Premium
Services
Premium
Ekrem Sarı

Ekrem Sarı

AI Researcher
37 Articles
Stay up-to-date on B2B Tech

Ekrem is an AI Researcher and Data Scientist at AIMultiple. He designs and runs hands-on benchmarks for AI and LLM systems.

Professional Experience

At AIMultiple, Ekrem benchmarks end-to-end AI systems and builds the data workflows and dashboards used to track benchmark and product metrics. His benchmarks cover embedding and reranker models, vector and graph databases, inference engines, quantization, GPU concurrency and multi-GPU scaling, cloud GPU pricing and providers, text-to-SQL, and RAG and agentic RAG frameworks.

Before AIMultiple, he worked as a Data Scientist at Yandex, where he queried and analyzed large datasets with SQL to evaluate search and ranking quality against detailed guidelines.

Research Interest

Ekrem's work focuses on measuring how LLM and retrieval systems perform in practice. He designs the test harness, runs the workloads on real hardware, and compares models, frameworks, and infrastructure on accuracy, throughput, latency, cost, and scalability, across the stack from embedding models and vector databases to inference engines and GPU infrastructure. His MSc thesis automates systematic literature reviews with a RAG-based pipeline.

Education

Ekrem holds an MSc in Management Information Systems from Başkent University, where his thesis automated systematic literature reviews with a RAG-based pipeline, and is pursuing a second MSc in Data and Knowledge Engineering at Hacettepe University.

Latest Articles from Ekrem

Data
Benchmark
Oct 14

Remote Browsers: Web Infra for AI Agents Compared

AI agents rely on remote browsers to automate web tasks without being blocked by anti-scraping measures. The performance of this browser infrastructure is critical to an agent’s success. We benchmarked 8 providers on success rate, speed, and features. To do this, we executed 160 automated tasks, running 4 distinct scenarios 5 times for each service…

AI
Benchmark
Oct 14

AIM-Text-to-SQL Benchmark: SQL Accuracy Across 70+ LLMs

SQL accuracy is the percentage of scored queries that return the reference result. Incorrect routes and references flagged as broken are excluded. A reference query is the SQL supplied as the expected answer. Each model reaches a different set of SQL questions because scoring depends on its database choices. Differences in these subsets and execution…

AI
Benchmark
Oct 14

AIM-Agentic RAG Benchmark: Routing Across 11 SQL Databases

Routing accuracy is the percentage of scored questions for which the model explicitly names the correct database in its final answer. The headline uses the 184 questions flagged as difficult by both our similarity test and a jury of three LLMs. Missing explicit declarations receive no credit. gemini-4-argon selected the correct database for 173 of…

AI
Benchmark
Oct 14

Benchmark of 90+ LLMs in Finance: Claude Opus 5.5 & Gemini 4 Argon

The test set is the hard subset of the FinanceReasoning benchmark (Tang et al.), with 238 questions. Accuracy is the percentage of questions answered correctly. Numerical answers receive a 0.2% relative tolerance. Output tokens are the tokens generated across the 238 answers. Input tokens are counted separately in the cost calculation. Cost is the calculated…

AI
Benchmark
Oct 14

RAG Evaluation Tools: W&B Weave vs Ragas vs TruLens

We benchmarked five retrieval-augmented generation (RAG) evaluation tools on 1,000+ adversarial questions, each paired with five passages. The comparison measures context relevance using GPT-4o and the tools’ default evaluation prompts. W&B Weave had the highest Top-1 accuracy at 94.5%. TruLens led the ranking metrics, including NDCG@5 at 0.932. Their positions depend on how each metric…

AI
Feature Comparison
Oct 13

Top 30+ AI Chip Makers: NVIDIA & Its Competitors

Based on our experience running AIMultiple’s cloud GPU benchmark, we compare chip makers by product, availability and architecture, covering data center GPUs, mobile chips, edge accelerators and foundries. Each row names a representative product, its type and its availability. Availability distinguishes hardware offered for sale, cloud services, internal deployments and announced products. Dates refer to…

Agentic AI
Benchmark
Oct 9

Video Search API Benchmark: Bright Data vs YouTube

We compared Bright Data Video Search with YouTube search on 50 English queries across five types of queries. Both engines were queried to exhaustion, and a three-model panel assessed sampled video frames. Usable clips per search estimates how many returned videos contain the queried subject or action. It combines the number of unique videos returned…

AI
Benchmark
Oct 1

Embedding Models: 25 Models Benchmarked for Retrieval

We benchmarked 25 embedding models on 200 retrieval questions across finance, legal contracts, technical support and medical literature. The four corpora contained 341,040 text passages in total. Each query searched its own domain. nDCG@10 measures how highly the recorded relevant passages appear in the first ten results. Scores range from 0 to 1, with higher…

Cybersecurity
Benchmark
Sep 29

Bot Detection: 9 Anti-Bot Vendors Benchmarked

We identified the anti-bot company protecting top ~3,800 domains and measured how often each domain blocked our bots through 50,000 requests. Cloudflare defends 73.3% of the 3,859 domains and recorded a 6.7% block rate. Imperva defends 3.7% and recorded 46.6%. The two measures rank the nine companies in close to opposite orders. Metrics explained: Block…

Cybersecurity
Benchmark
Sep 29

DLP Software Benchmark

We benchmarked Acronis DeviceLock DLP and ManageEngine DLP Plus on identical Windows Server 2022 VMs with 28 scenarios: 23 data leak tests (including 12 adversarial evasion files), 3 agent security tests, and 2 tests under high CPU and memory consumption. For the other DLP products, Netwrix Endpoint Protector, Sophos Intercept X, Teramind DLP, and Trellix…