Services
Contact Us
Ekrem Sarı

Ekrem Sarı

AI Researcher
35 Articles
Stay up-to-date on B2B Tech

Ekrem is an AI Researcher and Data Analyst at AIMultiple. He designs and runs hands-on benchmarks for AI and LLM systems.

Professional Experience

At AIMultiple, Ekrem benchmarks end-to-end AI systems and builds the data workflows and dashboards used to track benchmark and product metrics. His benchmarks cover embedding and reranker models, vector and graph databases, inference engines, quantization, GPU concurrency and multi-GPU scaling, cloud GPU pricing and providers, text-to-SQL, and RAG and agentic RAG frameworks.

Before AIMultiple, he worked as an Assessor at Yandex, where he evaluated search quality and labeled large volumes of data against detailed guidelines to support ranking and model quality.

Research Interest

Ekrem's work focuses on the MLOps and LLMOps lifecycle and on measuring the performance of AI systems. He compares models, frameworks, and infrastructure on metrics such as accuracy, throughput, API cost, and scalability, across the stack from embedding models and vector databases to GPU and cloud infrastructure. His MSc thesis automates systematic literature reviews with a RAG-based pipeline.

Education

Ekrem holds a BA from Hacettepe University and is completing an MSc at Başkent University.

Latest Articles from Ekrem

AI
Benchmark
Aug 14

Open Source Embedding Models Benchmark for RAG

NVIDIA Llama-Embed-Nemotron-8B leads in accuracy. On cost, Google’s EmbeddingGemma-300m runs roughly 4x cheaper than Nemotron at the cost of a small accuracy loss. nDCG@3: Normalized discounted cumulative gain at cutoff 3. With one relevant document per query, it is 1 / log2(rank + 1) when the gold document lands in the top 3, and 0…

AI
Benchmark
Aug 14

Multimodal Embedding Models: Apple vs Meta vs OpenAI

Multimodal embedding models excel at identifying objects but struggle with relationships. Current models struggle to distinguish “phone on a map” from “map on a phone.” We benchmarked 7 leading models across MS-COCO and Winoground to measure this specific limitation. To ensure a fair comparison, we evaluated every model under identical conditions using NVIDIA A40 hardware…

AI
Benchmark
Aug 14

Benchmark of 40+ LLMs in Finance: Claude Fable 5 & GPT-5.6 Sol

We evaluated LLMs on 238 hard questions from the FinanceReasoning benchmark (Tang et al.).1 This subset targets the most challenging financial-reasoning tasks, assessing complex, multi-step quantitative reasoning involving financial concepts and formulas. Our evaluation employed a custom prompt design and scoring criteria of accuracy and token consumption. For a detailed explanation of how these metrics…

AI
Benchmark
Aug 14

Embedding Models: OpenAI vs Gemini vs Voyage

We benchmarked 15 English text-embedding models and a BM25 baseline on over 500 manually curated queries across three retrieval domains: legal contracts (CUAD), customer support (IBM TechQA), and healthcare (MedRAG PubMed). Voyage-3.5 ranks first overall. Perplexity Embed V1 0.6b reaches the upper-mid tier at the lowest price point in our benchmark. nDCG@3: Normalized discounted cumulative…

Enterprise Software
Benchmark
Aug 14

Email Archiving Software Benchmark

We provisioned a Microsoft 365 tenant, populated it with a 10,000-mail synthetic corpus and 1,700 attachments across 8 file-type subtypes, then benchmarked NinjaOne SaaS Archiver, Barracuda Cloud Archiving Service, Acronis Cyber Protect Cloud Email Archiving, and MailPiler on the same tenant against 10 dimensions covering ingestion, search, attachment recall, export, immutability, legal hold, audit, encryption,…

Data
Benchmark
Aug 14

E-Commerce Scraper: 4 Providers Benchmarked

We benchmarked four web data providers across 100 e-commerce domains, fetching 65,000 product and search pages each at 5 to 5,000 concurrent requests. Averaged across concurrency levels, Bright Data reached the highest success rate (76%) at a 16-second median. Apify recorded the highest median response time at 46 seconds. Response time is reported as the…

Cybersecurity
Open World Evaluation
Aug 14

Disaster Recovery Benchmark: Acronis vs Comet vs MSP360

We benchmarked Acronis Cyber Protect Cloud, Comet Backup, and MSP360 Managed Backup on disaster recovery. Each vendor imaged a live Windows Server 2022 and a live Ubuntu 24.04 server carrying the same deterministic workload, a web service, a 10,000-row database, and 50 files, then recovered the whole machine onto a separate server after a ransomware-style…

Data
Benchmark
Aug 14

Graph Database Benchmark: Neo4j vs FalkorDB vs Memgraph

We benchmarked Neo4j, FalkorDB, and Memgraph on a synthetic graph derived from 120,000 Amazon product reviews (381K nodes, 804K edges). We ran 12 query templates with 1,000 measurements each, tested ingestion at 6 batch sizes, sustained concurrent load for 60 seconds at up to 32 threads, and measured memory, cold start, mixed workload, and index…

Cybersecurity
Benchmark
Aug 12

Bot Detection: 9 Anti-Bot Vendors Benchmarked

We ran two benchmarks against live websites, then identified the anti-bot company in front of every domain and measured how often each company’s domains returned the content we requested. That covers over 3,800 labeled domains and over 50,000 requests. The first benchmark fetched product and search pages on 100 e-commerce domains, 50 URLs each. The…

AI
Feature Comparison
Aug 11

Top 30+ AI Chip Makers: NVIDIA & Its Competitors

Based on our experience running AIMultiple’s cloud GPU benchmark with 10 different GPU models in 4 different scenarios, these are the top AI hardware companies for data center workloads. *Selected AI chip is the part, platform, or announced project that best represents each vendor. Some entries are systems, services or licensable IP rather than a…