Ekrem Sarı
Ekrem is an AI Researcher and Data Scientist at AIMultiple. He designs and runs hands-on benchmarks for AI and LLM systems.
Professional Experience
At AIMultiple, Ekrem benchmarks end-to-end AI systems and builds the data workflows and dashboards used to track benchmark and product metrics. His benchmarks cover embedding and reranker models, vector and graph databases, inference engines, quantization, GPU concurrency and multi-GPU scaling, cloud GPU pricing and providers, text-to-SQL, and RAG and agentic RAG frameworks.
Before AIMultiple, he worked as a Data Scientist at Yandex, where he queried and analyzed large datasets with SQL to evaluate search and ranking quality against detailed guidelines.
Research Interest
Ekrem's work focuses on measuring how LLM and retrieval systems perform in practice. He designs the test harness, runs the workloads on real hardware, and compares models, frameworks, and infrastructure on accuracy, throughput, latency, cost, and scalability, across the stack from embedding models and vector databases to inference engines and GPU infrastructure. His MSc thesis automates systematic literature reviews with a RAG-based pipeline.
Education
Ekrem holds an MSc in Management Information Systems from Başkent University, where his thesis automated systematic literature reviews with a RAG-based pipeline, and is pursuing a second MSc in Data and Knowledge Engineering at Hacettepe University.
Latest Articles from Ekrem
Compare Relational Foundation Models
We benchmarked SAP-RPT-1-OSS against gradient boosting (LightGBM, CatBoost) on 17 tabular datasets spanning the semantic-numeral spectrum, small/high-semantic tables, mixed business datasets, and large low-semantic numerical datasets. Our goal is to measure where a relational LLM’s pretrained semantic priors may provide advantages over traditional tree models and where they face challenges under scale or low-semantic structure.…
Top 70+ Cloud GPU Providers
Cloud GPU providers fall into three tiers. Hyperscalers run broad cloud platforms with GPU rental as one product among many. Specialist neoclouds focus on GPU and AI infrastructure as their core product. Community marketplaces aggregate inventory from many small operators, often at the floor of the published price spread. Column definitions: Ranking: Subscribers are linked…
Cloud GPU Pricing, Performance & Provider Comparison
Cloud GPU list prices for the same model can differ several times over from one provider to another. We curated the lowest rate, provider, market range, and median for 40+ GPU configurations across all three pricing tiers, plus a throughput-per-dollar benchmark on 10 models. See the most cost-effective GPU for your workload across 13 hyperscaler…
RAG Evaluation Tools: Weights & Biases vs Ragas vs DeepEval
When a RAG pipeline retrieves the wrong context, the LLM confidently generates the wrong answer. Context relevance scorers are the primary defense. We benchmarked five tools across 1,460 questions and 14,600+ scored contexts under identical conditions: same judge model (GPT-4o), default configurations, and no custom prompts. Under standard conditions, WandB, TruLens, and Ragas emerged as…
Embedding Models: OpenAI vs Gemini vs Voyage
We benchmarked 15 English text-embedding models and a BM25 baseline on over 500 manually curated queries across three retrieval domains: legal contracts (CUAD), customer support (IBM TechQA), and healthcare (MedRAG PubMed). Voyage-3.5 ranks first overall. Perplexity Embed V1 0.6b reaches the upper-mid tier at the lowest price point in our benchmark. nDCG@3: Normalized discounted cumulative…
RAG Frameworks: LangChain vs LangGraph vs LlamaIndex
We benchmarked 5 RAG frameworks: LangChain, LangGraph, LlamaIndex, Haystack, and DSPy, by building the same agentic RAG workflow with standardized components: identical models (GPT-4.1-mini), embeddings (BGE-small), retriever (Qdrant), and tools (Tavily web search). This isolates each framework’s true overhead and token efficiency. The benchmark consisted of 100 queries, with each framework running the full set…
Backup software benchmark: Acronis vs NinjaOne vs Comet vs MSP360
We benchmarked NinjaOne Backup, Acronis Cyber Protect Cloud Backup, Comet Backup, and MSP360 Managed Backup on identical AWS infrastructure. Each vendor ran a file-mode backup of the same 625,946-file / 50 GB workload and a full image backup of the system disk, then restored the 15 GB medium subdirectory. Here are the four backup products…
Open Source Embedding Models Benchmark for RAG
NVIDIA Llama-Embed-Nemotron-8B leads in accuracy. On cost, Google’s EmbeddingGemma-300m runs roughly 4x cheaper than Nemotron at the cost of a small accuracy loss. nDCG@3: Normalized discounted cumulative gain at cutoff 3. With one relevant document per query, it is 1 / log2(rank + 1) when the gold document lands in the top 3, and 0…
Reranker Benchmark: Top 8 Models Compared
We benchmarked 8 reranker models on ~145k English Amazon reviews to measure how much a reranking stage improves dense retrieval. We retrieved top-100 candidates with multilingual-e5-base, reranked them with each model, and evaluated the top-10 results against 300 queries, each referencing concrete details from its source review. The best reranker lifted Hit@1 from 62.67% to…
LLM VRAM Calculator for Self-Hosting
Self-hosting an LLM means running inference on hardware the operator controls rather than via a third-party API, which changes the cost, data control, and privacy profile. Whether a model runs at all depends on memory. The calculator estimates the VRAM or unified memory a model needs to run locally, based on the model, its precision,…
AIMultiple Newsletter
1 free email per week with the latest B2B tech news & expert insights to accelerate your enterprise.