Artificial Intelligence
Explore practical insights, research, and benchmarks on artificial intelligence, including generative AI, large language models, RAG, governance frameworks, MLOps practices, and AI hardware. Gain an understanding of key tools, implementation strategies, and enterprise use cases shaping the AI landscape.
Explore Artificial Intelligence
- AI CodingBenchmarkSep 22
AI Code Review Tools Benchmark
With the increased use of AI coding tools, codebases have become more prone to vulnerabilities, which increased the need for effective code reviews. To address this, we introduce RevEval (AI Code Review Eval), which benchmarks the top four AI code review tools across 309 pull requests from repositories of varying sizes and evaluates their performance…
- LLMInsightSep 21
LLM VRAM Calculator for Self-Hosting
Self-hosting an LLM means running inference on hardware the operator controls rather than via a third-party API, which changes the cost, data control, and privacy profile. Whether a model runs at all depends on memory. The calculator estimates the VRAM or unified memory a model needs to run locally, based on the model, its precision,…
- AI HardwareBenchmarkSep 21
Multi-GPU Benchmark: B200 vs H200 vs H100 vs MI300X
For over two decades, optimizing compute performance has been a cornerstone of my work. We benchmarked NVIDIA’s B200, H200, H100, and AMD’s MI300X to assess how well they scale for Large Language Model (LLM) inference. Using the vLLM framework with the meta-llama/Llama-3.1-8B-Instruct model, we ran tests on 1, 2, 4, and 8 GPUs. We analyzed…
- AI HardwareBenchmarkSep 21
GPU Concurrency Benchmark: H100 vs H200 vs B200 vs MI300X
I have spent the last 20 years focusing on system-level computational performance optimization. We benchmarked the latest NVIDIA GPUs, including the NVIDIA’s H100, H200, and B200, and AMD’s MI300X, for concurrency scaling analysis. Using the vLLM framework with the gpt-oss-20b model, we tested how these GPUs handle concurrent requests, from 1 to 512. By measuring…
- LLMBenchmarkSep 18
Agentic IT: Can AI Agents Design a Benchmark
We tested 16 models on a benchmark design in text-to-SQL and tool calling. Each model built one benchmark per topic, for a total of 32 submissions. No submission passed every rubric criterion. The agents could build and run tests, but none demonstrated both a blank-answer test and a correct-answer test of their own scorer. Text-to-SQL…
- LLMBenchmarkSep 17
AIM Enterprise: Agentic Enterprise Benchmark
Enterprises use LLMs every day for their regular tasks. To find the most cost-efficient LLMs, we designed AIM Enterprise, an agentic enterprise benchmark, where we used 69 real enterprise tasks across strategy, marketing, HR, sales, and operations. Two judge models scored every file that passed the checks. On 32.7% of the individual scores the two…
- AI HardwareBenchmarkSep 17
DGX Spark alternatives: RTX, Ryzen AI Halo and Mac Studio
We compare published DGX Spark, RTX and Ryzen AI Halo inference benchmarks, covering prompt processing, token generation and longer contexts. Alternatives include Framework Desktop, Mac Studio and GB10 systems from other manufacturers. Decode speed measures output tokens generated per second. Higher values mean faster response generation. All six results use GPT-OSS 20B MXFP4 in Ollama…
- AI ModelsInsightSep 15
World Foundation Models: 10 Use Cases
Training robots and autonomous vehicles (AVs) in the physical world can be costly, time-consuming and risky. World Foundation Models offer a scalable alternative by enabling realistic simulations of real-world environments. These models accelerate development and deployment in robotics, AVs, and other domains by reducing reliance on physical testing. Explore how World Foundation Models work, their…