Research Library
Explore our research, benchmarks, and practical guides on enterprise AI and software.
- AI AgentsSep 21
Agent benchmarks
Agent benchmark
- Network SecurityOpen World EvaluationSep 21
Top 14 Network Planning Tools: Key features & Benefits
Network planning tools help businesses optimize performance, manage resources efficiently, and ensure scalable, reliable network designs for growth and stability. Here is an overview of the most valuable tools available today, organized by their primary function: These solutions are primarily used to architect a network from scratch, but can also be leveraged to reorganize an…
- LLMInsightSep 21
LLM VRAM Calculator for Self-Hosting
Self-hosting an LLM means running inference on hardware the operator controls rather than via a third-party API, which changes the cost, data control, and privacy profile. Whether a model runs at all depends on memory. The calculator estimates the VRAM or unified memory a model needs to run locally, based on the model, its precision,…
- AI HardwareBenchmarkSep 21
Multi-GPU Benchmark: B200 vs H200 vs H100 vs MI300X
For over two decades, optimizing compute performance has been a cornerstone of my work. We benchmarked NVIDIA’s B200, H200, H100, and AMD’s MI300X to assess how well they scale for Large Language Model (LLM) inference. Using the vLLM framework with the meta-llama/Llama-3.1-8B-Instruct model, we ran tests on 1, 2, 4, and 8 GPUs. We analyzed…
- AI AgentsBenchmarkSep 21
Mobile AI Agents Tested Across 65 Real-World Tasks
We spent 3 days benchmarking four mobile AI agents (DroidRun, Mobile-Agent, AutoDroid, and AppAgent) across 65 real-world tasks using an Android emulator with applications such as calendar management, contact creation, photo capture, audio recording, and file operations. See benchmark results including real-world performance comparison, costs and execution times: Highest success rate (43%) with high cost…
- AI HardwareBenchmarkSep 21
GPU Concurrency Benchmark: H100 vs H200 vs B200 vs MI300X
I have spent the last 20 years focusing on system-level computational performance optimization. We benchmarked the latest NVIDIA GPUs, including the NVIDIA’s H100, H200, and B200, and AMD’s MI300X, for concurrency scaling analysis. Using the vLLM framework with the gpt-oss-20b model, we tested how these GPUs handle concurrent requests, from 1 to 512. By measuring…