AI Agents
AI agents are software systems that use reasoning, planning, and tools to assist or automate complex tasks. We compare the top open-source and commercial agents.
Large Action Models: Hype or Real?
Following the launch of Rabbit, an AI device that can use mobile apps, the term large action models (LAMs) is getting popular. These models move beyond conversation by turning LLMs into “agents” that can connect the siloed, app-driven world without requiring users to click on apps or integrate APIs. The line between hype and reality…
A-CODE-CLI Bench: Agentic CLI Benchmark
Agentic CLI tools are AI coding tools that can create and delete files, run commands, plan, and execute the coding of the entire project. We benchmarked the leading tools across 10 real-world web development scenarios, performing ~600 atomic validation checks per agent and more than ~5,000 total automated test executions, including backend logic, frontend functionality,…
Mobile AI Agents Tested Across 65 Real-World Tasks
We spent 3 days benchmarking four mobile AI agents (DroidRun, Mobile-Agent, AutoDroid, and AppAgent) across 65 real-world tasks using an Android emulator with applications such as calendar management, contact creation, photo capture, audio recording, and file operations. See benchmark results including real-world performance comparison, costs and execution times: Highest success rate (43%) with high cost…
SAP AI Agents: Joule Studio case studies
SAP predicts that AI agents could support up to 80% of the most-used business tasks in SAP. 31 We evaluated SAP’s AI agent portfolio, related case studies, and supporting infrastructure and identified three shifts shaping SAP’s AI agent strategy: The table shows some other enterprise AI agent builder providers that can be an alternative to…
Best 50+ Open Source AI Agents Listed
Everyone has been building AI agents so after hands-on testing with popular AI coding agents, AI agent builders and tools use benchmarks to evaluate their real-world capabilities, we put together a curated list of the best 50+ open source AI agents. Click the category headers to jump straight to our top picks: Agent development &…
Agentic Search: Benchmark 8 Search APIs for Agents
Agentic search plays a crucial role in bridging the gap between traditional search engines and AI search capabilities. Search APIs are the first layer of an agentic tool, where performance caps the quality of everything downstream. We benchmarked 8 search APIs across 100 real-world AI/LLM queries, evaluating 4,000 retrieved results with an LLM judge that…
A-CODE-LLM Bench: Agentic Coding Benchmark
We benchmarked the top Large Language Models (LLMs) across 10 software development tasks using an agentic CLI tool. We executed ~3,500 automated validation steps per model across both API and UI layers. Each alias ran 3 times across 10 tasks (30 samples per alias, 400 cells per iteration across 40 aliases). See more details on…
Top 30+ Agentic AI Companies
Though AI agents are being hyped and some companies rebrand their chatbots as agentic tools, there are still a few agents in production. Previously, we benchmarked several capable AI agents over several real-world tasks. We listed: These companies primarily focus on agentic AI research and development, offering initiatives like ethical AI guidelines and environments for…
Top Agent Harnesses: Claude Code vs Codex
Agent harnesses serve as the production runtime for AI agents, with design choices that create performance variation across identical underlying models. We benchmarked 17 agent harnesses across 10 coding tasks. To isolate the harness rather than the model, we ran every agentic CLI on a single foundation model, Claude Sonnet 4.6 (non-reasoning), and AI code…
Agentic AI Finance Benchmark: FinRobot vs FinRL vs FinGPT
79% of executives report that their companies have started adopting AI agents, yet 34% are currently using them in accounting and finance.83 We conduct a benchmark on 3 agentic AI finance tools tailored for financial workflows. Results suggest that We present benchmark results alongside use cases and implementation challenges. The findings highlight several important patterns:…