Joey 1 added
The newest model in the System One benchmarks is Joey 1.
AIM System One Index
v1.2We tested 64 decision models on 1,655 classification, 64 bias, and 50 browser automation tasks. See AIM System One Index Methodology for further details.
AIM System One Index vs time per decision
Higher and further left is better. Time: mean per AIM-Decision Classification question.
AIM System One Index
Leaderboard
System One models ranked by the AIM System One Index.
Filter & Sort
# | Model | Index | AIM-AI-Bias (%) | AIM-Decision Browser (%) | AIM-Decision Classification (%) | AIM-Decision Browser time (s) | Cost per 1k decisions ($) | Latency per decision (ms) | Context window (tokens) |
|---|---|---|---|---|---|---|---|---|---|
| 1 |  d1-ultra Liquid | 60 | 80.1 | 76 | 52.1 | 3.8 | 0.026 | 597 | 32,768 |
| 2 |  AutoJev-27B Perplexity | 60 | 94.8 | 68 | 48.4 | 6 | 0.11 | 223 | 262,144 |
| 3 | decider-12b Mapika | 59 | 93.8 | 32 | 54.4 | 3 | 0.06 | 127 | 262,144 |
| 4 |  GLiDE Fastino | 59 | 90.0 | 76 | 48.3 | 46.8 | 0.479 | 15,686 | 40,000 |
| 5 |  Drex Max Nace AI | 58 | 80.7 | 86 | 48.7 | 5.9 | 0.04 | 632 | 262,144 |
| 6 |  Solar Decide Flash Upstage | 58 | 97.6 | - | 46.2 | - | 0.046 | 1,132 | 524,288 |
| 7 | Solar Decide Upstage | 58 | 97.2 | - | 46.0 | - | 0.046 | 7,231 | 524,288 |
| 9 |  Jev 1.14 TypeSafe | 57 | 87.5 | 68 | 48.1 | 3.2 | 0.039 | 634 | 262,144 |
| 10 | d1.1 Liquid | 57 | 70.3 | 58 | 54.5 | 2.5 | 0.013 | 277 | 65,536 |
| 11 |  Shisa DE-1 Shisa.AI | 57 | 83.9 | - | 48.9 | - | 0.021 | 65 | 262,144 |
Page 1 of 2
Cost$0.026 / 1k
Latency597 ms
Context33k
AIM-AI-Bias (%)
80.1
AIM-Decision Browser (%)
76
AIM-Decision Classification (%)
52.1
Cost$0.110 / 1k
Latency223 ms
Context262k
AIM-AI-Bias (%)
94.8
AIM-Decision Browser (%)
68
AIM-Decision Classification (%)
48.4
Cost$0.060 / 1k
Latency127 ms
Context262k
AIM-AI-Bias (%)
93.8
AIM-Decision Browser (%)
32
AIM-Decision Classification (%)
54.4
Cost$0.479 / 1k
Latency15,686 ms
Context40k
AIM-AI-Bias (%)
90.0
AIM-Decision Browser (%)
76
AIM-Decision Classification (%)
48.3
Cost$0.040 / 1k
Latency632 ms
Context262k
AIM-AI-Bias (%)
80.7
AIM-Decision Browser (%)
86
AIM-Decision Classification (%)
48.7
Cost$0.046 / 1k
Latency1,132 ms
Context524k
AIM-AI-Bias (%)
97.6
AIM-Decision Classification (%)
46.2
Cost$0.046 / 1k
Latency7,231 ms
Context524k
AIM-AI-Bias (%)
97.2
AIM-Decision Classification (%)
46.0
Cost$0.039 / 1k
Latency634 ms
Context262k
AIM-AI-Bias (%)
87.5
AIM-Decision Browser (%)
68
AIM-Decision Classification (%)
48.1
Cost$0.013 / 1k
Latency277 ms
Context66k
AIM-AI-Bias (%)
70.3
AIM-Decision Browser (%)
58
AIM-Decision Classification (%)
54.5
Cost$0.021 / 1k
Latency65 ms
Context262k
AIM-AI-Bias (%)
83.9
AIM-Decision Classification (%)
48.9
Page 1 of 2
Individual benchmarks
Tasks completed out of 50 on the same browser tasks.
Browser tasks completed out of 50
*One attempt per task.
1,655 single-decision questions in four tasks: score against cost per 1,000 decisions.
Score vs cost and latency
*Score: average of the two main tasks. Cost per 1,000 decisions: API price or bill for hosted models, A100 rental time with the GPU fully loaded for self-hosted models. Latency: mean seconds per decision.
Frequently asked questions
A System One model, also called a decision model, picks one answer from a fixed set of options and returns a probability for each option. Unlike an LLM, it does not write out reasoning or generate its answer token by token. When a task needs text, such as a value typed into a form, a separate text model writes it.
They suit automation steps whose answer comes from a fixed set of options, such as a browser agent's next action or a support ticket's label. An LLM spends time and money on every token it generates, and a decision model generates none. On our browser tasks, Drex Max completed as many tasks as Gemini 3.8 Flash in less than half the median time per task, at about 1/20 of the cost per completed task.
On our 50 browser tasks, the best System One model, Drex Max, completed 43, as many as Gemini 3.8 Flash. GPT-6 Astra completed 47, with both LLMs at low reasoning effort. Drex Max took a median of 5.9 seconds per task, against 13.2 for Gemini and 15.5 for Astra. Drex Max cost about $0.001 per completed task, against $0.025 for Gemini and $0.224 for Astra.
d1-ultra leads the AIM System One Index with 60. AutoJev-27B (60), decider-12b (59) and GLiDE (59) are within one point of it. d1-ultra leads none of the three benchmarks on its own. Of the 64 models, 31 score above 50.
We average each model's score over two classification tasks, security advisory severity and FDA recall class. The best averages are 54.5% (d1.1) and 54.4% (decider-12b), and the median model averages 46.7%. The score is accuracy in which rare answers count as much as common ones. Random answers average 25% on the severity task.
The page lists browser results for 50 of the 64 models. Some of the others accept too few options per question or too short an input for the browser pages. The median model completed 22 tasks (44%), and the best, Drex Max, completed 43 (86%).
AIM-AI-Bias counts the share of unbiased answers to multiple-choice questions about people. 7 of the 64 models answer at least 90% without bias, led by Solar Decide Flash at 97.6%. The median model scores 75.1%, and Laya scores 1.6%.
The median model takes 366 ms per decision, and 59 of the 64 take under a second. Laya is fastest at 30 ms, but its index score is 5. Among models that score above 50, Shisa DE-1 is fastest at 65 ms.
The median model costs $0.039 per 1,000 decisions. Span-02, Decider Flash V1.2, Joey 1 and Laya cost $0.007 after rounding, the lowest price. Of these, only Span-02 scores above 50 on the index. Hosted models use API prices, and self-hosted models use the cost of renting an A100 GPU.
The 45 models with a listed context window range from 512 tokens (Laya) to 1,050,000 (GPT-6 Luna Decisions). The median is 131,072 tokens. The context window is the most input a model reads at once, counted in tokens, which are word pieces.
AIM System One Index Methodology
AIM System One Index combines each model's scores on AIM-Decision Browser, AIM-Decision Classification and AIM-AI-Bias.
Each score becomes a z-score, its distance from the benchmark's average in standard deviations, measured against the models that have all three scores. The index is 50 + 10 × the average of a model's z-scores, so 50 is an average model and every 10 points is one standard deviation. A missing score is skipped; the leaderboard shows it as a dash.
Latency and cost are shown for reference and do not count toward the index.
Cost and latency per decision come from AIM-Decision Classification. Cost is the API price for hosted models and the A100 rental time with the GPU fully loaded for self-hosted models.
Benchmarks We Decided not to Use
These public benchmarks have been excluded from our index since vendors are training their models on them: BANKING77, CLINC150, ContractNLI, Amazon ESCI, RAGTruth, SGD / SGD-X (Schema-Guided Dialogue), Typed Decisions, GoEmotions, When2Call.
Benchmarks We Used
AIMultiple
AIM-Decision Browser50 browser tasks, one attempt each. Its success rate counts toward the index.AIMultiple
AIM-Decision Classification1,655 single-decision questions in four tasks: security advisory severity, FDA recall class, paper topic and browser task done check. Its score counts toward the index.AIMultiple
AIM-AI-BiasMultiple-choice bias questions, each asked with the “cannot be determined” option in every position. Its score, the share of unbiased answers averaged over those positions, counts toward the index. The first versions of GLiDE, Nimble 9B and the two Tev1 models were tested with that option last only.Download all data
Raw results of 181 data points from the AIMultiple benchmarks this index is built from. Download them as a ZIP file containing 3 CSV files and a README.
Last updated: November 3, 2026
DownloadWant the granular data behind it? Join Premium
Changelog
19 updates- v1.2
2 models entered the index: d1.1, Joey 1.
- v1.2
3 models entered the index: Drex Max, GLiDE v1.2, SubQ Snap 1.1.
- v1.2
Celeris 1.1 Decision entered the index.
- v1.2
6 models entered the index: Classify-1, d3-flash, d3-mini, Decider Flash V1.2, Decider V1.2 and 1 more.
- v1.2
2 models entered the index: d1-ultra, Jev 1.14.
- v1.2
7 models entered the index: APUS Reflex 35B-A3B, Decision-1.0-Lux-9B, MATILDA JEV v1.5, NeoHorse-Jev-4B, Nimble 9B v2 and 2 more.
- v1.2
12 models entered the index: d1-3B, d3, Decision-2.0-Vega-27B, Decision 4B v1.6, GEV-26B-Decide and 7 more.
- v1.2
2 models entered the index: Drex v2.0, SubQ Snap.
- v1.2
3 models entered the index: Clef Omni, Drex v1.5, Microsoft-Decision-1.
- v1.2
5 models entered the index: d1, Decider V1.1 27B, GPT-6 Luna Decisions, Mercury Decide, Solar Decide Flash.