Benchmark of 90+ LLMs in Finance: Claude Opus 5.5 & Gemini 4 Argon

The test set is the hard subset of the FinanceReasoning benchmark (Tang et al.), with 238 questions.¹ Accuracy is the percentage of questions answered correctly. Numerical answers receive a 0.2% relative tolerance.

Output tokens are the tokens generated across the 238 answers. Input tokens are counted separately in the cost calculation.

Cost is the calculated USD spend for the question-answering run, using the token counts and rates recorded in our cost analysis.

Accuracy, token use and cost findings

Claude Fable 5.2, Muse Wonder and Claude Opus 5.5 tie at 218 correct answers

Claude Fable 5.2, Muse Wonder and Claude Opus 5.5 share the highest recorded score, 91.60% (218/238). Gemini 4 Argon and GPT-5.6 Sol Pro answer two fewer questions correctly, scoring 90.76% (216/238). GPT-5.6 Sol, Claude Fable 5.1, Claude Fable 5, Grok 4.6 and GPT-6.1 Sol Pro each answer 215 correctly, scoring 90.34%.

Claude Fable 5.2 generates 169,295 output tokens for $7.4887, using 89,614 measured input tokens. Muse Wonder generates 407,149 output tokens for $8.4098, compared with Opus 5.5’s 144,834 output tokens and $3.2532. Muse Wonder’s cost uses 66,706 measured input tokens. Opus input usage is estimated from a sibling model. Their execution setups differ, and repeated-run reliability has not been established.

Gemini 4 Argon matches GPT-5.6 Sol Pro at 90.76% for $2.65

Gemini 4 Argon scores 90.76% (216/238), generating 249,825 output tokens for $2.6477. GPT-5.6 Sol Pro answers the same number correctly with 385,886 output tokens at $16.3533. Argon returned a nonempty response for every question. Its cost uses measured input and output token counts.

GPT-6.1 Sol Pro has the lowest cost among models tied at 90.34%

GPT-6.1 Sol Pro answers 215 questions correctly for $3.28, the lowest calculated cost among the five models tied at 90.34%. GPT-5.6 Sol costs $3.85, Grok 4.6 costs $4.45, Claude Fable 5.1 costs $8.64 and Claude Fable 5 costs $10.05.

GPT-6.1 Sol Pro generates 187,502 output tokens, compared with GPT-5.6 Sol’s 117,735. Its lower recorded token rates offset the longer output and higher input count. Among these tied models, GPT-5.6 Sol still uses the fewest output tokens.

GPT-6.1 Terra generates the fewest output tokens

GPT-6.1 Terra uses 79,655 output tokens, the lowest count among the 98 models. It answers 207 questions correctly (86.97%) for $0.4617. GPT-6 Sol uses 85,228 output tokens and answers 212 correctly (89.08%) for $0.98. GPT-6 Astra generates 87,774 tokens, answers 208 correctly (87.39%) and costs $5.02 at its recorded rates.

GPT-6.1 Sol answers 214 questions correctly (89.92%), using 89,153 output tokens for $1.02. Claude Sonnet 5.5 reaches 212 correct (89.08%) with 142,728 tokens at $1.61. Each comparison covers one completed run on the same 238 questions.

GPT-6 Luna exceeds 83% accuracy for $0.0754

GPT-6 Luna answers 207 questions correctly (86.97%) for $0.0754, the lowest recorded cost among models above 83%. DeepSeek V4.1 Flash also answers 207 correctly for $0.2572. Muse Ember answers 212 questions correctly (89.08%) for $0.4361, the lowest recorded cost among models at or above that score. GPT-6 Sol reaches the same score for $0.9792.

GPT-OSS-120B scores 81.09% for $0.0611, while Llama 4 Maverick scores 75.21% for $0.0995. Mistral Small 3.2 24B has the cheapest run overall at $0.0352, with 41.18% accuracy.

Claude Haiku 5.5 answers 204 questions for $0.1506

Claude Haiku 5.5 scores 85.71% (204/238), generating 283,280 output tokens for $0.1506. Mistral Large 4 scores 87.39% (208/238), generating 1,562,401 output tokens for $3.3083. Both runs returned a nonempty response for every question.

Higher spending does not guarantee a higher score

O1 Pro has the highest calculated cost, $381.28, and scores 80.67%. O3 Pro costs $39.23 for 78.15%, while O1 costs $46.59 for 74.79%. Each scores below Gemini 3 Flash Preview at $0.39.

Hy4 Preview generates 4,119,093 output tokens, the largest count in the comparison, and scores 89.92%. GPT-6.1 Sol reaches the same 214 correct answers with 89,153 tokens. DeepSeek R1 0528 generates 1,251,064 tokens at 62.18%, another example of output length varying independently of accuracy.

Results for all 98 models

Recorded accuracy values use different rounding in some older entries. GPT-5, Grok 4.5 and Kimi K3 each have 210 correct answers, despite the displayed 88.23% and 88.24% values. Model identifiers are preserved, and costs are rounded to four decimal places.

How models answered the same financial question

Eight models answered the same bond-pricing question. The bond costs $950 and has annual cash flows of $50, $50 and $1,050. The corresponding Treasury spot rates are 1.5%, 1.8% and 2.0%.

These excerpts come from saved model responses. They preserve the original wording, with formatting simplified and omitted passages marked […]. Each header gives the answer extractor’s selected value and the stored benchmark result. Seven models used OpenRouter, while Opus 5.5 used Claude Code.

The reference answer is 4.92%. We checked it by solving the cash-flow equation, which gives approximately 4.916972%. Six models passed with extracted answers of 4.92%. Gemini 4 Argon returned 4.94%, outside the benchmark’s 0.2% relative tolerance.

Muse Spark Open gave a range of 4.90% to 4.95% and also mentioned 4.92%. The answer extractor selected 4.90%, producing a stored failure. This response shows how multiple candidate values can affect answer extraction.

MiniMax M3.1 Flash passed with 4.92%. Its explanation also describes that spread as about 49 basis points. A spread of 4.92 percentage points equals 492 basis points. The benchmark scores the extracted final number, so a passing score can coexist with an error in the explanation.

This selected question illustrates individual responses and scoring behavior. The results for all 238 questions provide the broader model comparison.

Retrieval experiment: 24 additional correct answers

Our existing experiment report compares standalone generation with retrieval-augmented generation (RAG), which adds retrieved documents to a model’s prompt. It records an increase from 104 to 128 correct answers on the same 238-question set.

Experiment notes label the tested model GPT-4o Mini. The main results table assigns the same baseline accuracy and token count to O4 Mini. RAG model identity needs confirmation against the original run records. Its token column also lacks a verified input/output split, so it cannot be compared directly with the output-token column above.

Recorded time rises from about 3 to 59 minutes. Per-request latency and the separate time spent on retrieval were not established here. Testing covered one reported model, which limits conclusions about RAG on the other models.

Recorded retrieval setup

Historical notes describe a Qdrant index containing financial explanations and Python functions from the benchmark’s knowledge corpus. Text-embedding-3-small produces 1,536-dimensional vectors. Documents are split by section, and each function forms a separate chunk.

For each question and its context, the pipeline searches two collections using cosine similarity, retrieving three document chunks and two functions. Those notes record two embedding calls per query and an increase from roughly 300–500 to 3,000–5,000 input tokens. Retrieved material is included in the generation prompt. These setup details remain part of the historical report. The local retrieval script inspected for this rewrite does not establish that this specific Qdrant run executed.

Financial reasoning benchmark methodology

Questions and prompting

We used FinanceReasoning’s hard subset of 238 questions. Each question supplies context and a reference answer. Context can include financial tables, time series or the inputs for a financial calculation.

One example asks for the last upper Keltner Channel band from 25 days of stock prices, using a 10-day exponential moving average, a 10-day average true range and a multiplier of 1.5. The answer is requested to two decimal places. Solving it requires calculating the moving average and true range, then applying the upper-band formula.

With chain-of-thought prompting, the model is instructed to work through the calculation and end with a numerical answer. It receives the question and context in a free-form generation task. The 42 models added from September through November 2026 each completed inference and evaluation for all 238 questions through OpenRouter, with no missing outputs. Opus 5.5’s accepted result comes from an execution setup that differed from the main API pipeline. Execution conditions therefore vary across the comparison.

Answer extraction and scoring

Evaluation extracts an answer from the generated response, then compares it with the reference. Our current configuration uses Claude Sonnet 4.5 for answer extraction. Numerical answers pass when their absolute error is at most 0.2% of the reference answer’s absolute value.

Accuracy is the correct-answer count divided by 238. One question changes the score by about 0.42 percentage points. Correct-answer counts make ties explicit where recorded percentages differ in rounding.

Token and cost calculations

Output-token totals come from model usage records. We calculate cost by combining input and output tokens with their recorded per-million-token prices:

Stored prices include historical rates for deprecated models. Estimated input counts are marked in the underlying results table. Opus 5.5’s cost combines its measured 144,834 output tokens with an estimated 89,138 input tokens from a sibling model, using $4 per million input tokens and $20 per million output tokens. That produces $3.2532.

Costs cover the final retained answer-generation responses. Evaluation-model calls, failed or superseded retries, and RAG indexing or embedding charges are outside this calculation. Differences in tokenizers, generation settings and execution setup also affect comparisons.

Don’t miss our benchmarks and data-driven insights. The button opens Google; selecting AIMultiple confirms that you wish to see AIMultiple more often in Google search results.

Add as preferred source

Benchmark limitations

This public dataset may have appeared in model training data. Scores therefore cannot establish how well a model handles unseen financial problems.

Results cover a fixed question set and the recorded runs. Repeated-run variation and statistical significance have not been established. LLM-based answer extraction can introduce errors, and scoring the final answer does not validate every step in the explanation.

Estimated input counts and stored prices limit cost precision. Our separate RAG report has unresolved model attribution and token accounting, as described above. Its figures remain historical observations pending source reconciliation.

This benchmark measures financial question answering. It does not test investment returns, regulatory compliance, fraud detection or deployment reliability.

Further reading

AI-based Stock Trading covers a separate financial application.a fixed question set.

Cite this benchmark

Pick the format that matches where you're publishing. Pasting the link version into your CMS preserves the backlink.

@misc{sari2026,
  author = {Sarı, Ekrem},
  title  = {{Benchmark of 90+ LLMs in Finance: Claude Opus 5.5 & Gemini 4 Argon}},
  year   = {2026},
  month  = nov,
  howpublished    = {\url{https://aimultiple.com/finance-llm}},
  note   = {AIMultiple. Retrieved November 3, 2026}
}

Reference Links

1.

[2506.05828] FinanceReasoning: Benchmarking Financial Numerical Reasoning More Credible, Comprehensive and Challenging

Ekrem Sarı

AI Researcher

Follow On

Ekrem is an AI Researcher and Data Scientist at AIMultiple. He designs and runs hands-on benchmarks for AI and LLM systems.

View Full Profile

Be the first to comment

Your email address will not be published. All fields are required. Comments are left in their original language.

Name

Email Address

Comment

0/450

Post Comment