Tabular Models Benchmark: Performance Across 19 Datasets
We benchmarked 13 tabular learning models on 19 real-world datasets covering roughly 272,000 samples, with dataset sizes from 435 to 48,800 rows. Every model ran with 5-fold cross-validation on identical splits.
Tabular learning models benchmark results
Each dataset is a round-robin of head-to-head matches between models, decided by the primary metric. Elo aggregates all 1,398 matches into a single rating per model; a 200-point gap means the higher-rated model wins about 3 out of 4 matches. Details are in the methodology.
- No single model leads. The significance test finds no significant gap among the top five: LimiX-2 (1503), Kumo-Tabular (1438), TabPFN-3.5 (1363), TabFM (1325) and Causilo (1192). The Elo intervals of the top four overlap.
- LimiX-2 finished first on 11 of the 19 datasets and in the top three on 17. Kumo-Tabular, which the test does not separate from it, averages 7.6 seconds per fold, about 15x faster.
- On 15 of the 18 datasets with a single winner, the winner’s lead is smaller than its own fold-to-fold standard deviation. LimiX-2 leads by more than that on the other three: diabetes, electricity and employee_salaries.
Elo also handles partial coverage fairly: Mitra is rated only on matches it could play, meaning the 12 datasets within its 10,000-row training cap1.
- LimiX-2 leads diabetes with 0.879 ROC-AUC against Kumo-Tabular‘s 0.844, and employee_salaries with RMSE 2,743 against TabPFN-3.5’s 3,060.
- On amazon_employee_access, whose largest categorical column has 7,518 values, LimiX-2 scores 0.898 against 0.863 for Logistic Regression, the best classical model. The gap is 3.5 times LimiX-2’s fold standard deviation.
The significance test (Friedman test, then Nemenyi critical difference) puts LimiX-2, Kumo-Tabular, TabPFN-3.5, TabFM and Causilo ahead of all four classical models, none of which won a dataset. LimiX-2 and Kumo-Tabular are also ahead of TabICLv2, Mitra-v2 and TabPFN 3. TabPFN-3.5’s lead over TabPFN 3 clears the threshold by 0.02 rank points, the narrowest significant gap in the test. Test details are in the methodology.
Latency of tabular models
- GPU pricing uses RunPod’s on-demand Community Cloud B200 rate of $5.98 per GPU-hour. The classical models ran on CPU (62 cores); each foundation model ran on one B200.
- TabFM averaged 173.6 seconds per fold (median 16.9) and LimiX-2 111.9 (median 28.4). At that rate, TabFM’s fold times for the full benchmark cost about $27 and LimiX-2’s about $18; TabPFN-3.5 did the same work for $0.68, and LightGBM ran on CPU.
- TabPFN-3.5 and Kumo-Tabular average 4.3 and 7.6 seconds per fold; Kumo-Tabular’s full run costs $1.19. Causilo (1.4 seconds) and Mitra-v2 (1.5) are the fastest foundation models.
- Fold times exclude TabFM’s cold start: its JAX backend spends roughly 20 to 25 minutes compiling GPU kernels before the first prediction. A persistent compilation cache removes this on later runs.
- LimiX-2 used about 124 GB of GPU memory on the largest datasets. TabFM’s JAX backend on B200 hardware hits a documented cuBLAS bug in the default CUDA 12 stack, which JAX flags as a silent data-corruption risk; the CUDA 13 stack avoids it.
- TabFM re-processes the training data as context on every prediction batch, so repeated scoring on large data compounds its cost. A trained XGBoost model predicts in microseconds.
Results by dataset size and feature type
- Small datasets (1,000 rows or fewer): foundation models take the top nine slots on both numeric and hybrid (numeric plus categorical) data. LimiX-2 leads on numeric data (93.7% average ROC-AUC). Logistic Regression holds up on numeric data (91.2%) and collapses on hybrid (77.5%).
- Medium datasets (1,001 to 10,000 rows): foundation models lead on both feature types. The hybrid gap is wide: LimiX-2 85.5% average ROC-AUC against 83.4% for the best classical model, CatBoost.
- Large datasets (over 10,000 rows): the old “boosting catches up at scale” pattern is gone. On large numeric data, LimiX-2, Kumo-Tabular, and TabPFN-3.5 average 99.6% against XGBoost’s 97.0%. On large hybrid data LimiX-2 and TabPFN-3.5 lead with 89.2% against LightGBM’s 87.6%.
- Mitra-v2 averages 95.6% on large numeric data, below XGBoost. That bucket holds one binary dataset, electricity, where Mitra-v2 predicts from a capped context.
Average rank by regime
The rank view adds two things the ROC-AUC averages hide.
- LimiX-2 holds the top average rank in five of six regimes, tied with TabFM on medium numeric data. On small hybrid data its inverted rank of 9.5 places it fourth, behind Kumo-Tabular (11.5), TabFM (10.0) and TabPFN-3.5 (9.8).
- Every foundation model except Mitra outranks every classical model in all six regimes.
Model wins by regime
LimiX-2 takes win points in all six regimes and 11.1 of the 19 points in total. TabFM scores in three regimes; Kumo-Tabular and TabPFN-3.5 in two. The classical models score no points. The eight-way tie on monks-problems-2 splits one point eight ways, so most foundation models show 0.1 on small hybrid data.
Model-specific observations
LimiX-2 (StableAI)
LimiX-2 has the best mean rank (2.16) and 11 outright wins. It also has the second-highest compute cost, at 111.9 seconds per fold and about 124 GB of GPU memory on the largest datasets2.
Kumo-Tabular (NVIDIA)
Kumo-Tabular has the second mean rank (2.63) at 7.6 seconds per fold, and the significance test does not separate it from LimiX-23. It finished in the top three on 16 of 19 datasets.
TabPFN-3.5 (Prior Labs)
TabPFN-3.5 has the third mean rank (3.26) at 4.3 seconds per fold, against TabPFN 3’s 4.2. Its rank lead over TabPFN 3 is significant by the narrowest margin in the test. SAP announced its acquisition of Prior Labs in May 20264, which matters for enterprise buyers evaluating vendor stability.
TabFM (Google Research)
TabFM has the fourth mean rank (3.53) and finished first on three datasets5. Its fold times are about 40x TabPFN 3’s, its first run compiles for over 20 minutes, and it is limited to 10 classes and optimized for up to 500 features.
Causilo (Nums AI)
Causilo has the fifth mean rank (4.89) at 1.4 seconds per fold, the fastest foundation model in the benchmark6. None of the four models above it is significantly ahead of it.
TabICLv2 (Inria SODA)
TabICLv2 has the sixth mean rank (6.58) at 3.1 seconds per fold7. It adds regression support, which the original TabICL lacked, and completed all 19 datasets.
TabPFN 3 (Prior Labs)
TabPFN 3 has the eighth mean rank (7.11) and completed all 19 datasets in 6.6 GPU-minutes8.
Mitra-v2 (Amazon)
Mitra-v2 has the seventh mean rank (7.00) at 1.5 seconds per fold9. It drops Mitra’s 10,000-row cap and completed all 19 datasets, but on five large binary datasets it predicts from a sample of at most 16,384 training rows.
Mitra (Amazon, via AutoGluon)
A small-data specialist by design: a hard 10,000-row training cap makes it N/A on 7 of 19 datasets. On its 12 datasets it averages a rank of 9.12 among the 13 models. Mitra-v2 removes the cap and ranks higher.
Gradient boosting: LightGBM, CatBoost, XGBoost
Gradient-boosted trees no longer lead on accuracy in any regime. LightGBM was the best classical model (mean rank 9.74). CatBoost was the best classical model on medium hybrid data. XGBoost finished last on mean rank (10.79) and showed one notable weakness: with native categorical splits on the high-cardinality employee_salaries regression, it scored RMSE 10,894 where LightGBM scored 4,367 on identical inputs. That is a one-dataset result, but worth checking before relying on XGBoost’s categorical mode.
The boosted trees’ case in 2026 is operational: sub-second training, microsecond inference, no GPU, no row caps, mature tooling.
Logistic Regression
Logistic regression has a mean rank of 10.53, yet on amazon_employee_access it beats every gradient-boosted tree, TabPFN 3, and TabICLv2. Sparse one-hot plus a linear model remains a strong, nearly free baseline on high-cardinality categorical data. Keep it in every benchmark.
Key takeaways
Five foundation models (LimiX-2, Kumo-Tabular, TabPFN-3.5, TabFM and Causilo) beat all four classical models with statistical significance across the 19 datasets, and foundation models lead in every size and feature regime. This holds only when each model receives the categorical input it is designed for; forcing one-hot encoding on in-context models cripples them on high-cardinality data.
The significance test does not separate LimiX-2, Kumo-Tabular, TabPFN-3.5, TabFM and Causilo. Their fold times range from 1.4 seconds (Causilo) to 173.6 seconds (TabFM).
Of these five, only Kumo-Tabular’s license allows commercial use of the model and its outputs without a separate agreement.
Zero-shot means zero training, not zero compute. Score your workload’s prediction volume before choosing TabFM or LimiX-2.
What are tabular foundation models?
Traditional tabular models train from scratch on each dataset. Tabular foundation models (TFMs) are pretrained once on large collections of tabular problems, mostly synthetic, and adapt to new datasets at inference time: the training rows go in as context, and predictions come out of a forward pass. No gradient training, no hyperparameter tuning.
TabPFN reframes tabular learning as amortized Bayesian inference: a transformer pretrained on millions of synthetic tasks conditions on the observed dataset to produce predictions8. TabPFN 3 is built for up to 1,000,000 rows and 2,000 features, with native categorical handling.
TabICL extends in-context learning to tables: rows become structured tokens, and the model infers decision rules from context examples10. TabICLv2 added regression support in February 2026.
TabFM combines the two approaches: alternating row and column attention over raw cells, compression of each row into a dense vector, then an in-context transformer over the compressed rows. It was trained on hundreds of millions of synthetic datasets generated from structural causal models.
Mitra follows the TabPFN line inside AutoGluon, tuned for small data.
LimiX-2, Kumo-Tabular, TabPFN-3.5, Causilo and Mitra-v2 work the same way in this benchmark: each predicts from the training rows as context, with no gradient training on the dataset.
Add as preferred sourceTabular Models Benchmark Methodology
Models (13): LogisticRegression (Ridge regression on the three regression datasets), XGBoost, LightGBM, CatBoost, TabPFN 3, TabICLv2, TabFM, Mitra, LimiX-2, Kumo-Tabular, TabPFN-3.5, Causilo, Mitra-v2.
Datasets (19, OpenML): 15 binary classification, 1 multiclass, 3 regression; 435 to 48,800 rows; numeric-only or hybrid (numeric plus categorical) features. Known leakage columns removed.
Evaluation: 5-fold stratified cross-validation (plain CV for regression), seed 42, identical splits for every model. Primary metrics: ROC-AUC (binary), macro-F1 (multiclass), RMSE (regression). Default hyperparameters, no tuning. This favors tuning-free models: a tuned XGBoost would score higher, and tuning budgets are themselves a cost we did not model.
Configuration: the benchmark’s fixed settings for the classical models, TabPFN 3, TabICLv2, TabFM and Mitra; for LimiX-2, Kumo-Tabular, TabPFN-3.5, Causilo and Mitra-v2, the default configuration in TabArena, a public leaderboard for tabular models. No model is fine-tuned. TabArena’s default for Mitra-v2 fine-tunes for 50 steps on each training fold; here Mitra-v2 runs without it.
Categorical handling, per model design: native category dtype for CatBoost, XGBoost, LightGBM, Causilo, and Mitra-v2; raw features with categorical indices for TabPFN 3, TabPFN-3.5 and TabFM, raw text columns for LimiX-2; column types inferred by the model for Kumo-Tabular, ordinal encoding for TabICLv2; one-hot plus StandardScaler for Logistic Regression only. No model that supports native categoricals is forced through one-hot. The trade-off: cells differ in input representation by design. Forcing one representation on every model looks fairer but penalizes the models built for a different input.
Environment: one RunPod pod with 2x NVIDIA B200 (183 GB), driver 580, Ubuntu 24.04. Classical models on CPU (62 cores), each foundation model on one B200. PyTorch 2.8.0 cu128 for TabPFN 3, TabICLv2, and Mitra; JAX 0.10.1 with CUDA 13 wheels for TabFM. LimiX-2, Kumo-Tabular, TabPFN-3.5, Causilo and Mitra-v2 ran on single-B200 RunPod pods (driver 595, CUDA 13.2), one model per GPU at a time, each in its own Python environment.
Latency: per-fold fit-plus-predict wall time, recorded inside the harness. TabFM’s one-time compilation is excluded from fold times and reported separately.
N/A policy: a cell is N/A only for a documented model limit, with the reason captured in logs. The only N/As in the final data are Mitra’s 7 datasets above its 10,000-row cap.
Mitra-v2 context cap: on binary datasets with more than 16,384 training rows (adult, amazon_employee_access, bank-marketing-full, default_credit, electricity), Mitra-v2 predicts from a class-balanced sample of 16,384 training rows, redrawn for each batch of 16,384 test rows. This is TabArena’s default. The cap is 32,768 rows for multiclass and regression, which no dataset here exceeds.
Rank metrics: within each dataset, models are ranked 1 (best) to k by the primary metric; a model’s mean rank averages these over the datasets it completed. The regime chart shows the inverted rank, k + 1 minus rank, so higher is better and 13 means first among 13. Win points award each dataset’s single point to the best score; exact ties split it equally, so an eight-way tie pays 0.125 each.
Statistics: Friedman test plus Nemenyi critical difference over the 12 full-coverage models. Mitra is excluded because it completed 12 of 19 datasets; including it would cut the test to those 12. The mean ranks quoted in the model sections come from this 12-model test; Mitra’s is computed among all 13 models over its 12 datasets.
The Friedman test gives chi-squared 165.8, p below 10^-29. The critical difference is CD = q × sqrt(k(k+1) / (6N)), with q = 3.268 at alpha 0.05 for k = 12 models and N = 19 datasets, giving CD = 3.82: two models differ significantly only if their mean ranks are more than 3.82 apart. The top four span 2.16 (LimiX-2) to 3.53 (TabFM), so no gap among them clears the threshold. LimiX-2’s gap to TabICLv2 is 4.42; TabPFN-3.5’s gap to TabPFN 3 is 3.84. Mean-rank confidence intervals come from a 10,000-sample bootstrap over datasets.
Elo ratings: each dataset is a round-robin between all models that completed it, 1,398 pairwise matches in total. The model with the better primary metric wins a match (score 1); exact ties score 0.5 each.
The formulas use four quantities. R(A) is model A’s current rating, the number the chart reports; every model starts at R = 1000. S(A) is the actual score model A took from a match: 1 for a win, 0.5 for a tie, 0 for a loss. E(A) is the expected score, the win probability the current ratings predict for A before the match is counted. K = 32 is the update step, which caps how much one match can move a rating.
Before a match between models A and B, the expected score is E(A) = 1 / (1 + 10^((R(B) − R(A)) / 400)): when the ratings are equal, it gives 0.5, and it approaches 1 as A’s rating pulls ahead. After the match, the rating updates by R(A) = R(A) + K × (S(A) − E(A)), so A gains when it does better than expected and loses points when it does worse; an upset win over a higher-rated model pays more than a routine win over a weaker one. The update is zero-sum, so the pool average stays at 1000, and a rating near 1000, such as Mitra-v2’s 977, reads as an average model in this field. The 400 divisor sets the scale: at a 200-point gap the expected score is 1 / (1 + 10^(−0.5)) = 0.76, which is where “wins about 3 out of 4 matches” comes from.
A single Elo pass depends on match order, and the 19 datasets are themselves a sample. We therefore run 1,000 bootstrap rounds; each round resamples the 19 datasets with replacement, rebuilds the match list, shuffles the match order, and runs Elo from scratch. The published rating is the mean across rounds, and the 95% intervals are the 2.5th and 97.5th percentiles, covering both dataset sampling and ordering noise. Elo is the summary visualization; the Friedman and Nemenyi test remains the significance arbiter.
Reproducibility: fixed seed 42, identical splits, per-fold results and all merge, statistics, and Elo scripts in the benchmark repository.
Cite this benchmark
Pick the format that matches where you're publishing. Pasting the link version into your CMS preserves the backlink.
@misc{kalelioglu2026,
author = {Kalelioğlu, Berk and Dilmegani, Cem},
title = {{Tabular Models Benchmark: Performance Across 19 Datasets}},
year = {2026},
month = jul,
howpublished = {\url{https://aimultiple.com/tabular-models}},
note = {AIMultiple. Retrieved July 3, 2026}
}Results and timestamps of 48 data points. Download the summary data shown in this article's charts and tables as a ZIP file containing 3 CSV files and a README.
Want the granular data behind it? Join Premium
Changelog
3 updatesAdded TabFM to the benchmark, leading every dataset regime.
Replaced TabPFN with TabPFN 3 across the benchmark results, rankings and model limitations.
Replaced TabPFN with TabPFN 3 in the benchmarked model list.
Reference Links
Cem's work at AIMultiple has been cited by leading global publications including Business Insider, Forbes, Morning Brew, and Washington Post, global firms like Deloitte and HPE, NGOs like World Economic Forum, and supranational organizations like European Commission. [1], [2], [3], [4], [5]
Throughout his career, Cem served as a tech consultant, tech buyer and tech entrepreneur. He advised enterprises on their technology decisions at McKinsey & Company and Altman Solon for more than a decade. He also published a McKinsey report on digitalization.
He led technology strategy and procurement of a telco while reporting to the CEO. He has also led commercial growth of deep tech company Hypatos that reached a 7 digit annual recurring revenue and a 9 digit valuation from 0 within 2 years. Cem's work in Hypatos was covered by leading technology publications like TechCrunch and Business Insider.
Cem regularly speaks at international technology conferences. He graduated from Bogazici University as a computer engineer and holds an MBA from Columbia Business School.
Be the first to comment
Your email address will not be published. All fields are required. Comments are left in their original language.