Benchmarks / Reasoning
MathVista
Uses Kaggle's MathVista testmini accuracy leaderboard. Random-chance accuracy is 17.9%. Score: MathVista testmini accuracy. Collected by Jake Boggs' benchmark dashboard.
MathVista
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | Gemini 3.5 Flash | xhigh | 90.4% |
| 2 | Gemini 3 Pro | high | 89.4% |
| 3 | Gemini 2.5 Pro | high | 84.6% |
| 4 | Opus 4.8 | max | 84.1% |
| 5 | Gemma 4 26B | xhigh | 84.1% |
| 6 | Gemma 4 31B IT | high | 83.5% |
| 7 | GPT-5.2025 | high | 82.7% |
| 8 | Gemini 2.5 Flash | medium | 81.2% |
| 9 | o3 | high | 81.1% |
| 10 | o4 mini | high | 77.8% |
| 11 | Opus 4.1 | high | 74.4% |
| 12 | Opus 4 | medium | 73.0% |
| 13 | Sonnet 4 | high | 72.7% |
| 14 | Grok 4.0709 | – | 72.2% |
| 15 | Gemini 2.0 Flash | medium | 72.2% |
| 16 | o1 | high | 72.1% |
| 17 | GPT-4.1 | high | 70.4% |
| 18 | Gemini 1.5 Pro | none | 68.3% |
| 19 | Sonnet 3.7 | medium | 67.7% |
| 20 | Sonnet 3.5 | medium | 66.2% |
| 21 | Gemma 3 27B IT | medium | 63.3% |
| 22 | Gemini 1.5 Flash | none | 60.5% |
| 23 | GPT-4o | high | 59.8% |
| 24 | Gemma 3 12B IT | – | 59.4% |
| 25 | Haiku 3.5 | medium | 55.2% |
| 26 | GPT-4o mini | high | 52.5% |
| 27 | Gemma 3 4B IT | – | 50.0% |
| 28 | DeepSeek V3.1 | none | 37.8% |