Benchmarks / Reasoning
MMMU
A multimodal benchmark of 11.5K college-level questions across six disciplines and 30 subjects, with 30 heterogeneous image types. Maintained by MMMU Team. Score: Zero-shot accuracy (percent) on multimodal college-level questions, reported overall and per discipline for the Val and Test splits.
MMMU
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | GPT-5.1 | high | 85.4% |
| 2 | GPT-5 Mini | high | 84.6% |
| 3 | GPT-5 | high | 84.2% |
| 4 | Gemini 2.5 Pro | high | 84.0% |
| 5 | o3 | high | 82.9% |
| 6 | o4 mini | high | 81.6% |
| 7 | Opus 4.5 | high | 80.7% |
| 8 | Gemini 2.5 Flash | medium | 79.7% |
| 9 | Qwen 3 | high | 78.7% |
| 10 | o1 | high | 78.2% |
| 11 | Grok 3 | none | 78.0% |
| 12 | Sonnet 4.5 | high | 77.8% |
| 13 | Opus 4.1 | high | 77.1% |
| 14 | Opus 4 | medium | 76.5% |
| 15 | Llama 4 Behemoth | – | 76.1% |
| 16 | GLM 4.5 V | none | 75.4% |
| 17 | Sonnet 3.7 | medium | 75.0% |
| 18 | GPT-4.5 | none | 74.4% |
| 19 | Sonnet 4 | high | 74.4% |
| 20 | Llama 4 Maverick | medium | 73.4% |
| 21 | Gemini 2.0 Pro | none | 72.7% |
| 22 | Gemini 2.0 Flash | medium | 71.7% |
| 23 | Llama 4 Scout | medium | 69.4% |
| 24 | GPT-4o | high | 69.1% |
| 25 | Gemini 2.0 Flash-Lite | none | 68.0% |
| 26 | GLM 4.1 V | – | 68.0% |
| 27 | Gemini 1.5 Pro | none | 65.8% |
| 28 | Gemma 3 27B | none | 64.9% |
| 29 | Qwen 2 | – | 64.5% |
| 30 | Llama 3.2 90b | none | 60.3% |
| 31 | Gemma 3 12B | none | 59.6% |
| 32 | Gemini 1.0 Ultra | none | 59.4% |
| 33 | Opus 3 | none | 59.4% |
| 34 | GPT-4o mini | high | 59.4% |
| 35 | Llama 3 76b | – | 58.3% |
| 36 | Gemini 1.5 Flash | none | 56.1% |
| 37 | Sonnet 3 | none | 53.1% |
| 38 | Qwen 2.5 | – | 53.1% |
| 39 | Llama 3.2 11b | none | 50.7% |
| 40 | Haiku 3 | medium | 50.2% |
| 41 | Gemma 2 9B | – | 49.8% |
| 42 | Gemma 3 4B | none | 48.8% |
| 43 | Llama 3 8b | none | 48.6% |
| 44 | Gemini 1.0 Pro | none | 47.9% |
| 45 | Llama 3 | – | 46.6% |
| 46 | Mistral | none | 35.3% |