Benchmarks / Reasoning
Arena-Hard-Auto v0.1
Uses LMArena's Arena-Hard-Auto v0.1 leaderboard CSV; GPT-4-0314 is the anchor model. Score: Arena-Hard-Auto score. Collected by Jake Boggs' benchmark dashboard.
Arena-Hard-Auto v0.1
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | GPT-4 | high | 82.6% |
| 2 | Sonnet 3.5 | medium | 79.3% |
| 3 | GPT-4o | high | 79.2% |
| 4 | GPT-4o mini | high | 74.9% |
| 5 | Gemini 1.5 Pro | none | 72.0% |
| 6 | Mistral Large 2407 | – | 70.4% |
| 7 | Llama 3.1 405b | none | 64.1% |
| 8 | GLM 4.0520 | – | 63.8% |
| 9 | Opus 3 | none | 60.4% |
| 10 | Gemma 2 27B IT | – | 57.5% |
| 11 | Llama 3.1 70b | none | 55.7% |
| 12 | GLM 4.0116 | – | 55.7% |
| 13 | GLM 4 Air | – | 50.9% |
| 14 | Gemini 1.5 Flash | none | 49.6% |
| 15 | Qwen 2 72B | none | 46.9% |
| 16 | Sonnet 3 | none | 46.8% |
| 17 | Llama 3 70b | none | 46.6% |
| 18 | Haiku 3 | medium | 41.5% |
| 19 | Mistral Large 2402 | – | 37.7% |
| 20 | Qwen 1.5 72B | – | 36.1% |
| 21 | Mistral Medium | none | 31.9% |
| 22 | Mistral | none | 27.4% |
| 23 | GPT-3.5 | high | 24.8% |
| 24 | Llama 3.1 8b | medium | 21.3% |
| 25 | Llama 3 8b | none | 20.6% |
| 26 | Gemma 1.1 7B IT | – | 12.1% |
| 27 | Llama 2 70b | none | 11.6% |
| 28 | Gemma 1.1 2B IT | – | 3.4% |