Benchmarks / Reasoning
ARC AI2
Uses Epoch AI's external ARC Challenge leaderboard. Random baseline is treated as 25% for multiple choice. Score: ARC AI2 Challenge score. Collected by Jake Boggs' benchmark dashboard.
ARC AI2
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | DeepSeek V3 | none | 95.3% |
| 2 | Llama 3.1 405b | none | 95.3% |
| 3 | Qwen 2.5 72B | none | 94.5% |
| 4 | DeepSeek V2 | none | 92.2% |
| 5 | GPT-3.5 | high | 87.4% |
| 6 | Mixtral 8x7B Instruct | none | 87.3% |
| 7 | Llama 3 8b | none | 82.8% |
| 8 | Mistral | none | 78.6% |
| 9 | Llama 2 70b | none | 78.3% |
| 10 | Qwen 2.5 Coder | none | 70.5% |
| 11 | Llama 6 5b | none | 69.5% |
| 12 | Llama 3 3b | – | 67.5% |
| 13 | Llama 2 13b | none | 60.3% |
| 14 | Llama 2 34b | – | 54.5% |
| 15 | Qwen 1 | – | 53.2% |
| 16 | Llama 1 3b | – | 52.7% |
| 17 | Llama 2 7b | none | 45.9% |
| 18 | GPT-2 | – | 25.0% |