Benchmarks / Reasoning
BIG-Bench Hard
Uses Epoch AI's external BIG-Bench Hard aggregate leaderboard. Score: BBH average. Collected by Jake Boggs' benchmark dashboard.
BIG-Bench Hard
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | Gemini 1.5 Pro | none | 89.2% |
| 2 | DeepSeek V3 | none | 87.5% |
| 3 | Llama 3.1 405b | none | 82.9% |
| 4 | Qwen 2.5 72B | none | 79.8% |
| 5 | DeepSeek V2 | none | 78.8% |
| 6 | GPT-4 | high | 75.1% |
| 7 | Llama 2 70b | none | 64.9% |
| 8 | GPT-3.5 | high | 61.6% |
| 9 | Llama 6 5b | none | 58.4% |
| 10 | Llama 2 13b | none | 58.2% |
| 11 | Mistral | none | 56.1% |
| 12 | Llama 3 3b | – | 50.0% |
| 13 | Llama 2 34b | – | 44.1% |
| 14 | Llama 2 7b | none | 39.2% |
| 15 | Llama 1 3b | – | 37.9% |
| 16 | Qwen 1 | – | 28.2% |