Benchmarks / Reasoning
WinoGrande
Uses Epoch AI's external WinoGrande accuracy leaderboard. Random baseline is 50% for binary choice. Score: WinoGrande accuracy. Collected by Jake Boggs' benchmark dashboard.
WinoGrande
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | Llama 3.1 405b | none | 89.2% |
| 2 | Opus 3 | none | 88.5% |
| 3 | GPT-4 | high | 87.5% |
| 4 | DeepSeek V2 | none | 86.3% |
| 5 | DeepSeek V3 | none | 85.2% |
| 6 | Llama 3 70b | none | 83.5% |
| 7 | Qwen 2.5 72B | none | 82.3% |
| 8 | GPT-3.5 | high | 81.6% |
| 9 | Qwen 2.5 Coder | none | 80.8% |
| 10 | Llama 2 70b | none | 80.2% |
| 11 | Mixtral 8x7B Instruct | none | 77.2% |
| 12 | Llama 6 5b | none | 77.0% |
| 13 | Llama 2 34b | – | 76.7% |
| 14 | Llama 3 3b | – | 76.0% |
| 15 | Llama 3 8b | none | 75.7% |
| 16 | Mistral | none | 75.3% |
| 17 | Sonnet 3 | none | 75.1% |
| 18 | Haiku 3 | medium | 74.2% |
| 19 | Llama 1 3b | – | 73.0% |
| 20 | Llama 2 13b | none | 72.8% |
| 21 | Llama 2 7b | none | 69.2% |
| 22 | GPT-2 | – | 58.3% |