Benchmarks / Reasoning
OpenBookQA
Uses Epoch AI's external OpenBookQA accuracy leaderboard. Random baseline is 25% for four-way multiple choice. Score: OpenBookQA accuracy. Collected by Jake Boggs' benchmark dashboard.
OpenBookQA
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | GPT-3.5 | high | 86.0% |
| 2 | Mixtral 8x7B Instruct | none | 85.8% |
| 3 | Llama 3 8b | none | 82.6% |
| 4 | Mistral | none | 79.8% |
| 5 | Llama 2 70b | none | 60.2% |
| 6 | Llama 6 5b | none | 60.2% |
| 7 | Llama 2 7b | none | 58.6% |
| 8 | Llama 3 3b | – | 58.6% |
| 9 | Llama 2 34b | – | 58.2% |
| 10 | Llama 2 13b | none | 57.0% |
| 11 | Llama 1 3b | – | 56.4% |
| 12 | GPT-2 | – | 22.4% |