Benchmarks / Coding
SWE-bench Verified (Epoch AI)
The human-verified SWE-bench subset: resolve real GitHub issues in a repository. Run by Epoch AI with its own scaffold; resolve rate. Data: Epoch AI, CC BY.
SWE-bench Verified (Epoch AI)
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | Opus 4.7 | max | 83.5% |
| 2 | GPT-5.5 | xhigh | 80.6% |
| 3 | Gemini 3.5 Flash | high | 79.3% |
| 4 | Opus 4.6 | max | 78.7% |
| 5 | GLM 5.2 | max | 78.7% |
| 6 | DeepSeek V4 Pro | max | 77.6% |
| 7 | Qwen 3.7 Max | max | 77.3% |
| 8 | GPT-5.4 | high | 76.9% |
| 9 | Qwen 3.6 Max | max | 76.7% |
| 10 | Kimi K2.6 | max | 76.7% |
| 11 | Opus 4.5 | high | 76.7% |
| 12 | Gemini 3.1 Pro | high | 75.6% |
| 13 | Gemini 3 Flash | high | 75.4% |
| 14 | Sonnet 4.6 | max | 75.2% |
| 15 | GPT-5.3 Codex | high | 74.8% |
| 16 | GLM 5.1 | max | 74.2% |
| 17 | Kimi K2.5 | max | 73.8% |
| 18 | GPT-5.2 | high | 73.8% |
| 19 | GPT-5 | high | 73.6% |
| 20 | Opus 4.1 | high | 73.3% |
| 21 | Gemini 3 Pro | high | 72.9% |
| 22 | GLM 5 | high | 72.1% |
| 23 | Sonnet 4.5 | high | 71.3% |
| 24 | Opus 4 | medium | 70.7% |
| 25 | GPT-5.1 | high | 68.0% |
| 26 | GPT-5 Mini | medium | 64.7% |
| 27 | o3 | medium | 62.3% |
| 28 | Sonnet 3.7 | medium | 61.0% |
| 29 | Qwen 3.6 Plus | max | 57.9% |
| 30 | Gemini 2.5 Pro | high | 57.6% |
| 31 | GPT-4.1 | high | 48.5% |
| 32 | GPT-4o | high | 31.0% |