Benchmarks / Coding
Scale SEAL: SWE-Bench Pro V2 Full
Scale AI’s refreshed public SWE-Bench Pro split (642 long-horizon tasks in 11 copyleft repositories) run under a locked protocol, scored by resolve rate. Maintained by Scale AI. Score: Resolve rate on the full V2 public split (percent); ± is the published confidence half-width.
Scale SEAL: SWE-Bench Pro V2 Full
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | Opus 5 | xhigh | 99.4% |
| 2 | Fable 5.1 | high | 99.1% |
| 3 | Kimi K3 | max | 97.7% |
| 4 | GPT-6 Astra Codex | high | 96.9% |
| 5 | GPT-6.1 Sol Mini | xhigh | 96.1% |
| 6 | GLM 5.3 | max | 95.6% |
| 7 | GPT-5.6 Sol Codex | xhigh | 95.5% |
| 8 | Gemini 3.8 Flash | high | 94.9% |
| 9 | Sonnet 5 | xhigh | 93.2% |
| 10 | GPT-5.6 Terra Codex | xhigh | 92.4% |
| 11 | Inkling | xhigh | 89.9% |
| 12 | Haiku 5.5 | high | 87.2% |