Benchmarks / Coding
SWE Atlas Refactoring (Scale AI)
Whether a coding agent restructures production code while preserving its behaviour, graded by tests and rubrics. Maintained by Scale AI. Score: Leaderboard score (percent); ± is the published confidence half-width.
SWE Atlas Refactoring (Scale AI)
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | GPT-6 Astra Codex | xhigh | 59.0% |
| 2 | Fable 5.1 | xhigh | 56.7% |
| 3 | Fable 5 | xhigh | 54.8% |
| 4 | Opus 4.7 | max | 48.6% |
| 5 | Opus 4.8 | max | 46.7% |
| 6 | GPT-5.5 Codex | xhigh | 44.8% |
| 7 | Gemini 3.8 Flash | max | 44.8% |
| 8 | GPT-5.4 Codex | xhigh | 44.3% |
| 9 | GLM 5.2 | max | 42.4% |
| 10 | GPT-5.3 Codex | xhigh | 42.4% |
| 11 | Opus 4.6 | max | 35.6% |
| 12 | Gemini 3.1 Pro | high | 33.8% |
| 13 | Sonnet 4.6 | max | 32.2% |
| 14 | GLM 5 | high | 24.2% |
| 15 | Kimi K2.5 | max | 20.9% |
| 16 | MiniMax M2.5 | high | 19.5% |
| 17 | Gemini 3 Flash | high | 10.0% |