Benchmarks / Coding
FrontierCode 1.1-AA
AA's run of Cognition's FrontierCode 1.1 Main (100 private tasks, three attempts each, 250-turn cap): would a repository's maintainers merge the change? Mean score; an attempt failing a blocking check scores 0. Separate from Cognition's own leaderboard, which uses different harnesses.
FrontierCode 1.1-AA
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | Opus 5.5 | medium | 57.1% |
| 2 | GPT-6.1 Sol | max | 49.1% |
| 3 | Fable 5.1 | max | 49.0% |
| 4 | GPT-6 Astra | max | 47.0% |
| 5 | Sonnet 5.5 | xhigh | 46.1% |
| 6 | Haiku 5.5 | xhigh | 45.5% |
| 7 | MiMo V2.6 Pro | – | 45.4% |
| 8 | DeepSeek V4.1 Flash | max | 44.0% |
| 9 | Kimi K3 | max | 43.6% |
| 10 | Gemini 4 | high | 43.5% |
| 11 | Qwen 3.8 Max | max | 43.2% |
| 12 | MiMo V2.6 Flash | – | 42.1% |
| 13 | GPT-6 Luna | max | 41.9% |
| 14 | GLM 5.3 Flash | max | 41.0% |
| 15 | Muse Spark 1.3 | max | 40.6% |
| 16 | GLM 5.3 | max | 40.1% |
| 17 | Grok 4.7 | low | 39.6% |
| 18 | Step 5 Preview | – | 38.4% |
| 19 | Gemini 3.8 Flash | high | 36.4% |
| 20 | Qwen 3.8 27B | xhigh | 34.2% |
| 21 | K2 Horizon 375B A23B | – | 32.1% |
| 22 | MiniMax M3 | – | 23.0% |
| 23 | Inkling | xhigh | 15.8% |
| 24 | Muse Glimmer (High) | high | 13.9% |
| 25 | Nemotron 3 Ultra | max | 10.5% |