Benchmarks / Coding

FrontierCode 1.1 Main

Mergeability of coding-agent changes on private maintainer-authored tasks, graded by unit tests, rubrics and verifiers (Cognition).

46models
–Open Index weight
–last updated
Cognitionpublisher

FrontierCode 1.1 Main

Higher is better · best effort per model

Chart settings
46 of 46 selected

full table below

#ModelEffortScoreCost / taskHarness
1Opus 5.5medium54.6%$0.802claude-code
2Fable 5xhigh53.5%$13.09claude-code
3Opus 5medium53.4%$4.31claude-code
4GPT-6 Astramax53.3%$4.59codex
5Sonnet 5.5xhigh52.1%$1.59claude-code
6Fable 5.1medium50.9%$3.28claude-code
7GPT-6.1 Solmedium50.2%$0.361codex
8SWE 2max50.0%$1.18devin
9GPT-6 Solmax49.3%$2.07codex
10Grok 4.6high48.0%$2.88grok-build
11Grok 4.7high47.6%$6.65grok-build
12Gemini 4medium47.5%$6.74chisel
13GPT-5.6 Solmax47.5%$5.19codex
14Opus 4.8max46.5%$9.62claude-code
15Haiku 5.5max46.4%$1.33claude-code
16Kimi K3none44.2%$3.82mini-swe-agent
17DeepSeek V4.1 Flashhigh43.9%$0.620chisel
18Gemini 3.7 Flashmedium43.6%$1.82chisel
19GPT-5.5xhigh43.0%$4.03codex
20Sonnet 5xhigh42.7%$10.07claude-code
21Grok 4.5high42.4%$1.30grok-build
22GPT-6 Lunamax42.4%$0.103codex
23SWE 1.7none42.0%$1.97chisel
24GPT-5.6 Terramax41.3%$1.87codex
25Gemini 3.8 Flashmedium41.2%$2.60chisel
26GLM 5.3max40.1%$16.91chisel
27GPT-5.6 Lunamax39.8%$0.370codex
28Mistral Large 4high39.4%$4.13chisel
29Opus 4.7max38.5%$9.09claude-code
30Gemini 3.6 Flashmedium34.4%$4.04chisel
31GLM 5.3 Flashmax31.8%$1.15chisel
32Kimi K2.7none30.1%$3.01mini-swe-agent
33DeepSeek V4 Pro 0813high28.5%$1.81chisel
34GPT-5.4 Minixhigh27.0%$1.52codex
35Opus 4.6high26.6%$3.98claude-code
36Composer 2.5none25.6%$3.09cursor-cli
37GLM 5.2none24.5%$2.47mini-swe-agent
38Sonnet 4.6max24.3%$2.90claude-code
39DeepSeek V4 Flash 0731high18.8%$1.53chisel
40DeepSeek V4 Pronone17.6%$1.55mini-swe-agent
41MiniMax M3none14.7%$0.676mini-swe-agent
42Inklingmax14.0%$3.60mini-swe-agent
43Nemotron 3 Ultra 550B A55Bnone13.6%$1.47chisel
44Qwen 3.7 Plusnone10.2%$0.241mini-swe-agent
45SWE 1.6none9.4%$0.489chisel
46Mistral 3.5none8.0%$1.35chisel