Benchmarks / Coding

FrontierCode 1.1 Extended

Mergeability of coding-agent changes on private maintainer-authored tasks, graded by unit tests, rubrics and verifiers (Cognition).

46models
–Open Index weight
–last updated
Cognitionpublisher

FrontierCode 1.1 Extended

Higher is better · best effort per model

Chart settings
46 of 46 selected

full table below

#ModelEffortScoreCost / taskHarness
1Opus 5.5medium65.3%$0.667claude-code
2Fable 5xhigh64.9%$10.53claude-code
3GPT-6 Astramax64.5%$3.93codex
4Sonnet 5.5xhigh64.4%$1.24claude-code
5Opus 5medium63.6%$3.51claude-code
6Fable 5.1medium63.6%$2.68claude-code
7SWE 2max62.5%$0.940devin
8Grok 4.6high61.3%$2.38grok-build
9GPT-6 Solmax60.7%$1.66codex
10Gemini 4low60.6%$6.14chisel
11GPT-5.6 Solmax60.6%$4.25codex
12GPT-6.1 Solmedium60.4%$0.309codex
13Opus 4.8max59.6%$8.05claude-code
14Grok 4.7high59.4%$5.59grok-build
15Haiku 5.5max58.4%$1.08claude-code
16Kimi K3none58.2%$3.12mini-swe-agent
17GPT-5.5xhigh56.7%$3.34codex
18DeepSeek V4.1 Flashhigh56.6%$0.500chisel
19Grok 4.5high56.5%$1.09grok-build
20Gemini 3.7 Flashmedium56.3%$1.53chisel
21Sonnet 5xhigh56.2%$8.23claude-code
22GPT-6 Lunamax56.1%$0.083codex
23GPT-5.6 Terramax55.8%$1.57codex
24GPT-5.6 Lunamax55.1%$0.307codex
25SWE 1.7none54.3%$1.64chisel
26Opus 4.7max53.9%$7.61claude-code
27Gemini 3.8 Flashmedium53.4%$2.21chisel
28Mistral Large 4high53.2%$3.48chisel
29GLM 5.3max52.6%$13.79chisel
30Gemini 3.6 Flashmedium48.0%$3.41chisel
31GLM 5.3 Flashmax46.2%$0.944chisel
32Kimi K2.7none45.4%$2.56mini-swe-agent
33DeepSeek V4 Pro 0813high44.5%$1.50chisel
34Opus 4.6high43.7%$3.30claude-code
35GPT-5.4 Minixhigh43.0%$1.26codex
36Composer 2.5none40.8%$2.57cursor-cli
37GLM 5.2none40.1%$2.14mini-swe-agent
38Sonnet 4.6max40.0%$2.44claude-code
39DeepSeek V4 Flash 0731high31.7%$1.31chisel
40DeepSeek V4 Pronone31.0%$1.37mini-swe-agent
41MiniMax M3none28.5%$0.592mini-swe-agent
42Nemotron 3 Ultra 550B A55Bnone27.2%$1.32chisel
43Inklingmax24.8%$3.20mini-swe-agent
44Qwen 3.7 Plusnone21.8%$0.212mini-swe-agent
45SWE 1.6none20.5%$0.450chisel
46Mistral 3.5none18.7%$1.24chisel