Add a benchmark

Benchmarks / Professional

GDPval-AA v2.1

GDPval-AA: economically valuable knowledge-work tasks across occupations, graded pairwise by AA; reported as Elo.

203models
–Open Index weight
–last updated

GDPval-AA v2.1

Higher is better · best effort per model

Chart settings
203 of 203 selected

full table below

#ModelEffortScoreCost / taskHarness
1Opus 5.5max1866$8.92Stirrup
2Muse Wonder Full (Max)max1857–Stirrup
3Sonnet 5.5max1838$6.84Stirrup
4Muse Wondermax1770$4.89Stirrup
5Fable 5.1max1754$9.77Stirrup
6Opus 5max1721$6.77Stirrup
7Grok 4.7xhigh1711$6.43Stirrup
8MiMo V2.6 Pro–1684$0.161Stirrup
9Muse Spark 1.3max1681$2.04Stirrup
10Qwen 3.8 Maxmax1666$4.54Stirrup
11GLM 5.3max1650$2.26Stirrup
12Grok 4.6xhigh1645$2.44Stirrup
13GLM 5.3 Flashmax1644$0.336Stirrup
14Qwen 3.8 Flash–1628$0.297Stirrup
15Gemini 4high1624$1.82Stirrup
16Haiku 5.5max1618$0.222Stirrup
17Ling 3.1 Flash–1618$1.18Stirrup
18Fable 5max1610$7.75Stirrup
19GPT-5.6 Solmax1609$2.81Stirrup
20Qwen 3.8 2.4T-A95B–1608$2.53Stirrup
21MiMo V2.6 Flash–1608$0.073Stirrup
22DeepSeek V4.1 Flashmax1600$0.300Stirrup
23GPT-6.1 Solmax1594$0.894Stirrup
24Step 5 Preview–1587$0.979Stirrup
25GPT-6 Astramax1573$4.53Stirrup
26DeepSeek V4 Flashmax1547$0.346Stirrup
27Kimi K3max1533$2.15Stirrup
28GPT-6 Solmax1508$1.28Stirrup
29Muse Spark 1.2xhigh1473$0.880Stirrup
30Sonnet 5max1464$4.70Stirrup
31GPT-5.6 Lunamax1459$0.110Stirrup
32Opus 4.8max1453$4.34Stirrup
33GPT-5.6 Terramax1452$1.24Stirrup
34DeepSeek V4 Pro 0813max1450$0.580Stirrup
35Gemini 3.8 Flashhigh1436$1.69Stirrup
36DeepSeek V4 Flash 0731max1434$0.290Stirrup
37GPT-6 Lunamax1432$0.089Stirrup
38Mistral Large 4–1422$0.874Stirrup
39Qwen 3.8 27Bxhigh1420$1.10Stirrup
40Muse Spark Open (Xhigh)xhigh1401$0.691Stirrup
41Gemini 3.7 Flashhigh1392$1.25Stirrup
42Grok 4.5high1384$0.481Stirrup
43GLM 5.2max1372$1.39Stirrup
44K2 Horizon 375B A23B–1361–Stirrup
45Opus 4.7max1352–Stirrup
46GPT-5.5xhigh1351$2.19Stirrup
47Agnes 2.5 Pro Beta–1329–Stirrup
48JT-4.1 Flash 236B A21B (Reasoning)–1324–Stirrup
49Gemini 3.6 Flashhigh1284$0.903Stirrup
50GPT-5.4xhigh1246–Stirrup
51MiniMax M3–1242$0.348Stirrup
52Sonnet 4.6max1232$1.94Stirrup
53Muse Spark 1.1xhigh1212$0.288Stirrup
54Gemini 3.5 Flashhigh1203$3.53Stirrup
55K2 Horizon MoVA 36B A4B–1191–Stirrup
56Apodex 1.1–1191$0.477Stirrup
57Quasar 438B (Max, Based on GLM-5.2)max1182$1.39Stirrup
58Beam 1.0–1177–Stirrup
59Nemotron 3.5 Super VL–1162$0.153Stirrup
60Ling 3.0 Flash–1162–Stirrup
61DeepSeek V4 Pro 0424max1156$0.057Stirrup
62Qwen 3.7 Maxmax1128$0.655Stirrup
63Inkling Small–1124$0.089Stirrup
64JT-4.1 Flash 236B A21B (Non-reasoning)none1123–Stirrup
65MiMo V2.5 Pro–1121$0.041Stirrup
66Motif 3–1117–Stirrup
67GLM 5.1–1116$0.478Stirrup
68Solar Pro 4–1106$0.246Stirrup
69G9v3-39A5B–1104–Stirrup
70Nex-N2-Pro–1103–Stirrup
71Motif 3 (Beta)–1103–Stirrup
72Ling 3.0 Flash Fin–1101–Stirrup
73Solar Mini 4–1089$0.393Stirrup
74Inklingxhigh1076–Stirrup
75Grok Build 0.1–1067–Stirrup
76Hy3–1067$0.036Stirrup
77Kimi K2.6–1038$0.603Stirrup
78Kimi K2.7 Code–1038$0.396Stirrup
79DeepSeek V4 Flash 0420max1036$0.039Stirrup
80K2 Horizon 7B–1023–Stirrup
81GLM 4.7–1014–Stirrup
82Nemotron 3 Ultramax1013$0.420Stirrup
83GPT-5.4 Minixhigh1012$0.656Stirrup
84MiniMax M2.7–1010–Stirrup
85Muse Spark–998–Stirrup
86MiMo V2.5–997$0.0095Stirrup
87Qwen 3.6 Plus–990–Stirrup
88Qwen 3.6 27B–986$0.442Stirrup
89Gemini 3.5 Flash-Litehigh983$0.140Stirrup
90Nemotron 3.5 Bolt 74B-A9B–979–Stirrup
91Solar Open2 250B–971–Stirrup
92A.X-K2–964–Stirrup
93GPT-5.4 Nanoxhigh948$0.241Stirrup
94Ling 3.0 Flash–947–Stirrup
95Grok 4.3none942$0.212Stirrup
96GPT-5high917–Stirrup
97Sonnet 4.5–910$0.508Stirrup
98Qwen 3.6 35B-A3B–892$0.429Stirrup
99K2 Horizon 3.7B–882–Stirrup
100LongCat 2.0–870$0.213Stirrup
101Step 3.7 Flashhigh856–Stirrup
102Kimi K2.5–839–Stirrup
103GPT-5.1high826–Stirrup
104Qwen 3.5 122B-A10B–814$0.240Stirrup
105Qwen 3.5 397B-A17B–793$0.371Stirrup
106Gemini 3.1 Pro–792$0.416Stirrup
107Muse Glimmer (High)high787$0.038Stirrup
108Qwen 3.7 Plus–768$0.068Stirrup
109GPT-5 Minihigh766$0.049Stirrup
110Mistral Medium 3.5high761$0.410Stirrup
111GLM 4.6–757–Stirrup
112Ring-2.6-1Txhigh746$0.287Stirrup
113KAT Coder Pro V2none733–Stirrup
114Haiku 4.5–732$0.238Stirrup
115KAT-Coder-Pro V1none731–Stirrup
116DeepSeek V3.1–709–Stirrup
117MiniCPM5-2B–707–Stirrup
118DeepSeek V3.2–692–Stirrup
119Sonnet 4–691–Stirrup
120Nemotron 3.5 Lightning–639$0.085Stirrup
121MiMo V2 Flashnone628–Stirrup
122Gemma 4 31B–620$0.0000Stirrup
123gpt-oss-120bhigh609$0.105Stirrup
124Qwen 3.5 35B-A3Bnone608–Stirrup
125Granite 4.2 30B–576$0.055Stirrup
126Gemma 4 26B–566–Stirrup
127Ling 3.0 Tiny–561$0.0000Stirrup
128Devstral 2none544–Stirrup
129Devstral Small 2none528–Stirrup
130Command A–515$0.0000Stirrup
131Qwen 3 Codernone514$0.399Stirrup
132GPT-5.5 Instant–509$0.074Stirrup
133Mercury 2high494$0.027Stirrup
134Granite 4.2 8B–492$0.026Stirrup
135Nemotron 3 Super 120B A12B (Reasoning)–490$0.545Stirrup
136Mercury 2.5–475$0.042Stirrup
137Nova 2.0 Pro Preview (Medium)medium470–Stirrup
138EXAONE 4.5 33B (Reasoning)–467–Stirrup
139Gemini 2.5 Pro–456$0.384Stirrup
140HyperNova 60B 2605 (High, Based on gpt-oss-120b)high442–Stirrup
141Nova 2.0 Pro Preview (Low)low436–Stirrup
142Gemini 3.1 Flash-Litehigh431$0.022Stirrup
143Gemma 4 12B–427–Stirrup
144Mistral Large 3none420$0.029Stirrup
145Qwen 3.5 9B–420$0.616Stirrup
146Mistral Medium 3.1none386–Stirrup
147Mistral Small 3.1none377$0.019Stirrup
148K-EXAONE (Reasoning)–367–Stirrup
149Nova 2.0 Lite (High)high366–Stirrup
150Mistral Small 4–363$0.0084Stirrup
151gpt-oss-20bhigh340$0.082Stirrup
152Trinity Large Thinking–337$0.154Stirrup
153Nova 2.0 Pro Preview (Non-reasoning)none332–Stirrup
154DiffusionGemma 26B A4B–321–Stirrup
155Ling 2.6 Flashnone319–Stirrup
156Qwen 3 235B-A22B–314–Stirrup
157North Mini Code–312$0.0000Stirrup
158Celeris-1none302$0.085Stirrup
159DeepSeek R1–295$0.989Stirrup
160GPT-4.1 Mininone270–Stirrup
161Nemotron Cascade 2 30B A3B–265–Stirrup
162Solar Pro 3high265$0.181Stirrup
163NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)–255$0.028Stirrup
164Ministral 3 14Bnone248$0.030Stirrup
165o3 minihigh230–Stirrup
166Nemotron 3 Nano Omni 30B A3B Reasoning–230–Stirrup
167Ministral 3 8Bnone214$0.011Stirrup
168Haiku 3.5none211–Stirrup
169Granite 4.2 3B–202$0.0065Stirrup
170Granite 4.1 30Bnone188–Stirrup
171Magistral Medium 1.2medium169–Stirrup
172K2 Think V2high130–Stirrup
173Qwen 3–128–Stirrup
174DeepSeek V3none70$0.012Stirrup
175Qwen 3 30B-A3B–68$0.163Stirrup
176Qwen 3 32B–34–Stirrup
177Ministral 3 3Bnone30$0.023Stirrup
178Magistral Small 1.2–4–Stirrup
179LFM2.5-2.6B–-8–Stirrup
180GPT-4onone-16–Stirrup
181GPT-4none-20–Stirrup
182GPT-4o mininone-22–Stirrup
183Qwen 3 14B–-24–Stirrup
184Gemma 4 E4B (Reasoning)–-36$0.051Stirrup
185Qwen 3.5 2B–-46–Stirrup
186Qwen 3 8B–-48–Stirrup
187NVIDIA Nemotron 3 Nano 4B–-58–Stirrup
188Granite 4.1 3Bnone-145–Stirrup
189Llama 4 Scoutnone-163$0.019Stirrup
190Llama 3.3 70bnone-174–Stirrup
191Gemma 4 E2B (Reasoning)–-188–Stirrup
192Mistral Small 3.2none-195$0.335Stirrup
193GPT-4.1 Nanonone-213–Stirrup
194Llama 4 Mavericknone-275$0.089Stirrup
195NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning)none-356–Stirrup
196MiniCPM-V 4.6 1.3Bnone-369–Stirrup
197Qwen 3.5 0.8Bnone-370–Stirrup
198Llama 3.1 8bnone-387–Stirrup
199Nanbeige4.1-3B–-404–Stirrup
200Gemma 3 27Bnone-408$0.0000Stirrup
201Gemma 3 12Bnone-408$0.0000Stirrup
202Phi-4 Mini Instructnone-408–Stirrup
203K2 Horizon 0.9B–-409–Stirrup