Terminal-Bench v2.1-AA

Terminal-Bench v2.1 run by AA: agentic command-line software engineering tasks; task pass rate.

183models

–Open Index weight

–last updated

Artificial Analysispublisher

#	Model	Effort	Score	Cost / task	Harness
1	Fable 5.1	max	91.4%	–	–
2	GPT-6 Astra	high	89.9%	–	–
3	GPT-5.6 Sol	xhigh	89.5%	–	–
4	Opus 5	max	89.1%	–	–
5	Qwen 3.8 Max	max	88.8%	–	–
6	Grok 4.6	high	88.4%	–	–
7	GPT-5.6 Terra	max	88.0%	–	–
8	Gemini 3.8 Flash	high	87.6%	–	–
9	Qwen 3.8 Flash	–	86.1%	–	–
10	Gemini 3.7 Flash	high	85.8%	–	–
11	Muse Spark 1.3	xhigh	85.4%	–	–
12	Kimi K3	max	85.0%	–	–
13	Fable 5	max	84.6%	–	–
14	Opus 4.8	max	84.6%	–	–
15	GLM 5.3 Flash	max	84.3%	–	–
16	GPT-5.5	xhigh	84.3%	–	–
17	GLM 5.3	max	83.9%	–	–
18	Opus 4.7	max	83.1%	–	–
19	Qwen 3.8 2.4T-A95B	–	82.0%	–	–
20	Grok 4.5	high	81.6%	–	–
21	GPT-5.6 Luna	max	80.9%	–	–
22	Sonnet 5	max	80.5%	–	–
23	Muse Spark 1.2	xhigh	80.1%	–	–
24	Qwen 3.8 27B	xhigh	79.8%	–	–
25	DeepSeek V4 Pro 0813	max	78.7%	–	–
26	DeepSeek V4 Flash 0731	max	78.7%	–	–
27	Gemini 3.5 Flash	high	78.7%	–	–
28	GPT-5.4	xhigh	78.3%	–	–
29	Muse Spark 1.1	xhigh	77.9%	–	–
30	GLM 5.2	max	77.9%	–	–
31	Gemini 3.6 Flash	high	77.5%	–	–
32	Muse Spark Open (Xhigh)	xhigh	76.4%	–	–
33	Motif 3	–	74.9%	–	–
34	Qwen 3.7 Max	max	74.5%	–	–
35	DeepSeek V4 Flash	max	74.2%	–	–
36	Gemini 3.1 Pro	–	73.8%	–	–
37	K2 Horizon 375B A23B	–	71.9%	–	–
38	Sonnet 4.6	max	71.2%	–	–
39	Motif 3 (Beta)	–	70.8%	–	–
40	KAT Coder Pro V2	none	70.0%	–	–
41	Agnes 2.5 Pro Beta	–	69.7%	–	–
42	Apodex 1.1	–	69.7%	–	–
43	Quasar 438B (Max, Based on GLM-5.2)	max	69.3%	–	–
44	Nex-N2-Pro	–	67.8%	–	–
45	Kimi K2.7 Code	–	67.4%	–	–
46	Kimi K2.6	–	65.9%	–	–
47	MiniMax M3	–	65.2%	–	–
48	MiMo V2.5 Pro	–	65.2%	–	–
49	DeepSeek V4 Pro 0424	high	64.8%	–	–
50	Hy3	–	64.4%	–	–
51	Ling 3.0 Flash	–	64.4%	–	–
52	MiMo V2.5	–	63.7%	–	–
53	Ling 3.0 Flash Fin	–	62.5%	–	–
54	Muse Spark	–	62.2%	–	–
55	GLM 5.1	–	61.8%	–	–
56	DeepSeek V4 Flash 0420	max	61.8%	–	–
57	MiMo V2 Flash	none	61.8%	–	–
58	Qwen 3.6 Plus	–	61.4%	–	–
59	Qwen 3.7 Plus	–	61.0%	–	–
60	Qwen 3.6 27B	–	60.7%	–	–
61	GPT-5.4 Nano	xhigh	60.7%	–	–
62	JT-4.1 Flash 236B A21B (Non-reasoning)	none	59.6%	–	–
63	GPT-5.4 Mini	xhigh	59.2%	–	–
64	Solar Pro 4	–	57.3%	–	–
65	K2 Horizon MoVA 36B A4B	–	56.6%	–	–
66	Sonnet 4.5	–	55.8%	–	–
67	MiniMax M2.7	–	55.4%	–	–
68	Ling 3.0 Flash	–	55.4%	–	–
69	Inkling Small	–	55.1%	–	–
70	Inkling	xhigh	55.1%	–	–
71	Nemotron 3 Ultra	max	53.9%	–	–
72	Gemini 3.5 Flash-Lite	high	53.6%	–	–
73	GPT-5.1	high	52.4%	–	–
74	Grok Build 0.1	–	52.1%	–	–
75	Muse Glimmer (High)	high	51.7%	–	–
76	Qwen 3.5 397B-A17B	–	51.3%	–	–
77	Mistral Medium 3.5	high	50.6%	–	–
78	LongCat 2.0	–	50.2%	–	–
79	GLM 4.6	–	49.4%	–	–
80	Qwen 3.5 122B-A10B	–	47.6%	–	–
81	DeepSeek V3.2	–	46.8%	–	–
82	Kimi K2.5	–	45.7%	–	–
83	GLM 4.7	–	45.3%	–	–
84	Qwen 3.6 35B-A3B	–	44.9%	–	–
85	DeepSeek V3.1	–	44.9%	–	–
86	Solar Open2 250B	–	44.2%	–	–
87	K2 Horizon 7B	–	44.2%	–	–
88	Haiku 4.5	–	44.2%	–	–
89	Gemma 4 31B	–	43.4%	–	–
90	Ring-2.6-1T	xhigh	43.1%	–	–
91	Qwen 3.5 35B-A3B	none	40.8%	–	–
92	K-EXAONE 2.0 0803	–	40.4%	–	–
93	Grok 4.3	high	39.7%	–	–
94	Step 3.7 Flash	high	39.3%	–	–
95	A.X-K2	–	39.0%	–	–
96	Gemma 4 26B	–	39.0%	–	–
97	Nemotron 3 Super 120B A12B (Reasoning)	–	38.6%	–	–
98	Qwen 3 Coder	none	38.2%	–	–
99	Sonnet 4	–	36.3%	–	–
100	North Mini Code	–	35.6%	–	–
101	GPT-5	high	35.2%	–	–
102	GPT-5.5 Instant	–	34.8%	–	–
103	G9v3-39A5B	–	32.6%	–	–
104	Gemini 3.1 Flash-Lite	high	31.1%	–	–
105	K-EXAONE (Reasoning)	–	30.3%	–	–
106	Devstral 2	none	30.3%	–	–
107	Nova 2.0 Pro Preview (Medium)	medium	29.6%	–	–
108	Devstral Small 2	none	29.6%	–	–
109	Qwen 3.5 9B	–	29.2%	–	–
110	Gemini 2.5 Pro	–	28.5%	–	–
111	K2 Horizon 3.7B	–	28.1%	–	–
112	Ling 3.0 Tiny	–	27.7%	–	–
113	Gemma 4 12B	–	27.3%	–	–
114	Mercury 2	high	27.3%	–	–
115	Granite 4.2 30B	–	26.6%	–	–
116	gpt-oss-120b	high	26.2%	–	–
117	Mistral Small 3.1	none	26.2%	–	–
118	Qwen 3.5 4B	–	25.8%	–	–
119	Nemotron 3.5 Lightning	–	24.3%	–	–
120	Ling 2.6 Flash	none	24.3%	–	–
121	Command A	–	22.8%	–	–
122	EXAONE 4.5 33B (Reasoning)	–	21.3%	–	–
123	Mistral Small 4	–	21.0%	–	–
124	Nemotron Cascade 2 30B A3B	–	20.6%	–	–
125	Trinity Large Thinking	–	20.6%	–	–
126	Nova 2.0 Pro Preview (Low)	low	19.5%	–	–
127	DeepSeek R1	–	19.1%	–	–
128	HyperNova 60B 2605 (High, Based on gpt-oss-120b)	high	18.4%	–	–
129	Granite 4.2 8B	–	18.4%	–	–
130	Nova 2.0 Pro Preview (Non-reasoning)	none	17.2%	–	–
131	DeepSeek V3	none	16.9%	–	–
132	Nova 2.0 Lite (High)	high	16.1%	–	–
133	K2 Think V2	high	15.0%	–	–
134	Mistral Medium 3.1	none	13.9%	–	–
135	Granite 4.2 3B	–	13.9%	–	–
136	gpt-oss-20b	high	13.9%	–	–
137	Magistral Medium 1.2	medium	12.4%	–	–
138	DiffusionGemma 26B A4B	–	12.4%	–	–
139	Qwen 3 235B-A22B	–	12.0%	–	–
140	Mistral Large 3	none	12.0%	–	–
141	Solar Pro 3	high	12.0%	–	–
142	Celeris-1	none	11.2%	–	–
143	GPT-4.1 Mini	none	10.1%	–	–
144	Haiku 3.5	none	10.1%	–	–
145	Ministral 3 14B	none	9.7%	–	–
146	MiniCPM5-2B	–	8.6%	–	–
147	Llama 4 Maverick	none	7.9%	–	–
148	Qwen 3	–	6.7%	–	–
149	Nemotron 3 Nano Omni 30B A3B Reasoning	–	6.7%	–	–
150	NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)	–	6.7%	–	–
151	G9v3-3B	–	6.0%	–	–
152	Mistral Small 3.2	none	5.6%	–	–
153	GPT-4o mini	none	5.6%	–	–
154	Qwen 3 32B	–	5.2%	–	–
155	Llama 3.3 70b	none	4.9%	–	–
156	Qwen 3 14B	–	4.9%	–	–
157	o3 mini	high	4.5%	–	–
158	Magistral Small 1.2	–	4.5%	–	–
159	LFM2.5-2.6B	–	4.5%	–	–
160	Gemma 3 27B	none	4.5%	–	–
161	Ministral 3 8B	none	4.1%	–	–
162	GPT-5 Mini	high	3.7%	–	–
163	GPT-4.1 Nano	none	3.7%	–	–
164	Llama 4 Scout	none	3.7%	–	–
165	NVIDIA Nemotron 3 Nano 4B	–	3.7%	–	–
166	Granite 4.1 8B	none	3.4%	–	–
167	Qwen 3.5 2B	–	3.0%	–	–
168	Granite 4.1 30B	none	2.6%	–	–
169	Qwen 3 8B	–	2.2%	–	–
170	Gemma 4 E4B (Reasoning)	–	1.9%	–	–
171	Qwen 3 30B-A3B	–	1.5%	–	–
172	Llama 3.1 8b	none	1.5%	–	–
173	Nanbeige4.1-3B	–	1.1%	–	–
174	Granite 4.1 3B	none	1.1%	–	–
175	K2 Horizon 0.9B	–	1.1%	–	–
176	Gemma 3n E4B Instruct	none	0.7%	–	–
177	Gemma 4 E2B (Reasoning)	–	0.4%	–	–
178	Phi-4 Mini Instruct	none	0.4%	–	–
179	Qwen 3.5 0.8B	none	0.4%	–	–
180	Gemma 3 4B	none	0.4%	–	–
181	MiniCPM-V 4.6 1.3B	none	0.0%	–	–
182	Ministral 3 3B	none	0.0%	–	–
183	Gemma 3 12B	none	0.0%	–	–