Terminal-Bench Hard-AA

Terminal-Bench Hard (AA subset of Terminal-Bench 2.0): agentic terminal tasks; pass rate.

288models

–Open Index weight

–last updated

Artificial Analysispublisher

#	Model	Effort	Score	Cost / task	Harness
1	GPT-5.6 Sol	max	65.9%	–	–
2	Fable 5	max	62.9%	–	–
3	GPT-5.6 Terra	xhigh	62.9%	–	–
4	GPT-5.5	xhigh	60.6%	–	–
5	Opus 4.8	max	58.3%	–	–
6	GPT-5.4	xhigh	57.6%	–	–
7	Opus 4.7	high	54.5%	–	–
8	Gemini 3.1 Pro	–	53.8%	–	–
9	GPT-5.3 Codex	xhigh	53.0%	–	–
10	Sonnet 4.6	max	53.0%	–	–
11	GPT-5.4 Mini	xhigh	52.3%	–	–
12	GLM 5.2	max	50.8%	–	–
13	Qwen 3.7 Max	max	50.8%	–	–
14	KAT Coder Pro V2	none	49.2%	–	–
15	Opus 4.6	high	48.5%	–	–
16	GPT-5.2	xhigh	47.0%	–	–
17	Opus 4.5	–	47.0%	–	–
18	Qwen 3.7 Plus	–	47.0%	–	–
19	Gemini 3.5 Flash	minimal	46.2%	–	–
20	DeepSeek V4 Pro 0424	max	46.2%	–	–
21	Muse Spark	–	45.5%	–	–
22	GPT-5.1	high	45.5%	–	–
23	Kimi K2.7 Code	–	44.7%	–	–
24	Qwen 3.6 Max	max	43.9%	–	–
25	Qwen 3.6 Plus	–	43.9%	–	–
26	Kimi K2.6	–	43.9%	–	–
27	GLM 5	–	43.2%	–	–
28	GLM 5.1	–	43.2%	–	–
29	MiMo V2.5 Pro	–	43.2%	–	–
30	MiniMax M3	–	42.4%	–	–
31	GPT-5.5 Instant	–	42.4%	–	–
32	GPT-5.4 Nano	xhigh	42.4%	–	–
33	Gemini 3 Pro	high	41.7%	–	–
34	MiMo V2.5	–	41.7%	–	–
35	MiMo V2 Pro	–	40.9%	–	–
36	Grok 4.20	–	40.9%	–	–
37	Qwen 3.5 397B-A17B	–	40.9%	–	–
38	MiniMax M2.7	–	39.4%	–	–
39	Gemini 3 Flash	–	38.6%	–	–
40	DeepSeek V4 Flash 0420	high	38.6%	–	–
41	GPT-5 Codex	high	37.9%	–	–
42	Grok 4.3	high	37.9%	–	–
43	GPT-5	medium	37.9%	–	–
44	Grok 4	–	37.9%	–	–
45	GPT-5.2 Codex	xhigh	37.1%	–	–
46	o3	–	37.1%	–	–
47	Nemotron 3 Ultra	max	36.4%	–	–
48	Gemma 4 31B	–	36.4%	–	–
49	MiMo V2	–	35.6%	–	–
50	DeepSeek V3.2	–	35.6%	–	–
51	Sonnet 4.5	–	35.6%	–	–
52	Step 3.7 Flash	high	35.6%	–	–
53	GPT-5.1 Codex	high	34.8%	–	–
54	Kimi K2.5	–	34.8%	–	–
55	MiniMax M2.5	–	34.8%	–	–
56	Qwen 3.6 27B	–	34.8%	–	–
57	Qwen 3.6 35B-A3B	–	34.8%	–	–
58	Opus 4.1	–	34.3%	–	–
59	Hy3	–	34.1%	–	–
60	GPT-5 Mini	high	33.3%	–	–
61	GPT-5.1 Codex Mini	high	33.3%	–	–
62	Mistral Medium 3.5	high	33.3%	–	–
63	GLM 5 V	–	32.6%	–	–
64	Qwen 3.5 27B	–	32.6%	–	–
65	Step 3.5 Flash 2603	–	32.6%	–	–
66	GLM 4.7	–	31.8%	–	–
67	DeepSeek V3.1	none	31.8%	–	–
68	MiMo V2 Flash	–	31.1%	–	–
69	Kimi K2	–	31.1%	–	–
70	Opus 4	–	31.1%	–	–
71	Sonnet 4	–	31.1%	–	–
72	Qwen 3.5 122B-A10B	–	31.1%	–	–
73	Ling-2.6-1T	none	31.1%	–	–
74	North Mini Code	–	31.1%	–	–
75	MiniMax M2.1	–	28.8%	–	–
76	JT-35B-Flash	none	28.8%	–	–
77	GLM 4.6	none	28.8%	–	–
78	Ring-2.6-1T	xhigh	28.8%	–	–
79	Nemotron 3 Super 120B A12B (Reasoning)	–	28.8%	–	–
80	Haiku 4.5	–	27.3%	–	–
81	Step 3.5 Flash	–	27.3%	–	–
82	Qwen 3.5 35B-A3B	–	26.5%	–	–
83	Doubao Seed Code	–	26.5%	–	–
84	Gemini 2.5 Pro	–	26.5%	–	–
85	Mercury 2	high	26.5%	–	–
86	MiniMax M2	–	25.8%	–	–
87	Gemma 4 26B	none	25.0%	–	–
88	ERNIE 5.0 Thinking Preview	–	25.0%	–	–
89	Command A	–	25.0%	–	–
90	Qwen 3 Max	max	24.2%	–	–
91	Grok 4.1 Fast	–	24.2%	–	–
92	Gemini 3.1 Flash-Lite	high	24.2%	–	–
93	Nova 2.0 Pro Preview (Medium)	medium	24.2%	–	–
94	Qwen 3.5 9B	–	24.2%	–	–
95	HyperNova 60B 2605 (High, Based on gpt-oss-120b)	high	23.5%	–	–
96	gpt-oss-120b	high	23.5%	–	–
97	K-EXAONE (Reasoning)	–	22.7%	–	–
98	Trinity Large Thinking	–	22.7%	–	–
99	GLM 4.7 Flash	–	22.0%	–	–
100	GLM 4.5	–	22.0%	–	–
101	Qwen 3.5	none	21.2%	–	–
102	Sonnet 3.7	–	21.2%	–	–
103	Nemotron Cascade 2 30B A3B	–	21.2%	–	–
104	Ling 2.6 Flash	none	21.2%	–	–
105	EXAONE 4.5 33B (Reasoning)	–	20.5%	–	–
106	GLM 4.5 Air	–	20.5%	–	–
107	Grok 4 Fast	–	18.9%	–	–
108	Qwen 3 Coder	none	18.9%	–	–
109	Devstral 2	none	18.9%	–	–
110	Gemma 4 12B	–	18.2%	–	–
111	Qwen 3.5 4B	–	18.2%	–	–
112	JT-MINI	none	18.2%	–	–
113	Grok 3 Mini	high	17.4%	–	–
114	Grok Code Fast 1	–	17.4%	–	–
115	GPT-5 Nano	medium	17.4%	–	–
116	Nova 2.0 Pro Preview (Low)	low	17.4%	–	–
117	Nova 2.0 Lite (Medium)	medium	17.4%	–	–
118	Mistral Small 4	–	17.4%	–	–
119	Gemini 2.5 Flash	–	16.7%	–	–
120	Nova 2.0 Lite (High)	high	16.7%	–	–
121	Nova 2.0 Pro Preview (Non-reasoning)	none	16.7%	–	–
122	Devstral Small 2	none	16.7%	–	–
123	Cogito v2.1 (Reasoning)	–	16.7%	–	–
124	DeepSeek R1	–	15.9%	–	–
125	Mistral Large 3	none	15.9%	–	–
126	o4 mini	high	15.2%	–	–
127	Qwen 3 235B-A22B	none	15.2%	–	–
128	DeepSeek V3	none	15.2%	–	–
129	Apriel-v1.6-15B-Thinker	–	14.4%	–	–
130	GLM 4.6 V	–	14.4%	–	–
131	GPT-4.1	none	13.6%	–	–
132	NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)	–	13.6%	–	–
133	o1	–	12.9%	–	–
134	Magistral Medium 1.2	medium	12.9%	–	–
135	Gemini 2.5 Flash Lite	–	12.9%	–	–
136	HyperCLOVA X SEED Think (32B)	–	12.1%	–	–
137	NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning)	none	12.1%	–	–
138	Qwen 3	–	11.4%	–	–
139	Grok 3	none	11.4%	–	–
140	Llama 3.1 405b	–	11.4%	–	–
141	Kimi Linear 48B A3B Instruct	none	11.4%	–	–
142	Apriel-v1.5-15B-Thinker	–	10.6%	–	–
143	LongCat Flash Lite	none	10.6%	–	–
144	gpt-oss-20b	high	10.6%	–	–
145	Mistral Medium 3.1	none	10.6%	–	–
146	Ling-1T	none	10.6%	–	–
147	Ling-flash-2.0	none	10.6%	–	–
148	K2-V2 (High)	high	9.8%	–	–
149	KAT-Coder-Pro V1	none	9.1%	–	–
150	INTELLECT-3 (Based on GLM-4.5-Air)	–	9.1%	–	–
151	Magistral Medium 1	medium	9.1%	–	–
152	Devstral Medium	none	9.1%	–	–
153	Nemotron 3 Nano Omni 30B A3B Reasoning	–	8.3%	–	–
154	K2-V2 (Medium)	medium	8.3%	–	–
155	Gemma 4 E4B (Reasoning)	–	8.3%	–	–
156	GPT-4o	none	8.3%	–	–
157	GPT-4.1 Mini	none	7.6%	–	–
158	Solar Pro 3	high	7.6%	–	–
159	Gemma 4 E4B (Non-reasoning)	none	7.6%	–	–
160	Ring-flash-2.0	–	7.6%	–	–
161	Mistral Small 3.1	none	7.6%	–	–
162	o3 mini	–	6.8%	–	–
163	Seed-OSS-36B-Instruct	–	6.8%	–	–
164	K2 Think V2	high	6.8%	–	–
165	K-EXAONE (Non-reasoning)	none	6.8%	–	–
166	Ring-1T	–	6.8%	–	–
167	Qwen 3 30B-A3B	none	6.8%	–	–
168	Llama 4 Maverick	none	6.8%	–	–
169	Nova Premier	none	6.8%	–	–
170	Nova 2.0 Lite (Non-reasoning)	none	6.8%	–	–
171	Nova 2.0 Omni (Non-reasoning)	none	6.8%	–	–
172	GLM 4.5 V	none	6.8%	–	–
173	Mistral Small 3.2	none	6.8%	–	–
174	NVIDIA Nemotron 3 Nano 4B	–	6.8%	–	–
175	Devstral Small	none	6.1%	–	–
176	Mistral Large 2	none	6.1%	–	–
177	ERNIE 4.5 300B A47B	none	6.1%	–	–
178	Nova Pro	none	6.1%	–	–
179	Llama Nemotron Super 49B v1.5 (Reasoning)	–	5.3%	–	–
180	Step3 VL 10B	–	5.3%	–	–
181	Qwen 3 14B	none	5.3%	–	–
182	Nova 2.0 Omni (Medium)	medium	4.5%	–	–
183	Qwen 3 4B	none	4.5%	–	–
184	Magistral Small 1.2	–	4.5%	–	–
185	Magistral Small 1	–	4.5%	–	–
186	Llama 3.1 70b	–	4.5%	–	–
187	Qwen 2.5 72B	none	4.5%	–	–
188	Llama 3.1	none	4.5%	–	–
189	NVIDIA Nemotron Nano 12B v2 VL (Reasoning)	–	4.5%	–	–
190	K2-V2 (Low)	low	4.5%	–	–
191	LFM2.5-8B-A1B	–	4.5%	–	–
192	Solar Pro 2 (Non-reasoning)	none	4.5%	–	–
193	Ministral 3 14B	none	4.5%	–	–
194	Ministral 3 8B	none	4.5%	–	–
195	Nova 2.0 Lite (Low)	low	3.8%	–	–
196	Nova 2.0 Omni (Low)	low	3.8%	–	–
197	Motif-2-12.7B-Reasoning	–	3.8%	–	–
198	Mistral Medium 3	none	3.8%	–	–
199	Gemini 2.0 Flash	none	3.8%	–	–
200	EXAONE 4.0 32B (Reasoning)	–	3.8%	–	–
201	GPT-4.1 Nano	none	3.8%	–	–
202	Llama Nemotron Super 49B v1.5 (Non-reasoning)	none	3.8%	–	–
203	Qwen 3.5 2B	–	3.8%	–	–
204	Phi-4	none	3.8%	–	–
205	Gemma 3 27B	none	3.8%	–	–
206	MiniMax M1	–	3.0%	–	–
207	Gemma 4 E2B (Reasoning)	–	3.0%	–	–
208	Llama 3.3 70b	none	3.0%	–	–
209	Solar Pro 2 (Reasoning)	high	3.0%	–	–
210	Qwen 3 32B	–	3.0%	–	–
211	Mi:dm K 2.5 Pro Preview	–	3.0%	–	–
212	Mi:dm K 2.5 Pro	–	2.3%	–	–
213	Solar Open 100B (Reasoning)	high	2.3%	–	–
214	Tri-21B-think Preview	–	2.3%	–	–
215	Haiku 3.5	none	2.3%	–	–
216	Falcon-H1R-7B	–	2.3%	–	–
217	Granite 4.1 30B	none	2.3%	–	–
218	Sarvam 30B (High)	high	2.3%	–	–
219	Gemma 4 E2B (Non-reasoning)	none	2.3%	–	–
220	Jamba 1.7 Large	none	2.3%	–	–
221	Granite 4.0 H Small	none	2.3%	–	–
222	Qwen 3 8B	none	2.3%	–	–
223	Granite 4.1 3B	none	2.3%	–	–
224	Sarvam M (Reasoning, Based on Mistral Small 3.1)	high	2.3%	–	–
225	Gemma 3n E4B Instruct	none	2.3%	–	–
226	Sarvam 105B (High)	high	1.5%	–	–
227	NVIDIA Nemotron Nano 9B V2 (Reasoning)	–	1.5%	–	–
228	Olmo 3 32B Think	–	1.5%	–	–
229	Llama 4 Scout	none	1.5%	–	–
230	EXAONE 4.0 32B (Non-reasoning)	none	1.5%	–	–
231	Nova Micro	none	1.5%	–	–
232	Granite 4.0 Micro	none	1.5%	–	–
233	Tri-21B-Think	–	0.8%	–	–
234	Llama 3.1 8b	none	0.8%	–	–
235	NVIDIA Nemotron Nano 9B V2 (Non-reasoning)	none	0.8%	–	–
236	Nova Lite	none	0.8%	–	–
237	Jamba Reasoning 3B	–	0.8%	–	–
238	Olmo 3 7B Think	–	0.8%	–	–
239	Haiku 3	none	0.8%	–	–
240	Ling-mini-2.0	none	0.8%	–	–
241	Llama 3 70b	none	0.8%	–	–
242	Llama 3.2 11b	none	0.8%	–	–
243	LFM2 2.6B	none	0.8%	–	–
244	Gemma 3 4B	none	0.8%	–	–
245	Gemma 3n E2B Instruct	none	0.8%	–	–
246	Gemma 3 12B	none	0.8%	–	–
247	Llama 3.3	–	0.0%	–	–
248	MiniCPM5-1B (Reasoning)	–	0.0%	–	–
249	MiniCPM5-1B (Non-reasoning)	none	0.0%	–	–
250	Nanbeige4.1-3B	–	0.0%	–	–
251	Olmo 3.1 32B Think	–	0.0%	–	–
252	Granite 4.1 8B	none	0.0%	–	–
253	Olmo 3.1 32B Instruct	none	0.0%	–	–
254	Phi-4 Mini Instruct	none	0.0%	–	–
255	Qwen 3.5 0.8B	–	0.0%	–	–
256	OLMo 2 32B	none	0.0%	–	–
257	LFM2 24B A2B	none	0.0%	–	–
258	Phi-3 Mini Instruct 3.8B	none	0.0%	–	–
259	NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning)	none	0.0%	–	–
260	MiniCPM-V 4.6 1.3B	none	0.0%	–	–
261	Reka Flash 3	–	0.0%	–	–
262	OLMo 2 7B	none	0.0%	–	–
263	Molmo 7B-D	none	0.0%	–	–
264	Exaone 4.0 1.2B (Reasoning)	–	0.0%	–	–
265	Olmo 3 7B Instruct	none	0.0%	–	–
266	Exaone 4.0 1.2B (Non-reasoning)	none	0.0%	–	–
267	LFM2.5-1.2B-Thinking	–	0.0%	–	–
268	Jamba 1.7 Mini	none	0.0%	–	–
269	LFM2.5-1.2B-Instruct	none	0.0%	–	–
270	Granite 4.0 H 1B	none	0.0%	–	–
271	Qwen 3 1.7B	–	0.0%	–	–
272	Gemma 3 270M	none	0.0%	–	–
273	Apertus 70B Instruct	none	0.0%	–	–
274	Granite 4.0 1B	none	0.0%	–	–
275	Molmo2-8B	none	0.0%	–	–
276	LFM2 8B A1B	none	0.0%	–	–
277	Granite 3.3 8B (Non-reasoning)	none	0.0%	–	–
278	Ministral 3 3B	none	0.0%	–	–
279	Qwen 3 0.6B	none	0.0%	–	–
280	LFM2 1.2B	none	0.0%	–	–
281	LFM2.5-VL-1.6B	none	0.0%	–	–
282	Granite 4.0 350M	none	0.0%	–	–
283	Llama 3.2 1b	none	0.0%	–	–
284	Tiny Aya Global	none	0.0%	–	–
285	Llama 3 8b	none	0.0%	–	–
286	Apertus 8B Instruct	none	0.0%	–	–
287	Granite 4.0 H 350M	none	0.0%	–	–
288	Gemma 3 1B	none	0.0%	–	–