τ²-Bench Telecom

τ²-Bench telecom domain: dual-control tool-using agent tasks; pass rate.

293models

–Open Index weight

–last updated

Artificial Analysispublisher

#	Model	Effort	Score	Cost / task	Harness
1	GLM 5.2	max	99.1%	–	–
2	JT-35B-Flash	none	99.1%	–	–
3	GLM 4.7 Flash	–	98.8%	–	–
4	Fable 5	max	98.5%	–	–
5	GLM 5	–	98.5%	–	–
6	GLM 5 V	–	98.5%	–	–
7	Step 3.7 Flash	high	98.5%	–	–
8	Qwen 3.6 Plus	–	97.7%	–	–
9	GLM 5.1	–	97.7%	–	–
10	Grok 4.3	high	97.7%	–	–
11	Grok 4.20	–	96.5%	–	–
12	DeepSeek V4 Pro 0424	max	96.2%	–	–
13	Qwen 3.6 Max	max	95.9%	–	–
14	Kimi K2.6	–	95.9%	–	–
15	Kimi K2.5	–	95.9%	–	–
16	GLM 4.7	–	95.9%	–	–
17	Gemini 3.5 Flash	medium	95.6%	–	–
18	Gemini 3.1 Pro	–	95.6%	–	–
19	DeepSeek V4 Flash 0420	high	95.6%	–	–
20	Qwen 3.5 397B-A17B	–	95.6%	–	–
21	MiniMax M2.5	–	95.3%	–	–
22	Qwen 3.6 35B-A3B	–	95.3%	–	–
23	MiMo V2 Pro	–	95.0%	–	–
24	MiMo V2 Flash	–	95.0%	–	–
25	Qwen 3.7 Max	max	94.7%	–	–
26	Opus 4.8	max	94.4%	–	–
27	Step 3.5 Flash	–	94.4%	–	–
28	MiMo V2.5 Pro	–	94.2%	–	–
29	Qwen 3.6 27B	–	94.2%	–	–
30	Mistral Medium 3.5	high	94.2%	–	–
31	GPT-5.5	xhigh	93.9%	–	–
32	Qwen 3.5 27B	–	93.9%	–	–
33	Qwen 3.5 122B-A10B	–	93.6%	–	–
34	Grok 4.1 Fast	–	93.3%	–	–
35	Tri-21B-think Preview	–	93.3%	–	–
36	Qwen 3.7 Plus	–	93.0%	–	–
37	Kimi K2	–	93.0%	–	–
38	JT-MINI	none	93.0%	–	–
39	Hy3	–	92.7%	–	–
40	Nova 2.0 Pro Preview (Medium)	medium	92.7%	–	–
41	Ring-2.6-1T	xhigh	92.4%	–	–
42	Opus 4.6	max	92.1%	–	–
43	GPT-5.2 Codex	xhigh	92.1%	–	–
44	Qwen 3.5 4B	–	92.1%	–	–
45	Muse Spark	–	91.5%	–	–
46	MiMo V2	–	91.2%	–	–
47	MiMo V2.5	–	90.6%	–	–
48	DeepSeek V3.2	–	90.6%	–	–
49	Nova 2.0 Pro Preview (Low)	low	90.6%	–	–
50	Grok 3 Mini	high	90.4%	–	–
51	Kimi K2.7 Code	–	90.1%	–	–
52	Trinity Large Thinking	–	90.1%	–	–
53	Ling-2.6-1T	none	89.8%	–	–
54	Opus 4.5	–	89.5%	–	–
55	KAT Coder Pro V2	none	89.5%	–	–
56	Qwen 3.5 35B-A3B	–	89.2%	–	–
57	MiniMax M3	–	88.9%	–	–
58	Opus 4.7	max	88.6%	–	–
59	KAT-Coder-Pro V1	none	88.6%	–	–
60	Qwen 3.5	none	88.3%	–	–
61	MiniCPM-V 4.6 1.3B	none	87.7%	–	–
62	Step 3.5 Flash 2603	–	87.4%	–	–
63	HyperCLOVA X SEED Think (32B)	–	87.4%	–	–
64	GPT-5.4	xhigh	87.1%	–	–
65	Gemini 3 Pro	high	87.1%	–	–
66	GPT-5 Codex	high	86.8%	–	–
67	MiniMax M2	–	86.8%	–	–
68	Qwen 3.5 9B	–	86.8%	–	–
69	GPT-5	medium	86.5%	–	–
70	Mi:dm K 2.5 Pro	–	86.5%	–	–
71	GPT-5.6 Terra	max	86.3%	–	–
72	Solar Pro 3	high	86.3%	–	–
73	GPT-5.3 Codex	xhigh	86.0%	–	–
74	Ling 2.6 Flash	none	86.0%	–	–
75	MiniMax M2.1	–	85.4%	–	–
76	GPT-5.6 Sol	max	85.1%	–	–
77	GPT-5.2	xhigh	84.8%	–	–
78	MiniMax M2.7	–	84.8%	–	–
79	ERNIE 5.0 Thinking Preview	–	83.9%	–	–
80	Qwen 3 Max	max	83.6%	–	–
81	GPT-5.4 Mini	xhigh	83.3%	–	–
82	Nemotron 3 Ultra	max	83.3%	–	–
83	GPT-5.1 Codex	high	83.0%	–	–
84	MiniCPM5-1B (Non-reasoning)	none	82.5%	–	–
85	GPT-5.1	high	81.9%	–	–
86	Nex-N2-Pro	–	81.6%	–	–
87	Qwen 3.5 2B	none	81.6%	–	–
88	Tri-21B-Think	–	81.0%	–	–
89	MiniCPM5-1B (Reasoning)	–	81.0%	–	–
90	o3	–	80.7%	–	–
91	Command A	–	80.7%	–	–
92	Gemini 3 Flash	–	80.4%	–	–
93	Nova 2.0 Omni (Medium)	medium	80.4%	–	–
94	Sonnet 4.6	none	79.5%	–	–
95	Qwen 3 Coder	none	79.5%	–	–
96	LongCat Flash Lite	none	79.5%	–	–
97	Sonnet 4.5	–	78.1%	–	–
98	EXAONE 4.5 33B (Reasoning)	–	78.1%	–	–
99	GLM 4.6	none	76.9%	–	–
100	GPT-5.4 Nano	xhigh	76.0%	–	–
101	Grok Code Fast 1	–	75.7%	–	–
102	Nova 2.0 Lite (Medium)	medium	75.7%	–	–
103	Grok 4	–	74.9%	–	–
104	K-EXAONE (Reasoning)	–	74.3%	–	–
105	Opus 4	–	73.4%	–	–
106	Nova 2.0 Lite (High)	high	72.8%	–	–
107	Nova 2.0 Lite (Low)	low	71.9%	–	–
108	Nova 2.0 Pro Preview (Non-reasoning)	none	71.6%	–	–
109	Opus 4.1	–	71.4%	–	–
110	GPT-5 Mini	medium	71.1%	–	–
111	Mercury 2	high	70.8%	–	–
112	Apriel-v1.6-15B-Thinker	–	69.3%	–	–
113	Apriel-v1.5-15B-Thinker	–	68.4%	–	–
114	Nemotron 3 Super 120B A12B (Reasoning)	–	67.8%	–	–
115	Nova 2.0 Omni (Low)	low	67.8%	–	–
116	Grok 4 Fast	–	65.8%	–	–
117	gpt-oss-120b	high	65.8%	–	–
118	Gemma 4 31B	none	65.5%	–	–
119	Qwen 3.5 0.8B	none	65.2%	–	–
120	Sonnet 4	–	64.6%	–	–
121	HyperNova 60B 2605 (High, Based on gpt-oss-120b)	high	63.2%	–	–
122	GPT-5.1 Codex Mini	high	62.9%	–	–
123	o1	–	62.6%	–	–
124	Nova 2.0 Lite (Non-reasoning)	none	62.0%	–	–
125	gpt-oss-20b	high	60.2%	–	–
126	K-EXAONE (Non-reasoning)	none	59.1%	–	–
127	Doubao Seed Code	–	58.2%	–	–
128	o4 mini	high	55.6%	–	–
129	Sonnet 3.7	–	54.7%	–	–
130	Haiku 4.5	–	54.7%	–	–
131	Gemini 2.5 Pro	–	54.1%	–	–
132	Qwen 3	–	54.1%	–	–
133	Qwen 3 235B-A22B	–	53.2%	–	–
134	Nemotron Cascade 2 30B A3B	–	53.2%	–	–
135	GPT-4.1 Mini	none	52.9%	–	–
136	Magistral Medium 1.2	medium	52.0%	–	–
137	GPT-5.5 Instant	–	49.4%	–	–
138	Seed-OSS-36B-Instruct	–	49.4%	–	–
139	Mi:dm K 2.5 Pro Preview	–	49.4%	–	–
140	Grok 3	none	48.8%	–	–
141	Solar Open 100B (Reasoning)	high	48.2%	–	–
142	GPT-4.1	none	47.1%	–	–
143	DeepSeek V3	none	47.1%	–	–
144	Sarvam 105B (High)	high	46.8%	–	–
145	GLM 4.5 Air	–	46.5%	–	–
146	Motif-2-12.7B-Reasoning	–	46.5%	–	–
147	Gemini 2.5 Flash	–	45.6%	–	–
148	Nemotron 3 Nano Omni 30B A3B Reasoning	–	45.3%	–	–
149	Nova 2.0 Omni (Non-reasoning)	none	44.7%	–	–
150	Gemma 4 26B	–	43.6%	–	–
151	GLM 4.5	–	43.0%	–	–
152	Granite 4.1 30B	none	42.1%	–	–
153	Mistral Small 4	–	41.2%	–	–
154	NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)	–	40.9%	–	–
155	Mistral Medium 3.1	none	40.6%	–	–
156	Nova Premier	none	38.3%	–	–
157	Devstral Small	none	38.0%	–	–
158	DeepSeek V3.1	–	37.4%	–	–
159	North Mini Code	–	37.4%	–	–
160	DeepSeek R1	–	36.5%	–	–
161	GPT-5 Nano	high	36.5%	–	–
162	Pixtral Large	none	36.5%	–	–
163	Gemma 4 12B	–	36.3%	–	–
164	Qwen 2.5 72B	none	34.5%	–	–
165	Qwen 3 14B	–	34.5%	–	–
166	Sarvam 30B (High)	high	34.5%	–	–
167	MiniMax M1	–	34.2%	–	–
168	Mistral Large 2	none	33.0%	–	–
169	Ling-1T	none	32.7%	–	–
170	Solar Pro 2 (Non-reasoning)	none	31.9%	–	–
171	GLM 4.6 V	–	31.6%	–	–
172	Gemini 3.1 Flash-Lite	high	31.3%	–	–
173	o3 mini	high	31.3%	–	–
174	Gemini 2.5 Flash Lite	–	30.7%	–	–
175	Qwen 3 32B	–	29.8%	–	–
176	Gemini 2.0 Flash	none	29.5%	–	–
177	Mistral Small 3.2	none	29.5%	–	–
178	GPT-4o	none	28.9%	–	–
179	Qwen 3 30B-A3B	–	28.1%	–	–
180	Llama Nemotron Super 49B v1.5 (Reasoning)	–	28.1%	–	–
181	Solar Pro 2 (Reasoning)	high	28.1%	–	–
182	NVIDIA Nemotron 3 Nano 4B	–	28.1%	–	–
183	K2-V2 (High)	high	27.8%	–	–
184	Magistral Small 1.2	–	27.8%	–	–
185	Falcon-H1R-7B	–	27.8%	–	–
186	Granite 4.1 8B	none	27.8%	–	–
187	Qwen 3 8B	–	27.8%	–	–
188	Ministral 3 14B	none	27.2%	–	–
189	Llama 3.3	–	26.9%	–	–
190	INTELLECT-3 (Based on GLM-4.5-Air)	–	26.6%	–	–
191	Qwen 3 4B	none	26.6%	–	–
192	Magistral Small 1	–	26.6%	–	–
193	Llama 3.3 70b	none	26.6%	–	–
194	Llama 3.1 405b	none	26.6%	–	–
195	Ministral 3 8B	none	26.6%	–	–
196	Ring-1T	–	26.3%	–	–
197	Gemma 4 E4B (Non-reasoning)	none	26.0%	–	–
198	Qwen 3 1.7B	–	26.0%	–	–
199	K2 Think V2	high	25.4%	–	–
200	NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning)	none	25.4%	–	–
201	Llama Nemotron Super 49B v1.5 (Non-reasoning)	none	25.1%	–	–
202	Mistral Small 3.1	none	25.1%	–	–
203	K2-V2 (Medium)	medium	24.9%	–	–
204	Devstral 2	none	24.9%	–	–
205	Ministral 3 3B	none	24.9%	–	–
206	Mistral Large 3	none	24.6%	–	–
207	Haiku 3.5	none	24.6%	–	–
208	Mistral Medium 3	none	24.3%	–	–
209	Devstral Small 2	none	23.4%	–	–
210	NVIDIA Nemotron Nano 9B V2 (Non-reasoning)	none	23.4%	–	–
211	Magistral Medium 1	medium	23.1%	–	–
212	Llama 3.1	none	23.1%	–	–
213	Granite 4.0 1B	none	22.8%	–	–
214	Llama 3.1 70b	–	22.5%	–	–
215	GLM 4.5 V	–	22.5%	–	–
216	Gemma 4 E2B (Non-reasoning)	none	22.2%	–	–
217	NVIDIA Nemotron Nano 9B V2 (Reasoning)	–	21.9%	–	–
218	Nanbeige4.1-3B	–	21.6%	–	–
219	NVIDIA Nemotron Nano 12B v2 VL (Reasoning)	–	21.3%	–	–
220	Olmo 3.1 32B Instruct	none	21.3%	–	–
221	Llama 3.2 3b	none	21.1%	–	–
222	Haiku 3	none	21.1%	–	–
223	Qwen 3 0.6B	–	21.1%	–	–
224	Gemma 4 E4B (Reasoning)	–	20.8%	–	–
225	Ling-flash-2.0	none	20.8%	–	–
226	Gemma 4 E2B (Reasoning)	–	20.8%	–	–
227	K2-V2 (Low)	low	20.8%	–	–
228	Exaone 4.0 1.2B (Non-reasoning)	none	20.5%	–	–
229	Solar Mini	none	20.2%	–	–
230	Devstral Medium	none	19.9%	–	–
231	Mistral Small 3	none	19.6%	–	–
232	Granite 4.1 3B	none	19.6%	–	–
233	LFM2.5-1.2B-Thinking	–	19.6%	–	–
234	Granite 4.0 H 1B	none	19.6%	–	–
235	NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning)	none	19.3%	–	–
236	Llama 4 Maverick	none	17.8%	–	–
237	Nova Lite	none	17.5%	–	–
238	EXAONE 4.0 32B (Reasoning)	–	17.3%	–	–
239	GPT-4.1 Nano	none	17.3%	–	–
240	Granite 4.0 H Small	none	17.3%	–	–
241	Llama 3.1 8b	none	16.4%	–	–
242	Exaone 4.0 1.2B (Reasoning)	–	16.4%	–	–
243	Step3 VL 10B	–	16.1%	–	–
244	LFM2.5-8B-A1B	–	16.1%	–	–
245	Jamba Reasoning 3B	–	15.8%	–	–
246	Llama 4 Scout	none	15.5%	–	–
247	Llama 3.2 11b	none	14.6%	–	–
248	Granite 4.0 H 350M	none	14.6%	–	–
249	Nova Pro	none	14.0%	–	–
250	Nova Micro	none	14.0%	–	–
251	Jamba 1.7 Large	none	13.5%	–	–
252	LFM2 2.6B	none	13.5%	–	–
253	Ling-mini-2.0	none	13.2%	–	–
254	Granite 4.0 350M	none	13.2%	–	–
255	Apertus 70B Instruct	none	12.9%	–	–
256	Olmo 3 7B Instruct	none	12.6%	–	–
257	Jamba 1.7 Mini	none	12.6%	–	–
258	Granite 4.0 Micro	none	12.6%	–	–
259	LFM2 1.2B	none	12.6%	–	–
260	Apertus 8B Instruct	none	11.4%	–	–
261	LFM2 24B A2B	none	11.1%	–	–
262	LFM2.5-1.2B-Instruct	none	10.8%	–	–
263	Gemma 3 12B	none	10.8%	–	–
264	LFM2 8B A1B	none	10.5%	–	–
265	Granite 3.3 8B (Non-reasoning)	none	10.5%	–	–
266	Gemma 3 27B	none	10.5%	–	–
267	Gemma 3 1B	none	10.5%	–	–
268	Gemma 3 270M	none	9.1%	–	–
269	LFM2.5-VL-1.6B	none	8.5%	–	–
270	Phi-4 Mini Instruct	none	8.2%	–	–
271	Gemma 3 4B	none	5.0%	–	–
272	Gemma 3n E4B Instruct	none	5.0%	–	–
273	EXAONE 4.0 32B (Non-reasoning)	none	4.1%	–	–
274	ERNIE 4.5 300B A47B	none	0.0%	–	–
275	Kimi Linear 48B A3B Instruct	none	0.0%	–	–
276	Ring-flash-2.0	–	0.0%	–	–
277	Olmo 3.1 32B Think	–	0.0%	–	–
278	Olmo 3 32B Think	–	0.0%	–	–
279	OLMo 2 32B	none	0.0%	–	–
280	Phi-4	none	0.0%	–	–
281	Phi-3 Mini Instruct 3.8B	none	0.0%	–	–
282	Reka Flash 3	–	0.0%	–	–
283	Olmo 3 7B Think	–	0.0%	–	–
284	OLMo 2 7B	none	0.0%	–	–
285	Molmo 7B-D	none	0.0%	–	–
286	Llama 3 70b	none	0.0%	–	–
287	Sarvam M (Reasoning, Based on Mistral Small 3.1)	high	0.0%	–	–
288	Mistral	none	0.0%	–	–
289	Molmo2-8B	none	0.0%	–	–
290	Llama 3.2 1b	none	0.0%	–	–
291	Tiny Aya Global	none	0.0%	–	–
292	Llama 3 8b	none	0.0%	–	–
293	Gemma 3n E2B Instruct	none	0.0%	–	–