τ³-Banking

τ³-Bench banking domain: multi-turn tool-using agent tasks with a simulated user; pass rate.

153models

–Open Index weight

–last updated

Artificial Analysispublisher

#	Model	Effort	Score	Cost / task	Harness
1	Qwen 3.8 Max	max	51.3%	–	–
2	Grok 4.6	high	50.7%	–	–
3	Muse Spark 1.3	max	50.5%	–	–
4	GLM 5.3	max	50.3%	–	–
5	Qwen 3.8 2.4T-A95B	–	49.1%	–	–
6	Qwen 3.8 27B	xhigh	48.0%	–	–
7	Fable 5.1	max	47.2%	–	–
8	GLM 5.3 Flash	max	47.2%	–	–
9	Kimi K3	max	46.0%	–	–
10	Gemini 3.8 Flash	medium	45.8%	–	–
11	Qwen 3.8 Flash	–	45.4%	–	–
12	Opus 5	high	44.7%	–	–
13	GPT-5.6 Sol	max	44.3%	–	–
14	GPT-6 Astra	xhigh	43.1%	–	–
15	Grok 4.5	high	42.1%	–	–
16	DeepSeek V4 Flash	max	41.0%	–	–
17	GPT-5.6 Terra	max	40.2%	–	–
18	GPT-5.4	xhigh	39.6%	–	–
19	DeepSeek V4 Pro 0813	max	39.6%	–	–
20	DeepSeek V4 Flash 0731	max	39.4%	–	–
21	GPT-5.5	xhigh	39.0%	–	–
22	Ling 3.0 Flash Fin	–	38.6%	–	–
23	Fable 5	max	38.1%	–	–
24	Sonnet 5	max	37.3%	–	–
25	Agnes 2.5 Pro Beta	–	35.7%	–	–
26	Gemini 3.7 Flash	medium	35.5%	–	–
27	Motif 3	–	35.3%	–	–
28	Muse Spark 1.2	xhigh	34.8%	–	–
29	Opus 4.7	max	34.6%	–	–
30	GLM 5.2	max	34.6%	–	–
31	Sonnet 4.6	max	34.4%	–	–
32	Ling 3.0 Flash	–	34.4%	–	–
33	Opus 4.8	max	34.2%	–	–
34	K2 Horizon 375B A23B	–	34.2%	–	–
35	Gemini 3.5 Flash	high	32.2%	–	–
36	K2 Horizon MoVA 36B A4B	–	32.0%	–	–
37	Muse Spark 1.1	xhigh	31.8%	–	–
38	GPT-5.6 Luna	max	31.1%	–	–
39	DeepSeek V4 Flash 0420	max	30.9%	–	–
40	DeepSeek V4 Pro 0424	max	30.1%	–	–
41	Gemini 3.6 Flash	high	29.9%	–	–
42	Inkling	xhigh	29.1%	–	–
43	Quasar 438B (Max, Based on GLM-5.2)	max	28.7%	–	–
44	K2 Horizon 7B	–	27.6%	–	–
45	GPT-5.4 Nano	xhigh	27.4%	–	–
46	Ling 3.0 Flash	–	27.2%	–	–
47	GPT-5.4 Mini	xhigh	25.6%	–	–
48	Apodex 1.1	–	25.2%	–	–
49	Sonnet 4.5	–	24.5%	–	–
50	Muse Glimmer (High)	high	23.5%	–	–
51	Kimi K2.6	–	23.3%	–	–
52	Solar Pro 4	–	23.3%	–	–
53	Hy3	–	22.9%	–	–
54	GPT-5	high	22.1%	–	–
55	G9v3-39A5B	–	22.1%	–	–
56	Solar Open2 250B	–	21.6%	–	–
57	Gemini 3.1 Pro	–	21.4%	–	–
58	DeepSeek V3.1	–	21.0%	–	–
59	Qwen 3.6 Plus	–	20.8%	–	–
60	Gemini 3 Flash	–	20.8%	–	–
61	MiniCPM5-2B	–	20.8%	–	–
62	Ling 3.0 Tiny	–	20.8%	–	–
63	Kimi K2.7 Code	–	20.2%	–	–
64	K2 Horizon 3.7B	–	19.6%	–	–
65	Inkling Small	–	18.8%	–	–
66	Ring-2.6-1T	xhigh	17.9%	–	–
67	Nex-N2-Pro	–	17.5%	–	–
68	Qwen 3.7 Plus	–	17.5%	–	–
69	Gemini 3.5 Flash-Lite	high	17.5%	–	–
70	Qwen 3.6 27B	–	16.7%	–	–
71	Sonnet 4	–	16.7%	–	–
72	A.X-K2	–	16.1%	–	–
73	GPT-5.1	high	15.9%	–	–
74	GPT-5 Mini	high	15.5%	–	–
75	MiniMax M3	–	15.3%	–	–
76	Qwen 3.5 122B-A10B	–	15.3%	–	–
77	Mistral Medium 3.5	high	15.1%	–	–
78	Gemma 4 31B	–	14.8%	–	–
79	Granite 4.2 30B	–	14.4%	–	–
80	Kimi K2.5	–	14.2%	–	–
81	Nemotron 3 Ultra	max	14.2%	–	–
82	GLM 5.1	–	13.6%	–	–
83	Grok Build 0.1	–	13.4%	–	–
84	GLM 4.6	–	13.4%	–	–
85	Qwen 3.5 397B-A17B	–	13.4%	–	–
86	LongCat 2.0	–	13.2%	–	–
87	gpt-oss-120b	high	12.8%	–	–
88	Grok 4.3	high	12.4%	–	–
89	GLM 4.7	–	12.2%	–	–
90	Step 3.7 Flash	high	12.0%	–	–
91	Gemma 4 26B	–	12.0%	–	–
92	Qwen 3.7 Max	max	11.8%	–	–
93	GPT-5.5 Instant	–	11.8%	–	–
94	K-EXAONE 2.0 0803	–	11.5%	–	–
95	GPT-5.2	none	11.1%	–	–
96	Devstral Small 2	none	10.7%	–	–
97	Devstral 2	none	10.5%	–	–
98	Nemotron 3 Super 120B A12B (Reasoning)	–	10.3%	–	–
99	MiMo V2.5 Pro	–	9.9%	–	–
100	MiniMax M2.7	–	9.9%	–	–
101	Gemini 2.5 Pro	–	9.7%	–	–
102	Gemini 3.1 Flash-Lite	high	9.7%	–	–
103	Mercury 2	high	9.5%	–	–
104	Qwen 3.6 35B-A3B	–	9.3%	–	–
105	Haiku 4.5	–	9.3%	–	–
106	Nemotron 3.5 Lightning	–	8.9%	–	–
107	MiMo V2.5	–	8.7%	–	–
108	Solar Pro 3	high	8.7%	–	–
109	Mistral Medium 3.1	none	8.2%	–	–
110	Qwen 3 235B-A22B	–	7.8%	–	–
111	Granite 4.2 8B	–	7.6%	–	–
112	Mistral Small 3.1	none	7.4%	–	–
113	LFM2.5-2.6B	–	7.2%	–	–
114	Qwen 3.5 9B	–	7.0%	–	–
115	gpt-oss-20b	high	7.0%	–	–
116	Qwen 3.5 4B	–	6.8%	–	–
117	Ministral 3 14B	none	6.6%	–	–
118	DeepSeek R1	–	6.4%	–	–
119	North Mini Code	–	6.4%	–	–
120	Qwen 3	–	6.2%	–	–
121	Mistral Small 3.2	none	6.2%	–	–
122	Command A	–	6.0%	–	–
123	NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)	–	6.0%	–	–
124	Trinity Large Thinking	–	5.8%	–	–
125	Mistral Large 3	none	5.8%	–	–
126	Magistral Medium 1.2	medium	5.6%	–	–
127	HyperNova 60B 2605 (High, Based on gpt-oss-120b)	high	5.6%	–	–
128	Granite 4.2 3B	–	5.6%	–	–
129	Qwen 3 14B	–	5.6%	–	–
130	GPT-4.1 Mini	none	5.4%	–	–
131	Qwen 3 30B-A3B	–	5.4%	–	–
132	Qwen 3 Coder	none	5.4%	–	–
133	Qwen 3 32B	–	5.4%	–	–
134	o3 mini	high	5.2%	–	–
135	Qwen 3.5 35B-A3B	none	4.9%	–	–
136	Mistral Small 4	–	4.9%	–	–
137	K2 Horizon 0.9B	–	4.9%	–	–
138	DeepSeek V3	none	4.7%	–	–
139	Qwen 3 8B	–	4.7%	–	–
140	Ministral 3 3B	none	4.7%	–	–
141	KAT Coder Pro V2	none	4.5%	–	–
142	Magistral Small 1.2	–	4.5%	–	–
143	Celeris-1	none	3.9%	–	–
144	Llama 4 Maverick	none	3.7%	–	–
145	Ministral 3 8B	none	3.7%	–	–
146	GPT-4.1 Nano	none	3.5%	–	–
147	Llama 4 Scout	none	3.3%	–	–
148	Ling 2.6 Flash	none	2.9%	–	–
149	GPT-4o mini	none	2.9%	–	–
150	Gemma 3 27B	none	0.8%	–	–
151	Gemma 3 12B	none	0.8%	–	–
152	Gemma 3 4B	none	0.4%	–	–
153	Gemma 3n E4B Instruct	none	0.2%	–	–