Benchmarks / Instruction Following

IFBench

IFBench: precise instruction-following with verifiable constraints; accuracy.

297models

–Open Index weight

–last updated

Artificial Analysispublisher

#	Model	Effort	Score	Cost / task	Harness
1	Grok 4.3	medium	83.3%	–	–
2	Grok 4.20	–	82.9%	–	–
3	MiniMax M3	–	82.9%	–	–
4	Nemotron 3 Ultra	max	81.4%	–	–
5	Qwen 3.7 Max	max	80.5%	–	–
6	Nemotron Cascade 2 30B A3B	–	80.4%	–	–
7	MiMo V2.5 Pro	–	79.9%	–	–
8	Nova 2.0 Pro Preview (Low)	low	79.6%	–	–
9	DeepSeek V4 Flash 0420	max	79.2%	–	–
10	Nova 2.0 Pro Preview (Medium)	medium	79.0%	–	–
11	Qwen 3.5 397B-A17B	–	78.8%	–	–
12	Gemini 3 Flash	–	78.0%	–	–
13	Qwen 3.7 Plus	–	78.0%	–	–
14	GPT-5.2 Codex	xhigh	77.6%	–	–
15	Gemini 3.1 Flash-Lite	high	77.2%	–	–
16	Gemini 3.1 Pro	–	77.1%	–	–
17	Qwen 3.6 Max	max	76.6%	–	–
18	DeepSeek V4 Pro 0424	max	76.5%	–	–
19	Gemini 3.5 Flash	high	76.3%	–	–
20	GLM 5.1	–	76.3%	–	–
21	Kimi K2.6	–	76.0%	–	–
22	Muse Spark	–	75.9%	–	–
23	GPT-5.4 Nano	xhigh	75.9%	–	–
24	GPT-5.5	xhigh	75.9%	–	–
25	MiniMax M2.7	–	75.7%	–	–
26	Qwen 3.5 122B-A10B	–	75.7%	–	–
27	Qwen 3.5 27B	–	75.6%	–	–
28	Gemma 4 31B	–	75.6%	–	–
29	GPT-5.2	xhigh	75.4%	–	–
30	GPT-5 Mini	high	75.4%	–	–
31	GPT-5.3 Codex	xhigh	75.4%	–	–
32	Qwen 3.6 Plus	–	75.2%	–	–
33	GPT-5 Codex	high	74.1%	–	–
34	GPT-5.4	xhigh	73.9%	–	–
35	Command A	–	73.9%	–	–
36	Gemma 4 12B	–	73.5%	–	–
37	GLM 5.2	max	73.3%	–	–
38	GPT-5.4 Mini	xhigh	73.3%	–	–
39	GLM 5	–	73.2%	–	–
40	GPT-5	high	73.1%	–	–
41	GPT-5.1	high	72.9%	–	–
42	GPT-5.6 Sol	max	72.7%	–	–
43	Qwen 3.5 35B-A3B	–	72.5%	–	–
44	Gemma 4 26B	–	72.4%	–	–
45	MiniMax M2	–	72.3%	–	–
46	MiMo V2 Flash	–	71.8%	–	–
47	MiniMax M2.5	–	71.6%	–	–
48	GPT-5.5 Instant	–	71.5%	–	–
49	Nemotron 3 Super 120B A12B (Reasoning)	–	71.5%	–	–
50	o3	–	71.4%	–	–
51	GPT-5.6 Terra	max	71.2%	–	–
52	Solar Pro 3	high	71.2%	–	–
53	NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)	–	71.1%	–	–
54	Qwen 3 Max	max	70.7%	–	–
55	Nova 2.0 Lite (High)	high	70.7%	–	–
56	Gemini 3 Pro	high	70.4%	–	–
57	o1	–	70.3%	–	–
58	Kimi K2.5	–	70.2%	–	–
59	GPT-5.1 Codex	high	70.0%	–	–
60	MiniMax M2.1	–	69.9%	–	–
61	Mercury 2	high	69.8%	–	–
62	Apriel-v1.6-15B-Thinker	–	69.1%	–	–
63	gpt-oss-120b	high	69.0%	–	–
64	MiMo V2 Pro	–	68.8%	–	–
65	Mistral Medium 3.5	high	68.8%	–	–
66	o4 mini	high	68.7%	–	–
67	Nova 2.0 Lite (Medium)	medium	68.5%	–	–
68	KAT-Coder-Pro V1	none	68.4%	–	–
69	Kimi K2	–	68.1%	–	–
70	GLM 4.7	–	67.9%	–	–
71	GPT-5.1 Codex Mini	high	67.9%	–	–
72	Qwen 3.6 27B	–	67.6%	–	–
73	GPT-5 Nano	high	67.6%	–	–
74	MiMo V2	–	67.3%	–	–
75	Step 3.7 Flash	high	67.3%	–	–
76	MiMo V2.5	–	67.1%	–	–
77	o3 mini	high	67.1%	–	–
78	Qwen 3.5 9B	–	66.7%	–	–
79	KAT Coder Pro V2	none	66.7%	–	–
80	Step 3.5 Flash 2603	–	66.5%	–	–
81	HyperNova 60B 2605 (High, Based on gpt-oss-120b)	high	66.5%	–	–
82	Nex-N2-Pro	–	66.2%	–	–
83	Nova 2.0 Omni (Medium)	medium	66.2%	–	–
84	Olmo 3.1 32B Think	–	66.0%	–	–
85	gpt-oss-20b	high	65.1%	–	–
86	K-EXAONE (Reasoning)	–	64.7%	–	–
87	Step 3.5 Flash	–	64.6%	–	–
88	Qwen 3.6 35B-A3B	–	64.4%	–	–
89	DeepSeek V3.2	–	63.9%	–	–
90	Fable 5	max	63.5%	–	–
91	Nemotron 3 Nano Omni 30B A3B Reasoning	–	63.2%	–	–
92	Kimi K2.7 Code	–	63.1%	–	–
93	Hy3	–	63.1%	–	–
94	K2 Think V2	high	62.8%	–	–
95	Opus 4.8	max	62.2%	–	–
96	Nova 2.0 Omni (Low)	low	61.8%	–	–
97	Apriel-v1.5-15B-Thinker	–	61.7%	–	–
98	Nova 2.0 Lite (Low)	low	61.2%	–	–
99	GLM 5 V	–	61.1%	–	–
100	GLM 4.7 Flash	–	60.8%	–	–
101	Qwen 3	–	60.7%	–	–
102	K2-V2 (High)	high	60.1%	–	–
103	DiffusionGemma 26B A4B	–	59.5%	–	–
104	Opus 4.7	max	58.6%	–	–
105	NVIDIA Nemotron 3 Nano 4B	–	58.2%	–	–
106	Opus 4.5	–	58.0%	–	–
107	EXAONE 4.5 33B (Reasoning)	–	58.0%	–	–
108	Solar Open 100B (Reasoning)	high	57.7%	–	–
109	North Mini Code	–	57.6%	–	–
110	Ling 2.6 Flash	none	57.4%	–	–
111	Sonnet 4.5	–	57.3%	–	–
112	DeepSeek V3.1	–	57.0%	–	–
113	Motif-2-12.7B-Reasoning	–	57.0%	–	–
114	Ling-2.6-1T	none	56.9%	–	–
115	Sonnet 4.6	max	56.6%	–	–
116	Trinity Large Thinking	–	56.3%	–	–
117	LFM2.5-8B-A1B	–	55.6%	–	–
118	Opus 4.1	–	55.4%	–	–
119	K2-V2 (Medium)	medium	55.1%	–	–
120	Sonnet 4	–	54.7%	–	–
121	Tri-21B-Think	–	54.6%	–	–
122	Falcon-H1R-7B	–	54.4%	–	–
123	Haiku 4.5	–	54.3%	–	–
124	Opus 4	–	53.7%	–	–
125	Grok 4	–	53.7%	–	–
126	Opus 4.6	max	53.1%	–	–
127	Grok 4.1 Fast	–	52.7%	–	–
128	Gemini 2.5 Flash Lite	–	52.6%	–	–
129	Jamba Reasoning 3B	–	52.4%	–	–
130	Gemini 2.5 Flash	–	52.3%	–	–
131	Nova 2.0 Pro Preview (Non-reasoning)	none	52.0%	–	–
132	Qwen 3.5 4B	–	52.0%	–	–
133	Doubao Seed Code	–	51.4%	–	–
134	Qwen 3 235B-A22B	–	51.2%	–	–
135	Qwen 3.5	none	51.2%	–	–
136	Qwen 3 30B-A3B	–	50.7%	–	–
137	Grok 4 Fast	–	50.5%	–	–
138	Step3 VL 10B	–	50.2%	–	–
139	Qwen 3 4B	–	49.8%	–	–
140	Mi:dm K 2.5 Pro	–	49.3%	–	–
141	MiniCPM5-1B (Reasoning)	–	49.3%	–	–
142	Olmo 3 32B Think	–	49.1%	–	–
143	Gemini 2.5 Pro	–	48.7%	–	–
144	Sonnet 3.7	–	48.3%	–	–
145	Mistral Small 4	–	48.2%	–	–
146	Tri-21B-think Preview	–	47.1%	–	–
147	Llama 3.3 70b	none	47.1%	–	–
148	Grok 3	none	46.9%	–	–
149	Cogito v2.1 (Reasoning)	–	46.3%	–	–
150	Grok 3 Mini	high	45.9%	–	–
151	LFM2 24B A2B	none	45.9%	–	–
152	Mi:dm K 2.5 Pro Preview	–	45.6%	–	–
153	Ring-2.6-1T	xhigh	44.6%	–	–
154	Ring-1T	–	44.6%	–	–
155	Magistral Small 1.2	–	44.4%	–	–
156	Granite 4.1 30B	none	44.4%	–	–
157	Gemma 4 E4B (Reasoning)	–	44.2%	–	–
158	GLM 4.5	–	44.1%	–	–
159	LFM2.5-1.2B-Instruct	none	43.8%	–	–
160	GLM 4.6	–	43.4%	–	–
161	Ring-flash-2.0	–	43.3%	–	–
162	LongCat Flash Lite	none	43.1%	–	–
163	GPT-4.1	none	43.0%	–	–
164	Magistral Medium 1.2	medium	43.0%	–	–
165	Llama 4 Maverick	none	43.0%	–	–
166	Haiku 3.5	none	42.8%	–	–
167	JT-35B-Flash	none	42.0%	–	–
168	Seed-OSS-36B-Instruct	–	41.9%	–	–
169	LFM2.5-1.2B-Thinking	–	41.8%	–	–
170	MiniMax M1	–	41.8%	–	–
171	Olmo 3 7B Think	–	41.5%	–	–
172	ERNIE 5.0 Thinking Preview	–	41.4%	–	–
173	Grok Code Fast 1	–	41.4%	–	–
174	Nova 2.0 Omni (Non-reasoning)	none	41.1%	–	–
175	DeepSeek V3	none	41.0%	–	–
176	K2-V2 (Low)	low	41.0%	–	–
177	Nova 2.0 Lite (Non-reasoning)	none	40.5%	–	–
178	Gemma 4 E4B (Non-reasoning)	none	40.5%	–	–
179	Qwen 3 14B	–	40.5%	–	–
180	Qwen 3 Coder	none	40.5%	–	–
181	Gemini 2.0 Flash	none	40.2%	–	–
182	Mistral Medium 3.1	none	39.8%	–	–
183	DeepSeek R1	–	39.6%	–	–
184	K-EXAONE (Non-reasoning)	none	39.6%	–	–
185	Llama 4 Scout	none	39.5%	–	–
186	Llama 3.3	none	39.5%	–	–
187	Mistral Medium 3	none	39.3%	–	–
188	Olmo 3.1 32B Instruct	none	39.2%	–	–
189	ERNIE 4.5 300B A47B	none	39.1%	–	–
190	Llama 3.1 405b	none	39.0%	–	–
191	QwQ 32B	–	38.8%	–	–
192	Granite 4.1 8B	none	38.6%	–	–
193	GPT-4.1 Mini	none	38.3%	–	–
194	Llama 3.1	max	38.2%	–	–
195	Devstral 2	none	38.1%	–	–
196	Nova Pro	none	38.1%	–	–
197	OLMo 2 32B	none	38.1%	–	–
198	Gemma 4 E2B (Reasoning)	–	38.0%	–	–
199	HyperCLOVA X SEED Think (32B)	–	37.9%	–	–
200	GLM 4.5 Air	–	37.6%	–	–
201	NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning)	none	37.5%	–	–
202	Solar Pro 2 (Reasoning)	high	37.1%	–	–
203	Llama 3 70b	none	37.1%	–	–
204	Llama Nemotron Super 49B v1.5 (Reasoning)	–	37.0%	–	–
205	Qwen 2.5 72B	none	36.9%	–	–
206	Gemma 3 12B	none	36.7%	–	–
207	JT-MINI	none	36.7%	–	–
208	EXAONE 4.0 32B (Reasoning)	–	36.3%	–	–
209	Qwen 3 32B	–	36.3%	–	–
210	Mistral Large 3	none	36.2%	–	–
211	Nova Premier	none	36.2%	–	–
212	Haiku 3	none	36.1%	–	–
213	GPT-4o	none	36.0%	–	–
214	Nanbeige4.1-3B	–	35.4%	–	–
215	MiniCPM5-1B (Non-reasoning)	none	35.2%	–	–
216	Jamba 1.7 Large	none	35.2%	–	–
217	Ling-1T	none	34.8%	–	–
218	Devstral Small	none	34.6%	–	–
219	Pixtral Large	none	34.5%	–	–
220	Llama 3.1 70b	none	34.4%	–	–
221	Sarvam 105B (High)	high	34.4%	–	–
222	Ling-flash-2.0	none	34.4%	–	–
223	GLM 4.5 V	–	34.2%	–	–
224	Nova Lite	none	34.1%	–	–
225	INTELLECT-3 (Based on GLM-4.5-Air)	–	34.0%	–	–
226	Solar Pro 2 (Non-reasoning)	none	33.7%	–	–
227	Granite 4.1 3B	none	33.7%	–	–
228	Gemma 4 E2B (Non-reasoning)	none	33.6%	–	–
229	Mistral Small 3.2	none	33.5%	–	–
230	EXAONE 4.0 32B (Non-reasoning)	none	33.5%	–	–
231	Qwen 3 8B	–	33.5%	–	–
232	GPT-4	none	33.2%	–	–
233	LFM2.5-VL-1.6B	none	33.1%	–	–
234	Llama Nemotron Super 49B v1.5 (Non-reasoning)	none	32.9%	–	–
235	Olmo 3 7B Instruct	none	32.8%	–	–
236	GPT-4.1 Nano	none	32.0%	–	–
237	Ministral 3 14B	none	32.0%	–	–
238	NVIDIA Nemotron Nano 12B v2 VL (Reasoning)	–	31.9%	–	–
239	Gemma 3 27B	none	31.8%	–	–
240	Sarvam M (Reasoning, Based on Mistral Small 3.1)	high	31.8%	–	–
241	Mistral Large 2	none	31.6%	–	–
242	Qwen 3.5 2B	–	31.5%	–	–
243	Granite 4.0 H Small	none	31.5%	–	–
244	Jamba 1.7 Mini	none	31.4%	–	–
245	Devstral Small 2	none	31.2%	–	–
246	GPT-4o mini	none	31.0%	–	–
247	Reka Flash 3	–	30.4%	–	–
248	Llama 3.2 11b	none	30.4%	–	–
249	GLM 4.6 V	–	30.1%	–	–
250	Devstral Medium	none	29.9%	–	–
251	Mistral Small 3.1	none	29.9%	–	–
252	Nova Micro	none	29.4%	–	–
253	Ministral 3 8B	none	29.1%	–	–
254	Llama 3.1 8b	none	28.6%	–	–
255	Gemma 3 4B	none	28.3%	–	–
256	Kimi Linear 48B A3B Instruct	none	28.1%	–	–
257	Gemma 3n E4B Instruct	none	27.9%	–	–
258	NVIDIA Nemotron Nano 9B V2 (Reasoning)	–	27.6%	–	–
259	NVIDIA Nemotron Nano 9B V2 (Non-reasoning)	none	27.1%	–	–
260	Molmo2-8B	none	26.9%	–	–
261	Qwen 3 1.7B	–	26.9%	–	–
262	Ministral 3 3B	none	26.8%	–	–
263	MiniCPM-V 4.6 1.3B	none	26.7%	–	–
264	Sarvam 30B (High)	high	26.5%	–	–
265	Mistral Small 3	none	26.4%	–	–
266	LFM2 8B A1B	none	26.3%	–	–
267	Llama 3.2 3b	none	26.2%	–	–
268	Granite 4.0 H 1B	none	26.2%	–	–
269	NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning)	none	25.9%	–	–
270	Apertus 70B Instruct	none	25.9%	–	–
271	Exaone 4.0 1.2B (Non-reasoning)	none	25.3%	–	–
272	Magistral Medium 1	medium	25.1%	–	–
273	Magistral Small 1	–	24.8%	–	–
274	Granite 4.0 Micro	none	24.8%	–	–
275	Llama 3 8b	none	24.6%	–	–
276	OLMo 2 7B	none	24.4%	–	–
277	Phi-3 Mini Instruct 3.8B	none	23.9%	–	–
278	Ling-mini-2.0	none	23.6%	–	–
279	Phi-4	none	23.5%	–	–
280	Qwen 3 0.6B	–	23.3%	–	–
281	Exaone 4.0 1.2B (Reasoning)	–	23.0%	–	–
282	Llama 3.2 1b	none	22.8%	–	–
283	Granite 3.3 8B (Non-reasoning)	none	22.4%	–	–
284	Apertus 8B Instruct	none	22.4%	–	–
285	Gemma 3n E2B Instruct	none	22.0%	–	–
286	LFM2 1.2B	none	22.0%	–	–
287	Qwen 3.5 0.8B	none	21.6%	–	–
288	Phi-4 Mini Instruct	none	21.1%	–	–
289	Granite 4.0 1B	none	20.5%	–	–
290	Tiny Aya Global	none	20.1%	–	–
291	Mistral	none	19.9%	–	–
292	Gemma 3 1B	none	19.9%	–	–
293	Molmo 7B-D	none	19.7%	–	–
294	LFM2 2.6B	none	19.5%	–	–
295	Granite 4.0 H 350M	none	17.6%	–	–
296	Granite 4.0 350M	none	15.9%	–	–
297	Gemma 3 270M	none	12.1%	–	–