MMLU-Pro

MMLU-Pro: broad knowledge multiple choice; accuracy.

235models

–Open Index weight

–last updated

Artificial Analysispublisher

#	Model	Effort	Score	Cost / task	Harness
1	Gemini 3 Pro	high	89.8%	–	–
2	Opus 4.5	–	89.5%	–	–
3	Gemini 3 Flash	–	89.0%	–	–
4	Opus 4.1	–	88.0%	–	–
5	MiniMax M2.1	–	87.5%	–	–
6	Sonnet 4.5	–	87.5%	–	–
7	GPT-5.2	xhigh	87.4%	–	–
8	Opus 4	–	87.3%	–	–
9	GPT-5	high	87.1%	–	–
10	GPT-5.1	high	87.0%	–	–
11	Grok 4	–	86.6%	–	–
12	GPT-5 Codex	high	86.5%	–	–
13	DeepSeek V3.2	–	86.3%	–	–
14	Gemini 2.5 Pro	–	86.2%	–	–
15	GPT-5.1 Codex	high	86.0%	–	–
16	GLM 4.7	–	85.6%	–	–
17	Doubao Seed Code	–	85.4%	–	–
18	Grok 4.1 Fast	–	85.4%	–	–
19	o3	–	85.3%	–	–
20	DeepSeek V3.1	–	85.1%	–	–
21	Grok 4 Fast	–	85.0%	–	–
22	Cogito v2.1 (Reasoning)	–	84.9%	–	–
23	DeepSeek R1	–	84.9%	–	–
24	Kimi K2	–	84.8%	–	–
25	o1	–	84.8%	–	–
26	MiMo V2 Flash	–	84.3%	–	–
27	Qwen 3 235B-A22B	–	84.3%	–	–
28	Gemini 2.5 Flash	–	84.2%	–	–
29	Sonnet 4	–	84.2%	–	–
30	Qwen 3 Max	none	84.1%	–	–
31	K-EXAONE (Reasoning)	–	83.8%	–	–
32	GPT-5 Mini	high	83.7%	–	–
33	Sonnet 3.7	–	83.7%	–	–
34	Qwen 3	–	83.6%	–	–
35	GLM 4.5	–	83.5%	–	–
36	o4 mini	high	83.2%	–	–
37	ERNIE 5.0 Thinking Preview	–	83.0%	–	–
38	Nova 2.0 Pro Preview (Medium)	medium	83.0%	–	–
39	GLM 4.6	–	82.9%	–	–
40	Llama 3.1 405b	–	82.9%	–	–
41	Grok 3 Mini	high	82.8%	–	–
42	Llama 3.1	max	82.5%	–	–
43	INTELLECT-3 (Based on GLM-4.5-Air)	–	82.2%	–	–
44	Ling-1T	none	82.2%	–	–
45	Nova 2.0 Pro Preview (Low)	low	82.2%	–	–
46	MiniMax M2	–	82.0%	–	–
47	GPT-5.1 Codex Mini	high	82.0%	–	–
48	DeepSeek V3	none	81.9%	–	–
49	Nova 2.0 Lite (High)	high	81.8%	–	–
50	EXAONE 4.0 32B (Reasoning)	–	81.8%	–	–
51	MiniMax M1	–	81.6%	–	–
52	Seed-OSS-36B-Instruct	–	81.5%	–	–
53	Magistral Medium 1.2	medium	81.5%	–	–
54	GLM 4.5 Air	–	81.5%	–	–
55	Llama Nemotron Super 49B v1.5 (Reasoning)	–	81.4%	–	–
56	Nova 2.0 Lite (Medium)	medium	81.3%	–	–
57	KAT-Coder-Pro V1	none	81.3%	–	–
58	Mi:dm K 2.5 Pro Preview	–	81.3%	–	–
59	Llama 3.1 70b	–	81.1%	–	–
60	K-EXAONE (Non-reasoning)	none	81.0%	–	–
61	Mi:dm K 2.5 Pro	–	80.9%	–	–
62	Nova 2.0 Omni (Medium)	medium	80.9%	–	–
63	Llama 4 Maverick	none	80.9%	–	–
64	Gemini 2.5 Flash Lite	–	80.8%	–	–
65	gpt-oss-120b	high	80.8%	–	–
66	Mistral Large 3	none	80.7%	–	–
67	GPT-4.1	none	80.6%	–	–
68	Ring-1T	–	80.6%	–	–
69	Solar Pro 2 (Reasoning)	high	80.5%	–	–
70	Qwen 3 30B-A3B	–	80.5%	–	–
71	Gemini 2.0 Pro	none	80.5%	–	–
72	GPT-4o	none	80.3%	–	–
73	o3 mini	high	80.2%	–	–
74	Haiku 4.5	none	80.0%	–	–
75	GLM 4.6 V	–	79.9%	–	–
76	Grok 3	none	79.9%	–	–
77	Qwen 3 32B	–	79.8%	–	–
78	Gemini 2.0 Flash	–	79.8%	–	–
79	Nova 2.0 Omni (Low)	low	79.8%	–	–
80	Motif-2-12.7B-Reasoning	–	79.6%	–	–
81	NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)	–	79.4%	–	–
82	Grok Code Fast 1	–	79.3%	–	–
83	Ring-flash-2.0	–	79.3%	–	–
84	Apriel-v1.6-15B-Thinker	–	79.0%	–	–
85	GLM 4.5 V	–	78.8%	–	–
86	Qwen 3 Coder	none	78.8%	–	–
87	Nova 2.0 Lite (Low)	low	78.8%	–	–
88	K2-V2 (High)	high	78.6%	–	–
89	HyperCLOVA X SEED Think (32B)	–	78.5%	–	–
90	Llama 3.3	–	78.5%	–	–
91	GPT-4.1 Mini	none	78.1%	–	–
92	GPT-5 Nano	high	78.0%	–	–
93	Celeris-1	none	78.0%	–	–
94	Ling-flash-2.0	none	77.7%	–	–
95	ERNIE 4.5 300B A47B	none	77.6%	–	–
96	Qwen 3 14B	–	77.4%	–	–
97	Apriel-v1.5-15B-Thinker	–	77.3%	–	–
98	Sonnet 3.5	none	77.2%	–	–
99	Nova 2.0 Pro Preview (Non-reasoning)	none	77.2%	–	–
100	Magistral Small 1.2	–	76.8%	–	–
101	EXAONE 4.0 32B (Non-reasoning)	none	76.8%	–	–
102	Solar Pro 2 (Preview) (Reasoning)	high	76.8%	–	–
103	QwQ 32B	–	76.4%	–	–
104	Olmo 3.1 32B Think	–	76.3%	–	–
105	Qwen 2.5 Max	none	76.2%	–	–
106	Devstral 2	none	76.2%	–	–
107	K2-V2 (Medium)	medium	76.1%	–	–
108	Mistral Medium 3	none	76.0%	–	–
109	Olmo 3 32B Think	–	75.9%	–	–
110	NVIDIA Nemotron Nano 12B v2 VL (Reasoning)	–	75.9%	–	–
111	Sonar Pro	none	75.5%	–	–
112	Magistral Medium 1	medium	75.3%	–	–
113	Llama 4 Scout	none	75.2%	–	–
114	Solar Pro 2 (Non-reasoning)	none	75.0%	–	–
115	Gemini 1.5 Pro	none	75.0%	–	–
116	gpt-oss-20b	high	74.8%	–	–
117	Magistral Small 1	–	74.6%	–	–
118	Nova 2.0 Lite (Non-reasoning)	none	74.3%	–	–
119	Qwen 3 4B	–	74.3%	–	–
120	Qwen 3 8B	–	74.3%	–	–
121	o1 mini	–	74.2%	–	–
122	NVIDIA Nemotron Nano 9B V2 (Reasoning)	–	74.2%	–	–
123	NVIDIA Nemotron Nano 9B V2 (Non-reasoning)	none	73.9%	–	–
124	Nova Premier	none	73.3%	–	–
125	Solar Pro 2 (Preview) (Non-reasoning)	none	72.5%	–	–
126	Falcon-H1R-7B	–	72.5%	–	–
127	Gemini 2.0 Flash-Lite	none	72.4%	–	–
128	Qwen 2.5 72B	none	72.0%	–	–
129	Nova 2.0 Omni (Non-reasoning)	none	71.9%	–	–
130	Phi-4	none	71.4%	–	–
131	Llama 3.3 70b	none	71.3%	–	–
132	K2-V2 (Low)	low	71.3%	–	–
133	Command A	none	71.2%	–	–
134	Grok 2	none	70.9%	–	–
135	Devstral Medium	none	70.8%	–	–
136	Grok Beta	none	70.3%	–	–
137	Pixtral Large	none	70.1%	–	–
138	Mistral Large 2	none	69.7%	–	–
139	Qwen 2.5 32B	none	69.7%	–	–
140	Opus 3	none	69.6%	–	–
141	Sarvam M (Reasoning, Based on Mistral Small 3.1)	high	69.6%	–	–
142	GPT-4	none	69.4%	–	–
143	Ministral 3 14B	none	69.3%	–	–
144	Llama Nemotron Super 49B v1.5 (Non-reasoning)	none	69.2%	–	–
145	Nova Pro	none	69.1%	–	–
146	Sonar	none	68.9%	–	–
147	Mistral Medium 3.1	none	68.3%	–	–
148	Mistral Small 3.2	none	68.1%	–	–
149	Gemini 1.5 Flash	none	68.0%	–	–
150	Devstral Small 2	none	67.8%	–	–
151	Ling-mini-2.0	none	67.1%	–	–
152	Llama 3.2 90b	none	67.1%	–	–
153	Reka Flash 3	–	66.9%	–	–
154	Gemma 3 27B	none	66.9%	–	–
155	DeepSeek V2.5	none	66.6%	–	–
156	Mistral Small 3.1	none	65.9%	–	–
157	GPT-4.1 Nano	none	65.7%	–	–
158	Olmo 3 7B Think	–	65.5%	–	–
159	Mistral Small 3	none	65.2%	–	–
160	NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning)	none	64.9%	–	–
161	QwQ 32B-Preview	–	64.8%	–	–
162	GPT-4o mini	none	64.8%	–	–
163	Ministral 3 8B	none	64.2%	–	–
164	Qwen 2.5 Coder	none	63.5%	–	–
165	Haiku 3.5	none	63.4%	–	–
166	Qwen 2.5 Turbo	none	63.3%	–	–
167	Devstral Small	none	63.2%	–	–
168	Granite 4.0 H Small	none	62.4%	–	–
169	Qwen 2 72B	none	62.2%	–	–
170	Mistral	none	61.1%	–	–
171	Gemma 3 12B	none	59.5%	–	–
172	Nova Lite	none	59.0%	–	–
173	Exaone 4.0 1.2B (Reasoning)	–	58.8%	–	–
174	Kimi Linear 48B A3B Instruct	none	58.5%	–	–
175	Sonnet 3	none	57.9%	–	–
176	NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning)	none	57.9%	–	–
177	Jamba Reasoning 3B	–	57.7%	–	–
178	Jamba 1.7 Large	none	57.7%	–	–
179	Llama 3 70b	none	57.4%	–	–
180	Jamba 1.5 Large	none	57.2%	–	–
181	Qwen 3 1.7B	–	57.0%	–	–
182	Jamba 1.6 Large	none	56.5%	–	–
183	Mixtral 8x22B Instruct	none	53.7%	–	–
184	Nova Micro	none	53.1%	–	–
185	Mistral Small	none	52.9%	–	–
186	Ministral 3 3B	none	52.4%	–	–
187	Olmo 3 7B Instruct	none	52.2%	–	–
188	Mistral Large	none	51.5%	–	–
189	OLMo 2 32B	none	51.1%	–	–
190	LFM2 8B A1B	none	50.5%	–	–
191	Exaone 4.0 1.2B (Non-reasoning)	none	50.0%	–	–
192	Claude 2.1	none	49.5%	–	–
193	Mistral Medium	none	49.1%	–	–
194	Gemma 3n E4B Instruct	none	48.8%	–	–
195	Claude 2.0	none	48.6%	–	–
196	Phi-4 Multimodal Instruct	none	48.5%	–	–
197	Gemma 3n E4B Instruct Preview (May '25)	none	48.3%	–	–
198	Llama 3.1 8b	none	47.6%	–	–
199	Granite 3.3 8B (Non-reasoning)	none	46.8%	–	–
200	Phi-4 Mini Instruct	none	46.5%	–	–
201	Llama 3.2 11b	none	46.4%	–	–
202	GPT-3.5	none	46.2%	–	–
203	Granite 4.0 Micro	none	44.7%	–	–
204	Phi-3 Mini Instruct 3.8B	none	43.5%	–	–
205	Claude Instant	none	43.4%	–	–
206	Command-R+ (Apr '24)	none	43.2%	–	–
207	Gemini 1.0 Pro	none	43.1%	–	–
208	DeepSeek Coder V2 Lite Instruct	none	42.9%	–	–
209	LFM 40B	none	42.5%	–	–
210	Gemma 3 4B	none	41.7%	–	–
211	Llama 2 13b	none	40.6%	–	–
212	Llama 2 70b	none	40.6%	–	–
213	Llama 3 8b	none	40.5%	–	–
214	DBRX Instruct	none	39.7%	–	–
215	Jamba 1.7 Mini	none	38.8%	–	–
216	Mixtral 8x7B Instruct	none	38.7%	–	–
217	Gemma 3n E2B Instruct	none	37.8%	–	–
218	Molmo 7B-D	none	37.1%	–	–
219	Jamba 1.5 Mini	none	37.1%	–	–
220	Jamba 1.6 Mini	none	36.7%	–	–
221	Llama 3.2 3b	none	34.7%	–	–
222	Qwen 3 0.6B	–	34.7%	–	–
223	Command-R (Mar '24)	none	33.8%	–	–
224	Granite 4.0 1B	none	32.5%	–	–
225	OpenChat 3.5 (1210)	none	31.0%	–	–
226	LFM2 2.6B	none	29.8%	–	–
227	OLMo 2 7B	none	28.2%	–	–
228	Granite 4.0 H 1B	none	27.7%	–	–
229	LFM2 1.2B	none	25.7%	–	–
230	Llama 3.2 1b	none	20.0%	–	–
231	Llama 2 7b	none	16.4%	–	–
232	Gemma 3 1B	none	13.5%	–	–
233	Granite 4.0 H 350M	none	12.7%	–	–
234	Granite 4.0 350M	none	12.4%	–	–
235	Gemma 3 270M	none	5.5%	–	–