MATH-500

MATH-500: competition math subset; accuracy.

141models

–Open Index weight

–last updated

Artificial Analysispublisher

#	Model	Effort	Score	Cost / task	Harness
1	GPT-5	high	99.4%	–	–
2	o3	–	99.2%	–	–
3	Grok 3 Mini	high	99.2%	–	–
4	Sonnet 4	–	99.1%	–	–
5	Grok 4	–	99.0%	–	–
6	o4 mini	high	98.9%	–	–
7	Gemini 2.5 Pro	–	98.6%	–	–
8	o3 mini	high	98.5%	–	–
9	Qwen 3 235B-A22B	–	98.4%	–	–
10	Llama Nemotron Super 49B v1.5 (Reasoning)	–	98.3%	–	–
11	DeepSeek R1	–	98.3%	–	–
12	Opus 4	–	98.2%	–	–
13	Gemini 2.5 Flash	–	98.1%	–	–
14	MiniMax M1	–	98.0%	–	–
15	GLM 4.5	–	97.9%	–	–
16	EXAONE 4.0 32B (Reasoning)	–	97.7%	–	–
17	Qwen 3 30B-A3B	–	97.6%	–	–
18	Kimi K2	none	97.1%	–	–
19	o1	–	97.0%	–	–
20	Gemini 2.5 Flash Lite	–	96.9%	–	–
21	Solar Pro 2 (Reasoning)	high	96.7%	–	–
22	GLM 4.5 Air	–	96.5%	–	–
23	Magistral Small 1	–	96.3%	–	–
24	Qwen 3 14B	–	96.1%	–	–
25	Qwen 3 32B	–	96.1%	–	–
26	Llama 3.3	–	95.9%	–	–
27	Sonar Reasoning Pro	–	95.7%	–	–
28	QwQ 32B	–	95.7%	–	–
29	R1 1776	–	95.4%	–	–
30	Llama 3.1	max	95.2%	–	–
31	Sonnet 3.7	–	94.7%	–	–
32	o1 mini	–	94.4%	–	–
33	Gemini 2.0 Flash	–	94.4%	–	–
34	Qwen 3 Coder	none	94.2%	–	–
35	DeepSeek V3	none	94.2%	–	–
36	EXAONE 4.0 32B (Non-reasoning)	none	93.9%	–	–
37	Qwen 3 4B	–	93.3%	–	–
38	ERNIE 4.5 300B A47B	none	93.1%	–	–
39	GPT-4.1 Mini	none	92.5%	–	–
40	Gemini 2.0 Pro	none	92.3%	–	–
41	Sonar Reasoning	–	92.1%	–	–
42	Magistral Medium 1	medium	91.7%	–	–
43	GPT-4.1	none	91.3%	–	–
44	QwQ 32B-Preview	–	91.0%	–	–
45	Mistral Medium 3	none	90.7%	–	–
46	Qwen 3 8B	–	90.4%	–	–
47	Solar Pro 2 (Preview) (Reasoning)	high	90.0%	–	–
48	Qwen 3 1.7B	–	89.4%	–	–
49	GPT-4o	none	89.3%	–	–
50	Reka Flash 3	–	89.3%	–	–
51	Llama 4 Maverick	none	88.9%	–	–
52	Solar Pro 2 (Non-reasoning)	none	88.9%	–	–
53	Mistral Small 3.2	none	88.3%	–	–
54	Gemma 3 27B	none	88.3%	–	–
55	Gemini 1.5 Pro	none	87.6%	–	–
56	Gemini 2.0 Flash-Lite	none	87.3%	–	–
57	Solar Pro 2 (Preview) (Non-reasoning)	none	87.1%	–	–
58	Grok 3	none	87.0%	–	–
59	Qwen 2.5 72B	none	85.8%	–	–
60	Gemma 3 12B	none	85.3%	–	–
61	GPT-4.1 Nano	none	84.8%	–	–
62	Sarvam M (Reasoning, Based on Mistral Small 3.1)	high	84.7%	–	–
63	Llama 4 Scout	none	84.4%	–	–
64	Nova Premier	none	83.9%	–	–
65	Qwen 2.5 Max	none	83.5%	–	–
66	Gemini 1.5 Flash	none	82.7%	–	–
67	Command A	none	81.9%	–	–
68	Sonar	none	81.7%	–	–
69	Phi-4	none	81.0%	–	–
70	Qwen 2.5 32B	none	80.5%	–	–
71	Qwen 2.5 Turbo	none	80.5%	–	–
72	GPT-4o mini	none	78.9%	–	–
73	Nova Pro	none	78.6%	–	–
74	Grok 2	none	77.8%	–	–
75	Llama 3.3 70b	none	77.3%	–	–
76	Sonnet 3.5	none	77.1%	–	–
77	Gemma 3n E4B Instruct	none	77.1%	–	–
78	Llama Nemotron Super 49B v1.5 (Non-reasoning)	none	77.0%	–	–
79	Qwen 2.5 Coder	none	76.7%	–	–
80	Gemma 3 4B	none	76.6%	–	–
81	Nova Lite	none	76.5%	–	–
82	DeepSeek V2.5	none	76.3%	–	–
83	Qwen 3 0.6B	–	75.0%	–	–
84	Gemma 3n E4B Instruct Preview (May '25)	none	74.9%	–	–
85	Sonar Pro	none	74.5%	–	–
86	DeepSeek-Coder-V2	none	74.3%	–	–
87	GPT-4	none	73.7%	–	–
88	Grok Beta	none	73.7%	–	–
89	Mistral Large 2	none	73.6%	–	–
90	Haiku 3.5	none	72.1%	–	–
91	Mistral Small 3	none	71.5%	–	–
92	Pixtral Large	none	71.4%	–	–
93	Devstral Medium	none	70.7%	–	–
94	Mistral Small 3.1	none	70.7%	–	–
95	Llama 3.1 405b	none	70.3%	–	–
96	Nova Micro	none	70.3%	–	–
97	Qwen 2 72B	none	70.1%	–	–
98	Phi-4 Mini Instruct	none	69.6%	–	–
99	Phi-4 Multimodal Instruct	none	69.3%	–	–
100	Gemma 3n E2B Instruct	none	69.1%	–	–
101	Devstral Small	none	68.4%	–	–
102	Mistral	none	67.7%	–	–
103	Granite 3.3 8B (Non-reasoning)	none	66.5%	–	–
104	Llama 3.1 70b	none	64.9%	–	–
105	Opus 3	none	64.1%	–	–
106	Llama 3.2 90b	none	62.9%	–	–
107	Jamba 1.5 Large	none	60.6%	–	–
108	Jamba 1.7 Large	none	60.0%	–	–
109	Jamba 1.6 Large	none	58.0%	–	–
110	Mistral Small	none	56.3%	–	–
111	Mixtral 8x22B Instruct	none	54.5%	–	–
112	Reka Flash (Sep '24)	none	52.9%	–	–
113	Mistral Large	none	52.7%	–	–
114	Llama 3.1 8b	none	51.9%	–	–
115	Llama 3.2 11b	none	51.6%	–	–
116	Llama 3 8b	none	49.9%	–	–
117	Llama 3.2 3b	none	48.9%	–	–
118	Gemma 3 1B	none	48.4%	–	–
119	Llama 3 70b	none	48.3%	–	–
120	LFM 40B	none	48.0%	–	–
121	Phi-3 Mini Instruct 3.8B	none	45.7%	–	–
122	GPT-3.5	none	44.1%	–	–
123	Sonnet 3	none	41.4%	–	–
124	Mistral Medium	none	40.5%	–	–
125	Gemini 1.0 Pro	none	40.3%	–	–
126	Haiku 3	none	39.4%	–	–
127	Claude 2.1	none	37.4%	–	–
128	Jamba 1.5 Mini	none	35.7%	–	–
129	Solar Mini	none	33.1%	–	–
130	Llama 2 13b	none	32.9%	–	–
131	Llama 2 70b	none	32.3%	–	–
132	OpenChat 3.5 (1210)	none	30.7%	–	–
133	Mixtral 8x7B Instruct	none	29.9%	–	–
134	DBRX Instruct	none	27.9%	–	–
135	Command-R+ (Apr '24)	none	27.9%	–	–
136	Claude Instant	none	26.4%	–	–
137	Jamba 1.7 Mini	none	25.8%	–	–
138	Jamba 1.6 Mini	none	25.7%	–	–
139	Command-R (Mar '24)	none	16.4%	–	–
140	Llama 3.2 1b	none	14.0%	–	–
141	Llama 2 7b	none	5.9%	–	–