SimpleQA Verified · Open Analysis

#	Model	Effort	Score	Cost / task	Harness
1	Gemini 3.1 Pro	high	77.5%	–	–
2	GPT-6 Astra	max	75.8%	–	–
3	Gemini 4	max	75.6%	–	–
4	Gemini 3.8 Flash	max	74.6%	–	–
5	GPT-6.1 Sol	max	73.8%	–	–
6	Gemini 3.7 Flash	max	73.2%	–	–
7	Gemini 3 Pro	high	72.1%	–	–
8	Gemini 3.5 Flash	xhigh	70.4%	–	–
9	GPT-5.6 Sol	max	69.2%	–	–
10	GPT-6 Sol	max	64.4%	–	–
11	Opus 5	max	63.3%	–	–
12	Grok 4.7	max	59.4%	–	–
13	Grok 4.5	max	53.8%	–	–
14	Qwen 3 235B-A22B	xhigh	53.7%	–	–
15	Grok 4.6	max	52.9%	–	–
16	o3	high	52.3%	–	–
17	Grok 4	high	51.9%	–	–
18	Opus 4.6	max	45.2%	–	–
19	Sonnet 5.5	max	44.7%	–	–
20	Opus 4.8	max	44.5%	–	–
21	GPT-5.6 Terra	max	41.5%	–	–
22	Grok 3	none	39.3%	–	–
23	GPT-5.6 Luna	max	39.1%	–	–
24	Opus 4.5	high	39.0%	–	–
25	GPT-5.1	high	34.9%	–	–
26	Gemini 3.5 Flash-Lite	xhigh	33.1%	–	–
27	Sonnet 5	max	32.1%	–	–
28	Opus 4.1	high	31.0%	–	–
29	Sonnet 4.6	max	30.9%	–	–
30	GPT-5.4	xhigh	30.5%	–	–
31	Sonnet 4.5	high	29.2%	–	–
32	DeepSeek R1	xhigh	28.6%	–	–
33	Gemini 2.5 Flash	medium	28.1%	–	–
34	DeepSeek V3	none	27.4%	–	–
35	Haiku 5.5	max	23.9%	–	–
36	Mistral Large 2	none	22.3%	–	–
37	o4 mini	high	21.2%	–	–
38	Grok 2	none	20.5%	–	–
39	Grok 3 Mini	high	20.2%	–	–
40	gpt-oss-120b	high	13.1%	–	–
41	GPT-3.5	high	11.2%	–	–
42	Gemma 3 27B	none	10.7%	–	–
43	Gemma 4 31B	max	10.7%	–	–
44	GPT-4o mini	high	9.1%	–	–
45	Gemma 4 26B	xhigh	9.1%	–	–
46	Gemma 3 12B	none	6.7%	–	–
47	Codestral	–	5.6%	–	–
48	Gemma 3 4B	none	4.6%	–	–