Add a benchmark

Benchmarks / Knowledge

MedQA (Vals)

Evaluating language model bias in medical questions. Scores are the values published by VALS AI. Source benchmark version 1.

78models
–Open Index weight
2026-04-16last updated
Vals AIpublisher

MedQA (Vals)

Higher is better · best effort per model

Chart settings
78 of 78 selected

full table below

#ModelEffortScoreCost / taskHarness
1o1high96.5%$0.053–
2GPT-5.1high96.4%$0.0098–
3Gemini 3.1 Prohigh96.4%$0.018–
4GPT-5high96.3%$0.021–
5GPT-5.4xhigh96.1%$0.044–
6o3high96.1%$0.0054–
7GPT-5 Minihigh96.1%$0.0030–
8Gemini 3 Prohigh96.0%$0.029–
9o4 minihigh96.0%$0.0028–
10Opus 4.5high95.9%$0.148–
11Gemini 3 Flashhigh95.8%$0.013–
12Opus 4.6max95.4%$0.027–
13Qwen 3.5 Plusnone95.2%$0.0083–
14o3 minihigh94.8%$0.0053–
15Sonnet 4.5high94.7%$0.031–
16Grok 4.20xhigh94.5%$0.0091–
17Kimi K2.5max94.4%$0.0070–
18GLM 5high94.3%$0.0080–
19GPT-5.2xhigh94.1%$0.022–
20DeepSeek V3.2high93.9%$0.0008–
21GLM 4.7none93.7%$0.0049–
22Opus 4.1high93.6%$0.088–
23GPT-5 Nanohigh93.3%$0.0013–
24Gemini 2.5 Prohigh93.1%$0.0029–
25Opus 4medium92.9%$0.038–
26Sonnet 4high92.7%$0.025–
27Kimi K2medium92.6%$0.0062–
28MiniMax M2.5high92.5%$0.0029–
29Grok 4high92.5%$0.026–
30Grok 2none92.3%$0.0033–
31GLM 4.6none92.2%$0.0043–
32Grok 4.1 Fasthigh92.1%$0.0006–
33Grok 4 Fastnone92.1%$0.0007–
34Sonnet 4.6max92.1%$0.035–
35Gemini 2.5 Flashmedium91.4%$0.0013–
36gpt-oss-120bhigh91.4%$0.0006–
37GPT-4.1high91.2%$0.0030–
38MiniMax M2.1–91.2%$0.0024–
39DeepSeek R1xhigh90.8%$0.0059–
40Qwen 3 235B–90.6%$0.0012–
41o1 minihigh90.2%$0.0043–
42Sonnet 3.7medium90.2%$0.017–
43Grok 3 Minihigh90.1%$0.0008–
44GLM 4.5xhigh90.0%$0.0034–
45Magistral Medium 1.2medium89.5%$0.0069–
46Gemini 2.5 Flash Litenone88.9%$0.0002–
47Llama 3.1max88.2%$0.0026–
48GPT-4ohigh88.2%$0.0028–
49Qwen 3 Maxmax87.4%$0.0032–
50GPT-4.1 Minihigh84.6%$0.0005–
51Grok 3none83.8%$0.0062–
52Claude 3.5 Sonnet Latest–83.2%$0.0060–
53gpt-oss-20bhigh82.9%$0.0003–
54Magistral Small 1.2–82.4%$0.0017–
55Mistral Large 3none82.2%$0.0008–
56DeepSeek V3none82.0%$0.0006–
57GPT-4high82.0%$0.012–
58Gemini 2.0 Flashmedium81.5%$0.0001–
59Command Anone80.5%$0.0035–
60Haiku 4.5xhigh79.6%$0.014–
61Mistral Medium 3.1none78.2%$0.0006–
62Qwen 2.5 Turbonone77.4%$0.0008–
63Gemini 1.5 Pronone76.5%$0.0030–
64Mistral Large 11–76.2%$0.0024–
65GPT-4o minihigh72.4%$0.0002–
66Mistral Small 3.1none69.1%$0.0001–
67GPT-4.1 Nanohigh68.2%$0.0002–
68Jamba 1.5 Largenone68.1%$0.0022–
69together/mistralai/Mixtral-8x22B-Instruct-v0.1–62.1%$0.0009–
70GPT-3.5high58.5%$0.0004–
71Mistral Small 02–57.0%$0.0002–
72Jamba 1.5 Mininone55.2%$0.0001–
73Mixtral 8x7B Instructnone53.2%$0.0004–
74Jamba 1.6 Mininone52.5%$0.0002–
75Llama 4 Scoutmedium50.9%$0.0003–
76Jamba 1.6 Largenone50.7%$0.0031–
77Llama 4 Maverickmedium43.3%$0.0007–
78Command-R+ (Apr '24)none2.7%$0.0035–