Benchmarks / Long Context

AA-LCR v1.1

AA-LCR: long-context reasoning over ~100k-token document sets; pass rate.

363models
–Open Index weight
–last updated

AA-LCR v1.1

Higher is better · best effort per model

Chart settings
363 of 363 selected

full table below

#ModelEffortScoreCost / taskHarness
1Kimi K3max88.7%$0.309–
2Step 5 Preview–88.3%$0.100–
3MiMo V2.6 Pro–86.3%$0.047–
4Fable 5.1max85.3%$1.58–
5Opus 5.5max84.7%$0.677–
6GPT-5.5xhigh84.3%$0.520–
7GPT-6.1 Sollow84.0%$0.190–
8GPT-5.6 Solmax84.0%$0.402–
9Gemini 3.8 Flashmedium84.0%$0.090–
10DeepSeek V4.1 Flashmax84.0%$0.032–
11Muse Wonder Full (Max)max83.7%––
12GPT-6 Solmax83.7%$0.198–
13GPT-5.6 Lunamax83.7%$0.021–
14GPT-6 Lunamax83.3%$0.010–
15GPT-5.3 Codexxhigh83.3%––
16Solar Mini 4–83.3%$0.012–
17Muse Glimmer (High)high83.3%$0.031–
18Muse Spark 1.3max83.0%$0.133–
19GPT-5.6 Terramax83.0%$0.213–
20Ling 3.1 Flash–83.0%$0.034–
21Gemini 3.7 Flashmedium83.0%––
22Agnes 2.5 Pro Beta–83.0%––
23MiniMax M3–83.0%$0.030–
24Sonnet 5.5max82.7%$0.339–
25Haiku 5.5max82.7%$0.019–
26GPT-5.2xhigh82.7%––
27Fable 5max82.3%$1.57–
28GPT-5.2 Codexxhigh82.3%––
29Opus 5medium82.0%$0.738–
30GPT-5.4xhigh82.0%––
31Sonnet 5max82.0%$0.368–
32Qwen 3.8 27Bxhigh82.0%$0.062–
33Gemini 3.1 Pro–82.0%$0.213–
34Nex-N2-Pro–81.7%––
35Mistral Large 4–81.3%$0.138–
36DeepSeek V4 Flashmax81.3%$0.048–
37Grok 4.6xhigh81.0%$0.204–
38Kimi K2.6–81.0%$0.102–
39GPT-6 Astramax80.7%$0.997–
40Muse Wondermax80.7%$0.437–
41Qwen 3.6 Maxmax80.7%––
42Qwen 3.8 Maxmax80.3%$0.229–
43Qwen 3.8 2.4T-A95B–80.3%$0.222–
44DeepSeek V4 Pro 0813max80.3%$0.139–
45Grok 4.7low80.0%$0.211–
46GLM 5.3 Flashmax80.0%$0.017–
47Gemini 3.6 Flashhigh80.0%$0.090–
48K2 Horizon 375B A23B–80.0%––
49Sonnet 4.6max80.0%$0.385–
50GPT-5.1high80.0%––
51Gemini 4high79.7%$0.236–
52GLM 5.3max79.7%$0.149–
53Qwen 3.8 Flash–79.7%$0.018–
54DeepSeek V4 Flash 0731max79.7%$0.048–
55MiMo V2.5 Pro–79.7%$0.048–
56Grok 4.5high79.3%$0.203–
57Apodex 1.1–79.3%$0.044–
58Kimi K2.7 Code–79.3%$0.097–
59Nemotron 3 Ultramax79.3%$0.076–
60Muse Spark 1.2xhigh79.0%$0.133–
61Qwen 3.7 Maxmax79.0%$0.279–
62Hy3–79.0%$0.015–
63Opus 4.7max78.7%––
64Muse Spark Open (Xhigh)xhigh78.3%$0.129–
65JT-4.1 Flash 236B A21B (Reasoning)–78.3%––
66GLM 5.2max78.3%$0.154–
67Qwen 3.6 Plus–78.3%––
68Ling 3.0 Flash–78.3%––
69MiniMax M2.7–78.3%––
70GPT-5high78.2%––
71Opus 4.6max78.0%––
72Muse Spark–78.0%––
73Beam 1.0–78.0%––
74Gemini 3 Flash–78.0%––
75Kimi K2.5–78.0%––
76Opus 4.8max77.7%$0.798–
77Muse Spark 1.1xhigh77.7%$0.130–
78Qwen 3.5 27B–77.7%––
79Opus 4.5–77.3%––
80Inklingxhigh77.3%––
81Qwen 3.5 397B-A17B–77.3%$0.072–
82Qwen 3.6 27B–77.3%$0.065–
83GPT-5.4 Minixhigh77.0%$0.098–
84GPT-5.4 Nanoxhigh76.7%$0.024–
85Quasar 438B (Max, Based on GLM-5.2)max76.3%$0.091–
86Qwen 3.5 122B-A10B–76.3%$0.053–
87Motif 3–76.0%––
88Gemini 3 Prohigh76.0%––
89Opus 4.1–76.0%––
90Gemini 3.5 Flash-Litehigh76.0%$0.039–
91GLM 5–75.7%––
92Inkling Small–75.7%$0.032–
93MiMo V2–75.0%––
94Grok 4.3medium75.0%––
95Motif 3 (Beta)–74.7%––
96DeepSeek V4 Pro 0424max74.7%$0.046–
97Grok Build 0.1–74.7%––
98o3–74.7%––
99MiMo V2.6 Flash–74.3%$0.015–
100Gemini 3.5 Flashmedium74.3%––
101DeepSeek V4 Flash 0420max74.3%$0.012–
102Qwen 3 Maxmax74.3%––
103Haiku 4.5–74.3%$0.121–
104Gemini 3.1 Flash-Litehigh74.3%$0.027–
105Grok 4.1 Fast–74.0%––
106GLM 5.1–73.7%$0.143–
107Ling 3.0 Flash Fin–73.7%––
108Step 3.7 Flashhigh73.7%––
109Grok 4 Fast–73.7%––
110MiniMax M2.5–73.3%––
111DeepSeek V3.2–73.3%––
112Qwen 3.7 Plus–73.0%$0.045–
113MiMo V2.5–73.0%$0.016–
114KAT Coder Pro V2none73.0%––
115Ling 3.0 Flash–73.0%––
116Sonnet 4.5–72.3%$0.339–
117GPT-5 Minimedium72.3%––
118Kimi K2–72.0%––
119Qwen 3.5 35B-A3B–72.0%––
120Qwen 3 235B-A22B–72.0%––
121K2 Horizon MoVA 36B A4B–71.7%––
122GPT-5 Codexhigh71.7%––
123Solar Open2 250B–71.7%––
124Qwen 3.6 35B-A3B–71.7%$0.041–
125Doubao Seed Code–71.7%––
126Mercury 2.5–71.7%$0.028–
127JT-4.1 Flash 236B A21B (Non-reasoning)none71.3%––
128MiMo V2 Flash–71.3%––
129GLM 4.7–71.0%––
130Gemini 2.5 Flash–71.0%––
131Solar Pro 4–70.7%$0.034–
132GLM 5 V–70.3%––
133Sonnet 4–70.3%––
134GPT-5.5 Instant–70.0%$0.481–
135A.X-K2–70.0%––
136Ring-2.6-1Txhigh70.0%$0.031–
137Qwen 3.5 9B–70.0%$0.015–
138Nemotron 3.5 Bolt 74B-A9B–69.7%––
139K2 Horizon 7B–69.7%––
140Gemma 4 31B–69.7%$0.0000–
141GPT-5.1 Codexhigh69.3%––
142Opus 4–69.3%––
143DeepSeek V3.1–69.3%––
144Mistral Medium 3.5high69.3%$0.171–
145Nemotron 3.5 Super VL–69.0%$0.024–
146Grok 4.20–69.0%––
147Gemini 2.5 Pro–69.0%$0.137–
148MiMo V2 Pro–68.3%––
149GPT-4.1none68.3%––
150Grok 4–68.0%––
151MiniMax M2.1–67.7%––
152GPT-5.1 Codex Minihigh66.7%––
153JT-35B-Flashnone65.7%––
154Gemma 4 26B–65.7%––
155Nemotron 3 Super 120B A12B (Reasoning)–65.7%$0.036–
156G9v3-39A5B–65.3%––
157LongCat 2.0–65.0%$0.081–
158o1–65.0%––
159Nova 2.0 Pro Preview (Low)low64.7%––
160Gemini 2.5 Flash Lite–64.7%––
161MiniMax M2–64.3%––
162Nova 2.0 Pro Preview (Medium)medium64.0%––
163Qwen 3.5none63.7%––
164Gemma 4 12B–63.7%––
165Qwen 3–63.7%––
166Step 3.5 Flash 2603–63.0%––
167Qwen 3.5 4B–63.0%––
168K2 Horizon 3.7B–62.3%––
169K-EXAONE (Reasoning)–61.3%––
170Seed-OSS-36B-Instruct–61.3%––
171Qwen 3 30B-A3B–61.3%$0.038–
172o4 minihigh61.0%––
173Nova 2.0 Lite (High)high60.3%––
174Nemotron 3.5 Lightning–60.3%$0.0079–
175Ling 3.0 Tiny–60.3%$0.0000–
176K-EXAONE 2.0 0803–60.0%––
177Nova 2.0 Lite (Medium)medium60.0%––
178Nova 2.0 Omni (Medium)medium59.7%––
179MiniCPM5-2B–59.3%––
180Grok 3none58.0%––
181DeepSeek R1–57.7%$0.194–
182MiniMax M1–57.7%––
183Nova 2.0 Omni (Low)low57.7%––
184K2 Think V2high57.0%––
185GPT-4onone56.0%––
186Grok 3 Minihigh55.3%––
187EXAONE 4.5 33B (Reasoning)–54.7%––
188Ring-1T–54.7%––
189Nova 2.0 Lite (Low)low54.3%––
190GLM 4.6–54.0%––
191K-EXAONE (Non-reasoning)none53.3%––
192Grok Code Fast 1–53.0%––
193Magistral Medium 1.2medium53.0%––
194Command A–52.7%$0.0000–
195GLM 4.5–52.7%––
196gpt-oss-120bhigh52.0%$0.015–
197Sonnet 3.7none51.7%––
198Step 3.5 Flash–50.0%––
199Llama 4 Mavericknone50.0%$0.024–
200Mistral Small 4–49.7%$0.015–
201Granite 4.2 30B–49.0%$0.019–
202GLM 4.6 V–48.7%––
203Qwen 3 Codernone47.0%$0.053–
204GLM 4.5 Air–46.7%––
205GPT-5 Nanohigh45.0%––
206Granite 4.2 8B–45.0%$0.0079–
207GPT-4.1 Mininone44.0%––
208Mercury 2high43.7%$0.025–
209o3 minihigh43.0%––
210Mistral Medium 3.1none42.7%––
211Ling-2.6-1Tnone41.7%––
212GLM 4.7 Flash–41.7%––
213G9v3-3B–41.3%––
214Solar Open 100B (Reasoning)high41.0%––
215NVIDIA Nemotron Nano 12B v2 VL (Reasoning)–41.0%––
216DeepSeek V3none40.7%$0.023–
217INTELLECT-3 (Based on GLM-4.5-Air)–39.7%––
218Nemotron 3 Nano Omni 30B A3B Reasoning–39.7%––
219Nemotron Cascade 2 30B A3B–39.3%––
220KAT-Coder-Pro V1none38.7%––
221Trinity Large Thinking–38.0%$0.024–
222NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)–38.0%$0.0056–
223Celeris-1none38.0%$0.021–
224North Mini Code–37.3%$0.0000–
225Qwen 3 4B–37.3%––
226Ling-1Tnone36.7%––
227HyperNova 60B 2605 (High, Based on gpt-oss-120b)high36.3%––
228Mistral Large 3none36.0%$0.048–
229K2-V2 (High)high35.0%––
230gpt-oss-20bhigh34.7%$0.0072–
231Nova Premiernone34.3%––
232Devstral 2none32.3%––
233Solar Pro 3high32.3%$0.018–
234Devstral Mediumnone32.0%––
235Gemma 4 E4B (Reasoning)–32.0%$0.0019–
236Devstral Smallnone32.0%––
237Ling 2.6 Flashnone31.3%––
238Mistral Medium 3none31.3%––
239Gemini 2.0 Flashnone31.3%––
240Nova 2.0 Pro Preview (Non-reasoning)none30.0%––
241LongCat Flash Litenone28.3%––
242K2-V2 (Medium)medium28.0%––
243Devstral Small 2none28.0%––
244Kimi Linear 48B A3B Instructnone28.0%––
245Llama 4 Scoutnone27.7%$0.0052–
246Haiku 3none27.7%––
247Haiku 3.5none27.3%––
248QwQ 32B–26.7%––
249Ministral 3 14Bnone26.3%$0.019–
250Ministral 3 8Bnone25.7%$0.014–
251Llama 3.1 405bnone25.3%––
252Nova 2.0 Omni (Non-reasoning)none24.7%––
253Llama Nemotron Super 49B v1.5 (Non-reasoning)none24.7%––
254Granite 4.2 3B–24.3%$0.0037–
255Gemma 4 E4B (Non-reasoning)none24.0%––
256NVIDIA Nemotron Nano 9B V2 (Non-reasoning)none24.0%––
257Granite 4.1 30Bnone23.7%––
258NVIDIA Nemotron Nano 9B V2 (Reasoning)–22.7%––
259Cogito v2.1 (Reasoning)–22.7%––
260Mistral Small 3.1none22.3%$0.0076–
261Tri-21B-think Preview–21.7%––
262NVIDIA Nemotron 3 Nano 4B–21.7%––
263Apriel-v1.5-15B-Thinker–21.3%––
264Ring-flash-2.0–21.0%––
265Nova Pronone21.0%––
266Qwen 3.5 2B–21.0%––
267NVIDIA Nemotron Nano 12B v2 VL (Non-reasoning)none20.7%––
268GPT-4.1 Nanonone20.3%––
269K2-V2 (Low)low20.3%––
270Mistral Small 3.2none20.3%$0.0072–
271DiffusionGemma 26B A4B–19.7%––
272Magistral Small 1.2–19.3%––
273Jamba 1.7 Largenone19.3%––
274Nova Litenone19.0%––
275Nova 2.0 Lite (Non-reasoning)none18.7%––
276Llama 3.3–18.3%––
277Llama 3.1 8bnone18.0%––
278Ling-flash-2.0none17.7%––
279Ministral 3 3Bnone17.0%$0.0095–
280Gemma 4 E2B (Reasoning)–16.3%––
281Gemma 4 E2B (Non-reasoning)none16.3%––
282Jamba 1.7 Mininone16.0%––
283Tri-21B-Think–15.7%––
284Llama 3.3 70bnone15.7%––
285Phi-4 Mini Instructnone15.3%––
286Motif-2-12.7B-Reasoning–14.7%––
287Mi:dm K 2.5 Pro Preview–14.7%––
288HyperCLOVA X SEED Think (32B)–13.7%––
289Granite 4.1 8Bnone13.3%––
290JT-MINInone13.0%––
291Nova Micronone13.0%––
292Llama 3.2 11bnone12.7%––
293Granite 4.0 H Smallnone11.3%––
294Falcon-H1R-7B–11.0%––
295NVIDIA Nemotron 3 Nano 30B A3B (Non-reasoning)none10.7%––
296Llama 3.1 70b–9.7%––
297Mi:dm K 2.5 Pro–9.0%––
298Qwen 3.5 0.8B–9.0%––
299Llama 3.1none8.3%––
300Gemma 3 12Bnone8.3%$0.0000–
301Gemma 3 27Bnone7.3%$0.0000–
302MiniCPM-V 4.6 1.3Bnone7.0%––
303Granite 4.0 H 1Bnone7.0%––
304Granite 4.0 Micronone7.0%––
305Gemma 3 4Bnone6.7%––
306Llama 3.2 1bnone6.7%––
307Jamba Reasoning 3B–6.3%––
308K2 Horizon 0.9B–6.3%––
309MiniCPM5-1B (Reasoning)–6.0%––
310Granite 4.1 3Bnone6.0%––
311Granite 4.0 1Bnone6.0%––
312LFM2.5-2.6B–5.7%––
313MiniCPM5-1B (Non-reasoning)none5.0%––
314Llama 3.2 3bnone4.3%––
315Mistral Large 2none2.0%––
316Magistral Medium 1medium0.0%––
317Sarvam 105B (High)high0.0%––
318Nanbeige4.1-3B–0.0%––
319Magistral Small 1–0.0%––
320Step3 VL 10B–0.0%––
321GLM 4.5 V–0.0%––
322Solar Pro 2 (Reasoning)high0.0%––
323Qwen 3 32Bnone0.0%––
324LFM2.5-8B-A1B–0.0%––
325Olmo 3.1 32B Think–0.0%––
326Solar Pro 2 (Non-reasoning)none0.0%––
327Qwen 3 14Bnone0.0%––
328Mistral Small 3none0.0%––
329Sarvam 30B (High)high0.0%––
330Olmo 3.1 32B Instructnone0.0%––
331Olmo 3 32B Think–0.0%––
332Qwen 3 8Bnone0.0%––
333OLMo 2 32Bnone0.0%––
334LFM2 24B A2Bnone0.0%––
335Phi-4none0.0%––
336Reka Flash 3–0.0%––
337Olmo 3 7B Think–0.0%––
338OLMo 2 7Bnone0.0%––
339Molmo 7B-Dnone0.0%––
340Llama 3 70bnone0.0%––
341Sarvam M (Reasoning, Based on Mistral Small 3.1)high0.0%––
342Exaone 4.0 1.2B (Reasoning)–0.0%––
343Olmo 3 7B Instructnone0.0%––
344Exaone 4.0 1.2B (Non-reasoning)none0.0%––
345LFM2.5-1.2B-Thinking–0.0%––
346LFM2 2.6Bnone0.0%––
347LFM2.5-1.2B-Instructnone0.0%––
348Qwen 3 1.7B–0.0%––
349Gemma 3 270Mnone0.0%––
350Apertus 70B Instructnone0.0%––
351Mistralnone0.0%––
352Molmo2-8Bnone0.0%––
353Qwen 3 0.6Bnone0.0%––
354LFM2 1.2Bnone0.0%––
355LFM2.5-VL-1.6Bnone0.0%––
356Granite 4.0 350Mnone0.0%––
357Tiny Aya Globalnone0.0%––
358Llama 3 8bnone0.0%––
359Apertus 8B Instructnone0.0%––
360Granite 4.0 H 350Mnone0.0%––
361Gemma 3n E4B Instructnone0.0%––
362Gemma 3n E2B Instructnone0.0%––
363Gemma 3 1Bnone0.0%––