AA-Briefcase v1.1

AA-Briefcase: agentic office-work tasks over real files (xlsx/pptx/docx/pdf) judged pairwise; reported as Elo.

151models

–Open Index weight

–last updated

Artificial Analysispublisher

#	Model	Effort	Score	Cost / task	Harness
1	Sonnet 5.5	max	1817	$20.47	Stirrup
2	Opus 5.5	max	1803	$21.05	Stirrup
3	Muse Wonder Full (Max)	max	1729	–	Stirrup
4	Fable 5.1	max	1672	$22.71	Stirrup
5	Opus 5	max	1659	$17.79	Stirrup
6	Muse Wonder	max	1659	$12.45	Stirrup
7	Grok 4.7	xhigh	1643	$9.30	Stirrup
8	Qwen 3.8 Max	max	1615	$19.58	Stirrup
9	Muse Spark 1.3	max	1581	$4.06	Stirrup
10	Qwen 3.8 Flash	–	1580	$1.60	Stirrup
11	Haiku 5.5	max	1577	$0.675	Stirrup
12	GPT-6 Astra	max	1570	$9.50	Stirrup
13	GPT-6.1 Sol	max	1558	$2.31	Stirrup
14	Grok 4.6	xhigh	1545	$4.88	Stirrup
15	Fable 5	max	1539	$22.30	Stirrup
16	MiMo V2.6 Pro	–	1516	$0.380	Stirrup
17	GLM 5.3	max	1507	$5.65	Stirrup
18	Kimi K3	max	1500	$6.73	Stirrup
19	MiMo V2.6 Flash	–	1491	$0.152	Stirrup
20	Gemini 4	high	1486	$5.83	Stirrup
21	GPT-6 Sol	max	1481	$2.62	Stirrup
22	GPT-5.6 Sol	max	1480	$4.02	Stirrup
23	GLM 5.3 Flash	max	1453	$0.984	Stirrup
24	Qwen 3.8 2.4T-A95B	–	1435	$4.16	Stirrup
25	Step 5 Preview	–	1425	$2.37	Stirrup
26	DeepSeek V4 Flash	max	1425	$0.805	Stirrup
27	DeepSeek V4.1 Flash	max	1420	$0.566	Stirrup
28	Ling 3.1 Flash	–	1400	$2.02	Stirrup
29	Qwen 3.8 27B	xhigh	1397	$2.83	Stirrup
30	Mistral Large 4	–	1393	$2.38	Stirrup
31	Sonnet 5	max	1358	$14.43	Stirrup
32	GPT-5.6 Luna	max	1342	$0.395	Stirrup
33	GPT-6 Luna	max	1336	$0.170	Stirrup
34	GPT-5.6 Terra	max	1334	$2.83	Stirrup
35	Muse Spark 1.2	xhigh	1328	$1.90	Stirrup
36	Opus 4.8	max	1321	$8.26	Stirrup
37	K2 Horizon 375B A23B	–	1296	–	Stirrup
38	Grok 4.5	high	1281	$1.08	Stirrup
39	Muse Spark Open (Xhigh)	xhigh	1279	$1.47	Stirrup
40	DeepSeek V4 Flash 0731	max	1259	$0.423	Stirrup
41	DeepSeek V4 Pro 0813	max	1257	$0.995	Stirrup
42	Opus 4.7	max	1254	–	Stirrup
43	GLM 5.2	max	1229	$2.82	Stirrup
44	Gemini 3.8 Flash	high	1203	$2.70	Stirrup
45	K2 Horizon MoVA 36B A4B	–	1167	–	Stirrup
46	GPT-5.5	xhigh	1137	$5.15	Stirrup
47	Quasar 438B (Max, Based on GLM-5.2)	max	1123	$3.76	Stirrup
48	Gemini 3.7 Flash	high	1114	$2.15	Stirrup
49	Solar Pro 4	–	1093	$3.77	Stirrup
50	MiniMax M3	–	1091	$0.940	Stirrup
51	JT-4.1 Flash 236B A21B (Reasoning)	–	1089	–	Stirrup
52	GPT-5.5 Instant	–	1086	$1.11	Stirrup
53	Sonnet 4.6	max	1063	$2.41	Stirrup
54	DeepSeek V4 Flash 0420	high	1044	$0.213	Stirrup
55	K2 Horizon 7B	–	1016	–	Stirrup
56	Apodex 1.1	–	1013	$1.04	Stirrup
57	Nemotron 3.5 Super VL	–	999	$0.703	Stirrup
58	Motif 3	–	992	–	Stirrup
59	Ling 3.0 Flash	–	986	–	Stirrup
60	Beam 1.0	–	976	–	Stirrup
61	GLM 5.1	–	965	$0.840	Stirrup
62	Ling 3.0 Flash Fin	–	961	–	Stirrup
63	Gemini 3.6 Flash	high	954	$1.56	Stirrup
64	Hy3	–	947	$0.113	Stirrup
65	Nex-N2-Pro	–	941	–	Stirrup
66	DeepSeek V4 Pro 0424	max	922	$0.093	Stirrup
67	Qwen 3.7 Max	max	916	$1.30	Stirrup
68	Qwen 3.7 Plus	–	915	$0.240	Stirrup
69	Nemotron 3.5 Bolt 74B-A9B	–	912	–	Stirrup
70	Inkling Small	–	912	$0.118	Stirrup
71	MiMo V2.5 Pro	–	883	$0.072	Stirrup
72	Gemini 3.5 Flash	medium	877	–	Stirrup
73	Nemotron 3 Ultra	max	876	$1.33	Stirrup
74	Solar Mini 4	–	870	$0.951	Stirrup
75	GPT-5.3 Codex	xhigh	868	–	Stirrup
76	Kimi K2.7 Code	–	859	$0.788	Stirrup
77	Muse Spark 1.1	xhigh	851	$2.52	Stirrup
78	Inkling	xhigh	832	–	Stirrup
79	Kimi K2.6	–	820	$1.17	Stirrup
80	Qwen 3.6 27B	–	816	$1.05	Stirrup
81	Ling 3.0 Flash	–	798	–	Stirrup
82	K2 Horizon 3.7B	–	793	–	Stirrup
83	Grok 4.3	high	762	$0.298	Stirrup
84	MiMo V2.5	–	754	$0.016	Stirrup
85	LongCat 2.0	–	740	$0.364	Stirrup
86	A.X-K2	–	737	–	Stirrup
87	GPT-5.4 Mini	xhigh	719	$0.374	Stirrup
88	MiniMax M2.7	–	719	–	Stirrup
89	Sonnet 4.5	–	713	$1.03	Stirrup
90	Qwen 3.6 35B-A3B	–	713	$0.732	Stirrup
91	GPT-5.4 Nano	xhigh	674	$0.588	Stirrup
92	Ring-2.6-1T	xhigh	658	$0.754	Stirrup
93	Gemini 3.5 Flash-Lite	high	650	$0.168	Stirrup
94	Muse Spark	–	646	–	Stirrup
95	Haiku 4.5	–	616	$0.458	Stirrup
96	KAT-Coder-Pro V1	none	601	–	Stirrup
97	Qwen 3.5 397B-A17B	–	562	$0.748	Stirrup
98	Mistral Medium 3.1	none	530	–	Stirrup
99	Mistral Medium 3.5	high	523	$0.792	Stirrup
100	Nemotron 3.5 Lightning	–	506	$0.307	Stirrup
101	Qwen 3.5 122B-A10B	–	497	$0.530	Stirrup
102	Devstral Small 2	none	495	–	Stirrup
103	Devstral 2	none	482	–	Stirrup
104	Muse Glimmer (High)	high	477	$0.068	Stirrup
105	Ling 3.0 Tiny	–	475	$0.0000	Stirrup
106	DeepSeek V3.1	–	458	–	Stirrup
107	Gemini 3.1 Pro	–	457	$1.17	Stirrup
108	GPT-5 Mini	high	428	$0.083	Stirrup
109	MiniCPM5-2B	–	423	–	Stirrup
110	Qwen 3 Coder	none	404	$2.16	Stirrup
111	Granite 4.2 30B	–	394	$0.195	Stirrup
112	Mercury 2.5	–	378	$0.110	Stirrup
113	Gemma 4 31B	–	368	$0.0000	Stirrup
114	Command A	–	356	$0.0000	Stirrup
115	Qwen 3.5 9B	–	341	$0.243	Stirrup
116	Gemma 4 26B	–	315	–	Stirrup
117	Mistral Small 3.1	none	313	$0.066	Stirrup
118	Granite 4.2 8B	–	310	$0.088	Stirrup
119	Gemini 2.5 Pro	–	302	$0.498	Stirrup
120	Mercury 2	high	265	$0.049	Stirrup
121	Trinity Large Thinking	–	255	$0.153	Stirrup
122	Mistral Small 4	–	241	$0.019	Stirrup
123	HyperNova 60B 2605 (High, Based on gpt-oss-120b)	high	232	–	Stirrup
124	Mistral Large 3	none	228	$0.051	Stirrup
125	North Mini Code	–	212	$0.0000	Stirrup
126	Gemini 3.1 Flash-Lite	high	207	$0.041	Stirrup
127	Qwen 3 235B-A22B	–	169	–	Stirrup
128	Celeris-1	none	151	$0.146	Stirrup
129	o3 mini	high	142	–	Stirrup
130	Ministral 3 14B	none	131	$0.052	Stirrup
131	DeepSeek R1	–	128	$0.610	Stirrup
132	Solar Pro 3	high	99	$0.135	Stirrup
133	Granite 4.2 3B	–	98	$0.016	Stirrup
134	DeepSeek V3	none	78	$0.055	Stirrup
135	Ministral 3 8B	none	77	$0.017	Stirrup
136	NVIDIA Nemotron 3 Nano 30B A3B (Reasoning)	–	77	$0.054	Stirrup
137	K2 Think V2	high	7	–	Stirrup
138	Qwen 3 30B-A3B	–	5	$0.148	Stirrup
139	Nemotron 3 Super 120B A12B (Reasoning)	–	0	$9.03	Stirrup
140	gpt-oss-120b	high	0	$0.565	Stirrup
141	Llama 4 Maverick	none	0	$0.172	Stirrup
142	gpt-oss-20b	high	0	$0.0082	Stirrup
143	Gemma 4 E4B (Reasoning)	–	0	$0.014	Stirrup
144	Qwen 3 32B	–	0	–	Stirrup
145	Mistral Small 3.2	none	0	$0.453	Stirrup
146	Llama 4 Scout	none	0	$0.729	Stirrup
147	Qwen 3 14B	–	0	–	Stirrup
148	Qwen 3 8B	–	0	–	Stirrup
149	Ministral 3 3B	none	0	$0.022	Stirrup
150	NVIDIA Nemotron 3 Nano 4B	–	0	–	Stirrup
151	K2 Horizon 0.9B	–	0	–	Stirrup