Benchmarks / Coding

SWE-bench Multimodal

A 480-instance SWE-bench variant whose issue descriptions include visual elements. Maintained by SWE-bench Team. Score: % Resolved: percentage of task instances solved.

11models

–Open Index weight

–last updated

benchmarkheavenpublisher

#	Model	Effort	Score	Cost / task	Harness
1	Opus 5	max	59.4%	–	–
2	Fable 5.1	max	54.7%	–	–
3	Fable 5	max	54.1%	–	–
4	Opus 4.8	max	38.4%	–	–
5	o3	high	36.0%	–	GUIRepair
6	Opus 4.7	max	34.5%	–	–
7	Sonnet 4	high	34.4%	–	OpenHands-Versa
8	o4 mini	high	33.9%	–	GUIRepair
9	GPT-4.1	high	31.1%	–	GUIRepair
10	GPT-4o	high	30.4%	–	GUIRepair
11	Sonnet 3.5	medium	12.2%	–	SWE-agent