Add a benchmark

Benchmarks

297 benchmarks: public ones collected from their publishers every two hours, and 1 submitted by their authors. Add yours.

BenchmarkCategoryPublisherModelsUpdated
AutomationBench (Zapier)in indexAgenticZapier39–
Terminal-Bench 4.0in indexAgenticTerminal-Bench23–
DeepSWE v1.1in indexCodingDatacurve28–
ProgramBench v1 Fully Resolvedin indexCodingProgramBench622026-10-12
GPQA Diamond (Vals)in indexKnowledgeVals AI1202026-09-01
FrontierMath Tiers 1–3 (v2)in indexMathEpoch AI84–
Finance Agent v2in indexProfessionalVals AI772026-10-12
ARC-AGI-2in indexReasoningARC Prize85–
BullshitBench v2in indexReasoningPeter Gostev108–
SimpleBenchin indexReasoningSimpleBench96–
AA-3D3DArtificial Analysis22–
BuzzBench3Dbenchmarkheaven18–
ClutterBench3Dclutterbench102026-10-15
ClutterBench, simple prompt3Dclutterbench102026-10-15
EQ-Bench 43Dbenchmarkheaven28–
EQ-Bench Creative Writing v33Dbenchmarkheaven109–
EQ-Bench Longform Creative Writing3Dbenchmarkheaven104–
Japanese-RP-Bench (Aratako)3Dbenchmarkheaven20–
Japanese-RP-Bench v2 (tegnike fork)3Dbenchmarkheaven15–
Judgemark v43Dbenchmarkheaven40–
LLM Creative Story-Writing Benchmark3Dbenchmarkheaven44–
MineBench3DMineBench75–
PingPong English (unadjusted mean)3Dbenchmarkheaven26–
RP-Bench Community Elo3Dbenchmarkheaven11–
Slop Score3Dbenchmarkheaven18–
SlopBench3Dbenchmarkheaven25–
The Slop Index3Dbenchmarkheaven36–
threejseval3Dthreejseval272026-10-16
UGI Writing3Dbenchmarkheaven165–
VoxelBench (image)3DVoxelBench16–
VoxelBench (text)3DVoxelBench55–
AA-AnalystAgentAgenticArtificial Analysis48–
AA-Briefcase v1.1AgenticArtificial Analysis151–
AA-RoboticsAgenticaa-robotics5–
Agent ArenaAgenticLMArena50–
APEX-Agents-AAAgenticArtificial Analysis33–
ApprenticeBench API (NeoCognition)Agenticbenchmarkheaven27–
ApprenticeBench CUA (NeoCognition)Agenticbenchmarkheaven18–
AutomationBench-AAAgenticArtificial Analysis147–
BioMysteryBenchAgenticVals AI272026-10-12
BU Bench V1 (Browser Use)Agenticbenchmarkheaven7–
Butter-BenchAgenticAndon Labs5–
CADBenchAgenticboggs12–
CheatBenchAgenticboggs9–
CUA-benchAgenticVals AI92026-10-13
DAYJOB: FinanceAgenticboggs30–
DAYJOB: HealthcareAgenticboggs30–
EBR-bench (Earthborne Rangers, Epoch AI)Agenticepoch-tasks26–
EdgeBenchAgenticboggs5–
Elimination Game BenchmarkAgenticbenchmarkheaven48–
EnterpriseBench CoreCraftAgenticboggs45–
EnterpriseOps-Gym-AAAgenticArtificial Analysis48–
Epoch Automation ReportsAgenticboggs6–
ExploitBench v0.1AgenticExploitBench4–
GDP.xlsxAgenticboggs19–
HANDBOOK.md AgentsAgenticboggs32–
Harvey's Legal Agent BenchmarkAgenticVals AI782026-10-12
Hermes BenchAgenticboggs14–
HiL-BenchAgenticboggs17–
ITBench-AAAgenticArtificial Analysis43–
ITSMBenchAgenticboggs5–
JobBenchAgenticboggs21–
Legal Research BenchAgenticVals AI772026-10-12
Long-Horizon Terminal-BenchAgenticbenchmarkheaven21–
MazeBenchAgenticboggs25–
MCP Atlas (Scale AI)Agenticbenchmarkheaven30–
MCPMark VerifiedAgenticbenchmarkheaven8–
Mercor APEXAgenticboggs52–
METR Time Horizon (50%)Agenticmetr21–
OSWorld 2.0AgenticOSWorld2–
OSWorld-VerifiedAgenticOSWorld13–
PostTrainBench v1.1AgenticPostTrainBench11–
Production SimulationAgenticboggs6–
Remote Labor IndexAgenticboggs16–
RSI-Exam 0.1 (aiming-lab)Agenticbenchmarkheaven13–
Scale SEAL: SWE-Bench Pro V2 HARDAgenticbenchmarkheaven11–
SkillsBenchAgenticVals AI342026-09-27
SpireClimb (Epoch AI)Agenticepoch-tasks7–
Tax Agent BenchAgenticVals AI682026-10-12
Terminal-Bench 2.0AgenticTerminal-Bench40–
Terminal-Bench 2.1AgenticTerminal-Bench13–
Terminal-Bench 4.0 (Vals)AgenticVals AI472026-10-13
Terminal-Bench Science (Vals)AgenticVals AI402026-10-14
Time Horizon Index: KSPAgenticVals AI132026-10-07
Toolathlon-Verified (HKUST NLP)Agenticbenchmarkheaven25–
Toolathlon, original release (HKUST NLP, archived board)Agenticbenchmarkheaven33–
Vending-Bench 2AgenticAndon Labs64–
Vending-Bench ArenaAgenticAndon Labs12–
VitaBenchAgenticbenchmarkheaven12–
Web Search IndexAgenticVals AI42026-07-16
WeirdML v3AgenticWeirdML22–
τ^τ-bench (Hyper-τ), release v1 (Sierra)Agenticbenchmarkheaven5–
τ²-Bench TelecomAgenticArtificial Analysis293–
τ³-BankingAgenticArtificial Analysis153–
Aider PolyglotCodingbenchmarkheaven37–
ALE-BenchCodingboggs110–
Arena Image-to-WebDevCodingLMArena51–
Arena WebDevCodingLMArena108–
Code MigrationCodingVals AI762026-10-12
CursorBench 4.0 (Cursor)Codingbenchmarkheaven16–
DeepSWE v1CodingDatacurve21–
Drone-BenchCodingAndon Labs11–
FrontierCode 1.1 ExtendedCodingCognition46–
FrontierCode 1.1 MainCodingCognition46–
FrontierCode 1.1-AACodingArtificial Analysis25–
FrontierSWE v1CodingFrontierSWE17–
FrontierSWE v2CodingFrontierSWE20–
GameDevBenchCodingboggs17–
GBA EvalCodingboggs23–
GSO software optimization, Opt@1 (UC Berkeley)Codingbenchmarkheaven26–
HumanEvalCodingboggs42–
IOICodingVals AI432026-10-14
LiveCodeBenchCodingArtificial Analysis232–
LiveCodeBench (Vals)CodingVals AI1252026-09-01
MirrorCode (Epoch AI, with METR)Codingepoch-tasks9–
MLS-Bench-LiteCodingbenchmarkheaven14–
ProgramBench v1 Almost ResolvedCodingProgramBench622026-10-12
ProgramBench v1 Raw Pass RateCodingProgramBench622026-10-12
React Native EvalsCodingbenchmarkheaven23–
Real-SWE (Specific Labs)Codingbenchmarkheaven8–
Scale SEAL: SWE-Bench Pro V2 FullCodingbenchmarkheaven12–
SciCodeCodingArtificial Analysis153–
Senior SWE-BenchCodingboggs22–
SlopCodeBenchCodingboggs13–
sudo L7Codingboggs12–
SWE Atlas Refactoring (Scale AI)Codingbenchmarkheaven17–
SWE Atlas Test Writing (Scale AI)Codingbenchmarkheaven23–
SWE Atlas-QnACodingScale AI22–
SWE-bench MultilingualCodingbenchmarkheaven25–
SWE-bench MultimodalCodingbenchmarkheaven11–
SWE-Bench Pro PublicCodingbenchmarkheaven20–
SWE-bench VerifiedCodingbenchmarkheaven59–
SWE-bench Verified (Epoch AI)Codingepoch-tasks32–
SWE-bench Verified (Vals)CodingVals AI852026-09-01
SWE-rebench, issues 15 May – 1 Jul 2026 (Nebius)Codingbenchmarkheaven13–
Terminal-Bench 2.0 (Vals)CodingVals AI622026-06-04
Terminal-Bench 2.1 (Vals)CodingVals AI722026-09-28
Terminal-Bench Hard-AACodingArtificial Analysis288–
Terminal-Bench Science-AACodingArtificial Analysis31–
Terminal-Bench v2.1-AACodingArtificial Analysis183–
Terminal-Bench v4-AACodingArtificial Analysis145–
Vibe Code Bench 1-100CodingVals AI252026-10-14
Vibe Code Bench v1.1CodingVals AI1042026-10-12
VulcanBench Frontier v4Codingbenchmarkheaven6–
WeirdML v2CodingWeirdML111–
AA Intelligence Index v4.3.2CompositeArtificial Analysis441–
Arena TextCompositeLMArena212–
Arena Text — Multi-TurnCompositeLMArena211–
Epoch Capabilities IndexCompositeepoch-eci184–
Vals IndexCompositeVals AI472026-10-13
Vals Multimodal Index v1.2CompositeVals AI332026-08-11
Vals RSI Index v1.1CompositeVals AI252026-10-15
Vals RSI Index v1.1: Harness Engineering: JudgeCompositeVals AI252026-10-15
Vals RSI Index v1.1: Post-training: Finance AgentCompositeVals AI252026-10-15
Vals RSI Index v1.1: Pre-training: CompressionCompositeVals AI252026-10-15
Vals RSI Index v1.1: Pre-training: LM TrainingCompositeVals AI252026-10-15
AA Intelligence Index Cost per TaskCostArtificial Analysis112–
Blended Price (USD per 1M tokens, 3:1)CostArtificial Analysis251–
Cost to Run AA Intelligence IndexCostArtificial Analysis112–
Subscription token price (USD per 1M tokens)CostReal API Pricing682026-10-16
AA Cyber IndexCyberArtificial Analysis20–
CWE-BenchCyberArtificial Analysis20–
CWE-Bench-AA Refusal RateCyberArtificial Analysis20–
CyberBench v1.1CyberVals AI442026-10-12
CyberGym-E2E-AACyberArtificial Analysis20–
CyberGym-E2E-AA Refusal RateCyberArtificial Analysis20–
DeepsecBench-AACyberArtificial Analysis22–
DeepsecBench-AA Refusal RateCyberArtificial Analysis22–
HalluHardCyberboggs29–
Spiral-BenchCyberbenchmarkheaven21–
SRE BenchCyberVals AI342026-10-14
UGICyberbenchmarkheaven166–
UGI WillingnessCyberbenchmarkheaven166–
IFBenchInstruction FollowingArtificial Analysis297–
AA-OmniscienceKnowledgeArtificial Analysis365–
AA-Omniscience AccuracyKnowledgeArtificial Analysis365–
AA-Omniscience Non-hallucinationKnowledgeArtificial Analysis365–
Arena SearchKnowledgeLMArena36–
CaseLaw (Vals)KnowledgeVals AI532026-05-04
Global-MMLU-LiteKnowledgeArtificial Analysis98–
GPQA DiamondKnowledgeArtificial Analysis404–
Harvey LAB-AA v1.1KnowledgeArtificial Analysis23–
MedCodeKnowledgeVals AI952026-10-13
MedQA (Vals)KnowledgeVals AI782026-04-16
MedScribeKnowledgeVals AI972026-10-13
MMLU Pro (Vals)KnowledgeVals AI1212026-09-01
MMLU-ProKnowledgeArtificial Analysis235–
MMMU Pro (Vals)KnowledgeVals AI812026-09-01
MMMU-ProKnowledgeArtificial Analysis160–
Public Benefits Bench v1.1KnowledgeVals AI502026-10-12
SimpleQA Verified (Epoch AI run)Knowledgeepoch-tasks80–
UGI NatIntKnowledgebenchmarkheaven166–
AA-LCR v1.1Long ContextArtificial Analysis363–
Arena DocumentLong ContextLMArena37–
Context Arena — MRCR v2 (8-needle)Long Contextbenchmarkheaven64–
LongBench v2Long Contextbenchmarkheaven28–
MLCR-AALong ContextArtificial Analysis71–
RULERLong Contextbenchmarkheaven21–
AIME (Vals)MathVals AI802026-04-16
AIME 2025MathArtificial Analysis180–
AIME 2026 (MathArena)Mathbenchmarkheaven27–
Apex 2025 (MathArena)Mathbenchmarkheaven41–
Apex Shortlist 2025 (MathArena)Mathbenchmarkheaven33–
ArXivMath 06/2026 (MathArena)Mathbenchmarkheaven26–
ArXivMath 08/2026 (MathArena)Mathbenchmarkheaven11–
BrokenArXiv 06/2026 (MathArena)Mathbenchmarkheaven26–
BrokenArXiv 08/2026 (MathArena)Mathbenchmarkheaven11–
FrontierMath ErdősMathEpoch AI7–
FrontierMath Tier 4 (v2)MathEpoch AI66–
GSM8KMathboggs18–
HMMT February 2026 (MathArena)Mathbenchmarkheaven27–
HMMT November 2025 (MathArena)Mathbenchmarkheaven17–
MATH 500 (Vals)MathVals AI472026-01-09
MATH Level 5 (Epoch AI)Mathepoch-tasks61–
MATH-500MathArtificial Analysis141–
MathArenaMathboggs81–
MGSM (Vals)MathVals AI612026-01-09
OEIS Open (Epoch AI)Mathepoch-tasks3–
OEIS Open Lite (Epoch AI)Mathepoch-tasks5–
OTIS Mock AIME 2024-2025Mathepoch-tasks161–
ProofBench v1.1MathVals AI512026-10-12
Riemann-BenchMathboggs39–
USAMO 2026 (MathArena)Mathbenchmarkheaven8–
AA Openness IndexOtherArtificial Analysis240–
AGI-Pilled BenchOtheragi-pilled14–
Arena VisionOtherLMArena107–
Hearing BenchOtherhearingbench92026-10-13
LLM Divergent Thinking Creativity BenchmarkOtherbenchmarkheaven18–
Poker AgentOtherVals AI172025-12-23
T3 Code usage shareOtherSlopalytics852026-10-08
VoiceCodeBenchOtherVals AI202026-09-24
CorpFin (Vals)ProfessionalVals AI1152026-08-12
Excel Modeling BenchmarkProfessionalVals AI732026-10-12
GDP.pdfProfessionalArtificial Analysis146–
GDPval-AA v2.1ProfessionalArtificial Analysis203–
MortgageTax (Vals)ProfessionalVals AI842026-09-01
Public Benefits Bench v1 (Vals)ProfessionalVals AI132026-06-09
TaxEval (Vals)ProfessionalVals AI1242026-09-01
AlpacaEval 2.0 VerifiedReasoningboggs20–
ARC AI2Reasoningboggs18–
ARC-AGI-1ReasoningARC Prize82–
ARC-AGI-3ReasoningARC Prize22–
Arena-Hard-Auto v0.1Reasoningboggs28–
AutoResearchExamReasoningboggs11–
BabyVisionReasoningboggs19–
benchmarks.bio Expected PerformanceReasoningboggs13–
BIG-Bench HardReasoningboggs16–
Biopharma Bench v0.1Reasoningboggs12–
Blueprint-Bench 2ReasoningAndon Labs28–
BoardBenchReasoningboardbench7–
CAD-EvalReasoningboggs12–
ChartographyReasoningbenchmarkheaven41–
CharXiv ReasoningReasoningbenchmarkheaven24–
Chess Puzzles (Epoch AI)Reasoningepoch-tasks83–
ClockBenchReasoningboggs36–
ComplexConstraintsReasoningboggs39–
CritPtReasoningArtificial Analysis368–
DROPReasoningboggs15–
EQ-Bench 3Reasoningboggs69–
FrontierChallengeReasoningboggs11–
Furniture AssemblyReasoningEpoch AI33–
GDP.pdfReasoningbenchmarkheaven36–
GPQA Diamond (Epoch AI run)Reasoningepoch-tasks164–
Hearing BenchcommunityReasoningAdam Holter92026-10-13
HellaSwagReasoningboggs17–
HELM LiteReasoningboggs40–
Humanity's Last ExamReasoningArtificial Analysis420–
Humanity's Sixth SenseReasoningScale AI26–
Kaggle Game ArenaReasoningboggs37–
LegalBenchReasoningVals AI1352026-10-13
LiveBenchReasoningbenchmarkheaven56–
MathVistaReasoningboggs28–
MMMUReasoningbenchmarkheaven46–
Mystery Game Puzzles (Epoch AI)Reasoningepoch-tasks77–
MysteryMechanismReasoningVals AI272026-10-12
NanoGPT SpeedrunReasoningboggs18–
Olam Poker EloReasoningboggs22–
OpenBookQAReasoningboggs12–
Opus Magnum BenchReasoningboggs20–
PerceptionBenchReasoningboggs15–
PPBenchReasoningboggs53–
ResearchClawBenchReasoningbenchmarkheaven21–
Roboflow Visual ReasoningReasoningRoboflow57–
SAGEReasoningVals AI802026-10-08
SciUniverseReasoningboggs6–
SimpleQA VerifiedReasoningSimpleQA48–
TasteValReasoningboggs20–
TriviaQAReasoningboggs16–
WinoGrandeReasoningboggs22–
ZeroBenchReasoningboggs61–
Provider countReliabilityOpenRouter182–
Tool-call error rateReliabilityOpenRouter173–
Uptime (7d)ReliabilityOpenRouter182–
Latency (Time to First Token)SpeedArtificial Analysis211–
Output Speed (tokens/s)SpeedArtificial Analysis211–
Provider latencySpeedOpenRouter179–
Provider throughputSpeedOpenRouter178–