Add a benchmark

A community benchmark is a folder with two files. Open a pull request and it appears on the site once checks pass and a maintainer merges it.

1. community/my-bench/benchmark.json

{
  "id": "my-bench",
  "name": "My Bench",
  "description": "What a model has to do, and how it is scored.",
  "url": "https://my-bench.example.com",
  "maintainer": "Your Name",
  "category": "reasoning",
  "unit": "percent",
  "higherIsBetter": true,
  "lastUpdated": "2026-10-15"
}

Categories: coding, agentic, reasoning, knowledge, math, long-context, instruction, creative (shown as 3D), safety (shown as Cyber), economics (shown as Professional), other. Units: percent, score (add "scale": {"min": 0, "max": 10}), elo, usd, seconds, count.

2. community/my-bench/results.csv

model,effort,score,cost_usd,duration_sec,harness,source_url
claude-opus-5.5,xhigh,43.0,0.19,103,Claude Code,https://example.com/run-1
gpt-6.1-sol,high,23.0,0.008,29,Codex,
gemini-4,medium,43.0,0.023,33,,https://example.com/run-3

• model: an id from the models page. Common names like "Claude Opus 5.5" also work.
• effort: none, minimal, low, medium, high, xhigh or max. Leave it blank if you didn't set one; the model's highest effort is assumed.
• score is required. cost_usd (per task), duration_sec (per task) and harness are optional; cost and duration can't be negative.
• source_url: a link to this result's source, optional. If blank, the benchmark's url is used.
• Blank cells are ignored. model and score are required; blank effort uses the model's highest known effort. Blank cost, duration and harness are omitted.
• One row per model and effort. Only publish complete runs.

3. Check it

npm install
npm run check:community

The same check runs on every pull request. It reports unknown models, duplicate rows and out-of-range scores by line number.

What gets accepted

• A public page explaining the task and scoring, linked in url.
• Results you ran yourself or that your benchmark publishes. No copies of other leaderboards.
• Joining the Open Index is a separate pull request to open-index.json, decided on coverage and how much the benchmark adds.

Browse community benchmarks on GitHub