Benchmarks / Agentic
WeirdML v3
Fully agentic: 11 hand-made ML and data-analysis tasks run in coding-agent harnesses, where models must explore unfamiliar data and build pipelines with limited data, unspecified goals or little feedback. Overall effective score (95% interval).
WeirdML v3
Higher is better · best effort per model
full table below
| # | Model | Effort | Score |
|---|---|---|---|
| 1 | GPT-6 Astra | xhigh | 42.2% |
| 2 | GPT-6.1 Sol | xhigh | 38.3% |
| 3 | Opus 5.5 | xhigh | 31.2% |
| 4 | Fable 5.1 | xhigh | 26.0% |
| 5 | Sonnet 5.5 | xhigh | 19.9% |
| 6 | GPT-6 Sol | xhigh | 19.7% |
| 7 | Opus 5 | xhigh | 19.0% |
| 8 | Fable 5 | xhigh | 15.6% |
| 9 | GPT-5.6 Sol | xhigh | 15.0% |
| 10 | Muse Wonder | xhigh | 14.9% |
| 11 | Haiku 5.5 | xhigh | 11.7% |
| 12 | Grok 4.7 | xhigh | 8.5% |
| 13 | GPT-6 Luna | xhigh | 7.9% |
| 14 | Kimi K3 | high | 7.5% |
| 15 | Gemini 3.8 Flash | high | 7.4% |
| 16 | Muse Spark 1.3 | xhigh | 6.7% |
| 17 | Opus 4.5 | high | 6.5% |
| 18 | DeepSeek V4.1 Flash | high | 5.9% |
| 19 | GPT-5.6 Luna | xhigh | 5.2% |
| 20 | GPT-5.2 | xhigh | 5.0% |
| 21 | Mistral Large 4 | high | 4.4% |
| 22 | Inkling | xhigh | 3.9% |