RU
← Home
Benchmark

Which AI models handle my work

Public AI leaderboards test other people's tasks. These are mine: find and sort information, write code, read a document or a scan, pick the right skill, do agency work. I keep the tasks private and the results open. Every run is dated, so you can see models change.

How I test

Project on hold

On 2026-09-26 I paused the benchmark. On simple tasks strong models solve almost everything, and a fair benchmark on tasks from my real work is a big project of its own. I may come back to it later. Below is what I managed to measure.

Plans and code on GitHub →

2026-09-26 — three runs on 29 simple tasks: each company's main models; the same models at every effort level; the rest, including the strongest (Fable, Astra) and free ones. Strong models solve almost everything, only small ones fall behind. Gemini sometimes reaches for the terminal, its program forbids it and returns no answer — those tasks are not counted; their number is shown under the model.

01/Latest run

Ranking

How many tasks each model solved out of those it got. There are only 29 tasks so far — a gap of one or two tasks between models may still be chance.

  • Anthropic
  • OpenAI
  • Google
  • OpenCode free
  1. Claude Opus 5.5
    Anthropic · effort: medium · isolation: strict · 2026-09-26
    29 of 29100%
    Tokens: read / written5,482 / 127
    Time per task5 s
    Would cost via API$0.5525
  2. Claude Sonnet 5
    Anthropic · effort: low · isolation: strict · 2026-09-26
    29 of 29100%
    Tokens: read / written5,549 / 147
    Time per task5 s
    Would cost via API$0.2891
  3. Claude Sonnet 5
    Anthropic · effort: max · isolation: strict · 2026-09-26
    29 of 29100%
    Tokens: read / written5,549 / 912
    Time per task11 s
    Would cost via API$0.5109
  4. Claude Sonnet 5
    Anthropic · effort: extra high · isolation: strict · 2026-09-26
    29 of 29100%
    Tokens: read / written5,549 / 304
    Time per task5 s
    Would cost via API$0.3344
  5. GPT-6 Sol
    OpenAI · effort: max · isolation: sandbox · 2026-09-26
    29 of 29100%
    Tokens: read / written16,435 / 143
    Time per task9 s
    Would cost via API$0.3801
  6. Gemini 3.1 Pro
    Google · effort: high · isolation: partial · 2026-09-26
    no answer — blocked by its own program: 11 · not supported by the program: 1
    17 of 17100%
    Tokens: read / written7,264 / 1,167
    Time per task15 s
    Would cost via API$0.5205
  7. Gemini 3.8 Flash
    Google · effort: high · isolation: partial · 2026-09-26
    no answer — blocked by its own program: 11 · not supported by the program: 1
    17 of 17100%
    Tokens: read / written17,707 / 1,417
    Time per task11 s
    Would cost via API$0.5096
  8. Claude Sonnet 5
    Anthropic · effort: medium · isolation: strict · 2026-09-26
    28 of 2997%
    Tokens: read / written5,549 / 145
    Time per task4 s
    Would cost via API$0.2883
  9. Claude Sonnet 5
    Anthropic · effort: high · isolation: strict · 2026-09-26
    28 of 2997%
    Tokens: read / written5,549 / 209
    Time per task6 s
    Would cost via API$0.3071
  10. GPT-5.6 Sol
    OpenAI · effort: medium · isolation: sandbox · 2026-09-26
    28 of 2997%
    Tokens: read / written15,858 / 60
    Time per task9 s
    Would cost via APIno data
  11. GPT-6 Astra
    OpenAI · effort: medium · isolation: sandbox · 2026-09-26
    28 of 2997%
    Tokens: read / written16,884 / 48
    Time per task9 s
    Would cost via API$1.7915
  12. GPT-6 Sol
    OpenAI · effort: medium · isolation: sandbox · 2026-09-26
    28 of 2997%
    Tokens: read / written17,062 / 116
    Time per task9 s
    Would cost via API$0.3800
  13. GPT-6 Sol
    OpenAI · effort: high · isolation: sandbox · 2026-09-26
    28 of 2997%
    Tokens: read / written16,439 / 112
    Time per task10 s
    Would cost via API$0.3712
  14. GPT-6 Sol
    OpenAI · effort: low · isolation: sandbox · 2026-09-26
    28 of 2997%
    Tokens: read / written16,920 / 59
    Time per task8 s
    Would cost via API$0.3592
  15. GPT-6 Sol
    OpenAI · effort: ultra · isolation: sandbox · 2026-09-26
    28 of 2997%
    Tokens: read / written16,481 / 149
    Time per task10 s
    Would cost via API$0.3845
  16. GPT-6 Sol
    OpenAI · effort: extra high · isolation: sandbox · 2026-09-26
    28 of 2997%
    Tokens: read / written16,434 / 126
    Time per task9 s
    Would cost via API$0.3749
  17. MiMo v2.6 Flash
    OpenCode free · isolation: sandbox · 2026-09-26
    not supported by the program: 1
    27 of 2896%
    Tokens: read / written9,474 / 72
    Time per task14 s
    Would cost via APIfree
  18. Gemini 3.8 Flash
    Google · effort: low · isolation: partial · 2026-09-26
    no answer — blocked by its own program: 8 · not supported by the program: 1
    19 of 2095%
    Tokens: read / written17,853 / 217
    Time per task8 s
    Would cost via API$0.3977
  19. Gemini 3.8 Flash
    Google · effort: medium · isolation: partial · 2026-09-26
    no answer — blocked by its own program: 10 · not supported by the program: 1
    17 of 1894%
    Tokens: read / written16,806 / 731
    Time per task9 s
    Would cost via API$0.4297
  20. Claude Fable 5.1
    Anthropic · effort: medium · isolation: strict · 2026-09-26
    27 of 2993%
    Tokens: read / written7,341 / 122
    Time per task5 s
    Would cost via API$1.3949
  21. GPT-6 Luna
    OpenAI · effort: medium · isolation: sandbox · 2026-09-26
    26 of 2990%
    Tokens: read / written16,256 / 49
    Time per task9 s
    Would cost via API$0.0191
  22. Muse Spark 1.3
    OpenCode free · isolation: sandbox · 2026-09-26
    not supported by the program: 1
    24 of 2886%
    Tokens: read / written11,306 / 83
    Time per task11 s
    Would cost via APIfree
  23. Claude Haiku 4.5
    Anthropic · isolation: strict · 2026-09-26
    22 of 2976%
    Tokens: read / written7,005 / 1,009
    Time per task11 s
    Would cost via API$0.3494
  24. Big Pickle
    OpenCode free · isolation: sandbox · 2026-09-26
    not supported by the program: 1
    13 of 2846%
    Tokens: read / written10,900 / 107
    Time per task7 s
    Would cost via APIfree
  25. Nemotron 3 Ultra
    OpenCode free · isolation: sandbox · 2026-09-26
    not supported by the program: 1
    13 of 2846%
    Tokens: read / written10,591 / 96
    Time per task10 s
    Would cost via APIfree

Every model gets the same tasks, a few hundred tokens each. The difference in “read” is the instructions each program adds on its own: Claude in strict mode about 2k tokens, OpenCode about 8k, Codex and Antigravity about 14k. Time and tokens are averages per task. The price uses API rates on the run date; subscriptions don't charge this.

02/Per task

Who solved what

Each model on each task in the latest run. ✓ solved, ✗ not solved, empty — not given.

 ask-01ask-02calc-01calc-02cli-flags-01code-review-01contract-defects-01longctx-32klongctx-8kreport-error-01research-notfound-01research-synth-01sandbox-hole-01scan-card-uz-latn-l1scan-invoice-en-l0scan-invoice-en-l2scan-invoice-en-l3scan-invoice-ru-l0scan-invoice-ru-l2scan-invoice-ru-l3scan-invoice-uz-cyrl-l0scan-invoice-uz-cyrl-l2scan-invoice-uz-cyrl-l3scan-invoice-uz-latn-l0scan-invoice-uz-latn-l2scan-invoice-uz-latn-l3scan-payment-ru-stampskill-route-01skill-route-20
Claude Opus 5.5solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
Claude Sonnet 5solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
Claude Sonnet 5solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
Claude Sonnet 5solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
GPT-6 Solsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
Gemini 3.1 Prosolvedsolvedsolvedsolvedsolvedsolvedsolvednot givensolvedsolvedsolvedsolvedsolvednot givensolvednot givensolvednot givennot givennot givennot givennot givennot givennot givennot givennot givensolvedsolvedsolved
Gemini 3.8 Flashsolvedsolvedsolvednot givensolvedsolvednot givennot givensolvedsolvedsolvedsolvedsolvednot givennot givensolvednot givennot givennot givennot givennot givensolvedsolvednot givensolvednot givensolvedsolvedsolved
Claude Sonnet 5solvedsolvedsolvedsolvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
Claude Sonnet 5not solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
GPT-5.6 Solsolvedsolvedsolvedsolvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
GPT-6 Astrasolvedsolvedsolvedsolvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
GPT-6 Solsolvedsolvedsolvedsolvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
GPT-6 Solsolvedsolvedsolvedsolvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
GPT-6 Solsolvedsolvedsolvedsolvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
GPT-6 Solsolvedsolvedsolvedsolvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
GPT-6 Solsolvedsolvedsolvedsolvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
MiMo v2.6 Flashsolvedsolvedsolvedsolvedsolvedsolvednot solvednot givensolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
Gemini 3.8 Flashsolvedsolvedsolvednot givensolvedsolvednot solvednot givensolvedsolvedsolvedsolvedsolvedsolvednot givensolvednot givennot givensolvedsolvednot givensolvednot givensolvednot givennot givensolvedsolvedsolved
Gemini 3.8 Flashsolvedsolvedsolvednot givensolvedsolvednot solvednot givensolvedsolvedsolvedsolvedsolvedsolvedsolvednot givennot givensolvednot givensolvednot givennot givennot givensolvednot givennot givennot givensolvedsolved
Claude Fable 5.1solvedsolvedsolvednot solvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
GPT-6 Lunasolvedsolvedsolvedsolvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvednot solvedsolvedsolvednot solvedsolvedsolvedsolved
Muse Spark 1.3solvedsolvedsolvedsolvedsolvedsolvednot solvednot givensolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvedsolvednot solvednot solvedsolvedsolvednot solvedsolvedsolvedsolved
Claude Haiku 4.5not solvedsolvedsolvednot solvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvednot solvedsolvedsolvedsolvedsolvedsolvedsolvednot solvednot solvednot solvedsolvedsolvedsolvedsolvedsolvedsolvedsolved
Big Picklesolvedsolvedsolvedsolvedsolvedsolvednot solvednot givensolvedsolvedsolvednot solvedsolvednot solvednot solvednot solvednot solvednot solvednot solvednot solvednot solvednot solvednot solvedsolvednot solvednot solvednot solvedsolvedsolved
Nemotron 3 Ultranot solvedsolvedsolvedsolvedsolvedsolvedsolvednot givensolvedsolvedsolvedsolvedsolvednot solvednot solvednot solvednot solvednot solvednot solvednot solvednot solvednot solvednot solvednot solvednot solvednot solvednot solvedsolvedsolved
03/History

Do models get dumber

Every week and after major updates I give the models the same tasks again. One bad day is chance. If a model fails several runs in a row, it really got worse.

The chart appears after the second run: so far everything was measured on one day.

04/Methodology

How I test

Own tasks, kept private

I don't publish tasks or correct answers. Otherwise they end up in model training and the test stops testing anything.

A program checks

Numbers, fields and files are compared to the correct answer automatically. A model saying “done” doesn't count.

Sometimes the right move is not to act

Some tasks lack data. The correct answer there is to ask, not to guess.

Only what was measured

Tasks solved, tokens and time spent. No forecasts or estimated ranges: there are few tasks so far, so draw conclusions carefully.

Every task from scratch

A new session with no memory and none of my settings. How strictly the model is fenced off from the rest of the computer is shown next to the result.

Tokens, time, price

How much text the model read and wrote, how long it took, and what it would cost at API rates on the run date.