Darwin-27B-ZTC-v9 Takes #1 on S1MB Across 197 Decision Models
aillmbenchmarksagents
The result
Darwin-27B-ZTC-v9 ranks #1 on the S1MB English v1 leaderboard, measured 20 October 2026.
| Rank | Model | Borda |
|---|---|---|
| 1 | Darwin-27B-ZTC-v9 | 90.38 |
| 2 | kettei-fat v1.0 | 90.14 |
| 3 | Superion S1 | 89.53 |
| 4 | D3 (27B) | 89.28 |
| 5 | Darwin-27B-ZTC-v5 | 88.44 |
The board carries 197 models over 137 benchmarks. Entries from Perplexity, Meta and Cloudflare sit further down the same table. Our own previous release, v5, is fifth, which gives a clean internal reference point for the step.
What the score actually measures
S1MB ranks by Borda score, not by a single accuracy number. Each of the 137 benchmarks produces its own ordering, every model gets a rank on each one, and those ranks are aggregated.
That distinction matters more than it first appears. A single averaged metric can be lifted by doing very well on a handful of tasks that happen to carry weight. Rank aggregation cannot be moved that way. To finish first you have to place well on most of the 137 boards at once, which is a far harder shape of result to produce by tuning for the test.
The board also publishes a task-average column alongside Borda. We report both: 72.20 on task average, which places second on that particular column. Borda is the board's default ranking and the one the headline refers to.
What the model does
ZTC stands for zero token choice. Given a situation and a typed question, the model returns a probability for every option in a single forward pass. It writes nothing.
Three question shapes cover most of what an agent needs to decide:
| Type | Returns | Example |
|---|---|---|
| noul | Probability the statement holds | Does this ticket need a reply today? |
| choice | A probability per option | Which team should own this? |
| score | A probability per level | How satisfied is this review, 1 to 5? |
There is no text to parse, no reasoning trace to wait for, and no format that can drift. An agent that needs to make a thousand small judgements per minute can afford to ask.
Calibration, not just accuracy
Decision benchmarks score whether the probabilities can be trusted, not only whether the top choice is right. Yes or no questions are scored with Brier score, choice questions on the probability mass placed on the correct option, and scales on expected score error.
This is the property that makes a decision model usable in production. A number that is right 70 percent of the time when it says 70 percent is a number a team can set a threshold on. An uncalibrated model that is accurate on average gives you no safe place to draw the line.
Selected per-task results from the model card, against the published baseline:
| Task | Baseline | v9 |
|---|---|---|
| boardgameqa-choice | 0.500 | 0.953 |
| snli-choice | 0.790 | 0.968 |
| gretel-pii-choice (entity type) | 0.653 | 0.981 |
| argument-quality-choice | 0.857 | 0.959 |
| ud-ewt-choice (part of speech) | 0.433 | 0.833 |
Running it
The model serves on the standard System One endpoint of Hugging Face transformers serve:
hf download FINAL-Bench/Darwin-27B-ZTC-v9 systemone_config.json --local-dir .
transformers serve FINAL-Bench/Darwin-27B-ZTC-v9 --decision-config systemone_config.json
Clients written against the public /v1/systemone shape work unchanged. Weights are Apache-2.0.
Where it fits
Routing, approval gates, triage, risk checks, content policy calls, and guards between agent steps. Anywhere a system needs a calibrated yes, no, pick or score many times a minute rather than a paragraph of prose.
Model: huggingface.co/FINAL-Bench/Darwin-27B-ZTC-v9 Leaderboard: S1MB
Scores above were read from the live leaderboard on 20 October 2026. Public boards move as new entries arrive.