How it works, and how well
Ask a language model for a few hundred random integers and it will not give you random integers. It gives you its habits: numbers it reaches for, numbers it avoids, a shape that survives rewording and is stable enough across sessions to tell models apart. This page describes the method, the two things this site adds on top of it, and the numbers that were measured rather than hoped for.
The protocol
Each challenge asks for between 292 and 333 integers from 1 to 355, chosen one at a time, with no tools and no repairs to the list afterwards. The identify tool hands out 3 such prompts and compares the replies against a bank of 14 models, each represented by 36 recorded answers.
The comparison is the ModelTrace method: each answer becomes a distribution over the 355 values, the distribution is transformed so that sampling noise and prompt wording matter less, and the result is scored against every model's centre. The bank, the scoring core, and the 36 challenge templates come from ModelTrace unchanged. This site vendors that core byte for byte and builds around it.
Length matters more than wording. Answers of 292 to 333 numbers fit their model with a median strength of 0.398 on the scale the verdict uses; answers of 218 to 257 fit at 0.302 and fall below the verdict's floor six times in 28. That is why every prompt on this site asks for at least 292.
Input gates
The scoring core is a closed-set classifier: whatever it is given, it ranks the bank and picks a winner. Fed a plain counted list from 1 to 300, it named a GPT model with 86% confidence. So before anything is scored, 6 rules decide whether the input is a model's answer at all.
- Nothing parsed, or every number the same.
- Sorted: nine in ten steps go the same direction. Real answers never exceed seven in ten.
- Fixed step: half or more of the steps are the same size. Real answers never exceed a quarter.
- Too few distinct values: fewer than one in twenty. One real answer in the bank goes as low as one in eleven, so the line sits below it.
- Too short to score: fewer than 55% of the requested count, which is the core's own floor.
Every threshold sits outside the most extreme value seen across all 576 real recorded answers, and a test fails if a threshold ever crosses one. The gates reject structure, not randomness: a uniformly random list passes them, because structurally it is indistinguishable from a model's answer. Rejecting it is the verdict's job.
The open-set verdict
The core's confidence number is a share of the bank, so it always adds up to one hundred percent over the models it knows. Uniform random input gets a median "probability" of 78 to 90 percent. That number is not used here.
Instead the verdict recovers what the core discards: how well the pasted answers agree with the winning model's centre in absolute terms (the fit), and how far ahead that model is of the runner-up (the margin). Fit below 0.25 means no model in the bank explains the answers, and the verdict says so. Margin of 0.08 or more means one model, and it is named. In between, if the family is clear at 0.9 or better, the verdict names the family and admits it cannot separate the models inside it.
Thresholds were chosen on 3,120 evidence records: held-out answers from the bank's own models, answers from each model scored against a bank rebuilt without it, uniform random input, and answers from two models collected through a different channel than the one in the bank.
What was measured
Only 3 of the 12 prompt environments the bank was built from can be reproduced by pasting into a chat window; the others need a real system field or a prefix of several thousand characters. So the figure the site quotes for itself is measured under its own conditions: three answers from one usable environment, verdict applied, over every model in the bank.
| Prompt style | Runs | Right model | Family only | No answer | Wrong model |
|---|---|---|---|---|---|
| 中文 | 14 | 11 | 3 | 0 | 0 |
| English, plain text | 14 | 13 | 1 | 0 | 0 |
| English, JSON array | 14 | 14 | 0 | 0 | 0 |
| All | 42 | 38 | 4 | 0 | 0 |
38 of 42 runs named the right model, 4 narrowed it to the family, and 0 named a wrong one. On the full held-out set across all twelve environments the verdict names the right model in 147 of 168 runs with 0 wrong. Uniform random input was given no answer in 600 of 600 runs and never matched a model.
What it cannot do
An unlisted version of a listed model is reported as its nearest sibling. With Claude Opus 4.7 removed from the bank, 7 of 12 three-answer runs confidently named Claude Opus 4.8. Close versions share a fingerprint, and no threshold separates what the feature space does not. Across all models removed one at a time, 16 of 168 runs still named a sibling. Each model's page states its own figure.
With one answer, roughly one random input in a hundred crosses the fit floor and can be told it belongs to a family. It was never named as a model in 640 tries, and with three answers none crossed the floor at all. Three answers are the recommendation for a reason.
The channel is not determined. Answers from Claude Sonnet 5 and Claude Haiku 4.5 collected through a coding tool were scored against bank entries collected through an API: 18 of 24 three-answer runs named the right model and 0 named a wrong one. The fingerprint follows the model, which is what makes a pasted chat reply recognisable, and it is also why the verdict cannot tell you which interface the reply came from.
Thirteen of the bank's entries are inherited. Their answers came through an API relay and the serving identity behind the label was never independently confirmed. They behave like the project's own entries in every test; the caveat is about provenance, not accuracy.
Credits
The fingerprint method, the scoring core, the challenge templates and thirteen of the bank's fingerprints are from ModelTrace by xqy2006, MIT licensed and used unmodified. The single-number baseline idea is credited to hlwy-ai-checker, which is not used in any form on this site. Everything else here, including the input gates, the open-set verdict and the Claude Fable 5.1 fingerprint, is this project's.