Fingerprint bank changelog

v2026.10.3 2026-10-17

24 models, 864 recorded answers.

Adds Claude Haiku 5.5 from ModelTrace 6cd4a8f under the Owner’s upstream-first source decision. The bank contains 24 models and 864 enrollment replies. All 18 upstream models use the pinned upstream raw references; the other 6 retain their existing data. GPT-6.1 Sol is not in the upstream main branch and keeps the existing local sample. Scoring, prompts, input checks and verdict thresholds are unchanged; all 7 older bank URLs keep their bytes.

Against v2026.10.2, the only failed admission rule is the existing-model three-reply CV rule: GPT-6 Astra changes 9/12→8/12. The Owner reaffirmed the upstream-first decision after this regression was disclosed. This exact exception is recorded; the assessment remains failed. In-bank wrong Clear changes 4/1/0→4/0/0 at 1/2/3 replies. On the same 1,932 removed-model cases it changes 233→229; the 84 new-model cases add 0 wrong Clear. Simulated random input remains 0 Clear.

184 comparisons changed. Haiku 5.5 CV is 36/36, 36/36 and 12/12 at 1/2/3 replies. Reporting-only single-reply changes: Opus 4.7 34/36→35/36, GPT-5.4 29/36→30/36, Astra 24/36→25/36, and GPT-6.1 Sol 21/36→20/36. Existing two-reply CV counts are unchanged.

The 5 existing independent holdout sets retain identical per-case results on both API and chat paste paths. GPT-6.1 Sol remains 5/9 single-reply and 1/3 grouped top-1, with 0 Clear. Haiku’s 36 enrollment replies cover all 36 templates and 12 environments; no independent Haiku holdout or first-attempt collection rate is available. Its enrollment replay passes both input paths; this is not independent validation.

The previous 612 upstream rows are byte-identical; only 36 Haiku rows were added on 2026-10-09, declared as OAIPro. Screening 30,438 pairs involving Haiku found 0 shared 12-number blocks. This does not attest identity or independent generation. No new model calls were made.

Uses 36 upstream ModelTrace 6cd4a8f replies collected on 2026-10-09 through the declared OAIPro relay. Cross-validation is 36/36, 36/36 and 12/12 at 1/2/3 replies; no independent holdout or first-attempt rate is available. v2026.10.3 follows the Owner’s upstream-first decision despite Astra three-reply CV changing 9/12→8/12. Provenance is declared, not independently attested.

One/two-reply CV changes (reported)

Cross-channel Clear (reported)

v2026.10.2 2026-10-02

Adds GPT-6.1 Sol under an explicit Owner exception after three admission rules failed. The bank now contains 23 models and 828 enrollment replies. This model overlaps with GPT-6 Astra under the recorded sampling conditions; inclusion does not establish reliable separation. Scoring, prompts, input checks and verdict thresholds are unchanged, and all six previous bank URLs keep their bytes.

Admission baseline: v2026.10.1. The Owner explicitly chose publication after reviewing the regressions. In-bank wrong Clear rises from 2/0/0 to 4/1/0 at 1/2/3 replies (candidate n = 828/828/276). GPT-6 Astra three-reply CV falls from 12/12 to 9/12. On the same 1,848 removed-model cases, wrong Clear rises from 137 to 180; the 84 new-model cases add 53 wrong Clear separately. Simulated random input remains at 0 Clear. These are recorded exceptions, not passing admission results.

396 comparisons changed. In this 23-model bank, GPT-6.1 Sol one/two/three-reply CV is 21/36, 22/36 and 6/12. Astra single-reply CV changes 35/36→24/36 and two-reply CV 36/36→25/36. Single-reply Opus 4.7 and GPT-5.6 Sol change 35/36→34/36; Fable 5.1 improves 35/36→36/36. The complete changed-case table is below.

Nine separately collected holdout replies, never fitted: GPT-6.1 Sol top-1 is 5/9 for single replies and 1/3 for three-reply groups, with 0/9 and 0/3 Clear matches. The Chinese and English prose groups still name Astra first; the English JSON group names GPT-6.1 Sol first. Both API and chat paste paths agree. The four existing models with independent holdouts retain their top-1 and Clear counts and have no wrong Clear.

Under the site-prompt CV conditions, GPT-6.1 Sol has 0 correct Clear, 9 family-only and 3 insufficient runs out of 12. Across the enlarged bank the headline is 56/69 correct-model runs, with 0 wrong, 9 family-only and 4 insufficient. The original core winner stays first even when the evidence is weak; claims never rerank it.

Collected on 2026-10-02 with official Codex CLI 0.159.1, existing ChatGPT sign-in and low reasoning effort, one fresh ephemeral process per challenge. First-attempt usable: 35/36 enrollment and 9/9 holdout. Query-31 initially returned four numbers; that original is retained and one unchanged fresh-context retry supplied the usable reply. Of 45 selected replies, 33 exactly match the requested count and 12 do not; none was trimmed or repaired. Two shared twelve-number blocks are a diagnostic, not proof of independence.

Codex built-in and host instructions remain present; challenge system blocks are supplied as developer instructions. Model identity is the requested CLI selection, not independent attestation, and this is not a clean-API-equivalence claim. Upstream PR #28 remains a separate experimental data contribution; this website release does not imply that upstream merged or enabled it. Capability score 52 is Artificial Analysis v4.3.2 at max effort, not our fingerprint measurement; OpenAI lists $2 input / $10 output per million tokens.

Limited separation from GPT-6 Astra in this Codex CLI sample. Fresh holdout top-1: 5/9 single replies, 1/3 three-reply groups; Clear: 0/9 and 0/3. Three admission rules failed, and the Owner explicitly admitted this bank with those regressions disclosed. The model name remains the nearest bank candidate, not authenticated identity.

One/two-reply CV changes (reported)

Cross-channel Clear (reported)

396 changed comparisons in total. The complete list is kept in this release's evidence record and is not expanded here to keep this page small.

v2026.10.1 2026-10-02

Adds Claude Sonnet 5.5 from ModelTrace commit d4131b3: 36 original enrollment replies across 12 environments, collected through the OAIPro relay on 2026-09-30. The bank now contains 22 models and 792 replies. All previous reference rows and all five previously published bank URLs keep their bytes. No scoring code, challenge prompts, input checks or verdict thresholds changed.

All four admission rules pass against published v2026.09.5. In-bank wrong Clear stays 2/0/0 for 1/2/3 replies; the candidate has 792/792/264 cases. All 21 existing models retain the same cross-validation counts at every reply count, including 12/12 at three replies. On the same 1,764 removed-model cases, wrong Clear falls from 141 to 136: 3 newly wrong, 8 resolved. Sonnet 5.5 adds 84 separate removed-model cases with 1 wrong Clear. Simulated random input remains at 0 Clear.

258 comparisons changed. Sonnet 5.5 reaches 36/36, 36/36 and 12/12 top-1 in one/two/three-reply cross-validation. Under the site prompt conditions, all 12 Sonnet runs are correct Clear matches; when Sonnet is absent, all 12 are insufficient. These are evaluations of upstream enrollment data, not an independent holdout or a guarantee of real-world accuracy.

Reporting-only site-condition changes, 12 runs per model: Opus 4.8 correct Clear 7→5, Sonnet 5 10→9, GPT-6 Sol 6→5, and GPT-5.6 Terra 11→12; DeepSeek keeps 4 Clear while 1 family-only result becomes insufficient. No run becomes a wrong Clear. The site headline changes from 55/63 to 58/66 correct-model runs with 0 wrong; the denominator grows because Sonnet joins the bank.

The four existing models with separately collected holdouts retain single-reply top-1 of 8/9, 9/9, 8/9 and 9/9, and three-reply group top-1 of 3/3 each, with 0 wrong Clear. Kimi single-reply correct Clear rises 5→6; DeepSeek falls 4→3. Gemini and Qwen are unchanged.

Capability and price sources were reread on 2026-10-02. Sonnet 5.5 has an Artificial Analysis v4.3.2 score of 56 (Adaptive Reasoning, Max Effort, Default Fallback), and Anthropic lists $2 input / $10 output per million tokens. GPT-6 Luna now scores 38. The index marks GPT-6 Sol and Sonnet 5 superseded, so they keep their fingerprint pages and identification coverage but leave the capability board.

ModelTrace d4131b3 supplies these 36 replies through the OAIPro relay. The site-prompt evaluation gives 12/12 correct Clear matches using upstream enrollment data. No separate Sonnet 5.5 holdout was collected; serving identity and independent generation remain declared provenance, not attested facts.

One/two-reply CV changes (reported)

Cross-channel Clear (reported)

258 changed comparisons in total. The complete list is kept in this release's evidence record and is not expanded here to keep this page small.

v2026.09.5 2026-09-23

Adds GPT-6 Sol, GPT-6 Luna and Claude Opus 5.5, taken as published from the upstream ModelTrace fingerprint bank (commit 55a2e4a, 2026-09-23): the GPT models collected through an official Codex subscription, Claude Opus 5.5 through the OAIPro relay. Each has upstream's 36 enrollment replies across the 12 challenge environments, none with an error. We collected no replies of our own and no separate holdout for these three models. No scoring code, prompts, input checks or verdict thresholds changed; all four earlier bank URLs keep their exact bytes.

Admission uses the same 2026-09-18 four rules, baselined against published 2026.09.4. This bank misses one of them by one case and was admitted by an explicit Owner decision. Rule 3, wrong Clear in the removed-model population compared on the same 1,512 cases, goes from 131 to 132 (21 newly wrong, 20 resolved). The other three pass: in-bank wrong Clear stays 2/0/0 at 1/2/3 replies (n grows 648→756, 648→756, 216→252 as the three new models join the pool); all 18 existing models keep 12/12 three-reply CV; simulated random input stays at 0 Clear across 1/2/3 replies. Either family alone would have passed all four rules (GPT-6 Sol and Luna alone: 131→127; Claude Opus 5.5 alone: 131→131); the Owner chose to add all three together. The new models' own 252 removed-model rows, which measure what the previous bank did to their replies, add 9 wrong Clear and are reported separately.

858 comparisons changed against the previous bank. Reporting-only one/two-reply CV moves for existing models: GPT-5.4 33/36→29/36 at one reply and 36/36→35/36 at two; GPT-5.5 35/36→34/36, GPT-6 Astra 36/36→35/36 and DeepSeek V4 Pro 0813 34/36→33/36 at one reply; Claude Sonnet 4.6 34/36→36/36 and GPT-5.6 Sol 34/36→35/36 at one reply improve. None of these move any model's three-reply CV. Cross-channel Clear moves 47/72→46/72 at one reply and 48/72→51/72 at two, and stays 18/24 at three. The complete list of changed comparisons is below.

The three new models, cross-validated on upstream's own replies (each reply scored with itself left out of the bank; this is not a holdout): GPT-6 Sol top-1 33/36, 36/36 and 12/12 at 1/2/3 replies; GPT-6 Luna and Claude Opus 5.5 36/36, 36/36 and 12/12. Under the site's own conditions (three replies to one of the three prompt styles offered here, verdict applied) Claude Opus 5.5 is named Clear in 12 of 12 runs, GPT-6 Luna in 8 (1 close call, 3 weak) and GPT-6 Sol in 6 (4 close calls, 2 weak), with no wrong Clear. With the previous bank all three were refused without a false Clear; Claude Opus 5.5 came out nearest GPT-5.4 in 11 of 12 runs, all weak.

Existing models whose site-condition Clear count moved (reporting only, 12 runs each): Claude Opus 4.7 11→10, Claude Opus 5 12→11, GPT-6 Astra 11→10, DeepSeek V4 Pro 0813 5→4 and Qwen3.8 Max 0902 12→11. Each moved run became a close call or a weak match, not a wrong model. Across all models the site-condition result goes from 48 of 54 runs named correctly to 55 of 63, still with 0 wrong. One real paste moves the same way: the three ChatGPT replies the Owner identified as GPT-5.6 Sol (2026-09-15, this site's JSON prompts) went from a close call inside the GPT family to a weak match; GPT-5.6 Sol is still the closest model, with the new GPT-6 Sol second.

The four existing models that have a separately collected holdout were rescored on those saved replies with both banks: Kimi K2.6, Gemini 3.8 Flash, DeepSeek V4 Pro 0813 and Qwen3.8 Max 0902 keep the same single-reply top-1 (8/9, 9/9, 8/9, 9/9) and the same three-environment group results (3/3 each), with no wrong Clear. Kimi's single-reply Clear count rises from 4 to 5; the others are unchanged.

Provenance: upstream's GPT rows come from an official Codex subscription and its Claude rows from the OAIPro relay, so serving identity is declared, not independently attested. The GPT-6 rows record Codex CLI 0.155.1 with reasoning effort low; the earlier upstream GPT rows record neither setting. The new rows were cut byte for byte from the end of upstream's two reference files, whose earlier part — the 13 upstream models already in this bank — is unchanged.

Taken as published from upstream ModelTrace 55a2e4a, with no replies or holdout of our own. Cross-validated on upstream's 36 replies (not a holdout), single-reply top-1 is 33/36 for GPT-6 Sol and 36/36 for GPT-6 Luna and Claude Opus 5.5; all three reach 12/12 at three replies. Added in v2026.09.5 by an Owner decision although admission rule 3 moved from 131 to 132 on 1,512 removed-model cases; the changelog lists every change.

One/two-reply CV changes (reported)

Cross-channel Clear (reported)

858 changed comparisons in total. The complete list is kept in this release's evidence record and is not expanded here to keep this page small.

v2026.09.4 2026-09-18

Adds DeepSeek V4 Pro 0813 and Qwen3.8 Max 0902, both collected through OpenRouter with providers pinned (deepseek, alibaba), reasoning left at the provider default, and a 16,384-token completion cap. Each has 36 enrollment replies and 9 separate holdout replies to this site's own prompts, all finishing normally with zero errors and zero cap exhaustions. No scoring code, prompts, input checks or verdict thresholds changed; all three earlier bank URLs keep their exact bytes.

Admission uses the same 2026-09-18 four rules, baselined against published 2026.09.3: in-bank wrong Clear stays 2/0/0 at 1/2/3 replies (n grows 576→648, 576→648, 192→216 as the two new models join the pool); all 16 existing models keep 12/12 three-reply CV; wrong Clear in the removed-model population, compared on the same 1,344 cases, falls from 138 to 128 (12 newly wrong, 22 resolved); simulated random input stays at 0 Clear across 1/2/3 replies. The two new models' own 168 removed-model rows, which measure what the previous bank did to their replies, add 3 wrong Clear and are reported separately rather than counted against the rule.

512 comparisons changed against the previous bank. Reporting-only one/two-reply moves for existing models: Claude Sonnet 4.6 35/36→34/36 and Claude Sonnet 5 32/36→31/36 at one reply, Claude Opus 4.8 36/36→35/36 at two replies; Claude Opus 4.8 29/36→30/36 and GPT-5.6 Terra 35/36→36/36 at one reply improve. None of these move any model's three-reply CV. The complete list of changed comparisons is below.

Fresh holdout on the site's nine prompts, identical on the page's API and chat input paths: DeepSeek's single-reply top-1 goes from 0/9 with the previous bank to 8/9 (Clear 1→4), and its three-environment groups from 0/3 to 3/3 (Clear 0→1). Qwen's single-reply top-1 goes from 0/9 to 9/9, Clear 0→9 — every held-out reply is a Clear match — and its groups from 0/3 to 3/3 (Clear 0→3). All 90 saved replies from both models (36 enrollment plus 9 holdout each) pass the page's own input-surface check on both the api and chat branches; none would have been refused.

This supersedes a WP5d attempt that excluded both models: DeepSeek with reasoning disabled had produced a wrong Clear on its own query-28 reply and looping outputs of 2,700+ numbers, and Qwen's mandatory reasoning exhausted an 8,192-token cap on many requests. This round leaves reasoning at the provider default and raises the cap to 16,384 tokens; both models finished every request with finish_reason stop and zero errors.

Collector fix, evidence-only: a truncated chunked HTTP response (http.client.IncompleteRead, which subclasses HTTPException rather than OSError or ValueError) was not in collect_common.py's caught exception tuple, so one truncated reply crashed an entire in-progress batch instead of counting as a single failed attempt. Fixed by adding http.client.HTTPException to the tuple; the collector still sends each POST once, with no added retry. Both collector test suites were rerun green after the fix, before this batch resumed.

OpenRouter is itself a relay: the bank id records the pinned provider and each request's generation record was read back, but this is declared provenance, not an independent attestation of model identity. A relay that serves a different DeepSeek or Qwen build may not match.

Fresh holdout on the site's nine prompts, identical on the API and chat input paths: with the previous bank, top-1/Clear were 0/9 and 0/9 for both models; with this bank, DeepSeek reaches 8/9 and 4/9 for single replies and 3/3 and 1/3 for the three environment groups, and Qwen reaches 9/9 and 9/9 for single replies (every reply Clear) and 3/3 and 3/3 for the groups. Collected through OpenRouter with the serving provider pinned; identity is declared provenance, not an independent attestation.

One/two-reply CV changes (reported)

Cross-channel Clear (reported)

512 changed comparisons in total. The complete list is kept in this release's evidence record and is not expanded here to keep this page small.

v2026.09.3 2026-09-18

Adds Gemini 3.8 Flash, collected through OpenRouter pinned to Google Vertex, with 36 enrollment replies and 9 separate holdout replies to this site's own prompts. Gemini is the first family in this bank that is neither GPT nor Claude nor an open-weight model. No scoring code, prompts, input checks or verdict thresholds changed; both earlier bank URLs keep their exact bytes.

Admission uses the four 2026-09-18 rules: in-bank wrong Clear counts stay 2/0/0 at 1/2/3 replies; all 15 existing models keep 12/12 three-reply CV; wrong Clear in the removed-model population, compared on the same 1,260 cases, falls from 136 to 129 (9 newly wrong, 16 resolved); simulated random input stays at 0 Clear. Gemini's own 84 removed-model cases, which measure what the previous bank did to Gemini replies, add 9 wrong Clear and are reported separately rather than counted against the rule.

Gemini 3.8 Flash reaches 35/36, 36/36 and 12/12 at 1/2/3 replies in grouped cross-validation. One/two-reply changes for existing models: Claude Haiku 4.5 36/36 to 35/36 and GPT-5.6 Sol 35/36 to 34/36 at one reply; GPT-5.4 32/36 to 33/36 at one reply and 35/36 to 36/36 at two. Cross-channel Clear moves 45 to 46 at one reply and 47 to 46 at two. The complete list of changed comparisons is below.

Fresh holdout on the site's nine prompts: the previous bank named Gemini in 0/9 single replies and 0/3 three-reply groups; this bank names it in 9/9 and 3/3, with Clear in 7/9 and 3/3. Every reply passes both the page's API input path and the chat branch, so the two paths agree for this model.

Excluded this round: DeepSeek V4 Pro 0813 (its own reply query-28 was Clear-named Claude Opus 4.6 in the candidate, breaking rule 1; with reasoning disabled it also produced looping outputs of 2,700+ numbers and needed retries on three challenges; 11 of 36 replies were refused by the page's API path at the time because of a few values above 355) and Qwen3.8 Max 0902 (mandatory reasoning exhausted the 8,192-token cap on 6 of 23 enrollment attempts and 5 of 9 holdout attempts; collection continues separately). Both remain research evidence only.

OpenRouter is itself a relay: the bank id records the pinned provider and each request's generation record was read back, but this is declared provenance, not an independent attestation of model identity. A relay that serves a different Gemini build may not match.

Input update, 2026-09-18, no bank change: a JSON-array reply made of plain whole numbers is no longer refused because a few values fall outside 1 to 355; those values are ignored, exactly as in every other reply form, and nothing is clipped, repaired or re-split. In the 576 saved enrollment replies this rule had refused 20 replies from seven GPT and Claude models; all 20 are now admitted with results identical to the chat path, at most 4 ignored values per reply. Arrays with signs, decimals, exponents, text or nested items are still refused, as are malformed arrays and all 648 simulated hard-wrap cuts.

Fresh holdout on the site's nine prompts: with the previous bank, top-1/Clear were 0/9 and 0/9; with this bank, 9/9 and 7/9 for single replies and 3/3 and 3/3 for the three environment groups. The page's API input path and the chat branch admit all nine replies. Collected through OpenRouter pinned to Google Vertex; identity is declared provenance, not an independent attestation.

One/two-reply CV changes (reported)

Cross-channel Clear (reported)

340 changed comparisons in total. The complete list is kept in this release's evidence record and is not expanded here to keep this page small.

v2026.09.2 2026-09-18

Adds Kimi K2.6 served by Cloudflare Workers AI, using 36 preserved enrollment replies and 9 separate holdout replies collected on September 17. This is a serving-provider fingerprint, not proof of model identity across relays. No scoring code, prompts, input checks or verdict thresholds changed. The previous bank URL retains its exact bytes.

Admission now requires non-increasing in-bank wrong Clear counts at each of 1/2/3 replies, no three-reply CV decline for any existing model, a non-increasing total wrong Clear count when models are removed, and zero random-input Clear matches. One/two-reply CV changes, individual flips and cross-channel changes are disclosed below rather than silently used as additional release gates.

All 14 existing models retain 12/12 three-reply CV top-1 results. Kimi reaches 32/36, 35/36 and 12/12 at 1/2/3 replies. These are grouped cross-validation measurements, not a universal accuracy claim.

There are 6 newly wrong Clear cases and 9 resolved cases in the removed-model population; the aggregate falls from 139 to 136. The complete 186-case list below includes every changed top-1 or verdict among the same 3,120 baseline comparisons. New-model cases are also included in the aggregate admission counts.

The separately preserved three-group Claude replay keeps all top-1 results; Haiku Clear falls from 2/3 to 1/3, Sonnet stays 3/3 and Opus stays 0/3. Historical capture and retry provenance limitations still apply.

DeepSeek V4 Pro 0813 is not included: the new 24k-token request and its unchanged-body timeout retry both returned HTTP 408. GLM 5.3 is not included: its completed reply was 25,348 characters, beyond the unchanged 20,000-character input limit. No qualifying Qwen flagship was found in the checked Workers AI catalogue; the 30B pilot was not substituted.

Fresh holdout on the site's nine prompts. At release the page's API input path admitted 6/9 original replies, because two JSON-array replies contained a few values above 355 and one reply (query-17) mixed several code blocks with prose. Since the 2026-09-18 input update such arrays are read like any other reply, with out-of-range values ignored, and the page admits 8/9: top-1/Clear 8/9 and 5/9, and 3/3 and 2/3 for the three environment groups, the JSON group using two of three replies. Query-17 is still refused. No input was repaired or resampled; three prompt groups do not establish independent cross-provider coverage.

One/two-reply CV changes (reported)

Cross-channel Clear (reported)

186 changed comparisons in total. The complete list is kept in this release's evidence record and is not expanded here to keep this page small.

v2026.09.1 2026-09-13

First public bank. Thirteen models inherited from ModelTrace (six GPT, seven Claude, all collected through the API channel and labelled upstream-derived), plus Claude Fable 5.1 collected through Claude Code, one independent session per challenge. Input gates and the open-set verdict ship with it.