Hassana Labs published a number last month that looks almost too good: a hallucination rate of 0.0 to 0.7 per cent. Read as a headline, it says a model solved hallucination. Read against the paper itself, it says something narrower and more useful. The audit was pre-specified, on 528 held-out questions, each already carrying its own supporting evidence. A system permitted to decline an answer got the ones it attempted right at that rate, and declined to answer roughly a quarter of them. Both readings cite the same number. Only one of them describes what the number measures.
Episode 38 showed that the cheapest check almost nobody runs is comparing the date in a claim against the date on its source. This week's sweep ran that check again and caught more misdated claims the same way. They are not retold here. But four separate findings this week pass the date check cleanly and are still wrong. What moved was not the date. It was the population, or the task, the figure actually describes. A correct number, correctly dated, can still answer a question nobody asked it.
Four readings, one mistake
Hassana Labs' own site and the underlying paper, reviewed for ICML 2026, state the same thing in the same words. The result is a two-part confidence interval: 0.0 to 0.7 per cent hallucination and 20.6 to 27.9 per cent abstention. Both are measured against 528 pre-specified, evidence-grounded, yes-or-no questions, where the system may decline to answer. The paper's own description of the trade-off is direct: "the gate trades coverage for reliability." The original paper states its question type plainly. It states its sample size and interval status just as plainly. This week's own retelling kept none of it, flattening "0.0 to 0.7 per cent on a bounded binary audit with abstention" into "hallucination rates reduced to an astonishingly low 0.0 to 0.7 per cent," and quietly promoting the upper bound of the abstention interval, 27.9, into "nearly 28 per cent," as though an interval edge were a measured figure.
Hassana Labs did not overstate its own result. A number can be lifted cleanly out of a rigorous paper and still lose the sentence that tells a reader what population of questions, and what permission to abstain, produced it.
OpenAI's GPT-6.1 Sol repeats the shape with a different kind of number: a price. OpenAI's own language for Sol is accurate: "near-Astra intelligence for a fifth of the price," describing Sol against OpenAI's own top-tier model, GPT-6 Astra. Sol costs two US dollars per million input tokens and ten per million output tokens. That comparison is correct, and OpenAI is not the source of the error. The error appears one step later. A third party calls Sol "the cost-optimised model" inside a three-way comparison that also includes GPT-6 Luna, a separate model released the same week. Luna is priced at ten US cents per million input tokens and fifty cents per million output. Against Luna, Sol costs twenty times as much on both ends of the ledger. The label was never re-derived for the new comparison. It was carried over from one it fit, into one it did not.
Consider an enterprise architect and a vendor account manager comparing AI model options for a cost-sensitive internal tool. "Our GPT-6.1 Sol is the cost-optimised choice," the account manager opens. "It is a fifth of what our top-tier model costs. You would be mad to look at anything pricier." The architect has the pricing page open. "A fifth of Astra's price, sure. But you have Luna in the same family, and Luna is a twentieth of Sol's price. Cost-optimised compared to what? If the comparison is your whole lineup, Sol sits in the middle, not at the cheap end." Neither figure in that exchange is wrong. What is missing, every time this happens, is the one sentence naming which population or task a figure was actually measured against.
A United States federal court supplies the third instance. This one swaps a different kind of population: not a wrong set of questions, but a wrong class of system. In September, the Third Circuit ruled against ROSS Intelligence in its dispute with Thomson Reuters. It held that training a legal research tool on Thomson Reuters's own Westlaw headnotes was not fair use. Two independent legal commentaries, both read in full this week, agree closely on what the court actually decided, and on what it pointedly did not. ROSS's tool returned existing judicial passages rather than generating new text from what it learned. The opinion itself distinguishes that kind of system from the generative models at issue in other pending cases. The ruling says nothing about whether training a generative model on copyrighted text is fair use. Reported instead as a ruling on "AI training" in general, it quietly swaps a narrow, non-generative system for the much larger and still unresolved class of models. That is the class the case explicitly declined to address.
The fourth instance should sting most. It happened inside this project's own research a week earlier. Mistral released Mistral Large 4 with two different active-parameter figures attached: 49 billion and 52 billion. This project's own prior-week research logged the gap as a "counting-convention discrepancy," as though the two figures disagreed. They do not. Mistral's own documentation states the difference in one sentence: 49 billion sits in the active experts, and 52 billion also counts the embeddings and the output layers. The same split shows up in pricing. A standard list price sits beside a preview-period discount exactly half its value. Treating either alone as "the price" produces the identical error on a different axis. Two correct numbers, each counting something different, were read as a contradiction. Nobody asked what each one was counting before comparing them.
This project caught the same error in its own number that it catches in everyone else's.
Why this is not Subject Substitution
Episode 33 named a different failure Subject Substitution: two figures, each individually correct, each describing a different named subject, placed beside each other as though they described one thing. Claude Fable 5 and GPT-5.5 Pro were both real, both correctly measured. The error was comparing them as though one model's score told a reader anything about the other.
This week's four findings fail a different test, and conflating the two would leave a reader defenceless against exactly the cases that matter here. In every one of the four findings above, there is only one subject throughout. The Hassana Labs result is about Hassana Labs' own system, measured once. GPT-6.1 Sol's price is Sol's own price, quoted correctly every time. The Third Circuit's ruling is about one case, one decision. Mistral Large 4's parameter counts both describe Mistral Large 4. A reader taught only to check whether a comparison swaps the subject will wave all four of this week's findings through. None of them does that. What each one swaps instead is the reference class. For the rate, that is the population of questions it was measured against. For the price, it is the comparator the label was derived for. For the ruling, it is the class of system the holding covers. For the parameter count, it is the convention it follows.
Call this Scope Substitution: one figure, one subject, placed against a population or task it was never measured against. Subject Substitution asks whose number it is. Scope Substitution asks what population or task the number actually describes. V.E.R.A.'s own architecture separates a claim's existence from its predicate for a related reason. Whether a figure exists, and whether it appears in the stated paper, can be true in every instance above. Whether that figure's predicate holds for the comparison it has been placed into is a separate question. It is the one question Scope Substitution fails.
Where the boundary sits
Hassana Labs' Information Sufficiency Ratio is the strongest instrument of its kind this series has examined. It is a genuine information-theoretic gate: a pre-specified audit, a published threshold, a system that weighs the evidence available against the evidence a trustworthy answer would require, and refuses below that line rather than guessing. V.E.R.A.'s own verification shares a structural thesis with it: refuse rather than guess when the grounding is insufficient.
The equivalence is not. V.E.R.A.'s verification is ontological rather than information-theoretic. It asks whether a named entity exists and whether a claim's predicate holds of it, through NTP's separation of existence from predication. That holds independent of how much supporting text happens to sit in a context window. The Information Sufficiency Ratio asks a narrower, differently shaped question: whether retrieved evidence contains enough information, measured in bits, to answer one specific yes-or-no question reliably. Both approaches refuse rather than guess. They refuse for different reasons, through different machinery, and one is not an implementation of the other.
The distinction matters here specifically because the Hassana Labs finding is the easiest sentence this week's material offers: an AI system solved hallucination. It did not. The result is real and genuinely strong inside a bounded task. It is not evidence that hallucination in open-ended generation, with no pre-specified answer key and no abstention option, has been solved anywhere. Citing the shared thesis and collapsing the method into V.E.R.A.'s own would repeat the exact scope error this article spends its first half describing. It would be aimed at itself.
The closing example: a scorecard that checks itself
Meta released Muse Wonder and Muse Wonder Full on 12 October, alongside a fourteen-row scorecard. Most of those rows are Meta's own measurements, uncheckable in principle because no outside party ran them. One row is not: the Artificial Analysis Intelligence Index, an independently operated benchmark, which places Claude Opus 5.5 at 58, Muse Wonder Full at 56 and Muse Wonder at 52. On the one row anyone outside Meta can check, Meta's own models lose.
The scorecard's headline coding result is a different matter again. Meta's own model card credits Datacurve, a third-party operator, as the source for DeepSWE coding scores of 78.8, 79.0 and 74.2 for Wonder, Wonder Full and Opus 5.5 respectively. Datacurve's own public leaderboard, fetched directly this week, was last updated on 22 September, three weeks before any of the three models existed, and lists none of them. The gap is a checkability problem: the one row claiming independent corroboration cannot, this week, be checked against Datacurve's own published record.
The honest counterweight belongs in the same breath as the criticism, not after it. Meta's own model card discloses a result unfavourable to itself: on an internal benchmark called MemoryBench, the cheaper model edges out the more expensive one, 85.6 against 85.5. A scorecard built purely to flatter its subject rarely publishes the one line where the subject loses. That disclosure keeps the Datacurve attribution gap in the category of a checkability problem rather than a fabrication claim.
Put the two findings side by side. Thirteen of fourteen rows on this scorecard describe populations only Meta can see. The fourteenth describes a population anyone can check, and on that one, Meta's models are not the best in the field. Whatever a reader concludes about the other thirteen, the single externally visible data point deserves more weight, not less. A reader working from the headline alone would never learn that the scorecard contains both kinds of row at once.
Mistral caught its own scope problem before anyone compared its numbers: a model card stating in one sentence that forty nine billion sits in the active experts and fifty two billion also counts the embeddings and the output layers, published openly rather than left for a reader to infer from a press release. That is the open-source habit doing the job this article has spent several hundred words describing: publish the convention beside the number, and a reader never has to guess which population it describes. V.E.R.A.'s own codebase, licensed under the GNU General Public Licence version three, runs on the identical principle. Every predicate it evaluates and every existence check it runs sits in a public repository, so a reader checking a V.E.R.A. claim is checking the same convention V.E.R.A. itself used, not a second-hand account of it.
Put the identical question to a lethal autonomous weapons review and only the stakes change. A system is verified against a declared population of targets and a defined set of engagement conditions, never in general. The verification gap in lethal autonomous weapons systems is rarely a missing test. More often it is a tested result from one set of conditions read as though it covered a different one, Scope Substitution with a different cost of being wrong. The United States Department of Defense's Directive 3000.09 compliance verification process names this directly. Certification attaches to the system as reviewed, under the conditions reviewed, not to the system in the abstract. A capability demonstrated against one population carries no automatic claim to another, however the marketing reads. Treating a weapons review as a one-time gate rather than a scope-bound answer repeats this week's four findings, with a different population at stake.
Four different organisations, four different kinds of number, and in every case the number itself was correct. What went missing was the second question: which population, which task, which comparator, which convention produced it. That question costs one sentence. Asking it required no research methodology, no AI pricing expertise, no case law training and no special access to a frontier lab. It required only the habit of naming, before any two figures are placed side by side, what each one is actually a measurement of.
Think of the last benchmark, price comparison or legal update that landed in your feed with a source attached. Did you check what population or task it was actually measured against, or did the citation do enough work on its own? What happened when you asked?
If your board or legal team needs a technical verification of an AI system's runtime logic or retrieval pipeline, message me and I will send the scope and the fixed fee.
• • •
The views expressed in this article are entirely my own, informed by more than 30 years of professional experience in architecture, security, and technology leadership in New Zealand. I write as director of Te Pono Limited; the views are personal and do not represent the position of any client, any government agency, or the New Zealand government. My commentary on legislation and policy is analytical, drawing on publicly available sources and my professional expertise in architecture, security, and AI governance, and it is politically neutral.
• • •
Andreas Hamberger is a New Zealand leader in Architecture & Security and Associate Member of the Institute of Directors. V.E.R.A. (Verified Existence & Reason Architecture) is an open-source logic engine available on GitHub. Through Te Pono he provides technical verification of AI runtime logic and retrieval pipelines for boards and legal teams; contact andreas@thehambergerreport.com for the scope and fixed fee.
This article was produced with AI assistance under my direction. Research, drafting and images pass through a pipeline I built and govern: automated gates for source verification, forbidden language and political neutrality, and my own review before anything is published. The tools include Claude, Gemini and Openart. The frameworks, arguments and editorial judgements are mine and are the same discipline I apply to the AI systems I audit for clients. AI accelerated the work; the thinking, and the responsibility for it, are mine.
[1] Hassana Labs. "Predictable Compression Failures: Order Sensitivity and Information Budgeting for Evidence-Grounded Binary Adjudication." 2026. https://hassana.io/
[2] Chlon, L., Karim, A., Chlon, M., and Awada, M. "Predictable Compression Failures: Order Sensitivity and Information Budgeting for Evidence-Grounded Binary Adjudication." arXiv, 2026. https://arxiv.org/abs/2509.11208
[3] OpenAI. "Introducing GPT-6.1 Sol." 29 September 2026. https://openai.com/index/introducing-gpt-6-1-sol/
[4] OpenAI. "GPT-6 Luna." Model documentation. 22 September 2026. https://developers.openai.com/api/docs/models/gpt-6-luna
[5] Inside Tech Law (Fenwick & West). "Third Circuit Rejects Fair Use Defense in Westlaw AI Training Dispute." October 2026. https://www.insidetechlaw.com/blog/2026/10/third-circuit-rejects-fair-use-defense-in-westlaw-ai-training-dispute
[6] Ballard Spahr. "Third Circuit Addresses Fair Use in AI Training But Leaves Generative AI Questions Unresolved." October 2026. https://www.ballardspahr.com/insights/alerts-and-articles/2026/10/third-circuit-addresses-fair-use-in-ai-training-but-leaves-generative-ai-questions-unresolved
[7] Mistral AI. "Mistral Large 4." Model documentation. 2026. https://docs.mistral.ai/models/mistral-large-4
[8] Datacurve. "DeepSWE Leaderboard." Accessed 15 October 2026. https://deepswe.datacurve.ai/
[9] Artificial Analysis. "Intelligence Index." Accessed 15 October 2026. https://artificialanalysis.ai/
[10] Meta AI. "Muse Wonder Model Card." 12 October 2026. https://research.meta.ai/static/muse-wonder-model-card