AI Verification: We Opened the Primary and Still Got It Wrong

This week's research pass did exactly what five weeks of this series have been asking of it. It opened the primary text of New York's new frontier AI law instead of trusting a summary. It read the definition of a covered "large developer" word for word. It found no revenue figure anywhere in it. The five hundred million dollar threshold circulating in commentary belongs to California's SB 53, not New York's RAISE Act. Press coverage had fused the two states' laws into one. Every word quoted was genuinely in the document. It was not the law now in force.

A verbatim quotation from a real primary source can still be the wrong edition of that source. New York's Responsible AI Safety and Education Act was signed in one form and amended into another. A research process built to distrust secondhand paraphrase has no particular reason to distrust the vintage of the primary document it opens for itself.

The document was real; it was the wrong edition

The as-passed RAISE Act defines a covered large developer as a person that has trained at least one frontier model and spent more than one hundred million dollars in aggregate compute costs training frontier models. A frontier model itself is defined by a compute threshold above ten to the twenty-sixth power of computational operations. No revenue figure appears anywhere in that text. A pass that opens the bill, reads that definition and reports it accurately has done its job on the document in front of it.

The trouble is the document in front of it is not the one New York now enforces. A later chapter amendment, taking effect from the first of January 2027, replaces the compute-spend test with an affiliated annual gross revenue test. A large frontier developer is now one that, with affiliates included, reported more than five hundred million dollars in annual gross revenue the prior year. Both thresholds are real. Only the revenue test is in force. The hundred million dollar compute figure the pass quoted correctly is the retired, as-passed vintage of a law since rewritten to track California's.

The pass's own rigour is what let it go wrong. A legislature's website does not serve the bill and the consolidated statute from the same place, and the bill, not the amended law, is the better indexed artefact. It has one stable page and one fixed date, and a search engine treats it as the canonical record of what the law says. The amendment sits elsewhere, under a separate chapter number, serving a smaller audience and carrying less link weight. A process that opens the primary source rather than trusting a summary will reliably find the primary source that is easiest to find. The easiest one to find is not guaranteed to be the one still standing. Distrust of secondhand paraphrase and distrust of document vintage are two different disciplines, and this series has only been practising the first one.

This matters beyond one New York statute. Non-Traditional Predication Theory, the formal foundation this project runs on, classifies a claim before it checks it: an existence or occurrence claim, an e-type claim, needs a traceable external referent before it is accepted. The RAISE Act's revenue threshold is exactly that kind of claim, and the pass's citation was traceable, specific and genuinely sourced. What the claim-type separation does not currently ask is which edition that traceable referent is. A citation can be real, checkable and still answer a question the law no longer poses, and nothing in the architecture as built today flags that gap on its own.

Reported, not resolved, and a second instance while checking the first

Two primary sources confirm the amended figure independently. A law firm's own published analysis of the amendment states the operative test in close agreement with the statute. Large frontier developers are those that, together with their affiliates, had annual gross revenues exceeding five hundred million dollars. A policy research organisation's own page, read separately, states the same figure. It adds that the original version was defined by annual compute spending above one hundred million dollars, matching what the pass found in the earlier text. Both documents also agree the amendment was signed in March, replacing a bill signed the preceding December. Exactly how a December signing and a March signing relate, as one amended instrument or as two distinct legislative acts under New York's chapter-amendment practice, is not established by either page on its own. Neither is it resolved here.

Staggered dates inside one instrument are not unusual, and reading the instrument's general date instead of the specific provision's own is an easy way to misread any of them. The European Union's AI Act makes the same point at larger scale. Article 113, its last operative article, quoted in The Hallucination Economy, declares that the Regulation "shall apply from 2 August 2026," while naming earlier dates for particular chapters in the same breath. A reader who cites "the AI Act's compliance date" as a single fact is making the identical category error this week's pass made with the RAISE Act. Both treat a multi-dated instrument as though it carried one vintage throughout.

The second instance turned up while checking the first, inside this project's own supporting material for this episode. The penalty figures this project had been carrying for the RAISE Act were the retired ones: ten million dollars for a first violation and thirty million for a subsequent one. Both sources above agree the amendment cut those figures to one million and three million. The hundred-million-to-ten-million pairing and the five-hundred-million-to-one-million pairing are the same chapter amendment, read correctly in one place and carried forward uncorrected in another. That happened inside one project, about one statute, in the same week it set out to demonstrate exactly this failure. It does not earn a second entry in a taxonomy that already has one legislative instance. The same chapter amendment caused both errors.

The honest limit, twice over

A dedicated pass over this week's own candidate terms, the kind of check this series has recommended since it started, returned nothing on the RAISE Act. The pass catches a re-arrival, a claim this project has already logged correctly before. The RAISE Act's version error was never logged at all, so there was nothing for the pass to catch it against. A check built to catch a stale memory cannot catch an error that was never in memory in the first place.

The same pass did catch something else this week, and the catch is instructive precisely because it looks, at first glance, like continuity rather than change. Mistral's newest flagship model shipped under a new licence, described consistently across independent sources as a modified permissive licence rather than a fully open one. A separate commercial licence is now required for anyone running it as a hosted service. The full weights were published on the twenty-ninth of October. On its own, a new model with a new licence name reads as routine. Set against this project's own held pattern, that each of Mistral's three prior flagship releases shipped under the Apache 2.0 licence, it reads as a flagship model leaving an open-source licence for the first time. The term pass supplied the comparison; nothing about the new announcement itself would have surfaced it.

A second, smaller correction ran the other way. This week's own first pass had down-rated a widely reported satellite-compute figure because it appeared, as far as that pass could tell, only in aggregator coverage rather than at a primary source. The dedicated term pass found that this project's own index had already held the figure since the first of October, sourced to the manufacturer's own mission page. That is better provenance than the figure had been credited with. This ordering discipline accumulates a record of what each new claim should be checked against before anyone decides how much to trust it.

Not every error this week came from a document. Two came from the layer that summarises documents before anyone reads them. A search summary this week attributed a desktop environment update to version "6.8.1"; the release announcement it was drawn from says "6.8," full stop, nothing after the decimal. One invented character, caught only because the source was opened in the same session that used the summary. A second search summary offered a specific subsection of a federal cybersecurity standard that does not exist in that document at all. The real content sits several sections earlier, under a different heading entirely.

The shape is the same in both cases: the summary was more specific than its own source, inventing a precision the document never claimed. It matters more than an equivalent error inside a dated article. A dated document has an author and stable text, so a mistake made about it can be corrected at its source and will not recur in the same form. A search summary has neither. An invented point release or a phantom subsection is not corrected anywhere; it is simply generated again, slightly differently, the next time someone asks. The fix is specific and cheap: when a summary gives a version number or a section number, open the document it names before repeating the number.

The ledger, one line each

Nothing this week asks V.E.R.A. to verify its own reasoning. Nor is anything in New York's legislature, the two firms whose analyses confirmed the amended figures, Mistral, the kernel project or the standards body shown to have misrepresented its own work. The section number it invented does not change that. Every correction here traces to a research process reading one edition of a document, or one summary of a document, rather than another.

This week's own research for this piece followed the discipline it is reporting on: the brief was read in full before a single search query went out. That is exactly how the penalty-figure error inside this project's own material got found. It was not an accident of needing to open a file for some other reason, but because the order was followed deliberately. It does not guarantee the next version error will be caught. It makes catching one before publication more likely.

Open source solved a version of this problem decades ago with semantic versioning and a published changelog. A dependency is pinned to an exact tagged release, not a floating branch. A consumer always knows precisely which revision's behaviour it is reading about, and can see, in one file, what changed between editions. A statute with no changelog and two dates in circulation is the same failure mode without the tooling. My own V.E.R.A. codebase, licensed under the GNU General Public License version 3 on GitHub, ships under that convention. Every release is tagged, every change is logged, and nobody verifying a claim against it has to guess which commit produced which answer. The same discipline, read into a weapons review instead of a dependency file, still holds.

The same discipline carries into weapons review. A lethal autonomous weapons system does not clear a verification gap once and stay cleared. The gap this series has named for six weeks is usually a tested result read as though it still described the system to a reviewer, not a missing test. Lethal autonomous weapons systems, LAWS, verification gaps exist because the standard a weapon was checked against can go out of force as a statute can. The configuration or operating envelope changes, and the clearance on file stops describing the fielded system. The North Atlantic Treaty Organisation's 2021 Principles of Responsible Use of Artificial Intelligence in Defence name reliability and traceability as conditions a system must meet. They are not properties a manufacturer asserts once and carries forward. The sovereignty implication is this week's statute: a review is only as current as the edition it was run against.

If a verification architecture is worth building at all, it has to survive discovering that its own operators made the mistake it exists to catch. What changed this week in your own reading or reviewing, once you went looking for the current edition instead of the one that was easiest to find?

This is one of seven weekly series in The Hamberger Report. Subscribe on LinkedIn and the next one arrives in your feed.


The views expressed in this article are entirely my own, informed by more than 30 years of professional experience in architecture, security, and technology leadership in New Zealand. I write as director of Te Pono Limited; the views are personal and do not represent the position of any client, any government agency, or the New Zealand government. My commentary on legislation and policy is analytical, drawing on publicly available sources and my professional expertise in architecture, security, and AI governance, and it is politically neutral.


Andreas Hamberger is a New Zealand leader in Architecture & Security and Associate Member of the Institute of Directors. V.E.R.A. (Verified Existence & Reason Architecture) is an open-source logic engine available on GitHub. Through Te Pono he provides technical verification of AI runtime logic and retrieval pipelines for boards and legal teams; contact andreas@thehambergerreport.com for the scope and fixed fee.


This article was produced with AI assistance under my direction. Research, drafting and images pass through a pipeline I built and govern: automated gates for source verification, forbidden language and political neutrality, and my own review before anything is published. The tools include Claude, Gemini and Openart. The frameworks, arguments and editorial judgements are mine and are the same discipline I apply to the AI systems I audit for clients. AI accelerated the work; the thinking, and the responsibility for it, are mine.


[1] Cooley LLP. "New York's Frontier AI Law Gets a California Makeover, With Some Key Differences." 31 March 2026. https://www.cooley.com/news/insight/2026/2026-03-31-new-yorks-frontier-ai-law-gets-a-california-makeover-with-some-key-differences

[2] Future of Privacy Forum. "The RAISE Act vs. SB 53: A Tale of Two Frontier AI Laws." 2026. https://fpf.org/blog/the-raise-act-vs-sb-53-a-tale-of-two-frontier-ai-laws/

[3] New York State Senate. "Senate Bill S8828, 2025-2026 Legislative Session." https://www.nysenate.gov/legislation/bills/2025/S8828

[4] Mistral AI. "Mistral Large 4, Open Weights." 29 October 2026. https://mistral.ai/news/mistral-large-4-oss/

[5] KDE Community. "Plasma 6.8." https://kde.org/announcements/plasma/6/6.8.0/

[6] National Institute of Standards and Technology. "Special Publication 800-207, Zero Trust Architecture." https://csrc.nist.gov/pubs/sp/800/207/final

[7] Linux Kernel Mailing List archive. "docs: remove Assisted-by references." https://lkml.iu.edu/2610.1/11683.html

[8] National Cyber Security Centre NZ. "CVE-2026-111497 Affecting Brotli Decoder." 29 October 2026. https://www.ncsc.govt.nz/alerts/cve-2026-111497-affecting-brotli-decoder/

Next
Next

AI Verification: The Ordering Worked