AI Verification: The Ordering Worked

Eight days ago, this series set itself a test it could not pass by accident. A short entry in this project's own research index made a plain observation: a web search answers one question, is this true, and cannot answer a different one, is this already ours. It set a condition for the next stage of the discipline. Run a search of the project's own prior work before running a search of the open web. Do it deliberately, not because the file happened to be open for some other reason. Then see whether that cheap, bounded pass catches something the expensive, unbounded pass would also have confirmed as true.

This week it did.

Before a single web query went out this week, a pass over roughly seventy candidate terms, drawn from every source feeding this week's research, searched the project's own index first. It came back with four hits: a set of figures reported once before, a mislabelled source, a corrected trio of numbers, and a claim already refused once and offered again anyway. Two of the four would have come back true from an open web search and read exactly like findings. They are repeats, caught before they could be filed as new. This series has a vocabulary for exactly this distinction. A check that is cheap, bounded, and runs first is a Heaven Vector property. A Skynet Vector's check is expensive, unbounded, and runs late.

This piece publishes on New Zealand's general election day. It carries no New Zealand material this week. That is the ordinary turn of the weekly research rota, not a deliberate choice to avoid the date. Every source below sits in the United States, Australia, or an international standards and analyst community.

The ordering, run on purpose

I think of two analysts at a research consultancy, briefing two different clients a month apart. The junior one finds a strong figure, checks it against the source this morning, confirms it is accurate, and is ready to add it to the slide. The senior one asks a single question before it goes in: have we used this before? A short search later, it turns out the answer is yes. It is the same study, re-found rather than found. Nothing about the check was wrong. The order was wrong. Checking the firm's own library first, then confirming against the source, costs thirty seconds. Those thirty seconds stop last month's work from re-appearing as this month's finding.

The clearest instance this week is Pew Research Center's own figures on American attitudes to artificial intelligence. Fifty-two per cent of American adults are now more concerned than excited about AI, up from 37 per cent when Pew last asked in 2021. Fifty-five per cent of adults under 30 feel the same, a majority for the first time on this question. Seventy-one per cent of all adults expect AI to reduce jobs in the United States over the next twenty years, up from 64 per cent two years earlier. Every one of those numbers checks out against Pew's own short-read article, published 18 August 2026. None of it is new. It was already sitting in this project's own index, catalogued weeks ago. The search this week found it there before a web query could confirm it a second time and call it a discovery.

The entry that first recorded the Pew figures states that the fieldwork ran on 18 August 2026. It did not. Pew's own methodology note, read directly this week, puts fieldwork at 22 to 28 June 2026. 18 August is the date the article itself was published, not the date anyone answered a survey question. An earlier, more careful version of the same entry had it right, naming the date as the publication date and nothing more. Somewhere between that version and this one, a correction written in the act of correcting something else introduced a smaller version of the same mistake it was fixing. This discipline has to be turned on this project's own corrections as well as on claims arriving from outside it.

The quietest catch of the four might be the cheapest. A claim about Australian cyber-insurance premiums was refused once before, because nobody could trace it to the study it claimed to come from. This week it turned up again, attached to the same study. It is refused again, for the same reason, at no further cost.

The honest limit

This method also has a limit, worth stating in the same breath as the catch above it. The same week that caught four repeats also produced three of its largest corrections, from material that had never been in the index before. A search against the index could not have caught any of them. There was nothing yet to match them against.

A United States federal standards body's draft guidance on operational technology security is one of them. The draft, released 21 September 2026, recommends applying zero trust principles to systems capable of supporting it. It specifies those as systems at the upper tiers of a long-established industrial control hierarchy. Many of the controllers and operator panels sitting below that line cannot take part. A sweep run this week read that scoped recommendation as a universal mandate covering every request on an operational technology network. Two independent analyses of the draft, read directly this week, state the scoping line explicitly and state nothing resembling a universal mandate. The same fact turned up elsewhere in this week's production, for a different series on a different day. A scoped recommendation stays scoped, wherever it turns up.

A global open-source industry body's own survey of contributors and maintainers is a second one. Its press release, published 20 October 2026 and read directly this week, states that an estimated quarter of incoming pull requests on the projects surveyed are low-quality, machine-generated contributions. That rises to roughly a third on larger projects. A sweep this week promoted that estimate into an unqualified figure for how much open-source code is now AI-written. That is a different, much larger claim than the one the survey supports. The release itself does not state the respondent count behind that quarter. Two further figures, which a prior entry in this project's own research attaches to the same survey, do not appear in the release either. That gap is disclosed rather than resolved: the fuller report behind the release may well carry those figures. A press release that skips a number from its own underlying document does not make the number wrong. It only means this week's check could not confirm it from that page alone.

None of this traces to carelessness on the part of the people who ran these studies. Every correction and every gap this week traces back to a restatement inside this project's own prior research, or to a landing page that does not carry every detail a fuller report might. None traces to the conduct of the organisation that did the work.

Scope is a choice, twice over

A second kind of fault closed a two-cycle question this week, and it widened rather than repeated what the prior week had already shown. A figure can be sourced correctly and dated correctly and still measure the wrong population, the wrong configuration, or the wrong comparison set. That fault now has four fresh instances, none of them a date problem.

The sharpest of the four concerns a well-known analyst firm's annual ranking of enterprise architecture tools. This week's own sweep read two named vendors' marketing of their own Leader placements as evidence of a market-wide shift, built on the strength of those two companies' own announcements. Checked directly this week against all four vendors publicly named as Leaders in the same ranking, the picture looks different. One of the four is marking a sixth consecutive year as a Leader, one a fifth, and one nineteen consecutive years. Naming two Leaders out of at least four and calling it a definitive market transformation is a comparator choice dressed up as a finding. Naming all four, including a vendor with nearly two decades in the category, tells a steadier story: most of the field already has the capability.

The same shape turns up twice more. A health-privacy regulator's own statistic about a rise in large-scale data breaches states, in its own words, that the affected population runs into the millions globally. A secondary restatement drops that one word. The number then reads as a national figure, which it is not. The same draft standards document turns up again here, from a different angle. The scoping question is not only which systems the recommendation applies to, but whether the reader of the headline knows that boundary exists at all.

The date check keeps earning its keep

A simpler version of the same discipline checks that a cited event actually happened inside the week it describes. It keeps finding problems. A regulatory inquiry presented as a development of this week actually opened four weeks earlier. A legal filing and its first broad press coverage both predate the window a sweep claimed them for. Two incidents involving the same company's AI systems ran on two different model versions, a few days apart. Only the later disclosure falls inside the window being described. A widely cited AI-spending forecast is three months old. It describes a different, much narrower slice of spending than the figure it is often compared against. A national cyber threat report has now carried the wrong release date for a fourth consecutive research cycle. The error changes from cycle to cycle. The correct date has sat the entire time in the name of the file on the publisher's own website.

The closing example, and its own counterweight

An annual American survey of public fears, released 20 October 2026, checks out in full this week. Artificial intelligence harming the political process now ranks third among the fears respondents named. That is just behind corrupt government officials, the country's top fear for an eleventh consecutive year running. It ranks just ahead of fear of the United States entering another major war, which jumped from sixth place a year ago to second this year. Sit those three next to each other and the AI figure reads less like a standalone alarm. It looks like one entry in a crowded list of things people are currently worried about.

The same survey carries its own counterweight, and it runs the opposite way to almost everything else in this week's sourcing. Broken down by how often someone actually uses AI, 46 per cent of weekly users fear it could harm the political process. That is against 55 per cent of people who have never used it at all. The people closest to the tool are less worried about it than the people furthest from it, on this one question, in this one survey.

A separate study of AI hallucination rates, testing more than four thousand prompts across seven AI systems, checks out in full against its own primary page this week too. That includes its own description of itself as a search engine optimisation and marketing firm. Read its single-vendor figures with caution, not as a settled industry benchmark. Searching for that same study surfaces, above the study itself in the search results, an aggregator page claiming an unrelated and much larger improvement figure under the same rounded percentage. That is the same attribution problem this project has caught from three separate directions. The hallucination-measurement literature has no agreed definition, no agreed denominator, and its most easily found surfaces are often aggregators rather than the people who did the measuring.

This is the exact problem V.E.R.A.'s existence-first verification is built to address: a measurement is only as good as agreement on what is being counted. It is not evidence that any outside tool has been checked against V.E.R.A., tested by it, or found wanting by it. Nothing this week claims otherwise. No external party marketed an instrument in V.E.R.A.'s own vocabulary this cycle. That is itself unremarkable: most weeks will not produce one.

Before you cite it

The practical version of all of this fits on one card. Search your own record before you search the world. A true thing you already have is an old thing, no matter how clean the new confirmation feels. Check the boundary before you repeat it. A scoped recommendation, a subset of respondents, a named comparator, and a qualifying word like globally are all part of what the number actually says. Then, and only then, verify against a primary source. The ordering above cannot catch what has never been written down before. That is a real limit.

As I wrote in The Hallucination Economy, "I searched part of my own earlier written material for prior statements of those four, and I did not find them. That is a floor and not a clearance, and I would rather publish the floor than round it up." The same principle, run the other way, means a floor you have already cleared is worth finding before you clear it again.

The open-source dimension of this discipline is the most familiar one in the room. Any large open-source project runs on exactly this ordering by convention, long before it had a name. A maintainer's own contribution guide asks anyone filing an issue to search existing issues first. A bug reported for the third time under a different title wastes review time that a five-second search would have saved. V.E.R.A.'s own codebase, released under the GNU General Public Licence and held on GitHub, follows the same discipline its own contribution process asks of anyone filing an issue against it: search first, file second. The practice is older than this project, and larger than it. That is why it travels so well into a different domain.

The implication on the sovereignty side runs on the same logic, one level up. A verification regime for autonomous weapons systems has to answer a structurally identical question before the one everybody expects: is this a new configuration, or an old one re-presented as new? The United States Department of Defense's Directive 3000.09 requires senior review before an autonomous or semi-autonomous weapon system can be fielded. Much of that directive's own text is a scoping exercise. It defines which systems and which changes trigger a fresh review, rather than falling under one already completed. The United Kingdom's Ministry of Defence runs a comparable discipline under JSP 936, its governance framework for autonomous systems. Both exist because a defence procurement process that cannot tell a genuinely new system from a familiar one re-badged ends up re-litigating decisions already made, at a cost nobody budgeted for.

The result this week is narrow. An ordering set as a test eight days ago was run this time on purpose, not by the accident of a file being open for another reason. It caught four repeats before a single web query confirmed any of them as true. The same discipline, turned immediately afterward on this project's own prior correction of one of those four, caught a further, smaller error sitting inside the correction itself. That is the stronger catch of the two. This method had only ever worked on other people's claims before; now it also works on its own output, which is closer to a habit than a filter.

Where in your own work does checking your own record first, before checking the world, actually save you the most time, and where have you been skipping that step without noticing?

If your board or legal team needs a technical verification of an AI system's runtime logic or retrieval pipeline, message me and I will send the scope and the fixed fee.

• • •

The views expressed in this article are entirely my own, informed by more than 30 years of professional experience in architecture, security, and technology leadership in New Zealand. I write as director of Te Pono Limited; the views are personal and do not represent the position of any client, any government agency, or the New Zealand government. My commentary on legislation and policy is analytical, drawing on publicly available sources and my professional expertise in architecture, security, and AI governance, and it is politically neutral.

• • •

Andreas Hamberger is a New Zealand leader in Architecture & Security and Associate Member of the Institute of Directors. V.E.R.A. (Verified Existence & Reason Architecture) is an open-source logic engine available on GitHub. Through Te Pono he provides technical verification of AI runtime logic and retrieval pipelines for boards and legal teams; contact andreas@thehambergerreport.com for the scope and fixed fee.


This article was produced with AI assistance under my direction. Research, drafting and images pass through a pipeline I built and govern: automated gates for source verification, forbidden language and political neutrality, and my own review before anything is published. The tools include Claude, Gemini and Openart. The frameworks, arguments and editorial judgements are mine and are the same discipline I apply to the AI systems I audit for clients. AI accelerated the work; the thinking, and the responsibility for it, are mine.


[1] Pew Research Center. "Young adults in the US are increasingly wary of AI, concerned it will take jobs." 18 August 2026. https://www.pewresearch.org/short-reads/2026/08/18/young-adults-in-the-us-are-increasingly-wary-of-ai-concerned-it-will-take-jobs/

[2] The Linux Foundation. "New LF Research Survey Shows Generative AI Cuts Open Source Coding Time in Half, But 25% of Incoming Pull Requests Are Considered 'AI Slop'." 20 October 2026. https://www.linuxfoundation.org/press/new-lf-research-survey-shows-generative-ai-cuts-open-source-coding-time-in-half-but-25-of-incoming-pull-requests-are-considered-ai-slop

[3] National Institute of Standards and Technology. "Guide to Operational Technology (OT) Security, SP 800-82 Revision 4 (Initial Public Draft)." 21 September 2026. https://doi.org/10.6028/NIST.SP.800-82r4.ipd

[4] Zscaler. "NIST's New OT Guide Draws the Zero Trust Line at Level 3. Attackers Are Working Below It." 2026. https://www.zscaler.com/blogs/product-insights/nist-s-new-ot-guide-draws-zero-trust-line-level-3-attackers-are-working

[5] Claroty. "The Global State of Operational Security 2026." 2026. https://claroty.com/resources/reports/the-global-state-of-operational-security-2026

[6] Chapman University, 2026 Survey of American Fears, via EurekAlert. "2026 American Fears Survey." 20 October 2026. https://www.eurekalert.org/news-releases/1148421

[7] First Page Sage. "AI Hallucination Statistics." 2026. https://firstpagesage.com/seo-blog/ai-hallucination-statistics/

[8] Ardoq. "Leader, Gartner Magic Quadrant for EA Tools 2026." 2026. https://www.ardoq.com/blog/leader-gartner-magic-quadrant-ea-2026

[9] Office of the Australian Information Commissioner data, via Insurance Business Australia. "Mega-breaches nearly triple as health sector tops Australia's data breach count." 2026. https://www.insurancebusinessmag.com/au/news/cyber/mega-breaches-nearly-triple-as-health-sector-tops-australias-data-breach-count-594893.aspx

[10] Hamberger, Andreas. The Hallucination Economy. Te Pono Limited, 2 October 2026. https://hamberger.short.gy/hallucination-economy

[11] Hamberger, Andreas. V.E.R.A. documentation. Te Pono Limited, GitHub, January 2026. (No public URL captured for the documentation itself; the codebase is referenced by component and architecture name, consistent with this project's own prior-episode practice.)

Next
Next

AI Verification: We Verified It, and We Already Had It