Verification Becomes a Scaling Axis: The Week the Judge Got a Probability Distribution
On RoboRewardBench, a benchmark that grades how well an AI system judges robot behaviour, two graders looked at the same work. One scored 70.8 per cent. The other scored 87.4 per cent. Same task. Same benchmark. The same family of models available to both. The distance between them was not a smarter model or more training. It was a single design choice about what a grader is allowed to say.
The lower score came from the standard tool for the job: an AI model asked to read a candidate answer and give it a mark on a discrete scale, one to five. The higher score came from a method that refuses to pick a single mark at all. That refusal, and what it makes possible, is the subject of a preprint posted on 6 July 2026 by researchers at Stanford, UC Berkeley and NVIDIA Research. It is an unreviewed preprint, not a peer-reviewed study, and it is worth reading carefully rather than at face value.
The claim underneath the numbers is larger than any one benchmark. It is that verification, the act of checking work an AI has already produced, has quietly become something you can scale, the way training a model is something you can scale. That is worth taking apart slowly, because it is half right in the half that matters.
The bottleneck nobody named
Start with the tool being displaced. The standard way to grade open-ended AI work is called LLM-as-a-Judge, a large language model asked to score another model's output. It has been the default since 2023, when the method was introduced alongside the MT-Bench and Chatbot Arena evaluations. Three years is a long life for a single evaluation habit in a field that rewrites itself every quarter.
The habit has a flaw the new work names directly. A discrete one-to-five judge runs out of room. Asked to compare two candidate solutions, it frequently gives them the same score, a tie, because both look like a four and the scale has no finer setting. In one case study reported in the paper, a discrete judge tied two different candidates in 88 of 100 comparisons. Across the broader Terminal-Bench evaluation, a tie rate of 27 per cent is reported. Those are two different measurements at two different scopes, one narrow case study and one benchmark-wide figure, and they should not be blurred into a single number. Both point the same way. A tie is not a verdict. It is the grader declining to discriminate, dressed up as an answer.
The proposed fix does not ask the model to choose a score token at all. It reads the full probability the model places across every possible score, a lot on four, less on three, a little on two, and computes the mathematical expectation: a single continuous number instead of a single discrete guess. Run on the same case study, that approach produced no ties.
Three ways to scale a verdict
Here is where the paper earns its title. Once a score is continuous rather than discrete, you can push on it. The authors describe three distinct ways to spend more effort on verification and get a better result, and they measured each on Terminal-Bench V2.
The first is score granularity. Read the model's belief across more possible score tokens, from one to twenty, and accuracy rose from 73.1 per cent to 77.5 per cent. Finer tokens, the authors write, give the decoder a finer space to project the model's belief. More resolution, more discrimination.
The second is repeated evaluation. Grade the same candidate several times independently and average the results. Variance falls at a rate proportional to one over the number of samples, and accuracy improved from 74.7 per cent at a single evaluation to roughly 77.4 to 77.5 per cent at sixteen. Two readings of that endpoint figure disagreed at the first decimal, so the honest statement is a range, not a false precision.
The third is criteria decomposition. Instead of asking one all-in question, is this solution correct, split the judgement into named sub-criteria, does it meet the specification, does the output behave, are there introduced errors, score each separately, and combine them. Any single sub-criterion scored between 75.2 and 76.4 per cent; the combined ensemble reached 78.3 per cent. Decomposing the rubric, the authors note, reduces the bias that comes from asking for one all-in verdict.
Granularity, repetition, decomposition. Three levers, all pointing the same way: verification is no longer a fixed act you perform once. It is an effort you can dial up, the way you dial up training compute or inference-time thinking. The authors put it plainly: verification is a fourth scaling axis, alongside pre-training, post-training, and test-time compute for generation. This is scaling applied to grading, not to producing, and it reaches state-of-the-art results across four unrelated domains, software terminals, code repositories, robotics, and medical agents, with a method that needs no additional training.
Not the only ones
A single research group reporting its own preprint results is a reason for interest, not yet for confidence. This paper has not been independently replicated; the secondary coverage that exists restates the authors' own figures rather than testing them, which is the limit any fresh preprint carries. So the strongest reason to take the scaling-axis claim seriously is not this paper at all.
It is a second one. A separately authored, peer-reviewed paper published in the Findings of the Association for Computational Linguistics in 2026, by Yuxuan Wan, Tianqing Fang and colleagues, argues independently that verification is an inference-time scaling dimension for AI agents. Its mechanism is different: rubric-guided iterative self-verification, which the authors call DeepVerifier, rather than reading a probability distribution over score tokens. Its problem is different: improving deep-research agents rather than grading general agentic work. Its venue is different: peer review, not an unreviewed preprint. And it reports its own numbers, F1 improvements of 12 to 48 per cent and accuracy gains of 8 to 11 per cent over baseline judges, again without additional training.
Two groups, two methods, two problems, one structural conclusion, reached within the same month. That is the shape of a real finding. It does not confirm any specific benchmark figure from the first paper, and the two systems are separate and must not be run together. What it confirms is the direction: verification effort is something you can scale, and scaling it measurably helps.
What V.E.R.A. already refused to do
Which brings the question home to this series. V.E.R.A., the Verified Existence and Reason Architecture, is an open-source logic engine built on Non-Traditional Predication Theory, Professor Horst Wessel's 1992 separation of existence from predication. It also refuses the single undivided judgement. Its first layer, the Krampitz Load Analyser, takes an AI output and breaks it into individual claims before any checking begins, sorting each by the kind of verification it needs. That is a decompose-before-verify move, structurally the same instinct as the paper's criteria decomposition.
The instinct is shared. The direction is opposite, and the difference is the whole point.
LLM-as-a-Verifier decomposes in order to add probability: finer tokens, more samples, more sub-scores, all combining into a continuous confidence estimate. V.E.R.A.'s E! Verification Service, for the class of claims it handles, removes probability entirely. Given an entity named in a claim, it queries an external corpus seeded from Wikipedia and Wikidata and returns one of three categorical verdicts: EXISTS, does not EXIST, or UNKNOWN. No confidence is attached. An entity does not 73 per cent exist. It is a lookup, not a score.
Both moves are correct, because they answer different kinds of question. LLM-as-a-Verifier grades open-ended solution quality, where there is no single right answer and a graduated score is the honest output. The E! Service checks whether a named thing is real, where a graduated score would be a category error. This matters enough to state plainly: neither paper validates V.E.R.A., and V.E.R.A. does not achieve the paper's benchmark scores. The systems work in different domains on different questions, and only the structural principle, do not verify a compound judgement as one undifferentiated block, connects them. The logic runs the other way too: V.E.R.A. does not verify its own reasoning. And this V.E.R.A., Andreas Hamberger's engine on GitHub, is not the several other systems that share the acronym.
What scaling verification does not buy you
A verifier you can scale is not the same as a verifier that is finished, and the paper is honest about this. Its authors list their own limits. The criteria decomposition is hand-designed for each domain, not learned. The repeated-evaluation gains come from spending compute uniformly, not from putting it where it is needed. And the method needs access to the model's scoring-token logits, the raw probabilities behind each score, which several frontier models behind restricted interfaces simply do not expose. The verifier models the authors used, Gemini 2.5 Flash and Qwen 3.6 35B, sit well below the current frontier, which is the interesting part: the claim is not that you need a bigger judge, but that a smaller one, asked the right way, can out-discriminate a one-shot verdict from a larger model.
Then there is the shape of the gains. Twenty times the score granularity moved accuracy from 73.1 to 77.5 per cent. Real, and worth having, and firmly bounded. The repeated-evaluation curve flattens on the same one-over-the-samples law that governs any averaging. Scaling verification buys genuine discrimination, and it buys that with diminishing returns. There is no setting on the dial that turns a grader into an oracle.
The implication
For anyone running a review gate, and increasingly that means anyone shipping AI-generated work, the practical question stops being only what error rate is tolerable and becomes how much verification effort is worth spending to approach it. That is a design decision, and it now has a research literature behind it.
It is also a governance gap sitting in plain sight. New Zealand's one central instrument here, the Government Chief Digital Office's Public Service AI Framework, names AI evaluation as an assurance activity but specifies no method for how that evaluation should be done, and it remains non-binding guidance last updated in January 2025, roughly eighteen months before this was written. That is a description of the published guidance as it stands, not a call for anyone to mandate a particular technique.
The same shape appears at larger scale. The European Union's AI Act, in Article 15, requires high-risk systems to reach an appropriate level of accuracy and to be tested against it, without saying what that testing must involve. The Reserve Bank of New Zealand's Operational Resilience Standard, arriving in 2027, will require boards to set quantified tolerance levels for AI-related failure. This week's finding gives that duty a sharper edge: not only what error rate is tolerable, but how much verification compute is worth spending to reach it, given that the returns diminish. Public unease about automated decisions is already on record, and local security guidance already insists that controls work under pressure; neither says how hard a control must look before its result can be trusted.
The open-source dimension here is not decoration; it is the mechanism. The scaling method the paper describes is published under an MIT licence, with reproduction scripts and cached agent trajectories anyone can run (593 stars on GitHub as of 24 July 2026, for whatever a star is worth). More to the point, it only works on models open enough to expose their scoring-token logits. Verification-by-scaling is available on open or open-weight models and closed to systems that hide their probabilities behind a restricted interface. Openness stops being a licensing preference and becomes a functional requirement for a certain kind of assurance. V.E.R.A. sits in the same tradition, a logic engine you can read, run, and audit under an open licence rather than take on trust.
The implication on the sovereignty and defence side is worth stating directly. A verifier you can scale changes what a governance requirement like adequate verification actually asks for. The North Atlantic Treaty Organisation's Principles of Responsible Use of Artificial Intelligence in Defence, adopted in 2021, require defence AI to be reliable and traceable, and allied test, evaluation, verification and validation doctrine treats verification as a named assurance activity. None of those frameworks says how much verification effort is enough. If verification is a scaling axis with diminishing returns, adequately verified stops being a fixed checkbox and becomes a budget set against the stakes of the decision. The governance gap that follows is quiet but real: a state that treats verification as a one-time gate and one that treats it as a scalable, diminishing-returns investment will not reach the same assurance for the same spend.
So here is the question I would put to anyone who has built, or is about to build, a gate that decides whether AI-generated work is good enough to ship. When your reviewer gives you a clean pass, is it giving you a verdict, or a tie it has learned to hide? And when you settled on how many checks were enough, did you decide that, or did the budget decide it for you?
The views expressed in this article are entirely my own, informed by more than 30 years of professional experience in architecture, security, and technology leadership in New Zealand. They do not represent the views of my employer, any government agency, or the New Zealand government. My commentary on legislation and policy is analytical, drawing on publicly available sources and my professional expertise in architecture, security, and AI governance. I follow the Public Service Commissioner's Code of Conduct for the Public Sector and social media guidance.
Andreas Hamberger is a New Zealand leader in Architecture & Security and Associate Member of the Institute of Directors. V.E.R.A. (Verified Existence & Reason Architecture) is an open-source logic engine available on GitHub.
I use AI tools, including Sudowrite, Claude, Perplexity AI, DeepSeek AI, ChatGPT, Grok, Copilot, Openart and Gemini, as deliberate production tools, not ghostwriters. This is consistent with my position: AI amplifies human judgement; it does not replace it. The frameworks, arguments, and editorial decisions in this series are original work. AI accelerated the process. The thinking is mine.
References
[1] Kwok, J., Li, S., Atreya, P., Liu, Y., Jiang, Y., Finn, C., Pavone, M., Stoica, I., and Mirhoseini, A. "LLM-as-a-Verifier: A General-Purpose Verification Framework." arXiv preprint 2607.05391, 6 July 2026. https://arxiv.org/abs/2607.05391
[2] Kwok, J., et al. "LLM-as-a-Verifier: A General-Purpose Verification Framework" (full text, version 2). arXiv, 7 July 2026. https://arxiv.org/html/2607.05391v2
[3] LLM-as-a-Verifier project page. Retrieved 24 July 2026. https://llm-as-a-verifier.com/
[4] LLM-as-a-Verifier source code repository. Retrieved 24 July 2026. https://github.com/llm-as-a-verifier/llm-as-a-verifier
[5] Wan, Y., Fang, T., Li, Z., Huo, Y., Wang, W., Mi, H., Yu, D., and Lyu, M. R. "Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification." Findings of the Association for Computational Linguistics: ACL 2026. https://aclanthology.org/2026.findings-acl.1243/
[6] Zheng, L., Chiang, W-L., Sheng, Y., Zhuang, S., et al. "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena." arXiv 2306.05685, June 2023 (NeurIPS 2023). https://arxiv.org/abs/2306.05685
[7] Government Chief Digital Office. "Public Service Artificial Intelligence Framework." digital.govt.nz, last updated 29 January 2025. https://www.digital.govt.nz/standards-and-guidance/technology-and-architecture/artificial-intelligence/public-service-artificial-intelligence-framework
[8] Hamberger, A. "V.E.R.A.: Verified Existence and Reason Architecture (documentation)." Te Pono Limited, January 2026. GitHub. (No captured document URL.)
[9] Reserve Bank of New Zealand. "Operational Resilience Standard 2027 (Tranche 3)." (No captured document URL; clause wording to be verified against the primary source before publication.)
[10] European Union. "Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 15." (No captured document URL.)
