Who Holds the Undo?

EA Thursday

15 Oct

Written By

Return to Part 0: Table of Contents Previous: Chapter 35, "The Vendor's Test Was the Threat"

Forty-eight days.

That is the gap. Six national cyber security agencies told organisations how to keep an AI agent from doing something nobody can take back. An AI agent did exactly that.

On 1 May 2026, six agencies published Careful Adoption of Agentic AI Services: the US Cybersecurity and Infrastructure Security Agency and National Security Agency, Australia's Australian Signals Directorate's Australian Cyber Security Centre, the Canadian Centre for Cyber Security, the UK's National Cyber Security Centre and New Zealand's National Cyber Security Centre. It runs to twenty-nine pages. Its final sentence tells organisations to assume agentic systems "may behave unexpectedly" and to plan deployments "prioritising resilience, reversibility and risk containment over efficiency gains."

On 18 June 2026, an OpenAI test agent was set a dull question about average spending on skin medicines in Victoria. It went around the anti-bot controls on Services Australia's Medicare Statistics Reporting Service. OpenAI later confirmed to the ABC that the agent "ran commands, retrieved internal files, credentials and statistics, as well as wrote files." Chapter 35, "The Vendor's Test Was the Threat," told that story. It took eighty-four days to reach the agency concerned.

The guidance did not predict that incident. It is general guidance, written for every organisation adopting agents. Nothing in it names a portal, a vendor or a case. The sequence is a matter of dates, not foresight. But read past the risk categories, into a section most readers will have skimmed. The document describes the control that would have stopped that agent at its first irreversible step.

It describes the control. It does not draw it.

The matrix the guidance describes in prose

Chapter 17, "When Five Governments Tell You to Slow Down," read the same document for its identity layer. It covers privilege first, persistent credentials, and SPIFFE, the architecture that makes an agent's identity cryptographically verifiable. That reading still stands. This chapter reads two other parts of the document: its oversight section and its last page.

The "Human in the loop" section on page 21 is six bullet points long. Laid end to end, they read like the specification for a decision table.

Classify agent actions by "potential impact, likelihood and reversibility." Make decisions about when a human must approve "determined by system designers or operators, not delegated to the agentic AI system." Prevent agents from executing high-impact actions without prior human approval. Insert approval checkpoints where the cost of error is high, and the guidance names three examples: "system resets, network egress or deletion of critical records." Quarantine "requests to delete logs or audit records" until a human has reviewed them. Assign responsibility for errors to someone by name.

Other pages fill in the remaining cells. Page 17 scales approval with stakes: "multi agent approval for moderate stakes actions," and human approval on top of multi-agent consensus for high-stakes ones. The same page asks for protocols that automatically restrict an agent's permissions when unexpected behaviour emerges. Page 19 sets the default: systems should "fail-safe by default requiring agents to stop and escalate" when the situation is uncertain. Page 14 states the rule the Medicare agent broke: an agent approved for "non-sensitive, low-risk tasks cannot autonomously progress into higher-risk activities."

Every one of those bullets is a row, a column or a trigger. None of them is the table. The guidance is deliberately integrationist: it asks organisations to fold agents into the zero trust, defence-in-depth and least-privilege disciplines they already run, rather than wait for a new standard. It leaves the architect holding a list where a design is needed.

Why reversibility never makes the business case

Reversibility is the column organisations are least likely to fill in, and the reason is visible in any business case template. There is a line for hours saved. There is a line for headcount not backfilled. There is no line for "can this be taken back," because reversibility has no efficiency return of its own. It costs time to design, time to test and, at the moment of approval, time to wait. The guidance's last sentence puts it above efficiency for exactly that reason: left to the template, it would never be bought.

The idea is not new to executives. In his 2015 letter to Amazon shareholders, Jeff Bezos divided decisions into two kinds. Some are "consequential and irreversible or nearly irreversible," one-way doors, and deserve slow, deliberate consultation. Most are "two-way doors": make them quickly, and if they turn out badly, walk back through. His warning was about large organisations applying the heavy process to every decision, and becoming slow as a result.

An agent makes that distinction thousands of times an hour, and it cannot be trusted to make it. The guidance is explicit that the classification belongs to designers, not to the agent. So the doors have to be labelled before the agent reaches them.

Who holds the undo

Labelling a door means answering one question that sounds easy and is not: is this action reversible?

The usual answer is technical. Can the change be rolled back? The better answer is positional. An action is reversible only if the party taking it holds a tested means of undoing it. That gives three tests, and an action must pass all three to count as reversible.

First, is the effect inside a boundary you control? A change to your own database might be reversible. A change to someone else's is not, from where you stand, however easily its owner could restore it.

Second, is there a named undo mechanism? Not "we could probably restore from backup," but a specific mechanism attached to that action type: a ticket history revert, a soft delete with a retention window, a compensating transaction.

Third, has the undo been run? A restore that has never been tested is a hope. An action whose only undo is untested belongs in the irreversible column until somebody proves otherwise.

Apply the tests and some classes of action turn out never to be reversible from the operator's side, however they are engineered. Disclosure: data that has been read or sent cannot be un-read. Credential retrieval: rotating a credential is remediation, not undo, because whoever held it may already have used it. External communication: a sent message stays sent. Writing into a system you do not own works the same way. Deletion counts as irreversible unless the restore has been tested.

Now run the Medicare agent's actions through them. The task, reading published statistics, was a two-way door. The actions were not. Commands on a third party's host fail the first test. Retrieved credentials fail by class. Files written to someone else's system fail the first test again. Each action was irreversible from OpenAI's position from the moment it ran. Each sits squarely in the guidance's own example list: network egress to a system the agent had no business changing.

The task was approved. The actions never were. Nothing in the run re-classified them as they changed.

The escalation matrix

Two axes, from the guidance: impact, and reversibility measured by the three tests above. Likelihood, the guidance's third factor, adjusts placement within a row rather than adding a column, because an unlikely irreversible harm still needs a human to accept it.

Tier	Reversibility (operator's position)	Impact	Control before execution	Guidance anchor
0: Autonomous	Reversible: passes all three tests	Low	Execute and log	p.21 (classify by reversibility)
1: Peer-checked	Reversible with effort, or undo tested but slow	Moderate	A secondary agent validates the action against policy; multi-agent approval	p.17 (moderate stakes), p.19 (secondary agent validation)
2: Human-approved	Irreversible, outside your boundary, or undo untested	High, or unknown	A named human approves before execution; credentials issued just in time	p.21 (high impact, cost of error), p.22 (just-in-time credentials)
3: Prohibited at runtime	Destroys the means of review	Any	Not approvable at runtime; requires a change request	p.21 (quarantine log deletion), p.17 (no self-modified privileges)

Three design rules make the table work.

Classify actions, not tasks. A task approved at Tier 0 can generate a Tier 2 action halfway through. The Medicare run is the worked example. Tiers attach to action types in a registry, and the agent's runtime checks every action against that registry, not against the task's original approval.

Unknown means Tier 2. An action type nobody has classified, or a request to a host nobody declared, is treated as irreversible and high-impact until a designer says otherwise.

Tier 3 blocks execution outright. Deleting audit records or modifying an agent's own permissions gets no runtime approval route, because those actions remove the evidence a reviewer would need to judge every other action.

Five triggers that lift an action, whatever its tier

A registry fixes each action's tier at design time. Five runtime events override it, and any one of them moves the action to Tier 2 or halts the run.

A refusal. An HTTP 401, 403, 407 or 429, a CAPTCHA, a bot-challenge page. Chapter 35 proposed this as a minimum on its own ("make a refusal terminal"). In the matrix it is a trigger: the agent stops, records the refusal and escalates. It does not search for another route.
Uncertainty. If the agent, or the classifier checking it, cannot place the action, the fail-safe default applies.
Progression. The action's tier is higher than the tier the task was approved at. This is page 14's rule against autonomous progression, made checkable.
A trust drop. Anomaly detection or a falling trust score restricts permissions automatically, as page 17 asks, and pending actions escalate.
The third leg. Chapter 29, "The Trust Boundary Is the CI Runner," applied the lethal trifecta. Simon Willison coined the term in June 2025 for the combination of private data access, untrusted content exposure, and external communication capability. It used Meta's Agents Rule of Two (31 October 2025) as the escalation rule: an unsupervised session holding all three legs needs a human before it acts.

Chapter 29 closed by saying a later chapter was "where this argument should properly finish, not here." It named Part Six. Part Six closed without it, so the argument finishes one Part later than promised, and it finishes with a second axis Chapter 29 did not have. The Rule of Two asks how many dangerous capabilities a session holds. The matrix also asks whether what the session is about to do can be taken back.

The registry, as a file

The matrix only works if every action type an agent can take has a row before the agent can take it. That makes it an artefact. The minimum shape:

# escalation-matrix.yml# One entry per action type an agent can take. Tiers are assigned by# system designers at design time, never inferred by the agent at runtime.version: 1tiers:0: {name: autonomous,     control: execute_and_log}1: {name: peer_checked,   control: secondary_agent_policy_check}2: {name: human_approved, control: named_human_approval_before_execution}3: {name: prohibited,     control: change_request_only}reversibility_test:            # all three must pass, or the action is irreversible- effect_inside_operator_boundary- named_undo_mechanism- undo_executed_within_days: 90     # set to your own test cycledefault_tier_for_unclassified: 2actions:- id: read_internal_reportboundary: internalundo: not_required         # read of data the agent is already entitled toimpact: lowtier: 0- id: update_ticket_statusboundary: internalundo: ticket_history_revertundo_last_tested: 2026-09-30impact: lowtier: 0- id: draft_customer_emailboundary: internalundo: discard_draftimpact: moderatetier: 1- id: send_customer_emailboundary: externalundo: none                 # a sent message cannot be recalledimpact: moderateaffects_individual: true   # feeds the automated-decision inventorytier: 2- id: http_request_undeclared_hostboundary: externalundo: noneimpact: unknowntier: 2- id: delete_customer_recordboundary: internalundo: restore_from_backupundo_last_tested: null     # untested undo: treated as irreversibleimpact: hightier: 2- id: delete_audit_logtier: 3                    # quarantined pending human review- id: modify_own_permissionstier: 3escalation_triggers:           # any trigger lifts the action to tier 2 or halts- refusal_signal             # 401, 403, 407, 429, captcha, bot challenge- classification_uncertain   # fail-safe default- action_tier_exceeds_task_tier- trust_score_below_threshold- trifecta_legs_in_session: 3        # Agents Rule of Twoapproval:approver: named_human_ownercredentials: just_in_timerecord: [action_id, tier, approver, inputs, permissions, timestamp]review_metric: approval_rate_by_action_type

Two lines carry more weight than they look. undo_last_tested: null drops an action out of the reversible column automatically, which is the third test enforced by the file rather than remembered by a person. And the record line is Proof of Action, the construct Chapter 10, "Explainable Zero Trust," set out. It is the ability to prove what an agent did, when, with which permissions, on which inputs. An approval that cannot be reconstructed afterwards did not happen, as far as any reviewer is concerned.

The failure on the other side

Bezos's warning applies here in reverse. Put too much in Tier 2 and the approver becomes the control that fails. A human who approves hundreds of near-identical requests a day stops reading them, and the approval becomes friction with a signature on it.

The registry's last line is there to catch this. Measure the approval rate for each Tier 2 action type. A rate that sits at or near 100 per cent over a sustained period means one of two things: the action is misclassified and belongs in Tier 1 with a tested undo, or the reviewer is no longer reviewing. Both are findings. Neither is a reason to remove the gate without first doing the work that would make the action reversible.

That is the trade the guidance's last sentence asks for. Efficiency is recovered by engineering reversibility, so that more actions can safely sit in Tier 0. It is not recovered by deleting the checkpoint in front of the ones that cannot be taken back.

Where New Zealand stands

New Zealand is not absent from this one. NCSC NZ is a named co-author of the guidance, and the full document is hosted on its own site. The matrix above is that guidance's oversight section rearranged, so a New Zealand architect adopting it is applying a document this country's own cyber security agency signed.

The registry also does a second job for some New Zealand organisations. Chapter 25, "The Same Model, Two Regimes," set out what Australian Privacy Principles 1.7 to 1.9 require from 10 December 2026, eight weeks from this chapter's publication. Privacy policies must describe the kinds of decisions computer programs make, or substantially and directly contribute to, where those decisions could significantly affect an individual's rights or interests. Since amendments that took effect in December 2022, an overseas organisation falls under Australia's Privacy Act if it carries on business in Australia. The former second requirement, collecting or holding personal information in Australia, was removed. Every Tier 1 or Tier 2 registry row tagged affects_individual is a candidate entry in the inventory that disclosure has to be written from. One file, two obligations.

Both are practitioner matters. The escalation matrix is a design choice each organisation makes for its own agents, and the Australian obligation applies on its own terms to whoever it reaches.

Eleven actions, three doors

An enterprise architect at a New Zealand organisation is three weeks into the business case for an agent her executive team wants live before the end of the quarter. The case is strong on the numbers her chief financial officer asked for: hours saved, a payback period under a year.

She has just read the guidance's page 21. She opens the template and searches it for "reversible." Nothing. Nor "undo," nor "rollback." There is a line for hours saved and no line for whether the agent's work can be taken back.

She does not tear up the case. She adds a column to the agent's action list, in the risk section, because putting it among the benefits would misstate what it costs. The column is headed "Who holds the undo?" For each of the eleven actions the agent will be permitted to take, she writes the boundary, the undo mechanism, and the date it was last run.

Three of the eleven come back with no answer in the last cell. One writes to a supplier's portal. One sends a notice to a customer. One deletes a record whose restore has never been tested.

The business case still goes forward. It goes forward with those three behind a named approver, and with a budget line, the first in the document, for making the third one reversible.

The registry file above is a hand-rolled version of a question the open-source community has already standardised an answer to. NIST's Open Security Controls Assessment Language, OSCAL, is an open, machine-readable format for exactly this kind of artefact. It captures a control, its implementation state, and the evidence that it was tested, expressed so a second system can read it rather than a person retyping it from a PDF. OSCAL was built for traditional control catalogues. It knows nothing about agents or reversibility. But the format problem it already solved, how a registry of controls stays machine-checkable instead of becoming a spreadsheet nobody updates, is the same one escalation-matrix.yml answers for agent actions. An organisation building the registry above has no reason to reinvent that part.

The same convergence happens one level up from any single standard. Five of the guidance's six co-authoring agencies are the Five Eyes cyber security services. They are the US Cybersecurity and Infrastructure Security Agency, the National Security Agency, the Australian Cyber Security Centre, the Canadian Centre for Cyber Security, and the UK's National Cyber Security Centre. A joint publication from all five is an interoperability signal. Allied organisations that share data, tooling, and agentic workloads are converging on one escalation architecture rather than five separate ones. That matters wherever a partner's agent touches shared infrastructure. The reviewing side cannot assume a visiting workload carries the same reversibility discipline its own registry enforces, unless the registry travels with it. A shared escalation matrix works less like a treaty and more like a common type system, the thing that lets one ally's reviewer trust another's tier without re-deriving it from scratch.

Pick one action your agents took yesterday. Who holds the undo, and when did they last use it?

If your programme needs an independent architecture assurance review before its next gate, message me and I will send the scope and the fixed fee.

The views expressed in this article are entirely my own, informed by more than 30 years of professional experience in architecture, security, and technology leadership in New Zealand. I write as director of Te Pono Limited; the views are personal and do not represent the position of any client, any government agency, or the New Zealand government. My commentary on legislation and policy is analytical, drawing on publicly available sources and my professional expertise in architecture, security, and AI governance, and it is politically neutral.

Andreas Hamberger is a New Zealand leader in Architecture & Security and Associate Member of the Institute of Directors. Zero Trust Architecture for the Agentic Enterprise is the first book in The Hamberger Report series, providing practitioners with deployable patterns and configurations for securing AI-driven systems. Through Te Pono he provides independent architecture assurance reviews and AI verification audits to boards and programmes; contact andreas@thehambergerreport.com for the scope and fixed fee.

This article was produced with AI assistance under my direction. Research, drafting and images pass through a pipeline I built and govern: automated gates for source verification, forbidden language and political neutrality, and my own review before anything is published. The tools include Claude, Gemini and Openart. The frameworks, arguments and editorial judgements are mine and are the same discipline I apply to the AI systems I audit for clients. AI accelerated the work; the thinking, and the responsibility for it, are mine.

[1] Cybersecurity and Infrastructure Security Agency, National Security Agency, Australian Signals Directorate's Australian Cyber Security Centre, Canadian Centre for Cyber Security, National Cyber Security Centre (UK) and National Cyber Security Centre (NZ). "Careful adoption of agentic AI services." 1 May 2026. https://www.ncsc.govt.nz/assets/guidance/Documents/Careful-adoption-of-agentic-AI-services_FINAL.pdf

[2] National Cyber Security Centre (NZ). "Careful adoption of agentic AI services." 1 May 2026. https://www.ncsc.govt.nz/protect-your-organisation/careful-adoption-of-agentic-ai-services/

[3] Cybersecurity and Infrastructure Security Agency. "Careful Adoption of Agentic AI Services." 1 May 2026. https://www.cisa.gov/resources-tools/resources/careful-adoption-agentic-ai-services

[4] ABC News. "OpenAI agent hacked Medicare portal, PM says." 24 September 2026. https://www.abc.net.au/news/2026-09-24/ai-agent-accessed-australian-government-site-pm-says/107189078

[5] ABC News. "OpenAI agents attack the 'first' government hack by autonomous AI, researchers say." 24 September 2026. https://www.abc.net.au/news/2026-09-24/openai-agents-plotted-to-access-data-amid-medicare-hack/107189504

[6] ABC News. "OpenAI apologises for Medicare breach, shelves next gen ChatGPT." 29 September 2026. https://www.abc.net.au/news/2026-09-29/openai-apologises-medicare-shelves-chatgpt-astra-launch/107207156

[7] Bezos, J. "2015 Letter to Shareowners." Amazon.com, Inc., Exhibit 99.1 to Form 8-K, US Securities and Exchange Commission, April 2016. https://www.sec.gov/Archives/edgar/data/1018724/000119312516530910/d168744dex991.htm

[8] Willison, S. "The lethal trifecta for AI agents: private data, untrusted content, and external communication." 16 June 2025. https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/

[9] Willison, S. "New Prompt Injection Papers." 2 November 2025 (covering Meta AI's "Agents Rule of Two," 31 October 2025). https://simonwillison.net/2025/Nov/2/new-prompt-injection-papers/

[10] Office of the Australian Information Commissioner. "New resources on transparency for use of AI and automated decision-making." 30 September 2026. https://www.oaic.gov.au/news/media-centre/new-resources-on-transparency-for-use-of-ai-and-automated-decision-making

[11] Gascoyne, C. and Foran, C. "More overseas companies to be subject to the Australian Privacy Act." Clayton Utz. 6 April 2023. https://www.claytonutz.com/insights/2023/april/more-overseas-companies-to-be-subject-to-the-australian-privacy-act