An accountability layer beyond models
Australia’s Robodebt scheme was not an AI-agent system, but it is a documented warning about automated authority. In Recommendation 10.1, the Royal Commission called for in-person and telephone options for people who lacked technology or digital literacy, and warned that government interactions could deepen financial stress. Recommendation 17.1 called for a clear path to review any automated decision. The correction process is part of the system, not an apology added after it fails.
The lesson is larger than one failed programme. Technical execution is not accountability. A system can identify the actor, record the decision and still leave the affected person powerless.
Many common web transactions bind a human account holder to a session. A person opens the page, clicks the button, approves the payment and carries the legal identity behind the account. Agents create distance inside that arrangement. Software can discover a service, negotiate a task, call a tool and attempt a payment while the person it represents is somewhere else.
This article concerns consequential agents: systems able to affect a person’s money, data, rights, access to services or physical environment. It proposes an architectural test for those systems, not a forecast that one universal agent stack will emerge.
Consequential agents need more than intelligence: identity, bounded authority, evidence, revocation and recourse. The final word is the important one. Recourse means a usable path to stop an action, correct the record and reach a human who can provide a remedy.
The components already exist in fragments. OAuth can express narrow grants. Workload-identity systems can authenticate software. A separate policy service can apply hard rules outside a model. Payment protocols can carry signed mandates. Content Credentials can preserve provenance.
The unsettled problem sits in the joins. Who is the agent acting for? What was it allowed to do? What happened? Who bears the loss? Can the person stop it, correct it and appeal?
In this article, “the next internet” is shorthand for that accountability layer. It names an architectural requirement, not a forecast.
The accountability test
The systems below solve different pieces of the same problem. Identity can establish who acted. A mandate can record permission. A receipt can preserve what happened. None of those mechanisms, by itself, gives a person power over the outcome.
That distinction should organise the architecture. Technically valid operation is insufficient when affected people or participating institutions lack visibility, a way to intervene and a path to remedy.
Authority and transactions
The next layer does not wait for today’s AI market to consolidate.
OAuth Rich Authorization Requests let a service describe a narrow permission instead of asking for one broad grant. SPIFFE gives a software workload a verifiable identity. A separate policy service can then decide whether the requested action fits the grant, without asking the model to police itself.
These are standards and operational building blocks, not evidence that one agent-accountability system has been deployed across providers. They do not establish competence or trustworthiness.
The architecture proposed here is compositional. One mechanism establishes machine identity. Another binds the machine to a person or organisation. A mandate grants a purpose, scope and expiry. A policy engine decides whether a requested action fits. A receipt records what happened. An institution handles disputes.
NIST’s 2026 agent-identity concept paper treats three questions as unresolved: how agents should be bound to the people or organisations responsible for them, how delegated authority should be revoked, and how audit should work across systems.
Identity is only the first question. A credential can prove who signed a claim. It cannot prove that the agent is competent, honest or appropriate for a particular task.
Payment evidence and disputes
Agent payments are moving into protocols and pilots.
The Agent Payments Protocol defines signed checkout and payment mandates, receipts and flows for transactions with or without a human present. Visa’s Trusted Agent Protocol helps merchants distinguish approved commerce agents from hostile bots. Mastercard Agent Pay uses substitute card credentials and device-based authentication rather than exposing a card number and password to the agent.
These are meaningful foundations. They do not create a universal economic system for agents.
A signed mandate can show what the user authorised. A receipt can preserve what the parties saw. Neither decides whether a product was misrepresented, a delivery failed or a vulnerable user should receive a remedy.
A dispute still needs judgment.
The durable layer needs mandate review, pause and revocation, notifications, reason codes, receipt retrieval, complaint handling, a remedy for a disputed transaction, and human appeal. Agent protocols must connect to those processes without reducing a dispute to cryptographic validity.
Agents can pay. The unresolved problem is whether a person can recover when a technically valid transaction was substantively wrong.
Data, evidence and institutional control
Personal AI sovereignty is an attractive phrase. Current systems prove narrower things.
The European Commission’s Data Act overview describes access to data from connected products and measures intended to support switching between data-processing services. Exact rights and duties depend on the actor and use case. Solid Pods separate applications from personal data stores, although Solid’s application-interoperability work remains a draft community report rather than a W3C standard.
A real counterweight to platform control needs three properties together: private execution, portable data and credible switching. A privacy feature inside a closed ecosystem provides one of them.
User-owned AI remains a hypothesis until a person can move memory, preferences, credentials and operating history between providers without rebuilding their digital life.
Content history
C2PA Content Credentials attach a signed record of origin and edits to a digital asset. The record can describe creation, tools, edits and participating signers. If a signed asset is modified and its credential remains available for validation, that modification can invalidate the credential. An absent credential proves nothing.
The standard has conforming products, including camera implementations, recorded in C2PA’s public conformance explorer. The same explorer publishes its product and trust lists. That is evidence of implementation, not universal adoption.
Its own documentation preserves the important limit. Provenance does not judge whether an assertion is good or bad. It cannot establish that a depicted event happened. Credentials can be absent or stripped.
This makes interpretation part of the infrastructure. A viewer needs to understand who signed the record, which parts are missing and what failed validation. C2PA’s user-experience guidance recommends showing simple information first, making further detail available, supporting accessibility, explaining failures and collecting continuous feedback.
The standard states its limits explicitly. Interfaces must preserve those limits so that provenance is not presented as truth.
The useful result is a verifiable history. No universal truth detector follows from it.
Physical safety boundaries
Industrial robotics has established safety standards. In 2025, ISO published requirements for industrial robots and for industrial robot applications and cells. The application standard excludes service, consumer, medical, military and airborne systems. Each needs its applicable safety regime.
That scope is the lesson.
A probabilistic planner should hand off to a bounded control interface. In plain language, the model proposes an allowed command, while separate machinery enforces where and how fast the system may move, watches for unsafe conditions and preserves an emergency stop. Those limits belong outside the model’s discretion.
A policy engine can decide whether an agent may call a tool. It cannot replace machinery, automotive or medical-device safety standards.
Across the domains examined here, one protocol cannot replace domain-specific control. That control remains part of the accountability layer.
Recourse and user control
Across these areas, technical capabilities and controls can exist while affected people or participating institutions still lack a usable way to intervene and obtain a remedy.
Revocation lets a person withdraw an identity grant. Appeal turns payment evidence into a path through a dispute. Portability lets protected data leave a provider. Clear presentation makes provenance useful to an ordinary viewer. A stop control gives physical safety authority to someone outside the model. The controls differ, but each one changes what a person can do after automation has started.
NIST’s AI Risk Management Framework calls for channels through which users and affected communities can report problems and appeal outcomes. That should be treated as infrastructure, not a final survey widget.
This accountability test evaluates consequential-agent systems against seven questions:
- Can a person see what authority was granted?
- Can they narrow, pause or revoke it?
- Can they inspect an intelligible receipt?
- Can they correct wrong data or context?
- Can they report harm or failure?
- Can they appeal to a human with power to change the result?
- Does resolved feedback change the system, policy or evaluation?
For this article’s accountability test, user agency is measured through revocation and recourse rather than the amount of personalisation offered.
The Robodebt Commission’s design warning returns here. A person who lacks reliable internet access or digital literacy cannot use a remedy that exists only behind the same interface that failed them. Visibility, intervention and appeal have to work under the conditions people actually face.
Two directions
One version develops shared accountability rails. Identity, mandates, receipts and provenance move across providers. Users can revoke authority, recover evidence and appeal. Platforms compete above common foundations.
Another version keeps each agent inside a private empire. Identity, memory, payments and recourse work well as long as the user never leaves. The experience may be safer than today’s fragmented systems and still concentrate control.
The evidence does not tell us which version wins. It gives us a way to measure the direction.
When the system is technically correct and substantively wrong, can an affected person obtain a remedy from someone with power?