Correction note (1 October 2026): This article now separates documentation from execution without implying bad faith, removes detailed regulatory applicability claims, and narrows the runtime guarantee to enforcement of a specified policy decision on paths that actually reach the boundary.
Governance documents describe what a system should do. That matters. Documentation does not, by itself, constrain the next tool call.
A consequential action is operationally constrained by a specified policy when the application routes it through an enforceable boundary. The boundary needs explicit policy inputs, a recorded decision and a defined failure mode. A policy document that never reaches the execution path cannot supply that control.
This argument is about agentic systems in workflows where one action can move money, change access, alter a record or create an external effect. The legal classification of any particular system still depends on its use case, operator, jurisdiction and effective dates. This article does not establish legal applicability.
For the incident history and human-factors argument behind action-level controls, see Runtime Is the Moment of Consequence in AI. This article answers a narrower question: when does a governance artifact become an operational control?
Documentation and execution are different artifacts
NIST’s Generative AI Profile treats confabulation, prompt injection, information integrity and other risks as things to measure and manage across the system lifecycle.1 OWASP’s agentic guidance separately identifies tool misuse, excessive agency and control-flow failures.2 Neither source says that a policy document automatically becomes an execution control.
That distinction is the gap. The document states intent. The execution path decides whether an action can happen.
Why this happens, structural causes
Documentation and enforcement solve different problems. Writing a policy, system instruction or model-behaviour objective can be cheaper than integrating a check into every consequential action path. The documentation can therefore arrive first.
Prompts, fine-tuning and preference optimization can reduce unwanted model behaviour. They remain model-behaviour controls. A separate action boundary can enforce a narrower property such as “this identity may not call this tool for this resource.” The rule can still be wrong. Deterministic evaluation makes the same normalized request, policy version and relevant state produce the same policy result. It does not remove false positives or false negatives.
The second structural cause is cycle mismatch. Periodic review often operates on a slower cycle than agent decisions. The review produces a snapshot. The agent produces a stream.
The third structural cause is evidence mismatch. A periodic assessment captures a system at one point. An agent produces a stream of proposed and executed actions. Investigating that stream needs event records, versions and effect receipts, not a claim that the model was instructed to behave.
Adjacent systems show the pattern. API gateways enforce authentication and rate limits at a request boundary. Databases enforce transaction rules around operations. Agent applications can use the same architectural idea, but only if every consequential path actually reaches the gate.
A public filing from Meta documents one concrete action path: attackers used an AI support experience in account-takeover attempts, while a separate code path failed to verify the supplied email before the recovery flow proceeded.3 The filing does not establish the model’s internal reasoning. It does show why the relevant control belongs on the account-change path, not only in instructions shown to a model.
None of this requires bad faith. It is a systems-integration problem with an evidence consequence.
The missing piece, action-level runtime governance
In the workflow class examined here, failures can take familiar shapes. Prompt injection can turn a benign instruction into unauthorized tool use. Multi-step plans can accumulate side effects no individual step would have triggered on its own. Model updates can produce silent policy drift between deployment and review. The relevant architectural risk is an action executed without a verifiable policy check at the boundary.
Some current frameworks already document pre-execution controls. NVIDIA NeMo, for example, documents execution rails that can inspect or block a tool call.4 General authorization systems also evaluate requested actions before access is granted. The useful question is no longer whether such a control can exist. It is whether the application routes every consequential action through it and preserves evidence that it ran.
The evidence needs careful language. A useful runtime record can preserve the request, retrieved inputs and provenance, model and policy versions, proposed tool call, authorization result, executed arguments, returned state and effect receipt. That can make the observed decision path reconstructable.
Reconstruction is not replay. Re-evaluating the same policy requires the same normalized inputs, policy version and relevant state. Reproducing a stochastic model response may also require model version, configuration and nondeterministic dependencies. Repeating an external side effect may be unsafe or impossible. W3C PROV helps represent lineage and transformations, but it does not promise byte-identical reproduction.5
Probabilistic alignment methods remain essential. RLHF, fine-tuning, system prompts and constitutional AI can reduce baseline risk and improve the distribution of model behavior. A separate boundary can enforce a specified policy decision for an individual action that reaches it. Runtime governance does not replace probabilistic alignment, guarantee that the policy is correct or prove that every action path is covered.
The runtime record therefore answers a bounded question: what did this application observe, decide, attempt and receive? That is narrower than perfect replay. It is still far more useful than an instruction with no execution evidence.
fivedrisk
While building DotOS, my personal AI systems lab, I wanted a control between an agent’s proposal and a consequential tool call. So I started developing fivedrisk and published the architecture as an Apache-2.0 open-source project. It evaluates proposed actions across five explicit dimensions and records the policy result. The public repository shows the implementation and its limits. It does not establish regulatory compliance.
Closing
Documentation is necessary. For consequential actions, it becomes operational when the application can prove that the relevant execution path reached an enforceable boundary.
Is “usually, my LLM follows its instructions” enough for your systems?
References
Footnotes
-
[Official guidance] National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024. https://doi.org/10.6028/NIST.AI.600-1. Verified 2026-09-30. Scope: generative-AI risk management. Limitation: voluntary guidance, not a compliance determination. ↩
-
[Official project guidance] OWASP GenAI Security Project, Top 10 for Agentic Applications 2026. https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/. Verified 2026-09-30. Scope: agentic-application threats. Limitation: community security guidance, not law. ↩
-
[Official filing] Maine Attorney General breach record containing Meta’s notice, 2026. https://www.maine.gov/agviewer/content/ag/985235c7-cb95-4be2-8792-a1252b4f8318/686120c8-63be-4e3c-b7ed-466d65b672f5.html. Verified 2026-09-30. Scope: the described verification failure and account-takeover attempts. Limitation: the filing does not establish the model’s internal reasoning or exact tool path. ↩
-
[Vendor documentation] NVIDIA, NeMo Guardrails: Rail types, current documentation. https://docs.nvidia.com/nemo/guardrails/latest/about-nemo-guardrails-library/rail-types. Verified 2026-09-30. Scope: documented pre-execution and post-execution controls. Limitation: a documented feature does not prove complete integration or deployed effectiveness. ↩
-
[Standard] W3C, PROV-DM: The PROV Data Model, Recommendation, 30 April 2013, sections 2 and 5. https://www.w3.org/TR/prov-dm/. Verified 2026-09-30. Scope: provenance representation. Limitation: provenance does not guarantee deterministic replay or truth. ↩