Five Rails and the Effect Guard — ข้อเสนอไม่ใช่ผลจริงFive Rails and the Effect Guard — A Proposal Is Not an Effect

🤔 The model proposes a 2,500-baht refund with a very plausible explanation — what should stop it: the prompt, a classifier, or code that does not read the explanation at all?

The previous post, #12 The Assurance Contract, ended on a single document — the assurance contract that forces us to stop saying a system is "safe and reliable" and to write instead, property by property, what is a structural guarantee, what is a semantic estimate, and what residual risk is left over.^[1] The contract says what we guarantee. This post answers the question an engineer asks immediately afterwards: where in the system do those obligations actually sit?

The one-line answer is on the rails, not in the instructions — five rails named Input, Dialog, Retrieval, Execution and Output, placing a mechanism at every seam an LLM system has, with a seven-check effect guard standing in the joint between "what the model proposed" and "what actually happened in the ledger". The sentence this chapter hands you is one sentence: the model proposes • deterministic control authorizes • authoritative state proves. The rest of this post unpacks it into five rails, three workshops you can genuinely fill in, and one set of test results about which I will be blunt on what it proves and what it does not.

1. A Proposal Is Not an Effect — a Boundary Needs Code Standing on It, Not a Sentence

The playbook runs one case through the whole Engineer group: Luma Commerce Thailand and a use case called CX-REFUND-01, a customer-care assistant that answers refund-policy questions and, in some cases, issues the refund itself (a fictional case from the playbook) — the company, the use-case ID and every number attached to this case are invented for teaching. There is no real customer here, and no measurement taken from a system in service.

So the scenario behind the opening question runs like this. The contract for CX-REFUND-01 states that a refund cleared by the guard will not exceed 2,000 baht, and will not execute at all without an authenticated approval token. One day the model proposes 2,500 baht with a tidy eligibility explanation: it cites the right policy clause, the tone is courteous, and every part of it looks correct. The question is who should be the one to say no.

There are three wrong answers, and I have met all three in real meeting rooms. The first is add a line to the prompt saying never exceed two thousand. The second is put a classifier in front and let it flag text that looks abnormal. The third is appoint a second model as the reviewer. All three share one property: the thing making the final decision is still a system that produces probabilistic output, and the evidence that the money stayed under two thousand is still a piece of text rather than a piece of state.

This chapter's answer is the exact opposite. What should stop 2,500 baht is code that does not read a single line of the explanation, and checks only whether the amount falls inside the range the contract allows, whether an authenticated approval token is present, and whether this request has already been executed once. If any one of those fails, it refuses and writes an escalation trace even if the explanation sounds plausible.^[1] This is what the playbook calls proposal–effect separation, and the mechanism that makes it real is effect mediation.

The difference is not a technical detail. It changes what you are able to say to a board. If the limit lives in the prompt, the most you can claim is "we told the model". If the limit lives in a tool path that every route is forced through, what you can claim is "a refund that travels this path cannot exceed two thousand baht" — and the second sentence is the only one anybody can audit. That is why the chapter opens the way it does.^[1]

"Assurance converts confidence into accountable testable obligations and makes failure routing part of the design."

The hardest-working phrase in that sentence is "failure routing". A well-designed system is not one that never breaks; it is one that, when it breaks, knows which way to go — refuse, narrow, escalate to a person, or stop fail-closed — and every one of those routes was written into the contract in advance rather than improvised during the incident. The playbook adds one more rule of modesty, and I think it is the line an engineering team should print and put on the wall.^[1]

"It should never claim that the whole AI system is safe. It should say narrowly what can be enforced, what can only be estimated, and what residual risk remains."

Accept that rule and the design question changes shape immediately. It stops being "how do we make the model safer" and becomes "what do we put where, so that each property we promised has a real mechanism behind it". The answer to the second question is five rails.

2. The Five Control Rails and the Assurance Envelope

Before we look at the figure, three things that people habitually lump together into one word — "guardrails" — have to come apart. The playbook keeps them as three distinct control classes, and that separation is a precondition for everything that follows.^[1]

• Hard enforcement — creates structural invariants such as authorized tools, valid parameters, transaction limits and schema-conforming payloads, subject to two conditions: every path must be mediated (complete mediation) and the implementation must be correct
• Soft detection — estimates semantic properties such as injection, faithfulness, relevance, privacy and policy alignment. Its verdicts carry false accepts and false rejects on a population that has to be stated
• Governance — supplies approval, trace retention, audit, rollback and incident response. It creates accountability and recovery, but it cannot make any individual answer correct

The three answer different questions and cannot stand in for one another. Hard enforcement answers "impossible". Soft detection answers "probably not, at this threshold". Governance answers "if it happened, we can find it and reverse it". An organisation that collapses all three into one phrase — our protections — will write a contract that promises at the hard level using a mechanism that only works at the soft level, without noticing it has done so.

The figure above is the five rails, in the order a single request actually travels: from the text a user types through to the effect that lands in the outside world. The red frame around all five is the assurance envelope — the boundary inside which we declare a guarantee. The dark bar underneath is the reconstructable trace, which is not a sixth rail but a span across every stage the system reaches.

The playbook states each rail's duty briefly: the Input Rail handles the user turn · the Dialog Rail governs multi-turn policy and permitted state transitions · the Retrieval Rail enforces source membership while estimating relevance and support · the Execution Rail authorizes and bounds every tool effect · the Output Rail enforces structure and estimates semantic quality before release · and trace retention spans them all.^[1]

The second and third columns are the heart of that table. Every time someone says "our system has checks in place", the only question worth asking back is which column those checks are in — because what sits in the third column cannot be written into a contract as a commitment. It can only be written as an estimate, with a threshold, a population, and error rates on both sides.

And here is the sentence I consider the most important in the chapter. The playbook sets it in bold: no rail covers everything, with four examples explaining why.^[1]

• An input filter cannot see malicious instructions arriving through retrieval — an instruction embedded in a document never passes through the channel the user types into
• An approved source can be poisoned — an allow-list says where a document came from, not what is written inside it
• An authorized tool call can oppose the user's intent — having permission is not the same as being right
• An output filter cannot recall data a tool has already exposed — an effect that has happened is beyond the reach of the last gate

That last item is the architectural reason the Execution Rail carries a different weight from the others. The other rails decide whether a piece of text goes forward. The Execution Rail decides whether money moves — and reversing those two things does not cost remotely the same.

3. The Seven-Check Effect Guard and Least Agency

Now down to the Execution Rail up close. The next figure is the one image I would most like to lodge in your memory from this whole series. On the left is what the model produces, in the middle is what decides, and on the right is what actually happens.

The box in the middle is the effect guard — an ordinary program that runs deterministically, does no learning, holds no probabilities, and never reads the accompanying rationale. It runs seven checks in order, and if any one of them fails there is exactly one destination: block and escalate.

💡 My view: of this chapter's five operating principles, the one I quote whole is principle 2 — "Put hard controls at release and effect boundaries — Complete mediation is a prerequisite." That sentence discards a great many long arguments, because it says the position of a control matters more than the cleverness of a control. The most accurate check in the world, placed at the wrong point, is still a check you can walk around. And "prerequisite" translates literally as: if any route can still bypass the guard, we cannot sign off yet — not sign now and close the gap later.

The principle did not begin in the LLM era. Saltzer and Schroeder's 1975 paper The Protection of Information in Computer Systems laid down complete mediation in a single line — "every access to every object must be checked for authority" — and set out least privilege as "every program and every user of the system should operate using the least set of privileges necessary to complete the job".^[2] Let us be exact: the 1975 work does not mention AI or language models once. Placing it beside the Execution Rail is the author's synthesis, and the playbook itself states the boundary on this source — enduring security principles require adaptation to contemporary distributed AI systems, not wholesale transplantation.

But there is one point that is nobody's synthesis — OWASP itself puts the two on the same page. In OWASP Top 10 for Large Language Model Applications 2025, under LLM06:2025 Excessive Agency, one prevention item carries the literal heading "Complete mediation", and its text reads "Implement authorization in downstream systems rather than relying on an LLM to decide if an action is allowed or not"^[3] — enforce authorization in the downstream system instead of trusting the model to decide whether an action is permitted. That is the one page where I will put a 2025 standard and a 1975 paper in the same sentence, and even there OWASP is endorsing the principle, not this playbook's five rails.

The same list separates the root causes of Excessive Agency into three, and they translate straight into the language of the workshop: excessive functionality is an agent with access to functions the intended operation does not need · excessive permissions is an extension holding permissions on downstream systems beyond what the application needs · excessive autonomy is an application that fails to independently verify and approve high-impact actions.^[3] Those three are the working definition of what the playbook calls least agency — hold the tools, the permissions, the degree of autonomy, the lifetime of that authority and the transaction limits down to what the job actually needs, not to what the system happens to be capable of.

One sentence on dates, said plainly: I cite the codes and wording of the 2025 edition (published 17 November 2024) because that is the edition whose text is still verifiable on the publisher's own site at the access date — OWASP published a 2026 edition on 3 August 2026 and announced it on 1 September 2026, but as of 5 September 2026 the per-risk pages of that new edition had not yet appeared on the publisher's site, so I cite no code and no ranking from the 2026 edition at all.

The other source worth reading beside it is OWASP's LLM Prompt Injection Prevention Cheat Sheet, which lays the defence out in layers — validate input, keep a human in the loop for high-risk requests, separate instructions from data using structured formats, grant least privilege, and only then validate the response.^[4] The sentence people citing this document tend to skip is the one saying that a guardrail which is itself a model can also be injected, and should be treated as one layer in a defence-in-depth design rather than as a replacement for input validation — the page carried no publication or revision date when I retrieved it on 5 September 2026, so I cite it by access date alone.

The last thing the guard must do is write a trace, and that trace has to keep the proposal and the effect visibly apart. The illustrative trace the playbook uses looks roughly like this (a fictional case from the playbook)

# CX-REFUND-01 — illustrative retained event
Begin       tr-8A41 · request rq-771 · authenticated customer c-204
            order o-919 · Thai · manifest rc4

Generate    candidate hash · protected text · citations
            proposed: issue_refund(order=o-919, amount=1850, currency=THB)
            ── no money has moved by this line; this is the "proposal" ──

Execution   retain Proposal · Verdict · Pre/Post-state
            route: Reject | Approve | Fail Closed

The proposed: line is what the model wanted to happen; the Execution line is what the system allowed to happen. If your trace has only one line and it reads "refund succeeded", you cannot tell an auditor whether the guard did anything at all, because you never kept what the guard refused.

4. The 517-Execution Specimen — What It Shows and What It Does Not Prove

Everything so far is an architectural proposal. The question to ask next is whether there is any evidence that putting hard controls at the effect boundary produces a different result from putting soft controls somewhere else. There is — but it is a very small piece of evidence, and I will give you its size in the same breath as its numbers.

The engineering paper Engineering AI-Core Systems, revision 8, includes a deliberately bounded authored specimen — 517 deterministic executions on a fixture the author built, using CX-REFUND-01 as the test case and a single configuration.^[5] There is no sample drawn from any population, no confidence interval and no comparison system. The number 517 describes only itself.

In the fixed suite — the tests the author wrote in advance and could therefore see in full — the results ran as follows: with the full envelope engaged the system completed 30 of 30 benign cases, allowed zero of 40 policy escapes, produced zero of 6 prohibited refund effects, and recorded 70 of 70 route traces.^[5] That set of numbers is dangerously easy to read, so let me attach three notes: 30 is the size of the benign arm in this suite, not 517 and not a number of customers · zero of 40 means zero in the visible suite, not zero in the world · and 6 is a denominator so small it cannot support any rate claim at all.

The more interesting arm is the second one. Adaptive-to-implementation testing means writing the attacks after seeing how this particular system defends itself — not generic attacks fired at any model. The result was that soft controls disclosed their own limits at once: 8 of 12 violating candidates were released, while hard execution mediation blocked all 4 prohibited effect attempts out of 4.^[5]

That pair of numbers is the whole substance of this post. Same system, same attacker, same run — yet the layer that estimates meaning let two-thirds through, while the layer that checks structure held completely. This does not make soft detection worthless: it catches things hard controls cannot see at all, such as a manipulative tone or a claim with no evidence behind it. What it means is that a property you cannot afford to get wrong must not be entrusted to a layer that estimates.

And the denominators need saying once more. 4 of 4 is four attempts, not four hundred. It is the smallest denominator in this entire post. Anyone who lifts it into a slide reading "the guard blocked 100%" is converting a demonstration of mechanism into a statistic, which is precisely what the specimen's own boundary forbids.

I will add three boundaries of my own. This work is the author's own manuscript, unpublished and with no public URL, so there is no link to check · there has been no replication by an independent team, and nobody outside this playbook has confirmed any of these numbers · and all of it was measured on one fixture and one fictional case, not on a system with real customers using it. I cite it because it localises where failure happens with unusual clarity, not because it measures safety.

5. Workshop — The Five-Rail Architecture Map (Artifact 3)

The rest of this post is material you can actually fill in, beginning with the map the playbook calls Artifact 3. It always fixes three things before the table itself — purpose, put a named control at every LLM-specific seam · use when, designing architecture, adding retrieval, memory, tools or conversation state, and investigating escapes · accountable owner, the application architect owns end-to-end coverage, each rail has its own operational owner, and security owns the threat model.^[1]

Notice that ownership sits at three levels on purpose. The person who answers "is every path mediated?" has to be one person. The person who watches how well each individual rail performs day to day is somebody else. And the person who thinks like an attacker is a third. If your organisation has one name in all three boxes, you do not have an owner; you have a bottleneck.

Two sentences sit under that table and they are, in my experience, the part teams forget most often. The first is the bypass rule: mark every bypass — batch, retry, webhook, cache, administrator tool, console, direct write, vendor fallback and recovery script — and then either mediate that path or exclude it from the guarantee.^[1] That sentence is complete mediation in its working form, because most systems are not breached through the front door; they are walked around at the recovery script nobody had in mind while drawing the architecture.

The second sentence is the whole chapter compressed into one line.^[1]

"The model proposes; an external guard authorizes; a transactional tool creates the effect."

The word external matters enormously. A guard living in the same process as the proposer, or in the same context as text that may have been injected, is not an external guard. And transactional matters just as much, because a tool that creates an effect must have only two states, committed or not committed — never an intermediate state where the thing is half done.

The completed example — CX-REFUND-01

The next table is the row-by-row example the playbook fills in for CX-REFUND-01. The right-hand column I would like you to read slowly is each owner's residual risk, because that is the column most often deleted when this document is turned into slides — and it is the entire reason the document exists.

Read the right-hand column downwards and a pattern appears. No rail's residual risk is "we have not finished checking"; every one of them is "there is something this rail cannot see, by its own nature". That is why the playbook insists no rail covers everything, and why this document has to carry real names — because a risk with no owner is a risk nobody is watching.

6. Workshop — The Decision and Consequence Matrix (Artifact 5)

The map in section 5 answers where the mechanisms are. The table in this section answers a different question: how much authority should the system have, for each individual decision? The playbook states the purpose of Artifact 5 as setting authority per decision and effect, not once for the whole application, and adds a sentence I think belongs in every organisation's AI policy: model indispensability does not determine acceptable autonomy.^[1]

That sentence cuts off the argument I hear most often — "the system cannot function without the model any more, so we have to let it run on its own". Indispensability is a fact about dependence; autonomy is a decision about risk. They answer different questions, and different people answer them: the business decision owner accepts the authority level · the risk owner challenges the expected harm · the tool owner enforces the route in code.

The playbook is very careful about how expected harm is used: as an aid, not as a single truth — likelihood multiplied by consequence, adjusted for exposure, detectability and recoverability, and then keeping four things apart: released text, reads, reversible writes, and irreversible effects. The rule that follows is that higher authority and lower recoverability require stronger evidence, and it closes with a sentence worth reading aloud: "Detectors cannot replace authority reserved to people" — the authority that law, policy, ethics or risk appetite has kept for a human being.^[1]

Read the third column downwards and you can see the four-rung ladder this table encodes — bounded autonomous → draft → act on named human approval → human decision only. What makes this ladder different from an ordinary policy is that it attaches to a "decision" rather than to a "system". The same system can therefore sit on rung one and rung four at the same time, depending on what it is doing — and that is least agency made concrete, with no need to argue about it in the abstract.

The column to fill in first is not the authority column; it is the stop condition. If the team cannot write the stop condition, it does not yet know what "wrong" means for that decision — and if you do not know what wrong looks like, every authority level you grant is a guess.

7. Workshop — The Contract and Attack Tabletop

The third workshop is the one I recommend actually running in a single room: two hours, every owner present. The playbook sets five scenarios: a benign request · an unsupported policy claim · an injected passage · an over-limit proposal · and a trace-write failure. For each one, complete a contract row and then walk all five rails, identifying the hard invariant, the soft signal, the threshold, the route, the evidence, the owner, the residual risk and the breach response — and then settle, before you leave, which failures block release and which require human escalation.^[1]

The fifth scenario is the one teams usually skip, and the one I insist on running early, because it answers the single most uncomfortable question: if the trace write fails, should the system carry on or stop? The playbook's answer is unambiguous and unconditional — the terminal trace is a hard control, and a failed write blocks release. An organisation that answers "let it keep going, we will collect the logs afterwards" is saying that its own assurance is optional.

To keep this workshop from becoming a session of imagining threats out of thin air, tie the five scenarios to an attack taxonomy somebody has already organised. The document I use is Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, NIST AI 100-2e2025, which as of 5 September 2026 is still the current final edition — published in March 2025, approved by the NIST Editorial Review Board on 20 March 2025, with a corrected file re-uploaded on 1 April 2025 and no superseding edition listed on its CSRC record.^[6]

The mapping that follows is mine, not one NIST has published, and NIST has in no way endorsed this playbook's five-rail architecture: the injected passage maps to indirect prompt injection in section 3.4 as an integrity violation · the trace-write failure maps to an availability violation in section 3.4.1 · the unsupported claim maps to an integrity violation in section 3.4.2 · and the over-limit proposal maps to the risks of tool-using agents in section 3.5.^[6]

The sentence I would like every architecture team to read word for word sits in section 3.4.4 — because current mitigations do not offer full protection against all attacker techniques, application designers may design systems on the assumption that prompt injection is possible if a model is exposed to untrusted input sources, for example by allowing models to interact with potentially untrustworthy data sources only through well-defined interfaces.^[6] That sentence is the entire justification for the Execution Rail: if we design on the assumption that injection succeeds, the question stops being "how do we filter it" and becomes "what can it then command", which is a question answered with permissions and transaction limits, not with a classifier.

The same document warns specifically about agents: because agents can take actions using tools, these attacks create additional risks in that context, such as hijacking an agent to execute arbitrary code or to exfiltrate data from the environment it operates in. And section 4.1.2 carries the most modest sentence in the whole publication — designing mitigations is an inherently ad hoc and fallible process.^[6] The boundary on this source must also be stated: NIST writes that the guidance remains voluntary and is not intended to serve as or supersede existing regulations, laws or other mandatory guidance.

Finally, once the table is filled in, tie each rail back to the risk codes your security team already knows, so that one document can carry a conversation with two audiences. This is how I map them — and again, this is the author's mapping, not one OWASP publishes.

8. Metrics That Matter, and the Failure Patterns

Before the table, let me set out this chapter's five operating principles as a whole set, because every metric below is one of them turned into a number. I quoted principle 2 in section 3; the other four are these.^[1]

1. State claims property by property — separate guarantees, estimates and governance duties
2. Put hard controls at release and effect boundaries — complete mediation is a prerequisite
3. Calibrate semantic gates — report threshold, population, false accepts, false rejects and correlated failure
4. Inspect state and traces — model narration is not evidence that an effect occurred correctly
5. Test beyond the visible suite — combine fixed, hidden, adaptive, stateful, failure and live evidence

The phrase most often skipped in principle 3 is "correlated failure". A great many teams stack several judges and then multiply their error probabilities together as though the judges were independent — when in practice those judges share a base model, share training data, and fail on the same cases. Three layers that fail simultaneously do not give you three times the confidence.

Principle 5, as it happens, has an independent piece of evidence supporting it. On 9 June 2026 NIST published a research summary reporting a mathematical proof that there is no finite set of guardrails that is universally robust against adversarial prompts, and recommending a shift towards continuous monitoring and updating rather than a fixed defensive posture — with NIST's own caveat that the approach will not completely solve the problem.^[7] Read it beside the specimen in section 4 and the same conclusion arrives from two directions: the visible test suite is never evidence about the invisible.

The metrics the playbook asks you to track are these, with a Scorecard column showing where each lands among the six columns we have used since the first post in the series.

The one rule the playbook sets in bold in its metrics section is never collapse utility and security into one score that hides the trade-off.^[1] The reason is pure arithmetic. A system that gets slightly better at routine work while letting one more prohibited effect through can post a higher combined score, even though it got worse on the dimension the organisation cannot accept. A single score does not make the decision easier; it only makes the wrong decision look reasoned.

Failure patterns

The playbook collects eight failure patterns. I have picked the seven I meet most often in the Thai organisations I am asked to look at, ordered from the hardest to fix.^[1]

• One score for utility and security — a combined number that improves while prohibited effects increase is a failure you cannot see until it is late, because the metric itself is what hides it
• Placing all protection at output — the last gate can filter text, but it cannot recall data a tool has already exposed, and it cannot stop money that has already moved
• Treating an allow-list as trustworthiness — a list of approved sources tells you where a document came from, not who edited the inside of it yesterday. An approved source can be poisoned
• Retry without idempotency — an upstream system that retries automatically turns one request into several records, and a key bound to the session instead of the order will not help at all
• Making the trace optional — both writing traces after release and letting the system continue when the trace write fails leave the assurance broken at exactly the point where it was needed most
• Approving narration instead of state — a reviewer reads the model's summary and clicks through, even though model narration is not evidence that the effect occurred correctly
• Claiming zero risk because the fixed suite found nothing — the suite we wrote is the suite we can see, zero in the visible suite is not zero in the world, and section 4 is the clearest evidence of this in the whole post

One more pattern the playbook names, which I want to leave as the closing warning, is suppressing escalation to make the automation number look better. That number does move, and it moves fast — but what actually happens is that the risk migrates from a queue of people into answers that have already been released, which is a far more expensive place for it to live.

9. The Road Ahead — From Control Rails to the Release Gate

If you take three things from this post into tomorrow, I would choose these. One — open the architecture of the system you already run and point to the line of code where the ceiling you promised your executives lives. If you cannot point to it within five minutes, it lives in the prompt. Two — build the bypass inventory from section 5 in full: batch, retry, webhook, cache, administrator tool, console, direct write, vendor fallback and recovery script, then decide path by path whether to mediate it or exclude it from the guarantee. Three — fill in the stop-condition column of section 6 before any other column.

What this post does not answer is how you would know these rails work before you release the system, and how you would watch them afterwards. Section 4 has already shown that a suite you write yourself has a ceiling — 0 of 40 in the visible suite alongside 8 of 12 released in the adaptive suite, both from the same system. That is the next post's work.