Three Control Classes, Five Rails — วางการ์ดครบทุกรอยต่อThree Control Classes at Five Rails — Hard, Soft, Governance

🤔 If you put an injection detector that is 95% accurate in front of your model and fix a deterministic rule — block whenever the score exceeds 0.8 — does your system now "have a guarantee"?

The previous post, Write the Assurance Contract, ended with one of the most valuable sheets of paper in the project: a per-property contract that separates which rows are structurally guaranteed (G), which are only estimates with error rates (E), who owns each row, and what evidence discharges it. But paper has never stopped an effect by itself. The question left open is purely mechanical: where on the path of a request must the mechanisms sit that make the G rows true — and how many kinds of mechanism do we actually have to place?

This whole post's answer comes from Section 7 of the paper^[1]: controls come in three kinds whose natures are completely different and which must never be conflated — hard enforcement gives invariants, soft detection gives fallible estimates with error rates, governance gives accountability and recovery — and all three are placed on the five rails of the request path: input, dialog, retrieval, execution, output, with governance crossing every rail. Three things must be accepted from the start: hard guards have a real footing at only three seams, every semantic verifier sits inside the attack surface, and the final product of all this rail-laying is declared residual risk — not a proof of safety.

1. Three Control Classes — and Why They Must Never Be Conflated

Guards in an AI-core system come in three kinds, and each kind gives something the others cannot substitute for: the first gives rules that cannot be violated within stated assumptions, the second gives fallible scores with measured error rates, the third gives the ability to notice and to reverse. Calling one kind by another kind's name is not linguistic sloppiness — it is writing a false claim into the very contract we spent the last post drafting.

A fixed threshold does not turn a fallible score into a reliable guard

Back to the opening question. Many teams believe that clamping a deterministic rule — "always block above 0.8" — onto a classifier's score produces a guarantee. The paper answers this more directly than anything else in the chapter: fixing the threshold makes the decision rule reproducible, while the decision stays wrong exactly as often as the score beneath it^[1]. What you gain from a fixed threshold is the ability to say "the system always decides by this rule" — genuinely valuable for an audit — but that is a different statement from "this rule decides correctly". The 0.79-scoring turn that is a real attack still slips through, and the 0.81-scoring turn that is an honest question still gets blocked. The only honest claim a soft detector supports therefore has exactly one shape: "catches …%, misses …%, measured on population …, at threshold …" — and no other shape.

Stacked detectors are not independent by construction

The next objection I hear often is "we stack three detectors, so the odds of all three missing at once are tiny" — and that multiplication of probabilities is valid only if the three detectors are independent, which the paper says is false by construction: model families share pretraining corpora, objectives, priors, and even the evaluator prompt templates. The result is correlated blind spots — an attack that fools one tends to fool the next for the same underlying reason^[1]. Independence is therefore something that must be measured — as joint false-negative behaviour on one shared adversarial set — never inferred from a change of vendor. Moving your judge from one provider to another does reduce risk in practice, but it does not establish independence until the joint false-negative number says it does.

A quality gate is a calibrated estimator, not a judge with authority

Once soft detection is accepted as measurement, the paper gives three disciplines for using it, which I reproduce in order^[1].

• Ground the metric — prefer checks that trace back to sources, such as span-level comparison of the answer against the retrieved passages, over unanchored preference scores tied to nothing at all
• Gate cheaply before judging expensively — cascade rule-based validation (schema, format, value ranges) ahead of model-graded judging, which is slower and costlier, and send only the residue onward
• Judge the judges — model-based evaluators carry position bias, verbosity bias, and self-enhancement bias, so calibrate them against human labels, and where possible use a judge from a different model family — without forgetting the previous point: a family change does not establish independence

💡 My view: almost everything sold under the name "guardrails" today is pure soft detection — a classifier plus a threshold plus a dashboard — and a great many teams finish the installation and report upward that the system "now has guarantees". The one question I use to open up such a report is: if the detector's score is wrong, what stops the effect? If the answer is "nothing", that organisation does not yet own a single hard guard. It owns measuring instruments dressed up as fences.

2. Five Rails on the Envelope — and an Honest Threat Model

With the three classes in hand, the question is where to place them. The paper draws the assurance envelope as a four-stage passage: assemble and screen the input → the probabilistic core generates → the verification and routing layer (hard plus soft) → a human takes responsibility for the residue, with control rails at every seam as defence in depth — and one property that must be read precisely: the system is deterministic only in its control flow. Every decision's route can always be traced back; the decision itself is not always right^[1].

The table that follows is the heart of this post — the five rails plus the governance layer crossing all of them. Each row states the control class, the property protected, the signal and decision rule, the fallback, and the rightmost column that matters most: what that rail cannot see by design, which the paper forces into the open instead of leaving the reader to guess^[1].

Read the "Class" column top to bottom and you see the fact this post is named after: hard invariants on the system's content exist at exactly three seams — retrieval provenance, execution, and output structure — plus one further hard condition on the trace write. Everything else is estimation plus governance. And the paper has empirical evidence on its own specimen that this dividing line is real rather than rhetorical: attacks authored after reading the implementation pushed policy-violating candidates through the soft rails in 8 of 12 cases, while the execution boundary stopped every prohibited effect — 0 of 4, against 4 of 4 with the envelope removed. Those numbers are conformance results on deterministic fixtures the author wrote himself — not field safety, and not the quality of any model^[1]. The full story is the next post's subject.

The threat model: assets, a boundary that has already collapsed, and the attacker

Rails end up in the right places only when you know what you are protecting from whom. The paper writes the AI-core threat model plainly and completely, starting with three assets^[1].

• Effect authority — the right to invoke tools that change the real world: moving money, editing orders, sending mail
• Data — the retrievable corpus, the session's memory, and every credential any tool can reach
• User trust — every answer goes out under the name of the system and the organisation behind it

On the trust boundary the paper is blunter than most security documents: in an AI-core system this boundary collapses into a single channel — system instructions, user messages and retrieved content all flow into one context window, so privilege flows with content. Anyone who can get text in front of the core has a chance of borrowing the core's authority. This is the textbook confused-deputy structure, and it is not a defect of any one implementation but a property of the architecture — one that Greshake and colleagues demonstrated against real deployed applications^[4]. The attacker's profile then fits in four lines.

• Can — plant content the core will read later in places the system retrieves from on its own: reviews, emails, web pages, documents — this is indirect prompt injection
• Can — craft adversarial conversation turns directly, as an ordinary user of the system
• Can — adaptively re-optimise the attack against externally observable signals: refusal messages, fallback behaviour, answer patterns
• Cannot — alter the model's weights, alter the orchestration code, or read traces already logged

The verifier is inside the threat surface

And this is the sentence I consider the most important in the whole threat model: a model-based verifier sits inside the attack surface, not outside it. Any gate that decides with a model — an injection classifier, an LLM judge, a semantic policy model — must read text the attacker has influence over, and so inherits the same instruction/data confusion as the core it is trying to protect^[1]. The field evidence points the same way: detector-based defences held attack success near 8% under AgentDojo's original attack suite^[2], while adaptive attacks that tune themselves against the defence significantly degraded the defences of that era^[3]. The paper's engineering conclusion is correspondingly strong: model-based verdicts are evidence, not authority — effect-bounding must rest on the mechanism at the execution rail, never on the judgement of any model — and the sum of the whole envelope is nameable residual risk, not a proof.

💡 My view: the mnemonic I teach is a single sentence — hard guards do not read; soft guards read, and can be lied to by what they read. A genuinely hard mechanism decides on structural properties the attacker cannot overwrite (identity, schema, limits, allow-list membership). The second any guard starts "reading" natural language in order to decide, it changes sides into the attack surface — and from that second on it must be treated as an estimate.

3. The Seven Steps

These seven steps are in the order you should actually work: threat model first (because it tells you which rails matter), then the rails laid along the request path left to right, closing with the governance layer across all of them. Every step ends with one beat of Nong Kram, the customer-support assistant for the KramKraft ceramics shop that we have been building since post #1.

Step 1 — Write the threat model before placing a single guard

Write a short document with three headings: assets (effect authority, data, trust), the boundaries where untrusted content enters the system, and the attacker's capabilities together with their budget — how many tries they get, which signals they can observe from outside, how fast they can adapt. Why this step comes first: rails placed without a threat model protect what is easy to protect, not what is valuable, and every residual-risk cell in step 7 must trace back to this document. NIST CAISI's work on agent-hijacking evaluations makes the same point: an evaluation that does not state the attacker's access and budget cannot be interpreted^[6].

# Nong Kram's threat-model.md (abridged — this step's artifact)
assets:
  effect_authority: refund(order_id, amount, reason)   # real money leaves the shop
  data: policy/ + catalog/ (pinned snapshots), session memory, order store
  user_trust: every answer goes out in KramKraft's name
attacker can:
  - plant text in product reviews, emails, or web pages the system will later retrieve
  - craft adversarial conversation turns directly, and retry while watching the responses
attacker cannot:
  - alter weights, alter orchestration code, or read traces already written
attacker budget: free customer accounts, unlimited turns — the system is public

Step 2 — Place the input rail

Put two classifiers on the user turn — one for scope, one for prompt injection — at a threshold declared in writing, together with the measurement population and both error rates. This is a soft guard by nature: the fallback is a polite refusal or a request to re-phrase, and every raw score goes into the trace so the threshold can be re-tuned later. Do not forget the limitation the table declares: this rail sees only the user's turn — instructions riding in on a passage retrieved later are invisible to it by design. For Nong Kram: the tutorial's rule is block when the injection score exceeds 0.85 — measured on a self-labelled set of 300 Thai-language support turns (all numbers here are the tutorial's, not the paper's) — and the refusal message says briefly that Nong Kram can only help with the shop's orders and products.

Step 3 — Place the dialog rail

Check policy conformance across turns, not on single turns in isolation — many attacks spread themselves over several messages that each look innocent. The paper's key rule: this rail can be a hard invariant only where the permitted flow is finitely enumerable; everywhere else it is an estimated score. For Nong Kram: the refund flow is compressed into a four-state machine — identify the order → check eligibility against policy → summarise amount and reason → confirm — and that sequence is enumerable, so skipping states can be enforced hard. General conversation outside that flow gets an estimated policy-conformance score, and when the score stays low for two consecutive turns the system steers back on topic or offers a hand-over to the admin.

Step 4 — Place the retrieval rail

This rail carries two stacked guards that must not be confused: provenance is hard — every passage entering the context must come from a source that is a member of the allow-list, enforced in code on a path with complete mediation — while support/relevance is soft — a score for whether the passage actually supports the question. Fallbacks run from light to heavy: drop that passage, answer without it, or abstain entirely. For Nong Kram: the allow-list is policy/@snapshot-2026-09-01 and catalog/@snapshot-2026-09-01, those two sources only — customer reviews, being content outsiders can write, are deliberately not on the allow-list — and any passage scoring below 0.6 support (the tutorial's threshold) is dropped before context assembly, always. But write the table's rightmost cell on your heart: a passage from policy/ itself that is accurate in every word yet has a command sentence embedded in it passes this rail — which is exactly why step 5 must exist.

Step 5 — Place the execution rail — your one hard effect bound

This is the most important rail in the whole system, because it is the only one that can hard-bound effects: every tool call must pass deny-by-default authorisation, schema and parameter-range validation, and transaction bounds before anything happens to the outside world — and it must decide on structural properties only, reading not one word of the model's explanation. Nong Kram's entire guard looks like this (the 1,500 THB cap is a tutorial value).

# Nong Kram's execution guard — deny by default
ALLOWED_TOOLS = {"refund"}                    # the only tool the system may call

def authorise(call, session, policy, store):
    if call.tool not in ALLOWED_TOOLS:
        return Verdict.REJECT("tool_not_allowed")     # not on the list = refused
    errors = REFUND_SCHEMA.validate(call.args)        # structure passes first
    if errors:
        return Verdict.REJECT("schema_invalid", errors)
    order = store.get(call.args["order_id"])
    if order is None or order.customer_id != session.customer_id:
        return Verdict.REJECT("not_order_owner")      # bound to the verified identity
    amount = call.args["amount"]
    if amount <= 0 or amount > order.paid_amount:
        return Verdict.REJECT("amount_out_of_range")  # never above what was paid
    if amount > policy.refund_cap:                    # per-transaction cap: 1,500 THB
        return Verdict.NEEDS_APPROVAL("over_cap")     # hard to reverse → human approves
    return Verdict.ALLOW                              # all checks passed, so allow

Notice that no line of the guard cares how beautifully the model explained this refund — and remember this rail's limit too: a call that is well-formed in every respect but serves an attacker's goal (a real customer talked into refunding to someone else's account) is one this rail cannot judge. That is work for the dialog rail and for the human at the NEEDS_APPROVAL step.

Step 6 — Place the output rail

Before any answer is released: the schema parse is a hard guard — a payload that fails to parse against the declared schema never leaves — while faithfulness, toxicity and privacy are soft guards with declared thresholds. The fallback is a ladder with a bounded number of rungs: repair → regenerate → withhold → escalate, never an endless loop. For Nong Kram: every turn's answer must first parse against the schema reply{text, order_refs[], refund_summary?}, then pass faithfulness against the retrieved passages (tutorial threshold 0.7) and a scan for card numbers, phone numbers, and other customers' addresses. Fail any check → repair once → regenerate once → still failing, withhold and send to the admin. And do not forget what this rail cannot do: data that a preceding tool call has already sent out cannot be recalled here — exfiltration is prevented at step 5, not at this rail.

Step 7 — Wire governance across every rail, then fill the residual-risk column honestly

Two final jobs. The first is code: make the trace write a hard condition of release — the release path stays blocked until the trace commits. This is the mechanical meaning of fail-closed for evidence. The second is documentation: review, rollback via the release manifest, incident response with a named owner, and — the part most teams skip — write the residual-risk cell of every rail honestly, copied from the "what this rail does not cover" column of the table in section 2, never written as "none".

# Nong Kram's release path — the trace is a hard condition
def execute_route(release_decision, payload, trace):
    trace_id = trace_store.commit(trace)      # always write the trace before release
    if trace_id is None:                      # write failed = release nothing at all
        return Route.FAIL_CLOSED              # fail closed — no evidence, no effect
    return deliver(payload, trace_id)         # release, carrying its evidence id

4. Nong Kram's Rail Table

This post's artifact is the single table that consolidates the seven steps — it is the seam register we built in post #3, The AI-OS Mental Model, promoted from "a list of places where classical guarantees break" to "a control plan stating each seam's guard, threshold, fallback and residual risk". Every number in the table is a tutorial number we measured or set ourselves for Nong Kram, not a number from the paper.

The table also does one more job: it says which rows of the assurance contract written in post #7 each rail discharges — because a contract with no mechanism behind it is only paper.

• Execution rail → closes the Authorised effects row (G) in full — this is the only mechanism that supports that row's G
• Output rail (schema side) → closes the Output structure row (G); the semantic side supplies evidence for the Semantic quality row (E)
• Retrieval rail → hard provenance supports the G side of the Injection containment row together with execution; the support score supplies evidence for Semantic quality (E)
• Input rail → supplies evidence for the detection side (E) of the Injection containment row — and must never claim more
• Dialog rail → supplies evidence for Task correctness (E) in its cross-turn conformance dimension; the state-machine part is a hard invariant only at that spot
• Governance across all rails → closes the Change control row (G) and the Auditable condition — complete traces on every path are its evidence

5. Validation Check — Audit Your Own Rails

Before moving on, answer these six against your own system. The series' standing rule applies: every answer must point at a tangible artifact, not an adjective^[1].

6. The Road Ahead

If this post had to shrink to one sentence, I would pick this one: put hard invariants at the seams that admit them, measure semantic properties as fallible estimates, and let governance do its own job — noticing early and reversing — and never let one class wear another's name. Nong Kram's system now has a guard at every seam, a threat model that admits the verifier can be attacked, and a table that ties every rail back to the previous post's contract.

What this post has not done at all is prove anything. We have run nothing yet: does the complete envelope route benign and fault-injected cases as specified? What escapes when each control is removed, one at a time? How far do attacks written after reading our rules get? And do a duplicate refund and a tool that dies mid-flight actually fail closed? All of that is the harness of the next post, Prove It — Golden Sets, Ablation, and Adaptive Attacks, where the paper's 517 executions get unpacked table by table.