When Is AI the Core? — อำนาจ × ความขาดไม่ได้ และ Context ในฐานะโปรแกรมWhen Is AI the Core? — Authority × Indispensability, and Context as Program

🤔 One and the same product can contain AI-assisted coding, a coding agent, AI-enabled software and an AI-core capability, all four at once — so which part of it actually needs the full assurance envelope?

The last five posts in this series were about redesigning the organization, closing with Federated by Design — Who Sets Standards, Builds, Approves and Operates, which ended on the division of roles: who lays the central rails, who builds on them, who approves, and who carries the outcome in the value stream. Now we cross into the third part of the series — Engineer · engineering the core — and the first engineering question is not "how do we build it" but "what is the thing we are about to build, in systems terms", because the answer to that question is what sets the size of the control budget.

This chapter's answer is shorter and sharper than most teams expect: AI-core is not a label you pin on a whole organization or a whole product. It is a class of runtime, defined by two axes — the model's decision authority and the model's indispensability — for one task set declared in advance — and you prove the second axis by actually removing the model and measuring, not by believing.

1. An AI-core system is not "software that calls a model"

Chapter 7 of the playbook this series follows^[1] opens with a single sentence that serves as both definition and warning — "AI-core is a task scoped runtime class defined by decision authority and model indispensability then governed according to consequence." Read it slowly and you will see it refusing the way most people use the term, on three separate counts: it is a runtime class, not the name of an architecture; it comes with a task scope attached, not an enterprise-level badge; and its governance is set by consequence, not by technical sophistication.

The chapter's English companion text spells this out even more directly, and I quote the whole sentence because every word in it is doing work:

An AI-core system is not simply software that calls a model. It is a scoped runtime class in which model output controls an important share of released content, action, or program branch, and a predeclared removal test shows that the model is indispensable to the specified task set. Classification belongs to a task, manifest, golden set, and threshold, not to a vendor, model family, or whole organization.^[1]

That sentence sets two conditions which must both hold at the same time before a system may be called AI-core. The first is that model output controls an important share of released content, action, or program branch — note that it does not say "the model is called", it says "controls": the model's output is what decides what reaches a real person, or which way the system goes next. The second is that a predeclared removal test shows the model is indispensable to the specified task set — and "predeclared" is the heart of it, because if you run the test first and set the threshold afterwards, you have not tested anything. You have merely described a result.

I meet a great many teams that clear the first condition and have never touched the second. A summarization system that pushes text straight out to customers clears the first one comfortably. But ask "if we removed the model today and let the declared fallback take over, would the declared task still succeed under your definition of task success — and have you ever measured it?" and the answer is silence. A system like that may not be AI-core at all — it may be an AI feature that nobody has ever switched off to check. The difference is expensive, because it moves the entire control budget.

Four ideas that get collapsed together until they mean nothing

The next paragraph in the chapter does the most valuable work on the page: it separates four terms that practitioners routinely use interchangeably until nobody can follow the conversation — "This separates four ideas. AI-assisted coding changes how software is developed. A coding agent performs delegated repository work. AI-enabled software contains a model-backed feature. An AI-core system makes model-generated behavior load-bearing at runtime."^[1]

The closing line of that paragraph is what stops the table above from being an academic taxonomy — "One product can contain an optional summarizer, an indispensable policy advisor, and an AI-core routing function."^[1] A single product can hold all three at once. So the right question is never "is our system AI-core?" but "which capability of our system is AI-core, which one is not, and on what grounds".

Two practical consequences follow. First, an enterprise AI register with a "type" column filled in by vendor name is useless; it has to be filled in per task set and per manifest. Second, a classification has a natural expiry date — if the task, authority, fallback, tools, thresholds or manifest change, the old classification no longer binds anything. That is exactly why the card in section 3 ends with a field called "reclassification triggers".

It is worth saying up front that the whole conceptual frame of this chapter — the two-axis classification, context as program, and the manifest — comes from an academic paper by the playbook's author that is unpublished and has no public URL.^[2] It is therefore one author's synthesis: not an industry standard, not a certification scheme, and not a consensus of the field. I think it is an unusually workable frame, but if you carry it into a governance document, cite it for what it is.

2. The 2×2 — decision authority × indispensability, and four cells that need different things

Once you know there are two conditions, the most direct way to see them is to lay them out as two axes and count the cases. The horizontal axis is decision authority — how far model output determines what is released, from "it drafts something for a human who decides" all the way to "the model's output is what reaches the customer, with nobody reading it line by line". The vertical axis is indispensability — if you removed the model and let the declared fallback run instead, would the declared task set still meet its success criterion?

The two axes really are orthogonal, and that is the main value of this picture. Most teams quietly assume they are the same thing — "if the model matters a lot, it must have a lot of authority" — which is not true at all. An internal policy advisor the legal team leans on daily, to the point of being indispensable, may have no power to act on anything beyond releasing text. Meanwhile an agent that presses buttons in an external system on its own may be removable this afternoon without anybody breaking stride, because the team can still do the same work by hand.

That rightmost column is the entire reason for separating the axes: these four cells call for different kinds of work, not for the same kind of work in different amounts.

• The indispensable-advisor cell — when the effect boundary is text release, all the control work concentrates at the point of release: can every statement cite its provenance, can the system refuse when the evidence is thin, and is there a route to a human when the question falls outside scope? Heavy execution guards in this cell are investment in the wrong place, because there is no effect to gate.
• The bounded-agent cell — the exact opposite. The system is removable and the business-continuity impact is low, but it can press buttons. So all the work sits in effect mediation: permissions bound to subject, purpose, amount, duration and transaction, and every action passing through a mediator you can inspect.
• The AI-feature cell — proportionate controls, which is not the same as casual controls. This line is habitually misread as "you need not do anything", when it means "do what the consequence demands, and record why that was enough".
• The AI-core cell — it needs everything above at once, plus the two things the other three cells do not: versioning the whole behaviour surface, and a reconstructable trace at every step the system walks through.

What I most want to underline, from reviewing real systems, is that these cells move, and they usually move without anyone announcing it. A system that sat in the AI-feature cell last year may be in the AI-core cell today, because one more tool got wired in, or because the human team that used to check every item was cut back until it could only sample. Nobody signed off on that move between cells. It happened because both axes crept upward a little at a time until they crossed together. This is the pattern post #4 called authority debt, and this diagram is where it becomes most visible.

3. The ten-field classification card, and a completed CX-REFUND-01

The 2×2 is a thinking tool; the thing you can actually take into a meeting is a single card. Appendix B of the playbook supplies the AI-core use-case classification card as a copy-ready form,^[1] with all three of that appendix's standard header lines filled in — Purpose: classify one released capability by two independent axes, and assess consequence separately · Use when: selecting a use case, and whenever task, authority, fallback, tools, thresholds or the release manifest change · Accountable owner: business or product owner, with the release owner verifying the evidence.

The word "separately" in that purpose line is a detail I want you to notice. Consequence is not a third axis of the 2×2. It is a distinct body of evidence, assessed on its own and then laid alongside. Folding it into the classification would produce the single most dangerous artefact in governance work: one composite score in which severity can offset authority, so that two pieces of work needing entirely different controls come out numerically equal.

The copy-ready card — ten fields, in order

The three fields teams most often cannot fill in are 4, 5 and 6 — which are the same three fields that give the indispensability axis any meaning. Without a declared fallback you have nothing to compare against. Without a golden set and a definition of task success you cannot measure the comparison. And without a threshold declared before the run, the result is just a number you can interpret in whichever direction you prefer. I have seen a card with field 7 confidently filled in as "AI-core" while the three fields before it were blank. That is not a classification. That is a feeling.

The completed example — CX-REFUND-01

Every example in Appendix B uses the same case: CX-REFUND-01, the Luma Commerce Thailand assistant (a fictional case from the playbook), which answers refund questions in two languages and may propose issue_refund, with an external guard permitted to execute one eligible refund of no more than THB 2,000 after confirmation; every other financial action goes to a person. The playbook attaches its own caveat in one sentence: "Values are illustrative, not universal thresholds."^[1]

One thing to read carefully: the blank card has ten fields, but the worked example in the book prints as seven rows — Scope, Authority, Fallback, Ablation, Class, Consequence and Decision.^[1] Those seven rows merge related fields for readability on the page; they do not reduce the number of fields you have to fill in. If you build your own card and start from seven rows, you will find there is nowhere to put "reclassification triggers" — the field with the longest useful life on the whole card.

The Ablation row is the one I would like every team to run for real at least once before reading the next post. It forces four things simultaneously: one, you must have a data set that is already sliced; two, you must have a fallback that actually runs, not one that exists in a document; three, you must declare the threshold first; and four, both sides must run on the same set. That last condition, "on the same set", is where most of these tests fall apart, because teams habitually compare the model's results on last month's live traffic against a fallback estimated from what the team remembers about two years ago.

4. Software 1.0, 2.0 and 3.0 live in the same call path

Once you can tell what the core is, the next question is what that core is assembled from. The playbook answers with what I consider the most operationally usable sentence in the chapter — "Software 1.0, 2.0, and 3.0 remain in the same call path. Explicit code owns interfaces, permissions, control flow, transactions, and effect mediation. Learned components contribute statistical capability. Runtime context supplies goals, examples, retrieved knowledge, tools, and state."^[1]

The phrase Software 1.0, 2.0 and 3.0 is now so widely used that many people have forgotten where it came from, and the playbook itself uses it without attribution. So let me supply the attribution as an addition of this article's own, not the book's: Software 1.0 and 2.0 come from Andrej Karpathy's essay Software 2.0, dated 11 November 2017,^[3] which defines 1.0 as "explicit instructions to the computer written by a programmer" and 2.0 as written in a language humans cannot read, "such as the weights of a neural network". The term Software 3.0 appears nowhere in that 2017 essay — not once. It comes from the keynote Software Is Changing (Again), delivered on 17 June 2025, which Y Combinator itself summarizes as the era in which "natural language becomes the new programming interface".^[4]

Both sources are an opinion essay and a recorded talk, not controlled research. I cite them for the provenance of the vocabulary only, not as evidence that any proposition is right or wrong.

What the playbook adds, and what is worth more than the vocabulary, is the insistence that the three do not replace one another: they sit in the same call path, and each owns something the others cannot take over.

💡 My view: operating principle 2 of this chapter is the shortest one written and the one that prevents the most damage — "Keep control and effects in code — models propose while services authorize, validate, transact and release." I like that it is written as a sequence of four verbs, because that makes it checkable on the spot in an architecture review: show me the line where authorize happens, the line where validate happens, where transact happens, and where release happens. If all four answers live inside the prompt, you do not have a system. You have a polite request.

The operating-system analogy — offered and broken in the same breath

The 2025 keynote mentioned above proposes the picture of an LLM as an operating system,^[4] which is a useful shortcut to the overall shape. The playbook takes that picture up and then breaks it in the very next sentence — "The operating-system analogy helps, but context has no true memory protection, retrieval has no stable read contract without versioning, and a natural-language tool request is not an authorized system call."^[1] I think that sentence deserves to be pinned to the wall of every architecture room, because those three fracture points are the origin of very nearly every real incident I have seen.

That third fracture point is proposal–effect separation, an idea that returns as the protagonist of post #13 — and it is not remotely new in security engineering. It is the principle of complete mediation, which Saltzer and Schroeder stated back in 1975 as "Every access to every object must be checked for authority."^[5] It travels with least privilege, that every program and every user should operate with the smallest set of privileges the job requires, and fail-safe defaults, that access decisions should be based on permission rather than exclusion.

I like to raise this 1975 paper in AI meetings because it changes the register of the conversation immediately. What we are discussing is not a novel worry of the LLM era; it is a principle the computing field settled in 1975 — what LLMs changed is that the requester now speaks human language extremely persuasively, until people feel there is nothing left to check. That feeling has to be resisted at every point where a real effect occurs. (The original paper sits behind IEEE's paywall, so I cite the DOI rather than a copy.)

5. Context as program — behaviour comes from the whole assembly

If the last section said who owns what, this one says what adds up to the system's "behaviour" — and the answer is longer than most architecture documents are willing to admit: "Behavior therefore depends on the assembly: model version, decoding, context, retrieval, tool semantics, orchestration, memory, and environment."^[1]

The idea of context as program comes down to one sentence, which is the normative line of the whole chapter: "A behavior-relevant change to any component is a program change and should trigger a new manifest and evidence review."^[1] — any change at all that bears on behaviour, in any component at all, is a program change, and it must produce a new manifest along with a review of the evidence.

That sounds needlessly strict, right up until you walk down the components one at a time and ask who in your organization can edit each of them, and through what process.

The rows I would have you dwell on longest are System context and Retrieval, because those are the two places where organizational change-control discipline fails most quietly. Organizations with three levels of approval for a code edit routinely let the prompt be edited on a settings screen by someone who needs nobody's sign-off, and let new documents be added to the corpus every week as "content work" rather than "release work". Both of those change the behaviour of the running system more than many code edits do.

I once saw a very clear case of this. A system's answers shifted materially over the course of a week. The engineering team insisted there had been no deploy, which was true — what had changed was that the content team had added a revised policy document to the corpus without removing the old one. The system now retrieved both and picked the one with the higher similarity score, which happened to be the old one. Under "context as program", that event is a code deploy that changed behaviour, with no manifest, no evidence review, and no identifiable way back.

6. The runtime-context manifest — six domains in one release

The playbook's answer to the problem above is the runtime-context manifest, the fourth instrument in Appendix B^[1] — Purpose: pin the whole behaviour-determining release, not only the model, with the manifest declaring how context should be assembled and the trace recording what one request actually received · Use when: any model, decoding, prompt, corpus, retrieval, tool, control, threshold, evaluator or orchestration component changes · Accountable owner: the release owner signs the integrated manifest, and component owners attest their own entries.

That distinction in the first line, between manifest and trace, matters more than it looks. The manifest is a declaration; the trace is a record of events. An organization with logs but no manifest can say what happened but not what should have happened, so it cannot prove whether the system deviated from what was declared. An organization with a manifest but no traces can declare beautifully, with nothing to confirm that reality matched.

The copy-ready manifest — six domains

The worked example — and why it has five rows, not six

The book's CX-REFUND-01 example merges Evaluation and trace with Operations and integrity into a single row headed "Evaluation and operations", so the example prints as five rows even though the copy-ready manifest has six domains.^[1] I have chosen not to "fix" that back into six rows, because the merge is a fact of the printed page you will see if you open the book, and because it reflects what really happens in teams — the owner of the test suites and the owner of the rollout plan are often the same person, until people forget these are two domains that have to be attested separately.

One small detail worth pointing out is the test-suite id. In the classification card in section 3 the same suite is written CXGS-2026-09-v3, while in the manifest it is written CXGS-v3. The playbook uses two forms in two instruments, and I have kept both as they stand rather than normalizing them — because that is itself a small lesson from this chapter: if the same thing carries different identifiers in two documents, automated checking cannot match them, and "manifest resolution" in the metrics table in section 8 will fail for a reason nobody thinks of as a reason.

For organizations that have to explain all this to an external auditor, there is one internationally recognized anchor. NIST AI 600-1, the Generative AI Profile of the AI RMF, published on 26 July 2024,^[6] sets out action MS-2.8-003, which calls for content-transparency tooling that documents each instance where content is generated, modified or shared, so as to give a tamper-proof and traceable history, and adds that "Robust version control systems can also be applied to track changes across the AI lifecycle over time." Put simply: versioning the whole behaviour surface does not conflict with the international frameworks — it is a more detailed, concrete form of what they already ask for. Bearing in mind, always, that NIST AI 600-1 is a voluntary framework whose own scope statement says organizations must select and tailor the actions to their own use case and risk tolerance.

7. Four layers, and the boundary-walk workshop

The chapter's reference architecture has four layers, plus a set of controls that crosses all of them.^[1]

• Layer 1 — Governed data foundation, supplying the authoritative records the organization treats as true, and versioned knowledge — if this layer has no versions, nothing above it can genuinely be versioned either
• Layer 2 — AI-core layer, combining the model, context assembly, retrieval and bounded memory — this is the layer where "capability" lives, and the layer that cannot offer structural guarantees
• Layer 3 — Orchestration, coordinating requests, specialist components, tools and state — it owns sequence and route, and therefore owns the trace as well
• Layer 4 — Applications and human operations, exposing outcomes and handling the residue — "the residue" is the word most teams skip past, though it means every case the system refuses, escalates or fails to complete, and that is always human work
• Crossing all four — Assurance controls, which are not a fifth layer but a cut across every layer, the same pattern as the "spine" that post #1 laid down at the start of this series

To show how those four layers behave inside a single request, the playbook walks CX-REFUND-01 step by step: an authenticated request enters the context assembler; the system retrieves only approved policy passages and reads current order state through bounded services; the model may draft an answer, select a route, or propose issue_refund(amount, reason) — and then comes the most important sentence in the whole passage: "The proposal is not the effect." What produces the effect is the execution guard, which checks identity, allow-list, schema, customer binding, amount, approval token, transaction budget and idempotency. After a tool result the system reassembles context, and only a structurally valid, sufficiently supported response reaches release; otherwise it is repaired once, withheld, or escalated.^[1]

The five operating principles

The chapter closes on five operating principles, which I give here in full and in the book's order (number 2 is the one I already expanded in section 4).

1. Scope AI-core behavior explicitly — declare task, authority, fallback, golden set and indispensability threshold
2. Keep control and effects in code — models propose while services authorize, validate, transact and release
3. Version the whole behavior surface — bind model, context, corpus, tools, policy, evaluators and thresholds
4. Grant minimum capability per request — bind tools to subject, purpose, amount, duration and transaction
5. Design fallback as a primary path — open-ended tasks often fall back to refusal, degradation or escalation

Principle 4 is the concrete form of what the playbook calls least agency, and principle 5 is the one teams overlook most often — "fallback as a primary path" does not mean the fallback is a backup plan. It means that for open-ended tasks, the route the system takes most often ought to be refusal, degradation or escalation, and those routes have to be designed as well as the successful one is, rather than left as an error message someone wrote at three in the morning.

The boundary-walk workshop — six cases × six questions

The chapter's practical instrument is to walk six cases through the architecture one transition at a time, asking the same six questions every time. The required output is a boundary inventory and the removal of every path that releases content or commits an effect without mediation.^[1]

The six questions asked again at every transition are these — (1) what crosses this boundary · (2) what is trusted · (3) what is estimated · (4) which hard invariant applies · (5) which trace proves this route · (6) who handles failure. Questions 2 and 3 always travel as a pair, and that pair is what separates a structural guarantee from a semantic estimate — the dividing line on which post #12 will build an entire contract.

The injected document case is the one I would have every team walk first rather than third, because it is the only case in which an adversary designs the input on purpose. NIST AI 600-1 files this under Information Security and separates the direct form from the indirect one — "Indirect prompt injection attacks occur when adversaries remotely… exploit LLM-integrated applications by injecting prompts into data likely to be retrieved."^[6] The phrase "data likely to be retrieved" is the heart of it, because it means the attack surface is not the chat window: it is every document your system might retrieve, including the ones your customers upload themselves.

Worth knowing alongside that: NIST AI 600-1 enumerates 12 risk categories unique to or exacerbated by generative AI,^[6] with the document's own scope statement noting that some of those risks remain unknown and are therefore hard to scope or evaluate properly, and that organizations must select and tailor the actions to their own use case and risk tolerance. I will say again that it is a voluntary framework, not a mandate, and that this article offers no legal opinion of any kind.

The output of the workshop is not a report but a boundary inventory — the full list of boundary crossings with all six answers for each — and there is exactly one success condition: no path remains that releases content or commits a real effect without mediation. If you finish all six cases and such a path is still there, the next task is not to write the report. It is to delete that path before the release.

8. Metrics that matter, and failure patterns

The playbook states what to monitor as one long list, running from mediated-path coverage all the way to authoritative post-state accuracy.^[1] I have turned it into a table with the failure signal for each, and tied each to a column of the six-column scorecard that post #1 laid out — with the caveat, stated first, that all of this is what to monitor, not what to target, and that the illustrative values from CX-REFUND-01 in section 3 must never be carried into this table as goals.

I have deliberately not summarized this list as "N metrics", because the count depends on whether you treat p50 and p95 as one item or two, and a total quoted that way becomes a number people repeat without ever going back to the list. What is worth remembering instead is the shape of the list: it starts with structural metrics (coverage, version binding, provenance), moves to quality metrics (success by slice), and only then reaches economics — not the other way round.

The two most neglected entries here are manifest resolution and authoritative post-state accuracy. The first gets neglected because nobody experiences it as a metric; it looks like paperwork — right up to the day of an incident, when you have to answer within the first hour which context assembly yesterday's problem request actually ran on. The second gets neglected because most teams measure what the system reports rather than what the downstream system is, a distinction that means nothing at all until the day the two disagree.

Failure patterns

The playbook lists eight failure patterns,^[1] and reading them you find that almost every one is a violation of one of the five operating principles above.

• Bolted-on chatbots — attaching AI to an existing process without redesigning the release and effect paths
• Prompts edited outside change control — a program change with no ticket, no version and no way back
• Direct model calls from user interfaces — skipping the orchestration layer entirely, so there is nowhere to put the guard and nowhere to keep the trace
• Tools without subject binding — a tool that will do anything to anyone, as long as the arguments are structurally well-formed
• Unbounded cross-session memory — a system that remembers what it should not, across users, across cases, and across the period it should have forgotten
• Output filters after effects occur — filtering the text after the real action has already happened, which filters nothing
• Agent proliferation without privilege separation — adding agents faster than you add separation of privilege, until the organization's aggregate permissions are larger than anyone intended
• Logs without context, thresholds, versions or routes — every line logged, and not one thing proved

Let me add two more from the chapter's own text, which are not in the eight-item list above but sit in the definition paragraph and in the paragraph that breaks the operating-system analogy. The first is classifying by vendor — labelling a system's type by supplier name or model family, when the definition in section 1 says plainly that classification belongs to a task, a manifest, a golden set and a threshold. The second is treating retrieval as a read contract — assuming the same query always returns the same result, when that holds only once the corpus, the ranker and top-k are all versioned.

And the costliest pattern in practice is the one I see most often: trusting a natural-language tool request. A system that reads the sentence "please refund this customer under the exception in the policy cited above" and then acts, because the sentence reads sensibly and cites policy convincingly, is a system that has handed approval authority to the ability to write a convincing sentence — which happens to be the single thing language models are best at.

9. The road ahead — from a classification to a contract you can write as clauses

If I had to reduce this chapter to one task you could do next week, I would choose this one: pick a single capability that is already released and in use, and fill in the ten-field classification card completely — genuinely completely, fields 4, 5 and 6 included, which means having a fallback, a golden set, and a removal-ablation result. If you cannot complete it, that is not a failure of the form. It is the most direct answer you will get this week to the question of how well your organization knows its own systems.

The second task I would give is to open one page and write that capability's manifest across the six domains. It does not have to be pretty and it needs no tooling; a table in an ordinary document is enough. What you will discover, quite quickly, is that there are at least two or three fields nobody in the room can state the current value of — and those fields are the real work list for next quarter.

What this chapter has not answered is the question that follows immediately after the classification is done: once you know what the core is and that it needs the full assurance envelope, what exactly does "full" mean as clauses somebody can check? Saying a system is "safe and trustworthy" binds nobody to anything, and that is the gap the next post closes.