Quality Gate — แยกด่านคุณภาพให้ชัดBuild a Separate Quality Gate

Part 9 of twelve. Separate usefulness and clarity from schema and grounding; no strong rubric value cancels a hard constraint.

Default decisions and draft text are authored mocks, not live Jev results. Use the tutorial with the matching companion code, tests and exercise linked to a fixed GitHub version.

Outcome and prerequisites

Use Node22.18+ and a terminal. Clone the GitHub companion using its README, or download the complete ZIP, extract it and enter ai-core-functional-lab. No npm install, API key or network inference is needed. The reading-time label excludes about25–45 minutes of hands-on work. Keep one reference copy and make exercises in another.

Download companion ZIP · README / lesson map

This part focuses on src/quality.mjs. Expected gate states describe content and continuation; no state sends a message or changes a real account.

Old functions and the new boundary

A response can be well-formed and grounded yet fail to help the user. The quality gate therefore checks a separate acceptance rubric: answer the question, provide usable steps and use clear language. It comes after hard validation and grounding rather than replacing them.

Each rubric item is a yes/no proposition evaluated with Noul. The value is probability of yes, not a score measuring how good the prose is. The code requires every item to meet the chosen threshold; it does not average a weak item away with two strong ones.

Four gate states communicate different next steps. Accept releases a draft artifact inside the workflow, reject stops forbidden content, revise requests bounded editing, and human_review stops automation for uncertainty. A good-looking average cannot decide these paths on behalf of application policy.

Seven practical steps

Step 1

Write one sentence describing the user’s desired outcome before defining the rubric. For DeskLite, a useful answer gives the export path in order and explains that it downloads rather than sends the file.

Step 2

Read RUBRIC in quality.mjs and match each key to questions.mjs. The key list is an application contract, not a new vendor quality API. Unexpected or missing data must not pass by default.

Step 3

Print the quality request and verify it contains the user question, trusted facts and current candidate. Evaluating a candidate in isolation can reward fluent text that answers the wrong question.

Step 4

Run the direct V008 quality judgment. The authored plainLanguage value is low, so the state is revise. Keep that local gate result distinct from the final pipeline state after the repair budget runs out.

Step 5

Run full V008 and inspect both generations. The reference stops with human_review after two drafts rather than trying indefinitely until the same model says yes. Repeated judgments are not independent votes.

Step 6

Run exact-boundary and invalid-value tests. All three0.8 values pass this teaching policy; a missing, nonfinite or out-of-range value reviews. One middle-band item also reviews even if another item is clearly weak.

Step 7

Run the hardFailure test with every rubric item equal to1. Reject must still win. This is the rule that prevents quality evaluation from overruling authorization, schema or other nonnegotiable constraints.

Files and runnable commands

Open the exact source and read its dependencies before editing. This is runnable reference code, not a fictional AI-Core SDK. The source links point to a fixed version of the public GitHub companion.

src/quality.mjs · Matching tests · This lesson’s exercise

import {gate} from './contracts.mjs';
export const RUBRIC=['answersUser','stepsUsable','plainLanguage'];
export function qualityGate(judgments,policy,{hardFailure=false}={}) {
  if(hardFailure) return gate('reject',['hard_constraint_failed']);
  if(RUBRIC.some(k=>typeof judgments?.[k]!=='number'||!Number.isFinite(judgments[k])||judgments[k]<0||judgments[k]>1))
    return gate('human_review',['quality_schema']);
  const low=RUBRIC.filter(k=>judgments[k]<=policy.noMax);
  const uncertain=RUBRIC.filter(k=>judgments[k]>policy.noMax&&judgments[k]<policy.yesMin);
  if(uncertain.length) return gate('human_review',uncertain.map(k=>'uncertain_'+k));
  if(low.length) return gate('revise',low.map(k=>'quality_'+k));
  return gate('accept',['all_rubric_checks_pass']);
}

Offline commands:

node src/cli.mjs request --stage quality --case V001
node src/cli.mjs judge --stage quality --case V008
node src/cli.mjs run --case V008
node --test --test-reporter=spec test/gates.test.mjs

Expected output and verification

The direct V008 gate revises; the full workflow reviews after exhausting bounded revisions. The test with a hard failure rejects despite perfect rubric values. These are policy behaviors over mock judgments, not a calibrated quality score for real users.

Inspect reasons, history and every attempt rather than only the final label. Unknown usage must remain unknown. A green test demonstrates the specified code path, not universal semantic safety.

A failure to understand

A weighted mean can accept a draft whose clarity is unusable because factuality is high. In this example the dimensions are separate acceptance conditions. If you intentionally permit tradeoffs, specify and validate them rather than hiding them in arithmetic.

Exercise in a disposable copy

Add a fourth criterion in a disposable copy. Update the question contract, parser, rubric, fixtures and tests together. First make a missing-answer test fail closed; do not merely append a UI label.

Follow the matching exercise file, retain the original and restore the reference before the next chapter. Do not use held-out results to choose a threshold.

Troubleshooting

If quality unexpectedly accepts, inspect whether every rubric key was validated. Undefined comparisons in JavaScript can quietly evaluate false; the explicit quality_schema branch prevents absence from masquerading as success.

Next checkpoint

Before continuing, name the property that code checks exactly, the judgment that may be wrong, the state that stops work and the evidence needed before release.

← Previous lesson · Next lesson →

Sources and version notes

Key takeaways