Budgets — จำกัดการแก้ร่าง retry และทางออกเมื่อพลาดBound Repairs, Retries and Failure Recovery

1. The engineering problem

An unbounded 'try again until it passes' loop converts uncertainty into uncontrolled cost and latency. It can also overfit an evaluator rather than improve the task. Define a shared budget before the first provider call. The companion defaults to five logical provider calls, at most one candidate revision, and a small elapsed-time limit for offline execution. These are teaching values, not production SLOs. A live decision adapter uses explicit HTTP timeouts and a longer workflow budget.

Count operations honestly. The first pass uses decide, generate and judge. A behavior failure can skip the judge and request one new candidate, so wrong-then-correct uses four logical calls and two candidate attempts. A persistent quality failure consumes all five logical calls and ends in review. The call counter measures the provider interface, not HTTP traffic: FixtureProvider has no network, while JevProvider supplies a local draft and makes HTTP requests only for decisions. Do not publish logical-call counts as token usage or cost.

2. The boundary to design

Transport retry and semantic repair solve different problems. A temporary 429 may justify bounded backoff according to the provider's error contract [1]. A valid but inadequate explanation may justify a new candidate. The tutorial adapter fails to review rather than implementing automatic transport retries. Its shell checks time before and after calls; it does not preempt a blocking call. A production implementation needs cancellation, remaining-deadline propagation, jitter, retry limits and reconciliation for ambiguous results. State those limitations before calling the mechanism a hard deadline.

3. Read the implementation

Implementation excerpt from the companion. Run the complete lesson below with its imports and fixtures.

from workbench.runtime import run, request, candidate, FixtureProvider

wrong = candidate("def discount_cents(price, percent):\n    return price - percent\n")
result = run(request(), FixtureProvider(candidates=[wrong, candidate()]),
             max_calls=5, max_revisions=1, seconds=2)
print(result["state"], result["attempts"], result["calls"])

Feedback-driven revision is a plausible workflow pattern, illustrated by Reflexion research [2], but improvement must be measured for the task. A repair must re-enter output and behavior checks and receive a newly bound quality verdict. It cannot reuse old approval. Never apply the same loop blindly to a PR, payment or deployment action: a timeout after the server committed requires a status check or idempotency mechanism. Our draft-only lab avoids effects entirely, making the narrower outcome easy to verify: bounded repair succeeds once; persistent failure terminates visibly.

4. Compare the outcomes

The following output comes from scripted fixtures that verify architecture behavior. It is not a measurement of live Jev accuracy, latency, or cost.

{
  "bounded_repair": "accepted_draft",
  "attempts": 2,
  "calls": 4,
  "no_budget": "review"
}

5. Run, break, and explain

Part 13 code · Implementation · Tests: test_bounded_repair · Exercise

git clone https://github.com/Anirach/ai-core-engineering-workbench.git
cd ai-core-engineering-workbench
git checkout 18cdfcd4156d3d02acb8badf3b3a0a823113e0ed
python3 -m workbench.lesson 13
python3 -m unittest tests.test_boundaries.BoundaryTests.test_bounded_repair -v

Your turn

Set max_calls to zero, then force a permanent quality failure. Verify finite termination and that rejected drafts are not returned as accepted.

Keep three artifacts: the input you tried, the resulting state and reason, and an explanation of which boundary owns that result. Do not change an expected result merely to make a test pass.

6. Sources and limits