Software engineering optimized for deterministic computers. Same input, same output, same CI gate. Agentic systems break that contract: the model samples, context drifts, tools mutate disk, and control flow leaks into the prompt. The industry answer is no longer a longer system prompt. It is harness engineering: Agent = Model + Harness.
I am happy to announce the following class after the successful completion of Cohort 1 on harness engineering.
Packt Cohort 2 of Hands-On Harness Engineering with Claude - C2 runs Saturday, October 17, 2026, from 9:00 AM to 11:30 AM EDT (6:00 AM to 8:30 AM PDT / 1:00 PM to 3:30 PM UTC).
I will teach the live lab. Register with the following link for 40% off on Packt's Eventbrite page: Hands-On Harness Engineering with Claude - C2 registration.
Paid Substack subscribers get the implementation stack and exclusive access to the updated Cohort 2 slide deck and relevant code.
Classic software engineering assumes a deterministic machine. Compilers, type checkers, unit tests, and SRE error budgets all ride on reproducible execution. That stack still matters. It is no longer sufficient as the only control plane once the "compute unit" is a sampling model that writes files and shells out.
Birgitta Böckeler's 2026 Thoughtworks write-up on harness engineering for coding agent users states the formula directly: Agent = Model + Harness. The harness is everything except the model. Guides steer before the agent acts; sensors correct after. Computational sensors (tests, linters, type checkers, hooks) stay cheap and deterministic. Inferential sensors (LLM-as-judge) stay bounded and expensive. The point is not to make the model deterministic. The point is to regulate the codebase with a control system the model cannot rewrite on a whim.
Anthropic's production containment data shows why human-in-the-loop alone fails as that regulator. In How we contain Claude across products, Anthropic reports users approved roughly 93% of permission prompts, then paid less attention as volume rose. An OS-level sandbox cut permission prompts by 84%. Probabilistic oversight saturates. Deterministic isolation and hooks remain.
Claude Code's own hook contract encodes the same judgment. The hooks guide and hooks reference specify that PreToolUse
runs before permission-mode checks, including under bypassPermissions
/ --dangerously-skip-permissions
. Exit code 2 or permissionDecision: deny
hard-stops the tool. That is harness logic in product form: policy in code, not in a hope that the model "remembers" the rule.
Across 2025–2026 papers, the failure diagnosis converges. Multi-agent LLM systems fail first on orchestration, not on raw reasoning. The RO (Reliable Orchestration) line of work, summarized in RO: Reliable Orchestration of LLM Programs, cites plan abandonment, state corruption, and tool-management errors as dominant modes across frameworks and 1,600+ traces. Their fix is a stored program runtime: control and state nodes stay deterministic; generative nodes are bounded oracles behind instruct–validate–repair with schema gates. Orchestration correctness becomes independent of LLM sampling.
Qiu, Ye, and coauthors make the same split concrete in Blueprint First, Model Second (arXiv:2508.02721, 2025). An expert procedure becomes a source-code Execution Blueprint. A deterministic engine walks the blueprint. The LLM runs only as a tool at bounded nodes. It never chooses the workflow path. On TravelPlanner with the same Claude Sonnet 4 backbone, their Source Code Agent reached a 35.56% final pass rate versus 18.00% for the ATLAS baseline, cut constraint violations by 96.0% (11 vs 275), and used 27.1% fewer steps (10.2 ± 0.7 vs 14.0). Same model. Different harness. Different reliability.
A related formalization appears in arXiv:2512.20660 (2025/2026 dual-state line): treat the LLM as part of the environment, not as the decision-making agent. Workflow state stays a finite deterministic machine. Environment state holds stochastic artifacts. Atomic action pairs bind generation to verification. Guards project opaque model output onto observable workflow truth. Across 13 models on diagnostic probes, they report reliability gains up to 66 percentage points at 1.2–2.1× baseline cost. That is the paradigm shift in one sentence: stochastic generation stays; stochastic control flow goes.
Component-level evidence keeps landing on context and action design, not magic prompts. An empirical harness study on SWE-Bench Verified and Terminal-Bench (summarized on alphaXiv 2609.20804) finds context management mainly prevents overflow failures as budgets tighten, planning shifts from accuracy scaffold to cost saver as models strengthen, and weaker bash models need predefined tools while stronger models can run cheaper on bash-only interfaces. Harness knobs move accuracy and cost even when the model weights stay fixed.
Agentic coding systems add three properties deterministic SE never assumed as the default runtime:
Sampling inside the loop. Token draws change the next tool call. Identical prompts do not guarantee identical trajectories.
Mutable shared artifacts. The agent edits the same repo humans edit. Merge conflicts and scope violations are first-class hazards, not edge cases.
Tool side effects. Bash, package installs, and
git_push
change the world. A wrong branch is not a wrong string. It is a production incident.
Cognition's Don't Build Multi-Agents (Walden Yan) argued that naive parallel agents fragment decisions and lose shared context. Their later Multi-Agents: What's Actually Working update still keeps writes single-threaded when multiple agents contribute intelligence. That is harness design: coordination policy outside the prompt.
HumanLayer's field write-up Skill Issue: Harness Engineering for Coding Agents reports the same production pattern: sub-agents act as context firewalls so intermediate tool noise never floods the parent orchestrator; verification must stay context-efficient because dumping thousands of passing test lines back into the window makes the agent lose the task. FutureAGI's Agent Harness Production Readiness checklist collapses readiness into four failure points: state/memory, control loop ceilings, tool verification, and observability. A chat transcript is not a trace. If you cannot answer which tool failed and what the loop did next from stored data, you cannot debug the incident.
Production Learning Review's taxonomy of agent failure signals by harness layer frames enterprise failures as harness defects: stale context, broken tool schemas, missing approval gates, retry logic without termination. The layering (execution sandbox, tooling, context/memory, lifecycle, observability, verification, governance) matches what coding-agent products now expose as hooks, sandboxes, and permission modes.
Enterprise practice is converging on the same split. Böckeler cites OpenAI's public harness notes: layered architecture enforced by custom linters and structural tests, plus recurring drift cleanup, with the hardest remaining work described as designing environments, feedback loops, and control systems. Stripe's "minions" write-up, also summarized there, pushes feedback left with pre-push hooks and blueprints that wire sensors into agent workflows. That is classical software engineering reused as agent regulation, not replaced by a chat transcript.
Net: deterministic software engineering remains the substrate for compilers, tests, and delivery. Harness engineering is the new control plane for agentic AI implementations because the model is probabilistic, the side effects are real, and production reliability now lives in scopes, hooks, budgets, and verification sensors around the model. Cohort 2 exists to teach that control plane as executable modules, not as slideware alone. Updated Cohort 2 slides stay behind the paywall for paid subscribers only.
Today we cover below the paywall:
Failure modes to kill in code: context decay, infinite retry, unverified edits, with the Cohort 2 control card.
Clone path and exclusive Cohort 2 slides: GitHub modules for all readers who subscribe; updated interactive slides for paid subscribers only.
Implementation stack: five pillars, 31 Claude Code hooks, SPEC.md fail-closed writes, PreToolUse denials, risk tiers, pytest repair loops.
Ship criteria: five-step SOP, worktree role isolation, five-gate audit, 14/14 capstone numbers, and the production checklist.
