Audit-Then-Execute: The New Frontier-Model Workflow

What Is This?

Audit-then-execute is a two-speed workflow for AI software work:

expensive frontier model -> inspect, map, plan, define checks
cheaper executor agent   -> make edits, run commands, leave a trail
human/operator           -> review the trail and merge only verified work

The expensive model is not used as a tireless typist. It is used where marginal intelligence matters most: finding the bug shape, reading architecture, spotting hidden constraints, decomposing the task, and writing the verification contract. A cheaper or narrower agent then performs the mechanical loop: edit files, run tests, fix failures, and report what changed.

This is not just cost optimisation. It is control-plane design. The frontier model becomes the auditor and architect. The executor becomes the worker. The written plan, test commands, diffs, logs, and final verification become the handoff surface.

Why Does It Matter?

Coding agents are now good enough to act inside real repositories, but not reliable enough to be trusted without structure. SWE-bench tracks agents on real GitHub issues. METR measures capability by the length of software tasks agents can complete and finds that task length still sharply affects reliability. GitHub describes agent mode as a loop that reads the workspace, edits code, runs terminal commands, observes errors, and iterates. Anthropic warns that fully autonomous agents are useful for open-ended work but carry higher cost and compounding-error risk.

Those sources point to the same lesson:

agent reliability is a workflow property, not just a model property

A single frontier model doing everything spends expensive tokens on low-leverage work. A cheap agent doing everything often misses the architectural move. Audit-then-execute splits the job by comparative advantage.

The Core Pattern

Use the expensive model for judgment-heavy work:

• inspect the issue, repo, logs, and failing tests;
• identify the likely failure mechanism;
• map affected files and interfaces;
• choose the smallest safe change;
• define acceptance tests and rollback criteria;
• write a plan that another agent can execute without improvising.

Use the executor for action-heavy work:

• edit files;
• install declared dependencies;
• run the named checks;
• paste bounded failures back into the trail;
• make incremental fixes;
• stop when the verification contract passes.

The handoff is the important artifact. If the plan says only "fix auth," it is not a handoff. If it says "reproduce with npm test -- auth, inspect middleware token parsing, do not edit generated clients, pass npm run typecheck and npm test -- auth," the executor has rails.

Why Smart People Get This Wrong

The common mistake is treating the best model as the whole worker. That feels natural because frontier models can reason, code, and explain. But long agent runs mix two very different activities:

1. high-value uncertainty reduction;
2. low-value execution churn.

The first deserves the best model. The second deserves a cheaper loop with strong instrumentation.

Another mistake is treating the executor's final message as evidence. It is not. Evidence is the written trail: the plan, diff, command output, test result, and live check. Without that trail, the human has to re-audit the work from scratch.

How To Use This

A practical audit-then-execute packet has six parts:

1. Problem statement: what is broken or desired.
2. Architecture map: files, boundaries, invariants, and known gotchas.
3. Hypothesis: the smallest likely mechanism behind the bug or change.
4. Execution plan: ordered edits with explicit non-goals.
5. Verification contract: exact commands, expected signals, and live checks.
6. Written trail requirement: executor must report diff summary, commands run, failures seen, and final status.

For Jamie's Jme-Loop world, this is the steward pattern: use frontier intelligence to audit the repo and write the work order; let the cheaper loop execute one bounded unit; keep the artifact trail visible enough that Lloyd, Jamie, or another agent can resume.

What This Does Not Prove

Audit-then-execute does not prove that cheap agents can safely handle arbitrary work. It works best when the frontier model can name constraints and the executor has runnable checks. It also does not remove the need for human review on dangerous, ambiguous, or high-blast-radius changes. The workflow lowers wasted intelligence and improves auditability; it does not make autonomy free.

Key Terms

• Auditor model: the stronger model used to inspect, reason, plan, and specify validation.
• Executor agent: the cheaper or narrower agent used to make changes and run checks.
• Verification contract: the exact pass/fail evidence required before work is considered done.
• Written trail: plan, diff, command output, failures, fixes, and final status.
• Compounding error: small agent mistakes accumulating across a long autonomous run.

Recall Questions

• What work should the frontier model do in audit-then-execute?
• Why is the executor's final message weaker evidence than command output and diffs?
• What belongs in a verification contract?
• When should the workflow stop and ask for human review?

Best Resources to Learn More

• Anthropic, "Building Effective Agents" — best general map of workflows, agents, evaluator-optimizer loops, and orchestrator-worker patterns.
• Aider chat modes documentation — concrete example of ask/code and architect/editor separation.
• METR, "Measuring AI Ability to Complete Long Software Tasks" — why long-horizon execution remains a reliability boundary.
• GitHub Copilot agent mode overview — useful description of the edit-run-observe loop.

Sources

1. Anthropic, "Building Effective Agents," 2024, https://www.anthropic.com/engineering/building-effective-agents
2. Anthropic, "Best practices for Claude Code," https://www.anthropic.com/engineering/claude-code-best-practices
3. Aider, "Chat modes," https://aider.chat/docs/usage/modes.html
4. METR, "Measuring AI Ability to Complete Long Software Tasks," 2025, https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/
5. SWE-bench, benchmark and leaderboards, https://www.swebench.com/
6. GitHub Blog, "Agent mode 101: All about GitHub Copilot's powerful mode," 2025, https://github.blog/ai-and-ml/github-copilot/agent-mode-101-all-about-github-copilots-powerful-mode/
7. GitHub Docs, "Adding repository custom instructions for GitHub Copilot," https://docs.github.com/en/copilot/customizing-copilot/adding-repository-custom-instructions-for-github-copilot