Review the AI Agent's Plan Before It Writes Code

An AI coding agent makes its most expensive mistakes before it writes a line: in the plan. A single planning pass forms a plausible story about the codebase, and that story can be wrong in ways that stay invisible until the implementation fails. Askpert, a marketplace where experts package private knowledge into protected AI Experts that run sealed server-side, ran dozens of Claude Code builds and found the most effective fix is an adversarial reviewer that attacks the plan file before any implementer touches a worktree. This post documents that workflow from the production builds that produced it.

Why review the plan instead of the code?

Plan review examines the written plan before any code exists, where code review examines committed code after it exists. A logic error caught on paper costs minutes, while the same error caught after implementation costs a rebase and a retest. Both gates police the same risk at different points in the pipeline.

Code review cannot see the architectural decision the agent made before it started writing, because it inspects the finished diff. The cheaper place to stop a bad decision is the plan, where nothing has been committed yet. For the post-implementation half of the gate, a separate guide covers how to review agent-produced code. Plan review sits before that guide and catches a different class of failure.

How many agents does a plan review workflow need?

Five roles run each task: a read-only Explorer, a lead planner, an adversarial critic, an implementer and a red-team reviewer. Context flows through written artifacts and reports with file:line references, never through a shared memory. The lead adjudicates every finding and records accepted or rejected with rationale.

The roles run sequentially because they share a working tree. The Explorer maps every caller, callee, config value and existing test before anything is written, a map a code graph turns from guesswork into a query. The lead writes the plan as files on disk, one per phase. The critic attacks those files. The implementer executes the locked plan in an isolated git worktree. A red-team reviewer and a simplify pass gate the ship. Written artifacts are the shared memory, because a shared conversational memory would let one role's assumption leak into another role's decisions.

What does plan review catch that a single pass misses?

Askpert built a critic agent that attacks a plan for wrong mechanisms, for evidence that does not exist in the worktree, and for guarantees the code cannot honor. One build caught all three classes before implementation.

The first catch was a wrong mechanism. A plan proposed fixing dropped uploads by raising the API server's keep-alive timeout, on the theory that the server closes connections mid-upload on large bodies. The reviewer read the installed HTTP server's protocol source and showed that mechanism impossible: the keep-alive timer only arms between requests and cancels on the first byte of the next one, so it never kills a transfer mid-body. The apparent correlation was a confound, because bigger files take the generating model longer to produce, which leaves the connection idle longer before the call. The reviewer also demanded production logs before implementation, and they showed the failed calls never reached the server, ruling out the one scenario where the fix would have been a placebo.

The second catch was a phantom reference. The plan cited a 23-line section of a tracking document at a specific line number. The reviewer checked the implementation worktree and found the file there was 23 lines shorter: the section existed only as an uncommitted edit in a different checkout.

The third catch was an overclaimed guarantee about to ship as documentation. The plan wanted to describe a retry operation as idempotent. The reviewer traced the datastore code and found the retry silently bumps the row's updated_at timestamp, which a separate publish flow uses as its concurrency check. Shipped as written, the documentation would have promised something false, so the wording was qualified instead.

How should a reviewer score a plan?

Findings are ranked BLOCKER, MAJOR or MINOR, and every claim is grounded in file:line evidence the reviewer verifies itself. The review ends in an explicit verdict: GO, GO-WITH-CHANGES or NO-GO. On one transport-bug task the verdict was GO-WITH-CHANGES with two blockers and four majors, all fixed before implementation began.

The file:line grounding is what separates this from a rubber stamp. Each finding names the file and the line the critic read, and a finding without evidence is rejected. The verdict forces a decision instead of an open-ended list of caveats. Askpert also runs two reviewer roles, the critic and the red team, so a consistent failure mode does not slip through because both reviewers happen to share one model's blind spot.

A real case: the plan that explained a bug that never happened

Uploads of skill files over MCP intermittently died with a closed socket. Small payloads mostly succeeded and 9.6 to 20 KB payloads failed, so the first written explanation blamed a transport cap near 8 KB. Plan review found no such cap in the codebase, and production logs showed the failed calls never reached the server.

The real cause was different. The server ran with HTTP keep-alive at the default five seconds, and a pooled connection the server had already hung up on was being reused. The fix was a single flag, raising keep-alive from five seconds to 620 seconds, plus a documented recovery recipe for agents. The original 8 KB cap belief was formally retracted from the tracking documents. The transport explanation is labeled a hypothesis and gets falsified or confirmed by a post-deploy probe rather than asserted as settled.

When is plan review too expensive?

The five-role review is calibrated to production code and costs real tokens and wall-clock time. It is overkill for throwaway scripts and one-off explorations. It pays for itself on code that ships, is maintained or becomes documentation, because every catch happens before a worktree exists.

Askpert applies plan review to the production marketplace it ships, not to every exploration. The same review-first discipline is packaged on the Askpert marketplace as a rented Expert, where an author's private knowledge and skills run sealed server-side and a buyer calls them over API or MCP and receives only the answer.

Does plan review replace code review?

No. Plan review stops errors before implementation, and code review catches errors the implementation introduced. Both run as separate gates in Askpert's workflow, because a plan that passes review can still produce downstream bugs.

A clean plan does not guarantee clean code, and clean code cannot undo a bad architectural decision. Running the two gates back to back is the point: the pre-code gate removes the expensive decision errors, and the post-code gate removes the mechanics errors the implementation added.

Can one model effectively review another model's plan?

A reviewer running the same model as the planner shares its failure modes and will miss the same errors. An adversarial reviewer is useful when it is instructed to attack assumptions and verify every claim against real files and logs rather than to confirm the plan it was handed.

The value does not come from a second opinion. It comes from a reviewer whose explicit instruction is to find the plan wrong, and whose evidence standard is a file and a line number it verified itself. That instruction, not the model choice, produced the catches in this post.

How do I start with just two agents?

Start with two roles: a planner that writes the plan to disk and a critic that reads it against the real codebase and returns file:line findings with a GO or NO-GO verdict. Add the Explorer and implementer roles once the two-role loop is stable, and keep plans as files so both agents read the same artifact.

Writing the plan to disk is non-negotiable from day one. A plan that exists only in a chat transcript cannot be reviewed as an object, and a critic cannot check a line reference against a file that was never saved. The file is the contract between the planner and the reviewer.