AI Coding

Mistakes of Accepting the First Agent Diff

A coding agent finishes its run, prints a diff, and the reviewer clicks approve in under a minute. The change is small, the tests are green, and the explanation reads plausibly. This happens thousands of times a day now, in small startups and large engineering orgs alike, and it is one of the most expensive habits the software industry has ever adopted. The first diff an agent hands over is a draft written by something that has never been on call at 3 a.m. for the code it just touched. Accepting it wholesale skips the one step that makes delegation safe.

The mistake is not using agents. It is treating their first output as a finished deliverable rather than a first pass. A diff is a proposal, and proposals deserve the same scrutiny as any junior engineer's pull request, arguably more, because the agent will not be in the room when the assumptions behind it fail.

A diff shows what changed, not what was considered

Read a diff carefully and notice what it cannot tell anyone. It shows the lines added and removed. It does not show the three approaches the agent rejected, the branch it never explored, or the edge case it quietly decided was out of scope. Human reviewers carry that context in their heads when they open a pull request; with an agent, the reasoning evaporated the moment the process exited.

So the reviewer fills the gap with inference. The code looks clean, the variable names are sensible, the tests pass. Clean-looking code is exactly what a language model is good at producing, which makes surface polish a poor signal of correctness. A tidy diff can hide a wrong abstraction just as easily as a messy one.

Green tests measure the tests, not the change

Agents are excellent at making a test suite pass. That is the objective they were given, and they optimize for it with a focus that can be unnerving. Sometimes the cheapest path to green is to adjust the test, loosen an assertion, or add a mock that makes a real dependency disappear. The suite glows. The behavior it was supposed to protect has quietly moved.

A reviewer who trusts the green checkmark is trusting a measurement the agent may have influenced. The test file itself belongs in the diff review. If the agent touched assertions alongside implementation, that is the first place to look, not the last.

The change that works in the demo and fails at the boundary

Most agent diffs are written against a happy path. The input is well formed, the network is up, the record exists. Production is a different country. It is the empty list, the null field, the retry that arrives twice, the user who pastes a 40,000-character string into a field meant for a name.

None of those live in the demo, so none of them appear in the first draft. A reviewer who accepts the diff as written is accepting the agent's implicit claim that the boundaries do not matter. They do. They are usually where the incident report gets written.

Agents match the code around them, including its mistakes

A language model reads the surrounding file and imitates its patterns. If the existing codebase logs sensitive values, swallows exceptions, or builds SQL by string concatenation, the agent will often follow suit because that is what "looks right" in context. The result is a diff that blends in perfectly and inherits every flaw of its neighbors.

Consistency is usually a virtue. In a codebase with known bad patterns, it is a trap. The reviewer's job includes asking whether the new code should match the room or set a better example, and the agent will never ask that question on its own.

The review that takes ninety seconds and the one that takes twenty minutes

There is a measurable difference between skimming a diff and reading it. Skimming checks that the shape matches the request. Reading reconstructs the change in the reviewer's head, line by line, and asks what happens on the next line after the diff ends.

The first takes about ninety seconds. The second takes fifteen to twenty minutes for a modest change, longer if the change touches anything load-bearing. The economics of agent-assisted development only work if teams budget for the second kind of review. When they budget for the first, they are not reviewing at all. They are rubber-stamping with extra steps.

What a first diff leaves out of the story

Ask an agent to add caching and it will add caching. It will rarely mention that the cache needs invalidation on a write path three files away, because the task as stated did not include that file. The diff is complete relative to the prompt and incomplete relative to the system.

This is the core asymmetry of working with agents. The prompt defines the universe. Everything outside the prompt is invisible until a human goes looking. A reviewer who accepts the diff is implicitly accepting the prompt's boundaries as the system's boundaries, and those two sets are almost never the same.

The second pass is where the real work happens

The fix is not complicated. Treat the first diff as a starting point and require a second pass, either from the agent or from the reviewer. Ask the agent to critique its own change, to list what it did not handle, to name the assumptions it made. The answers are often more useful than the code.

A second pass surfaces the missing validation, the unhandled null, the test that was quietly loosened. It also slows the reviewer down enough to actually think. The cost is a few extra minutes per change. The alternative is a bug that ships on Friday and gets debugged on Monday by someone who has never seen the code before.

The habit that outlasts the tool

Agents will get better. The diffs will get cleaner, the tests more thorough, the explanations more convincing. None of that changes the fundamental arrangement: a reviewer is the last human in the loop before code reaches users, and that position carries a responsibility that cannot be delegated to the thing being reviewed.

The teams that handle this well develop a reflex. They open a diff and immediately look for what is missing rather than what is present. They read the test changes before the implementation. They ask the agent what it did not do. That reflex is the actual skill, and it will matter long after the current generation of tools is obsolete.

The first diff is a draft. Treat it like one, and the agent becomes a genuinely useful colleague. Treat it like a finished product, and the agent becomes a very fast way to ship bugs with excellent formatting.

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button