Development

Refactors With AI: Small Diffs, Hard Tests, No Rewrite the App

Kevin PlumleeOctober 2, 20264 min read
Refactors With AI: Small Diffs, Hard Tests, No Rewrite the App

Ask an agent to “clean this up” and it will offer to rebuild civilization.

New folder structure. New names for everything. A utility layer you did not request. Tests rewritten to match the new religion. Your PR turns into a crime scene.

Use AI for refactors constantly. Also keep it on a short leash. Small diffs. Hard tests. No “while we are here.”

Refactors fail when the goal is aesthetic

A refactor should reduce a concrete pain:

  • this module is untestable
  • this function does four jobs
  • this API client duplicates auth in six places
  • this React component is a junk drawer

“It feels messy” is not enough. Feelings create 2,000-line PRs. Pain creates surgical ones.

The brief you give the agent

Paste something like:

“Refactor BillingClient to extract retry and auth into HttpClient. Do not change behavior. Keep public method names. Update tests. Do not touch unrelated folders. Show a plan with file list first. Wait for approval. Then produce a minimal diff.”

The “wait for approval” line matters. Agents sprint. You need a gate.

Tests first, or do not pretend

If the code has no tests, I do not start with a rewrite.

Ask the agent to add characterization tests around current behavior. Golden outputs. A few edge cases. Then refactor under that harness.

No tests means you are not refactoring. You are rewriting and hoping. Hope is not a strategy. It is how you spend Friday night blaming git.

Slice the work like an adult

Bad: “modernize the auth stack.”

Better:

  1. extract interface
  2. move one caller
  3. delete dead path
  4. rinse

Each PR should be reviewable in one sitting by a human who still has a life. If the reviewer needs a sandwich and a map, the PR is too big.

Also ban drive-by renames in the same PR unless the rename is the point. Diff noise hides bugs.

What to accept from the model

Useful:

  • extracting pure functions
  • replacing copy-paste with one helper
  • migrating a call site to a new client
  • suggesting edge cases for tests
  • explaining a gnarly block in plain English before we touch it

Not useful:

  • framework migrations mid-refactor
  • “simplifying” by introducing a clever abstraction used once
  • changing unrelated lint rules
  • rewriting comments into essays

If the model introduces a pattern we do not use elsewhere, you make it match the house style or reject it. Consistency beats novelty in a working system.

Review like a suspicious senior

the checklist:

  • behavior preserved?
  • error handling preserved?
  • logs and metrics still fired?
  • any new network calls?
  • any widened permissions?
  • tests fail when purposefully broken? (I sabotage one assertion to see)

AI PRs look clean. Clean is not correct. Read the diff. Run the app. Click the thing.

Rollback is part of the design

Prefer refactors that can ship behind a flag or a side-by-side adapter. Keep the old path until the new one proves itself in production traffic or at least in staging soak.

Big-bang cutovers are how you learn exciting new things about your customers’ data shapes.

A real example pattern

I had a folder of scripts that each talked to the same API with slightly different timeout soup. Instead of “rewrite all scripts,” we:

  1. added tests around one script’s outputs
  2. built a tiny shared client
  3. migrated one script
  4. migrated the rest over a week

The agent did most of the typing. Did the sequencing and the nos. That division of labor is the whole skill.

Punchline

AI is a power saw. Refactors are furniture repair. You still measure twice.

Keep the blast radius small. Make tests mean enough that they can say no for you. Refuse the grand rewrite unless the system is truly on fire and you have time, tests, and a rollback story.

Bottom line

You will ship more improvements and fewer legends about the weekend the agent “helped.”

Kevin Plumlee

Founder, RVNOPS

Kevin Plumlee

15+ years running healthcare SEO, paid media, and operations. Now I advise the teams who do that work, across marketing, intake, technology, and AI.

More about me

Tell me what’s not working.

Stalled growth, spend you can’t trace, systems that fight your team. That’s a good place to start.

Start a conversation