The Review Checklist We Run on Every AI-Generated Pull Request
Most of the code we ship is written by agents, and every line of it is reviewed. AI-written code fails differently from human code. It is rarely sloppy on the surface. It is fluent, consistently formatted, and plausible, which is exactly why problems slide through: the missing authorization check looks just as tidy as the rest.
So we review against a fixed checklist in six areas, and every finding gets a severity, a location, and a suggested fix. Here is the checklist and what we look for in each area that is specific to AI-generated code.
Structure: does it fit the codebase it landed in
Check separation of concerns, module boundaries, naming consistency, and circular dependencies. With agents, the specific failure is parallel structure: a new helper that duplicates one that already exists three folders away, or a new pattern for data access sitting next to the established one. The agent did not search before writing.
Also check the diff size against the request. Unrelated reformatting, renamed variables, and deleted "dead" code inside a feature pull request are a review cost and a hiding place. Send them back as a separate change, or not at all.
Error handling: caught, logged, and honest with the user
Every error should be caught where it can be handled, logged at the right level, and shown to the user as a clear message, never as a raw stack trace. Agents tend to fail in two opposite ways here. Either they wrap everything in a catch that logs and continues, so failures vanish, or they handle errors for states that cannot occur while missing the ones that do.
Look hardest at the boundaries: network calls, payment providers, webhooks, file reads. Ask what the user sees when that call fails, and whether the system degrades gracefully or leaves half-written state behind.
Performance: N+1 queries and blocking calls
The classic AI performance bug is the N+1 query: a list page that loads fifty records, then makes a database call per record inside a loop or a component. It is invisible with seed data and painful with real data. Check every loop that touches the database or an API.
The second is blocking work in async contexts: synchronous file or network calls inside an event loop, or heavy computation in a request handler. Then check algorithm choice against real data size, and whether caching is used where it helps and absent where it would serve stale data.
If your app was built mostly by AI tools and these first three areas already worry you, the remaining three are where production incidents usually come from. That combination is what a Production Rescue diagnostic is built for.
Is the AI-built app the risk now?
A $7,500 diagnostic week, then we stabilize what you have.
Security: strict, every time
We review security strictly on every pull request, not in a quarterly pass. Secrets: nothing hardcoded, nothing logged, nothing shipped to the client bundle. Injection: no string-built SQL, shell commands, or HTML from user input. Inputs validated and sanitized at the boundary. HTTPS enforced.
Authorization is the one AI code gets wrong most often. The agent adds an endpoint that checks the user is logged in, but not that the record belongs to them. Read every new route and ask: who can call this, and what stops them from passing someone else's ID? Then uploads: type and size limits, storage outside the web root, and no trust in the file name.
Tests: critical paths, not coverage padding
Agents will happily write tests that assert a mock returns what the mock was told to return. That raises coverage and proves nothing. We check that critical paths have meaningful tests: sign-in, permissions, checkout and webhooks, anything that sends email or moves money, and the specific bug the pull request claims to fix.
Edge cases matter more than count: empty lists, duplicates, expired tokens, retries. Mocks are fine at the edges of the system, not in the middle of the logic under test.
Debuggability: can we fix it at 2am
When this code fails in production, will the logs tell us why? Check for enough context in log lines, clear error messages that include what was being attempted, and a request or correlation ID that follows a request across services and background jobs.
The red flag is a swallowed exception: a catch block that does nothing, or logs a generic "something went wrong" with no context. Agents produce these often, because it makes the code "not crash". It does not crash. It just lies.
Severity levels and the review output
Every finding is written as severity, category, file and line, a description, and a suggested fix. Severities are CRITICAL, HIGH, MEDIUM, and LOW. Critical means a security hole, data loss, or money at risk, and blocks the merge. High means a likely production failure and normally blocks. Medium and low get fixed or ticketed with a reason.
The fixed format matters with agents in the loop: the review output becomes the next task for the coding agent, and a numbered list with file and line is something it can act on precisely. One round of changes, then the reviewer merges, edits, or closes.
Key takeaways
- AI code looks clean, so review against a checklist rather than by feel.
- Check every new route for authorization on the record, not just authentication.
- Hunt N+1 queries and blocking calls in async code; seed data hides both.
- Tests should cover critical paths and edge cases, not assert what mocks were told to return.
- Write findings as severity, category, file and line, and fix, so the agent can act on them.
Frequently asked questions
An agent can run the checklist as a first pass and catch a lot. It should not be the final verifier of its own work. A human signs off, especially on security and money paths.
For a focused pull request, minutes. That is why we keep agent diffs small. A two-thousand-line AI pull request cannot be reviewed properly in any reasonable time and should be split.
One of the most common we see is missing ownership checks on endpoints: the user is authenticated, but nothing stops them from reading or changing another user's records.
Usually not. If the data model is coherent, most issues are fixed in place, in order of severity. A rewrite is for codebases with no seams to test and no real authorization at all.
The 15-Day AI MVP Playbook
Day-by-day plan, a starter CLAUDE.md, the Lovable-to-production checklist, eval and cost worksheets, and 10 prompts. Pure value, no pitch.
Book 20 minutes, or send a brief
A former CTO replies. You leave with a written plan and a fixed estimate.
Prefer LinkedIn? Connect with Harish.