AI pull request review checklist for agent-authored code

Quick answer

Review agent-authored code as untrusted code. The author label does not change the acceptance bar: a maintainer must understand the requested change, verify the diff, run tests that the agent did not write solely to satisfy itself, and own the result after merge. The checklist below is for maintainers, engineering leads, and security reviewers deciding whether a patch is safe to ship.

The merge decision

Gate	Evidence required	Stop when
Scope	Issue, acceptance criteria, and a diff limited to that job	Unrelated refactors or generated files obscure the change
Correctness	Independent tests plus manual inspection of affected paths	Tests only assert the agent’s preferred implementation
Security	Input, auth, secret, dependency, and outbound-action review	Permissions expand without a documented need
Operations	Logs, failure behavior, migration, and rollback plan	A partial failure can corrupt state or strand users
Ownership	Named human reviewer and follow-up owner	Nobody can explain or maintain the patch

Start with the diff, not the agent transcript. Transcripts are supporting context; they are not proof that the repository now behaves correctly.

Review checklist

1. Restate the intended behavior in one sentence and map each changed file to it.
2. Reject scope creep before spending time on line-level polish.
3. Trace user input through validation, authorization, storage, external calls, and output.
4. Inspect removed checks as closely as added code.
5. Run the existing suite, then add at least one independent edge or failure case.
6. Review new packages, lockfile changes, install scripts, licenses, and network destinations.
7. Verify secrets never enter prompts, logs, fixtures, snapshots, or generated artifacts.
8. Confirm migrations are reversible or safely retryable and that partial writes are handled.
9. Read generated SQL, shell, CI, infrastructure, and policy files line by line.
10. Require the human approver to be accountable for the merged behavior.

Example: a small patch with a large blast radius

An agent changes a newsletter endpoint to retry failed requests. The diff is only 30 lines, but a retry after an ambiguous timeout can create duplicate subscriptions. The correct review adds an idempotency key, tests the timeout path, confirms log redaction, and checks the provider’s retry contract. Line count would have been a poor proxy for risk.

Tool-assisted review

Promptfoo can help regression-test model-facing behavior, while Agent Audit is relevant when the change adds agent actions or tool permissions. Neither replaces repository-native tests or a maintainer’s security review. For the selection step, use the AI coding tools category and the AI PR review tools comparison to compare evidence, not popularity.

Risks and limits

An agent can produce a plausible explanation for incorrect code, and a green test suite can still encode the same mistaken assumption as the implementation. Large generated diffs also create reviewer fatigue. Keep tasks narrow, cap change size, preserve protected branches, and escalate authentication, payments, deletion, cryptography, and infrastructure changes to domain owners.

Decision

Merge only if an independent reviewer can explain the patch, reproduce its evidence, and operate its failure modes. Otherwise, reduce scope or return the change for revision.

Evidence package and risk tiers

A reviewable agent pull request should contain a short problem statement, exact scope, test commands and results, and a note about risks or untested paths. Record dependencies, migrations, generated artifacts, and external documentation explicitly. A reviewer should reproduce the evidence from a clean checkout; a test that depends on an uncommitted fixture, global package, or undocumented credential is not a reliable gate.

Different change types require different proof. A user-interface patch needs keyboard, responsive, loading, and error-state checks, not only a screenshot. A data migration needs row counts, constraints, retry behavior, rollback, and a production-shaped sample. An API change needs authorization, validation, compatibility, rate-limit, idempotency, and observability checks. Infrastructure changes require a rendered plan, failure-domain review, and recovery path. Dependency updates need changelog, license, transitive package, install-script, and vulnerability inspection.

Classify documentation, narrowly scoped tests, and reversible presentation fixes as low risk when they have no permission or data effect. Application logic, dependencies, and background jobs are medium risk and need independent failure tests. Authentication, secrets, payments, deletion, infrastructure, security controls, and irreversible migrations are high risk and require a domain specialist plus staged rollout. Consequence and reversibility determine the tier, not the agent’s confidence or line count.

Review the process as well as the patch

Capture commands, changed files, approvals, and network destinations without placing secrets in the record. If the agent accessed a forbidden path and later produced a clean diff, the outcome is still unsafe. Ask which assumptions were uncertain, then verify them against code and primary documentation. Turn an important undocumented assumption into a test or remove the dependency.

Record the review cost as well as the review outcome: initial reading, clarification, correction, CI reruns, and post-merge repair all belong in the coding-agent review-debt scorecard. A fast first patch does not reduce engineering work when it transfers that work to reviewers or on-call owners.

Inspect generated comments and tests for circular reasoning. A test written to mirror the implementation can pass while the requirement remains unmet. Prefer externally observable behavior and tests that fail on the starting commit. For model-facing code, store representative inputs and an acceptance rubric, then report repeated trials instead of one favorable answer.

After merge

Watch the signals that reveal a mistaken assumption: error rate, retry volume, latency, permission denials, support contacts, data reconciliation, or rollback events. Connect the pull request to those signals and name the responder. For risky changes, use a feature flag, canary, or limited tenant release and keep the prior version deployable.

Feed confirmed failures back into repository tests and the coding-agent evaluation set. Do not solve every review problem by lengthening an agent prompt. Decide whether the durable control belongs in code, CI, policy, documentation, or sandbox configuration. The goal is a stronger engineering system that remains understandable to humans.

Keep the shared checklist concise enough to run on every patch, with specialist addenda for migrations, security controls, infrastructure, and regulated workflows.

Related guides

• Build a repository task set for coding-agent evaluation
• Compare coding-agent evaluation and rollout tools
• Regression-test prompts before a model or workflow change