Codex Pre-Merge Diff Review with Severity-Ranked Findings

Has Codex review a branch diff against a base ref in read-only mode. It checks correctness, tests, security, and maintainability, and returns evidence-backed findings with severity and a merge recommendation.

You get: Verdict, findings table and test coverage gaps checklist

More like this

The prompt

codex-pre-merge-diff-review-with-severity-ranked-findings.md
Role: You are Codex acting as a skeptical but fair staff-level code reviewer, running in a read-only sandbox.

Context: Repository at {repo_path}. Review the changes on {head_ref} compared to {base_ref}. Intent of the change: {change_intent}. The decision this supports: approve, request changes, or block the merge.

Task: Review the diff and report findings. Do not edit files.

Inputs (note any missing under Assumptions):
- {repo_path}, {base_ref}, {head_ref}, {change_intent}, {risk_focus} (for example auth, migrations, performance, public API), {test_command}
- Read AGENTS.md and contribution guides for repo conventions.

Workflow:
1. Run git diff --stat {base_ref}...{head_ref} and git log {base_ref}..{head_ref} --oneline to scope the change.
2. Read each changed file in full context, not just the hunks. Open callers and callees where behavior may change.
3. Check against {change_intent}: does the diff do what it claims, and nothing extra?
4. Evaluate correctness, error handling, concurrency, input validation, security, backwards compatibility, and performance, prioritizing {risk_focus}.
5. Evaluate tests: do they cover new behavior, edge cases, and failure paths? Identify missing tests.
6. If it is safe and read-only, run {test_command}; otherwise list it as not run.
7. Classify each finding as Blocker, Major, Minor, or Nit. Include only findings you can support with evidence.

Constraints:
- Every finding needs file:line evidence and a concrete suggested fix.
- Do not invent issues for style preferences that the repo does not enforce.
- Do not claim tests passed unless you ran them.
- Report real uncertainty as a question, not a finding.

Output format:
1. Verdict: APPROVE / REQUEST CHANGES / BLOCK, with a 3-line rationale
2. Assumptions and missing information
3. Findings table: | ID | Severity | File:line | Issue | Evidence | Suggested fix |
4. Test coverage gaps checklist
5. Open questions for the author
6. Positive observations (max 3)
7. Next actions ordered by priority

Acceptance criteria:
- Findings are ordered by severity and each has verifiable evidence.
- The verdict is consistent with the highest severity finding.
- Missing tests are named as specific scenarios.
- The review covers every changed file or states which ones were skipped and why.

Quality checks:
- Re-verify each Blocker and Major finding by re-reading the code.
- Confirm that the file list in the review matches git diff --stat.
- Remove any finding that rests on speculation.

Example output

AI-written example

1. Verdict: BLOCK

  • Refund creation checks the idempotency key and inserts it in separate steps, so concurrent retries can still double-refund.
  • The migration adds a non-unique index, so the database does not enforce the guarantee the change promises.
  • Both issues are in the payment path and have straightforward fixes.

2. Assumptions and missing information

  • Repo: /workspace/billing-service, origin/main...feature/idempotent-refunds (4 commits, 5 files, +212/−18). All 5 files were reviewed.
  • Intent: "Add idempotency keys to POST /refunds." No design doc was provided.
  • Ran pytest tests/refunds -q -p no:cacheprovider: 38 passed, 0 failed. Integration tests needing Postgres were not run (no DB in sandbox).

3. Findings

IDSeverityFile:lineIssueEvidenceSuggested fix
F1Blockerapp/refunds/service.py:74-89Check-then-insert race on the idempotency keyget_by_key() at L74, create_refund() at L86, no transaction or lock between themInsert the key first and handle IntegrityError by returning the stored response, or use SELECT ... FOR UPDATE
F2Blockermigrations/0042_refund_idempotency.py:12Index is not uniqueop.create_index(..., unique=False)Use unique=True on (merchant_id, idempotency_key)
F3Majorapp/refunds/api.py:41Same key with a different payload silently returns the old resultHandler never compares the request hashStore a payload hash and return 422 on mismatch
F4Minorapp/refunds/service.py:95Key TTL hardcoded to 24hLiteral 86400Move to settings.REFUND_KEY_TTL
F5Nittests/refunds/test_api.py:18Duplicate fixtureSame merchant fixture as conftest.py:9Reuse the shared fixture

4. Test coverage gaps

  • [ ] Two concurrent requests with the same key create exactly one refund
  • [ ] Same key, different amount returns 422
  • [ ] Expired key is accepted as a new request
  • [ ] Same key used by two different merchants does not collide
  • [ ] Missing or oversized key (>255 chars) returns 400
  • [ ] Migration downgrade removes the index cleanly

5. Open questions for the author

  • Should failed (5xx) refund attempts store their key, or be retryable with the same key?
  • Is the 24h TTL a product requirement or a placeholder?

6. Positive observations

  • The new IdempotencyRecord model is small and clearly named.
  • Error responses reuse the existing problem+json format.
  • Commit history is clean and easy to follow.

7. Next actions

  1. Fix F1 and F2, then add the concurrency test.
  2. Add payload-hash validation (F3) with a test.
  3. Run the Postgres integration suite in CI.
  4. Address F4 and F5 at your discretion.
Generated from this prompt by AI. Your results will vary.

Tips and common edits

Run with codex exec in read-only mode from the repo root, after fetching both refs locally. Set {risk_focus} to steer attention toward the riskiest areas. Treat the output as a first-pass review that supplements, not replaces, human approval.

Questions about this prompt

More in Codex

Codex Pull Request Review With Comments Ranked by Severity

Has Codex review a pull request and return comments sorted into blocking, should-fix and nit, each with the file and line, the risk, and a suggested change.

Pull Request Review with Merge Recommendation

Reviews a diff against your standards and returns a merge, merge-with-changes, or do-not-merge recommendation. Findings are ranked by severity, with evidence, suggested fixes, and open questions.

Codex Read-Only Repo Investigation for a Bug Report

Runs a read-only Codex investigation that maps the relevant code paths, test commands, and likely root causes for a bug report. It ends with a ranked hypothesis table and a go/no-go recommendation before any edits.

Conventional Commit and PR Description from a Diff and Ticket

Produces a Conventional Commits message and a reviewer-ready PR description covering what changed, why, how to test it, and the risks, based on your diff and ticket.

Pre-PR Self-Review and Pull Request Description from a Git Diff

Has GitHub Copilot review your branch diff against the base branch, flag defects and missing tests, and draft a PR description with evidence. Use it to decide whether to open the PR or fix issues first.

Codex Characterization-Test-First Refactor with Verification Handoff

Runs a behavior-preserving refactor in Codex exec. It pins current behavior with characterization tests, refactors in small verified steps inside an allowed scope, and ends with an evidence-based handoff for the merge decision.

Codex CI Failure Triage: Reproduce, Classify, Fix or Hand Off

Guides Codex through reproducing a failing CI job locally, classifying the cause as code, test, environment, or flaky, applying a minimal fix when safe, and producing an evidence-based handoff for anything it cannot resolve.

Codex Exec CI Fixer: Diagnose, Minimal Fix, Rerun, Stop After 3 Attempts

Gives codex exec a non-interactive procedure to diagnose a failing CI output, apply a minimal fix, rerun the checks and stop after three attempts with a clear report.

Outcome-First Codex Task Writer for a Feature or Bug Request

Turns a raw feature or bug request into a paste-ready Codex task with a goal, constraints, files in scope, definition of done and exact verification commands.

Codex Writes an AGENTS.md for This Repository From the Real Setup

Has Codex inspect your repository and write a short, accurate AGENTS.md with the real build, test and lint commands, conventions and boundaries, checked by actually running them.

Pre-Merge Code Review Checklist Skill (SKILL.md)

A save-ready SKILL.md that has your coding agent review a branch before merge: correctness, security, tests and migrations, with findings ranked by severity and no style nitpicks.

Coding Agent Task Brief with Guardrails, Test Plan, and Rollback Steps

Creates a scoped brief for an AI coding agent that limits file changes, defines tests, and includes a rollback plan and review checklist.