Codex Characterization-Test-First Refactor with Verification Handoff

Runs a behavior-preserving refactor in Codex exec. It pins current behavior with characterization tests, refactors in small verified steps inside an allowed scope, and ends with an evidence-based handoff for the merge decision.

You get: Verdict, baseline vs final and characterization tests added

More like this

The prompt

codex-characterization-test-first-refactor-with-verification-handoff.md
Role
You are Codex running non-interactively in {repo_path} through codex exec. You act as a careful senior engineer doing a behavior-preserving refactor. You do not change product behavior, public APIs, or files outside the allowed scope.

Context
The user wants to refactor {target_module} to achieve this goal: {refactor_goal}. The decision this run supports: should the refactor be merged as-is, merged after follow-ups, or held back? Base ref for comparison: {base_ref}. Repository instructions in AGENTS.md, CONTRIBUTING, or README take priority over your defaults.

Task
Pin the current behavior of {target_module} with characterization tests, refactor in small steps, and prove through command output that behavior is unchanged.

Inputs
- repo_path: {repo_path}
- target_module: {target_module}
- refactor_goal: {refactor_goal}
- base_ref: {base_ref}
- allowed_paths: {allowed_paths}
- forbidden_paths: {forbidden_paths}
- test_command: {test_command}
- lint_command: {lint_command}
- typecheck_command: {typecheck_command}
- max_files_changed: {max_files_changed}
If an input is blank, discover it from package manifests, Makefile, CI config, or AGENTS.md and list it under Assumptions. Do not invent commands. If you cannot find a working test command, stop after Step 1 and report it as a blocker.

Workflow
1. Orient: read AGENTS.md and repo docs, run git status and git log -n 5, and confirm the working tree is clean or list pre-existing changes you will not touch. Locate {target_module}, its callers, its existing tests, and its public surface (exports, routes, CLI flags, schemas).
2. Baseline: run {test_command}, {lint_command}, and {typecheck_command} before editing. Record pass/fail and any pre-existing failures. Pre-existing failures are not yours to fix; record them.
3. Characterize: find behavior in {target_module} that existing tests do not cover (branches, error paths, edge inputs, side effects). Add characterization tests that assert current actual behavior, even if it looks odd. Mark suspected bugs with a comment and do not fix them. Re-run tests and confirm the new tests pass on unmodified code.
4. Plan: write a numbered refactor plan of small steps, each with its expected touched files and the check that verifies it. Confirm every file is within {allowed_paths} and the total stays within {max_files_changed}. If it will not, shrink the scope and note what was deferred.
5. Refactor: apply one step at a time. After each step run the narrowest relevant tests; after the last step run the full {test_command}, {lint_command}, and {typecheck_command}. If a step breaks a characterization test, revert that step and try a smaller one. Never edit a characterization test to make it pass.
6. Compare: run git diff --stat {base_ref}, confirm no forbidden or out-of-scope files changed, and confirm the public surface is unchanged or list each difference.
7. Hand off: do not commit or push unless explicitly told. Leave changes in the working tree.

Constraints
- No behavior changes, dependency upgrades, formatting-only sweeps, or unrelated cleanup.
- Do not touch {forbidden_paths}, secrets, lockfiles, or generated files unless required and justified.
- No network access unless the test command requires it; say so if it does.
- Report only commands you actually ran and their real results. Never claim a check passed if it was not run.
- If blocked, stop and report the blocker, what you tried, and what input would unblock you.

Output format
Return the final message in this structure:
1. Verdict: Ready to merge / Merge after follow-ups / Hold, with a one-sentence reason.
2. Assumptions: bullets for inferred inputs and commands.
3. Baseline vs final: table with columns Check | Command | Before | After | Notes.
4. Characterization tests added: table with columns File | Behavior pinned | Suspected bug (Y/N).
5. Refactor steps: table with columns Step | Files | Verification run | Result.
6. Diff summary: files changed, lines added/removed, and public-surface differences (or none).
7. Risks and missing information: bullets with severity (High/Med/Low) and what a reviewer should check by hand.
8. Next actions: ordered checklist, including exact commands to re-verify locally.

Acceptance criteria
- Baseline and final results exist for every check that could be run.
- At least one characterization test was added for each previously uncovered branch you touched, or a justification is given.
- Every changed file is inside {allowed_paths} and the count is at most {max_files_changed}.
- No characterization test was weakened or deleted.
- The public surface is unchanged, or each change is listed and flagged.
- All remaining failures are identified as pre-existing or as new, with evidence.

Quality checks
Before the final message, confirm: (a) each claim in the report maps to a command you ran; (b) git diff contains no unrelated edits; (c) suspected bugs are reported, not fixed; (d) the verdict matches the evidence; (e) next actions are specific enough to run without further questions.

Example output

AI-written example
  1. Verdict: Merge after follow-ups. All checks match baseline and the public surface is unchanged, but one suspected bug is pinned and not fixed, and it needs an owner decision.
  2. Assumptions
  • Commands came from package.json and AGENTS.md: npm test, npm run lint, npm run typecheck.
  • allowed_paths was blank, so I used src/billing/ and test/billing/. max_files_changed was blank, so I used 6.
  • Working tree was clean at a41c9e7. No network was needed.
  1. Baseline vs final
CheckCommandBeforeAfterNotes
Testsnpm test212 pass, 1 skipped221 pass, 1 skipped+9 characterization tests; skip is pre-existing
Lintnpm run lintPassPassNone
Typesnpm run typecheckPassPassNone
  1. Characterization tests added
FileBehavior pinnedSuspected bug
test/billing/proration.characterization.test.tsMid-cycle upgrade rounds half-up per line itemN
sameZero-day remaining period returns 0 and no credit lineN
sameNegative quantity throws InvalidQuantityErrorN
sameLeap-day cycle uses 366-day divisorN
sameDowngrade on the final day of the cycle yields a credit of 0.01 too manyY

The remaining 4 tests cover currency-minor-unit handling and the empty-items path.

  1. Refactor steps
StepFilesVerification runResult
1. Extract daysRemaining()src/billing/proration.tsnpm test -- prorationPass
2. Extract roundLine()src/billing/proration.tsnpm test -- prorationPass
3. Move helpers to proration-helpers.tssrc/billing/proration.ts, src/billing/proration-helpers.tsnpm test -- billingPass
4. Replace nested conditionals with early returnssrc/billing/proration.tsnpm test -- billing, then the full test, lint and typecheck commandsPass

An attempt to merge the upgrade and downgrade branches broke the leap-day test. I reverted it and did not retry.

  1. Diff summary
  • git diff --stat origin/main: 3 files changed, 214 insertions, 87 deletions.
  • Files: src/billing/proration.ts, src/billing/proration-helpers.ts (new), test/billing/proration.characterization.test.ts (new).
  • No forbidden paths or lockfiles were touched.
  • Public surface: unchanged. The exports of src/billing/index.ts are identical, and the new helpers are not re-exported.
  1. Risks and missing information
  • Med: The final-day downgrade credit looks like
Generated from this prompt by AI. Your results will vary.

Tips and common edits

Run from the repo root, for example: codex exec --sandbox workspace-write "<prompt with variables filled>". Adjust flags to your installed Codex version. Keep the scope small, since large modules work better when split into several runs. Review the characterization tests yourself, because they pin current behavior including quirks.

Questions about this prompt

More in Codex

Codex CI Failure Triage: Reproduce, Classify, Fix or Hand Off

Guides Codex through reproducing a failing CI job locally, classifying the cause as code, test, environment, or flaky, applying a minimal fix when safe, and producing an evidence-based handoff for anything it cannot resolve.

Codex Exec CI Fixer: Diagnose, Minimal Fix, Rerun, Stop After 3 Attempts

Gives codex exec a non-interactive procedure to diagnose a failing CI output, apply a minimal fix, rerun the checks and stop after three attempts with a clear report.

Codex Read-Only Repo Investigation for a Bug Report

Runs a read-only Codex investigation that maps the relevant code paths, test commands, and likely root causes for a bug report. It ends with a ranked hypothesis table and a go/no-go recommendation before any edits.

Outcome-First Codex Task Writer for a Feature or Bug Request

Turns a raw feature or bug request into a paste-ready Codex task with a goal, constraints, files in scope, definition of done and exact verification commands.

Codex Pre-Merge Diff Review with Severity-Ranked Findings

Has Codex review a branch diff against a base ref in read-only mode. It checks correctness, tests, security, and maintainability, and returns evidence-backed findings with severity and a merge recommendation.

Codex Pull Request Review With Comments Ranked by Severity

Has Codex review a pull request and return comments sorted into blocking, should-fix and nit, each with the file and line, the risk, and a suggested change.

Codex Writes an AGENTS.md for This Repository From the Real Setup

Has Codex inspect your repository and write a short, accurate AGENTS.md with the real build, test and lint commands, conventions and boundaries, checked by actually running them.

Agent Run Pre-Mortem and Self-Correction Loop Designer

Designs verification checks, error handling, and a self-correction loop for an agent task before it runs, based on likely failure modes.

Characterization Tests Before a Refactor

Writes tests that lock in a module's current behavior, including edge cases and quirks, so you can refactor safely.

Claude Code Behavior-Preserving Refactor with Characterization Tests and Staged Commits

Guides Claude Code to refactor a module safely by first pinning current behavior with characterization tests, then applying small reversible steps with verification after each. Delivers a diff, risk register, and rollback plan.

Cursor Characterization-Test-First Refactor of a Legacy Module

Has Cursor lock in current behavior with characterization tests before restructuring a legacy module in small steps, so you can decide whether the refactor preserved behavior and is safe to merge.

Cursor Refactor of a Component With Zero Behaviour Change

Has the Cursor agent restructure a messy component into smaller pieces while proving the behaviour stays identical, using existing tests or new characterization tests first.