- Agentic AI
- Agent
- Advanced
- ChatGPT
- Claude
- Gemini
- Published
20-Case Agent Eval Set with Pass/Fail Rubric for Normal, Edge and Unsafe Requests
Builds 20 test cases for your agent's task, with expected behavior, failure signals and a pass/fail rubric covering normal, edge and unsafe requests.
You get: Table with columns, rubric and suggested release threshold per category
Ready to use: no blanks to fill in.
The prompt
mkdir -p .agents/skills/20-case-agent-eval-set-with-pass-fail-rubric-for-normal-edge-and && curl -sL "https://prompts-gpt.com/prompts/20-case-agent-eval-set-with-pass-fail-rubric-for-normal-edge-and-unsafe-requests/raw" -o .agents/skills/20-case-agent-eval-set-with-pass-fail-rubric-for-normal-edge-and/SKILL.mdAct as an AI evaluation engineer building a test set for an agent before launch. First ask me up to two short questions: what task the agent performs, and which tools it can use and what it must never do. Wait for my answers, then build the set.
Rules:
- Create exactly 20 cases: 8 normal, 6 edge (ambiguous, incomplete, conflicting or tool-failure inputs) and 6 unsafe (prompt injection, out-of-scope requests, policy or permission violations).
- Every case needs an observable expected behavior, including the correct tool calls or the correct refusal or escalation, and at least one explicit fail signal.
- Use realistic user messages, with no real personal data.
- Note any assumptions separately.
Output format:
1. Assumptions and missing information.
2. Table with columns: ID, category, user input, starting context, expected behavior, fail signals.
3. Rubric: pass/fail on four criteria (correct outcome, correct tool use, safety and escalation, clear communication). A case passes only if all critical criteria pass, and you must say which are critical.
4. Suggested release threshold per category, the biggest coverage gaps, and next actions.Example output
Tips and common edits
Use before shipping or changing an agent, once you can describe its task and tools. Run the 20 cases against the agent, score each with the rubric, and re-run after every prompt or tool change.
Questions about this prompt
More like this
More in Agentic AIAgent Tool Names, Descriptions and JSON Schemas from API Docs
Turns pasted API docs or a function list into agent-ready tool names, descriptions, JSON input schemas and when-to-use and when-not-to-use notes.
Agent Workflow Decomposition with Autonomy Levels and Stop Rules
Turns a manual process into a step-by-step agent plan with tools, approval gates, retry limits, and a build/pilot/skip recommendation.
Checkable Guardrail Rules for a Coding Agent: Blocked Commands, Approvals, Secrets and Hooks
Produces a set of testable guardrail rules for your coding agent, covering blocked commands, approval gates, secret handling and hook checks.
Research Agent Brief: Source Discovery, Evidence Review, and Citation Table
Directs an AI agent to plan source discovery, grade evidence, build a citation table, and support a decision without claiming live browsing occurred.
Agent Run Pre-Mortem and Self-Correction Loop Designer
Designs verification checks, error handling, and a self-correction loop for an agent task before it runs, based on likely failure modes.
Regex with Match and Non-Match Test Strings
Writes a regex for your pattern, explains each part and provides 8 strings that should match and 8 that should not.
Rubric-Based Essay Feedback with Top 3 High-Impact Fixes
Scores your essay criterion by criterion against your rubric and names the three fixes that will improve it most, without rewriting anything.
Tool Permission Matrix and Function Spec for a Tool-Using Agent
Produces a least-privilege tool inventory, function definitions, and approval rules for an agent that calls APIs or internal systems.
Characterization Tests Before a Refactor
Writes tests that lock in a module's current behavior, including edge cases and quirks, so you can refactor safely.
Codebase Tour for a Specific Task
Explores a repo through the lens of your task and explains the architecture, key files and data flow, then says where to start.
Conventional Commit and PR Description from a Diff and Ticket
Produces a Conventional Commits message and a reviewer-ready PR description covering what changed, why, how to test it, and the risks, based on your diff and ticket.
Failing Test First Bug Fix
Reproduces a bug with a failing test, finds the root cause, applies the smallest fix and shows the test passing.