How to Evaluate AI Prompts: A Repeatable Regression Workflow

Most prompt changes are judged by running the new version once, reading the answer and deciding it looks better. That works until an edit that fixes one case quietly breaks three others. Prompt evaluation replaces that one-off impression with a repeatable check: the same inputs, the same scoring, run against every version you consider shipping.

This guide shows a workflow you can run with a spreadsheet, an open-source tool or a hosted product. The examples are illustrative; the method is what matters.

What a prompt evaluation is (and is not)

A prompt evaluation runs a prompt against a fixed set of test inputs and scores the outputs. A regression evaluation does this for two or more versions of the same prompt so you can see what an edit changed, case by case.

Two things are often confused with it:

• Grading the prompt text. A prompt linter or grader reads the prompt itself and flags missing ingredients such as a role, an output format or examples. That is useful before you start, but it says nothing about the outputs a model actually produces.
• Benchmarks. Public model benchmarks measure models on generic tasks. Your evaluation measures your prompt on your inputs.

The vendors who build models give consistent advice here. OpenAI's evaluation guide recommends that you "Adopt eval-driven development: Evaluate early and often" and lists "Vibe-based evals" as an anti-pattern (OpenAI evaluation best practices, checked 2026-11-05). Anthropic's guide says to "Design evals that mirror your real-world task distribution" and to "Automate when possible" (Anthropic: define success criteria and build evaluations, checked 2026-11-05).

Step 1: Decide what "good" means

Before writing test cases, write down the two to five properties that make an output acceptable for this prompt. Make them observable. "High quality" is not observable; "lists every changed file" is.

For an illustrative prompt that turns a git diff into a pull request description, success criteria might be:

1. Faithful: mentions only changes that are in the diff.
2. Complete: covers each changed area.
3. Structured: uses the Summary / Changes / Testing sections.
4. Proportionate: short for small diffs, with a Risks section only when migrations change.

Each criterion should map to at least one check in Step 4.

Step 2: Build a test set

A test set is a list of inputs, each with a name and, where possible, what a good output must or must not contain. Good test sets mix three kinds of case:

• Typical inputs that look like everyday usage.
• Edge cases that are rare but expensive to get wrong: empty input, very long input, mixed languages, inputs that invite the model to invent something.
• Regression cases copied from real failures. Every time a prompt fails in use, turn that input into a case so the failure cannot come back unnoticed.

There is no universal right size. Start with a handful of cases that cover your criteria, run them on every change, and add cases as you discover failures. Anthropic's guidance leans toward more automatically graded cases over fewer hand-graded ones; Promptfoo's workflow ends with a feedback loop of expanding test cases as you collect examples and user feedback (Promptfoo introduction, checked 2026-11-05).

Illustrative example — not a measured result. A starter set for the PR-description prompt:

Case	Input	Must include	Must not include	Length
migration-change	Diff adding a column in migrations/	"Risks"	—	—
docs-only	Diff touching only README.md	"README"	"migration", "Risks"	≤ 600 chars
large-refactor	Diff renaming a module across many files	"Summary", "Changes"	—	—
typo-fix	One-line spelling fix	—	"Risks"	≤ 300 chars
empty-diff	Empty string	"No changes"	"Changes:"	≤ 200 chars

Keep inputs realistic. A test set built from invented, tidy inputs tends to pass while real usage fails.

Step 3: Write a rubric

Some criteria (faithfulness, tone, usefulness) cannot be reduced to string checks. A rubric turns them into something a reviewer, human or model, can apply consistently. Keep the scale small and define every level.

Illustrative rubric:

Score the PR description from 0 to 100 using these criteria:

Faithfulness (0-40): 40 = every claim is supported by the diff; 20 = one minor
unsupported detail; 0 = invents a change, file or migration.
Completeness (0-30): 30 = every changed area is mentioned; 15 = one area missing;
0 = most changes missing.
Structure (0-20): 20 = Summary, Changes and Testing sections present and in order;
10 = one missing; 0 = unstructured.
Proportion (0-10): 10 = length fits the size of the diff and Risks appears only
for migration changes; 0 = padded or a spurious Risks section.

Return the score and a one-sentence reason naming the biggest problem.

Write the rubric before you look at outputs from the new prompt version, so you are not tuning it to make your preferred version win.

Step 4: Combine deterministic and model-graded checks

Check type	Examples	Strengths	Weaknesses
Deterministic	Required terms, forbidden terms, length limits, valid JSON, regex, exact match	Repeatable, free to rerun, easy to explain	Only catches what you can spell out in advance
Model-graded	A judge model scores the output against your rubric	Handles fuzzy criteria like faithfulness and tone	Varies between runs and judge models; can be confidently wrong
Human review	You or a teammate read a sample of outputs	Highest judgment quality	Slow; does not scale to every change

Use deterministic checks wherever a criterion allows it, and add a model judge for what is left. Two habits make model grading more trustworthy:

• Use a different model as the judge than the one that produced the output. Anthropic's eval examples note that it is "Generally best practice to evaluate with a different model from the one used to generate the evaluated output" (Anthropic develop tests, checked 2026-11-05).
• Calibrate the judge against people. OpenAI's guide recommends you "Use human feedback to calibrate automated scoring" (OpenAI evaluation best practices, checked 2026-11-05). Read a sample of judged outputs yourself; if you disagree with the judge often, fix the rubric before trusting the scores.

Step 5: Record the settings that affect results

An evaluation is only reproducible if you can rerun it under the same conditions. Record, for every run:

• Prompt version (or the full prompt text) and any system prompt.
• Model identifier exactly as you called it, plus the provider or router.
• Sampling settings: temperature, top_p and maximum output tokens. Higher temperature means more run-to-run variation; see our temperature and top_p guide.
• Judge model and rubric text, if you used model grading.
• Date of the run. Hosted models can change behind the same name.
• Number of runs per case. At non-zero temperature, one run per case can mislead; repeat important cases and look at the spread.
• Latency and token counts, if cost or speed is part of your criteria.

Step 6: Run, compare and read the failures

Run the old and new prompt versions against the full test set. Then read failures before you read averages: an average can rise while a critical case gets worse.

Illustrative failure examples — constructed to show what to look for, not measured results.

• Hallucinated content. On empty-diff, the new version writes a full description of changes that do not exist. The must-include, must-not-include and length checks all fail; a judge would give faithfulness 0. Fix: instruct the model to say "No changes found" when the diff is empty.
• Over-applied instruction. On typo-fix, the output adds "Risks: none." The forbidden-term check catches it. Fix: make the Risks rule conditional and say what to do otherwise.
• Judge disagreement. On large-refactor, the judge scores 85 while a human reviewer notices a missing module. Fix the rubric to name "every renamed module" explicitly, then rerun.
• Format drift. Sections appear in a different order on one model but not another. Decide whether order matters; if it does, add a deterministic check.

Step 7: Iterate one change at a time

1. Change one thing in the prompt.
2. Rerun the whole test set, not just the case you were fixing.
3. Accept the change only if the cases you care about improve and none of your must-pass cases regress.
4. Turn each new failure you find in real usage into a test case.
5. Write a change note on the new prompt version explaining what the evaluation showed.

Promptfoo, an open-source CLI and library for this workflow, describes the same loop: define test cases, configure the evaluation, run it, analyze results and expand the test cases as you learn (Promptfoo introduction, checked 2026-11-05). It is a good fit if you want evaluations in your repository and CI.

Running this workflow in HyperPrompt

HyperPrompt includes evaluation suites for prompts you manage there. Here is what the current product does, so you can map the steps above onto it. See the prompt evaluation feature page and the evaluations documentation for walkthroughs.

• Suite. A suite holds a name, a description, a prompt template that uses {{input}} for each case's input, and a judge rubric (Steps 1 and 3).
• Test cases. Each suite can hold up to 20 cases. A case has a name, an input, an optional expected output or key terms, and deterministic assertions: must-include terms, must-not-include terms, and minimum and maximum character counts (Steps 2 and 4).
• Running. You choose one to three OpenRouter model IDs and, optionally, a separate judge model. Leaving the judge blank runs deterministic checks only. Each run is capped at 10 candidate calls and 10 judge calls, so cases multiplied by models must stay within 10. Running evaluations requires a paid plan; see pricing.
• Results. For each case and model you see the output, an assertion pass rate, a term-coverage percentage (the share of distinct words in your expected output that also appear in the response, a rough signal rather than a semantic match), a 0–100 judge score with a short reason when a judge model is set, latency and token count.
• Settings to note (Step 5). Candidate calls currently use a fixed temperature of 0.7 and a maximum of 2,000 output tokens; these are not configurable per run. Because of that, outputs can vary between runs, so rerun a suite before treating a small score difference as real.

HyperPrompt does not run your evaluations in CI or trace production traffic. If you need that, a tool such as Promptfoo in your repository is a better fit, and you can keep using HyperPrompt suites for the prompts your team runs by hand.

Where the free prompt grader fits

The free prompt grader is a lint step, not an evaluation. It runs a heuristic checklist in your browser over the prompt text, looking for signals such as a stated role, an action verb, an output format, examples or delimiters, template placeholders and enough detail, and suggests what to add. It does not call a model or look at outputs. Use it to tidy a prompt before you write test cases, then evaluate the outputs with the workflow above.

Common mistakes

• Testing only the case you just fixed. Always rerun the full set.
• Letting the judge grade its own output. Use a different judge model and spot-check it.
• Changing the rubric and the prompt at the same time. You will not know which change moved the score.
• Ignoring run-to-run variation. At non-zero temperature, one run is a sample, not a verdict.
• Never adding cases. A test set that never grows stops reflecting how the prompt is used.

Related reading

• What is prompt management?
• Temperature and top_p settings
• System prompts vs user prompts
• How to write better AI prompts