# How do you evaluate a prompt change without a labelled test set?

- **Asked by:** [Kabir Singh (@kabir_singh)](https://www.promptabide.com/kabir_singh)
- **Asked:** 2026-02-26
- **Answers:** 2
- **Upvotes:** 62
- **Views:** 1945

## Question

Every guide starts with 'assemble a golden set of 200 examples'. We do not have one and building it is weeks of work. Is there a defensible middle ground for deciding whether a prompt edit made things better, or is it labels or nothing?

## Answers

### Accepted answer

- **By:** [Dmytro Shevchenko (@dmytro_shevchenko)](https://www.promptabide.com/dmytro_shevchenko)
- **Answered:** 2026-08-22
- **Upvotes:** 67

You do not need 200 labelled examples to start. Take 20 real inputs, run both prompts, and look at the outputs side by side yourself. Twenty is enough to catch a regression that matters, and you will build the labelled set as a by-product of the reviewing.

Blind yourself to which output came from which prompt. Otherwise you will find the new one better, because you wrote it.

### Answer

- **By:** [Riya Bansal (@riya_bansal)](https://www.promptabide.com/riya_bansal)
- **Answered:** 2026-08-22
- **Upvotes:** 178

Format rules go in the system prompt, and it caches, which matters more as the prompt grows.

The "closer to the end is followed better" effect is real but it is about recency across the whole assembled context, not about which field the text sits in. If you have a long user message, a one-line reminder at the end of it is cheap and closes most of the gap without duplicating 400 tokens.

---

Canonical HTML: https://www.promptabide.com/questions/evaluating-prompt-changes-no-labels
Agent guide: https://www.promptabide.com/llms.txt · https://www.promptabide.com/agent-instructions.md
Sitemap: https://www.promptabide.com/sitemap.xml