Write down what an acceptable result contains
“The assistant answers questions well” leaves too much room for interpretation. A support assistant may need to identify a documented limitation, explain the next supported step and avoid inventing a refund promise. Those requirements can become evaluation criteria before a model is selected. The examples then describe the service a user should receive.
The Datasheets for Datasets paper proposes documenting a dataset's motivation, composition, collection process and recommended uses. An internal evaluation set benefits from the same questions. Where did these examples come from? Which users and situations are absent? Who supplied the reference answer, and what information was available when it was written?
Build cases around behavior
CheckList presents behavioral testing as a way to examine NLP systems beyond aggregate held-out accuracy. Its work motivates testing particular capabilities and transformations. It does not make a small hand-written suite statistically representative of an entire user population.
For a practical first suite, group examples by the decision they test. Include ordinary requests, ambiguous wording, missing evidence, contradictory documents and requests outside the product's remit. Add variations that preserve meaning while changing formatting or terminology. Label synthetic examples as synthetic so nobody later interprets them as observed customer demand.
Reference answers should explain their basis. A bare sentence can conceal an outdated policy or a judgment call. Keep a source version and a short rationale with the expected result. When qualified reviewers disagree, record the disagreement and refine the criterion. Forcing a single label before understanding the dispute can turn the evaluation into an arbitrary style preference.
Separate development feedback from acceptance evidence
Use one collection to diagnose failures and another reserved collection to check candidate changes. Once a difficult acceptance example has been repeatedly inspected and used to tune a prompt, describe that exposure in the run record. Its score can remain useful for regression checks while providing weaker evidence of performance on unfamiliar cases.
Report results by meaningful groups as well as overall. A system can handle common lookup questions while failing requests that require abstention. Combining both into one average can hide the behavior that matters most to an operator. Keep sample counts visible; a strong-looking percentage from a tiny group invites more confidence than the evidence supports.
A model-based judge can help organize review, but its instructions, version and disagreements need inspection. For consequential criteria, include human examination of a documented sample and concrete failure examples. Agreement between automated graders alone does not demonstrate that the answers are supported by the source material.
Keep the evaluation connected to operation
Google's ML Test Score paper treats testing and monitoring as production-system concerns. Our application of that principle is to maintain an evaluation release alongside the product release: examples, evidence snapshot, criteria and grader configuration should all be identifiable.
When a real failure arrives, first investigate the individual case. Then decide whether it reveals a missing class of behavior worth adding to the suite. This keeps the collection connected to observed use without silently rewriting its history. A useful evaluation set becomes a maintained account of the product's promises, with explicit limits on what its passing results establish.