Evaluate the application, not just the model.
A model score from a public benchmark may have little relationship to whether a specific employee can complete an approved workflow or whether a product user will trust a generated answer.
Which failure would stop you from shipping?
Make that failure a release case with an expected behavior, a reproducible setup and a named review responsibility. A public model score is not an application's acceptance decision.
The evidence behind a release decision.
This illustrative dossier exposes what must be judged. It contains no test results: outcomes belong to an actual, versioned evaluation run.
Answer an exact-value question
- Case setup
- A versioned source contains a value and a nearby qualification.
- Acceptance condition
- The value, applicable condition and supporting passage agree. A plausible answer without the qualification fails.
- Review responsibility
- Domain reviewer; deterministic checks where the expected output is exact.
Recognize insufficient evidence
- Case setup
- The requested fact is absent, or only an obsolete source supports it.
- Acceptance condition
- Return a clear no-answer or request the missing information. Do not reward a confident guess.
- Review responsibility
- Domain reviewer calibrates automated judgments against inspected examples.
Respect another tenant's boundary
- Case setup
- A tempting answer exists only in a source this identity cannot read.
- Acceptance condition
- The source never reaches context and no title, snippet or result exposes it.
- Review responsibility
- Security and engineering review the access trace, beyond answer text.
Survive a model or prompt change
- Case setup
- Replay the held regression set with the candidate model, prompt, retrieval and tool versions.
- Acceptance condition
- Review costly tail failures, evidence specificity, elapsed time and cost against agreed limits.
- Review responsibility
- The release owner considers automated evidence and human exception review.
Illustrative acceptance cases. Acceptance criteria and release authority must be agreed for the real task.
Ask for the cases behind the score.
- A representative dataset with provenance, difficult cases and a separately held regression set.
- Human calibration examples explaining when an automated judge is wrong.
- A run manifest tying outputs and review decisions to exact application versions.
Automate coverage; retain judgment.
Deterministic tasks deserve deterministic checks. Open-ended tasks may require expert review. More automated judging is useful only when its errors and blind spots are understood for the decision at hand.
The application evaluation argument
For CTO, CIO, CPO, COO
The situation behind the request
A team changes a model version. Offline accuracy improves, but the assistant begins answering with less specific references and spending more time on requests that should return “not enough evidence.”
Where the obvious solution falls short
Generic judges and happy-path prompts can miss permission leakage, extraction errors and user-interface failure. Comparing only model outputs ignores changes in retrieval and tools.
How we would approach the work
Define task-specific acceptance cases, make failure categories visible and hold a representative regression set. Include retrieval evaluation, answer evidence, authorization, latency, cost and human review.
What the engagement may include
- Evaluation taxonomy and representative datasets.
- Reproducible harness, prompt/model/source version control.
- Human calibration for automated evaluator scores.
- Quality gate and monitoring/rollback plan.
The choices worth making explicitly
Automation can broaden test coverage but cannot replace domain and security judgment. Some tasks need deterministic expected outputs; others require expert review.
What we would test before calling it ready
Include false positives, no-answer, stale sources, adversarial inputs, cross-tenant access and upgrade regressions. Review not only average performance but costly tail failures.
A sensible first step
Identify the failure most expensive to the buyer and make it a first-class acceptance case.