How to evaluate an LLM feature before you ship it

Teams ask us how to evaluate an LLM feature far less often than they should. The usual approach is to read a handful of outputs, decide they look good, and ship. That works right up until a prompt tweak quietly breaks a case nobody re-checked. Evaluation is what replaces “looks good to me” with a number you can defend.

Here’s a practical way to evaluate an LLM feature — before and after it goes live.

Start with a golden set

Before you can score anything, you need examples worth scoring. A golden set is a curated collection of representative inputs paired with either known-good outputs or a clear rubric for what “good” means.

• Pull real cases from logs or user requests, including the awkward ones.
• Cover the edge cases that matter: long inputs, missing fields, adversarial phrasing.
• Keep it version-controlled and growing — every bug you find becomes a new test.

A golden set of fifty real cases beats a thousand synthetic ones.

Offline evaluation: score before you ship

Offline evaluation runs your feature against the golden set on every change, so you catch regressions before users do:

• Deterministic checks for anything you can assert exactly — formats, schemas, required fields.
• Model-graded scoring for open-ended quality, with a rubric and a stronger model as judge.
• A pass threshold wired into delivery, so a change that drops quality doesn’t merge.

The goal isn’t a perfect score; it’s a baseline you can hold and improve.

Online evaluation: guardrails after launch

Offline tests can’t see everything real users will do, so launch behind guardrails and keep measuring:

• Log inputs and outputs (within your privacy rules) so you can build new test cases from production.
• Track proxy signals — thumbs, edits, retries, escalations — that hint at quality without manual review.
• Roll out behind a flag and compare against the previous version before going wide.

Make it a habit, not an event

The teams that ship reliable AI don’t evaluate once at the end; they evaluate continuously, and every incident adds a case to the set. Evaluation stops being a gate you dread and becomes the thing that lets you change prompts and models with confidence. That’s the difference between iterating on an AI feature and being afraid to touch it.