A demo copilot has to be impressive for fifteen minutes, on inputs the presenter half-chose in advance. A production copilot has to survive eight hours a day of real, messy, adversarial-by-accident input from people who did not read a prompt guide. Building for the first goal actively works against the second, and we have shipped both kinds long enough to know exactly where the paths split.
The fifteen-minute trap
Demos reward breadth — the copilot that can answer questions about sales, support, and scheduling in one chat window looks more impressive than one that only does scheduling well. In daily use, that breadth becomes the problem: a support agent typing a scheduling question does not want a chatbot guessing at intent across three domains, they want a fast, narrow tool that is right almost every time on the one thing they actually asked for.
We now scope copilots the opposite way from how they get pitched. One domain, one tool surface, a hard boundary on what it will decline to answer — and we say that boundary out loud in the UI rather than letting the model improvise an answer it is not equipped to give well.
Latency is a feature, not an ops metric
A three-second response feels magical once, in a demo. It feels like friction the fortieth time someone uses it before lunch. We treat p95 latency as a product requirement scoped alongside accuracy, not an infrastructure concern bolted on afterward — which usually means choosing a smaller, faster model for the narrow task instead of routing everything through the biggest general-purpose one available.
- Route simple, high-frequency requests to a smaller/faster model; reserve the large model for genuinely hard cases.
- Stream partial responses for anything that takes over a second — perceived latency drops even when total time does not.
- Set a timeout with a graceful fallback to the manual path, never a spinner that runs forever.
The eval set that is not a demo script
Before launch we build an evaluation set from real historical requests, including the ugly ones — typos, half-finished sentences, questions outside the tool's scope. A copilot that scores well on hand-picked demo questions and poorly on this set is not ready, no matter how good the pitch looked. We re-run the eval set on every prompt or model change, the same way a test suite gets re-run on every code change.
eval_set.jsonl -> 340 real historical requests
pass_rate -> 91% (target: 90%)
out_of_scope -> correctly declined 97% of the timeWhat ships in the copilot, and what does not
The demo trap's last form is scope creep after launch: "since it's already talking to users, can it also handle X." We keep a written boundary for what the copilot will and will not attempt, reviewed the same way we review any other product scope — because a copilot that quietly grows past its evaluated domain is a copilot nobody has actually tested for its new job.
