Drift is the normal state
AI features rarely degrade because something failed. They degrade because every individual change was reasonable: a longer prompt to fix an edge case, more retrieved documents to improve coverage, a retry to smooth over a flaky call. Each is defensible and together they double your latency and cost.
The prompt accumulates instructions
Every production incident tends to add a sentence to the system prompt, and nothing ever removes one. Six months later the model is processing a page of accumulated corrections on every call, some of which now contradict each other.
Treat the prompt as code: review it, delete from it, and confirm each removal against your evaluation set rather than by intuition.
Retrieved context grows quietly
Coverage problems are usually met by retrieving more documents, which increases cost per call and often reduces answer quality by burying the relevant passage. This is the failure mode behind scoping the corpus by question rather than by system.
Most latency regressions in AI features are self-inflicted context growth, not model slowdown.
Retries hide the real failure rate
A retry that succeeds looks like a success in your metrics and costs you two calls and double the wait. Log retries separately from failures, or you will not notice that your underlying error rate tripled.
Instrument the three numbers
Track tokens per call, latency at the 95th percentile, and cost per successful outcome — not per call. Cost per outcome is the one that exposes retries, abandoned conversations, and work that had to be redone by a human.
Review them monthly. Drift is gradual enough that nobody notices it happening and obvious the moment somebody charts it.