A model trained on past data assumes the future will look similar. When that stops being true, performance degrades — often silently.
Data Drift
The distribution of inputs changes. New customer segments, a new product line, a changed form, a different camera or seasonal shifts all alter what the model sees.
Concept Drift
The relationship between inputs and the outcome changes. Fraudsters adopt new tactics; customer preferences shift; a policy change alters behaviour. The same inputs now mean something different.
Detecting Drift
- Monitor input distributions: compare recent data with training data using statistics such as population stability index or distribution tests.
- Monitor predictions: sudden shifts in prediction rates or confidence.
- Monitor outcomes: when true labels arrive, track accuracy over time — the most direct signal, though often delayed.
- Watch data quality: many "drifts" are actually broken pipelines.
Responding
- Investigate the cause before reacting.
- Retrain on recent data, on a schedule or when triggered.
- Adjust features or thresholds.
- Fall back to rules or human review during major disruptions.
Design for It
Choose features that are stable over time, keep training data pipelines ready to rerun, and version models so you can roll back.
Set Expectations
Every model has a shelf life. Budget for monitoring and retraining from the start.