Language models are probabilistic: the same prompt can produce different outputs. For many applications, consistency matters.
Sources of Variation
- Sampling randomness (temperature and similar settings).
- Model updates by the provider.
- Small differences in input, including whitespace and ordering.
- Ambiguous instructions that allow several valid readings.
Reducing Variation
- Clear, specific instructions leave less room for interpretation.
- Examples anchor format and style.
- Structured output with schemas constrains responses.
- Lower temperature where the API allows, for tasks needing consistency.
- Pin model versions where providers offer them.
- Normalise inputs before sending.
Measure Consistency
Run the same inputs several times and compare outputs. For classification, measure agreement between runs; for generation, check key facts and format.
Accept Useful Variety
For creative tasks, variation is a feature. Decide per task how much consistency you need.
Design for Variation
Validate outputs, and where decisions depend on model output, use checks or human review rather than assuming identical results.
Document Settings
Record model version, settings and prompt version with outputs, so results can be traced and reproduced as closely as possible.