AI incidents need the same discipline as other security incidents, plus some AI-specific steps.
Types of AI Incidents
- Data leaked through model outputs.
- An agent taking unauthorised actions after prompt injection.
- Poisoned data or a compromised model discovered.
- Model weights or prompts stolen.
- Harmful content generated publicly.
- Runaway costs from abuse.
Prepare
- Include AI systems in incident response plans.
- Maintain an inventory of models, data sources, tools and owners.
- Build the ability to quickly disable tools, switch models, roll back prompts and take features offline.
- Keep logs sufficient for investigation.
Respond
- Contain: disable affected features or tools; revoke credentials.
- Investigate: review logs of prompts, retrievals and actions.
- Eradicate: remove poisoned data, fix vulnerabilities, update controls.
- Recover: restore service with fixes verified.
- Notify: meet legal obligations for data breaches.
Learn
Hold a blameless review. Add the attack to test suites and update the threat model.