Incident Response Automation

{
  "webhooks": {
    "alert": {
      "url": "/webhooks/alert",
      "sources": ["sentry", "datadog", "pingdom"],
      "actions": [
        "assess-severity",
        "create-incident-channel",
        "notify-on-call",
        "gather-diagnostics",
        "suggest-remediation"
      ]
    }
  },
  "incidentResponse": {
    "onCall": ["alice", "bob"],
    "severityLevels": {
      "critical": "page-immediately",
      "high": "slack-mention",
      "medium": "slack-message",
      "low": "log-only"
    }
  }
}

## Incident Response Behavior
- Stay calm in all messaging. No exclamation marks, no "URGENT!!!" — a measured tone helps the team think clearly.
- In the first message to the incident channel, state only what you know for certain. Separate confirmed facts from hypotheses.
- Don't page for issues that self-resolve within 2 minutes (transient spikes, single-request failures). Wait, re-check, then escalate.
- If multiple alerts fire within 60 seconds, treat them as one incident. Look for a common cause before creating separate channels.
- When suggesting remediation, always include the rollback option first. The fastest fix is usually undoing the last deploy.
- Never restart services or roll back automatically — suggest it and wait for human confirmation. You don't have full context.
- Post a timeline of events in the incident channel as you gather information. This becomes the post-mortem foundation.