Coding
AI Prompt for Production Incident Triage
Triage production incidents with this AI prompt: turn alerts, logs and metrics into ranked hypotheses, 5-minute checks, and a safe mitigation plan.
The prompt
Role: You are an incident commander and SRE helping me triage a production issue calmly and systematically. Context: Service: [SERVICE_NAME], stack: [STACK]. Alert/monitoring signal: [ALERT_OR_METRIC, e.g. p99 latency spiked 5x at 14:02 UTC, error rate 2%→18%]. Recent changes: [RECENT_DEPLOYS_OR_CONFIG_CHANGES]. Relevant logs (trimmed): [LOG_EXCERPT]. Current user impact: [USER_IMPACT]. Task: 1) Generate 3-5 hypotheses ranked by likelihood, each with the evidence that supports or weakens it. 2) For each hypothesis, give me ONE quick check (log query, metric, or command) that would confirm or rule it out in under 5 minutes. 3) Recommend the safest immediate mitigation to reduce user impact (rollback, feature flag, scale-up, traffic shift) with rollback risks noted. 4) List what data to preserve now for the postmortem. Format: Hypotheses as a ranked table (hypothesis | evidence for | evidence against | 5-min check), then mitigation recommendation, then evidence-preservation checklist. Constraints: Prioritize reducing user impact over finding the perfect root cause. Flag any suggested action that is destructive or hard to reverse. Keep it actionable — I am under pressure.
Variables to fill in
Replace each [variable] in the prompt with your own details:
[SERVICE_NAME] [STACK] [ALERT_OR_METRIC] [RECENT_DEPLOYS_OR_CONFIG_CHANGES] [LOG_EXCERPT] [USER_IMPACT]
Example input
SERVICE=checkout-api, STACK=Node.js/Postgres on Kubernetes, alert=error rate 2%→18% at 14:02 UTC, recent=deploy v2.14.1 at 13:55, impact=users cannot complete payment
How to use this prompt
Paste this during the incident, not after — it is designed for triage under pressure. Always include recent deploys; most incidents correlate with a recent change.
Tips for the best results
- Work the 5-minute checks in order; do not jump to mitigation before ruling out the top hypothesis.
- Ask the AI to draft the postmortem timeline from your chat once the incident is resolved.
- Save the evidence checklist — missing logs are the #1 postmortem regret.
Frequently asked questions
Should I follow the AI's mitigation blindly?
No. Treat it as a structured second opinion; the on-call engineer makes the final call, especially for destructive actions.
What if I have almost no logs?
Say so — the prompt will prioritize hypotheses checkable via metrics and suggest what instrumentation to add afterward.