Production Alerts: Notify on Critical Errors
A production error that goes unnoticed for hours can result in significant revenue loss and user churn. Production alerts notify you immediately when critical errors spike, crash rates jump, or performance degrades. Without alerts, you are waiting for users to complain on Twitter. With alerts, you are proactively monitoring your app 24/7.
Types of Alerts You Should Set Up
Configure alerts for these scenarios:
- New error types: A previously unseen error appears (high signal-to-noise ratio).
- Error spike: Error rate jumps above normal (5x the 7-day average).
- Crash-free regression: Release health drops below threshold (< 95%).
- Performance regression: Core Web Vitals degrade (LCP > 4s, CLS > 0.3).
- High-impact errors: Errors affecting more than N% of sessions.
- Service degradation: API timeout rate spikes (> 10%).
Setting Up Sentry Alert Rules
In Sentry, go to Alerts > Alert Rules > Create Alert Rule:
Alert Rule: New Error Type
Trigger: A new issue is created
Conditions: (leave empty or filter by environment)
Actions: Send a Slack notification to #oncall
This fires immediately when Sentry detects a new error pattern. New errors are high-priority because they often indicate a real problem, not just noise.
Alert Rule: Error Spike
Trigger: For all projects
When: The issue's error event count exceeds 100 in 5 minutes
Conditions: Environment is "production"
Actions: Send to #alerts and create a PagerDuty incident
This fires if an existing error suddenly increases. A spike of 100 errors in 5 minutes is abnormal and warrants investigation.
Alert Rule: Crash-Free Regression
Trigger: Release health
When: Crash-free rate of release drops below 95%
Conditions: Environment is "production"
Actions: Send to Slack #oncall and send SMS to on-call engineer
This catches regressions in new releases quickly. A 5% threshold allows for normal variation but catches real problems.
Alert Rule: Performance Degradation
Trigger: Metric alert
When: Average page load time (LCP) exceeds 4 seconds
Conditions: Environment is "production"
Duration: For more than 5 minutes
Actions: Send to #performance-oncall
Tracks Web Vitals and alerts if performance degrades. This might indicate a backend issue, a new bundle size regression, or a third-party service slowdown.
Routing Alerts to Notification Channels
Sentry supports multiple notification channels:
- Slack: Instant notifications in your team channel.
- PagerDuty: Creates incidents and escalates to on-call engineers.
- Email: Less reliable for critical alerts but good for summaries.
- SMS/Phone: For critical incidents affecting all users.
- Webhooks: Custom integration with your tooling.
Configure a Slack integration in Sentry:
- Go to Settings > Integrations > Slack.
- Click Install and authorize the app to your Slack workspace.
- Select which channel to post alerts to (e.g., #oncall, #alerts).
In each alert rule, select Send a Slack notification and choose the channel:
Alert: Error spike
Condition: Error count > 100 in 5 minutes
Notification: Slack #oncall message:
"High error rate spike in [Project]: [Error Type]
Errors: [Count]
View: [Link to Sentry]"
For critical alerts, integrate with PagerDuty to create incidents and notify the on-call team:
- Go to Settings > Integrations > PagerDuty.
- Add your PagerDuty service key.
- In alert rules, select Create a PagerDuty incident.
Now when a critical error spikes, Sentry automatically creates a PagerDuty incident, notifies the on-call engineer via Slack and phone, and escalates if not acknowledged within 5 minutes.
Alert Fatigue and Threshold Tuning
Too many alerts lead to alert fatigue: engineers ignore them because most are noise. Too few alerts and you miss real issues. Balance with these strategies:
Use smart grouping: Sentry groups similar errors automatically. An error that occurs 1,000 times in one hour is one group, not 1,000 alerts.
Set high thresholds initially: Start with conservative thresholds:
- New error: Always alert (high signal).
- Error spike: 10x the 7-day average (very abnormal).
- Crash-free: < 90% (severe regression).
Then tighten thresholds based on false positives.
Silence known issues: If you are debugging a known problem, silence the alert temporarily:
In Sentry alert rule:
Add condition: Except for the issue titled "Known bug: checkout error"
Aggregate similar errors: Group errors by root cause before alerting:
Sentry alert configuration:
Group by: exception.type (catch all TypeError separately from ReferenceError)
This prevents 100 similar errors from triggering 100 alerts.
Creating a Custom Alert Workflow
For most teams, a multi-level alert strategy works best:
Level 1 — High-priority alerts (instant notification):
- New errors in production
- Error spike (10x average)
- Crash-free < 90%
- P1 errors affecting > 5% of users
- Action: PagerDuty incident + SMS to on-call
Level 2 — Medium-priority alerts (1-hour batch):
- Error spike (5x average)
- Crash-free 90–95%
- Performance degradation (LCP > 4s)
- Action: Slack #alerts
Level 3 — Low-priority alerts (daily digest):
- New low-impact errors
- Performance warnings
- Unusual patterns
- Action: Email summary
Configure this in Sentry by creating multiple alert rules with different thresholds and channels.
Testing Alert Rules
After creating an alert, test it to ensure notifications work:
- In Sentry, open the alert rule.
- Click Test alert to send a test notification.
- Verify the message appears in your chosen channel (Slack, PagerDuty, etc.).
For Slack, the test message looks like:
[TEST] High error rate spike in my-app
Project: my-app
Errors: 150 (in 5 minutes)
View: https://sentry.io/my-app/issues/...
Example Alert Rule Setup for a Typical React App
Here is a complete alert configuration for a production React app:
| Condition | Threshold | Channel | Escalation |
|---|---|---|---|
| New error in production | 1 occurrence | Slack #alerts | None |
| Error spike | > 10x 7-day average | Slack #oncall | PagerDuty after 5 min |
| Crash-free regression | < 95% | Slack #oncall | SMS to on-call |
| Crash-free critical | < 90% | Slack + PagerDuty | Phone call to on-call |
| Performance degradation | LCP > 4s for 5 min | Slack #performance | None |
| User-impacting errors | > 5% of sessions | PagerDuty | Phone escalation |
This setup catches real problems without drowning your team in noise.
Integrating with On-Call Schedules
For teams with on-call rotations, integrate Sentry alerts with your on-call schedule:
- PagerDuty integration (recommended): PagerDuty rotates alerts to the current on-call engineer based on your schedule.
- Slack + GitHub: Use a bot to rotate alerts to the assigned owner.
- Email routing: Set up email rules to route critical errors to the on-call engineer's address.
Example PagerDuty escalation policy:
Level 1: Slack notification to #oncall (immediate)
Level 2: Page the on-call engineer (5 min if not acknowledged)
Level 3: Escalate to the on-call manager (15 min if unresolved)
Level 4: Create a war room bridge (30 min if unresolved)
This ensures critical issues get human attention quickly.
Key Takeaways
- Set up alerts for new errors, spikes, regressions, and performance issues to catch problems before users notice.
- Configure multiple alert thresholds (L1 critical, L2 medium, L3 low) to avoid alert fatigue.
- Route critical alerts to PagerDuty and on-call schedules for instant escalation.
- Test alert rules before deploying them.
- Use smart grouping and silencing to reduce false positives.
Frequently Asked Questions
What is the right threshold for an error spike alert?
Start with 10x the 7-day average (very abnormal). If you get false positives, increase to 20x. If you miss real issues, decrease to 5x. Monitor and adjust monthly based on your traffic patterns.
Should I alert on every new error?
Yes, initially. New errors are high-signal—they often indicate a real bug. Once your system stabilizes, you can filter out low-impact new errors (affecting < 0.1% of users). But do not ignore them entirely.
What is a reasonable response time for a production alert?
Critical alerts (affecting > 5% of users): respond within 15 minutes. Medium alerts: respond within 1 hour. Low alerts: respond the next business day. Adjust based on your SLA.
Can I snooze or silence an alert?
Yes. Most alerting systems let you silence an issue for a specific time:
Silence this issue for 8 hours
Silence this issue until the fix is deployed
This is useful for known issues you are already tracking.
How do I prevent alerts from being sent outside business hours?
Many alerting services support quiet hours:
Slack: Send to #alerts during business hours, #alerts-quiet after hours
PagerDuty: Quiet hours 6pm–9am (no paging, but create incident)
Adjust based on your team's timezone and on-call policy.