Incident Management

PingZen automatically creates incidents when monitors go DOWN and resolves them when they recover. You can update statuses, add comments, and track the full timeline.

Zoom

Features

Incident Statuses

The incidents page

The page has three tabs: Active, Resolved and All (Active, then Resolved). The tabs show their counts, and the search does not change them.

Active

Every open incident, grouped so that one cause reads as one block. The groups are checked in this order, and an incident lands in the first one that fits:

• Location … looks down — PingZen noticed on its own that checks from this location fail for monitors across all users, and gathered here the failures from that location only (a location that is offline altogether shows a “Location … is offline” line at the top).
• host · went down together — two or more monitors of the same host went down within 15 minutes.
• Unreachable from region · … — partial outages: the monitors fail from these locations and still answer from the others. This group is folded by default.
• date, time–time · went down together — three or more monitors went down within 15 minutes of each other.
• Failures — everything else, newest first. This group never folds.

Each row shows:

• the monitor status, in the same words as the Monitors list;
• the reason and the location (“from Novosibirsk”, or “only from Moscow” for a partial outage). The reason is what the monitor fails with now, read from its latest failed check rather than the one that opened the incident: a long outage can change its error, say from 401 to 404. The full error is in the tooltip. How the incident started is in its details;
• how long the incident has lasted.

A monitor that keeps going down and coming back gets “flapping · N times in M h”. A monitor with several separate outages over the last three days gets “N outages in M h” instead. An outage younger than an hour gets a “new” pill. When a monitor has been failing from all of its locations for a week or more and now gets an authorization error (401, 403), a DNS error or “not found” (404, 410), the row reads “Looks like a stale monitor setup” and links to Edit check. Manually created incidents never get this hint.

The browser remembers which groups you folded, separately for each workspace. The search opens the groups that contain matches.

Resolved

Resolved incidents of one monitor that are less than 15 minutes apart make up one episode. Each row shows:

• what happened: “flapped · N times”, “2 outages” or “1 outage”;
• the share of the episode the monitor was down;
• when the episode started and ended.

The period switch (24 hours, 7 days, 30 days) applies here. You can sort by repeats or by time. The list comes in pages of 50 episodes.

• ongoing means an open incident continues the episode. Click it to open that incident.
• started before the period means the episode began before the start of the chosen period, so its earlier incidents are not counted.

Incident details

Click a row, or tap it on a phone, to open the incident timeline. The dialog has Open monitor, Edit check and Acknowledge.

Owners and editors can acknowledge incidents:

• one incident, from the row menu “⋯” or the dialog;
• every row of a group at once, with Acknowledge all in the group menu. Regional outages have no such item.

Incidents API

Manage incidents programmatically via REST API.