Skip to content

Blog

How Upcertain decides a site is down

"Is it down?" sounds like a yes-or-no question. In practice a monitor has to decide from a handful of requests whether to wake someone up. Here is exactly how Upcertain makes that call.

Updated 25 October 2026 · By the Upcertain team

1. A scheduled check runs

Every monitor has an interval, from 30 seconds to 24 hours. When it is due, a worker in Seoul runs the check. That is an HTTP request, a keyword match on the response, a ping, a TCP connection or a certificate inspection. For heartbeat monitors it is a look at when the job last reported in.

Each result is recorded with its response time. A failure also records a reason we can name: a status code outside what you expect, a timeout, a missing or unwanted keyword, a DNS failure or a refused connection. For certificates, the reason is that one is expired, issued for the wrong host or badly chained.

2. A failure is not an outage yet

The internet drops packets. Containers restart. Load balancers move traffic. A single failed request says very little, so the first failure starts a short investigation instead of an incident.

The monitor re-checks after its confirmation delay (15, 30, 60 or 120 seconds) as many times as you set, from 0 to 5. If any re-check passes, the episode ends quietly. The failures stay in the check log, but nobody is alerted and nothing appears on your status page.

New monitors use one re-check after 15 seconds. Services that fail predictably during deploys can use more; we walk through an example in downtime alerts during deploys.

3. Confirmed: the incident opens

When every re-check fails, Upcertain opens an incident. Its start time is the first failed check, not the moment it was confirmed, so the recorded downtime is honest about how long users were affected.

4. The right people hear once

  • Each monitor lists its alert channels. Each channel can wait a set delay before alerting (0 to 1440 minutes), so email to a wider group can wait while the on-call Slack channel hears at once.
  • If several monitors fail within a few minutes and alert grouping is on, they arrive as one alert listing them all. A shared database going down should be one message, not twenty.
  • Reminders repeat while the incident stays open, if you asked for them, every 5 to 1440 minutes.
  • Someone acknowledges the incident in Upcertain. Reminders stop on every channel and the incident records who took it.

Failed deliveries are retried at 30 seconds, 1, 2 and 4 minutes. Email goes to each recipient separately, with a personal link to stop alerts and an Acknowledge button only for people whose role can acknowledge.

5. Recovery

The first passing check resolves the incident. Every channel that received the alert gets the recovery, with the total downtime, in the same place. There is no confirmation on the way up: if the service fails again, a new episode starts.

6. What the status page shows

Status pages read the same incidents. Incidents that start within 2 minutes of each other are shown as one event ("Web app, API and 8 more were unavailable"), with each service's own downtime, because services on different check cycles notice the same failure a few seconds apart. Short incidents can be folded into one line per day. The reasoning is in brief outages on status pages.

What we do not do

We check from one region. A tool that checks from several places can tell a regional network problem from a real outage and report regional availability; ours cannot. We chose confirmation over time instead, which handles the most common cause of false alarms we see: short failures on the service itself.

Why this order

Every rule above trades a little speed for a lot of trust. Teams that get false alarms soon stop reading alerts. With the defaults, a real outage on a 30-second monitor reaches Slack within about 45 seconds; a monitor tuned to ignore 90-second deploy failures reports real outages within about two minutes. We would rather you hear about a real outage a minute later than learn to ignore the channel.