# When you get alerted

> How up4 confirms an outage before it alerts you: observations, regions, quiet mode, recovery, slowdowns, scheduled jobs and escalation.

Source: https://upfour.io/docs/how/when-you-get-alerted

up4 waits for several matching results before it calls an outage, so one hiccup does not wake you. This page lists every rule and what it adds to the wait.

> The timeline needs JavaScript. The rules are written out below.
>
> Time since the outage started. A range means it depends on where in the interval the last check fell.

## Confirming an outage

A state change, up to down or down to up, needs 3 consistent observations. The first check that differs from the current state is the first observation. It starts 2 re-checks at once, so confirmation does not wait for the next interval. With several regions, each region runs its own re-checks.

The count has a shelf life. If the observations do not line up, the streak expires after 3 times the check interval, but not sooner than 30 seconds, and counting starts over.

**Extra confirmation time** adds a minimum wait on top, and only for the failure direction. The outage counts as confirmed once the latest failing check lies at least that long after the first failing one and the observation count is reached. Because the regular checks are an interval apart, the alert comes with the first regular check after that time. The setting No extra wait still means 3 observations, never one.

## Several regions

A monitor that checks from several regions changes state only when a strict majority of its regions agree. up4 counts the regions that have a healthy checker at that moment, each by its latest result, and ignores a result older than 2 minutes. A region without a fresh result counts as not agreeing.

With three regions two must agree. With two healthy regions both must agree, so one region reporting alone never starts an incident. The same majority applies on the way back up. More on regions is in [Checks and regions](https://upfour.io/docs/how/checks-and-regions.md).

## Right away or skip short blips

The monitor form asks **When should we alert you?**. The answer decides only when the first notice goes out.

| Choice | First notice |
| --- | --- |
| **Right away** | As soon as the outage is confirmed. One alert per outage, and an all-clear when it is over. |
| **Skip short blips** | Once confirmed downtime adds up to 3 minutes. A shorter outage stays in the history and sends no notice. |

Downtime adds up from the gaps between successive failing checks. A gap counts only up to the larger of 1.5 times the interval and 30 seconds. A longer gap adds nothing, so missed checks do not count as downtime. Short blips that recover within 15 minutes share one window and add up; the recovery period extends that window when it is longer.

Either way the monitor status, the incident and the status page show the outage as soon as it is confirmed.

## Coming back up

Recovery is confirmed like an outage: the first good check plus its re-checks. At that moment the monitor is up again, and an all-clear goes to the channels that heard about the outage, but only if an alert went out. The incident stays open while **Recovery period** runs and closes with the first good check after that time. With Resolve immediately there is no wait, so the next check closes it.

## Monitors that keep flipping

A monitor that goes down again within 15 minutes of its last recovery counts as unstable. up4 still opens the incident, so the dashboard and the status page show it, but sends one unstable notice instead of a down and up pair for every flip.

Once the monitor has stayed up for 15 minutes you get a stable notice and alerts resume. If it stays down instead, a regular down notice follows after 2 times the interval, but not sooner than 5 minutes. up4 sets both windows; the values here are the defaults.

## Slow, not down

A slowdown warning is a separate message and off until you turn on **Warn when the target gets slow but stays up**. It is confirmed after 3 slow checks and cleared after 3 fast ones. It never opens an incident and never goes to phone call, SMS, PagerDuty, Jira or Grafana channels.

## What shows and what notifies

| What | When it changes |
| --- | --- |
| Monitor status, incident, status page | When the outage is confirmed, in both notification modes. |
| Notice to direct recipients | With Right away, when the outage is confirmed. With Skip short blips, after 3 minutes of downtime. |
| Alert plan steps | See escalation timing below. |
| All-clear | After recovery is confirmed and the recovery period has passed. |
| Paused monitors, maintenance windows | Nothing alerts. Checks inside a maintenance window keep running. |

## Scheduled jobs

A scheduled job is late when its last check-in plus the expected interval plus the grace period has passed. The alerter looks for missed deadlines every 30 seconds, so a missed check-in is noticed within that sweep after the deadline. The defaults are 5 minutes for the interval and 1 minute for the grace period; the interval cannot be shorter than 30 seconds.

No check runs for a scheduled job, so there is no three-result confirmation; the deadline and the grace period do that work. A missed check-in goes to the channels that fit the job's Company, client or team, never to an alert plan and never as a call. A job that keeps changing state gets its alerts bundled once an hour until it settles. [Watch a scheduled job](https://upfour.io/docs/watch-a-scheduled-job.md) shows the setup.

## Escalation timing

Direct recipients get one notice at the moment shown above. An [alert plan](https://upfour.io/docs/build-an-alert-plan.md) adds to it. Its first step fires at once or after its own wait. Every later step fires after its wait, at least 1 minute, and only while nobody has acknowledged the incident. After the last step the plan can start over up to 8 times. Acknowledging or resolving the incident stops it.

## When checking slows down

When the balance of a company is used up after a first payment, up4 moves its checks to one every 5 minutes for 7 days before monitoring pauses. Checks that far apart notice an outage later, up to one interval after it starts. A paused monitor runs no checks and sends no alerts; nothing is deleted. More in [Credit and plans](https://upfour.io/docs/credit-and-plans.md).

## What adds to the wait

| Rule | What it adds |
| --- | --- |
| **Check every** | Up to one interval until a check meets the outage. |
| Request timeout | A failing check can take as long as the timeout, by default 30 seconds. |
| Re-checks | Nothing noticeable. They start right away. |
| **Extra confirmation time** | At least the time you set, failure direction only. |
| **Skip short blips** | 3 minutes of confirmed downtime. |
| Alert plan | The wait of each step. |
| Channel and provider | Not in up4's hands: mail servers, chat services and phone networks. |

## What up4 does not do here

- up4 never promises a delivery time. The table above shows what depends on you and what does not.
- The timeline shows ranges because a check can land anywhere within its interval. It does not model flapping or the channel.
- An outage shorter than the interval can pass between two checks and never be seen.

[Per API POST /v1/monitors](https://upfour.io/docs/api/monitors#post_monitors)

| In the app | API field |
| --- | --- |
| When should we alert you? · Right away alerts once checks confirm an outage. The quiet option skips short blips. | `notification_mode` |
| Extra confirmation time · seconds; 0 means no extra wait | `confirmation_period` |
| Recovery period · How long the monitor must stay healthy before the outage counts as over. | `recovery_period` |
| Alert plan · An alert plan pages people step by step until someone acknowledges, on top of the channels. | `escalation_policy_id` |

[Full reference](https://upfour.io/docs/api/monitors#post_monitors)