Heartbeat Monitors

A Heartbeat Monitor watches a job that runs on its own: a nightly backup, a cron job, a queue worker. We do not call the job. The job calls us, at a URL of its own, each time it has run. When a call does not come in time, the Monitor is down and we tell you.

A Heartbeat Monitor has one Check, uptime. No location probes it, so it has no locations to choose.

The URL your job calls

Once you create a Heartbeat Monitor, its Checks card shows two things to copy. The Heartbeat URL is the address your job calls. It ends in a secret of its own:

https://api.uup.dev/hb/your-token

The Line to add to your job is the call, ready to paste at the end of your job:

curl -fsS -m 10 --retry 3 https://api.uup.dev/hb/your-token

Add it so it runs only when the job succeeded. Heartbeat examples shows how for crontab, systemd timers, GitHub Actions, Kubernetes CronJobs, a shell script, Node, Python, Go, Ruby and PHP.

The URL takes GET, POST and HEAD, and answers 200 with the body ok. It sets no cookie and never redirects, so a bare curl succeeds with no options. We never read the body or the query of a call, so nothing your job sends is kept. We record what we saw: the time, the method, where the call came from and its User-Agent. The Checks card shows the latest call as Last signal, and when the next one is due as Next due.

The URL answers otherwise only when the call cannot count:

• 404 with the body unknown token: the URL is not one we know. It was rotated, its Monitor was deleted, or part of it is missing.
• 405: the call used another method, such as PUT. Use GET, POST or HEAD.

Calls closer together than 5 seconds count as one.

The URL is a secret: anyone who has it can call it, and keep the Monitor up for a job that stopped. Keep it out of public repositories and shared logs, and rotate it if it gets out.

How often your job calls

Runs every says how often your job runs and calls us. Choose 1 minute, 5 minutes, 1 hour or 1 day, Another interval in minutes, or Custom cron. An interval can be up to 31 days.

On an interval, each call is a promise of the next one. The next call is due one interval after the last, and late once the Grace after that runs out.

Custom cron

For a job that does not run on a fixed period, such as weekdays at 09:00 or the first of each month, choose Custom cron. Then enter the job’s schedule in Cron schedule, as five fields, the way a crontab writes it, such as 0 9 * * MON-FRI. The form shows the next runs, so you can check them.

• The schedule is read in the Project’s timezone, the way a crontab is read in its server’s local time. If they differ, write the schedule for the Project’s timezone.
• Its runs must be at least 1 minute apart and at most 31 days apart.
• After a call, the next one is due at the next scheduled run, and late once the Grace after that run runs out. Your job calls when it finishes, not when it starts, so make the Grace longer than the job takes.

Grace

Grace is how late a call may be before we tell you. A job never runs to the second: it starts a little late, or takes longer on a busy day.

Left empty, the Grace is a tenth of the interval, or of the gap between a schedule’s next two runs, and never less than 1 minute: an hourly job’s is 6 minutes. The form shows the Grace it will take. You can choose any Grace from 1 minute up to the interval, or up to the shortest gap between a schedule’s runs.

When a call is late

When the Grace runs out with no call, the Monitor is Down and an Incident opens, from the moment the call was due. The alert names the Monitor and says it has not checked in.

The next call closes the Incident. One call is enough, because only your job can make it.

Waiting for the first call

A new Heartbeat Monitor watches nothing until its job first calls. Until then the Checks card says Waiting for the first call, the Monitor list says Waiting for first signal, and Next due says After the first call. Nothing can be late, and the time before the first call does not count toward its uptime. So you can create the Monitor before the job is running.

Rotating the URL

To replace the URL, press Rotate URL on the Checks card. The dashboard asks first, because the current URL stops working at once. Press Rotate, and the card shows the new URL.

From that moment the old URL answers 404, so your job’s calls to it no longer count. Put the new URL in your job before its next call is due, or the Monitor goes down when the Grace runs out. Owners and admins can rotate a URL.

Paused, and outside your Plan

While a Monitor is paused, its URL still answers 200 and records each call, but nothing is due and nothing alerts. When you resume it, we watch it as if its job had just called. One that never called is still waiting for the first call.

A Monitor your Plan no longer covers keeps its URL. The URL still answers 200 and records nothing, so a change of Plan never fails your job’s call.

When something is wrong on our side

When our side cannot take calls, your job’s call fails: it gets a 5xx answer, or no answer in time. The line on the Checks card gives up after 10 seconds and tries again up to 3 times, so a short break on our side does not fail it.

We never count a call against your job when we could not take it. We keep a record of each time our side could not take calls. When a job’s call was due inside such a time, we open no Incident: we wait one more interval and Grace from the moment we were back. So a job that kept running never alerts for our outage, and a job that really stopped during it alerts later than usual.

Your job’s own log may still show the failed call. That failure is ours, not your job’s.