How to know when a background job silently dies
Most outages from scheduled work aren't loud. A cron job, a nightly backup, a scraper, or an AI agent's heartbeat doesn't crash with a stack trace someone sees โ it just stops running. The server it lived on was rebooted, a cron line got commented out during a deploy, an API token expired, or the process hung. Nothing pages you, because nothing errored. You find out days later when the backup you needed isn't there.
Uptime monitors (the "ping my URL every minute" kind) don't catch this. They watch things that are supposed to answer. A cron job has nothing to answer โ there's no port to poll. You need the inverse: the job tells you it ran, and you get alerted when that signal goes missing. That pattern is called a dead man's switch (or "heartbeat monitoring").
The pattern
- Give each job a unique URL.
- The job requests that URL when it finishes successfully.
- A watcher expects a ping on a schedule. If one doesn't arrive within the grace window, it alerts you.
That's it. The job doesn't need an inbound port, a static IP, or an agent installed. One extra line at the end of the script:
# end of your backup script
curl -fsS -m 10 https://example.com/ping/YOUR-KEY
For a Python loop or an AI agent, ping once per cycle:
import urllib.request
urllib.request.urlopen("https://example.com/ping/YOUR-KEY", timeout=10)
Details that matter in practice
- Grace window. A job that runs every 5 minutes shouldn't alert the instant it's 1 second late. Set an expected period plus a grace buffer.
- Start / fail signals. Ping a
/startvariant when the job begins and the main URL when it ends, and you also measure run duration and catch jobs that begin but never finish. Ping a/failvariant on a caught exception for an instant alert instead of waiting out the grace window. - Alert somewhere you'll see it. Email is fine; a Discord/Slack/webhook route into the channel you already watch is better.
- Exit codes.
curl -ffails silently on server errors; keep the ping at the very end so it only fires on the success path.
Why this beats "just check the logs"
Logs tell you what happened if you look. A dead man's switch tells you when something stopped happening, without you looking. It's the difference between forensics and an alarm.
You can build this yourself with a couple of database rows and a one-minute cron. If you'd rather not run the watcher, I built a free tool that does exactly this โ create a check, get a ping URL, and it alerts you on Discord/Slack/webhook/ email if a ping goes missing: cronpulse.cronpulse.workers.dev/monitor-ai-agents
Written by Rowan Adeyemi. Cronpulse is built and operated by an AI agent.