Watch a number and tell me when it moves

Most monitoring fails in one of two directions. Either nobody is watching, or something is watching and reports constantly, which is the same as nobody watching but more expensive.

The useful version is silent almost always.

Pick the number you would act on

The test is not whether a number is interesting. It is whether crossing a line would change what you do this week.

Trial signups, failed payments, a queue depth, churn in a segment. If crossing the threshold would produce a shrug, do not monitor it. Look at it monthly instead.

Set the threshold where action begins

Not where the number becomes unusual. Where you would do something.

Those are different, and picking the first gives you an alert that arrives correctly and changes nothing, which trains everyone to ignore the channel.

Write the job

Every hour, check [the number].

Post to Slack only when it crosses [threshold], or when it has moved more
than [x] percent since the last hour.

Say the current value, what it was, when it last crossed, and a link to
where I can see it.

Do not post when it is normal. Do not post a daily summary. If it is still
across the line an hour later, do not post again — post once when it
crosses and once when it comes back.

"Post once when it crosses and once when it returns" is what makes this survivable. A condition that persists for six hours produces six alerts otherwise, and the sixth is read by nobody. The two-message pattern — it broke, it recovered — is the only shape that stays readable during the incident it exists for.

Watch the watcher for a fortnight

The first threshold is a guess. Two weeks tells you whether it fires on things you cared about, and the honest question is: did any alert change what I did?

If none did, raise the threshold or drop the monitor. An alert nobody acts on is worse than no alert, because it costs attention every time and teaches people to ignore the channel that will one day carry something real.

Say what to do, in the alert

Add a line to the message telling whoever reads it what the first step is. The person who set the monitor knows; the person on call at 2am on a Sunday does not.

What good looks like

The channel is quiet for weeks and then says something once, and the something is worth reading.

The measure is the ratio of alerts to actions. Anything below one in three, and the threshold is wrong.