Health Checks

A health check is Muppy's unit of "look at this, decide whether it is fine, and say so if it is not". It is the shared mechanism behind almost everything Muppy watches: PostgreSQL clusters and their replication, App Servers, Kubernetes packages, Pack8s releases, and the cert-manager chain on your clusters.

Where to find them

Muppy › High Availability › Health Checks

The menu says High Availability for historical reasons — health checks were built there first. They are not specific to it: seven Muppy addons create and use them. The companion entry Fix Health-Checks lists the ones currently failing.

How one works

Every check follows the same four steps, and only the first differs between types:

1. Produce a raw result. For most types this is an HTTP request — an endpoint, a port, a path. Some types read something Muppy already holds, in process, and make no network call at all.
2. Parse it with the check's Result Parser — a Smart Config. json_parser is the usual one.
3. Judge it with the check's Result Processor — another Smart Config. This is where the thresholds live: it turns the parsed data into an overall state, a set of metrics, and a list of alerts.
4. Record it. A Health Check Result row is written whatever the outcome — success, timeout, connection error. The check's own state is updated from it.

The types

Type	What it reads
Basic State check	an endpoint that answers with a state
Prometheus Exporter	an exporter's metrics endpoint
Remote Dump	a remote dump of something Muppy then parses
Muppy K8s Internal	Muppy's own Kubernetes objects, in process — no HTTP request, no cluster call. See K8s Monitoring

The last one is the pattern for anything Muppy is already the source of: rather than exposing an endpoint and scraping itself, the check reads the records directly. Any addon can add a type this way.

Cadence — the one thing to get right

A check has an Execute Every setting, and Muppy ships one scheduled action per interval:

Scheduled action	Serves checks set to
Muppy: Run Health Checks - every Minute	1 minute
Muppy: Run Health Checks - every Hour	1 hour
Muppy: Run Health Checks - every Day	1 day
Muppy: Run Health Checks - every 2 Hours	2 hours

A check is run by the scheduled action whose interval equals its own — and by no other

The match is exact. If you set a check to 3 hours and no scheduled action runs every 3 hours, that check is dispatched by nobody and never runs. Nothing warns you: the check sits there looking armed, with no results and no error.

To change a cadence, set the same interval in both places — the check's Execute Every, and a matching scheduled action (create it if none exists).

Checks are dispatched through the message queue, so an IMQ worker must be running. Without one, the scheduled actions enqueue work that nothing consumes.

Notifications

A check has a Notify alerts flag. When it is on, the alerts its processor produced are pushed through Pushover, which needs your application and user key in pushover.apptoken_userkey under Settings › Technical › System Parameters.

The flag is on; the credentials may not be

The checks Muppy ships have Notify alerts ticked — a check that finds something and stays quiet is the failure this whole layer exists to end. But the push itself needs pushover.apptoken_userkey to be set, and nothing warns you when it is not: the alerts are still computed and still recorded, they simply never leave the building.

If you are relying on a check to wake you up, verify the system parameter first, then send yourself one with Test Health Check.

What goes out is bounded on purpose. A processor puts a short classified code in the alert and keeps free-text diagnostics — which can carry tokens, addresses or provider query ids — inside Muppy. Pushover is a third party.

Reading a result

Open a check and use its Results button.

On the result	What it holds
Result Status	ok, or why not — timeout, connection_refused, read_timeout, error
Metrics	what the processor measured, as name/value pairs
Alerts	what the processor decided a human should know
Result	the raw text the check produced, before parsing
Producer Duration (s)	how long producing the raw result took

Results are purged automatically after 96 hours. They are a trail for diagnosis, not a history to reason over — that distinction is the point.

Test Health Check, in the header, runs the whole chain once, now, and writes a result. Use it after changing a parser or a processor.

Extending

An addon adds a type by declaring it on health_check_type and implementing the step that produces the raw result. Everything downstream — parsing, state, results, alerts, the GUI — is already generic and does not change.

The contract for that step lives in mpy.health_check._produce_raw_result's docstring, which is the reference: what to return, what never to raise, and why the guard on how long it may take is a measurement rather than a limit.