Health Checks

A health check is Muppy's unit of "look at this, decide whether it is fine, and say so if it is not". It is the shared mechanism behind almost everything Muppy watches: PostgreSQL clusters and their replication, App Servers, Kubernetes packages, Pack8s releases, and the cert-manager chain on your clusters.

Where to find them

Muppy › High Availability › Health Checks

The menu says High Availability for historical reasons — health checks were built there first. They are not specific to it: seven Muppy addons create and use them. The companion entry Fix Health-Checks lists the ones currently failing.

How one works

Every check follows the same steps. Only the first two differ between types:

1. Produce a raw result. For most types this is an HTTP request — an endpoint, a port, a path. Some types read something Muppy already holds, in process, and make no network call at all.
2. Parse it with the check's Result Parser — a Smart Config. json_parser is the usual one. A type that hands its processor the records themselves skips this step, and the Result Parser field is hidden on its form.
3. Judge it with the check's Result Processor — another Smart Config. This is where the thresholds live: it turns what it received into an overall state, a set of metrics, and a list of alerts.
4. Record it. A Health Check Result row is written whatever the outcome — success, timeout, connection error. The check's own state is updated from it.

The types

Type	What it reads
Basic State check	an endpoint that answers with a state
Prometheus Exporter	an exporter's metrics endpoint
Remote Dump	a remote dump of something Muppy then parses
Muppy Internal	Muppy's own records — any model, chosen by a saved filter. In process: no HTTP request, no network call

The last one is the pattern for anything Muppy is already the source of: rather than exposing an endpoint and scraping itself, the check reads its own rows. One type covers every model — point it at a saved filter and write a processor. The cert-manager monitoring on your clusters is one such check; see K8s Monitoring.

Watching Muppy's own records

A Muppy Internal check needs one thing: a Target Filter, naming the records to read.

1. Open the list of the model to watch — K8s Objects, Hosts, Databases.
2. Filter, group and sort until the list shows exactly what the check should read.
3. Open Save current search, tick Shared, and save it.
4. On the check, set Target Filter to that saved search.

The filter must be shared

A filter that is not shared belongs to one account and is deleted with it. A check pointed at one would go silently empty the day that account is removed — and an empty read looks exactly like a healthy park. Muppy refuses a private filter rather than let that happen.

The processor receives the selected records and reads whatever fields it needs. Widening the filter widens what the check reads, with no code to change.

Cadence — the one thing to get right

A check has an Execute Every setting, and Muppy ships one scheduled action per interval:

Scheduled action	Serves checks set to
Muppy: Run Health Checks - every Minute	1 minute
Muppy: Run Health Checks - every Hour	1 hour
Muppy: Run Health Checks - every Day	1 day
Muppy: Run Health Checks - every 2 Hours	2 hours

A check is run by the scheduled action whose interval equals its own — and by no other

The match is exact. If you set a check to 3 hours and no scheduled action runs every 3 hours, that check is dispatched by nobody and never runs. Nothing warns you: the check sits there looking armed, with no results and no error.

To change a cadence, set the same interval in both places — the check's Execute Every, and a matching scheduled action (create it if none exists).

A scheduled action does not run the checks itself: it queues them, and they execute in the background. Follow a run in Muppy › Qs, or simply watch the check's Results.

Notifications

A check has a Notify alerts flag. When it is on, the alerts its processor produced are pushed through Pushover, which needs your application and user key in pushover.apptoken_userkey under Settings › Technical › System Parameters.

The flag is on; the credentials may not be

The checks Muppy ships have Notify alerts ticked — a check that finds something and stays quiet is the failure this whole layer exists to end. But the push itself needs pushover.apptoken_userkey to be set, and nothing warns you when it is not: the alerts are still computed and still recorded, they simply never leave the building.

If you are relying on a check to wake you up, verify the system parameter first, then send yourself one with Test Health Check.

What goes out is bounded on purpose. A processor puts a short classified code in the alert and keeps free-text diagnostics — which can carry tokens, addresses or provider query ids — inside Muppy. Pushover is a third party.

Reading a result

Open a check and use its Results button.

On the result	What it holds
Result Status	ok, or why not — timeout, connection_refused, read_timeout, error
Metrics	what the processor measured, as name/value pairs
Alerts	what the processor decided a human should know
Result	the raw text the check produced, before parsing
Producer Duration (s)	how long producing the raw result took
Processing Duration (s)	how long parsing, judging and notifying took. For a check whose processor is handed records, this is where nearly all the time goes — Producer Duration only covers the read

Results are purged automatically after 96 hours. They are a trail for diagnosis, not a history to reason over — that distinction is the point.

Test Health Check, in the header, runs the whole chain once, now, and writes a result. Use it after changing a parser or a processor.

Extending

Most of the time you do not need a new type. Muppy Internal already reads any model: a saved filter and a processor cover it, with no Python.

An addon that genuinely needs one declares it on health_check_type and implements the step that produces the raw result. A type that must hand its processor something other than parsed text — a recordset, thresholds of its own — also overrides _processor_extra_kwargs(). Everything else — state, results, alerts, the GUI — is already generic and does not change.

The contracts live in two docstrings, which are the reference: mpy.health_check._produce_raw_result (what to return, what never to raise, and why the guard on how long it may take is a measurement rather than a limit) and mpy.health_check._processor_extra_kwargs (why those argument names are a published interface, and why they must be rebuilt rather than stashed).