Odoo Filestore rsync and monitoring¶

Three ways to hold a filestore¶

Odizy manages an Odoo filestore in three different ways. Which one is right depends on the size of the filestore and the shape of the deployment:

Mode	How	When
In the database	`inouk_attachments_storage`, storage location PostgreSQL Database	Small filestores, up to roughly 50 GB. Backup and replication come for free — the filestore is the database.
In an S3 bucket	`inouk_attachments_storage` in S3 mode (`IK_IR_ATTACHMENT_S3_*`)	Odoo 18 and later. Scaling and Kubernetes deployments, where no single node owns the files.
On disk, replicated by rsync	storage location File System, replicated primary → standby by the script on this page	Legacy servers with huge filestores — too large for the database, not portable to S3.

The first two modes make the filestore follow the database on their own. The third does not, and that is the gap this page fills: streaming replication replicates the database and nothing else, so a standby with a perfectly current database and a stale filestore serves broken attachments — silently, and only for the records whose attachments are missing.

The script closes that gap with a periodic rsync. The part that makes it operable is that it publishes its own outcome as Prometheus metrics, so "the sync stopped" becomes an alert instead of a discovery.

Configuring the other two modes

In-database and S3 storage are configured in the inouk_attachments_storage addon (Settings, then Attachment Storage Location), not here. This page covers the rsync mode only.

The script runs only on a replica by default — it checks pg_is_in_recovery() before doing anything — so the same crontab line is safe to deploy on every node of the cluster.

What it produces¶

Metrics, all labelled with the sync name (sync="odoo_filestore"):

Metric	Meaning
`mpy_rsync_replica_check`	Is this host a replica: `1` yes, `0` no, `-1` not checked
`mpy_rsync_last_success_timestamp`	Unix timestamp of the last successful sync
`mpy_rsync_last_failure_timestamp`	Unix timestamp of the last failed sync
`mpy_rsync_duration_seconds`	Duration of the last successful rsync
`mpy_rsync_success_total`	Successful syncs (counter)
`mpy_rsync_failure_total`	Failed syncs (counter)
`mpy_rsync_skipped_total`	Syncs skipped because one was already running (counter)

Files, one set per sync name:

Path	Role
`/opt/muppy/node_exporter/textfile/mpy_rsync_{name}.prom`	Metrics, scraped by node_exporter
`/tmp/mpy_rsync_{name}.lock`	`flock` guard against overlapping runs
`/tmp/mpy_rsync_{name}.state`	Persisted counters and timestamps
`/var/log/mpy_rsync/rsync_{name}_*.log`	Full rsync output, 5 most recent kept

Progress also goes to syslog: journalctl -f -t mpy_rsync.

Prerequisites on the standby¶

Directories. The script creates its own log directory, but that fails when it does not run as root, so create all three up front and give them to the user that will run the cron:

sudo mkdir -p /opt/muppy/tools /var/log/mpy_rsync /opt/muppy/node_exporter/textfile
sudo chown -R muppy:muppy /var/log/mpy_rsync /opt/muppy/node_exporter
sudo chmod -R 755 /opt/muppy/node_exporter

SSH from the standby to the primary for the source user (odoo). See Managing authorized_keys.
node_exporter with the textfile collector. Open the standby's exporter under Muppy › Metrics › Prometheus › Exporters, add this to systemd_execstart_params, then press Reconfigure — which regenerates the systemd unit and restarts the exporter:
```
--collector.textfile.directory=/opt/muppy/node_exporter/textfile
```
Decide how the replica check reaches PostgreSQL. The script sources /etc/muppy.env, whose PG* variables point at the database the application uses. On an Odoo standby that is the primary — so the check would ask the primary whether it is a replica, get f, and skip forever. Two valid answers: pass --pg-cluster (see below) so the check asks the local cluster, which needs passwordless sudo -u postgres for the cron user; or pass --dont-check-replica and do without the guard.

You cannot fix this from the crontab line

The script exports every variable it reads from /etc/muppy.env, so those values override anything the caller set. Setting PGHOST before the command has no effect. --pg-cluster is the way.

Without step 3, nothing is monitored

The script writes its .prom file whether or not the collector is enabled. Skip step 3 and everything looks fine on the standby while no metric ever reaches Prometheus.

Step 1 — Upload the script¶

The script ships as a Muppy template. Go to Muppy › Configuration › Templates, open Muppy filestore rsync With Metrics v1, and click Upload to Host:

Field	Value
Host	the standby
Target Directory	`/opt/muppy/tools`
Filename	`mpy_rsync_with_metrics.sh` (already pre-filled)
Owner (chown)	the user that will run the cron, e.g. `muppy:muppy`
Permissions (chmod)	`755`

See Muppy Templates for the wizard itself.

This template has no Jinja2 placeholders

The script is parameterized at run time, by command-line flags — not at render time. So the wizard's Rendering tab (Target Object, Additional Parameters) stays empty, and here the wizard is simply "copy this file over with the right owner and mode".

It is a system template

The template is flagged is_system, so it cannot be edited in place. Duplicate it if you need to change the script body — the copy is editable and gets its own code.

Step 2 — Schedule it¶

Add one crontab line per folder to replicate, in the crontab of the user that owns the destination path, can SSH to the primary, and can run psql:

# m h  dom mon dow   command
*/1 * * * * /opt/muppy/tools/mpy_rsync_with_metrics.sh --name=odoo_filestore --pg-cluster=18/main --source-ssh=odoo@odoo-primary.example-customer.tld --source-path=/opt/odoo/odoo_files/odoo_data_dir/* --dest-path=/postgresql_data/odoo_files/odoo_data_dir
*/1 * * * * /opt/muppy/tools/mpy_rsync_with_metrics.sh --name=reflex_files --pg-cluster=18/main --source-ssh=odoo@odoo-primary.example-customer.tld --source-path=/opt/odoo/odoo_files/reflex_interfaces_files/* --dest-path=/postgresql_data/odoo_files/reflex_interfaces_files

A one-minute schedule is fine even when a sync takes longer than a minute: flock makes an overlapping run increment mpy_rsync_skipped_total and exit immediately.

Three flags worth knowing:

--pg-cluster=<version>/<name> — run the replica check against the local cluster, as sudo -n -u postgres psql -X --cluster=…. Use it on any host whose /etc/muppy.env points PG* at another server. Omit it and the check keeps its historical behaviour: connect with the ambient PG*.
--dont-check-replica — skip the pg_is_in_recovery() test entirely, for a host that is not a PostgreSQL standby.
--metric-folder — write the .prom elsewhere than /opt/muppy/node_exporter/textfile.

--dont-check-replica is a legitimate escape hatch

Use it when the host is not a PostgreSQL standby, or when the replica check cannot work and you need the sync running now. It is what unblocks an install.

What you trade: mpy_rsync_replica_check reports -1 instead of 1, and after a failover the promoted node keeps pulling from the demoted one. For the Odoo filestore that costs little — its files are content-addressed, so a same-named file is same-content. For a sync carrying mutable files, such as interface directories, a stale overwrite is real.

Once --pg-cluster is in place you can drop the flag and get the guard back.

Systemd timer (beta)

Muppy can also schedule this through a oneshot service plus a timer, created as Unit Definitions (see Creating Systemd Units), which would put the schedule under Muppy's control and send the output to journald. That path has not been tested with this script — treat it as beta and prefer crontab for now.

Step 3 — Verify by hand¶

Run the script once, then check the three surfaces in order:

# 1. the metrics file exists and carries a non-zero success timestamp
cat /opt/muppy/node_exporter/textfile/mpy_rsync_odoo_filestore.prom

# 2. the rsync itself did what you expect
ls -1t /var/log/mpy_rsync/ | head -1

# 3. node_exporter actually re-exports the metrics
curl -s localhost:9100/metrics | grep mpy_rsync

Do not create the health check before the third command returns something. It is the only step that proves the textfile collector flag from the prerequisites took effect.

Step 4 — Monitor it¶

Go to Muppy › High Availability › Health Checks and create a record:

Field	Value
Type	`Prometheus Exporter`
Prometheus Exporter	the standby's node_exporter — picking it fills in the endpoint, port `9100`, path `/metrics`, method and expected code
Frequency	`1` `Minutes` — match the sync cadence
Result parser	leave empty — the processor reads the raw Prometheus text
Result processor	`Prometheus Metric Threshold Processor`
Health Check target	the standby host — supplies `{target_name}` in the alert
Notify alerts	on

Then fill Other Processor params (YAML) on the Check tab:

metric_name: mpy_rsync_last_success_timestamp
compute_duration: true
threshold: 360
description: "File replication ({labels}) from the Odoo primary has been interrupted for {duration_minutes:.0f} minutes"
alerts:
  sound: cosmic

compute_duration: true turns the timestamp into an age. The comparison defaults to gt, so the alert fires when that age passes threshold — 360 seconds, or six missed one-minute runs.

Set description and leave title and message alone. The processor supplies both, and its default message already prints the target, the metric, the label, the value and the threshold. description is the one sentence those lines cannot write for you.

Placeholders: {target_name}, {health_check_name}, {metric_name}, {labels}, {value}, {value_display}, {threshold}, {duration}, {duration_minutes}, {duration_formatted}, {description}. They accept Python format specs. {duration_minutes:.0f} rounds to the minute — without it you get 7.383333333333334 minutes.

alerts.sound is the Pushover ringtone. Give this check its own, so you recognise it.

Press Test Health Check in the header to run it now. Open Results to read the metrics and alerts it produced.

One health check covers every sync on the host

The processor parses all metrics matching metric_name. It emits one result and one alert per label. A standby running both syncs above needs one health check, not two.

Watch the success timestamp, not the failure one

mpy_rsync_last_failure_timestamp looks like the natural target. It is a trap. With compute_duration and gt it fires when the last failure is old — so almost always, and permanently while the counter is still 0. Freshness on mpy_rsync_last_success_timestamp works, and covers failure anyway: a failed sync does not advance the success timestamp.

Alerts repeat every minute, on purpose

A notification goes out on every run while the threshold is exceeded. There is no state-change deduplication. That is the point: a broken filestore on a production Odoo should keep asking for attention. Use Mute alerts until while you fix the cause. Notifications go through Pushover and need the pushover.apptoken_userkey system parameter set to APP_TOKEN:USER_KEY. Each one deep-links back to the health check.

The default message says PostgreSQL Cluster

The processor labels the target PostgreSQL Cluster: whatever the target is, so this check reads PostgreSQL Cluster: <standby host>. Harmless, confusing on first sight. Override message if it bothers you.

What the alert actually catches¶

One freshness check covers four different failures, which is the reason to prefer it over anything that inspects the script's exit status:

rsync fails, or the primary is unreachable — the success timestamp stops advancing.
the crontab line is removed, or the script deleted — same signal, no special case.
the standby is promoted — the script takes its not-a-replica branch and overwrites the .prom file with only mpy_rsync_replica_check 0 and a skip timestamp. The success metric disappears entirely, the processor reports Metric Not Found, and you are alerted. Expected after a failover, and worth acting on: a promoted node is no longer syncing from anyone.
the replica check cannot answer — psql fails, sudo is refused, the cluster is down. The script logs a skip and exits 0 having synced nothing. With --pg-cluster the reason is logged; without it, the failure is swallowed and reads exactly like "this host is not a replica". Either way the script's exit status stays 0, which is why the alert is anchored on the freshness of the data rather than on the outcome of the run.