DumbMonit
FR
GitHub Démo en ligne

Surveillances

NAS et sauvegarde

Synology, TrueNAS, Unraid, Proxmox Backup Server et Veeam : disques, pools, et l'âge de la dernière bonne sauvegarde.

Les descriptions des surveillances et les textes des règles d'alerte sont en anglais, comme dans le produit. Les libellés de la page suivent votre langue.

Synology DSM

synology

Volumes, pools, disk health, temperature, Hyper Backup and Active Backup jobs, Synology Drive and Photos.

Exemples

  • DiskStation
  • RackStation

Ce qu'il faut

  • DSM account

Port par défaut: 5001

Règles d'alerte livrées 17

  • AvisSynology disk SMART warningDSM no longer reports this disk's SMART status as normal.
  • AlerteSynology disk failedDSM reports the disk as crashed or failed.
  • AlerteSynology disk bad sectorsThe disk's bad-sector count exceeds the threshold set in DSM.
  • AvisSynology disk bad sectors growingNew unreadable sectors appeared in the last 24 hours (delta(dumbmonit_synology_disk_unc_count[24h])).
  • AvisSynology SSD wearing outAn SSD has less than 10 % of its rated life left; clears above 12 %.
  • AlerteSynology volume degradedA volume is degraded or crashed: its redundancy is gone.
Afficher 11 de plus
  • AlerteSynology storage pool degradedA storage pool or SSD cache is degraded or crashed: a disk failed and the redundancy is gone or rebuilding.
  • AvisSynology volume almost fullA volume is 90 % full or more, not counting the space reserved by thick-provisioned iSCSI LUNs; clears below 88 %. A volume filled by a thick LUN is full by design and does not alert.
  • AvisSynology thin LUN over-allocationThe thin LUNs of a volume were promised more space than the volume still has free; clears below 95 %.
  • AvisSynology thin LUN almost fullA thin LUN has 95 % or more of its size written. A thick LUN is opaque from the NAS: the rules of the monitor that uses it (datastore, filesystem) alert instead.
  • AvisSynology temperature highA disk is above 55 °C, or DSM raised its own temperature warning; clears below 52 °C.
  • AvisSynology memory highMemory usage of the NAS above 95 %; clears below 90 %.
  • AlerteActive Backup task failedThe last run of an Active Backup for Business task failed, or backed up only part of its devices. The notification names the task.
  • AvisActive Backup too oldNo successful Active Backup for Business run for this task for more than two days. The series does not exist for a task that has never succeeded; "Active Backup task failed" speaks then.
  • InfoActive Backup task disabledThe task has no schedule, or its continuous backup is paused: it will not back up anything until someone runs it.
  • AvisActive Backup device overdueA device has gone longer without a successful Active Backup for Business run than its own rhythm allows, off-days excluded).
  • AlerteActive Backup device failingThe last two or more attempts of a device failed; a cancelled run counts as neither.
Étapes de mise en place: Create a monitoring account in DSM

  1. In DSM, open Control Panel → User & Group → User → Create and name the account as follows, with a long password used nowhere else.

    dumbmonit

  2. Join groups: tick administrators.

  3. Assign shared folder permissions: tick No access on every shared folder.

  4. Assign application permissions: Deny every application except DSM, and Active Backup for Business if you want its tasks read.

  5. Keep two-step verification off for this account (Control Panel → Security → Account).

  6. In DumbMonit, enter the NAS address (HTTPS, port 5001 by default) and this account's user name and password.

À savoir. The administrators group is required by DSM's storage API, not by DumbMonit: that is why the account gets no shared folder and no other application.

Documentation

TrueNAS

truenas

Pool health, the failing disk, scrubs, snapshots, replication and TrueNAS's own alerts.

Exemples

  • TrueNAS SCALE
  • TrueNAS Community Edition

Ce qu'il faut

  • API keyThe key TrueNAS shows once when it is created, as "3-AbCd…".

Port par défaut: 443

Règles d'alerte livrées 13

  • AlerteNAS pool degradedZFS no longer calls a pool healthy. A degraded pool still serves its data, which is exactly why nobody notices until the next disk fails. The monitor page names the disk.
  • AvisNAS disk errorsDisks of a pool have reported read, write or checksum errors since the last zpool clear.
  • AvisNAS pool almost fullA pool is more than 85 % full. ZFS slows down well before it is full. Clears at 80 %.
  • AlerteNAS scrub found errorsThe last scrub of a pool found damaged data.
  • InfoNAS scrub overdueA pool has not completed a scrub in more than 45 days. Absent after a resilver, which erases the record of the last scrub.
  • AlerteNAS disk failed its SMART testA SMART self-test in a disk's log failed. A disk never tested has no series.
Afficher 7 de plus
  • AvisNAS disk too hotA disk is above 55 °C. Clears at 50 °C.
  • AvisNAS dataset near its quotaA dataset has used more than 90 % of its quota: at 100 % its writes fail while the pool still has room. No quota, no series.
  • AvisNAS replication failedAn enabled replication task is in error: the copy on the other side is getting older.
  • AvisNAS snapshot task failedAn enabled periodic snapshot task is in error.
  • AvisNAS snapshots staleAn enabled periodic snapshot task has not run for more than eight days: a task that stops running does not fail, it goes quiet.
  • AvisTrueNAS alert raisedTrueNAS itself has raised an alert of level ERROR or above. Its own sentence is on the monitor page.
  • AvisNAS service stoppedA service set to start with the NAS is stopped. A service switched off on purpose has no series.
Étapes de mise en place: Create an API key for DumbMonit in TrueNAS

  1. Open Credentials → Groups → Add, name the group as follows and give it the Local Administrator privilege.

    dumbmonit

  2. Open Credentials → Users → Add, name the account as follows and make dumbmonit its primary group. Leave shell access, sudo and SSH off.

    dumbmonit

  3. Open Credentials → Users → API Keys → Add, pick the dumbmonit user, name the key and copy it: TrueNAS shows it only once.

  4. On TrueNAS 24.10, the path is the user menu at the top right → API Keys → Add.

  5. In DumbMonit, enter the NAS address (for example nas.lan) and paste the key. Keep HTTPS.

À savoir. This key can change anything on the NAS, because TrueNAS offers no read-only key over its REST API: keep it for DumbMonit alone and revoke it if it leaks. TrueNAS ships a self-signed certificate: if the connection is refused, tick "Accept an unverifiable certificate" in the options.

Documentation

Unraid

unraid

An Unraid server through its GraphQL API: array stopped, disks disabled or with errors, parity checks, cache pool usage, containers that should be running.

Exemples

  • Unraid 7
  • Unraid 6.12 with Unraid Connect

Ce qu'il faut

  • API keyA key with the Viewer role: it reads everything and can change nothing.

Port par défaut: 80

Règles d'alerte livrées 9

  • AlerteUnraid array stoppedThe Unraid array is not started: shares, containers and VMs that live on it are unavailable.
  • AlerteUnraid disk disabled or missingAn Unraid disk is disabled, missing or invalid: the array runs degraded, and one more failure loses data.
  • AvisUnraid disk read errorsAn Unraid disk has reported read errors since the array started: it may be disabled at the next one.
  • AvisUnraid disk too hotAn Unraid array or parity disk has stayed above 55 °C for fifteen minutes.
  • AvisUnraid parity check found errorsThe last Unraid parity check failed or found sync errors: parity no longer matches the data.
  • AvisUnraid parity check overdueThe last Unraid parity check is more than 40 days old: a monthly check was missed.
Afficher 3 de plus
  • AvisUnraid cache nearly fullAn Unraid cache pool is more than 90 % full: writes fall back to the slow array, or fail.
  • AvisUnraid container stoppedA Docker container set to start with the array is not running.
  • AlerteMonitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
Étapes de mise en place: Create a read-only API key for DumbMonit

  1. On Unraid 6.12 to 7.1, install the Unraid Connect plugin from the Apps tab first. Unraid 7.2 or later has the API built in.

  2. Open Settings → Management Access → API Keys and create a key named as follows, with the Viewer role.

    dumbmonit

  3. Copy the key: Unraid shows it only once.

  4. In DumbMonit, enter the Unraid address (for example tower.lan), paste the key as API key and choose https if the web interface is served over HTTPS.

À savoir. Unraid reports no temperature for a disk that is spun down, and DumbMonit never wakes one: a sleeping disk simply has no temperature until it spins up.

Documentation

QNAP QTS / QuTS hero

Bêtaqnap

A QNAP NAS through the CGI pages of its web interface: volumes filling up, disks whose SMART health is no longer good or too hot, processor temperature, firmware updates.

Exemples

  • QNAP TS-453D
  • QuTS hero NAS
  • QTS 5.x

Ce qu'il faut

  • User name and passwordThe dumbmonit account created on the NAS. The password is sent base64-encoded to the login page, as the web interface does.

Port par défaut: 443

Règles d'alerte livrées 6

  • AvisQNAP volume nearly fullA QNAP volume is more than 90 % full: shares, snapshots and apps stop being able to write.
  • AvisQNAP disk health not goodThe SMART health of a QNAP disk is no longer good (warning or abnormal): replace it before the RAID loses redundancy.
  • AvisQNAP disk too hotA QNAP disk has stayed above 55 °C for fifteen minutes.
  • AvisQNAP processor too hotThe QNAP processor has stayed above 90 °C for fifteen minutes.
  • InfoQNAP firmware updateQNAP offers a newer QTS or QuTS hero release.
  • AlerteMonitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
Étapes de mise en place: Create a limited account for DumbMonit

  1. Open Control Panel → Privilege → Users → Create → Create a User and create the account as follows, with a long random password. Tick no share and no application privilege.

    dumbmonit

  2. Add the account to the administrators group: QTS serves the system status, volumes and disks to that group only.

  3. Optional: under Control Panel → System → Security → Allow/Deny List, allow the web interface only from the DumbMonit server's address.

  4. In DumbMonit, enter the NAS address (for example nas.lan) with that user name and password. Choose http and port 8080 only if HTTPS is off.

À savoir. QTS has no read-only role for its system status, so the account sits in the administrators group: give it a unique password, and an account without 2-step verification. Over HTTP the password travels base64-encoded, not encrypted: prefer HTTPS.

Documentation

Ceph

Bêtaceph

A Ceph cluster through the REST API of the manager's dashboard: health and its checks, OSDs down or nearly full, placement groups not clean, capacity, pools, monitors out of the quorum.

Exemples

  • Ceph 18 (Reef)
  • Ceph 19 (Squid)
  • cephadm clusters

Ce qu'il faut

  • Dashboard user and passwordThe dumbmonit user of the Ceph dashboard, with the read-only role.

Port par défaut: 8443

Règles d'alerte livrées 10

  • AlerteCeph health errorThe Ceph cluster reports HEALTH_ERR: data may be unavailable or at risk.
  • AvisCeph health warningThe Ceph cluster has reported HEALTH_WARN for 30 minutes: see its health checks.
  • AvisCeph OSD downOne or more Ceph OSDs are down: the cluster rebuilds their data elsewhere and loses redundancy meanwhile.
  • AvisCeph OSD nearly fullA Ceph OSD is more than 85 % full (the default nearfull ratio): writes to it stop at the full ratio.
  • AvisCeph placement groups not cleanSome Ceph placement groups have not been active+clean for 30 minutes: data is degraded, misplaced or inactive.
  • AlerteCeph objects unfoundCeph reports unfound objects: no copy of some data can be found.
Afficher 4 de plus
  • AvisCeph capacity highThe Ceph cluster is more than 80 % full: it turns read-only at 95 %, and rebuilding after a failure needs free space.
  • AvisCeph pool nearly fullA Ceph pool is more than 90 % full.
  • AlerteCeph monitor out of quorumA Ceph monitor is out of the quorum: one more loss and the cluster stops.
  • AlerteMonitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
Étapes de mise en place: Create a read-only dashboard user for DumbMonit

  1. Check that the dashboard is enabled and note the address of the active manager.

    ceph mgr services

  2. Write a long random password to a file, and keep it: DumbMonit needs it.

    openssl rand -base64 24 | tr -d '\n' | tee /tmp/dumbmonit.pwd

  3. Create the user with the built-in read-only role.

    ceph dashboard ac-user-create dumbmonit -i /tmp/dumbmonit.pwd read-only

  4. Delete the password file.

    shred -u /tmp/dumbmonit.pwd

  5. In DumbMonit, enter the address of the active manager (for example ceph.lan) with that user name and password.

À savoir. A standby manager redirects to the active one: use the address of the active manager. The dashboard ships a self-signed certificate: tick "Accept an unverifiable certificate" unless you installed your own.

Documentation

MinIO

Bêtaminio

A MinIO object store through its Prometheus metrics: nodes and drives offline, capacity, failed heals, replication failures and S3 server errors.

Exemples

  • MinIO single node
  • MinIO distributed cluster

Ce qu'il faut

  • Prometheus tokenThe bearer_token printed by mc admin prometheus generate. It opens the metrics pages only.

Aucun compte nécessaire.

Port par défaut: 9000

Règles d'alerte livrées 7

  • AlerteMinIO node offlineA MinIO node is offline: the cluster runs with less parity.
  • AvisMinIO drive offlineA MinIO drive is offline: objects are rebuilt from parity until it is back.
  • AvisMinIO capacity highThe usable capacity of the MinIO cluster is more than 85 % used.
  • AvisMinIO heal failuresMinIO could not heal some objects in its current self-healing run.
  • AvisMinIO replication failuresObjects of a MinIO bucket failed to replicate to the remote site.
  • AvisMinIO S3 server errorsMore than 5 % of the S3 requests MinIO served in the last 10 minutes failed with a server error (5xx) (100 * rate(dumbmonit_minio_s3_requests_5xx_total[10m]) / rate(dumbmonit_minio_s3_requests_total[10m])).
Afficher 1 de plus
  • AlerteMonitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
Étapes de mise en place: Generate a Prometheus token for DumbMonit

  1. With the mc client configured for your MinIO (alias myminio), generate the token of the cluster metrics.

    mc admin prometheus generate myminio cluster

  2. Copy the value after bearer_token in the output: that is the token.

  3. In DumbMonit, enter the MinIO address (for example minio.lan), paste the token as Prometheus token and choose https if MinIO serves TLS.

À savoir. The token only opens the metrics pages, but anyone holding it can read them. If MinIO runs with MINIO_PROMETHEUS_AUTH_TYPE=public, the metrics need no token: choose No authentication.

Documentation

Proxmox Backup Server

pbs

Backup freshness per machine, failed tasks, sync and verify jobs, datastore space.

Exemples

  • Proxmox Backup Server

Ce qu'il faut

  • API token (recommended)No expiry and no session opened: the right choice for monitoring.
  • Username / passwordOpens a two-hour session, renewed automatically. Use it only if tokens are not an option.

Port par défaut: 8007

Règles d'alerte livrées 21

  • AvisPBS datastore almost fullPBS datastore more than 90% full.
  • AvisPBS datastore filling upAt the current rate, PBS estimates the datastore full within seven days.
  • AvisPBS datastore not mountedA removable datastore is not mounted: nothing can be written to it.
  • AvisPBS backup too oldNo new snapshot for this backup group (one per machine) for more than two days. Edit the threshold for machines backed up weekly; the notification names the group and datastore.
  • AlertePBS backup verification failedVerification of the latest snapshot for this machine failed.
  • AvisPBS task failedAt least one PBS task failed in the review window.
Afficher 15 de plus
  • AvisPBS garbage collection too oldNo successful garbage collection on this datastore for more than eight days.
  • AvisPBS garbage collection failedThe last garbage collection on a datastore failed.
  • AlertePBS corrupt chunks foundThe garbage collection found unreadable chunks on a datastore: some backups can no longer be restored.
  • AvisPBS prune job failedThe last run of a PBS prune job failed: nothing is pruned and the datastore keeps filling up. The notification names the job and datastore.
  • AvisPBS verification job failedThe last run of a PBS verification job failed.
  • AlertePBS sync job failedThe last run of a PBS sync job failed.
  • AlertePBS tape backup failedThe last tape backup run failed: the offline copy is not being made.
  • AvisPBS job never ranA job is enabled and scheduled but has never run.
  • AlertePBS service downA service Proxmox Backup Server needs is not running; no backup can be taken in while it is down.
  • AlertePBS disk SMART failureA disk of the backup server reports SMART health FAILED. The notification reads /dev/sdb.
  • AvisPBS SSD worn outAn SSD of the backup server has used more than 90 % of its rated endurance.
  • AlertePBS ZFS pool degradedA ZFS pool of the backup server is not ONLINE.
  • AvisPBS certificate expiringThe certificate served by the backup server expires in less than three weeks. Needs the certificates option.
  • InfoPBS restart pending after upgradeThe package was upgraded but the running daemon is still the old one.
  • InfoPBS updates pendingMore than twenty package updates are pending on the backup server.
Étapes de mise en place: Create a read-only user and token in Proxmox Backup Server

  1. Open Configuration → Access Control → User Management → Add, enter the user name dumbmonit, choose the realm Proxmox Backup authentication server, then click Add.

    dumbmonit@pbs

  2. Open Configuration → Access Control → API Token → Add: user dumbmonit@pbs, token name monitor. Copy the Token ID and the Secret now, the secret is shown only once.

    dumbmonit@pbs!monitor

  3. Open Configuration → Access Control → Permissions → Add → User Permission: path /, user dumbmonit@pbs, role Audit.

  4. Add a second permission with Add → API Token Permission: path /, token dumbmonit@pbs!monitor, role Audit. A token has its own permissions, so both are needed.

  5. Optional, to see pull sync jobs and the tape tier: repeat both permissions with role RemoteAudit on path /remote and role TapeAudit on path /tape.

  6. In DumbMonit, enter the server address (for example pbs.lan:8007), the Token ID and the Secret.

À savoir. Both the user and the token need the permission: given to the token alone, it grants nothing and PBS reports nothing. The server uses a self-signed certificate by default: tick "Accept an unverifiable certificate" in the options if the connection is refused.

Documentation

Veeam Backup & Replication

veeam

A Veeam backup server through its REST API: jobs whose last run failed, failed sessions of the last 24 hours, repositories filling up, license expiry.

Exemples

  • Veeam Backup & Replication 12
  • Veeam Backup & Replication 13
  • Veeam Community Edition

Ce qu'il faut

  • User name and passwordThe account given the Veeam Backup Viewer role, in the form HOST\user or DOMAIN\user.

Port par défaut: 9419

Règles d'alerte livrées 6

  • AvisClient device staleA device has not connected or not backed up within the staleness window configured for this target.
  • AlerteVeeam job failedThe last run of an enabled Veeam job failed: what it protects has no new restore point.
  • AvisVeeam job ended with a warningThe last run of an enabled Veeam job ended with a warning: some objects may not be protected.
  • AvisVeeam repository nearly fullA Veeam backup repository is more than 90 % full: the next jobs will fail for lack of space.
  • AvisVeeam license expiringThe Veeam license expires in less than 30 days: jobs stop at the end of the grace period.
  • AlerteMonitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
Étapes de mise en place: Give DumbMonit a Veeam Backup Viewer account

  1. On the backup server, open Computer Management → Local Users and Groups → Users → New User and create the account as follows, with a long random password.

    dumbmonit

  2. In the Veeam console, open the main menu > Users and Roles > Security, click Add, enter the account (for example VBR01\dumbmonit) and give it the Veeam Backup Viewer role only.

  3. In DumbMonit, enter the backup server address (for example vbr.lan) and the account as HOST\dumbmonit or DOMAIN\dumbmonit, with its password.

  4. Tick Accept an unverifiable certificate unless you installed your own: the REST API (port 9419) uses a self-signed one.

À savoir. Veeam keeps some REST API routes for the Backup Administrator role, the license among them on current versions: with the Viewer role DumbMonit shows the license as not readable and never asks for more.

Documentation

Kopia

Bêtakopia

A Kopia server through its REST API: the last snapshot of each source (age, errors, size, files), sources that missed their schedule, and the repository connection.

Exemples

  • Kopia server
  • KopiaUI
  • Kopia in Docker

Ce qu'il faut

  • Server user name and passwordThe user name and password the Kopia server was started with.

Aucun compte nécessaire.

Port par défaut: 51515

Règles d'alerte livrées 4

  • AvisKopia snapshot with errorsThe last snapshot of a Kopia source finished with errors: some files were not saved.
  • AvisKopia snapshot too oldA scheduled Kopia source has no snapshot for more than two days.
  • AlerteKopia repository disconnectedThe Kopia server is not connected to its repository: no snapshot can run.
  • AlerteMonitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
Étapes de mise en place: Start the Kopia server with its own user name and password

  1. On the machine that runs the backups, start the Kopia server with a generated certificate and its own user name and password.

    kopia server start --address=0.0.0.0:51515 --tls-generate-cert --server-username=dumbmonit --server-password=PASTE-A-LONG-RANDOM-PASSWORD

  2. Check that the repository is connected to that server and that the DumbMonit server can reach port 51515.

  3. In DumbMonit, enter the server address (for example nas.lan) with that user name and password.

  4. Tick Accept an unverifiable certificate: the generated certificate is self-signed. If the server runs with --insecure, choose the http protocol instead.

À savoir. Kopia's API has no read-only account: the server password also lets its holder start and delete snapshots. DumbMonit only reads, but keep port 51515 on your local network.

Documentation

Duplicati

Bêtaduplicati

A Duplicati server through its API: the age of each task's last successful backup, its last result, duration, size, warnings and errors, and the task running now.

Exemples

  • Duplicati 2.1
  • Duplicati in Docker
  • Duplicati on a NAS

Ce qu'il faut

  • Web interface passwordDuplicati has no user accounts: the password of its web interface is all it asks for.

Port par défaut: 8200

Règles d'alerte livrées 3

  • AlerteDuplicati backup failedThe last run of a Duplicati backup failed: what it protects has no new version.
  • AvisDuplicati backup too oldA scheduled Duplicati backup has not succeeded for more than two days.
  • AlerteMonitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
Étapes de mise en place: Let DumbMonit read the Duplicati API

  1. In the Duplicati web interface, open Settings and set a long random password for the interface: Duplicati 2.1 requires one.

  2. If the server only listens on the machine itself, start it with these options so that DumbMonit can reach it.

    --webservice-interface=any --webservice-allowed-hostnames=*

  3. In DumbMonit, enter the Duplicati address (for example nas.lan) and paste that password.

À savoir. Duplicati 2.1 or later is required: version 2.0 signs in through a different mechanism that DumbMonit does not use. The password gives full access, so keep port 8200 on your local network.

Documentation

Autres familles