DumbMonit
中文
GitHub 在线演示

监控项

数据库与可观测性

PostgreSQL、MySQL、MongoDB、Redis、RabbitMQ、MQTT，以及 VictoriaMetrics、Loki 和 Graylog 服务器。

监控项描述和告警规则文本为英文，与产品一致。页面标签使用您的语言。

PostgreSQL database

postgres

Logs in and runs a query: catches a database that is up but no longer answering.

示例

  • The database behind Nextcloud or Immich
  • A Home Assistant recorder
  • An application database
  • A TimescaleDB instance

您需要准备

  • Username / passwordA read-only account created for monitoring, never an application one.

默认端口: 5432

内置告警规则 3

  • 警告Service downThe service has not responded correctly for three minutes.
  • 提示Service flappingThe service changed state more than six times in thirty minutes (changes(dumbmonit_probe_success{probe!="push"}[30m])).
  • 提示Slow serviceThe service takes more than three seconds to respond; failed checks are left out.
设置步骤: Monitor a PostgreSQL database

  1. In psql, create a role that may only log in.

    CREATE ROLE dumbmonit LOGIN PASSWORD 'a-long-password';

  2. Allow it to connect from the DumbMonit host with this line in pg_hba.conf (adjust the address), then reload PostgreSQL.

    host all dumbmonit 10.0.0.5/32 scram-sha-256

  3. In DumbMonit, enter the server, for example db.home.lan:5433 (port 5432 by default), then the role's user and password.

  4. Optional: replace the default query SELECT 1 with one that returns a single number, to chart it.

注意. The check opens a real connection on every poll: on an instance close to its connection limit, give it a longer interval.

文档

MySQL or MariaDB database

mysql

Logs in and runs a query: catches a database that is up but no longer answering.

示例

  • The database behind a WordPress
  • A Nextcloud or a Kimai
  • An application database
  • A MariaDB in Docker

您需要准备

  • Username / passwordA read-only account created for monitoring, never an application one.

默认端口: 3306

内置告警规则 3

  • 警告Service downThe service has not responded correctly for three minutes.
  • 提示Service flappingThe service changed state more than six times in thirty minutes (changes(dumbmonit_probe_success{probe!="push"}[30m])).
  • 提示Slow serviceThe service takes more than three seconds to respond; failed checks are left out.
设置步骤: Monitor a MySQL or MariaDB database

  1. In the mysql client, create an account with no privilege (adjust the host part to the DumbMonit machine).

    CREATE USER 'dumbmonit'@'%' IDENTIFIED BY 'a-long-password';

  2. In DumbMonit, enter the server, for example db.home.lan:3307 (port 3306 by default), then the account's user and password.

  3. Optional: replace the default query SELECT 1 with one that returns a single number, to chart it.

注意. The check opens a real connection on every poll: on an instance close to its connection limit, give it a longer interval.

文档

SQL Server

mssql

The database engine itself: databases that are not online, blocked requests, sessions against the limit, disk space of the data volumes, SQL Server Agent jobs and AlwaysOn availability groups.

示例

  • SQL Server 2019 to 2025
  • SQL Server Express
  • Azure SQL Managed Instance

您需要准备

  • SQL Server loginA SQL Server authentication login, not a Windows account. Sent in the TDS login packet over TLS.

默认端口: 1433

内置告警规则 7

  • 警告SQL Server database unavailableA database is SUSPECT, RECOVERY_PENDING, OFFLINE or EMERGENCY.
  • 提示SQL Server blocked requestsA request has been waiting on a lock for more than 5 minutes: a transaction was probably left open.
  • 提示SQL Server volume almost fullA volume holding database files is more than 90 % full; clears below 85 %.
  • 提示SQL Server Agent job failedThe last run of an enabled SQL Server Agent job failed (backups, maintenance).
  • 警告SQL Server AlwaysOn replica unhealthyA database replica of an availability group is not synchronizing in good health.
  • 提示SQL Server connections near the limitUser sessions use more than 90 % of the configured connection limit. No series without a limit.
再显示 1 条
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Create a read-only SQL Server login

  1. Connect to the instance with SQL Server Management Studio or sqlcmd, with a login allowed to create logins, and create the login DumbMonit will use.

    CREATE LOGIN dumbmonit WITH PASSWORD = 'choose-a-long-password', CHECK_POLICY = ON;

  2. Give it the server-wide read rights it needs to see the state of the engine, and nothing else.

    GRANT VIEW SERVER STATE TO dumbmonit;
GRANT VIEW ANY DEFINITION TO dumbmonit;

  3. Optional, for the SQL Server Agent jobs: let it read the job history in msdb.

    USE msdb;
CREATE USER dumbmonit FOR LOGIN dumbmonit;
ALTER ROLE SQLAgentReaderRole ADD MEMBER dumbmonit;

  4. Check that the instance accepts SQL Server authentication (mixed mode) and listens on a fixed TCP port: SQL Server Configuration Manager, Network Configuration, TCP/IP.

  5. In DumbMonit, enter the server address, for example sql.lan, or sql.lan:1444 for another port, then the login and its password.

  6. Tick Accept an unverifiable certificate while the server still presents its self-signed certificate.

注意. Windows authentication is not supported: use a SQL Server login, on an instance that allows SQL Server authentication (mixed mode). Named instances need their fixed TCP port in the address.

文档

Oracle Database

oracle

The database through Oracle REST Data Services: an instance that is not open, tablespaces filling up, sessions against the limit, Data Guard lag, the fast recovery area and failed scheduler jobs.

示例

  • Oracle Database 19c, 21c, 23ai
  • Oracle Database Free
  • Oracle XE

您需要准备

  • Username / passwordThe database account enabled for REST in ORDS, sent as HTTP basic authentication.

默认端口: 8080

内置告警规则 7

  • 警告Oracle instance not openThe instance status is not OPEN (MOUNTED, STARTED...): the database serves nothing.
  • 提示Oracle tablespace almost fullA tablespace is more than 90 % full of its maximum size; clears below 85 %.
  • 提示Oracle sessions near the limitSessions use more than 90 % of the sessions parameter (ORA-00018 when it is reached).
  • 提示Oracle Data Guard apply lagA standby applies redo more than 10 minutes behind the primary.
  • 警告Oracle recovery area almost fullThe fast recovery area is more than 90 % full: archiving can stall and freeze the database.
  • 提示Oracle scheduler job failedAt least one scheduler job run failed in the last 24 hours.
再显示 1 条
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Let DumbMonit read Oracle through ORDS

  1. Make sure Oracle REST Data Services (ORDS) runs in front of the database with REST-enabled SQL switched on.

    ords config set restEnabledSql.active true

  2. As a privileged user, create the account DumbMonit will use, with the two rights it needs.

    CREATE USER dumbmonit IDENTIFIED BY "choose-a-long-password";
GRANT CREATE SESSION, SELECT_CATALOG_ROLE TO dumbmonit;

  3. Connected as that account, enable its schema for REST with the alias dumbmonit.

    EXEC ORDS.ENABLE_SCHEMA(p_enabled => TRUE, p_url_mapping_type => 'BASE_PATH', p_url_mapping_pattern => 'dumbmonit');

  4. In DumbMonit, enter the ORDS address, for example ords.lan:8080 or https://ords.lan:8443, then the account and its password.

  5. If the schema alias or the ORDS path differ from the defaults, set them in the options below.

注意. There is no native Oracle client: DumbMonit reads the database through ORDS over HTTP, and never contacts the listener on port 1521. The REST-enabled SQL settings vary between ORDS versions: check the ORDS documentation of yours.

文档

MongoDB

mongodb

The document database itself: replica set members and lag, a missing primary, connections against the limit, WiredTiger cache and assertions.

示例

  • MongoDB replica set
  • Standalone mongod
  • The database behind UniFi or Rocket.Chat

您需要准备

  • Username / passwordAn account with the clusterMonitor role only, logged in with SCRAM-SHA-256.

无需账户。

默认端口: 27017

内置告警规则 7

  • 警告MongoDB replica set without primaryNo member is primary: no write is accepted.
  • 警告MongoDB member unreachableA replica set member is unreachable from the others.
  • 提示MongoDB replication lagA secondary is more than a minute behind the primary; clears below 30 s.
  • 提示MongoDB connections near the limitMore than 80 % of the available connections in use; clears below 75 %.
  • 提示MongoDB cache under pressureMore than 20 % of the WiredTiger cache is dirty; clears below 15 %.
  • 提示MongoDB internal errorsRegular assertions in the last hour (increase_prometheus(dumbmonit_mongodb_asserts_total{type="regular"}[1h])).
再显示 1 条
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Create a read-only MongoDB account

  1. In mongosh, create the account with the clusterMonitor role.

    use admin
db.createUser({user: "dumbmonit", pwd: passwordPrompt(), roles: [{role: "clusterMonitor", db: "admin"}]})

  2. In DumbMonit, enter the address of one mongod, for example db1.lan:27018, then the user and its password.

  3. In a replica set, add every member as its own monitor.

  4. Tick TLS if the server requires it.

注意. A mongodb+srv:// address is not resolved (enter each member's host), and MongoDB older than 3.6 and mongos routers are not supported.

文档

Redis / Valkey

redis

The in-memory store itself: memory against maxmemory, refused connections, replication link and lag, failed saves and keyspace size.

示例

  • Redis
  • Valkey
  • The cache behind Nextcloud, Immich or Authentik

您需要准备

  • ACL user (recommended)A Redis user allowed to run INFO and PING only. Sent as AUTH user password.
  • Password onlyFor Redis 5 or older, or a server with only requirepass set. Sent as AUTH password.

无需账户。

默认端口: 6379

内置告警规则 7

  • 提示Redis memory near maxmemoryMemory used above 90 % of maxmemory; clears below 85 %. No series without maxmemory.
  • 警告Redis refusing connectionsClient connections refused in the last 15 minutes: maxclients reached (increase_prometheus(dumbmonit_redis_rejected_connections_total[15m])).
  • 警告Redis replication link downA replica lost its link to the primary.
  • 提示Redis replica laggingSeen from the primary, a replica has not acknowledged for more than 30 seconds.
  • 警告Redis snapshot failedThe last BGSAVE failed; with stop-writes-on-bgsave-error, writes are refused.
  • 警告Redis append-only file write failedThe last AOF write failed.
再显示 1 条
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Create a read-only Redis user

  1. In redis-cli, create the user (Redis 6 or later, or Valkey).

    ACL SETUSER dumbmonit on >a-long-password -@all +info +ping

  2. Save it: ACL SAVE if the server uses an aclfile, CONFIG REWRITE otherwise.

    ACL SAVE

  3. In DumbMonit, enter the server address, for example cache.lan:6380, then the user and its password.

  4. Tick TLS if the server only accepts TLS on that port (tls-port).

注意. On Redis 5 or older, or with only requirepass set, pick Password only: that password carries every right.

文档

RabbitMQ

rabbitmq

The message broker: memory and disk alarms that block publishers, stopped nodes and partitions, queues filling up with no consumer.

示例

  • RabbitMQ single node
  • RabbitMQ cluster

您需要准备

  • Username / passwordA user tagged monitoring, sent as HTTP basic authentication.

默认端口: 15672

内置告警规则 6

  • 警告RabbitMQ memory or disk alarmA node raised a resource alarm: every publisher is blocked.
  • 警告RabbitMQ node downA cluster node is not running.
  • 警告RabbitMQ network partitionA node sees a network partition.
  • 提示RabbitMQ queue without consumerA queue holds messages and has no consumer; the value is the number of messages waiting.
  • 提示RabbitMQ queue unavailableA queue is not running: quorum queue without majority, or its node is down.
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Create a RabbitMQ monitoring user

  1. Open the RabbitMQ management interface, for example http://rabbit.lan:15672, and log in with a user allowed to manage users.

  2. Go to Admin → Users → Add a user, name it dumbmonit, set a password and the tag monitoring, then click Add user.

  3. Click dumbmonit in the list, and under Set permission pick each virtual host, set Configure, Write and Read to ^$, then click Set permission.

  4. In DumbMonit, enter the address of one node, for example rabbit.lan, or https://rabbit.lan:15671 when the API is served over TLS.

注意. A memory or disk alarm blocks every publisher in the cluster, and clients get no error.

文档

Apache Kafka

kafka

The event streaming cluster: offline and under-replicated partitions, a missing controller, and the lag of the consumer groups.

示例

  • Apache Kafka (KRaft or ZooKeeper)
  • Redpanda
  • Confluent Platform

您需要准备

  • SASL/PLAINFor a SASL_SSL listener only: the password is accepted over TLS and never sent in clear text.

无需账户。

默认端口: 9092

内置告警规则 5

  • 警告Kafka partitions offlineA partition has no leader: reads and writes to it fail.
  • 提示Kafka under-replicated partitionsPartitions have fewer in-sync replicas than replicas for 10 minutes: a broker is down or lagging.
  • 警告Kafka without controllerThe cluster reports no controller: no leader election is possible.
  • 提示Kafka consumer group lagA consumer group is more than 10 000 messages behind for 15 minutes. Adjust the threshold to the traffic of your topics.
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Give DumbMonit read-only access to Kafka

  1. Skip this step for a cluster without an authorizer. Otherwise give a dedicated principal read-only ACLs, from a host that can reach the brokers.

    kafka-acls.sh --bootstrap-server kafka.lan:9092 --add --allow-principal User:dumbmonit --operation Describe --cluster
kafka-acls.sh --bootstrap-server kafka.lan:9092 --add --allow-principal User:dumbmonit --operation Describe --topic '*'
kafka-acls.sh --bootstrap-server kafka.lan:9092 --add --allow-principal User:dumbmonit --operation Describe --group '*'

  2. In DumbMonit, enter the address of one broker, for example kafka.lan:9092: DumbMonit then contacts the other brokers at the addresses they advertise.

  3. Tick TLS if the listener is SSL or SASL_SSL, and enter the SASL/PLAIN user and password if the listener asks for them.

注意. Each broker must be reachable at the address it advertises (advertised.listeners): a broker DumbMonit cannot reach leaves the consumer lag incomplete. SASL/PLAIN is accepted only over TLS; SCRAM and Kerberos are not supported.

文档

MQTT broker

mqtt

Connects, subscribes to a topic, and can wait for a retained message.

示例

  • Mosquitto
  • The broker behind Home Assistant
  • Zigbee2MQTT
  • ESPHome sensors

您需要准备

  • Username / passwordSent once the connection is encrypted, never before.

无需账户。

默认端口: 1883

内置告警规则 3

  • 警告Service downThe service has not responded correctly for three minutes.
  • 提示Service flappingThe service changed state more than six times in thirty minutes (changes(dumbmonit_probe_success{probe!="push"}[30m])).
  • 提示Slow serviceThe service takes more than three seconds to respond; failed checks are left out.
设置步骤: Monitor an MQTT broker

  1. In DumbMonit, enter the broker, for example broker.home.lan:1884 (port 1883 by default).

  2. Tick Encrypted connection for a broker listening on 8883.

  3. If the broker requires an account, fill in its user and password.

  4. Optional: name a topic, to check that the account may still subscribe to it.

  5. Optional: tick Expect a retained message only for a topic that carries one, typically an availability topic.

注意. The check never publishes, and it cannot tell how old a retained message is: MQTT does not date them.

文档

Elasticsearch / OpenSearch

elasticsearch

The search cluster: red or yellow status, unassigned shards, disk filling up on a node and JVM heap pressure.

示例

  • Elasticsearch 7, 8, 9
  • OpenSearch 1, 2, 3
  • The index behind Graylog or Wazuh

您需要准备

  • Username / passwordA user holding only the cluster privilege monitor, sent as HTTP basic authentication.
  • Bearer tokenA service account token, sent as a bearer token. Elasticsearch API keys use another scheme and are not supported.

无需账户。

默认端口: 9200

内置告警规则 6

  • 警告Elasticsearch cluster redThe cluster status is red: at least one primary shard is not allocated and part of the data is unavailable.
  • 提示Elasticsearch cluster yellowThe cluster has been yellow for 30 minutes: replicas are missing and the next node failure will lose data.
  • 提示Elasticsearch unassigned shardsShards have stayed unassigned for 15 minutes.
  • 提示Elasticsearch node disk almost fullA node's data disk is more than 85 % full (the high watermark is 90 %, the read-only flood stage 95 %); clears below 80 %.
  • 提示Elasticsearch JVM heap pressureA node's JVM heap stays above 90 % for 15 minutes: garbage collection will slow the node.
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Create a monitoring-only Elasticsearch user

  1. Create a role that only holds the cluster privilege monitor, from Kibana (Stack Management, Roles) or with the API.

    curl -u elastic -X PUT "https://es.lan:9200/_security/role/dumbmonit" -H "Content-Type: application/json" -d '{"cluster":["monitor"]}'

  2. Create the user with that role.

    curl -u elastic -X PUT "https://es.lan:9200/_security/user/dumbmonit" -H "Content-Type: application/json" -d '{"password":"choose-a-long-password","roles":["dumbmonit"]}'

  3. OpenSearch: create a role with the cluster permission cluster_monitor in Security, Roles, and map the user to it.

  4. In DumbMonit, enter the address of one node, for example es.lan, then the user and its password.

  5. Elasticsearch 8 and later serve HTTPS with a generated certificate: pick HTTPS, and tick Accept an unverifiable certificate unless you installed your own.

注意. The account needs no index privilege: DumbMonit never reads a document. One node reports the whole cluster, so enter a node that is not the one you expect to fail.

文档

InfluxDB

influxdb

The time series database: health and readiness, uptime, memory, file descriptors and server errors, for InfluxDB 2 and 3.

示例

  • InfluxDB 2.x
  • InfluxDB 3 Core
  • InfluxDB 3 Enterprise

您需要准备

  • API tokenSent as a bearer token. InfluxDB 3 needs it on /health; InfluxDB 2 does not.

无需账户。

默认端口: 8086

内置告警规则 4

  • 警告InfluxDB unhealthyInfluxDB answers its health check with a failure.
  • 提示InfluxDB not readyInfluxDB is up but does not report itself ready for queries and writes.
  • 提示InfluxDB server errorsMore than 10 API requests answered with a 5xx status in the last 15 minutes (increase_prometheus(dumbmonit_influxdb_http_requests_5xx_total[15m])).
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Let DumbMonit read InfluxDB's health

  1. InfluxDB 2: nothing to create, /health, /ready and /metrics are open. Leave the credential on No authentication.

  2. InfluxDB 3 asks for a token on /health and /metrics by default: paste a token as API token, or start the server with --disable-authz health,ping,metrics.

  3. In DumbMonit, enter the server address, for example influx.lan, with the port 8086 for InfluxDB 2 or 8181 for InfluxDB 3.

  4. Tick HTTPS if the server, or the reverse proxy in front of it, serves TLS.

注意. InfluxDB 3 answers 401 on /health without a token unless it runs with --disable-authz. The token is only used to read health and metrics.

文档

VictoriaMetrics

victoriametrics

The time series database itself: ingestion, samples it refuses, disk headroom before it turns read-only, slow inserts and active series.

示例

  • VictoriaMetrics single-node
  • Prometheus long-term storage

您需要准备

  • Username / passwordSent as HTTP basic authentication.
  • Bearer tokenSent as "Authorization: Bearer …" on every request.

无需账户。

默认端口: 8428

内置告警规则 6

  • 警告Metrics or log storage read-onlyFree disk space fell below -storage.minFreeDiskSpaceBytes and the storage refuses every write.
  • 提示Metrics or log storage almost read-onlyLess than 10% of the disk left before read-only; clears above 12 %.
  • 警告Metrics or log server unhealthy/health does not answer OK.
  • 提示Metrics or log data refusedSamples or log lines refused in the last hour: timestamps outside retention, labels too long, series limit (increase_prometheus(dumbmonit_victoriametrics_rows_rejected_total[1h]) or increase_prometheus(dumbmonit_victorialogs_rows_rejected_total[1h])).
  • 提示VictoriaMetrics slow insertsMore than 5% of new samples took the slow path over 15 minutes: not enough memory for the active series ((100 * increase_prometheus(dumbmonit_victoriametrics_slow_inserts_total[15m]) / increase_prometheus(dumbmonit_victoriametrics_rows_added_total[15m])) and on (target) dumbmonit_victoriametrics_uptime_seconds > 900); clears below 3 %. Only judged once the server has run for 15 minutes, so a restart (cold cache, counters back to zero inside the window) does not leave it pending.
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Let DumbMonit read VictoriaMetrics' own health

  1. Nothing to install: check from the DumbMonit host that VictoriaMetrics answers on its API port (8428 by default).

    curl http://victoriametrics.lan:8428/health

  2. If it runs with -httpAuth.username and -httpAuth.password, pick Username / password below; behind vmauth or a proxy that expects a bearer token, pick Bearer token.

  3. In DumbMonit, enter the server address, for example "victoriametrics.lan" or "http://10.0.0.5:8428".

注意. Free disk space is what to watch: below -storage.minFreeDiskSpaceBytes (100 MB by default) VictoriaMetrics turns read-only and refuses every new sample. DumbMonit shows the headroom left and warns under 10%.

文档

VictoriaLogs

victorialogs

The log database itself: lines ingested, lines it refuses, disk headroom before it turns read-only, errors.

示例

  • VictoriaLogs single-node

您需要准备

  • Username / passwordSent as HTTP basic authentication.
  • Bearer tokenSent as "Authorization: Bearer …" on every request.

无需账户。

默认端口: 9428

内置告警规则 6

  • 警告Metrics or log storage read-onlyFree disk space fell below -storage.minFreeDiskSpaceBytes and the storage refuses every write.
  • 提示Metrics or log storage almost read-onlyLess than 10% of the disk left before read-only; clears above 12 %.
  • 警告Metrics or log server unhealthy/health does not answer OK.
  • 提示Metrics or log data refusedSamples or log lines refused in the last hour: timestamps outside retention, labels too long, series limit (increase_prometheus(dumbmonit_victoriametrics_rows_rejected_total[1h]) or increase_prometheus(dumbmonit_victorialogs_rows_rejected_total[1h])).
  • 提示VictoriaMetrics slow insertsMore than 5% of new samples took the slow path over 15 minutes: not enough memory for the active series ((100 * increase_prometheus(dumbmonit_victoriametrics_slow_inserts_total[15m]) / increase_prometheus(dumbmonit_victoriametrics_rows_added_total[15m])) and on (target) dumbmonit_victoriametrics_uptime_seconds > 900); clears below 3 %. Only judged once the server has run for 15 minutes, so a restart (cold cache, counters back to zero inside the window) does not leave it pending.
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Let DumbMonit read VictoriaLogs' own health

  1. Nothing to install: check from the DumbMonit host that VictoriaLogs answers on its API port (9428 by default).

    curl http://victorialogs.lan:9428/health

  2. If it runs with -httpAuth.username and -httpAuth.password, pick Username / password below; behind vmauth or a proxy that expects a bearer token, pick Bearer token.

  3. In DumbMonit, enter the server address, for example "victorialogs.lan" or "http://10.0.0.6:9428".

注意. Free disk space is what to watch: below -storage.minFreeDiskSpaceBytes (10 MB by default) VictoriaLogs turns read-only and refuses every new log line. DumbMonit shows the headroom left and warns under 10%.

文档

Grafana Loki

loki

The log server itself: readiness, log lines it refuses and why, chunks that fail to reach storage, write-ahead log disk, request errors.

示例

  • Loki single binary
  • Loki simple scalable

您需要准备

  • Username / passwordSent as HTTP basic authentication.
  • Bearer tokenSent as "Authorization: Bearer …" on every request.

无需账户。

默认端口: 3100

内置告警规则 6

  • 警告Loki not ready/ready does not answer ready.
  • 提示Loki refusing log linesLines refused in the last hour: too old, rate or stream limit, line too long (increase_prometheus(dumbmonit_loki_discarded_lines_total[1h])).
  • 警告Loki cannot flush to storageChunks failed to reach storage in the last 30 minutes; they are lost if Loki restarts (increase_prometheus(dumbmonit_loki_flush_failures_total[30m])).
  • 警告Loki write-ahead log disk fullWAL writes lost to a full disk (increase_prometheus(dumbmonit_loki_wal_disk_full_failures_total[30m])).
  • 提示Loki request errorsMore than five requests answered with a server error in 15 minutes (increase_prometheus(dumbmonit_loki_request_errors_total[15m])).
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Let DumbMonit read Loki's own health

  1. Nothing to install: check from the DumbMonit host that Loki answers on its HTTP port (3100 by default).

    curl http://loki.lan:3100/ready

  2. Loki has no login of its own: only if a reverse proxy in front asks for one, pick Username / password (or Bearer token) below.

  3. In DumbMonit, enter Loki's address, for example "loki.lan" or "https://logs.lan/loki" behind a reverse proxy.

  4. With Loki split into components (simple scalable or microservices), add at least the write path.

注意. Refused lines are the failure to watch: when Loki refuses a line (too old, over a rate or stream limit, too long), the sender usually drops the batch and Grafana only shows a gap in the logs.

文档

Graylog

graylog

The log server and its search cluster: processing, journal backlog, buffers, throughput in and out, OpenSearch health, failed inputs and indexing failures.

示例

  • Graylog Open
  • Graylog with OpenSearch

您需要准备

  • Access token (recommended)A token of the dedicated Graylog user, created from its Edit tokens page.
  • User name / passwordThe dedicated Graylog user itself, sent as HTTP basic authentication.

默认端口: 9000

内置告警规则 7

  • 警告Graylog not processingMessage processing paused on the node.
  • 警告Graylog search cluster downOpenSearch or Elasticsearch red, or unreachable from Graylog.
  • 提示Graylog search cluster yellowReplica shards unassigned for half an hour.
  • 提示Graylog journal fillingThe journal is more than half full: the output does not keep up; clears below 40 %.
  • 提示Graylog input failedAn input failed to start.
  • 提示Graylog indexing failuresWrites to the search cluster or message processing failed in the last hour (increase_prometheus(dumbmonit_graylog_output_failures_total[1h]) + increase_prometheus(dumbmonit_graylog_processing_failures_total[1h])).
再显示 1 条
  • 警告Monitor unreachableNo measurement received for more than three minutes. bottomk by (target) (1, time() - tlast_over_time(dumbmonit_up[7d]))
设置步骤: Create a read-only user and token in Graylog

  1. Open System → Users and Teams → Create user, name it as follows, give it a long random password and assign the Reader role only.

    dumbmonit

  2. Open that user's Edit tokens page, create a token named after DumbMonit and copy it: Graylog shows it only once.

  3. Optional, for Graylog's own notifications: with an account that can manage roles, create a role holding only notifications:read and give it to the dumbmonit user.

    curl -u YOUR_USER -H 'X-Requested-By: cli' -H 'Content-Type: application/json' -X POST https://graylog.lan/api/roles -d '{"name":"DumbMonit Notifications","description":"Read system notifications","permissions":["notifications:read"],"read_only":false}'
curl -u YOUR_USER -H 'X-Requested-By: cli' -X PUT 'https://graylog.lan/api/roles/DumbMonit%20Notifications/members/dumbmonit'

  4. In DumbMonit, enter the Graylog node address (for example graylog.lan, port 9000) and paste the token. In a cluster, add each node.

注意. Do not reuse your own account: a leaked token carries every right of its user. Graylog serves plain HTTP unless TLS is configured: across an untrusted network, put it behind HTTPS.

文档

其他系列