外观
NATS 运维与观测
本页结论:NATS 的可观测性入口是 HTTP 监控端点(
/varz、/connz、/jsz、/healthz)与官方 CLI(nats);JetStream 的积压看 Consumer 的 pending/ack_pending,容量看 Stream 存储与账户配额。
监控端点(-m 8222 开启)
| 端点 | 内容 |
|---|---|
/healthz | 就绪检查(本仓库 compose healthcheck 使用) |
/varz | 服务器全局:连接数、入/出消息速率、内存、slow consumers |
/connz | 连接明细:每个客户端的订阅数、缓冲、收发计数 |
/subsz | 订阅明细 |
/jsz?consumers=1 | JetStream:Stream/Consumer 状态、存储占用、副本信息 |
/accountz | 账户与配额(多租户部署时) |
本仓库 compose 已将 8222 仅绑定 127.0.0.1。
官方 CLI 速查
bash
nats stream ls # Stream 列表与消息数
nats stream info ORDERS # 消息数、字节数、首末序列
nats consumer ls ORDERS # Consumer 列表
nats consumer info ORDERS orders-first # 位点、pending、ack_pending、重投统计
nats stream get ORDERS 3 --raw # 按序列号读取单条(回放排查)
nats pub orders.events '...' # 手工发布
nats sub 'orders.>' # 手工订阅(Core 层)积压定位决策树
text
消费延迟增大
├── Core 层:/varz slow_consumers 增长
│ └── 订阅者处理不过来被断开 ⇒ 加 Queue Group 成员 / 迁移到 JetStream
├── JetStream:consumer info 的 Num Pending 持续增长
│ ├── ack_pending 高:投递了没确认
│ │ └── AckWait 太短导致重复重投,或业务处理变慢
│ └── ack_pending 低:生产速率 > 消费速率
│ └── 增加共享该 Consumer 的客户端数;评估 Stream 吞吐上限
└── /jsz 存储接近账户 max_bytes
└── 调整保留策略/限额,或扩容存储关键指标映射
| 指标类别 | NATS 对应 |
|---|---|
| Producer | PublishAck 延迟与错误率(客户端埋点);/varz in_msgs 速率 |
| Broker | /varz:connections、in/out_msgs、mem;/jsz:storage、reserved |
| Consumer | consumer info:delivered、ack_pending、num_redelivered |
| Backlog | Consumer 的 num_pending(未投递)+ ack_pending(未确认) |
| DLQ 等价物 | 自建 DLQ Stream 的消息数与最老序列年龄 |
| Business | 幂等表 duplicate_skipped 计数(应用侧埋点) |
安全基线
- 认证:用户名密码(开发)、Token、NKey/签名挑战(推荐)、JWT(多租户运营)。
- 授权:按用户限制可发布/订阅的 Subject 集合;Account 层做租户隔离与导入导出。
- TLS:客户端-服务器与路由(cluster)链路均可启用;生产默认开启。
- 本仓库实验未开启认证(localhost 单节点);默认开放连接是开发配置,不是生产配置。
常见故障
| 现象 | 常见原因 | 处置 |
|---|---|---|
| 发布报 no space left | 账户/Stream 存储配额满 | 调整 max_bytes、清理或扩容 |
| 消息反复重投 | AckWait < 业务处理时长 | 调大 AckWait 或拆小批处理 |
| 副本长时间降级 | R3 节点失联/磁盘慢 | 检查节点健康与磁盘 IO;必要时 stream restore |
| 客户端频繁重连 | 网络抖动或服务器过载 | 检查 /varz 负载与客户端重连缓冲配置 |
cli-tools:纯自带 CLI 实验
nats:2.11.5 官方镜像是 distroless:文件系统仅含 /nats-server 单一二进制,无 shell、无自带收发 CLI(nats CLI 由 nats-io/natscli 独立发行)。因此本实验只做二进制盘点(--version/--help)与宿主机 8222 监控端点体检(/healthz、/varz、/connz、/subsz);CLI 缺口如实记录在 gap.out.txt(容器内 exec nats 实测 executable file not found)。收发需外部客户端,Java 版见 core-pubsub 实验。
nats / docker
bash demos/nats/docker/run.shassert.out.txt
--- status: verified capturedAt: "2026-08-23T15:19:49.324Z" dockerImage: "nats:2.11.5@sha256:47710a58674c152355730c69fc89d5a86369e3d6d1329cc91767650ce8e0dde6" exitCode: 0 --- PASS binVersion: 1 PASS healthzOk: ok PASS varzVersion: 2.11.5 PASS varzHasServerId: true PASS varzJetstream: true PASS connzConnections: 0 PASS subszParsed: true PASS gapRecorded: 1 PASS gapEvidence: 1 RESULT: all assertions passed
bin-list.out.txt
# nats:2.11.5 官方镜像为 distroless:文件系统仅含 /nats-server 单一二进制(无 shell、无其他 CLI),故无 bin 目录可列
[cli-tools] $ compose exec nats /nats-server --version
nats-server: v2.11.5
[cli-tools] $ compose exec nats /nats-server --help
Usage: nats-server [options]
Server Options:
-a, --addr, --net <host> Bind to host address (default: 0.0.0.0)
-p, --port <port> Use port for clients (default: 4222)
-n, --name
--server_name <server_name> Server name (default: auto)
-P, --pid <file> File to store PID
-m, --http_port <port> Use port for http monitoring
-ms,--https_port <port> Use port for https monitoring
-c, --config <file> Configuration file
-t Test configuration and exit
-sl,--signal <signal>[=<pid>] Send signal to nats-server process (ldm, stop, quit, term, reopen, reload)
<pid> can be either a PID (e.g. 1) or the path to a PID file (e.g. /var/run/nats-server.pid)
--client_advertise <string> Client URL to advertise to other servers
--ports_file_dir <dir> Creates a ports file in the specified directory (<executable_name>_<pid>.ports).
Logging Options:
-l, --log <file> File to redirect log output
-T, --logtime Timestamp log entries (default: true)
-s, --syslog Log to syslog or windows event log
-r, --remote_syslog <addr> Syslog server addr (udp://localhost:514)
-D, --debug Enable debugging output
-V, --trace Trace the raw protocol
-VV Verbose trace (traces system account as well)
-DV Debug and trace
-DVV Debug and verbose trace (traces system account as well)
--log_size_limit <limit> Logfile size limit (default: auto)
--max_traced_msg_len <len> Maximum printable length for traced messages (default: unlimited)
JetStream Options:
-js, --jetstream Enable JetStream functionality
-sd, --store_dir <dir> Set the storage directory
Authorization Options:
--user <user> User required for connections
--pass <password> Password required for connections
--auth <token> Authorization token required for connections
TLS Options:
--tls Enable TLS, do not verify clients (default: false)
--tlscert <file> Server certificate file
--tlskey <file> Private key for server certificate
--tlsverify Enable TLS, verify client certificates
--tlscacert <file> Client certificate CA for verification
Cluster Options:
--routes <rurl-1, rurl-2> Routes to solicit and connect
--cluster <cluster-url> Cluster URL for solicited routes
--cluster_name <string> Cluster Name, if not set one will be dynamically generated
--no_advertise <bool> Do not advertise known cluster information to clients
--cluster_advertise <string> Cluster URL to advertise to other servers
--connect_retries <number> For implicit routes, number of connect retries
--cluster_listen <url> Cluster url from which members can solicit routes
Profiling Options:
--profile <port> Profiling HTTP port
Common Options:
-h, --help Show this message
-v, --version Show version
--help_tls TLS help
connz.out.txt
[cli-tools] $ curl -fsS http://127.0.0.1:8222/connz
{
"server_id": "NBWDRAPRGUPMWTMYXIZKNZRK26AKOZNPIHAIFUMH5ZSY5DN2ZOKGD45T",
"now": "2026-08-23T15:17:22.004747792Z",
"num_connections": 0,
"total": 0,
"offset": 0,
"limit": 1024,
"connections": []
}
gap.out.txt
# 缺口记录:nats 官方镜像不含 nats CLI(由 nats-io/natscli 独立发行),distroless 镜像连 shell 都没有, # 纯镜像自带命令无法收发消息;收发需外部客户端(Java 实验见 demos/nats/core-pubsub)。 [cli-tools] $ compose exec nats nats --version # 尝试在容器内寻找自带 CLI,预期失败 OCI runtime exec failed: exec failed: unable to start container process: exec: "nats": executable file not found in $PATH exit=127(预期失败:镜像不含 nats 二进制)
inventory.out.txt
---
status: verified
capturedAt: "2026-08-23T15:19:49.324Z"
dockerImage: "nats:2.11.5@sha256:47710a58674c152355730c69fc89d5a86369e3d6d1329cc91767650ce8e0dde6"
exitCode: 0
---
# nats:2.11.5 官方镜像为 distroless:文件系统仅含 /nats-server 单一二进制(无 shell、无其他 CLI),故无 bin 目录可列
[cli-tools] $ compose exec nats /nats-server --version
nats-server: v2.11.5
[cli-tools] $ compose exec nats /nats-server --help
Usage: nats-server [options]
Server Options:
-a, --addr, --net <host> Bind to host address (default: 0.0.0.0)
-p, --port <port> Use port for clients (default: 4222)
-n, --name
--server_name <server_name> Server name (default: auto)
-P, --pid <file> File to store PID
-m, --http_port <port> Use port for http monitoring
-ms,--https_port <port> Use port for https monitoring
-c, --config <file> Configuration file
-t Test configuration and exit
-sl,--signal <signal>[=<pid>] Send signal to nats-server process (ldm, stop, quit, term, reopen, reload)
<pid> can be either a PID (e.g. 1) or the path to a PID file (e.g. /var/run/nats-server.pid)
--client_advertise <string> Client URL to advertise to other servers
--ports_file_dir <dir> Creates a ports file in the specified directory (<executable_name>_<pid>.ports).
Logging Options:
-l, --log <file> File to redirect log output
-T, --logtime Timestamp log entries (default: true)
-s, --syslog Log to syslog or windows event log
-r, --remote_syslog <addr> Syslog server addr (udp://localhost:514)
-D, --debug Enable debugging output
-V, --trace Trace the raw protocol
-VV Verbose trace (traces system account as well)
-DV Debug and trace
-DVV Debug and verbose trace (traces system account as well)
--log_size_limit <limit> Logfile size limit (default: auto)
--max_traced_msg_len <len> Maximum printable length for traced messages (default: unlimited)
JetStream Options:
-js, --jetstream Enable JetStream functionality
-sd, --store_dir <dir> Set the storage directory
Authorization Options:
--user <user> User required for connections
--pass <password> Password required for connections
--auth <token> Authorization token required for connections
TLS Options:
--tls Enable TLS, do not verify clients (default: false)
--tlscert <file> Server certificate file
--tlskey <file> Private key for server certificate
--tlsverify Enable TLS, verify client certificates
--tlscacert <file> Client certificate CA for verification
Cluster Options:
--routes <rurl-1, rurl-2> Routes to solicit and connect
--cluster <cluster-url> Cluster URL for solicited routes
--cluster_name <string> Cluster Name, if not set one will be dynamically generated
--no_advertise <bool> Do not advertise known cluster information to clients
--cluster_advertise <string> Cluster URL to advertise to other servers
--connect_retries <number> For implicit routes, number of connect retries
--cluster_listen <url> Cluster url from which members can solicit routes
Profiling Options:
--profile <port> Profiling HTTP port
Common Options:
-h, --help Show this message
-v, --version Show version
--help_tls TLS help
session.out.txt
---
status: verified
capturedAt: "2026-08-23T15:19:49.324Z"
dockerImage: "nats:2.11.5@sha256:47710a58674c152355730c69fc89d5a86369e3d6d1329cc91767650ce8e0dde6"
exitCode: 0
---
## status
[cli-tools] $ curl -fsS http://127.0.0.1:8222/healthz
{"status":"ok"}
## gap
# 缺口记录:nats 官方镜像不含 nats CLI(由 nats-io/natscli 独立发行),distroless 镜像连 shell 都没有,
# 纯镜像自带命令无法收发消息;收发需外部客户端(Java 实验见 demos/nats/core-pubsub)。
[cli-tools] $ compose exec nats nats --version # 尝试在容器内寻找自带 CLI,预期失败
OCI runtime exec failed: exec failed: unable to start container process: exec: "nats": executable file not found in $PATH
exit=127(预期失败:镜像不含 nats 二进制)
## varz
[cli-tools] $ curl -fsS http://127.0.0.1:8222/varz
{
"server_id": "NBWDRAPRGUPMWTMYXIZKNZRK26AKOZNPIHAIFUMH5ZSY5DN2ZOKGD45T",
"server_name": "NBWDRAPRGUPMWTMYXIZKNZRK26AKOZNPIHAIFUMH5ZSY5DN2ZOKGD45T",
"version": "2.11.5",
"proto": 1,
"git_commit": "94fb8b0",
"go": "go1.24.4",
"host": "0.0.0.0",
"port": 4222,
"max_connections": 65536,
"ping_interval": 120000000000,
"ping_max": 2,
"http_host": "0.0.0.0",
"http_port": 8222,
"http_base_path": "",
"https_port": 0,
"auth_timeout": 2,
"max_control_line": 4096,
"max_payload": 1048576,
"max_pending": 67108864,
"cluster": {},
"gateway": {},
"leaf": {},
"mqtt": {},
"websocket": {},
"jetstream": {
"config": {
"max_memory": 12574804992,
"max_storage": 68064571392,
"store_dir": "/tmp/nats/jetstream",
"sync_interval": 120000000000
},
"stats": {
"memory": 0,
"storage": 0,
"reserved_memory": 0,
"reserved_storage": 0,
"accounts": 1,
"ha_assets": 0,
"api": {
"level": 1,
"total": 0,
"errors": 0
}
},
"limits": {}
},
"tls_timeout": 2,
"write_deadline": 10000000000,
"start": "2026-08-23T15:17:21.243592603Z",
"now": "2026-08-23T15:17:21.997112291Z",
"uptime": "0s",
"mem": 15831040,
"cores": 4,
"gomaxprocs": 4,
"cpu": 3,
"connections": 0,
"total_connections": 0,
"routes": 0,
"remotes": 0,
"leafnodes": 0,
"in_msgs": 0,
"out_msgs": 0,
"in_bytes": 0,
"out_bytes": 0,
"slow_consumers": 0,
"subscriptions": 63,
"http_req_stats": {
"/healthz": 2,
"/varz": 1
},
"config_load_time": "2026-08-23T15:17:21.243592603Z",
"config_digest": "",
"system_account": "$SYS",
"slow_consumer_stats": {
"clients": 0,
"routes": 0,
"gateways": 0,
"leafs": 0
}
}
## connz
[cli-tools] $ curl -fsS http://127.0.0.1:8222/connz
{
"server_id": "NBWDRAPRGUPMWTMYXIZKNZRK26AKOZNPIHAIFUMH5ZSY5DN2ZOKGD45T",
"now": "2026-08-23T15:17:22.004747792Z",
"num_connections": 0,
"total": 0,
"offset": 0,
"limit": 1024,
"connections": []
}
## subsz
[cli-tools] $ curl -fsS http://127.0.0.1:8222/subsz
{
"num_subscriptions": 63,
"num_cache": 5,
"num_inserts": 63,
"num_removes": 0,
"num_matches": 6,
"cache_hit_rate": 0.16666666666666666,
"max_fanout": 1,
"avg_fanout": 0.8
}
status.out.txt
[cli-tools] $ curl -fsS http://127.0.0.1:8222/healthz
{"status":"ok"}
subsz.out.txt
[cli-tools] $ curl -fsS http://127.0.0.1:8222/subsz
{
"num_subscriptions": 63,
"num_cache": 5,
"num_inserts": 63,
"num_removes": 0,
"num_matches": 6,
"cache_hit_rate": 0.16666666666666666,
"max_fanout": 1,
"avg_fanout": 0.8
}
varz.out.txt
[cli-tools] $ curl -fsS http://127.0.0.1:8222/varz
{
"server_id": "NBWDRAPRGUPMWTMYXIZKNZRK26AKOZNPIHAIFUMH5ZSY5DN2ZOKGD45T",
"server_name": "NBWDRAPRGUPMWTMYXIZKNZRK26AKOZNPIHAIFUMH5ZSY5DN2ZOKGD45T",
"version": "2.11.5",
"proto": 1,
"git_commit": "94fb8b0",
"go": "go1.24.4",
"host": "0.0.0.0",
"port": 4222,
"max_connections": 65536,
"ping_interval": 120000000000,
"ping_max": 2,
"http_host": "0.0.0.0",
"http_port": 8222,
"http_base_path": "",
"https_port": 0,
"auth_timeout": 2,
"max_control_line": 4096,
"max_payload": 1048576,
"max_pending": 67108864,
"cluster": {},
"gateway": {},
"leaf": {},
"mqtt": {},
"websocket": {},
"jetstream": {
"config": {
"max_memory": 12574804992,
"max_storage": 68064571392,
"store_dir": "/tmp/nats/jetstream",
"sync_interval": 120000000000
},
"stats": {
"memory": 0,
"storage": 0,
"reserved_memory": 0,
"reserved_storage": 0,
"accounts": 1,
"ha_assets": 0,
"api": {
"level": 1,
"total": 0,
"errors": 0
}
},
"limits": {}
},
"tls_timeout": 2,
"write_deadline": 10000000000,
"start": "2026-08-23T15:17:21.243592603Z",
"now": "2026-08-23T15:17:21.997112291Z",
"uptime": "0s",
"mem": 15831040,
"cores": 4,
"gomaxprocs": 4,
"cpu": 3,
"connections": 0,
"total_connections": 0,
"routes": 0,
"remotes": 0,
"leafnodes": 0,
"in_msgs": 0,
"out_msgs": 0,
"in_bytes": 0,
"out_bytes": 0,
"slow_consumers": 0,
"subscriptions": 63,
"http_req_stats": {
"/healthz": 2,
"/varz": 1
},
"config_load_time": "2026-08-23T15:17:21.243592603Z",
"config_digest": "",
"system_account": "$SYS",
"slow_consumer_stats": {
"clients": 0,
"routes": 0,
"gateways": 0,
"leafs": 0
}
}