4.0 KiB
4.0 KiB
Go Service Observability
ECS
Host: 115.29.187.205
The Go services expose local-only health and metrics endpoints. They are intended for ECS-local checks, Prometheus scraping through an agent, or SSH troubleshooting. They are not public business APIs.
Endpoints
| Service | systemd unit | Address | Health | Readiness | Metrics |
|---|---|---|---|---|---|
| Gateway | lingniu-go-gateway.service |
127.0.0.1:20211 |
/healthz |
/readyz |
/metrics |
| History writer | lingniu-go-history-writer.service |
127.0.0.1:20212 |
/healthz |
/readyz |
/metrics |
| Stat writer | lingniu-go-stat-writer.service |
127.0.0.1:20213 |
/healthz |
/readyz |
/metrics |
| NATS Kafka bridge | lingniu-go-nats-kafka-bridge.service |
127.0.0.1:20214 |
/healthz |
/readyz |
/metrics |
| Realtime API | lingniu-go-realtime-api.service |
0.0.0.0:20200 |
/healthz |
/readyz |
/metrics |
Quick Checks
生产事故处理顺序和阈值解释见 车辆接入生产运行手册。
curl -fsS http://127.0.0.1:20211/readyz
curl -fsS http://127.0.0.1:20211/metrics
curl -fsS http://127.0.0.1:20212/readyz
curl -fsS http://127.0.0.1:20213/readyz
curl -fsS http://127.0.0.1:20214/readyz
curl -fsS http://127.0.0.1:20200/readyz
Core Counters
| Metric | Meaning |
|---|---|
vehicle_gateway_active_connections |
Current TCP connections by protocol. Labels: protocol. |
vehicle_gateway_frames_total |
Protocol frames received and parsed by the gateway. Labels: protocol, status. |
vehicle_gateway_frame_duration_ms |
Last observed Gateway frame handling duration and histogram buckets for p95/p99. Labels: protocol, status. |
vehicle_gateway_publish_total |
Gateway publish result for raw events. Unified appears only when the compatibility stream is explicitly enabled. Labels: protocol, kind, status. |
vehicle_bridge_messages_total |
NATS messages fetched by the bridge. Labels: subject, status. |
vehicle_bridge_kafka_writes_total |
Bridge writes to Kafka. Labels: topic, status. |
vehicle_bridge_nats_acks_total |
NATS ack results after Kafka write. Labels: subject, status. |
vehicle_bridge_nats_consumer_pending |
JetStream messages pending for the bridge durable consumer. |
vehicle_bridge_nats_consumer_ack_pending |
JetStream messages delivered to bridge but not yet acked. |
vehicle_bridge_nats_consumer_waiting |
Pull requests waiting on the bridge durable consumer. |
vehicle_bridge_batch_pending_messages |
NATS messages fetched by the bridge but not yet written to Kafka and acked. |
vehicle_bridge_batch_pending_kafka_messages |
Kafka messages prepared for the current bridge batch. |
vehicle_bridge_batch_duration_ms_histogram |
Bridge batch duration histogram covering Kafka write and NATS ack. Labels: status. |
vehicle_history_writes_total |
TDengine history writes. Labels: topic, status. |
vehicle_history_batch_pending_messages |
Messages fetched by history writer but not yet flushed and committed. |
vehicle_history_batch_pending_rows |
Parsed raw envelopes waiting in the current TDengine batch. |
vehicle_history_batch_flush_duration_ms |
Last TDengine batch flush duration. Labels: status. |
vehicle_stat_writes_total |
MySQL metric writes. Labels: topic, status. |
vehicle_realtime_updates_total |
Redis/MySQL realtime projector updates. Labels: topic, status. |
vehicle_history_kafka_lag |
Estimated Kafka lag for history writer by topic and partition. |
vehicle_stat_kafka_lag |
Estimated Kafka lag for stat writer by topic and partition. |
vehicle_realtime_kafka_lag |
Estimated Kafka lag for realtime projector by topic and partition. |
First Principle
These metrics are operational telemetry. They should stay out of MySQL and TDengine business tables unless a product query explicitly requires persisted aggregates. Persisted aggregate metrics should be derived from Kafka replay rather than by widening realtime or raw tables.