# Go Service Observability

## ECS

Host: `115.29.187.205`

The Go services expose local-only health and metrics endpoints. They are intended for ECS-local checks, Prometheus scraping through an agent, or SSH troubleshooting. They are not public business APIs.

## Endpoints

| Service | systemd unit | Address | Health | Readiness | Metrics |
| --- | --- | --- | --- | --- | --- |
| Gateway | `lingniu-go-gateway.service` | `127.0.0.1:20211` | `/healthz` | `/readyz` | `/metrics` |
| History writer | `lingniu-go-history-writer.service` | `127.0.0.1:20212` | `/healthz` | `/readyz` | `/metrics` |
| Stat writer | `lingniu-go-stat-writer.service` | `127.0.0.1:20213` | `/healthz` | `/readyz` | `/metrics` |
| NATS Kafka bridge | `lingniu-go-nats-kafka-bridge.service` | `127.0.0.1:20214` | `/healthz` | `/readyz` | `/metrics` |
| Realtime API | `lingniu-go-realtime-api.service` | `0.0.0.0:20200` | `/healthz` | `/readyz` | `/metrics` |

## Quick Checks

生产事故处理顺序和阈值解释见 [车辆接入生产运行手册](vehicle-ingest-runbook.md)。

```bash
curl -fsS http://127.0.0.1:20211/readyz
curl -fsS http://127.0.0.1:20211/metrics
curl -fsS http://127.0.0.1:20212/readyz
curl -fsS http://127.0.0.1:20213/readyz
curl -fsS http://127.0.0.1:20214/readyz
curl -fsS http://127.0.0.1:20200/readyz
```

## Core Counters

| Metric | Meaning |
| --- | --- |
| `vehicle_gateway_active_connections` | Current TCP connections by protocol. Labels: `protocol`. |
| `vehicle_gateway_frames_total` | Protocol frames received and parsed by the gateway. Labels: `protocol`, `status`. |
| `vehicle_gateway_frame_duration_ms` | Last observed Gateway frame handling duration and histogram buckets for p95/p99. Labels: `protocol`, `status`. |
| `vehicle_gateway_publish_total` | Gateway publish result for raw events. Unified appears only when the compatibility stream is explicitly enabled. Labels: `protocol`, `kind`, `status`. |
| `vehicle_bridge_messages_total` | NATS messages fetched by the bridge. Labels: `subject`, `status`. |
| `vehicle_bridge_kafka_writes_total` | Bridge writes to Kafka. Labels: `topic`, `status`. |
| `vehicle_bridge_nats_acks_total` | NATS ack results after Kafka write. Labels: `subject`, `status`. |
| `vehicle_bridge_nats_consumer_pending` | JetStream messages pending for the bridge durable consumer. |
| `vehicle_bridge_nats_consumer_ack_pending` | JetStream messages delivered to bridge but not yet acked. |
| `vehicle_bridge_nats_consumer_waiting` | Pull requests waiting on the bridge durable consumer. |
| `vehicle_bridge_batch_pending_messages` | NATS messages fetched by the bridge but not yet written to Kafka and acked. |
| `vehicle_bridge_batch_pending_kafka_messages` | Kafka messages prepared for the current bridge batch. |
| `vehicle_bridge_batch_duration_ms_histogram` | Bridge batch duration histogram covering Kafka write and NATS ack. Labels: `status`. |
| `vehicle_history_writes_total` | TDengine history writes. Labels: `topic`, `status`. |
| `vehicle_history_batch_pending_messages` | Messages fetched by history writer but not yet flushed and committed. |
| `vehicle_history_batch_pending_rows` | Parsed raw envelopes waiting in the current TDengine batch. |
| `vehicle_history_batch_flush_duration_ms` | Last TDengine batch flush duration. Labels: `status`. |
| `vehicle_stat_writes_total` | MySQL metric writes. Labels: `topic`, `status`. |
| `vehicle_realtime_updates_total` | Redis/MySQL realtime projector updates. Labels: `topic`, `status`. |
| `vehicle_history_kafka_lag` | Estimated Kafka lag for history writer by topic and partition. |
| `vehicle_stat_kafka_lag` | Estimated Kafka lag for stat writer by topic and partition. |
| `vehicle_realtime_kafka_lag` | Estimated Kafka lag for realtime projector by topic and partition. |

## First Principle

These metrics are operational telemetry. They should stay out of MySQL and TDengine business tables unless a product query explicitly requires persisted aggregates. Persisted aggregate metrics should be derived from Kafka replay rather than by widening realtime or raw tables.