shishengliang/lingniu-vehicle-ingest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: align runbooks with tdengine raw frames

Browse Source
This commit is contained in:
lingniu
2026-07-01 06:11:10 +08:00
parent 56149c884e
commit f91fa21463
3 changed files with 33 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
docs/operations/gb32960-service-split-runbook.md
View File

@@ -8,17 +8,17 @@ This runbook covers the split GB32960 ingest, history, and analytics runtimes. K
| --- | --- | --- | --- |
| GB32960 ingest | `:gb32960-ingest-app` | Accept GB32960 TCP connections, decode/authenticate frames, produce raw and normalized records to Kafka, and send protocol ACKs after required Kafka production succeeds. | TCP `32960`, HTTP `20100` |
| JT808 ingest | `:jt808-ingest-app` | Accept JT/T 808 TCP connections on the production receive port, parse raw frames, archive frame bytes, and produce raw/event envelopes to Kafka. | TCP `808`, HTTP `20482` |
| Vehicle history | `:vehicle-history-app` | Consume GB32960 raw/event Kafka records, store event-history records and raw archive references, and expose event/history/GB32960 query APIs. Kafka raw envelopes carry archive URI/size metadata, not raw frame bytes. | HTTP `20200` |
| Vehicle history | `:vehicle-history-app` | Consume GB32960/JT808 raw/event Kafka records, store compact history rows plus TDengine `raw_frames` `parsedJson`/`metadataJson`/`rawUri`, and expose TDengine-backed history APIs. | HTTP `20200` |
| Vehicle analytics | `:vehicle-analytics-app` | Consume GB32960 event Kafka records and update analytics outputs such as daily vehicle statistics. Vehicle state can also be enabled from this runtime. | HTTP `20300` |
## Kafka Contract
| Topic | Producer | Consumers | Purpose |
| --- | --- | --- | --- |
| `vehicle.raw.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app` | Raw archive reference records. The envelope contains URI/size metadata, not the raw `.bin` payload. |
| `vehicle.raw.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app` | Raw frame records. History writes the parsed JSON, metadata JSON, and raw URI to TDengine `raw_frames`. |
| `vehicle.event.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app`, `vehicle-analytics-app` | Normalized vehicle events keyed by VIN when available. |
| `vehicle.dlq.gb32960.v1` | Kafka producer/consumer error paths | Operators/replay tooling | Dead-letter records for failed production or consumer processing. |
| `vehicle.raw.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app` | JT808 raw archive reference records. |
| `vehicle.raw.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app` | JT808 raw frame records for TDengine `raw_frames` and raw-frame troubleshooting APIs. |
| `vehicle.event.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app` | JT808 parsed location/session/alarm events keyed by phone or mapped VIN. |
| `vehicle.dlq.jt808.v1` | Kafka producer/consumer error paths | Operators/replay tooling | Dead-letter records for failed JT808 production or consumer processing. |
@@ -121,7 +121,7 @@ java --sun-misc-unsafe-memory-access=allow \
-jar modules/apps/vehicle-history-app/target/vehicle-history-app.jar
```
`SINK_ARCHIVE_PATH` is intentionally omitted in the split-service Kafka verification path. Current Kafka raw envelopes contain `RawArchiveRef` URI/size metadata only; they do not contain raw bytes that `RawArchiveEventSink` could materialize into `.bin` files in the history JVM.
`SINK_ARCHIVE_PATH` is intentionally omitted from the history JVM. The history hot path is Kafka to TDengine: raw envelopes are stored in TDengine `raw_frames` with `parsedJson`, `metadataJson`, and `rawUri`. Ingest services may still write original `.bin` files as cold backup, but history API queries must not depend on a shared local archive mount.
Vehicle analytics:
@@ -203,9 +203,9 @@ Use the templates under `deploy/local/launchctl/` for the local production-verif
- JT808 ingest: `com.lingniu.jt808`, TCP `808`, HTTP `20482`
- History: `com.lingniu.vehicle-history`, HTTP `20200`
The two ingest services and history service must share the same `SINK_ARCHIVE_PATH`. Kafka raw messages contain `archive://...` references, so history can only read raw frames when its archive root points to the same directory or shared volume that ingest used to write the `.bin` files.
The launchctl history template does not inject `SINK_ARCHIVE_PATH`. Use TDengine `raw_frames` as the authoritative hot-query store; keep any ingest-side raw archive path as an optional cold backup for operator inspection.
For Portainer deployments, mirror this same rule with a shared volume mounted to all three services, for example `/archive` in `deploy/portainer/docker-compose.yml`.
For Portainer deployments, mirror this same shape: no history archive volume, history consumes raw/event Kafka topics and writes TDengine history rows.
## Rollback Guidance
@@ -268,4 +268,4 @@ Observed on 2026-06-29 in worktree `.worktrees/vehicle-ingest-redesign-phase1`:
- A GB32960 synthetic realtime frame sent to TCP `32960` received a binary ACK beginning with `2323`. The same VIN `LTEST202606290001` was queryable from TDengine-backed history APIs with speed, mileage, longitude, and latitude fields.
- History raw and event Kafka consumption used separate bindings so raw-frame writes remained current while location/field event ingestion continued.
This verifies the local path `GB32960/JT808 TCP -> protocol app -> shared raw archive -> Kafka raw/event topics -> vehicle-history-app -> TDengine -> Swagger/API query`.
This verifies the local path `GB32960/JT808 TCP -> protocol app -> Kafka raw/event topics -> vehicle-history-app -> TDengine raw_frames/location tables -> Swagger/API query`.