shishengliang/lingniu-vehicle-ingest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: document local ingest deployment verification
All checks were successful
ci/woodpecker/push/woodpecker Pipeline was successful
Details

Browse Source
This commit is contained in:
lingniu
2026-06-29 16:18:52 +08:00
parent d2bc4a0d0a
commit ef9ac327fe
6 changed files with 391 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

29
docs/operations/gb32960-service-split-runbook.md
View File

@@ -7,6 +7,7 @@ This runbook covers the split GB32960 ingest, history, and analytics runtimes. K
| Service | Module | Role | Local ports |
| --- | --- | --- | --- |
| GB32960 ingest | `:gb32960-ingest-app` | Accept GB32960 TCP connections, decode/authenticate frames, produce raw and normalized records to Kafka, and send protocol ACKs after required Kafka production succeeds. | TCP `32960`, HTTP `20100` |
| JT808 ingest | `:jt808-ingest-app` | Accept JT/T 808 TCP connections on the production receive port, parse raw frames, archive frame bytes, and produce raw/event envelopes to Kafka. | TCP `808`, HTTP `20482` |
| Vehicle history | `:vehicle-history-app` | Consume GB32960 raw/event Kafka records, store event-history records and raw archive references, and expose event/history/GB32960 query APIs. Kafka raw envelopes carry archive URI/size metadata, not raw frame bytes. | HTTP `20200` |
| Vehicle analytics | `:vehicle-analytics-app` | Consume GB32960 event Kafka records and update analytics outputs such as daily vehicle statistics. Vehicle state can also be enabled from this runtime. | HTTP `20300` |
@@ -17,6 +18,9 @@ This runbook covers the split GB32960 ingest, history, and analytics runtimes. K
| `vehicle.raw.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app` | Raw archive reference records. The envelope contains URI/size metadata, not the raw `.bin` payload. |
| `vehicle.event.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app`, `vehicle-analytics-app` | Normalized vehicle events keyed by VIN when available. |
| `vehicle.dlq.gb32960.v1` | Kafka producer/consumer error paths | Operators/replay tooling | Dead-letter records for failed production or consumer processing. |
| `vehicle.raw.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app` | JT808 raw archive reference records. |
| `vehicle.event.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app` | JT808 parsed location/session/alarm events keyed by phone or mapped VIN. |
| `vehicle.dlq.jt808.v1` | Kafka producer/consumer error paths | Operators/replay tooling | Dead-letter records for failed JT808 production or consumer processing. |
Default local consumer groups:
@@ -173,6 +177,18 @@ Do not expect `vehicle-history-app` to create raw `.bin` archive files from Kafk
Do not mark any of these as verified unless the matching command was run and the output was inspected.
## Local launchctl Deployment
Use the templates under `deploy/local/launchctl/` for the local production-verification machine. They codify the current three-service deployment and keep the runtime knobs in one place:
- GB32960 ingest: `com.lingniu.gb32960`, TCP `32960`, HTTP `20100`
- JT808 ingest: `com.lingniu.jt808`, TCP `808`, HTTP `20482`
- History: `com.lingniu.vehicle-history`, HTTP `20200`
The two ingest services and history service must share the same `SINK_ARCHIVE_PATH`. Kafka raw messages contain `archive://...` references, so history can only read raw frames when its archive root points to the same directory or shared volume that ingest used to write the `.bin` files.
For Portainer deployments, mirror this same rule with a shared volume mounted to all three services, for example `/archive` in `deploy/portainer/docker-compose.yml`.
## Rollback Guidance
If the split deployment is unhealthy:
@@ -222,3 +238,16 @@ Observed on 2026-06-23 with Kafka `114.55.58.251:9092`:
- Runtime logs showed successful GB32960 ACK flushes for platform login, vehicle login, and vehicle logout after the Kafka-backed split ingest service was running.
This verifies the live path `GB32960 TCP 32960 -> gb32960-ingest-app -> Kafka raw/event topics` against the remote Kafka broker. Downstream `vehicle-history-app` and `vehicle-analytics-app` were not started in this pass.
## 2026-06-29 Local Production-Path Verification
Observed on 2026-06-29 in worktree `.worktrees/vehicle-ingest-redesign-phase1`:
- `jt808-ingest-app` was running through launchctl on TCP `808` and HTTP `20482`; `curl -sS http://127.0.0.1:20482/actuator/health` returned top-level status `UP`.
- `gb32960-ingest-app` was running through launchctl on TCP `32960` and HTTP `20100`; `curl -sS http://127.0.0.1:20100/actuator/health` returned top-level status `UP`.
- `vehicle-history-app` was running through launchctl on HTTP `20200`; `curl -sS http://127.0.0.1:20200/actuator/health` returned top-level status `UP`.
- External JT808 traffic on TCP `808` was parsed and written through Kafka into TDengine. Direct TDengine checks showed more than `6000` JT808 `raw_frames`, more than `1900` JT808 `vehicle_locations`, and more than `15000` JT808 `telemetry_fields`.
- A GB32960 synthetic realtime frame sent to TCP `32960` received a binary ACK beginning with `2323`. The same VIN `LTEST202606290001` was queryable from TDengine-backed history APIs with speed, mileage, longitude, and latitude fields.
- History raw and event Kafka consumption used separate bindings so raw-frame writes remained current while location/field event ingestion continued.
This verifies the local path `GB32960/JT808 TCP -> protocol app -> shared raw archive -> Kafka raw/event topics -> vehicle-history-app -> TDengine -> Swagger/API query`.