feat: build vehicle data platform and production pipeline

This commit is contained in:
lingniu
2026-07-14 12:35:33 +08:00
parent b452be3b94
commit bb59303a4b
270 changed files with 88016 additions and 1975 deletions

View File

@@ -58,8 +58,8 @@ Go 服务使用 systemd 裸机部署,不使用 Docker。
最近 release
```text
/opt/lingniu-go-native/releases/production-track-telemetry-20260714082451
/opt/lingniu-go-native/releases/100k-hardening-20260703175730
/opt/lingniu-go-native/releases/parsed-fields-once-20260703163821
```
服务重启:
@@ -511,14 +511,28 @@ vehicle_gateway_frame_duration_ms_histogram_sum{protocol,status}
该耗时覆盖单帧从解析、identity resolve、parsed fields 生成、publish enqueue 到协议响应写入的整体入口处理路径。带帧率压测时用它按协议观察 p95/p99不能只看 `vehicle_gateway_frame_duration_ms` 最后一条 gauge。
### Gateway identity resolve duration
2026-07-11 已为 Gateway 身份解析单独增加耗时 histogram 和超时分类:
```text
vehicle_gateway_identity_total{protocol,status}
vehicle_gateway_identity_duration_ms_histogram_bucket{protocol,status,le}
vehicle_gateway_identity_duration_ms_histogram_count{protocol,status}
vehicle_gateway_identity_duration_ms_histogram_sum{protocol,status}
```
生产默认使用周期身份快照,每帧只查本机内存;`status="timeout"` 主要用于兼容关闭 `IDENTITY_SNAPSHOT_ONLY_ENABLED` 后的回源查询模式。快照模式应重点观察 `vehicle_gateway_identity_snapshot_ready` 和最近成功刷新时间RDS 故障时入口继续接收,上一版快照继续服务,未知 phone 暂时 unresolved。
### History writer TDengine 批写
2026-07-03 已实现 history-writer 第一阶段批写:
- `history.Writer.AppendAllBatch` 按 TDengine child table 聚合 raw/location 多行 `INSERT`
- `cmd/history-writer` 默认 `HISTORY_BATCH_SIZE=200``HISTORY_BATCH_WAIT_MS=100`
- `cmd/history-writer` 默认 `HISTORY_WORKERS=3``HISTORY_BATCH_SIZE=200``HISTORY_BATCH_WAIT_MS=20`
- TDengine batch 成功后才批量提交 Kafka messages失败不提交 offset。
- 保留单条 `AppendAll` 路径作为回退。
- `production-track-telemetry-20260714082451` 起,`vehicle_locations` stable 增加 `soc_percent`。history-writer 在建表后执行幂等 `ALTER STABLE ... ADD COLUMN soc_percent DOUBLE`,只忽略 TDengine 明确的重复列错误;其他迁移错误会阻止 ready禁止带病继续写入。单条和批量位置 INSERT 都必须携带 SOC。
新增指标: