docs: add tdengine verification runbook
This commit is contained in:
lingniu
166
docs/operations/vehicle-ingest-tdengine-verification.md
Normal file
166
docs/operations/vehicle-ingest-tdengine-verification.md
Normal file
@@ -0,0 +1,166 @@
|
||||
# Vehicle Ingest TDengine 链路验收手册
|
||||
|
||||
这份手册用于验收当前 32960/JT808 接入重构链路：
|
||||
|
||||
1. 协议 App 接收 TCP 报文。
|
||||
2. 协议 App 归档原始帧元数据，并把事件信封发布到 Kafka。
|
||||
3. `vehicle-history-app` 消费 Kafka。
|
||||
4. history 写入 raw、location、telemetry field 到 TDengine。
|
||||
5. Swagger/API 通过 TDengine 做历史分页查询。
|
||||
|
||||
每一步都必须执行命令并检查输出后，才能标记为已验证。
|
||||
|
||||
## 必需运行参数
|
||||
|
||||
启动服务前先设置：
|
||||
|
||||
```bash
|
||||
export KAFKA_BROKERS=114.55.58.251:9092
|
||||
export TDENGINE_HISTORY_ENABLED=true
|
||||
export TDENGINE_JDBC_URL='jdbc:TAOS-RS://<tdengine-host>:6041/vehicle_history'
|
||||
export TDENGINE_USERNAME=root
|
||||
export TDENGINE_PASSWORD='<tdengine-password>'
|
||||
export TDENGINE_MIN_IDLE=0
|
||||
export TDENGINE_MAX_POOL_SIZE=32
|
||||
```
|
||||
|
||||
`TDENGINE_MIN_IDLE=0` 用于减少冷启动时 TDengine 连接重试日志。TDengine 地址稳定后，再按生产并发调大连接池。
|
||||
|
||||
## 构建
|
||||
|
||||
```bash
|
||||
mvn -pl modules/apps/jt808-ingest-app,modules/apps/gb32960-ingest-app,modules/apps/vehicle-history-app -am package -DskipTests
|
||||
```
|
||||
|
||||
预期产物：
|
||||
|
||||
- `modules/apps/jt808-ingest-app/target/jt808-ingest-app.jar`
|
||||
- `modules/apps/gb32960-ingest-app/target/gb32960-ingest-app.jar`
|
||||
- `modules/apps/vehicle-history-app/target/vehicle-history-app.jar`
|
||||
|
||||
## 启动 JT808 接入服务，监听 808 端口
|
||||
|
||||
```bash
|
||||
HTTP_PORT=20400 \
|
||||
JT808_PORT=808 \
|
||||
KAFKA_CONSUMER_ENABLED=false \
|
||||
java -jar modules/apps/jt808-ingest-app/target/jt808-ingest-app.jar
|
||||
```
|
||||
|
||||
验证：
|
||||
|
||||
```bash
|
||||
curl -sS http://127.0.0.1:20400/actuator/health
|
||||
lsof -nP -iTCP:808 -sTCP:LISTEN
|
||||
```
|
||||
|
||||
预期：健康检查为 `UP`，并且 Java 进程监听 TCP `808`。
|
||||
|
||||
## 启动 GB32960 接入服务
|
||||
|
||||
```bash
|
||||
HTTP_PORT=20100 \
|
||||
GB32960_PORT=32960 \
|
||||
KAFKA_CONSUMER_ENABLED=false \
|
||||
java -jar modules/apps/gb32960-ingest-app/target/gb32960-ingest-app.jar
|
||||
```
|
||||
|
||||
验证：
|
||||
|
||||
```bash
|
||||
curl -sS http://127.0.0.1:20100/actuator/health
|
||||
lsof -nP -iTCP:32960 -sTCP:LISTEN
|
||||
```
|
||||
|
||||
## 启动 TDengine 版历史服务
|
||||
|
||||
```bash
|
||||
HTTP_PORT=20200 \
|
||||
KAFKA_CONSUMER_ENABLED=true \
|
||||
EVENT_FILE_STORE_ENABLED=true \
|
||||
EVENT_FILE_STORE_PATH=./target/tdengine-verification/event-store \
|
||||
SINK_ARCHIVE_PATH=./target/tdengine-verification/archive \
|
||||
java -jar modules/apps/vehicle-history-app/target/vehicle-history-app.jar
|
||||
```
|
||||
|
||||
验证：
|
||||
|
||||
```bash
|
||||
curl -sS http://127.0.0.1:20200/actuator/health
|
||||
curl -sS http://127.0.0.1:20200/v3/api-docs \
|
||||
| grep -E '/api/event-history/jt808/locations|/api/event-history/telemetry/fields'
|
||||
```
|
||||
|
||||
预期：健康检查为 `UP`，OpenAPI 中包含这两个 TDengine 查询接口。
|
||||
|
||||
## Kafka Topic 验证
|
||||
|
||||
历史服务必须消费事件 topic 和 raw topic：
|
||||
|
||||
- `vehicle.event.gb32960.v1`
|
||||
- `vehicle.raw.gb32960.v1`
|
||||
- `vehicle.event.jt808.v1`
|
||||
- `vehicle.raw.jt808.v1`
|
||||
|
||||
使用 Kafka CLI 或管理工具检查 topic 是否存在，并确认 `vehicle-history` consumer group 在测试流量后没有持续堆积。
|
||||
|
||||
## JT808 实时转发验收
|
||||
|
||||
如果外部平台已经把 JT808 报文转发到本机 TCP `808`，至少观察 60 秒日志和 Kafka 数据。
|
||||
|
||||
必须拿到以下证据：
|
||||
|
||||
- `jt808-ingest-app` 日志显示接入连接或解析到上游消息 ID。
|
||||
- Kafka 的 `vehicle.event.jt808.v1` 和 `vehicle.raw.jt808.v1` 有新增记录。
|
||||
- `vehicle-history-app` 日志没有持续出现 consumer 或 TDengine 写入失败。
|
||||
|
||||
消费完成后，使用已知终端手机号查询：
|
||||
|
||||
```bash
|
||||
curl -sS 'http://127.0.0.1:20200/api/event-history/jt808/locations?phone=<phone>&dateFrom=2026-06-29T00:00:00%2B08:00&dateTo=2026-06-29T23:59:59%2B08:00&limit=10'
|
||||
```
|
||||
|
||||
预期：响应包含 `items`，并且每条记录包含 `eventTime`、`phone`、`longitude`、`latitude`、`speedKmh`、`rawUri`、`metadataJson`。
|
||||
|
||||
## Telemetry Field 查询验收
|
||||
|
||||
查询 GB32960 字段历史：
|
||||
|
||||
```bash
|
||||
curl -sS 'http://127.0.0.1:20200/api/event-history/telemetry/fields?protocol=GB32960&vin=<vin>&fieldKey=<field-key>&dateFrom=2026-06-29T00:00:00%2B08:00&dateTo=2026-06-29T23:59:59%2B08:00&limit=10'
|
||||
```
|
||||
|
||||
查询 JT808 字段历史。没有 VIN 映射时，优先使用 `phone`：
|
||||
|
||||
```bash
|
||||
curl -sS 'http://127.0.0.1:20200/api/event-history/telemetry/fields?protocol=JT808&phone=<phone>&fieldKey=location.speed_kmh&dateFrom=2026-06-29T00:00:00%2B08:00&dateTo=2026-06-29T23:59:59%2B08:00&limit=10'
|
||||
```
|
||||
|
||||
预期：响应包含 `items`，可能包含 `nextCursor`。下一页使用 `nextCursor` 里的 `cursorTs` 和 `cursorId` 查询。
|
||||
|
||||
## TDengine 直接检查
|
||||
|
||||
使用 TDengine CLI 或 JDBC 客户端检查超级表和子表是否创建：
|
||||
|
||||
```sql
|
||||
USE vehicle_history;
|
||||
SHOW STABLES;
|
||||
SELECT COUNT(*) FROM raw_frames;
|
||||
SELECT COUNT(*) FROM vehicle_locations;
|
||||
SELECT COUNT(*) FROM telemetry_fields;
|
||||
```
|
||||
|
||||
预期：
|
||||
|
||||
- `raw_frames`、`vehicle_locations`、`telemetry_fields` 存在。
|
||||
- 有实时流量后，计数持续增加。
|
||||
|
||||
## 失败语义
|
||||
|
||||
- TDengine 不可达时，TDengine 查询接口应该返回 HTTP `503`，消息类似 `tdengine history query failed`。
|
||||
- TDengine 未启用时，TDengine 专属接口不应该出现在 OpenAPI 中。
|
||||
- 路径不存在时，API 层应该返回 HTTP `404`，而不是泛化的存储失败。
|
||||
|
||||
## 当前本地缺口
|
||||
|
||||
本次实现验证环境中，Kafka `114.55.58.251:9092` 可连通，但本机 `127.0.0.1:6041` 不可连通。完整 TDengine 写入和查询验收需要提供真实可用的 `TDENGINE_JDBC_URL`。
|
||||
Reference in New Issue
Block a user