docs: align history and analytics production surface
This commit is contained in:
lingniu
@@ -591,9 +591,9 @@ flowchart LR
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/sinks/tdengine-history-store</td>
|
||||
<td>生产历史库边界，负责 TDengine schema、raw_frames/location/telemetry field 行映射、批量写入和分页查询语句；raw_frames 保留完整 payloadJson.parsed，位置表保持轻量字段并通过 rawUri 关联原始帧。</td>
|
||||
<td>生产历史库边界，负责 TDengine schema、raw_frames/location 行映射、批量写入和分页查询语句；raw_frames 保留完整 payloadJson.parsed，位置表保持轻量字段并通过 rawUri 关联原始帧。</td>
|
||||
<td>Kafka Protobuf Envelope、Raw Envelope。</td>
|
||||
<td>TDengine <code>raw_frames</code>、位置表、可选 telemetry field 表。</td>
|
||||
<td>TDengine <code>raw_frames</code> 和位置表。</td>
|
||||
<td><span class="tag green">生产历史</span><span class="tag green">分页查询</span><span class="tag amber">raw 追溯</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
|
||||
@@ -17,10 +17,10 @@ This runbook covers the default production runtimes: GB32960 ingest, JT808 inges
|
||||
| Topic | Producer | Consumers | Purpose |
|
||||
| --- | --- | --- | --- |
|
||||
| `vehicle.raw.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app` | Raw frame records. History writes the parsed JSON, metadata JSON, and raw URI to TDengine `raw_frames`. |
|
||||
| `vehicle.event.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app`, `vehicle-analytics-app` | Normalized vehicle events keyed by VIN when available. |
|
||||
| `vehicle.event.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app` | Normalized vehicle events keyed by VIN when available. |
|
||||
| `vehicle.dlq.gb32960.v1` | Kafka producer/consumer error paths | Operators/replay tooling | Dead-letter records for failed production or consumer processing. |
|
||||
| `vehicle.raw.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app` | JT808 raw frame records for TDengine `raw_frames` and raw-frame troubleshooting APIs. |
|
||||
| `vehicle.event.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app` | JT808 parsed location/session/alarm events keyed by phone or mapped VIN. |
|
||||
| `vehicle.event.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app`, `vehicle-analytics-app` | JT808 parsed location/session/alarm events keyed by phone or mapped VIN; analytics uses location total mileage for daily mileage metrics. |
|
||||
| `vehicle.dlq.jt808.v1` | Kafka producer/consumer error paths | Operators/replay tooling | Dead-letter records for failed JT808 production or consumer processing. |
|
||||
| `vehicle.raw.mqtt-yutong.v1` | `yutong-mqtt-app` | `vehicle-history-app` | Yutong MQTT raw payload records for TDengine `raw_frames` and troubleshooting APIs. |
|
||||
| `vehicle.event.mqtt-yutong.v1` | `yutong-mqtt-app` | `vehicle-history-app` | Yutong MQTT parsed telemetry events keyed by mapped VIN or device identity. |
|
||||
@@ -318,7 +318,7 @@ Observed on 2026-06-29 in worktree `.worktrees/vehicle-ingest-redesign-phase1`:
|
||||
- `jt808-ingest-app` was running through launchctl on TCP `808` and HTTP `20400`; `curl -sS http://127.0.0.1:20400/actuator/health` returned top-level status `UP`.
|
||||
- `gb32960-ingest-app` was running through launchctl on TCP `32960` and HTTP `20100`; `curl -sS http://127.0.0.1:20100/actuator/health` returned top-level status `UP`.
|
||||
- `vehicle-history-app` was running through launchctl on HTTP `20200`; `curl -sS http://127.0.0.1:20200/actuator/health` returned top-level status `UP`.
|
||||
- External JT808 traffic on TCP `808` was parsed and written through Kafka into TDengine. Direct TDengine checks showed more than `6000` JT808 `raw_frames`, more than `1900` JT808 `vehicle_locations`, and more than `15000` JT808 `telemetry_fields`.
|
||||
- External JT808 traffic on TCP `808` was parsed and written through Kafka into TDengine. Direct TDengine checks showed more than `6000` JT808 `raw_frames` and more than `1900` JT808 `vehicle_locations`; current production history does not maintain a `telemetry_fields` table.
|
||||
- A GB32960 synthetic realtime frame sent to TCP `32960` received a binary ACK beginning with `2323`. The same VIN `LTEST202606290001` was queryable from TDengine-backed history APIs with speed, mileage, longitude, and latitude fields.
|
||||
- History raw and event Kafka consumption used separate bindings so raw-frame writes remained current while location/field event ingestion continued.
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
1. 协议 App 接收 TCP 报文。
|
||||
2. 协议 App 归档原始帧元数据，并把事件信封发布到 Kafka。
|
||||
3. `vehicle-history-app` 消费 Kafka。
|
||||
4. history 写入 raw、location 到 TDengine；按需开启 telemetry field 逐字段宽表。
|
||||
4. history 写入 raw、location 到 TDengine；不写 telemetry_fields。
|
||||
5. Swagger/API 通过 TDengine 做历史分页查询。
|
||||
|
||||
每一步都必须执行命令并检查输出后，才能标记为已验证。
|
||||
@@ -305,7 +305,7 @@ JT808 真实转发链路已验证：
|
||||
- `raw_frames` 中 `protocol='JT808'` 的真实行数已超过 `36000`。
|
||||
- 外部平台真实 0x0100 注册帧已超过 `347` 条，真实 0x0200 位置帧已超过 `12400` 条。
|
||||
- `jt808_locations` 中位置行数已超过 `8783`。
|
||||
- 在旧的字段宽表开启模式下，`telemetry_fields` 中 `protocol='JT808'` 的字段行数曾超过 `15000`；当前高吞吐默认不要求该表持续增长。
|
||||
- 当前高吞吐默认链路不维护 `telemetry_fields`；JT808 明细验证以 `raw_frames`、`vehicle_locations` 和 API 分页结果为准。
|
||||
- 分页 API 已用终端号 `13079963301` 验证第一页和 nextCursor 第二页，样例位置包含 `longitude=118.913846`、`latitude=31.927309`、`statusFlag=3` 和 `archive://2026/06/29/JT808/...` 原始帧引用。
|
||||
- 注册帧样例：终端号 `13079963320`、deviceId `9963320`、deviceType `SEG-9888G`、plate `沪A61559F`、maker `70112`、province `31`、city `113`、plateColor `2`。生产 VIN 反写依赖 MySQL identity store，后续 raw/event 的 VIN 解析也从 MySQL 绑定读取。
|
||||
|
||||
@@ -367,7 +367,7 @@ GB32960 链路已用正式平台登录和合成实时帧验证：
|
||||
- 当前正式平台尚未发送车辆实时上报 `0x02`；非测试 VIN 的 `message_id=2` 计数为 `0`，因此不能把平台登录误判为车辆数据在线。
|
||||
- TCP `32960` 返回 GB32960 `2323` ACK。
|
||||
- `raw_frames` 中测试 VIN `LTEST202606290001` 已有 `2` 条 raw 记录。
|
||||
- 在旧的字段宽表开启模式下，`telemetry_fields` 中测试 VIN `LTEST202606290001` 曾有 `34` 条字段记录；当前默认通过 GB32960 snapshot/fields 即时解析 raw，不要求该表增长。
|
||||
- 当前默认通过 GB32960 snapshot/fields 即时解析 `raw_frames` 中的 RAW JSON，不维护字段宽表增长。
|
||||
- latest raw 包含 `archive://2026/06/29/GB32960/LTEST202606290001/...`。
|
||||
- snapshots API 可查到 `speedKmh=62.4`、`totalMileageKm=223456.7`、`longitude=116.397128`、`latitude=39.916527`。
|
||||
|
||||
|
||||
@@ -380,6 +380,28 @@ class MavenModuleProfileTest {
|
||||
.doesNotContain("Spring Boot 3.4.x");
|
||||
}
|
||||
|
||||
@Test
|
||||
void productionDocsMatchCompactHistoryAndJt808OnlyAnalyticsSurface() throws Exception {
|
||||
String moduleDataFlow = Files.readString(repositoryRoot().resolve("docs/module-data-flow.html"));
|
||||
String tdengineVerification = Files.readString(repositoryRoot()
|
||||
.resolve("docs/operations/vehicle-ingest-tdengine-verification.md"));
|
||||
String runbook = Files.readString(repositoryRoot()
|
||||
.resolve("docs/operations/gb32960-service-split-runbook.md"));
|
||||
|
||||
assertThat(moduleDataFlow)
|
||||
.contains("TDengine raw_frames + locations")
|
||||
.contains("TDengine <code>raw_frames</code> 和位置表")
|
||||
.doesNotContain("telemetry field 行映射")
|
||||
.doesNotContain("可选 telemetry field 表");
|
||||
assertThat(tdengineVerification)
|
||||
.contains("history 写入 raw、location 到 TDengine；不写 telemetry_fields")
|
||||
.doesNotContain("按需开启 telemetry field");
|
||||
assertThat(runbook)
|
||||
.contains("| `vehicle.event.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app` |")
|
||||
.contains("| `vehicle.event.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app`, `vehicle-analytics-app` |")
|
||||
.doesNotContain("| `vehicle.event.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app`, `vehicle-analytics-app` |");
|
||||
}
|
||||
|
||||
@Test
|
||||
void supersededDesignDocsPointToCurrentArchitecture() throws Exception {
|
||||
for (String path : List.of(
|
||||
|
||||
Reference in New Issue
Block a user