docs: clarify kafka-only production channel

This commit is contained in:
lingniu
2026-07-01 11:25:00 +08:00
parent 61ad4a2439
commit ebbf2ca8fb
4 changed files with 27 additions and 11 deletions

View File

@@ -2,11 +2,11 @@
> 本文件记录 lingniu-vehicle-ingest v2 的关键架构决策,后续每次重大调整追加条目(不删除旧条目)。 > 本文件记录 lingniu-vehicle-ingest v2 的关键架构决策,后续每次重大调整追加条目(不删除旧条目)。
## ADR-001 MQ 选型Kafka ## ADR-001 消息通道Kafka
- **Status**: Accepted - **Status**: Accepted
- **Context**: 需要高吞吐、严格单车有序、成熟生态 - **Context**: 需要高吞吐、严格单车有序、成熟生态
- **Decision**: 使用 Kafka分区 key = VIN - **Decision**: 使用 Kafka分区 key = VIN
- **Consequences**: 下游消费者需使用消费组 + 分区内顺序消费;不使用 RocketMQ 的事务/顺序特性 - **Consequences**: 下游消费者需使用消费组 + 分区内顺序消费;生产链路只支持 Kafka。
## ADR-002 线上消息格式Protobuf ## ADR-002 线上消息格式Protobuf
- **Status**: Accepted - **Status**: Accepted

View File

@@ -414,7 +414,7 @@ sequenceDiagram
participant Dispatcher as Dispatcher participant Dispatcher as Dispatcher
participant Handler as Protocol Handler<br/>Mapper participant Handler as Protocol Handler<br/>Mapper
participant EventBus as DisruptorEventBus participant EventBus as DisruptorEventBus
participant MQ as sink-kafka<br/>Kafka Envelope participant Kafka as sink-kafka<br/>Kafka Envelope
participant Archive as sink-archive<br/>ArchiveStore participant Archive as sink-archive<br/>ArchiveStore
participant History as vehicle-history-app participant History as vehicle-history-app
participant TDengine as tdengine-history-store<br/>raw_frames + locations participant TDengine as tdengine-history-store<br/>raw_frames + locations
@@ -430,12 +430,12 @@ sequenceDiagram
Handler->>Handler: 协议字段映射为内部字段 Handler->>Handler: 协议字段映射为内部字段
Dispatcher->>Handler: 绑定 rawArchiveKey / rawArchiveUri Dispatcher->>Handler: 绑定 rawArchiveKey / rawArchiveUri
Handler->>EventBus: VehicleEvent(Realtime / Location / Alarm / Login ...) Handler->>EventBus: VehicleEvent(Realtime / Location / Alarm / Login ...)
EventBus->>MQ: 投递规整后的业务事件 EventBus->>Kafka: 投递规整后的业务事件
MQ->>MQ: EnvelopeMapper 转 Protobuf Kafka->>Kafka: EnvelopeMapper 转 Protobuf
MQ->>History: Kafka event/raw topickey=vin单车有序 Kafka->>History: event/raw topickey=vin单车有序
History->>TDengine: 写 raw_frames、locations 和 raw parsed JSON History->>TDengine: 写 raw_frames、locations 和 raw parsed JSON
TDengine->>Consumer: 分页历史查询、原始帧追溯、位置查询 TDengine->>Consumer: 分页历史查询、原始帧追溯、位置查询
MQ->>Consumer: Redis 热状态、指标统计、告警工单等下游消费 Kafka->>Consumer: Redis 热状态、指标统计、告警工单等下游消费
</pre> </pre>
</div> </div>
<div class="callout"> <div class="callout">

View File

@@ -24,13 +24,12 @@ export TDENGINE_USERNAME=root
export TDENGINE_PASSWORD='<tdengine-password>' export TDENGINE_PASSWORD='<tdengine-password>'
export TDENGINE_MIN_IDLE=0 export TDENGINE_MIN_IDLE=0
export TDENGINE_MAX_POOL_SIZE=32 export TDENGINE_MAX_POOL_SIZE=32
export EVENT_FILE_STORE_ENABLED=false
export TDENGINE_TELEMETRY_FIELDS_ENABLED=false export TDENGINE_TELEMETRY_FIELDS_ENABLED=false
``` ```
`TDENGINE_MIN_IDLE=0` 用于减少冷启动时 TDengine 连接重试日志。TDengine 地址稳定后,再按生产并发调大连接池。 `TDENGINE_MIN_IDLE=0` 用于减少冷启动时 TDengine 连接重试日志。TDengine 地址稳定后,再按生产并发调大连接池。
高吞吐生产验收默认关闭 `EVENT_FILE_STORE_ENABLED` `TDENGINE_TELEMETRY_FIELDS_ENABLED`,验证 TDengine `raw_frames`、位置表和分页查询闭环。RAW 帧的 `parsedJson``metadataJson``rawUri` 直接进入 TDengineGB32960 snapshot/fields 接口优先基于 `raw_frames` 中的结构化 JSON 返回结果;接入服务写出的原始 `.bin` 只作为冷备复核材料。需要兼容旧 `/api/event-history/records` 低层记录接口或逐字段趋势宽表时,再显式开启对应开关,并单独压测写入放大。 高吞吐生产验收默认保持 `TDENGINE_TELEMETRY_FIELDS_ENABLED=false`,验证 TDengine `raw_frames`、位置表和分页查询闭环。RAW 帧的 `parsedJson``metadataJson``rawUri` 直接进入 TDengineGB32960 snapshot/fields 接口基于 `raw_frames` 与当前解析器即时返回结构化结果;接入服务写出的原始 `.bin` 只作为冷备复核材料。需要逐字段趋势宽表时,再显式开启 `TDENGINE_TELEMETRY_FIELDS_ENABLED=true`,并单独压测写入放大。
## 构建 ## 构建
@@ -83,7 +82,6 @@ lsof -nP -iTCP:32960 -sTCP:LISTEN
```bash ```bash
HTTP_PORT=20200 \ HTTP_PORT=20200 \
KAFKA_CONSUMER_ENABLED=true \ KAFKA_CONSUMER_ENABLED=true \
EVENT_FILE_STORE_ENABLED=false \
EVENT_FILE_STORE_PATH=./target/tdengine-verification/event-store \ EVENT_FILE_STORE_PATH=./target/tdengine-verification/event-store \
TDENGINE_TELEMETRY_FIELDS_ENABLED=false \ TDENGINE_TELEMETRY_FIELDS_ENABLED=false \
java -jar modules/apps/vehicle-history-app/target/vehicle-history-app.jar java -jar modules/apps/vehicle-history-app/target/vehicle-history-app.jar

View File

@@ -116,7 +116,9 @@ class PortainerComposeResourceLimitsTest {
.contains("TDengine `raw_frames`") .contains("TDengine `raw_frames`")
.doesNotContain("三个服务必须共用同一个 `SINK_ARCHIVE_PATH`") .doesNotContain("三个服务必须共用同一个 `SINK_ARCHIVE_PATH`")
.doesNotContain("再回读共享 archive") .doesNotContain("再回读共享 archive")
.doesNotContain("读取共享 archive 中的原始 `.bin`"); .doesNotContain("读取共享 archive 中的原始 `.bin`")
.doesNotContain("EVENT_FILE_STORE_ENABLED")
.doesNotContain("/api/event-history/records");
assertThat(splitRunbook) assertThat(splitRunbook)
.contains("TDengine `raw_frames`") .contains("TDengine `raw_frames`")
.doesNotContain("must share the same `SINK_ARCHIVE_PATH`") .doesNotContain("must share the same `SINK_ARCHIVE_PATH`")
@@ -124,6 +126,22 @@ class PortainerComposeResourceLimitsTest {
.doesNotContain("shared volume mounted to all three services"); .doesNotContain("shared volume mounted to all three services");
} }
@Test
void architectureDocsNameKafkaInsteadOfGenericMq() throws IOException {
String decisions = Files.readString(repositoryRoot().resolve("DECISIONS.md"));
String dataFlow = Files.readString(repositoryRoot().resolve("docs/module-data-flow.html"));
assertThat(decisions)
.contains("生产链路只支持 Kafka")
.doesNotContain("MQ 选型")
.doesNotContain("RocketMQ");
assertThat(dataFlow)
.contains("participant Kafka as sink-kafka")
.doesNotContain("participant MQ")
.doesNotContain("->>MQ")
.doesNotContain("MQ->>");
}
@Test @Test
void splitRunbookKeepsAnalyticsAsJt808MetricRuntimeOnly() throws IOException { void splitRunbookKeepsAnalyticsAsJt808MetricRuntimeOnly() throws IOException {
String splitRunbook = Files.readString(repositoryRoot() String splitRunbook = Files.readString(repositoryRoot()