diff --git a/.dockerignore b/.dockerignore deleted file mode 100644 index 08b6fcca..00000000 --- a/.dockerignore +++ /dev/null @@ -1,5 +0,0 @@ -.git -.idea -.worktrees -archive -**/.DS_Store diff --git a/CHANGELOG.md b/CHANGELOG.md deleted file mode 100644 index e85b9c04..00000000 --- a/CHANGELOG.md +++ /dev/null @@ -1,113 +0,0 @@ -# Changelog - -本文件记录本仓库的显著变更。日期使用 `YYYY-MM-DD` 格式。 - -格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)，版本遵循 [SemVer](https://semver.org/lang/zh-CN/)。 - -> 说明：v0.1.0 为本仓库 git 初始化时的基线快照。下面列出的"主题改动"实际上都被压在 -> `chore: initial import of lingniu-vehicle-ingest` 这一个 commit 里——这是 git -> 初始化前已经完成的工作，无法事后拆分。从下一次提交开始走"小步分主题提交"流程。 - ---- - -## [Unreleased] - -### Added —— 原始报文冷存（Raw Archive）接入 EventSink 扇出 -- 新增 `VehicleEvent.RawArchive` sealed 变体（`ingest-api`），携带一帧入站的原始字节 + - `command` / `infoType` 供按协议命令分片归档。 -- `Dispatcher.dispatch` 在 interceptor **之前**按 `RawFrame.rawBytes()` 产出一条 - `RawArchive` 事件，保证 dedup / rate-limit **不过滤** raw archive，满足"每帧必留痕"目标。 -- 新增 `ArchiveEventSink implements EventSink`（`sink-archive`）： - - `accepts` 仅 `VehicleEvent.RawArchive` - - 写入 key `yyyy/MM/dd///.bin`（UTC 日期分片） - - 同 eventId 重复写按 put 语义覆盖 - - 失败 completeExceptionally，EventBus 打 WARN - - AutoConfig 在 `ArchiveStore` bean 存在时自动装配 -- `KafkaEventSink.accepts(RawArchive)` 返回 `false`：本期 Kafka 不投递原始字节， - 只落 ArchiveStore。未来需要 Kafka 回填 URI 时再开（`TopicRouter` / `EnvelopeMapper` - 已补好 switch 分支）。 -- 覆盖测试 `ArchiveEventSinkTest` —— accepts 过滤 / 稳定 key 写入 / 幂等覆盖 3 case。 - -### Changed —— GB/T 32960 Body Parser 单块异常隔离 -- `Gb32960BodyParser` 主循环对单信息块的 parser 异常不再放弃整帧： - - 固定长度块（`fixedLength ≥ 0`）：回滚 reader → 按 `fixedLen` 截取字节兜成 - `InfoBlock.Raw` → `continue` 继续解析后续块； - - 变长块（`fixedLength == -1`）或剩余字节不足：剩余字节全部兜成 `Raw` 后终止循环； - - `parser` 契约违反（声明 `fixedLen=X` 但实际读了 `Y`）仍抛 `DecodeException`， - 以暴露 parser bug，不走 lenient 兜底，避免静默误解析长期误判； - - 只捕获 `DecodeException` / `BufferUnderflowException` / `IndexOutOfBoundsException`， - `RuntimeException` 继续向上抛，保留 bug 可见性。 -- 新增配置 `lingniu.ingest.gb32960.parse.lenient-block-failure`（默认 `true`）； - 需要回退严格模式时设为 `false`。 -- `InfoBlock.Raw` record 结构**未变**，下游 `Gb32960EventMapper` 按 `findBlock(Class)` - 类型匹配，新增 Raw 不影响业务事件映射。 -- 新增测试 `Gb32960BodyParserIsolationTest` —— 3 种隔离场景 + 严格模式回退，共 4 case。 - ---- - -## [0.1.0] — 2026-04-15 - -初始基线。多模块 Spring Boot 车辆遥测接入服务。 - -### Added -- **GB/T 32960.3 接入**（`protocol-gb32960`）：Netty server、帧分帧器、消息解码器、 - V2016/V2025 双版本信息体 parser、平台登入鉴权、VIN 白名单、读空闲告警。 -- **JT/T 808 / JT/T 1078 / JSATL12 接入**（`protocol-jt808` / `protocol-jt1078` / - `protocol-jsatl12`）。 -- **MQTT 入站**（`inbound-mqtt`）。 -- **Disruptor 事件总线**（`ingest-core`）：高吞吐解耦解码与下游 sink。 -- **Sink**：本地归档（`sink-archive`）、Kafka（`sink-kafka`，含 protobuf envelope）。 -- **会话状态**（`session-core`）。 -- **可观测性**：Micrometer/Actuator 装配（`observability`）。 -- **终端控制命令网关**（`command-gateway`）。 - -### Removed -- 旧文件型事件索引模块和 DuckDB 依赖；历史查询统一收敛到 TDengine 与 `archive://...` 原始报文引用。 - -### Fixed —— GB/T 32960 应答帧时间字段 -- `Gb32960MessageDecoder` 对 `VEHICLE_LOGIN` / `VEHICLE_LOGOUT` / - `PLATFORM_LOGIN` / `PLATFORM_LOGOUT` 命令预解析 body 首 6B 采集时间，写入 - `header.eventTime`。修复前 handler 回 ACK 时落到 `Instant.now()`，导致应答时间与原 - 报文时间不一致，违反 §6.3.2 应答规则。 - -### Added —— Netty 读空闲告警 -- 配置项 `lingniu.ingest.gb32960.idle-read-seconds`（默认 60s，0 禁用）。 -- pipeline 注入 `IdleStateHandler(idleReadSeconds, 0, 0)`，仅打 WARN 日志不断链， - 用于排查"对端登入成功后长时间不推数据"场景。 - -### Changed —— GB32960 信息体 parser 包结构重构 -- 原 `codec/parser/` 下的 16 个共用类按版本严格拆分到 `parser/v2016/`（10 类） - 和 `parser/v2025/`（11 类）两个独立子包。 -- 取消 `Vehicle/DriveMotor/Engine/Position` 的 `v2016()/v2025()` 工厂方法，每个 - 版本独立类，零代码共用。 -- `Alarm` 解析中跨版本静态调用 `readFaultList` 也已改为各版本类内部私有方法。 -- `Gb32960AutoConfiguration` 同步更新所有 import 与 `@Bean` 工厂。 - -### Removed —— 违反 2016 标准的 typeCode 注册 -- 经核对 GB/T 32960.3-2016 附录 B 表 B.3，`0x30~0x7F` 整段为预留区， - **2016 标准里没有 0x30 / 0x31 / 0x32 任何字段定义**。 -- 删除 `FuelCellStackV2016BlockParser`（曾把 V2025 字段布局硬套到 V2016 帧的 0x30 段， - 解出来的电压/电流远超规范上限，是非物理值）。 -- `Gb32960AutoConfiguration` 移除 `gb32960V2016FuelCellStack` bean。 -- `Gb32960DecoderGoldenTest` 中 749B 真实生产帧的断言由 - "`isEmpty()`（不应有 Raw 兜底）"翻转为 "`isPresent()`（peer 越界使用 0x30+ 预留 - typeCode，应有 Raw 兜底）"。这才是符合 2016 国标的正确预期。 - -### Diagnostics —— BodyParser hex dump 增强 -- `Gb32960BodyParser` 在遇到未知 typeCode 触发 unknown 路径时，额外打一条 WARN， - 内容为整段 info-block 区域的 16 字节/行 hex dump，并在 `typeStartPos` 字节后插入 - `<` 标记，便于人工反查上一个块的字段对齐。 - -### Notes —— 待办 -- peer 在 V2016 帧（`2323` 起始）里下发 0x30/0x31/0x32 是越界使用 2016 预留区段。 - 建议推动对端： - 1. 切换到 V2025 帧头（`2424`）； - 2. 或迁移到 `0x80~0xFE` 用户自定义区段； - 3. 或提供私有协议文档，再决定是否实现 vendor-specific parser。 -- `Gb32960BodyParser` 当前对 unknown 块的策略是"吞掉剩余所有字节为单个 Raw 后终止 - 循环"，未做单块异常隔离。后续可改为对每个 parser.parse 调用加 try/catch + - position 回滚，实现单块失败不拖垮整帧。 -- `platformAuthorizer.authenticate` 是 EventLoop 同步调用，存在 worker 阻塞风险， - 建议挪到独立业务线程池（与 0x30 议题无关，但属于同模块潜在风险）。 - -[0.1.0]: #010--2026-04-15 diff --git a/DECISIONS.md b/DECISIONS.md deleted file mode 100644 index 7acb356f..00000000 --- a/DECISIONS.md +++ /dev/null @@ -1,105 +0,0 @@ -# 架构决策记录（ADR 汇总） - -> 本文件记录 lingniu-vehicle-ingest v2 的关键架构决策，后续每次重大调整追加条目（不删除旧条目）。 - -## ADR-001 消息通道：Kafka -- **Status**: Accepted -- **Context**: 需要高吞吐、严格单车有序、成熟生态 -- **Decision**: 使用 Kafka，分区 key = VIN -- **Consequences**: 下游消费者需使用消费组 + 分区内顺序消费；生产链路只支持 Kafka。 - -## ADR-002 线上消息格式：Protobuf -- **Status**: Accepted -- **Context**: 需要向前兼容、体积小、性能好 -- **Decision**: 线上用 Protobuf，调试/诊断保留 JSON 序列化能力 -- **Consequences**: 需维护 `.proto` schema，`ingest-api` 模块负责生成 Java stub - -## ADR-003 command-gateway 独立模块 -- **Status**: Accepted -- **Context**: 下行命令链路不应与接入进程耦合 -- **Decision**: 拆出独立 `command-gateway` 模块，复用 `session-core`；本次重构同步交付 -- **Consequences**: HTTP 对外接口（原 JT808Controller / JT1078Controller）迁移到此模块 - -## ADR-004 信达 Push：Removed -- **Status**: Superseded by ADR-011 -- **Context**: 旧实现硬编码凭证、无重连、0300/0401 空实现 -- **Decision**: 删除信达 Push 源码、Maven profile、私仓依赖和部署入口。 -- **Consequences**: 信达 Push 后续已从源码和构建面删除，优化优先投向 GB32960、JT808、宇通 MQTT、历史和统计链路。 - -## ADR-005 JT1078 / JSATL12 进第一批 -- **Status**: Superseded by ADR-012 -- **Decision**: 第一批迁移即覆盖 JT1078 信令 + JSATL12 报警附件上传 -- **Consequences**: Phase 2 工期相应拉长；JSATL12 需要对象存储后端（本地 / S3 / OSS） - -## ADR-006 部署形态：GB32960 三应用拆分 -- **Status**: Superseded by ADR-011 -- **Context**: 原 `bootstrap-all` 一体化启动已被 GB32960 拆分部署替代 -- **Decision**: 当时生产默认部署 `gb32960-ingest-app`、`vehicle-history-app`、`vehicle-analytics-app`；旧 `bootstrap-all` 模块删除 -- **Consequences**: 该三应用边界已由 ADR-011 的三协议接入 + 历史 + 统计生产面取代；协议接入、历史查询、统计消费仍保持独立发布和回滚。 - -## ADR-007 Java 25 + 虚拟线程 + Disruptor -- **Status**: Accepted -- **Decision**: - - Netty EventLoop 只做解码与 RingBuffer 投递，严禁阻塞 - - 业务 Handler 走虚拟线程（`Thread.ofVirtual()`） - - 协议间背压通过 Disruptor RingBuffer 表达 - - 禁用 `synchronized`，使用 `ReentrantLock` 避免虚拟线程 pinning -- **Consequences**: 需开启 `jdk.VirtualThreadPinned` JFR 监控 - -## ADR-008 协议接入应用不直接写业务库 -- **Status**: Accepted -- **Decision**: GB32960、JT808、Yutong MQTT 等协议接入应用只负责收、解析、校验、规整和投递 Kafka，不直接写业务表。 -- **Consequences**: 业务落库由独立 Kafka 消费应用承接：`vehicle-history-app` 写入 TDengine 历史与 RAW JSON，`vehicle-analytics-app` 写入 MySQL `vehicle_stat_metric`。 - -## ADR-009 构建工具：Maven -- **Status**: Accepted -- **Decision**: 沿用 Maven，统一 BOM 管理版本，Spotless 格式化，ArchUnit 守护分层 - -## ADR-010 框架：Spring Boot 3.5 -- **Status**: Accepted -- **Decision**: Spring Boot 3.5.x，支持 JDK 25；AutoConfiguration 用于按需启停 - -## ADR-011 默认生产面：三协议接入 + 历史 + 统计 -- **Status**: Accepted -- **Context**: 生产接入范围已经从 GB32960 三应用拆分，扩展为三条活跃接入链路和两个消费应用；信达 Push 已废弃并删除，不应再出现在构建、部署或优化目标里。 -- **Decision**: - 1. 默认生产面包含 GB32960、JT808、Yutong MQTT、vehicle-history-app、vehicle-analytics-app。 - 2. Xinda Push 源码、Maven profile、Woodpecker 镜像发布和历史消费绑定全部移除。 - 3. `command-gateway` 和 JT1078 继续作为可选能力，通过 `optional-command-gateway` profile 显式启用。 -- **Consequences**: 默认构建和部署保持精简；后续性能、可靠性、字段解析和存储优化都服务三条活跃协议链路。 - -## ADR-012 JSATL12 附件上传：Optional only -- **Status**: Accepted -- **Context**: 当前默认生产面只部署 GB32960、JT808、Yutong MQTT、vehicle-history-app、vehicle-analytics-app；JSATL12 附件上传没有独立生产 app，也不在 Portainer 和 Woodpecker 活跃镜像列表中。 -- **Decision**: - 1. `protocol-jsatl12` 不进入默认 Maven reactor。 - 2. JSATL12 仅保留在 `optional-attachments` profile，需要附件上传能力时显式构建。 - 3. 默认优化和验证优先覆盖活跃接入链路，附件上传能力保持源码可用但不增加默认构建面。 -- **Consequences**: 默认构建更轻，生产部署边界更清晰；附件上传相关测试需要通过 `-Poptional-attachments` 显式运行。 - -## ADR-013 最新状态服务：Optional only -- **Status**: Accepted -- **Context**: Redis 最新状态查询是独立消费能力，但当前默认 Portainer 部署和 Woodpecker 镜像发布只包含三条接入链路、history 和 analytics；`vehicle-state-service` 没有独立 app，也不应增加默认构建面。 -- **Decision**: - 1. `vehicle-state-service` 不进入默认 Maven reactor。 - 2. vehicle-state-service 仅保留在 `optional-latest-state` profile，需要 Redis 最新状态查询能力时显式构建。 - 3. 默认优化和验证优先覆盖活跃接入、TDengine 历史和 MySQL 指标链路。 -- **Consequences**: 默认构建和部署边界继续收窄；最新状态能力保持源码可用，但其测试需要通过 `-Poptional-latest-state` 显式运行。 - -## ADR-014 raw-archive-store 原型已删除 -- **Status**: Accepted -- **Context**: 默认生产 raw bytes 写入由 `sink-archive` 负责，历史查询通过 TDengine `raw_frames` 和 `archive://...` 引用追溯；`raw-archive-store` 只是未部署的独立读写原型，继续保留会造成 raw archive 路径歧义。 -- **Decision**: - 1. 删除 `raw-archive-store` 模块和对应 optional profile。 - 2. 默认生产链路只保留 `sink-archive`、Kafka raw topic、TDengine raw_frames 这条 raw 路径。 - 3. 如需新的 archive store 能力，先以生产链路需求重新设计，不恢复旧原型。 -- **Consequences**: 默认构建面和可选构建面继续收窄；raw bytes 写入职责集中在 `sink-archive`。 - -## ADR-015 文件型事件索引：Removed -- **Status**: Accepted -- **Context**: 默认历史查询已收敛到 TDengine `raw_frames`、`vehicle_locations` 和按需解码；旧文件型索引会增加一套无生产部署的查询和依赖边界。 -- **Decision**: - 1. 删除旧文件型事件索引模块和 profile。 - 2. 父 POM 不再管理旧索引驱动依赖。 - 3. 历史查询和 RAW 回放统一以 TDengine + `archive://...` 引用为准。 -- **Consequences**: 默认构建面和可选构建面都更小；需要历史查询时只维护 TDengine 一条路径。 diff --git a/Dockerfile b/Dockerfile deleted file mode 100644 index 41021641..00000000 --- a/Dockerfile +++ /dev/null @@ -1,19 +0,0 @@ -FROM eclipse-temurin:25-jre - -ARG APP_NAME -ARG APP_VERSION - -ENV APP_NAME=${APP_NAME} -ENV APP_VERSION=${APP_VERSION} -ENV JAVA_OPTS="--sun-misc-unsafe-memory-access=allow -Dio.netty.transport.noNative=true" - -RUN apt-get update \ - && apt-get install -y --no-install-recommends curl \ - && rm -rf /var/lib/apt/lists/* - -WORKDIR /app -COPY modules/apps/${APP_NAME}/target/${APP_NAME}.jar /app/app.jar - -EXPOSE 20100 20200 20310 20400 20500 32960 808 - -ENTRYPOINT ["sh", "-c", "exec java $JAVA_OPTS -jar /app/app.jar"] diff --git a/README.md b/README.md deleted file mode 100644 index bad5be4d..00000000 --- a/README.md +++ /dev/null @@ -1,104 +0,0 @@ -# lingniu-vehicle-ingest - -> 羚牛车辆数据接入平台 v2 - -## 设计目标 - -- **协议接入统一抽象**：GB/T 32960、JT/T 808、Yutong MQTT 是默认生产接入；JT/T 1078、JSATL12 作为显式 profile 的可选能力；信达 Push 已废弃并删除源码 -- **原子能力化**：每个协议一个独立 Maven 模块 + 独立 AutoConfiguration + 独立配置开关 -- **业务 / 实时彻底解耦**：协议接入应用只负责收、解析、校验、规整和投递 Kafka；历史与统计由独立 Kafka 消费应用落库 -- **高并发低延迟**：Netty + Disruptor + Java 25 虚拟线程；目标单节点 ≥ 5 万 msg/s，P99 < 50 ms -- **可观测、可回放、可灰度**：统一 traceId、Envelope、幂等键、DLQ、原始报文冷存 - -## 技术栈 - -| 类别 | 选型 | -|---|---| -| 语言 | Java **25** | -| 框架 | Spring Boot 3.5.3 | -| 网络 | Netty 4.2.9.Final | -| 并发 | Disruptor 4 + 虚拟线程 | -| 事件流 | **Kafka**（唯一生产消息通道，vin 分区，保证单车有序） | -| 线上消息格式 | **Protobuf**，调试走 JSON | -| MQTT 客户端 | Eclipse Paho 1.2.5 | -| 会话索引 | Redis | -| 熔断/限流 | Resilience4j 2 | -| 可观测 | Micrometer + Prometheus | -| 冷存 | 本地文件系统 | -| 构建 | Maven | - -## 模块划分 - -``` -lingniu-vehicle-ingest/ -├── modules/ -│ ├── core/ -│ │ ├── ingest-api/ SPI + sealed 领域事件 + 注解 -│ │ ├── ingest-codec-common/ 公共编解码工具（BCD/CRC/BCC/bit utils） -│ │ ├── ingest-core/ Pipeline / Dispatcher / Disruptor / Session 桥 -│ │ ├── session-core/ 设备会话 + 鉴权 + Token + Redis SessionStore -│ │ ├── vehicle-identity/ 跨协议车辆身份解析 + MySQL 外部标识绑定 -│ │ └── observability/ metrics / health -│ ├── protocols/ -│ │ ├── protocol-gb32960/ GB/T 32960 -│ │ ├── protocol-jt808/ JT/T 808（统一身份映射 + 事件 metadata 内部 VIN + 注册/鉴权/心跳/注销/位置/批量位置/参数/属性/媒体/透传/未知上行和坏帧兜底/断链清理会话/下行分包） -│ │ ├── protocol-jt1078/ JT/T 1078（optional-command-gateway profile；808 信令按需桥接 + 常用下行信令编码 + TCP/UDP RTP 媒体流分段归档 + 事件 metadata 内部 VIN + 坏 RTP 统一 RawArchive/Passthrough + 归档失败兜底 + SIM 身份映射） -│ │ └── protocol-jsatl12/ 苏标主动安全报警附件（optional-attachments profile） -│ ├── inbound/ -│ │ └── inbound-mqtt/ MQTT 接入（endpoint 生命周期 + profile 注册扩展 + 统一身份映射 + PEM 双向 TLS + 未知 profile/解析失败/profile异常/连接订阅失败兜底 + 统一 UNKNOWN 身份 metadata） -│ ├── sinks/ -│ │ ├── sink-kafka/ Kafka producer + Protobuf Envelope -│ │ └── sink-archive/ 原始报文冷存 -│ ├── services/ -│ │ ├── event-history-service/ Kafka 全字段事件消费 + 历史查询 -│ │ ├── vehicle-state-service/ Kafka 全字段事件消费 + Redis 热状态查询（optional-latest-state profile） -│ │ └── vehicle-stat-service/ Kafka 全字段事件消费 + 可配置日统计 -│ ├── testing/ -│ │ └── vehicle-identity-test-support/ 协议测试共享身份解析夹具 -│ └── apps/ -│ ├── command-gateway/ 可选 HTTP → 设备下行命令（optional-command-gateway profile） -│ ├── gb32960-ingest-app/ GB32960 TCP 接入 + Kafka 投递 -│ ├── jt808-ingest-app/ JT808 TCP 接入 + Kafka 投递 -│ ├── yutong-mqtt-app/ 宇通 MQTT 接入 + Kafka 投递 -│ ├── vehicle-history-app/ TDengine 历史查询 + RAW JSON -│ └── vehicle-analytics-app/ JT808 每日里程指标消费 -├── docs/ 架构文档、模块图、实施计划 -└── reference/ 参考资料 -``` - -## 快速开始 - -```bash -# 要求：JDK 25, Maven 3.9+ -mvn -v -mvn -pl :gb32960-ingest-app,:jt808-ingest-app,:yutong-mqtt-app,:vehicle-history-app,:vehicle-analytics-app -am package -Dmaven.test.skip=true -``` - -生产服务只在 ECS/Portainer 上运行；本机只用于源码检查、Maven 构建和单元测试。部署、健康检查和真实流量验证步骤见 `docs/operations/gb32960-service-split-runbook.md` 与 `docs/operations/vehicle-ingest-tdengine-verification.md`。 - -信达 Push 已废弃并删除源码，不参与 Maven reactor、CI、Portainer 部署和历史消费链路。 - -JSATL12 附件上传不在默认 Maven reactor 中；需要时使用 `-Poptional-attachments` 构建。 - -Command Gateway/JT1078 下行与音视频信令不在默认 Maven reactor 中；需要时使用 `-Poptional-command-gateway` 构建。 - -Redis 最新状态查询不在默认 Maven reactor 中；需要时使用 `-Poptional-latest-state` 构建 `vehicle-state-service`。 - -## 迁移说明 - -本项目是 `lingniu-vehicle-data-reception` 的 v2 重构，采用 strangler fig 渐进式迁移，旧项目保留只读参考。迁移路径与决策参见 `../REFRACTOR_PLAN.md`。 - -## 架构文档 - -- 目标架构：`docs/target-architecture.md` -- 内部字段模型：`docs/vehicle-telemetry-internal-fields.md` -- 模块与数据流：`docs/module-data-flow.html` - -## 核心原则 - -1. **接入与业务落库分离**：协议接入应用只负责收、解析、校验、规整和投递 Kafka；`vehicle-history-app` 写入 TDengine 历史与 RAW JSON，`vehicle-analytics-app` 将 JT808 `daily_mileage_km` 写入 MySQL `vehicle_stat_metric` -2. **Kafka 唯一消息通道**：生产链路只支持 Kafka；接入、历史、统计统一通过 Kafka Envelope 解耦 -3. **协议即插拔**：每个 `protocol-*` / `inbound-*` 都可独立开关（`lingniu.ingest..enabled`） -4. **顺序保证**：同一 VIN 严格有序（Disruptor hash + Kafka 分区 key） -5. **幂等消费**：Envelope 带 `eventId`，下游去重 -6. **原始可回放**：协议接入会把 `rawArchiveKey/rawArchiveUri` 追加到事件 metadata；Kafka Envelope、TDengine `raw_frames` 和导出都使用 `archive://...` 逻辑 URI 追溯原始 bytes，实际文件由 `sink-archive` 管理 diff --git a/deploy/portainer/docker-compose-go.yml b/deploy/portainer/docker-compose-go.yml deleted file mode 100644 index 32bbadd5..00000000 --- a/deploy/portainer/docker-compose-go.yml +++ /dev/null @@ -1,117 +0,0 @@ -version: "3.8" - -# Go sidecar stack for the vehicle ingest redesign. -# It is intentionally separate from deploy/portainer/docker-compose.yml so the -# current Java production stack can keep running until Go evidence is captured. - -x-common-env: &common-env - TZ: Asia/Shanghai - KAFKA_BROKERS: ${KAFKA_BROKERS:-172.17.111.56:9092} - KAFKA_TOPIC_GB32960_RAW: ${KAFKA_TOPIC_GB32960_RAW:-vehicle.raw.gb32960.v1} - KAFKA_TOPIC_JT808_RAW: ${KAFKA_TOPIC_JT808_RAW:-vehicle.raw.jt808.v1} - KAFKA_TOPIC_YUTONG_MQTT_RAW: ${KAFKA_TOPIC_YUTONG_MQTT_RAW:-vehicle.raw.yutong-mqtt.v1} - KAFKA_TOPIC_UNIFIED: ${KAFKA_TOPIC_UNIFIED:-vehicle.event.unified.v1} - -x-restart: &restart-policy - restart: unless-stopped - networks: - - vehicle-ingest - logging: - driver: json-file - options: - max-size: "100m" - max-file: "5" - -services: - go-vehicle-gateway: - <<: *restart-policy - image: ${LINGNIU_GO_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com}/${LINGNIU_GO_IMAGE_NAMESPACE:-oneos}/vehicle-gateway-go:${LINGNIU_GO_IMAGE_VERSION:?set LINGNIU_GO_IMAGE_VERSION} - container_name: go-vehicle-gateway - command: ["/app/gateway"] - mem_limit: ${GO_GATEWAY_MEM_LIMIT:-512m} - environment: - <<: *common-env - GB32960_TCP_ADDR: ":32960" - JT808_TCP_ADDR: ":808" - TCP_READ_BUFFER_BYTES: ${TCP_READ_BUFFER_BYTES:-65536} - TCP_IDLE_TIMEOUT_SECONDS: ${TCP_IDLE_TIMEOUT_SECONDS:-180} - TCP_MAX_CONNECTIONS: ${TCP_MAX_CONNECTIONS:-20000} - YUTONG_MQTT_ENABLED: ${YUTONG_MQTT_ENABLED:-false} - YUTONG_MQTT_ENDPOINT: ${YUTONG_MQTT_ENDPOINT:-yutong} - YUTONG_MQTT_URI: ${YUTONG_MQTT_URI:-} - YUTONG_MQTT_TOPICS: ${YUTONG_MQTT_TOPICS:-/ytforward/shln/+} - YUTONG_MQTT_QOS: ${YUTONG_MQTT_QOS:-2} - YUTONG_MQTT_CLIENT_ID: ${YUTONG_MQTT_CLIENT_ID:-lingniu-go-yutong-mqtt} - YUTONG_MQTT_USERNAME: ${YUTONG_MQTT_USERNAME:-} - YUTONG_MQTT_PASSWORD: ${YUTONG_MQTT_PASSWORD:-} - YUTONG_MQTT_CLEAN_SESSION: ${YUTONG_MQTT_CLEAN_SESSION:-false} - YUTONG_MQTT_KEEP_ALIVE_SECONDS: ${YUTONG_MQTT_KEEP_ALIVE_SECONDS:-20} - YUTONG_MQTT_CONNECTION_TIMEOUT_SECONDS: ${YUTONG_MQTT_CONNECTION_TIMEOUT_SECONDS:-10} - YUTONG_MQTT_TLS_CA_PEM: ${YUTONG_MQTT_TLS_CA_PEM:-} - YUTONG_MQTT_TLS_CLIENT_PEM: ${YUTONG_MQTT_TLS_CLIENT_PEM:-} - YUTONG_MQTT_TLS_CLIENT_KEY: ${YUTONG_MQTT_TLS_CLIENT_KEY:-} - YUTONG_MQTT_TLS_HOSTNAME_VERIFICATION_ENABLED: ${YUTONG_MQTT_TLS_HOSTNAME_VERIFICATION_ENABLED:-true} - IDENTITY_MYSQL_DSN: ${IDENTITY_MYSQL_DSN:-} - VEHICLE_IDENTITY_TABLE: ${VEHICLE_IDENTITY_TABLE:-vehicle_identity_binding} - ports: - - "${GO_GB32960_TCP_PORT:-32960}:32960" - - "${GO_JT808_TCP_PORT:-808}:808" - volumes: - - "${KAFKA_SPOOL_HOST_DIR:-/opt/lingniu-go/spool/gateway}:${KAFKA_SPOOL_DIR:-/data/spool/gateway}" - - "${YUTONG_MQTT_CERT_HOST_DIR:-/opt/lingniuServices/certificate/yutong/vehicledatareception}:${YUTONG_MQTT_CERT_CONTAINER_DIR:-/opt/lingniuServices/certificate/yutong/vehicledatareception}:ro" - - go-history-writer: - <<: *restart-policy - image: ${LINGNIU_GO_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com}/${LINGNIU_GO_IMAGE_NAMESPACE:-oneos}/vehicle-gateway-go:${LINGNIU_GO_IMAGE_VERSION:?set LINGNIU_GO_IMAGE_VERSION} - container_name: go-history-writer - command: ["/app/history-writer"] - mem_limit: ${GO_HISTORY_WRITER_MEM_LIMIT:-768m} - environment: - <<: *common-env - KAFKA_GROUP: ${KAFKA_GROUP_GO_HISTORY:-go-history-writer} - KAFKA_TOPICS: ${KAFKA_TOPICS_GO_HISTORY:-vehicle.raw.gb32960.v1,vehicle.raw.jt808.v1,vehicle.raw.yutong-mqtt.v1} - TDENGINE_DRIVER: ${TDENGINE_DRIVER:-taosWS} - TDENGINE_DSN: ${TDENGINE_DSN:?set TDENGINE_DSN, example root:password@ws(172.17.111.57:6041)/lingniu_vehicle_ts} - TDENGINE_DATABASE: ${TDENGINE_DATABASE:-lingniu_vehicle_ts} - TDENGINE_ENSURE_SCHEMA: ${TDENGINE_ENSURE_SCHEMA:-true} - - go-stat-writer: - <<: *restart-policy - image: ${LINGNIU_GO_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com}/${LINGNIU_GO_IMAGE_NAMESPACE:-oneos}/vehicle-gateway-go:${LINGNIU_GO_IMAGE_VERSION:?set LINGNIU_GO_IMAGE_VERSION} - container_name: go-stat-writer - command: ["/app/stat-writer"] - mem_limit: ${GO_STAT_WRITER_MEM_LIMIT:-512m} - environment: - <<: *common-env - KAFKA_GROUP: ${KAFKA_GROUP_GO_STAT:-go-stat-writer} - KAFKA_TOPICS: ${KAFKA_TOPICS_GO_STAT:-vehicle.raw.gb32960.v1,vehicle.raw.jt808.v1} - MYSQL_DSN: ${MYSQL_DSN:?set MYSQL_DSN} - MYSQL_ENSURE_SCHEMA: ${MYSQL_ENSURE_SCHEMA:-true} - LOCAL_TZ: ${LOCAL_TZ:-Asia/Shanghai} - - go-realtime-api: - <<: *restart-policy - image: ${LINGNIU_GO_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com}/${LINGNIU_GO_IMAGE_NAMESPACE:-oneos}/vehicle-gateway-go:${LINGNIU_GO_IMAGE_VERSION:?set LINGNIU_GO_IMAGE_VERSION} - container_name: go-realtime-api - command: ["/app/realtime-api"] - mem_limit: ${GO_REALTIME_API_MEM_LIMIT:-512m} - environment: - <<: *common-env - HTTP_ADDR: ":20210" - KAFKA_GROUP: ${KAFKA_GROUP_GO_REALTIME:-go-realtime-api} - KAFKA_TOPICS: ${KAFKA_TOPICS_GO_REALTIME:-vehicle.event.unified.v1} - REDIS_ADDR: ${REDIS_ADDR:-r-bp1u741kij7e51i481.redis.rds.aliyuncs.com:6379} - REDIS_USERNAME: ${REDIS_USERNAME:-} - REDIS_PASSWORD: ${REDIS_PASSWORD:?set REDIS_PASSWORD} - REDIS_DB: ${REDIS_DB:-50} - ONLINE_TTL_SECONDS: ${ONLINE_TTL_SECONDS:-600} - MYSQL_DSN: ${MYSQL_DSN:-} - TDENGINE_DRIVER: ${TDENGINE_DRIVER:-taosWS} - TDENGINE_DSN: ${TDENGINE_DSN:-} - TDENGINE_DATABASE: ${TDENGINE_DATABASE:-lingniu_vehicle_ts} - ports: - - "${GO_REALTIME_HTTP_PORT:-20210}:20210" - -networks: - vehicle-ingest: - name: vehicle-ingest diff --git a/deploy/portainer/docker-compose.yml b/deploy/portainer/docker-compose.yml deleted file mode 100644 index 5d601133..00000000 --- a/deploy/portainer/docker-compose.yml +++ /dev/null @@ -1,216 +0,0 @@ -version: "3.8" - -# Default production stack: GB32960, JT808, Yutong MQTT, history, analytics. -# Legacy and optional services stay outside this compose file. -# Set LINGNIU_IMAGE_VERSION in Portainer Stack env, for example: -# main-0.1.0-SNAPSHOT-22468e7 -# Override LINGNIU_IMAGE_REGISTRY / LINGNIU_IMAGE_NAMESPACE only when publishing to another registry. -x-common-env: &common-env - TZ: Asia/Shanghai - JAVA_OPTS: ${JAVA_OPTS:---sun-misc-unsafe-memory-access=allow -Dio.netty.transport.noNative=true -XX:+UseZGC -XX:MaxRAMPercentage=75} - NACOS_CONFIG_ENABLED: ${NACOS_CONFIG_ENABLED:-true} - NACOS_SERVER_ADDR: ${NACOS_SERVER_ADDR:-127.0.0.1:8848} - NACOS_NAMESPACE: ${NACOS_NAMESPACE:-} - NACOS_GROUP: ${NACOS_GROUP:-DEFAULT_GROUP} - NACOS_CONFIG_FILE_EXTENSION: ${NACOS_CONFIG_FILE_EXTENSION:-yml} - NACOS_REFRESH_ENABLED: ${NACOS_REFRESH_ENABLED:-true} - NACOS_USERNAME: ${NACOS_USERNAME:-} - NACOS_PASSWORD: ${NACOS_PASSWORD:-} - KAFKA_BROKERS: ${KAFKA_BROKERS:-172.17.111.56:9092} - -x-identity-mysql-env: &identity-mysql-env - VEHICLE_IDENTITY_MYSQL_JDBC_URL: ${MYSQL_JDBC_URL:-} - VEHICLE_IDENTITY_MYSQL_USERNAME: ${MYSQL_USERNAME:-} - VEHICLE_IDENTITY_MYSQL_PASSWORD: ${MYSQL_PASSWORD:-} - -x-redis-session-env: &redis-session-env - REDIS_HOST: ${REDIS_HOST:-r-bp1u741kij7e51i481.redis.rds.aliyuncs.com} - REDIS_PORT: ${REDIS_PORT:-6379} - REDIS_DATABASE: ${REDIS_DATABASE:-50} - REDIS_USERNAME: ${REDIS_USERNAME:-} - REDIS_PASSWORD: ${REDIS_PASSWORD:-} - -x-readiness-healthcheck: &readiness-healthcheck - interval: 30s - timeout: 5s - retries: 3 - start_period: 60s - -services: - gb32960-ingest-app: - image: ${LINGNIU_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com}/${LINGNIU_IMAGE_NAMESPACE:-oneos}/gb32960-ingest-app:${LINGNIU_IMAGE_VERSION:?set LINGNIU_IMAGE_VERSION} - container_name: gb32960-ingest-app - restart: unless-stopped - mem_limit: ${GB32960_MEM_LIMIT:-768m} - environment: - <<: - - *common-env - - *identity-mysql-env - - *redis-session-env - HTTP_PORT: 20100 - GB32960_PORT: 32960 - KAFKA_NODE_ID: ${KAFKA_NODE_ID:-gb32960-ingest-portainer} - KAFKA_TOPIC_GB32960_EVENT: ${KAFKA_TOPIC_GB32960_EVENT:-vehicle.event.gb32960.v1} - KAFKA_TOPIC_GB32960_RAW: ${KAFKA_TOPIC_GB32960_RAW:-vehicle.raw.gb32960.v1} - KAFKA_TOPIC_GB32960_DLQ: ${KAFKA_TOPIC_GB32960_DLQ:-vehicle.dlq.gb32960.v1} - GB32960_AUTH_ENABLED: ${GB32960_AUTH_ENABLED:-false} - GB32960_PLATFORM_USER_HYUNDAI: ${GB32960_PLATFORM_USER_HYUNDAI:-Hyundai} - GB32960_PLATFORM_PWD_HYUNDAI: ${GB32960_PLATFORM_PWD_HYUNDAI:-} - GB32960_PLATFORM_IP_HYUNDAI: ${GB32960_PLATFORM_IP_HYUNDAI:-} - GB32960_PLATFORM_USER_YUEJIN: ${GB32960_PLATFORM_USER_YUEJIN:-YueJin} - GB32960_PLATFORM_PWD_YUEJIN: ${GB32960_PLATFORM_PWD_YUEJIN:-} - GB32960_PLATFORM_IP_YUEJIN: ${GB32960_PLATFORM_IP_YUEJIN:-} - GB32960_TLS_ENABLED: ${GB32960_TLS_ENABLED:-false} - SINK_ARCHIVE_PATH: ${GB32960_ARCHIVE_PATH:-/data/archive} - ports: - - "${GB32960_HTTP_PORT:-20100}:20100" - - "${GB32960_TCP_PORT:-32960}:32960" - volumes: - - gb32960-ingest-data:/data - networks: - - vehicle-ingest - healthcheck: - <<: *readiness-healthcheck - test: ["CMD-SHELL", "curl -fsS http://127.0.0.1:20100/actuator/health/readiness >/dev/null"] - - jt808-ingest-app: - image: ${LINGNIU_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com}/${LINGNIU_IMAGE_NAMESPACE:-oneos}/jt808-ingest-app:${LINGNIU_IMAGE_VERSION:?set LINGNIU_IMAGE_VERSION} - container_name: jt808-ingest-app - restart: unless-stopped - mem_limit: ${JT808_MEM_LIMIT:-768m} - environment: - <<: - - *common-env - - *identity-mysql-env - - *redis-session-env - HTTP_PORT: 20400 - JT808_PORT: 808 - KAFKA_NODE_ID: ${KAFKA_NODE_ID_JT808:-jt808-ingest-portainer} - KAFKA_TOPIC_JT808_EVENT: ${KAFKA_TOPIC_JT808_EVENT:-vehicle.event.jt808.v1} - KAFKA_TOPIC_JT808_RAW: ${KAFKA_TOPIC_JT808_RAW:-vehicle.raw.jt808.v1} - KAFKA_TOPIC_JT808_DLQ: ${KAFKA_TOPIC_JT808_DLQ:-vehicle.dlq.jt808.v1} - SINK_ARCHIVE_PATH: ${JT808_ARCHIVE_PATH:-/data/archive} - ports: - - "${JT808_HTTP_PORT:-20400}:20400" - - "${JT808_TCP_PORT:-808}:808" - volumes: - - jt808-ingest-data:/data - networks: - - vehicle-ingest - healthcheck: - <<: *readiness-healthcheck - test: ["CMD-SHELL", "curl -fsS http://127.0.0.1:20400/actuator/health/readiness >/dev/null"] - - yutong-mqtt-app: - image: ${LINGNIU_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com}/${LINGNIU_IMAGE_NAMESPACE:-oneos}/yutong-mqtt-app:${LINGNIU_IMAGE_VERSION:?set LINGNIU_IMAGE_VERSION} - container_name: yutong-mqtt-app - restart: unless-stopped - mem_limit: ${YUTONG_MQTT_MEM_LIMIT:-512m} - environment: - <<: - - *common-env - - *identity-mysql-env - HTTP_PORT: 20500 - YUTONG_MQTT_ENABLED: ${YUTONG_MQTT_ENABLED:-false} - YUTONG_MQTT_AUTO_STARTUP: ${YUTONG_MQTT_AUTO_STARTUP:-true} - KAFKA_NODE_ID: ${KAFKA_NODE_ID_MQTT:-yutong-mqtt-portainer} - KAFKA_TOPIC_YUTONG_MQTT_EVENT: ${KAFKA_TOPIC_YUTONG_MQTT_EVENT:-vehicle.event.mqtt-yutong.v1} - KAFKA_TOPIC_YUTONG_MQTT_RAW: ${KAFKA_TOPIC_YUTONG_MQTT_RAW:-vehicle.raw.mqtt-yutong.v1} - KAFKA_TOPIC_YUTONG_MQTT_DLQ: ${KAFKA_TOPIC_YUTONG_MQTT_DLQ:-vehicle.dlq.mqtt-yutong.v1} - YUTONG_MQTT_ENDPOINT_NAME: ${YUTONG_MQTT_ENDPOINT_NAME:-yutong} - YUTONG_MQTT_URI: ${YUTONG_MQTT_URI:-} - YUTONG_MQTT_TOPIC: "${YUTONG_MQTT_TOPIC:-#}" - YUTONG_MQTT_QOS: ${YUTONG_MQTT_QOS:-1} - YUTONG_MQTT_CLIENT_ID: ${YUTONG_MQTT_CLIENT_ID:-lingniu-yutong-mqtt} - YUTONG_MQTT_USERNAME: ${YUTONG_MQTT_USERNAME:-} - YUTONG_MQTT_PASSWORD: ${YUTONG_MQTT_PASSWORD:-} - YUTONG_MQTT_CLEAN_SESSION: ${YUTONG_MQTT_CLEAN_SESSION:-false} - YUTONG_MQTT_KEEP_ALIVE_SECONDS: ${YUTONG_MQTT_KEEP_ALIVE_SECONDS:-20} - YUTONG_MQTT_CONNECTION_TIMEOUT_SECONDS: ${YUTONG_MQTT_CONNECTION_TIMEOUT_SECONDS:-10} - YUTONG_MQTT_TLS_CA_PEM: ${YUTONG_MQTT_TLS_CA_PEM:-} - YUTONG_MQTT_TLS_CLIENT_PEM: ${YUTONG_MQTT_TLS_CLIENT_PEM:-} - YUTONG_MQTT_TLS_CLIENT_KEY: ${YUTONG_MQTT_TLS_CLIENT_KEY:-} - YUTONG_MQTT_TLS_HOSTNAME_VERIFICATION_ENABLED: ${YUTONG_MQTT_TLS_HOSTNAME_VERIFICATION_ENABLED:-true} - SINK_ARCHIVE_PATH: ${YUTONG_MQTT_ARCHIVE_PATH:-/data/archive} - ports: - - "${YUTONG_MQTT_HTTP_PORT:-20500}:20500" - volumes: - - yutong-mqtt-data:/data - networks: - - vehicle-ingest - healthcheck: - <<: *readiness-healthcheck - test: ["CMD-SHELL", "curl -fsS http://127.0.0.1:20500/actuator/health/readiness >/dev/null"] - - vehicle-history-app: - image: ${LINGNIU_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com}/${LINGNIU_IMAGE_NAMESPACE:-oneos}/vehicle-history-app:${LINGNIU_IMAGE_VERSION:?set LINGNIU_IMAGE_VERSION} - container_name: vehicle-history-app - restart: unless-stopped - mem_limit: ${VEHICLE_HISTORY_MEM_LIMIT:-1536m} - environment: - <<: *common-env - HTTP_PORT: 20200 - KAFKA_CONSUMER_ENABLED: ${KAFKA_CONSUMER_ENABLED_HISTORY:-true} - KAFKA_CONSUMER_CLIENT_ID_PREFIX: ${KAFKA_CONSUMER_CLIENT_ID_PREFIX_HISTORY:-vehicle-history} - KAFKA_CONSUMER_MAX_POLL_RECORDS: ${KAFKA_CONSUMER_MAX_POLL_RECORDS_HISTORY:-2000} - KAFKA_CONSUMER_MAX_POLL_INTERVAL_MILLIS: ${KAFKA_CONSUMER_MAX_POLL_INTERVAL_MILLIS_HISTORY:-1800000} - KAFKA_TOPIC_GB32960_EVENT: ${KAFKA_TOPIC_GB32960_EVENT:-vehicle.event.gb32960.v1} - KAFKA_TOPIC_GB32960_RAW: ${KAFKA_TOPIC_GB32960_RAW:-vehicle.raw.gb32960.v1} - KAFKA_TOPIC_JT808_EVENT: ${KAFKA_TOPIC_JT808_EVENT:-vehicle.event.jt808.v1} - KAFKA_TOPIC_JT808_RAW: ${KAFKA_TOPIC_JT808_RAW:-vehicle.raw.jt808.v1} - KAFKA_TOPIC_YUTONG_MQTT_EVENT: ${KAFKA_TOPIC_YUTONG_MQTT_EVENT:-vehicle.event.mqtt-yutong.v1} - KAFKA_TOPIC_YUTONG_MQTT_RAW: ${KAFKA_TOPIC_YUTONG_MQTT_RAW:-vehicle.raw.mqtt-yutong.v1} - KAFKA_TOPIC_HISTORY_DLQ: ${KAFKA_TOPIC_HISTORY_DLQ:-vehicle.dlq.history.v1} - KAFKA_GROUP_HISTORY: ${KAFKA_GROUP_HISTORY:-vehicle-history} - TDENGINE_HISTORY_ENABLED: ${TDENGINE_HISTORY_ENABLED:-true} - TDENGINE_JDBC_URL: ${TDENGINE_JDBC_URL:-jdbc:TAOS-WS://172.17.111.57:6041/vehicle_ts} - TDENGINE_HISTORY_DATABASE: ${TDENGINE_HISTORY_DATABASE:-vehicle_ts} - TDENGINE_USERNAME: ${TDENGINE_USERNAME:-root} - TDENGINE_PASSWORD: ${TDENGINE_PASSWORD:-taosdata} - TDENGINE_CONNECTION_TIMEOUT_MILLIS: ${TDENGINE_CONNECTION_TIMEOUT_MILLIS:-30000} - TDENGINE_MIN_IDLE: ${TDENGINE_MIN_IDLE:-1} - TDENGINE_MAX_POOL_SIZE: ${TDENGINE_MAX_POOL_SIZE:-32} - ports: - - "${VEHICLE_HISTORY_HTTP_PORT:-20200}:20200" - networks: - - vehicle-ingest - healthcheck: - <<: *readiness-healthcheck - test: ["CMD-SHELL", "curl -fsS http://127.0.0.1:20200/actuator/health/readiness >/dev/null"] - - vehicle-analytics-app: - image: ${LINGNIU_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com}/${LINGNIU_IMAGE_NAMESPACE:-oneos}/vehicle-analytics-app:${LINGNIU_IMAGE_VERSION:?set LINGNIU_IMAGE_VERSION} - container_name: vehicle-analytics-app - restart: unless-stopped - mem_limit: ${VEHICLE_ANALYTICS_MEM_LIMIT:-1024m} - environment: - <<: *common-env - HTTP_PORT: 20310 - KAFKA_CONSUMER_ENABLED: ${KAFKA_CONSUMER_ENABLED_ANALYTICS:-true} - KAFKA_CONSUMER_CLIENT_ID_PREFIX: ${KAFKA_CONSUMER_CLIENT_ID_PREFIX_ANALYTICS:-vehicle-analytics} - KAFKA_CONSUMER_MAX_POLL_INTERVAL_MILLIS: ${KAFKA_CONSUMER_MAX_POLL_INTERVAL_MILLIS_ANALYTICS:-900000} - KAFKA_TOPIC_JT808_EVENT: ${KAFKA_TOPIC_JT808_EVENT:-vehicle.event.jt808.v1} - KAFKA_TOPIC_JT808_DLQ: ${KAFKA_TOPIC_JT808_DLQ:-vehicle.dlq.jt808.v1} - KAFKA_GROUP_STAT: ${KAFKA_GROUP_STAT:-vehicle-stat} - VEHICLE_STAT_ENABLED: ${VEHICLE_STAT_ENABLED:-true} - VEHICLE_STAT_ZONE_ID: ${VEHICLE_STAT_ZONE_ID:-Asia/Shanghai} - VEHICLE_STAT_JT808_MILEAGE_ENABLED: ${VEHICLE_STAT_JT808_MILEAGE_ENABLED:-true} - MYSQL_JDBC_URL: ${MYSQL_JDBC_URL:-} - MYSQL_USERNAME: ${MYSQL_USERNAME:-} - MYSQL_PASSWORD: ${MYSQL_PASSWORD:-} - ports: - - "${VEHICLE_ANALYTICS_HTTP_PORT:-20310}:20310" - networks: - - vehicle-ingest - healthcheck: - <<: *readiness-healthcheck - test: ["CMD-SHELL", "curl -fsS http://127.0.0.1:20310/actuator/health/readiness >/dev/null"] - -networks: - vehicle-ingest: - name: vehicle-ingest - -volumes: - gb32960-ingest-data: - jt808-ingest-data: - yutong-mqtt-data: diff --git a/deploy/systemd/README.md b/deploy/systemd/README.md deleted file mode 100644 index 3a6f16b2..00000000 --- a/deploy/systemd/README.md +++ /dev/null @@ -1,66 +0,0 @@ -# Go Vehicle Gateway Native Deployment - -本目录用于 ECS 原生部署。当前 goal 后续不再使用 Docker/Portainer 作为 Go 接入链路的部署方式。 - -## Runtime Layout - -```text -/opt/lingniu-go-native/ - current -> /opt/lingniu-go-native/releases/ - releases// - gateway - history-writer - stat-writer - realtime-api - env/ - gateway.env - history-writer.env - stat-writer.env - realtime-api.env - spool/gateway/ -``` - -## Services - -| systemd unit | Binary | Purpose | -|---|---|---| -| `lingniu-go-gateway.service` | `gateway` | GB32960 TCP `32960`、JT808 TCP `808`、宇通 MQTT 接入，写 Kafka RAW/unified | -| `lingniu-go-history-writer.service` | `history-writer` | 消费 RAW topic，写 TDengine `raw_frames` 和核心时序表 | -| `lingniu-go-stat-writer.service` | `stat-writer` | 消费 RAW topic，写 MySQL `vehicle_daily_metric` | -| `lingniu-go-realtime-api.service` | `realtime-api` | 消费 unified topic，写 Redis，并提供 realtime/raw/stat 查询 API | - -## Build - -在开发机或 CI 上构建 Linux amd64 二进制： - -```bash -cd go/vehicle-gateway -CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' -o /tmp/lingniu-go-native/gateway ./cmd/gateway -CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' -o /tmp/lingniu-go-native/history-writer ./cmd/history-writer -CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' -o /tmp/lingniu-go-native/stat-writer ./cmd/stat-writer -CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' -o /tmp/lingniu-go-native/realtime-api ./cmd/realtime-api -``` - -## Cutover Notes - -1. 先上传二进制到新 release 目录并更新 `current` symlink。 -2. 从现有生产 env 生成四个 systemd 专用 env 文件；不要把密钥写入仓库。 -3. 停止旧 Docker Go 容器或 Java 容器，释放 `808`、`32960`、`20210`。 -4. 执行： - -```bash -systemctl daemon-reload -systemctl enable lingniu-go-gateway lingniu-go-history-writer lingniu-go-stat-writer lingniu-go-realtime-api -systemctl restart lingniu-go-gateway lingniu-go-history-writer lingniu-go-stat-writer lingniu-go-realtime-api -``` - -## Verification - -```bash -systemctl is-active lingniu-go-gateway lingniu-go-history-writer lingniu-go-stat-writer lingniu-go-realtime-api -ss -lntp | egrep ':(808|32960|20210)\b' -journalctl -u lingniu-go-gateway --since '5 minutes ago' --no-pager -curl -sS 'http://127.0.0.1:20210/api/history/raw-frames?protocol=JT808&limit=1' -curl -sS 'http://127.0.0.1:20210/api/history/raw-frames?protocol=GB32960&limit=1' -curl -sS 'http://127.0.0.1:20210/api/history/raw-frames?protocol=YUTONG_MQTT&limit=1' -``` diff --git a/deploy/systemd/lingniu-go-gateway.service b/deploy/systemd/lingniu-go-gateway.service deleted file mode 100644 index 40ef12f2..00000000 --- a/deploy/systemd/lingniu-go-gateway.service +++ /dev/null @@ -1,18 +0,0 @@ -[Unit] -Description=Lingniu Go Vehicle Gateway -After=network-online.target -Wants=network-online.target - -[Service] -Type=simple -WorkingDirectory=/opt/lingniu-go-native/current -EnvironmentFile=/opt/lingniu-go-native/env/gateway.env -ExecStart=/opt/lingniu-go-native/current/gateway -Restart=always -RestartSec=3 -LimitNOFILE=1048576 -KillSignal=SIGTERM -TimeoutStopSec=30 - -[Install] -WantedBy=multi-user.target diff --git a/deploy/systemd/lingniu-go-history-writer.service b/deploy/systemd/lingniu-go-history-writer.service deleted file mode 100644 index b08bed08..00000000 --- a/deploy/systemd/lingniu-go-history-writer.service +++ /dev/null @@ -1,18 +0,0 @@ -[Unit] -Description=Lingniu Go History Writer -After=network-online.target -Wants=network-online.target - -[Service] -Type=simple -WorkingDirectory=/opt/lingniu-go-native/current -EnvironmentFile=/opt/lingniu-go-native/env/history-writer.env -ExecStart=/opt/lingniu-go-native/current/history-writer -Restart=always -RestartSec=3 -LimitNOFILE=1048576 -KillSignal=SIGTERM -TimeoutStopSec=30 - -[Install] -WantedBy=multi-user.target diff --git a/deploy/systemd/lingniu-go-realtime-api.service b/deploy/systemd/lingniu-go-realtime-api.service deleted file mode 100644 index 6ef81160..00000000 --- a/deploy/systemd/lingniu-go-realtime-api.service +++ /dev/null @@ -1,18 +0,0 @@ -[Unit] -Description=Lingniu Go Realtime API -After=network-online.target -Wants=network-online.target - -[Service] -Type=simple -WorkingDirectory=/opt/lingniu-go-native/current -EnvironmentFile=/opt/lingniu-go-native/env/realtime-api.env -ExecStart=/opt/lingniu-go-native/current/realtime-api -Restart=always -RestartSec=3 -LimitNOFILE=1048576 -KillSignal=SIGTERM -TimeoutStopSec=30 - -[Install] -WantedBy=multi-user.target diff --git a/deploy/systemd/lingniu-go-stat-writer.service b/deploy/systemd/lingniu-go-stat-writer.service deleted file mode 100644 index 6b46c108..00000000 --- a/deploy/systemd/lingniu-go-stat-writer.service +++ /dev/null @@ -1,18 +0,0 @@ -[Unit] -Description=Lingniu Go Stat Writer -After=network-online.target -Wants=network-online.target - -[Service] -Type=simple -WorkingDirectory=/opt/lingniu-go-native/current -EnvironmentFile=/opt/lingniu-go-native/env/stat-writer.env -ExecStart=/opt/lingniu-go-native/current/stat-writer -Restart=always -RestartSec=3 -LimitNOFILE=1048576 -KillSignal=SIGTERM -TimeoutStopSec=30 - -[Install] -WantedBy=multi-user.target diff --git a/deploy/tools/EnsureKafkaTopics.java b/deploy/tools/EnsureKafkaTopics.java deleted file mode 100644 index e99e0c42..00000000 --- a/deploy/tools/EnsureKafkaTopics.java +++ /dev/null @@ -1,86 +0,0 @@ -import java.time.Duration; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; -import java.util.Properties; -import java.util.Set; -import java.util.concurrent.ExecutionException; -import java.util.stream.Collectors; -import org.apache.kafka.clients.admin.AdminClient; -import org.apache.kafka.clients.admin.AdminClientConfig; -import org.apache.kafka.clients.admin.NewPartitions; -import org.apache.kafka.clients.admin.NewTopic; -import org.apache.kafka.clients.admin.TopicDescription; - -public final class EnsureKafkaTopics { - private static final short REPLICATION_FACTOR = 1; - - private record TopicSpec(String name, int partitions) {} - - private EnsureKafkaTopics() {} - - public static void main(String[] args) throws Exception { - String brokers = args.length > 0 ? args[0] : "172.17.111.56:9092"; - List specs = - List.of( - topic("vehicle.event.gb32960.v1", 12), - topic("vehicle.raw.gb32960.v1", 12), - topic("vehicle.dlq.gb32960.v1", 3), - topic("vehicle.event.jt808.v1", 12), - topic("vehicle.raw.jt808.v1", 12), - topic("vehicle.dlq.jt808.v1", 3), - topic("vehicle.event.mqtt-yutong.v1", 12), - topic("vehicle.raw.mqtt-yutong.v1", 12), - topic("vehicle.dlq.mqtt-yutong.v1", 3)); - - Properties props = new Properties(); - props.put(AdminClientConfig.BOOTSTRAP_SERVERS_CONFIG, brokers); - props.put(AdminClientConfig.REQUEST_TIMEOUT_MS_CONFIG, "15000"); - props.put(AdminClientConfig.DEFAULT_API_TIMEOUT_MS_CONFIG, "30000"); - - try (AdminClient admin = AdminClient.create(props)) { - Set existing = admin.listTopics().names().get(); - List missing = - specs.stream() - .filter(spec -> !existing.contains(spec.name())) - .map(spec -> new NewTopic(spec.name(), spec.partitions(), REPLICATION_FACTOR)) - .toList(); - if (!missing.isEmpty()) { - try { - admin.createTopics(missing).all().get(); - } catch (ExecutionException e) { - System.out.println("createTopics result=" + e.getCause()); - } - } - - Map byName = - specs.stream().collect(Collectors.toMap(TopicSpec::name, spec -> spec)); - Map descriptions = - admin.describeTopics(byName.keySet()).allTopicNames().get(); - - Map increases = - descriptions.entrySet().stream() - .filter(entry -> entry.getValue().partitions().size() < byName.get(entry.getKey()).partitions()) - .collect( - Collectors.toMap( - Map.Entry::getKey, - entry -> NewPartitions.increaseTo(byName.get(entry.getKey()).partitions()))); - if (!increases.isEmpty()) { - admin.createPartitions(increases).all().get(); - Thread.sleep(Duration.ofSeconds(2).toMillis()); - descriptions = admin.describeTopics(byName.keySet()).allTopicNames().get(); - } - - List lines = new ArrayList<>(); - descriptions.entrySet().stream() - .sorted(Map.Entry.comparingByKey()) - .forEach(entry -> lines.add(entry.getKey() + " partitions=" + entry.getValue().partitions().size())); - System.out.println("bootstrap=" + brokers); - lines.forEach(System.out::println); - } - } - - private static TopicSpec topic(String name, int partitions) { - return new TopicSpec(name, partitions); - } -} diff --git a/docs/module-data-flow.html b/docs/module-data-flow.html deleted file mode 100644 index 377abbc0..00000000 --- a/docs/module-data-flow.html +++ /dev/null @@ -1,749 +0,0 @@ - - - - - - lingniu-vehicle-ingest 模块与数据流 - - - -
-

lingniu-vehicle-ingest 模块与数据流

-

- 本图基于当前 Maven 多模块项目整理，用于后续讨论协议接入、内部字段、氢能车辆运营统计、安全告警和下游消费边界。 - 当前架构将接入层、历史明细、Redis 热状态和日统计拆成独立模块，通过统一 Kafka Envelope 解耦。 -

-
- -
-
-

一、系统定位

-

- 当前项目不是业务库写入服务。它的核心职责是把 GB/T 32960、JT/T 808、Yutong MQTT - 等默认生产来源的车辆数据统一转换为 VehicleEvent 和全字段 TelemetrySnapshot； - JT/T 1078、JSATL12 作为显式 profile 的可选能力保留， - 再通过 Kafka、原始报文归档、TDengine 历史索引、Redis 热状态和独立统计模块交给下游业务。 -

-
-
- 接入方式 - Netty TCP、MQTT、后续可扩展更多 Inbound Adapter；信达 Push 源码已删除。 -
-
- 统一模型 - 所有协议归一到 VehicleEvent 和全字段内部 TelemetrySnapshot。 -
-
- 输出方式 - Kafka Protobuf Envelope + 原始 bytes 冷存 + TDengine raw_frames/locations + Redis 热状态。 -
-
- 业务边界 - 日统计由 vehicle-stat-service 消费 Kafka 实现，告警工单、资产业务继续由下游消费。 -
-
-
- -
-

二、模块分层图

-
- 
-flowchart TB
-  subgraph external["外部来源"]
-    vehicle["车辆终端
GB/T 32960 / JT808 / JT1078"]
-    mqttSource["车企或平台 MQTT"]
-    commandClient["业务系统 / 运维平台
HTTP 下行命令"]
-  end
-
-  subgraph protocolModules["协议模块 modules/protocols"]
-    gb["protocol-gb32960
32960 Netty 接入、鉴权、ACK、解析"]
-    jt808["protocol-jt808
808 Netty 接入、会话、位置/批量位置、媒体、透传、下行"]
-    jt1078["protocol-jt1078
1078 信令 + TCP/UDP RTP 媒体流归档"]
-    jsatl12["protocol-jsatl12
主动安全附件流接入"]
-  end
-
-  subgraph inbound["入口适配 modules/inbound"]
-    mqtt["inbound-mqtt
MQTT endpoint 生命周期、PEM TLS"]
-    mqttProfile["MqttProfileRegistry
按 endpoint profile 解析/映射"]
-  end
-
-  subgraph protocolBase["核心与公共能力 modules/core"]
-    codec["ingest-codec-common
BCD / CRC / BCC / bit 工具"]
-    session["session-core
设备会话、命令分发接口、会话存储"]
-    identity["vehicle-identity
跨协议身份解析、MySQL 绑定表"]
-    obs["observability
metrics / health"]
-    api["ingest-api
ProtocolId / RawFrame / VehicleEvent / 注解 SPI"]
-    registry["HandlerRegistry
按协议、命令、infoType 路由"]
-    dispatcher["Dispatcher
RawFrame 到 Handler 到 EventBus"]
-    bus["DisruptorEventBus
高吞吐事件发布"]
-  end
-
-  subgraph sink["输出与明细存储 modules/sinks"]
-    kafka["sink-kafka
VehicleEvent 到 Protobuf Envelope 到 Kafka"]
-    archive["sink-archive
RawArchive 到本地文件系统冷存"]
-    tdengineStore["tdengine-history-store
TDengine raw_frames + locations"]
-  end
-
-  subgraph services["消费服务 modules/services"]
-    history["event-history-service
Kafka event/raw 到 TDengine 历史查询"]
-    state["vehicle-state-service
可选 Redis 热状态
optional-latest-state"]
-    stat["vehicle-stat-service
Kafka 全字段事件到指标表"]
-  end
-
-  subgraph apps["应用入口 modules/apps"]
-    gbApp["gb32960-ingest-app
GB32960 TCP 接入"]
-    jtApp["jt808-ingest-app
JT808 TCP 接入"]
-    yutongApp["yutong-mqtt-app
宇通 MQTT 接入"]
-    historyApp["vehicle-history-app
历史查询与 TDengine 写入"]
-    analyticsApp["vehicle-analytics-app
808 日里程指标消费"]
-    gateway["command-gateway
可选 HTTP 设备命令入口"]
-  end
-
-  vehicle --> gbApp
-  vehicle --> jtApp
-  vehicle --> jt1078
-  vehicle --> jsatl12
-  mqttSource --> yutongApp
-  gbApp --> gb
-  jtApp --> jt808
-  yutongApp --> mqtt
-  mqtt --> mqttProfile
-  mqttProfile --> api
-  commandClient --> gateway
-
-  gb --> codec
-  jt808 --> codec
-  jt1078 --> codec
-  jsatl12 --> codec
-
-  gb --> api
-  jt808 --> api
-  jsatl12 --> archive
-  jsatl12 --> jt808
-
-  api --> registry
-  registry --> dispatcher
-  dispatcher --> bus
-  bus --> kafka
-  bus --> archive
-  kafka --> historyApp
-  kafka --> analyticsApp
-  historyApp --> history
-  analyticsApp --> stat
-  kafka -.可选热状态.-> state
-  history --> tdengineStore
-
-  gateway --> session
-  session --> jt808
-  session --> gb
-  identity --> jt808
-  identity --> mqttProfile
-
-  obs -.监控.-> dispatcher
-  obs -.监控.-> bus
-  obs -.监控.-> kafka
-
-  classDef entry fill:#eff6ff,stroke:#2563eb,color:#1e3a8a;
-  classDef core fill:#ecfdf3,stroke:#0f766e,color:#134e48;
-  classDef sink fill:#fffbeb,stroke:#b45309,color:#7c2d12;
-  classDef command fill:#f5f3ff,stroke:#6d28d9,color:#4c1d95;
-  classDef support fill:#f8fafc,stroke:#64748b,color:#334155;
-
-  class gb,jt808,jt1078,jsatl12,mqtt,mqttProfile,gbApp,jtApp,yutongApp,historyApp,analyticsApp entry;
-  class api,registry,dispatcher,bus,identity core;
-  class kafka,archive,tdengineStore sink;
-  class gateway,session command;
-  class codec,obs support;
-
-
-
- 入口/协议模块 - 核心管线 - 输出层 - 下行命令/会话 -
-
- -
-

三、上行数据流转

-
- 
-sequenceDiagram
-  autonumber
-  participant Device as 车辆/平台
-  participant Inbound as Inbound Adapter
Netty/MQTT
-  participant Decoder as FrameDecoder
MessageDecoder
-  participant Dispatcher as Dispatcher
-  participant Handler as Protocol Handler
Mapper
-  participant EventBus as DisruptorEventBus
-  participant Kafka as sink-kafka
Kafka Envelope
-  participant Archive as sink-archive
ArchiveStore
-  participant History as vehicle-history-app
-  participant TDengine as tdengine-history-store
raw_frames + locations
-  participant Consumer as 下游业务消费者
-
-  Device->>Inbound: 原始报文 bytes / MQTT message
-  Inbound->>Decoder: 粘包拆帧、校验、协议体解析
-  Inbound->>Inbound: MQTT 按 endpoint profile 选择厂商解析器
-  Decoder->>Dispatcher: RawFrame(protocol, command, infoType, payload, rawBytes)
-  Dispatcher->>EventBus: 先发布 RawArchive 事件
-  EventBus->>Archive: 保存原始 bytes，生成可回放材料
-  Dispatcher->>Handler: 根据 protocol + command + infoType 路由
-  Handler->>Handler: 协议字段映射为内部字段
-  Dispatcher->>Handler: 绑定 rawArchiveKey / rawArchiveUri
-  Handler->>EventBus: VehicleEvent(Realtime / Location / Alarm / Login ...)
-  EventBus->>Kafka: 投递规整后的业务事件
-  Kafka->>Kafka: EnvelopeMapper 转 Protobuf
-  Kafka->>History: event/raw topic，key=vin，单车有序
-  History->>TDengine: 写 raw_frames、locations 和 raw parsed JSON
-  TDengine->>Consumer: 分页历史查询、原始帧追溯、位置查询
-  Kafka->>Consumer: Redis 热状态、指标统计、告警工单等下游消费
-
-
-
- 关键边界： - 协议字段只在协议模块内解释，出了 Mapper 以后都应该使用内部字段。统计每日里程、每日用电量、每日用氢量、储氢安全和氢泄露告警时，应消费全字段 TelemetrySnapshot，不要直接依赖某个协议的原始字段名。 -
-
- 原始追溯： - Dispatcher 为所有带 rawBytes 的 RawFrame 生成统一的 rawArchiveKey，业务事件 metadata 使用 rawArchiveUri=archive://... 指向同一份原始 bytes。归档文件的真实本地路径只属于 ArchiveStore，查询、导出和 Kafka Envelope 使用逻辑 URI 解耦存储实现。 -
-
- -
-

四、32960 细化链路

-
- 
-flowchart LR
-  terminal["氢能车 TBOX
GB/T 32960"] --> netty["Gb32960NettyServer
TCP / TLS / Idle 检测"]
-  netty --> access["Gb32960AccessService
VIN 白名单 / 平台登录鉴权"]
-  netty --> frame["Gb32960FrameDecoder
拆帧、转义、校验"]
-  frame --> decoder["Gb32960MessageDecoder
Header + Body 解析"]
-  decoder --> diag["Gb32960FrameDiagnostics
首帧、Raw 块、异常诊断"]
-  decoder --> handler["Gb32960ChannelHandler
ACK / NACK / dispatch"]
-  handler --> ack["Gb32960AckService
登录应答、失败断开"]
-  handler --> dispatcher["Dispatcher"]
-  dispatcher --> rtHandler["Gb32960RealtimeHandler"]
-  rtHandler --> mapper["Gb32960EventMapper
协议字段到内部字段"]
-  mapper --> realtime["RealtimePayload
速度、里程、电量、氢量、压力、温度"]
-  mapper --> alarm["AlarmPayload
安全分类、氢泄露、告警等级"]
-  mapper --> login["Login / Logout / Heartbeat"]
-  realtime --> bus["DisruptorEventBus"]
-  alarm --> bus
-  login --> bus
-  bus --> kafka["Kafka Envelope"]
-  bus --> archive["RawArchive 冷存"]
-
-  classDef safety fill:#fff1f3,stroke:#b42318,color:#7a271a;
-  classDef core fill:#ecfdf3,stroke:#0f766e,color:#134e48;
-  classDef entry fill:#eff6ff,stroke:#2563eb,color:#1e3a8a;
-  classDef sink fill:#fffbeb,stroke:#b45309,color:#7c2d12;
-
-  class terminal,netty,frame,decoder,handler entry;
-  class access,diag,ack,dispatcher,rtHandler,mapper,bus core;
-  class alarm safety;
-  class kafka,archive sink;
-
-
-
- -
-

五、模块职责表

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
模块职责主要输入主要输出业务开发关注点
modules/core/ingest-api定义系统边界：协议 ID、RawFrame、IngestContext、VehicleEvent、Payload、Handler 注解、Sink SPI；ProtocolId 提供 UNKNOWN 隔离桶，供消费侧兼容未来协议或脏数据，不作为正式接入协议使用；consumer 包定义 EnvelopeIngestor、EnvelopeConsumerProcessor 和 EnvelopeDeadLetterSink，统一 Kafka 消费结果与死信边界。无直接外部输入。全项目共享的接口和内部事件模型。内部字段定义新增事件类型字段兼容性
modules/core/ingest-codec-common公共编解码工具，承载 BCD、CRC、BCC、bit 操作等协议底层能力。协议模块传入的 byte、bit、校验材料。解析辅助结果。协议基础能力
modules/core/ingest-core核心管线。扫描 Handler，按协议和命令路由，执行拦截器，发布 VehicleEvent 到 Disruptor。RawFrame。VehicleEvent、RawArchive。吞吐路由原始可回放
modules/core/session-core设备会话、命令下发抽象、SessionStore、CommandDispatcher 默认实现。SessionStore 仅保留 Redis 后端；协议进程本地只保存真实 Netty Channel，Redis 保存 sessionId、VIN 和 phone 三索引及 TTL，协议模块只依赖 SPI，不感知存储细节。设备连接、命令请求。会话状态、命令分发结果、Redis 会话索引。在线状态下行控制多实例会话索引
modules/core/vehicle-identity跨协议车辆身份解析，维护 phone、deviceId、plate 到 VIN 的绑定关系；生产运行使用 MySQL 绑定表，避免重启丢失绑定并保证多实例一致。协议模块传入的外部标识。稳定 VIN、解析来源、是否已命中绑定、MySQL 绑定记录。统计主键多协议归一持久化绑定
modules/core/observability监控、指标、健康检查等横切能力。核心管线和 Sink 运行状态。Micrometer 指标、健康信息。可用性延迟监控
modules/protocols/protocol-gb32960GB/T 32960 接入、鉴权、ACK、报文解析、实时数据和告警映射。32960 TCP 报文。Realtime、Location、Alarm、Login、Logout、Heartbeat、RawFrame。氢泄露储罐安全电量/氢量里程
modules/protocols/protocol-jt808JT/T 808 接入、注册/鉴权/心跳/注销、位置/位置附加项、批量位置、参数/属性、多媒体、透传、未知上行兜底透传、分帧边界异常和协议解析异常坏帧兜底、下行命令和超长下行分包能力；0x0200 位置附加项 0x01 总里程按 0.1km 解码为内部字段 total_mileage_km，后续每日里程统计不依赖 808 原始字段；0x0102 鉴权帧解析会保留完整 token，并尽量提取 IMEI 和软件版本；注册和鉴权会话均使用共享身份解析后的内部 VIN，避免 deviceId/IMEI 污染后续会话和下行边界；终端连接断开时同步解绑 Channel 并清理 SessionStore 会话，避免 command-gateway 误判离线车辆仍可下行；正常上行 RawArchive 和业务事件 metadata 均使用共享身份解析后的内部 vin，并保留 phone、identityResolved、identitySource 便于冷存和业务事件按同一车辆查询；无法解析终端身份的坏帧 RawArchive 和 Passthrough 均按 vin=unknown、identityResolved=false、identitySource=UNKNOWN 标记；0x0900 透传事件按原始消息 ID 归类，透传类型写入 passthroughType metadata。808 TCP 报文。Location、Login、Logout、Heartbeat、MediaMeta、Passthrough、未知上行 Raw 兜底、坏帧 Passthrough、会话状态、下行响应。GPS 位置在线状态媒体证据命令链路
modules/protocols/protocol-jt1078JT/T 1078 音视频能力。信令在 JT808 mapper 存在时桥接复用 JT808 连接和包头；0x1005 乘客流量保留原始 body，同时结构化 channelId、startTime、endTime、passengerGetOn、passengerGetOff metadata 便于查询/导出；0x1205 文件列表保留原始 body，同时结构化 responseSerialNo、fileCount 摘要；下行信令编码由本模块提供，覆盖 0x1003 音视频属性查询、0x9101 实时预览、0x9102 实时控制、0x9201 历史回放、0x9202 回放控制、0x9205 资源列表查询、0x9206 文件上传和 0x9207 文件上传控制，命令发送仍由 command-gateway 经 CommandDispatcher 复用 JT808 在线通道；TCP/UDP RTP 媒体流独立端口接入，生产默认端口按旧接收服务对齐为 11078，按 VIN/通道/时间分段写 ArchiveStore，Kafka 只发送 MediaMeta 引用，事件 metadata 暴露内部 vin、sim、channelId、dataType、packetType、segment、sequence、segmentSizeBytes、archiveKey 和 archiveRef，MediaMeta.sizeBytes 同步当前片段大小，MediaMeta 按 archiveKey 去重，避免同一 SIM 运行中从 fallback 身份切换到内部 VIN 时漏发新归档引用，便于文件明细库追踪、展示和导出；超长/短包/坏魔数等 RTP 入口异常和媒体归档失败都会兜底产出 Passthrough 错误事件，坏 RTP 统一走 Dispatcher 产出 RawArchive 和 Passthrough，统一标记 parseError 和 parseErrorMessage，并保留内部 vin、原始 bytes、peer、长度、原因、RawArchive 引用、segment 和 archiveKey 便于排障；坏 RTP 头部可读时会提取 SIM 并通过共享身份服务映射 VIN。1078 信令报文、TCP/UDP RTP 媒体流。MediaMeta、乘客流量 Passthrough、文件列表 Passthrough、坏 RTP/归档失败 Passthrough、归档媒体分段。视频扩展信令事件化大流冷存
modules/protocols/protocol-jsatl12苏标主动安全报警附件接入（仅 optional-attachments profile 显式构建，不属于默认生产面）。依赖 ArchiveStore 和 JT808 decoder；端口按旧接收服务对齐为 7612；附件 DataPacket 按旧服务真实布局 01cd + 50B文件名 + offset + length + data 分帧解析，不再把文件名前 4 字节误判为帧长；附件数据写 ArchiveStore 后通过 Dispatcher 产出 MediaMeta 引用事件，事件 metadata 保留 fileName、fileOffset、declaredChunkSizeBytes、archiveKey 和 archiveRef，便于附件明细查询、导出和补传诊断；内置附件状态机处理 T1210 文件清单、DataPacket 分块区间和 T1212 上传完成消息，数据分块会按 T1210 文件清单中的文件名继承手机号并通过共享身份解析器得到内部 VIN，入口 RawFrame、ArchivedChunk 和 MediaMeta 均保留 phone、vin、identityResolved、identitySource，只有找不到文件归属时才标记 UNKNOWN；按文件名合并无身份数据分块并返回 0x9212 完成/补传应答，补传应答包含缺失 offset/length 区间；附件归档失败会产出 JSATL12 Passthrough 错误事件并保留原始分块；JT_MESSAGE 信令复用 JT808 decoder 后进入 Dispatcher，桥接 RawFrame 使用共享身份解析后的内部 VIN并保留 phone、identityResolved、identitySource，坏 JT 信令入口 RawFrame 同样标记 UNKNOWN 身份；超长/畸形附件帧、未闭合超长 JT 信令和坏 JT 信令都以 JSATL12 Passthrough 兜底，附件帧异常同时标记 frameError 与统一 parseError/parseErrorMessage，保留原始字节、peer、长度和错误元数据，帧长和 worker 线程支持部署配置。主动安全附件流、JT808 信令帧。归档文件、MediaMeta、JT808 统一事件、坏帧/坏信令/归档失败 Passthrough。附件归档报警证据信令复用错误隔离
modules/inbound/inbound-mqttMQTT 多 endpoint 生命周期管理，支持 PEM CA、客户端证书/私钥双向 TLS，按 endpoint profile 路由到厂商解析/映射实现，默认提供宇通 profile；生产默认接入口径参考旧接收服务：ssl://cpxlm.axxc.cn:38883、topic /ytforward/shln/+、QoS 2、cleanSession=false、keepAlive=20s、connectTimeout=10s，用户名和密码仍通过环境变量注入；入站 RawFrame 和事件映射均接入统一车辆身份解析，解析器保留 externalVin、phone、mqttDeviceId、plateNo，优先按设备号/终端 ID/IMEI 绑定解析内部 VIN，未绑定且字段形态为 VIN 时再作为显式 VIN 兜底，冷存和事件 metadata 均保留这些外部标识、identityResolved、identitySource；未知 profile、解析失败、profile 运行时异常、endpoint 初始化/连接/订阅失败均兜底为 Passthrough，解析失败诊断和 profile 异常诊断都会同步写入 RawArchive metadata，并在最终 Passthrough 事件保留 parseErrorMessage 或 operational reason，无法解析设备身份的兜底事件和运行类归档都按 vin=unknown、identityResolved=false、identitySource=UNKNOWN 标记，保留原始 payload 或 operational 错误信息便于排障和归档追溯。MQTT topic/message、endpoint profile、TLS PEM 配置。统一 RawFrame、Realtime、Location、Passthrough。车企平台接入profile 扩展身份映射双向 TLS错误隔离
modules/sinks/sink-kafkaKafka Sink。把 VehicleEvent 转成 Protobuf Envelope，按 VIN 分区投递；业务事件携带全字段 TelemetrySnapshot 和 rawArchiveUri，显式 RawArchive Envelope 会填充 archive:// 逻辑 URI 与 size，默认 Kafka Sink 仍不发送原始 bytes 本体；同时提供 KafkaEnvelopeConsumerFactory、KafkaEnvelopeConsumerRunner、KafkaEnvelopeConsumerWorker 和 KafkaEnvelopeDeadLetterSink，统一服务侧 Kafka 消费启动、独立 group 绑定、死信发布和 topic 到 EnvelopeConsumerProcessor 的分发。VehicleEvent。Kafka Protobuf 消息、消费侧 DLQ 记录。单车有序Schema 演进安全字段下发给消费者
modules/sinks/sink-archive原始报文冷存，当前为本地文件系统实现；写入键与业务事件 metadata 中的 rawArchiveKey 保持一致。RawArchive 事件、附件流。可回放原始文件、archive://... 逻辑引用。问题排查合规留痕
modules/sinks/tdengine-history-store生产历史库边界，负责 TDengine schema、raw_frames/location 行映射、批量写入和分页查询语句；raw_frames 保留完整 payloadJson.parsed，位置表保持轻量字段并通过 rawUri 关联原始帧。Kafka Protobuf Envelope、Raw Envelope。TDengine raw_frames 和位置表。生产历史分页查询raw 追溯
modules/services/event-history-service消费 32960、808、宇通 MQTT 的 Kafka event/raw topic，生产默认写入 TDengine raw_frames 和位置表，并提供 raw 帧、位置历史等分页查询边界；Raw 查询返回完整 parsed JSON，位置历史只保留核心字段并通过 rawUri 关联原始帧；生产消费可使用 tryIngest 和自动装配的 EnvelopeConsumerProcessor 将坏 protobuf、缺快照和存储异常收敛为结构化结果，并通过 EnvelopeDeadLetterSink 发布死信，避免阻塞 Kafka 分区。Kafka Protobuf Envelope。TDengine raw_frames、位置分页结果、raw archive 逻辑引用。历史明细分页查询raw JSON
modules/services/vehicle-state-service消费 Kafka 全字段事件，更新车辆最新状态、位置、安全和最后事件到 Redis；仅通过 optional-latest-state profile 显式构建，不属于默认生产 reactor；生产消费可使用 tryIngest 和自动装配的 EnvelopeConsumerProcessor 隔离坏 protobuf 和缺快照 Envelope，并把失败记录交给死信出口。Kafka Protobuf Envelope。vehicle:state:{vin}、vehicle:location:{vin}、vehicle:safety:{vin}。毫秒级热查询氢泄露安全
modules/services/vehicle-stat-service消费 Kafka 全字段事件，808 每日里程只使用 0x0200 附加项 0x01 上报的总里程做当日首末差值；生产消费可使用 tryIngest 和自动装配的 EnvelopeConsumerProcessor 隔离坏 protobuf，缺少 telemetry_snapshot 的消息会标记为 SKIPPED 并进入死信出口。Kafka Protobuf Envelope、808 total_mileage_km。vehicle_stat_metric：metric_key=daily_mileage_km、calculation_method=JT808_TOTAL_MILEAGE_DIFF。每日里程指标表
modules/apps/command-gateway可选 HTTP 到设备下行命令入口。支持会话查询、808 位置查询、参数查询/设置、终端控制、平台通用应答、终端属性查询 0x8107、区域删除 0x8601、人工报警确认 0x8203，以及 1078 音视频属性查询、实时预览/控制、历史回放/控制、资源列表查询、文件上传/控制；通过 CommandDispatcher 复用 protocol-jt808 的在线通道、流水号和同步应答等待能力。业务系统命令请求。CommandDispatcher 调用、设备应答摘要。远程控制参数设置属性查询区域删除报警确认视频控制
modules/apps/gb32960-ingest-app生产 GB32960 TCP 接入应用，只负责接收、鉴权、解析、冷存引用和 Kafka 投递。GB/T 32960 TCP 32960。vehicle.raw.gb32960.v1、vehicle.event.gb32960.v1。生产接入32960
modules/apps/jt808-ingest-app生产 JT808 TCP 接入应用，只负责 808 注册/鉴权/位置等上行解析、冷存引用和 Kafka 投递。JT/T 808 TCP 808。vehicle.raw.jt808.v1、vehicle.event.jt808.v1。生产接入808
modules/apps/yutong-mqtt-app生产宇通 MQTT 接入应用，按 endpoint profile 解析 MQTT 报文并投递 Kafka。宇通 MQTT topic。vehicle.raw.mqtt-yutong.v1、vehicle.event.mqtt-yutong.v1。生产接入MQTT
modules/apps/vehicle-history-app生产历史应用，消费 32960、808、宇通 MQTT 的 raw/event Kafka Envelope，写 TDengine 并提供历史查询。Kafka Protobuf Envelope。TDengine raw_frames、位置表和历史查询 API。历史查询raw JSON
modules/apps/vehicle-analytics-app生产指标应用，当前只消费 JT808 事件并用 GPS 总里程首末差值写每日里程指标。vehicle.event.jt808.v1。MySQL vehicle_stat_metric。每日里程指标表
-
- -
-

六、氢能业务开发落点

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
业务主题当前建议落点说明
车辆状态、速度、位置RealtimePayload、LocationPayload32960 和 808 都可能提供位置。统计侧需要明确优先级，例如优先 32960，缺失时用 808 补点。
每日里程Kafka 下游统计服务接入层只投递总里程和实时点位；日增里程建议下游按 VIN、自然日、事件时间聚合，处理回补和乱序。
每日用电量内部电池字段 + 下游统计接入层保留 SOC、电压、电流等瞬时字段；若有累计电耗字段，可加入内部字段模型后统一投递。
每日用氢量hydrogenRemainingKg + 下游统计优先使用累计氢耗字段；没有累计值时用氢余量差值估算，并在统计结果里标记估算口径。
储氢安全RealtimePayload 压力/温度 + AlarmPayload.safetyCategory高压、低压、温度、异常 bit 都应统一归入储罐安全域，下游可做趋势、阈值和连续异常判断。
氢气泄露AlarmPayload.hydrogenLeakDetected当前已作为高优先级安全事件。32960 中出现 HYDROGEN_LEAK 时强制映射为 CRITICAL。
原始报文追溯RawArchive + rawArchiveUri + sink-archive业务统计出现争议时，用事件 metadata 中的 archive://... 逻辑 URI 回查原始 bytes，结合 eventId、traceId、VIN、时间复现解析链路。
明细展示和导出vehicle-history-app + tdengine-history-store生产查询通过 TDengine raw_frames、位置表和 rawUri 关联完成；不再维护文件型事件索引旁路。
-
- 后续开发建议： - 当前 Kafka Protobuf 已加入全字段 TelemetrySnapshot。后续业务统计服务继续只依赖内部字段，不依赖 32960 原始字段名，这样接入 808、MQTT、车企私有协议时不会重写统计口径。 -
-
-
- - - - diff --git a/docs/operations/current-ecs-deployment.md b/docs/operations/current-ecs-deployment.md deleted file mode 100644 index cd6b8bf2..00000000 --- a/docs/operations/current-ecs-deployment.md +++ /dev/null @@ -1,227 +0,0 @@ -# 当前 ECS Go 原生部署说明 - -更新时间：2026-07-02 01:30 CST - -本文档记录当前 `lingniu-vehicle-ingest` 的生产运行面。当前 goal 的接入链路已经切到 Go 原生 systemd 部署，不再以 Docker/Portainer 作为生产运行方式。密钥只维护在 ECS 环境文件或受控凭据中，不写入 Git。 - -## 部署范围 - -| systemd 服务 | 二进制 | 说明 | 对外端口 | -| --- | --- | --- | --- | -| `lingniu-go-gateway.service` | `gateway` | GB32960 TCP、JT808 TCP、宇通 MQTT 接入，解析后写 Kafka RAW 和统一事件 | TCP `32960`、TCP `808` | -| `lingniu-go-history-writer.service` | `history-writer` | 消费 Kafka RAW，写 TDengine RAW、位置点、里程点核心表 | 无 HTTP | -| `lingniu-go-stat-writer.service` | `stat-writer` | 消费 Kafka RAW，按总里程差值法写 MySQL 每日指标 | 无 HTTP | -| `lingniu-go-realtime-api.service` | `realtime-api` | 消费统一事件写 Redis，并提供实时、历史、统计查询 API | HTTP `20210` | - -信达 Push 已废弃，不参与当前 Go 生产链路。旧 Java/Docker 服务不应占用 `808`、`32960`、`20210`。 - -## 应用 ECS - -| 项 | 值 | -| --- | --- | -| 公网 IP | `115.29.187.205` | -| 登录用户 | `root` | -| 部署目录 | `/opt/lingniu-go-native` | -| 当前 release | `/opt/lingniu-go-native/current` | -| 环境文件目录 | `/opt/lingniu-go-native/env` | -| systemd unit 目录 | `/etc/systemd/system` | -| 对外服务 | `32960`、`808`、`20210` | - -运行目录结构： - -```text -/opt/lingniu-go-native/ - current -> /opt/lingniu-go-native/releases/ - releases// - gateway - history-writer - stat-writer - realtime-api - env/ - gateway.env - history-writer.env - stat-writer.env - realtime-api.env - spool/gateway/ -``` - -## 中间件 - -| 组件 | 内网地址 | 公网地址 | 用途 | -| --- | --- | --- | --- | -| Kafka | `172.17.111.56:9092` | `114.55.58.251:9092` | RAW 和统一事件消息总线 | -| TDengine | `172.17.111.57:6041` | `115.29.185.82:6041` | RAW、位置、里程点时序热存储 | -| MySQL RDS | `rm-bp179zbv481rnw3e2.mysql.rds.aliyuncs.com:3306` | `rm-bp179zbv481rnw3e2no.mysql.rds.aliyuncs.com:3306` | 身份映射、JT808 注册、每日指标 | -| Redis RDS | `r-bp1u741kij7e51i481.redis.rds.aliyuncs.com:6379` | 无 | 准实时车辆状态缓存 | - -生产应用之间访问中间件优先使用内网地址。RDS 白名单需要允许应用 ECS 私网访问。 - -## Kafka Topic - -| Topic | 生产者 | 消费者 | 内容 | -| --- | --- | --- | --- | -| `vehicle.raw.go.gb32960.v1` | `gateway` | `history-writer`、`stat-writer` | GB32960 完整 RAW 记录 | -| `vehicle.raw.go.jt808.v1` | `gateway` | `history-writer`、`stat-writer` | JT808 完整 RAW 记录 | -| `vehicle.raw.go.yutong-mqtt.v1` | `gateway` | `history-writer` | 宇通 MQTT 完整 RAW 记录 | -| `vehicle.event.go.unified.v1` | `gateway` | `realtime-api` | 三类协议统一事件，用于 Redis 实时状态 | - -`gateway` 配置了 Kafka retry；生产 env 中启用 `KAFKA_SPOOL_DIR` 时，本地 spool 可在 Kafka 短暂不可用后回放。 - -Kafka 消费组： - -| Consumer Group | Topic | 当前验证 | -| --- | --- | --- | -| `go-history-writer` | 三个 `vehicle.raw.go.*` topic | 2026-07-02 验证 lag 为 `0` | -| `go-stat-writer` | `vehicle.raw.go.gb32960.v1`、`vehicle.raw.go.jt808.v1` | 2026-07-02 验证 lag 为 `0` | -| `go-realtime-api` | `vehicle.event.go.unified.v1` | 2026-07-02 验证 lag 为 `0` | - -可重复 smoke： - -```bash -python3 tools/go_kafka_prod_smoke.py --host 114.55.58.251 --user root --max-lag 100 -``` - -该脚本通过 SSH 登录 Kafka ECS，检查 Go 生产 topic 是否存在，并校验 `go-history-writer`、`go-stat-writer`、`go-realtime-api` 的消费组 lag。运行环境需要已配置 SSH 免密或已建立可用的 SSH 认证方式。 - -## TDengine 数据层 - -默认数据库：`lingniu_vehicle_ts` - -| 表 | 类型 | 说明 | -| --- | --- | --- | -| `raw_frames` | stable | 完整 RAW 帧，包含 `raw_hex`、`parsed_json`、`fields_json`、`source_endpoint` 和协议/车辆 tags | -| `vehicle_locations` | stable | 最小化位置历史，保留 `frame_id` 回链 RAW | -| `vehicle_mileage_points` | stable | 最小化总里程点，供历史查询和统计复核 | - -设计原则： - -- 完整结构化 payload 只落在 `raw_frames.parsed_json` / `fields_json`。 -- `vehicle_locations` 和 `vehicle_mileage_points` 只保留核心查询字段，不重复保存完整 JSON。 -- `telemetry_fields` 不由当前服务维护，字段配置解析由独立子服务处理。 -- API 传入东八区时间时，服务会转换为 TDengine 当前存储使用的 UTC 字面量再查询。 - -## MySQL 数据层 - -默认业务库：`lingniu_vehicle_data` - -| 表 | 维护方 | 说明 | -| --- | --- | --- | -| `vehicle_identity_binding` | 人工/外部主数据 | VIN 映射主表，用 `phone`、`device_id`、`plate` 降级定位 VIN | -| `jt808_registration` | `gateway` 自动写入 | 仅 JT808 注册和鉴权记录，以 `phone` 作为主键，记录设备、车牌、厂家、鉴权、来源端点 | -| `vehicle_daily_metric` | `stat-writer` 自动写入 | 每日指标表，保存 `daily_mileage_km` 和 `daily_total_mileage_km` | - -每日里程算法： - -```text -daily_mileage_km = 当日最大 total_mileage_km - 当日最小 total_mileage_km -daily_total_mileage_km = 当日最大 total_mileage_km -``` - -统计按 `Asia/Shanghai` 自然日归档，当前支持 GB32960、JT808、宇通 MQTT 中能解析出 `total_mileage_km` 的数据。JT808 的总里程来自位置附加信息 `0x01`，单位按协议转换为 km；宇通 MQTT 的 `TOTAL_MILEAGE` 原始单位为米，入库前转换为 km。 - -## Redis 实时层 - -`realtime-api` 消费 `vehicle.event.go.unified.v1`，按 VIN/vehicle key 维护准实时快照。主要能力： - -| API | 说明 | -| --- | --- | -| `/api/realtime/vehicles/{vin}` | 查询车辆合并实时快照 | -| `/api/realtime/vehicles/{vin}/online` | 查询车辆是否在线 | -| `/api/realtime/vehicles/{vin}/protocols/{protocol}` | 查询指定协议的实时快照 | - -Redis 只作为实时缓存；历史和统计以 TDengine/MySQL 为准。 - -## API 入口 - -公网基础地址： - -```text -http://115.29.187.205:20210 -``` - -常用查询： - -```bash -curl -sS 'http://115.29.187.205:20210/api/history/raw-frames?protocol=GB32960&dateFrom=2026-07-02T00:00:00%2B08:00&dateTo=2026-07-03T00:00:00%2B08:00&limit=1' -curl -sS 'http://115.29.187.205:20210/api/history/raw-frames?protocol=JT808&dateFrom=2026-07-02T00:00:00%2B08:00&dateTo=2026-07-03T00:00:00%2B08:00&limit=1' -curl -sS 'http://115.29.187.205:20210/api/history/raw-frames?protocol=YUTONG_MQTT&dateFrom=2026-07-02T00:00:00%2B08:00&dateTo=2026-07-03T00:00:00%2B08:00&limit=1' -curl -sS 'http://115.29.187.205:20210/api/history/locations?protocol=JT808&dateFrom=2026-07-02T00:00:00%2B08:00&dateTo=2026-07-03T00:00:00%2B08:00&limit=1' -curl -sS 'http://115.29.187.205:20210/api/history/mileage-points?protocol=JT808&dateFrom=2026-07-02T00:00:00%2B08:00&dateTo=2026-07-03T00:00:00%2B08:00&limit=1' -curl -sS 'http://115.29.187.205:20210/api/stats/daily-metrics?dateFrom=2026-07-02&dateTo=2026-07-02&limit=20' -``` - -分页参数统一使用 `limit` 和 `offset`。 - -部署后推荐直接运行 Go 原生生产 smoke： - -```bash -python3 tools/go_prod_acceptance.py --date 2026-07-02 -``` - -该聚合脚本会串行执行 systemd/端口检查、Kafka topic/consumer lag 检查、HTTP 数据链路检查，任一子检查失败时整体退出码为非 0。需要 SSH 可登录应用 ECS 和 Kafka ECS。 - -也可以单独运行子检查： - -```bash -python3 tools/go_systemd_prod_smoke.py --host 115.29.187.205 --user root -python3 tools/go_native_prod_smoke.py --date 2026-07-02 --timeout 8 -``` - -`go_systemd_prod_smoke.py` 通过 SSH 检查四个 Go systemd 服务是否 `active` 且 `enabled`，并确认 `808`、`32960` 由 `gateway` 监听、`20210` 由 `realtime-api` 监听，同时要求 `/opt/lingniu-go-native/current` 指向有效 release、四个生产二进制均可执行、`/opt/lingniu-go-native/spool/gateway` 没有待回放文件，避免旧 Java/Docker 进程占用生产端口、部署目录错乱或 Kafka spool 静默堆积。运行环境需要已配置 SSH 免密或已建立可用的 SSH 认证方式。 - -`go_native_prod_smoke.py` 通过 `20210` HTTP API 验证 GB32960、JT808、宇通 MQTT 的 RAW 查询及结构化 `parsed_json`，三类协议的位置和里程点查询及 `frame_id` RAW 回链，GB32960/JT808 的 `daily_mileage_km`、`daily_total_mileage_km` 统计公式，以及三类协议的 Redis realtime snapshot、online、protocol 查询。实时 snapshot/protocol 查询会校验 `updated_at_ms` 新鲜度；默认检查今天东八区数据时，还会要求三类 RAW 最新样本不超过 15 分钟，避免旧数据误判为接收正常；任一检查不达标时退出码为非 0。 - -## 部署命令 - -推荐使用仓库脚本发布完整 Go 原生 release。脚本会在本机/CI 构建 Linux amd64 四个二进制，打包上传到应用 ECS，切换 `/opt/lingniu-go-native/current`，逐个重启四个 systemd 服务，等待 gateway Kafka spool 清空，并在发布后运行聚合生产验收： - -```bash -python3 tools/go_native_deploy.py --date 2026-07-02 -``` - -脚本不会保存 SSH 或数据库密钥；认证仍使用当前操作环境可用的 SSH 凭据。需要跳过发布后验收时可显式追加 `--skip-acceptance`，但生产发布默认应保留验收。发布后 spool 默认最多等待 `900` 秒，可用 `--spool-drain-timeout` 和 `--spool-poll-interval` 调整。 - -脚本执行的发布动作等价于： - -```bash -CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' ./cmd/gateway -CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' ./cmd/history-writer -CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' ./cmd/stat-writer -CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' ./cmd/realtime-api -scp release.tar.gz root@115.29.187.205:/tmp/ -ln -sfn /opt/lingniu-go-native/releases/ /opt/lingniu-go-native/current -systemctl daemon-reload -systemctl restart lingniu-go-gateway lingniu-go-history-writer lingniu-go-stat-writer lingniu-go-realtime-api -``` - -只更新查询 API 时，可以仅替换 `/opt/lingniu-go-native/current/realtime-api` 并重启： - -```bash -systemctl restart lingniu-go-realtime-api.service -``` - -## 运行检查 - -```bash -python3 tools/go_systemd_prod_smoke.py --host 115.29.187.205 --user root - -systemctl is-active lingniu-go-gateway lingniu-go-history-writer lingniu-go-stat-writer lingniu-go-realtime-api -ss -lntp | egrep ':(808|32960|20210)\b' -journalctl -u lingniu-go-gateway --since '5 minutes ago' --no-pager -journalctl -u lingniu-go-history-writer --since '5 minutes ago' --no-pager -journalctl -u lingniu-go-stat-writer --since '5 minutes ago' --no-pager -journalctl -u lingniu-go-realtime-api --since '5 minutes ago' --no-pager -``` - -2026-07-02 01:30 CST 已验证： - -- 四个 Go systemd 服务均为 `active` 且 `enabled`。 -- `/opt/lingniu-go-native/current` 指向有效 release，四个生产二进制均存在且可执行。 -- `gateway` 正在监听 `32960` 和 `808`。 -- `realtime-api` 正在监听 `20210`。 -- gateway Kafka spool 当前无待回放文件。 -- GB32960、JT808、YUTONG_MQTT 的东八区 RAW 查询均可命中生产数据，且最新 RAW 样本包含结构化 `parsed_json`。 -- GB32960、JT808、YUTONG_MQTT 的最新 RAW 样本均在 15 分钟内。 -- GB32960、JT808、YUTONG_MQTT 的 `vehicle_locations`、`vehicle_mileage_points` 查询均可命中生产数据，且样本包含 `frame_id` 回链 RAW。 -- `vehicle_daily_metric` 可查到 `daily_mileage_km` 和 `daily_total_mileage_km`，且公式满足 `daily_mileage_km = latest_total_mileage_km - first_total_mileage_km`、`daily_total_mileage_km = latest_total_mileage_km`。 -- Redis realtime 可查到 GB32960、JT808、YUTONG_MQTT 的在线状态和协议快照，且 snapshot/protocol 快照最近刷新。 diff --git a/docs/operations/gb32960-service-split-runbook.md b/docs/operations/gb32960-service-split-runbook.md deleted file mode 100644 index 05005a66..00000000 --- a/docs/operations/gb32960-service-split-runbook.md +++ /dev/null @@ -1,273 +0,0 @@ -# GB32960 Split Service Runbook - -This runbook covers the default production runtimes: GB32960 ingest, JT808 ingest, Yutong MQTT ingest, history, and analytics. Kafka is the durability boundary between protocol ingest services and downstream business services. - -## Service Roles - -| Service | Module | Role | Production ports | -| --- | --- | --- | --- | -| GB32960 ingest | `:gb32960-ingest-app` | Accept GB32960 TCP connections, decode/authenticate frames, produce raw and normalized records to Kafka, and send protocol ACKs after required Kafka production succeeds. | TCP `32960`, HTTP `20100` | -| JT808 ingest | `:jt808-ingest-app` | Accept JT/T 808 TCP connections on the production receive port, parse raw frames, archive frame bytes, and produce raw/event envelopes to Kafka. | TCP `808`, HTTP `20400` | -| Yutong MQTT ingest | `:yutong-mqtt-app` | Connect to Yutong MQTT, parse subscribed messages, archive payloads, and produce raw/event envelopes to Kafka. | HTTP `20500` | -| Vehicle history | `:vehicle-history-app` | Consume GB32960/JT808/Yutong MQTT raw/event Kafka records, store compact history rows plus TDengine `raw_frames` `parsedJson`/`metadataJson`/`rawUri`, and expose TDengine-backed history APIs. | HTTP `20200` | -| Vehicle analytics | `:vehicle-analytics-app` | Consume JT808 event Kafka records, calculate daily mileage from reported GPS total mileage, and write metrics to `vehicle_stat_metric`. | HTTP `20310` | - -## Kafka Contract - -| Topic | Producer | Consumers | Purpose | -| --- | --- | --- | --- | -| `vehicle.raw.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app` | Raw frame records. History writes the parsed JSON, metadata JSON, and raw URI to TDengine `raw_frames`. | -| `vehicle.event.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app` | Normalized vehicle events keyed by VIN when available. | -| `vehicle.dlq.gb32960.v1` | Kafka producer/consumer error paths | Operators/replay tooling | Dead-letter records for failed production or consumer processing. | -| `vehicle.raw.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app` | JT808 raw frame records for TDengine `raw_frames` and raw-frame troubleshooting APIs. | -| `vehicle.event.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app`, `vehicle-analytics-app` | JT808 parsed location/session/alarm events keyed by phone or mapped VIN; analytics uses location total mileage for daily mileage metrics. | -| `vehicle.dlq.jt808.v1` | Kafka producer/consumer error paths | Operators/replay tooling | Dead-letter records for failed JT808 production or consumer processing. | -| `vehicle.raw.mqtt-yutong.v1` | `yutong-mqtt-app` | `vehicle-history-app` | Yutong MQTT raw payload records for TDengine `raw_frames` and troubleshooting APIs. | -| `vehicle.event.mqtt-yutong.v1` | `yutong-mqtt-app` | `vehicle-history-app` | Yutong MQTT parsed telemetry events keyed by mapped VIN or device identity. | -| `vehicle.dlq.mqtt-yutong.v1` | Kafka producer/consumer error paths | Operators/replay tooling | Dead-letter records for failed Yutong MQTT production or consumer processing. | -| `vehicle.dlq.history.v1` | History app consumer error paths | Operators/replay tooling | Dead-letter records for failed history storage processing. | - -Default consumer groups: - -- History: `vehicle-history` -- Analytics statistics: `vehicle-stat` - -## ACK Semantics - -GB32960 success ACKs are sent only after the ingest service completes the required Kafka production boundary for the accepted frame. If required Kafka production fails, the ingest service must not claim a successful ACK for that frame. - -History and analytics failures do not block GB32960 ACKs. They are downstream Kafka consumer failures and should be handled through retry, DLQ inspection, offset replay, and service-specific recovery. - -Authorization failures are rejected before the Kafka durability boundary and can be ACKed/rejected immediately according to protocol handling. - -## Build Prerequisites - -- Java 25 and Maven available on PATH. -- Kafka CLI tools such as `kafka-topics` and optionally `kafka-console-consumer` available on the ECS/operator host. -- Session state requires Redis. GB32960/JT808 ingest services keep live Netty - channels in-process, but session indexes and TTL metadata use Redis only. -- Redis is configured through Portainer environment variables such as `REDIS_HOST` and `REDIS_DATABASE=50`; there is no memory session-store fallback. - -If your shell needs an explicit JDK: - -```bash -export JAVA_HOME=/opt/homebrew/opt/openjdk@25/libexec/openjdk.jdk/Contents/Home -export PATH="$JAVA_HOME/bin:/opt/homebrew/bin:$PATH" -``` - -## Build - -From the repository root: - -```bash -mvn -pl :gb32960-ingest-app,:jt808-ingest-app,:yutong-mqtt-app,:vehicle-history-app,:vehicle-analytics-app -am package -Dmaven.test.skip=true -``` - -Expected result: `BUILD SUCCESS` and these runnable jars: - -- `modules/apps/gb32960-ingest-app/target/gb32960-ingest-app.jar` -- `modules/apps/jt808-ingest-app/target/jt808-ingest-app.jar` -- `modules/apps/yutong-mqtt-app/target/yutong-mqtt-app.jar` -- `modules/apps/vehicle-history-app/target/vehicle-history-app.jar` -- `modules/apps/vehicle-analytics-app/target/vehicle-analytics-app.jar` - -## Create Topics - -Run these only on the ECS/operator host after Kafka is reachable. The default bootstrap address is the ECS private Kafka address used by the production stack: - -```bash -KAFKA_BOOTSTRAP="${KAFKA_BOOTSTRAP:-172.17.111.56:9092}" -kafka-topics --bootstrap-server "$KAFKA_BOOTSTRAP" --create --if-not-exists --topic vehicle.raw.gb32960.v1 --partitions 12 --replication-factor 1 -kafka-topics --bootstrap-server "$KAFKA_BOOTSTRAP" --create --if-not-exists --topic vehicle.event.gb32960.v1 --partitions 12 --replication-factor 1 -kafka-topics --bootstrap-server "$KAFKA_BOOTSTRAP" --create --if-not-exists --topic vehicle.dlq.gb32960.v1 --partitions 3 --replication-factor 1 -kafka-topics --bootstrap-server "$KAFKA_BOOTSTRAP" --create --if-not-exists --topic vehicle.raw.jt808.v1 --partitions 12 --replication-factor 1 -kafka-topics --bootstrap-server "$KAFKA_BOOTSTRAP" --create --if-not-exists --topic vehicle.event.jt808.v1 --partitions 12 --replication-factor 1 -kafka-topics --bootstrap-server "$KAFKA_BOOTSTRAP" --create --if-not-exists --topic vehicle.dlq.jt808.v1 --partitions 3 --replication-factor 1 -kafka-topics --bootstrap-server "$KAFKA_BOOTSTRAP" --create --if-not-exists --topic vehicle.raw.mqtt-yutong.v1 --partitions 12 --replication-factor 1 -kafka-topics --bootstrap-server "$KAFKA_BOOTSTRAP" --create --if-not-exists --topic vehicle.event.mqtt-yutong.v1 --partitions 12 --replication-factor 1 -kafka-topics --bootstrap-server "$KAFKA_BOOTSTRAP" --create --if-not-exists --topic vehicle.dlq.mqtt-yutong.v1 --partitions 3 --replication-factor 1 -kafka-topics --bootstrap-server "$KAFKA_BOOTSTRAP" --create --if-not-exists --topic vehicle.dlq.history.v1 --partitions 3 --replication-factor 1 -``` - -Optional sanity check: - -```bash -kafka-topics --bootstrap-server "$KAFKA_BOOTSTRAP" --list | grep -E 'vehicle\.(raw|event|dlq)\.(gb32960|jt808|mqtt-yutong|history)\.v1' -``` - -## Production Runtime - -Production services run on ECS through Portainer/Docker. Do not start GB32960, JT808, Yutong MQTT, history, or analytics services on local developer machines; local machines are for code build, unit tests, and source inspection only. - -Use `deploy/portainer/docker-compose.yml` as the production runtime manifest. The default stack is: - -- `gb32960-ingest-app` -- `jt808-ingest-app` -- `yutong-mqtt-app` -- `vehicle-history-app` -- `vehicle-analytics-app` - -`vehicle-history-app` intentionally has no raw archive volume. The history hot path is Kafka to TDengine: raw envelopes are stored in TDengine `raw_frames` with `parsedJson`, `metadataJson`, and `rawUri`. Ingest services may still write original `.bin` files as cold backup, but history API queries must not depend on a shared local archive mount. - -Keep `TDENGINE_HISTORY_ENABLED=true` in the `vehicle-history-app` environment so history reads and writes use TDengine in production. - -`vehicle-analytics-app` is intentionally a metric runtime only. Latest-state APIs belong to `vehicle-state-service`, which stays outside the default reactor and should be built with `-Poptional-latest-state` and deployed as a separate consumer if that surface is needed. - -## Health Verification - -Run these from the ECS host or inside the matching container: - -```bash -curl -sS http://127.0.0.1:20100/actuator/health -curl -sS http://127.0.0.1:20100/actuator/health/liveness -curl -sS http://127.0.0.1:20100/actuator/health/readiness -curl -sS http://127.0.0.1:20400/actuator/health -curl -sS http://127.0.0.1:20400/actuator/health/liveness -curl -sS http://127.0.0.1:20400/actuator/health/readiness -curl -sS http://127.0.0.1:20500/actuator/health -curl -sS http://127.0.0.1:20500/actuator/health/liveness -curl -sS http://127.0.0.1:20500/actuator/health/readiness -curl -sS http://127.0.0.1:20200/actuator/health -curl -sS http://127.0.0.1:20200/actuator/health/liveness -curl -sS http://127.0.0.1:20200/actuator/health/readiness -curl -sS http://127.0.0.1:20310/actuator/health -curl -sS http://127.0.0.1:20310/actuator/health/liveness -curl -sS http://127.0.0.1:20310/actuator/health/readiness -``` - -Expected: each aggregate, liveness, and readiness endpoint returns `{"status":"UP"}` or an equivalent Spring Boot health JSON with top-level status `UP`. - -## End-to-End Verification - -Use a known-valid GB32960 fixture, for example one of the hex samples under `modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/`. - -Example send command: - -```bash -xxd -r -p modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_001.hex | nc 127.0.0.1 32960 -``` - -To capture and inspect the binary ACK, write the response to a file: - -```bash -rm -f /tmp/gb32960-ack.bin -xxd -r -p modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_001.hex \ - | nc -w 3 127.0.0.1 32960 > /tmp/gb32960-ack.bin -xxd -p /tmp/gb32960-ack.bin -``` - -Expected: the ACK file is non-empty and the decoded bytes begin with the GB32960 frame marker `2323`. Treat the ACK as verified only after checking the response bytes from your run. - -Verify only what was actually run in your environment: - -```bash -kafka-console-consumer --bootstrap-server "$KAFKA_BOOTSTRAP" --topic vehicle.raw.gb32960.v1 --from-beginning --max-messages 1 --timeout-ms 10000 -kafka-console-consumer --bootstrap-server "$KAFKA_BOOTSTRAP" --topic vehicle.event.gb32960.v1 --from-beginning --max-messages 1 --timeout-ms 10000 -kafka-console-consumer --bootstrap-server "$KAFKA_BOOTSTRAP" --topic vehicle.raw.mqtt-yutong.v1 --from-beginning --max-messages 1 --timeout-ms 10000 -kafka-console-consumer --bootstrap-server "$KAFKA_BOOTSTRAP" --topic vehicle.event.mqtt-yutong.v1 --from-beginning --max-messages 1 --timeout-ms 10000 -``` - -Use TDengine CLI or a JDBC client to verify history writes: - -```sql -USE vehicle_ts; -SELECT COUNT(*) FROM raw_frames WHERE protocol = 'GB32960'; -SELECT COUNT(*) FROM vehicle_locations WHERE protocol = 'GB32960'; -SELECT COUNT(*) FROM raw_frames WHERE protocol = 'JT808'; -SELECT COUNT(*) FROM jt808_locations WHERE protocol = 'JT808'; -SELECT COUNT(*) FROM raw_frames WHERE protocol = 'MQTT_YUTONG'; -SELECT COUNT(*) FROM vehicle_locations WHERE protocol = 'MQTT_YUTONG'; -``` - -Useful HTTP query checks after events are consumed. For `realtime_001.hex`, the fixture VIN is `LTEST000000000001`; replace ``, ``, and `` with values that cover the consumed record's event time: - -```bash -curl -sS 'http://127.0.0.1:20200/api/event-history/locations?protocol=GB32960&dateFrom=&dateTo=&vin=LTEST000000000001&limit=10' -curl -sS 'http://127.0.0.1:20200/api/event-history/raw-frames?protocol=GB32960&dateFrom=&dateTo=&vin=LTEST000000000001&limit=10' -curl -sS 'http://127.0.0.1:20200/api/event-history/gb32960/dictionary' -curl -sS 'http://127.0.0.1:20310/api/vehicle-stat/LTEST000000000001/daily?date=' -``` - -Expected E2E result when Kafka and all services are running: - -- A GB32960 client receives a binary success ACK only after required Kafka production succeeds, and the captured ACK bytes are inspected. -- `vehicle.raw.gb32960.v1` receives a raw frame envelope. -- `vehicle.event.gb32960.v1` receives one or more normalized records. -- TDengine `raw_frames` receives GB32960/JT808 RAW rows with parsed JSON and metadata. -- TDengine `raw_frames` receives Yutong MQTT RAW rows with parsed JSON and metadata when MQTT payloads are consumed. -- TDengine location tables receive compact GB32960/JT808/Yutong MQTT location rows when applicable telemetry is present. -- JT808 analytics writes daily mileage rows to MySQL `vehicle_stat_metric` after consuming applicable `vehicle.event.jt808.v1` records. - -Do not expect `vehicle-history-app` to create raw `.bin` archive files from Kafka raw records in the current implementation. `RawArchiveEventSink` can write archive files only when it receives `VehicleEvent.RawArchive.rawBytes()` inside the same JVM; the Kafka envelope carries only `RawArchiveRef` metadata. - -Do not mark any of these as verified unless the matching command was run and the output was inspected. - -## Runtime Location Policy - -Production services run on ECS through Portainer/Docker. - -- Do not create local service runners for GB32960, JT808, Yutong MQTT, history, or analytics. -- Do not run the service jars directly on developer machines for production verification. -- Keep local work limited to source edits, builds, and automated tests. - -For Portainer deployments, keep history without an archive volume: history consumes raw/event Kafka topics and writes TDengine history rows. - -## Rollback Guidance - -If the split deployment is unhealthy: - -1. Stop `gb32960-ingest-app` first so new GB32960 ACKs are not issued against an unhealthy Kafka boundary. -2. Keep Kafka topics intact for replay unless storage or privacy policy requires deletion. -3. Restart or roll back `vehicle-history-app` and `vehicle-analytics-app` independently; their failures do not require protocol ACK rollback because they consume from Kafka offsets. -4. If Kafka itself is unavailable, stop `gb32960-ingest-app` or route GB32960 traffic to a previously verified release that preserves the Kafka durability boundary. -5. After rollback, compare Kafka consumer group lag and DLQ contents before resuming the split services. - -Rollback commands depend on the deployment environment. For the default production stack, roll back the Portainer stack image version and let Docker restart the affected containers. - -## Task 8 Verification Notes - -Observed on 2026-06-23 in worktree `.worktrees/gb32960-service-split`: - -- Package build should use the current active-app command shown in the Build section. -- Repository-local Kafka setup inspection found no Kafka script, no Docker Compose file, and no compose YAML within the searched repository paths. -- `nc -z -w 2 127.0.0.1 9092` exited `1`, so no local Kafka broker was reachable at `127.0.0.1:9092`. -- `kafka-topics`, `kafka-topics.sh`, `docker`, and `docker-compose` were not found on PATH. -- Kafka topic creation, service startup, health checks, Kafka record checks, archive checks, TDengine checks, stat output checks, and ACK observation were not run because the local Kafka prerequisite was absent. - -## Latest Build Verification - -Observed on 2026-06-23 in worktree `.worktrees/gb32960-service-split`: - -- Targeted module tests: `mvn -pl :sink-kafka,:ingest-core,:protocol-gb32960,:event-history-service,:vehicle-state-service,:vehicle-stat-service test` ended with `BUILD SUCCESS`; Maven reported 55 protocol/sink/core tests, 30 event-history tests, 14 vehicle-state tests, and 19 vehicle-stat tests with 0 failures/errors/skips. -- Split app tests should cover the current active apps listed in the Service Roles section. -- Package verification should use the active-app package command from the Build section. -- Split app startup: not run in this verification pass because an ECS/Portainer runtime was outside that build-only check. -- Kafka raw records: not verified in that build-only check. -- Kafka event records: not verified in that build-only check. -- TDengine history rows: not verified; downstream services were not started without Kafka. -- Analytics output: not verified; downstream services were not started without Kafka. -- ACK behavior: not verified against a live broker in this pass. Unit tests cover the GB32960 ACK boundary and ordering, but an operator should still run the ACK capture command above on ECS or staging. - -## Remote Kafka Live Verification - -Observed on 2026-06-23 with Kafka `114.55.58.251:9092`: - -- `gb32960-ingest-app` handled TCP `32960` and HTTP `20100`. -- Ingest app health check `curl -sS http://127.0.0.1:20100/actuator/health` returned `{"status":"UP"}`. -- Kafka AdminClient confirmed topics `vehicle.raw.gb32960.v1` and `vehicle.event.gb32960.v1` existed, and created `vehicle.dlq.gb32960.v1`. -- The app accepted the live Hyundai platform connection from `8.134.95.166`, authenticated platform login, and received real GB32960 realtime/report/login/logout frames. -- Probe consumer with a temporary group read records from `vehicle.event.gb32960.v1`, including VIN keys such as `LB9A32A2XR0LS1575` and `LNXNEGRR3SR319485`. -- Probe consumer with a temporary group read records from `vehicle.raw.gb32960.v1`, including VIN keys such as `LNXNEGRR0SR319444`, `LNXNEGRR2SR318201`, and `LB9A32A23R0LS1532`. -- Runtime logs showed successful GB32960 ACK flushes for platform login, vehicle login, and vehicle logout after the Kafka-backed split ingest service was running. - -This verifies the live path `GB32960 TCP 32960 -> gb32960-ingest-app -> Kafka raw/event topics` against the remote Kafka broker. Downstream `vehicle-history-app` and `vehicle-analytics-app` were not started in this pass. - -## 2026-06-29 Superseded Local Verification Record - -Observed on 2026-06-29 in worktree `.worktrees/vehicle-ingest-redesign-phase1`. This record is retained only as historical evidence of parser/storage behavior; do not repeat it as a deployment procedure because production services now run on ECS through Portainer/Docker. - -- External JT808 traffic on TCP `808` was parsed and written through Kafka into TDengine. Direct TDengine checks showed more than `6000` JT808 `raw_frames` and more than `1900` JT808 `vehicle_locations`; current production history does not maintain a `telemetry_fields` table. -- A GB32960 synthetic realtime frame sent to TCP `32960` received a binary ACK beginning with `2323`. The same VIN `LTEST202606290001` was queryable from TDengine-backed history APIs with speed, mileage, longitude, and latitude fields. -- History raw and event Kafka consumption used separate bindings so raw-frame writes remained current while location/field event ingestion continued. - -The verified behavior was `GB32960/JT808 TCP -> protocol app -> Kafka raw/event topics -> vehicle-history-app -> TDengine raw_frames/location tables -> Swagger/API query`; the deployment method from that historical run is superseded. diff --git a/docs/operations/go-vehicle-gateway-verification-result-2026-07-01.md b/docs/operations/go-vehicle-gateway-verification-result-2026-07-01.md deleted file mode 100644 index 1c766bb8..00000000 --- a/docs/operations/go-vehicle-gateway-verification-result-2026-07-01.md +++ /dev/null @@ -1,888 +0,0 @@ -# Go Vehicle Gateway 旁路验证记录 - -验证时间：2026-07-01 22:00-22:10 CST - -## 部署版本 - -- Git commit：`60ca42e` -- 镜像：`crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com/oneos/vehicle-gateway-go:go-60ca42e-20260701220622` -- 部署方式：ECS `115.29.187.205` 上 Docker Compose 旁路部署 -- Compose 目录：`/opt/lingniu-go` - -## 旁路端口和 topic - -为了不影响现有 Java 生产服务，本次没有占用生产端口和生产 topic。 - -| 项 | 值 | -|---|---| -| GB32960 旁路 TCP | `115.29.187.205:23296 -> container :32960` | -| JT808 旁路 TCP | `115.29.187.205:18080 -> container :808` | -| Realtime API | `115.29.187.205:20210` | -| GB32960 RAW topic | `vehicle.raw.go.gb32960.v1` | -| JT808 RAW topic | `vehicle.raw.go.jt808.v1` | -| Yutong MQTT RAW topic | `vehicle.raw.go.yutong-mqtt.v1` | -| Unified topic | `vehicle.event.go.unified.v1` | - -## 容器状态 - -ECS 上四个 Go 容器均已启动： - -```text -go-vehicle-gateway Up -go-history-writer Up -go-stat-writer Up -go-realtime-api Up -``` - -## Kafka 验证 - -向 `18080` 注入 JT808 样例帧后，`vehicle.raw.go.jt808.v1` 和 -`vehicle.event.go.unified.v1` 均消费到统一 JSON envelope。 - -样例解析结果包含： - -- `protocol=JT808` -- `message_id=0x0200` -- `phone=013307795425` -- `longitude=121.069881` -- `latitude=30.590151` -- `speed_kmh=23` -- `total_mileage_km=10241.2` -- 表 27 附加项 `0x01` 原始值 `0001900C` - -## TDengine 验证 - -目标库：`lingniu_vehicle_ts` - -```text -raw_frames count = 3 -vehicle_locations count = 2 -vehicle_mileage_points count = 2 -``` - -最新 RAW 标识： - -```text -GB32960 | LNBSCB3D4R1234567 | LNBSCB3D4R1234567 | -JT808 | JT808:013307795425 | | 013307795425 -JT808 | JT808:013307795425 | | 013307795425 -``` - -说明：JT808 样例 phone 当前没有在 `vehicle_identity_binding` 中解析到 VIN，因此 -TDengine tag 使用 `vehicle_key=JT808:013307795425`。 - -## Redis 验证 - -向 `23296` 注入带 VIN 的 GB32960 合成帧后，Realtime API 可查询： - -```text -GET /api/realtime/vehicles/LNBSCB3D4R1234567 -``` - -返回字段包含： - -- `vin=LNBSCB3D4R1234567` -- `protocols=["GB32960"]` -- `online=true` -- `speed_kmh=30` -- `total_mileage_km=10000` -- `soc_percent=85` -- `longitude=121` -- `latitude=30.56` - -## MySQL 统计验证 - -向 `23296` 注入带 VIN 的 GB32960 合成帧后，`vehicle_daily_metric` 写入： - -```text -LNBSCB3D4R1234567 | 2020-07-01 | GB32960 | daily_mileage_km | 0.000 | 10000.000 | 10000.000 | 1 -LNBSCB3D4R1234567 | 2020-07-01 | GB32960 | daily_total_mileage_km | 10000.000 | 10000.000 | 10000.000 | 1 -``` - -说明：本次 GB32960 合成帧的设备时间为测试值，因此统计日期为 `2020-07-01`。 - -## 已修复问题 - -- TDengine WebSocket 普通 `INSERT` 使用参数占位符时，字符串未按预期加引号，导致语法错误。已在 `60ca42e` 中改为显式 SQL literal 转义。 -- MySQL Go DSN 中 `charset=utf8mb4%2Cutf8` 会被 driver 传给 MySQL，导致语法错误；ECS 旁路 env 已改为 `charset=utf8mb4`。 -- TDengine ECS 当前可用 root 密码与早先记录不一致，本次旁路 env 已按实际可认证值修正。 - -## 未完成 - -- 未切换真实生产端口 `32960/808`。 -- 未将真实外部 32960/808 流量转发到 Go 旁路端口。 -- 未开启宇通 MQTT 正式订阅。 -- JT808 样例未解析到 VIN，需维护 `vehicle_identity_binding` 后再验证 808 每日统计。 - -## 2026-07-01 22:22 真实归档帧复验 - -部署版本已更新： - -- Git commit：`5f4f4fd` -- 镜像：`crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com/oneos/vehicle-gateway-go:go-5f4f4fd-20260701222128` - -本轮补充验证内容： - -- JT808 支持 2019 版 versioned header，真实帧 `0x0200` 可解析出 `phone=14894135060`、`longitude=117.178187`、`latitude=30.570155`、`speed_kmh=33.4`、`total_mileage_km=7868.9`。 -- GB32960 真实燃料电池帧 `VIN=LB9A32A21R0LS1707` 可解析 `0x01` 整车、`0x02` 驱动电机、`0x03` 燃料电池、`0x04` 发动机、`0x05` 位置、`0x06` 极值、`0x07` 报警、`0x08` 电压、`0x09` 温度；后续 `0x30` 广东燃料电池扩展暂以 unknown raw 保留。 -- GB32960 真实帧字段包含 `fuel_cell_hydrogen_consumption_kg_per_100km=1.8`、`longitude=120.800326`、`latitude=31.634907`、`total_mileage_km=53490.9`。 -- Realtime API `GET /api/realtime/vehicles/LB9A32A21R0LS1707` 已返回上述经纬度和氢耗字段。 -- TDengine `vehicle_locations` 和 `vehicle_mileage_points` 已写入真实 GB32960 位置和里程点： - -```text -2020-07-01 08:09:22 | GB32960 | LB9A32A21R0LS1707 | lon=120.800326 | lat=31.634907 | mileage=53490.9 -``` - -后续仍需补齐： - -- JT808 真实 phone `14894135060` 仍未解析到 VIN，需维护身份绑定后再验证 VIN 维度实时和统计。 - -## 2026-07-01 22:28 广东燃料电池扩展复验 - -部署版本已更新： - -- Git commit：`a15de91` -- 镜像：`crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com/oneos/vehicle-gateway-go:go-a15de91-20260701222743` - -本轮补充验证内容： - -- GB32960 真实燃料电池帧 `VIN=LB9A32A21R0LS1707` 的 vendor 尾部已从 `unknown_unit` 补为结构化块： - `0x30 gd_fc_stack`、`0x31 gd_fc_auxiliary`、`0x32 gd_fc_dcdc`、`0x33 gd_fc_air_conditioner`、 - `0x34 gd_fc_vehicle_info`、`0x80 gd_fc_demo_extension`、`0x83 gd_fc_vendor_tlv`。 -- Kafka `vehicle.raw.go.gb32960.v1` 最新消息包含上述 block，并输出关键扁平字段： - `gd_fc_stack_hydrogen_inlet_pressure_kpa=97`、`gd_fc_stack_cell_count=108`、 - `gd_fc_vehicle_hydrogen_mass_kg=4.3`、`gd_fc_demo_stack_temp_c=65`。 -- Realtime API `GET /api/realtime/vehicles/LB9A32A21R0LS1707` 已返回 vendor 扁平字段： - -```text -gd_fc_dcdc_controller_temp_c=47 -gd_fc_demo_stack_temp_c=65 -gd_fc_stack_cell_count=108 -gd_fc_stack_frame_cell_count=108 -gd_fc_stack_hydrogen_inlet_pressure_kpa=97 -gd_fc_stack_water_outlet_temp_c=65 -gd_fc_vehicle_hydrogen_mass_kg=4.3 -``` - -- TDengine `raw_frames` 中 GB32960 行数增加到 `4`，最新 RAW 标识： - -```text -2026-07-01 14:28:24.869 | 172.20.0.1:47082 | GB32960 | LB9A32A21R0LS1707 | OK -``` - -后续仍需补齐： - -- JT808 真实 phone `14894135060` 到 VIN 的绑定数据维护与统计复验。 -- Go 旁路尚未切换到生产 `32960/808` 端口；仍运行在 `23296/18080`。 - -## 2026-07-01 22:43 JT808 前导零 phone 身份解析复验 - -部署版本已更新： - -- Git commit：`6b55694` -- 镜像：`crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com/oneos/vehicle-gateway-go:go-6b55694-20260701224300` - -本轮修复内容： - -- JT808 2011 老版包头的终端手机号按 BCD[6] 解析会保留前导 `0`，例如包头为 `013079963379`。 -- `vehicle_identity_binding.phone` 中实际维护的是去前导零后的手机号 `13079963379`。 -- 身份解析候选键已调整为先查原始 phone，再查去前导零 phone，之后再按 `device_id`、`plate` 降级。 - -真实帧复验样本： - -- 归档帧：`/opt/lingniu/vehicle-ingest/gb32960/archive/2026/07/01/JT808/unknown/1782903942845000.bin` -- 消息：`0x0200` 位置信息汇报 -- 包头 phone：`013079963379` -- 绑定表 phone：`13079963379` -- 解析 VIN：`LKLG7C4E3NA774736` - -Kafka `vehicle.raw.go.jt808.v1` 最新消息确认： - -```text -vin=LKLG7C4E3NA774736 -phone=013079963379 -parsed.header.phone_trim_zero=13079963379 -parsed.identity.resolved=true -parsed.identity.source=phone -parsed.identity.value=13079963379 -``` - -Realtime API 确认： - -```text -GET /api/realtime/vehicles/LKLG7C4E3NA774736 - -longitude=119.557997 -latitude=29.049092 -speed_kmh=0 -total_mileage_km=0 -device_time=2026-07-01T19:05:40+08:00 -``` - -TDengine 确认： - -```text -raw_frames: -2026-07-01 14:49:10.913 | JT808 | LKLG7C4E3NA774736 | phone=013079963379 | message_id=512 - -vehicle_locations: -2026-07-01 11:05:40.000 | JT808 | LKLG7C4E3NA774736 | lon=119.557997 | lat=29.049092 | speed=0 | mileage=0 - -vehicle_mileage_points: -2026-07-01 11:05:40.000 | JT808 | LKLG7C4E3NA774736 | mileage=0 -``` - -MySQL `vehicle_daily_metric` 确认： - -```text -LKLG7C4E3NA774736 | JT808 | 2026-07-01 | daily_mileage_km | 0.000 | sample_count=1 | TOTAL_MILEAGE_DIFF -LKLG7C4E3NA774736 | JT808 | 2026-07-01 | daily_total_mileage_km | 0.000 | sample_count=1 | TOTAL_MILEAGE_DIFF -``` - -说明：该真实位置帧携带的表 27 附加项 `0x01` 总里程值为 `0`，因此本次只验证身份解析、链路落地和差值统计写入；要验证非零日里程，需要回放或等待同一 VIN 的非零总里程连续位置点。 - -## 2026-07-01 22:55 Kafka 发布重试部署复验 - -部署版本已更新： - -- Git commit：`7e7066d` -- 镜像：`crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com/oneos/vehicle-gateway-go:go-7e7066d-20260701225503` - -本轮修复内容： - -- gateway 的 Kafka 发布增加同步重试封装。 -- 默认配置： - - `KAFKA_PUBLISH_ATTEMPTS=3` - - `KAFKA_PUBLISH_BACKOFF_MS=100` - - `KAFKA_PUBLISH_TIMEOUT_MS=3000` -- RAW 仍先于 unified event 写入；RAW 写失败不会继续写 unified。 -- 单元测试覆盖首次失败后成功、重试耗尽返回错误、上下文取消时停止重试。 - -部署后容器状态： - -```text -go-vehicle-gateway | go-7e7066d-20260701225503 | Up -go-history-writer | go-7e7066d-20260701225503 | Up -go-stat-writer | go-7e7066d-20260701225503 | Up -go-realtime-api | go-7e7066d-20260701225503 | Up -``` - -真实 808 帧回放复验： - -```text -source_endpoint=172.20.0.1:51966 -protocol=JT808 -message_id=0x0200 -phone=013079963379 -vin=LKLG7C4E3NA774736 -parsed.identity.value=13079963379 -``` - -Kafka `vehicle.raw.go.jt808.v1` 确认新消息写入，partition 6 offset 从 `1` 增加到 `2`。最新消息包含： - -```text -received_at_ms=1782917751903 -source_endpoint=172.20.0.1:51966 -vin=LKLG7C4E3NA774736 -fields.longitude=119.557997 -fields.latitude=29.049092 -fields.total_mileage_km=0 -``` - -TDengine `raw_frames` 确认新 RAW 落地： - -```text -2026-07-01 14:55:51.903 | JT808 | LKLG7C4E3NA774736 | phone=013079963379 | source=172.20.0.1:51966 -``` - -Realtime API 确认 Redis 已刷新： - -```text -GET /api/realtime/vehicles/LKLG7C4E3NA774736 - -received_at_ms=1782917751903 -source_endpoint=172.20.0.1:51966 -longitude=119.557997 -latitude=29.049092 -total_mileage_km=0 -``` - -说明：当前重试层只能覆盖 Kafka 短暂抖动；如果 Kafka 长时间不可用，仍需后续增加本地磁盘 spool/WAL 和恢复补发。 - -## 2026-07-01 23:02 切换 Go 到生产 32960/808 端口 - -本轮操作： - -- 停止 ECS 上原有 Java Docker 容器： - - `gb32960-ingest-app` - - `jt808-ingest-app` - - `yutong-mqtt-app` - - `vehicle-history-app` - - `vehicle-analytics-app` - - `vehicle-state-app` - - `telemetry-field-parser-app` -- 保留 Go sidecar 和 Portainer agent。 -- 修改 `/opt/lingniu-go/.env`： - - `GO_GB32960_TCP_PORT=32960` - - `GO_JT808_TCP_PORT=808` -- Go gateway 使用镜像： - - `crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com/oneos/vehicle-gateway-go:go-243de5c-20260701230028` -- 开启 Kafka 本地 spool： - - host：`/opt/lingniu-go/spool/gateway` - - container：`/data/spool/gateway` - -端口验证： - -```text -0.0.0.0:32960 -> go-vehicle-gateway:32960 -0.0.0.0:808 -> go-vehicle-gateway:808 -18080/23296 -> 已释放 -``` - -启动日志确认： - -```text -kafka durable spool enabled dir=/data/spool/gateway replay_interval_ms=1000 -tcp listener started protocol=GB32960 addr=[::]:32960 -tcp listener started protocol=JT808 addr=[::]:808 -``` - -真实连接确认： - -```text -GB32960 external remote: 8.134.95.166:50084 -JT808 external remote: 115.231.168.135:13801 -``` - -生产 `808` 回放复验： - -```text -source_endpoint=172.20.0.1:54680 -vin=LKLG7C4E3NA774736 -phone=013079963379 -longitude=119.557997 -latitude=29.049092 -total_mileage_km=0 -``` - -TDengine 最新 RAW 复验显示生产 808 已有真实外部数据持续落地： - -```text -2026-07-01 15:03:02.960 | JT808 | LKLG7C4E1NA774802 | 013079963291 | 222.66.200.68:28394 -2026-07-01 15:03:02.114 | JT808 | | 14894135583 | 122.152.221.156:33871 -2026-07-01 15:03:01.352 | JT808 | | 013307795519 | 115.231.168.135:13801 -2026-07-01 15:03:01.330 | JT808 | LKLG7C4E3NA774736 | 013079963379 | 222.66.200.68:28393 -``` - -Kafka spool 目录确认： - -```text -/opt/lingniu-go/spool/gateway file count = 0 -``` - -说明：生产 808 中仍有部分 phone 未解析出 VIN，后续需要继续维护 `vehicle_identity_binding` 的 phone/device_id/plate 到 VIN 映射，或者增强 808 注册/鉴权帧中的 identity 回写。 - -## 2026-07-01 23:09 808 里程统计防 0 污染复验 - -部署版本已更新： - -- Git commit：`d347525` -- 镜像：`crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com/oneos/vehicle-gateway-go:go-d347525-20260701230845` - -本轮修复内容： - -- `stat-writer` 不再把 `total_mileage_km <= 0` 的采样写入 MySQL 每日指标。 -- MySQL upsert 对历史 `first_total_mileage_km=0` / `latest_total_mileage_km=0` 做纠偏；后续首次有效总里程会替换掉历史 0。 -- `realtime-api` 合并实时快照时，非正 `total_mileage_km` 不再覆盖已有正值；速度、位置、状态等其他字段仍按事件时间更新。 - -测试验证： - -```text -go test ./... -GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build ./cmd/gateway -GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build ./cmd/history-writer -GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build ./cmd/stat-writer -GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build ./cmd/realtime-api -``` - -生产回放验证： - -- 第一条 808 帧：`device_time=2026-07-01T23:09:35+08:00`，`total_mileage_km=12345.6` -- 第二条 808 帧：`device_time=2026-07-01T23:09:36+08:00`，`total_mileage_km=0` -- 两条帧使用同一 VIN：`LKLG7C4E3NA774736` - -Redis merged 快照确认：第二条 0 里程帧没有覆盖已有有效总里程。 - -```text -event_time_ms=1782918576000 -source_endpoint=172.20.0.1:40946 -speed_kmh=0 -longitude=119.557997 -latitude=29.049092 -total_mileage_km=12345.6 -field_times_ms.total_mileage_km=1782918575000 -``` - -MySQL `vehicle_daily_metric` 确认：0 里程采样未进入统计，sample_count 只因有效总里程采样增加一次。 - -```text -LKLG7C4E3NA774736 | JT808 | 2026-07-01 | daily_mileage_km | 0.000 | sample_count=13 | first=12345.600 | latest=12345.600 -LKLG7C4E3NA774736 | JT808 | 2026-07-01 | daily_total_mileage_km | 12345.600 | sample_count=13 | first=12345.600 | latest=12345.600 -``` - -说明：生产 TDengine 中已经存在大量 808 非零 `total_mileage_km` 采样，但许多 phone 尚未映射到 VIN，因此 MySQL 按 VIN 的统计只会覆盖已解析 VIN 的车辆。下一步应继续补齐 `vehicle_identity_binding`，让更多 808 车辆进入统计。 - -## 2026-07-01 23:23 808 注册身份自动维护复验 - -部署版本已更新： - -- Git commit：`7b44f97` -- 镜像：`crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com/oneos/vehicle-gateway-go:go-7b44f97-20260701232200` - -本轮修复内容： - -- 808 `0x0100` 注册帧解析 `province`、`city`、`manufacturer`、`device_type`、`device_id`、`plate_color`、`plate`。 -- 808 `0x0102` 鉴权帧解析 `auth_token`。 -- Go gateway 在 identity resolve 后写入 `jt808_registration`，普通上行也会刷新 `latest_seen_at`。 -- 注册帧通过车牌解析到 VIN 后，会尝试把 phone/device/plate 反写到 `vehicle_identity_binding`。 -- 注册帧车牌支持 GBK 解码，避免 `Incorrect string value` 写入错误。 - -验证命令： - -```text -go test ./... -GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build ./cmd/gateway -GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build ./cmd/history-writer -GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build ./cmd/stat-writer -GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build ./cmd/realtime-api -``` - -ECS 运行状态： - -```text -go-vehicle-gateway | go-7b44f97-20260701232200 | 0.0.0.0:808->808, 0.0.0.0:32960->32960 -go-history-writer | go-7b44f97-20260701232200 | Up -go-stat-writer | go-7b44f97-20260701232200 | Up -go-realtime-api | go-7b44f97-20260701232200 | 0.0.0.0:20210->20210 -``` - -生产日志确认： - -- 新版本启动后真实 808、32960 连接持续进入。 -- GBK 车牌注册帧不再出现 `Incorrect string value`。 - -MySQL `jt808_registration` 确认： - -```text -registration_rows=214 -distinct_phone=214 -vin_rows=74 -plate_rows=74 -device_rows=72 - -013079963379 | device_id=9963379 | plate=沪A06788F | vin=LKLG7C4E3NA774736 | latest_registered_at=2026-07-01 23:23:01 -013079963321 | device_id=9963321 | plate=沪A59613F | vin=LKLG7C4EXNA774796 | latest_registered_at=2026-07-01 23:23:04 -``` - -说明：`jt808_registration.phone` 保存 808 包头原始 phone；`vehicle_identity_binding.phone` 中已有部分去前导零 phone。Go resolver 会同时查询原始 phone 和去前导零 phone。 - -## 2026-07-01 23:28 生产链路与指标清理复验 - -本轮复验目标： - -- 确认 Go gateway 的 Kafka durable spool 是重启窗口残留，而不是持续发布失败。 -- 确认 stat-writer / history-writer / realtime-api 的 Kafka consumer group 无 lag。 -- 确认真实 808 车辆 registration、TDengine RAW/里程点、Redis 实时查询均在持续更新。 -- 清理之前生产合成回放产生的 JT808 当日指标测试数据。 - -Kafka topic / consumer group： - -```text -vehicle.raw.go.jt808.v1 有持续 offset -vehicle.raw.go.gb32960.v1 有持续 offset -vehicle.event.go.unified.v1 有持续 offset - -go-stat-writer LAG=0 -go-history-writer LAG=0 -go-realtime-api LAG=0 -``` - -TDengine： - -```text -raw_frames: -JT808 2922 -GB32960 27 - -vehicle_mileage_points: -JT808 2050 -GB32960 2 -``` - -Redis 实时查询样例： - -```text -GET /api/realtime/vehicles/LKLG7C4E3NA774736 -vin=LKLG7C4E3NA774736 -source_endpoint=222.66.200.68:29646 -plate=沪A06788F -longitude=119.556866 -latitude=29.047911 -total_mileage_km=12345.6 -``` - -说明：该 Redis merged 快照保留了此前有效 `total_mileage_km=12345.6`，后续真实 808 位置帧上报的 `total_mileage_km=0` 不会覆盖该字段。TDengine 中该 VIN 后续真实里程点的 `total_mileage_km` 均为 0，因此 stat-writer 按“只统计正总里程”的规则不再写 JT808 每日指标。 - -MySQL 指标清理： - -```text -删除前： -LKLG7C4E2NA774775 | JT808 | 2026-07-01 | daily_mileage_km | first=0 | latest=0 | sample_count=7 -LKLG7C4E2NA774775 | JT808 | 2026-07-01 | daily_total_mileage_km | first=0 | latest=0 | sample_count=7 -LKLG7C4E3NA774736 | JT808 | 2026-07-01 | daily_mileage_km | first=12345.6 | latest=12345.6 | sample_count=13 -LKLG7C4E3NA774736 | JT808 | 2026-07-01 | daily_total_mileage_km | first=12345.6 | latest=12345.6 | sample_count=13 - -删除后等待真实流量窗口： -JT808 2026-07-01 metric rows = 0 -``` - -清理原因： - -- `LKLG7C4E3NA774736` 的 12345.6 来自此前生产连通性合成回放，不是真实平台上报。 -- `LKLG7C4E2NA774775` 的 0 指标来自旧逻辑污染，当前 stat-writer 已跳过非正总里程。 - -Kafka spool： - -```text -23:24 spool_count=904 -23:25 spool_count=862 -23:28 spool_count=722 -new_spool_1min=0 -``` - -说明：spool 正在回放下降，且最近 1 分钟没有新增文件；生产 gateway 日志未见 `Incorrect string value`、Kafka publish error 或 replay error。 - -## 2026-07-01 23:34 每日指标查询 API 复验 - -部署版本已更新： - -- Git commit：`56f4811` -- 镜像：`crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com/oneos/vehicle-gateway-go:go-56f4811-20260701233311` - -新增接口： - -```text -GET /api/stats/daily-metrics -``` - -查询参数： - -```text -vin 可选 -protocol 可选，GB32960 / JT808 / YUTONG_MQTT -metricKey 可选，daily_mileage_km / daily_total_mileage_km -dateFrom 可选，YYYY-MM-DD -dateTo 可选，YYYY-MM-DD -limit 可选，默认 50，最大 1000 -offset 可选，默认 0 -``` - -生产可访问样例： - -```text -http://115.29.187.205:20210/api/stats/daily-metrics?vin=LB9A32A21R0LS1707&protocol=GB32960&dateFrom=2020-07-01&dateTo=2020-07-01&limit=10 -``` - -返回确认： - -```json -{ - "items": [ - { - "vin": "LB9A32A21R0LS1707", - "stat_date": "2020-07-01", - "protocol": "GB32960", - "metric_key": "daily_mileage_km", - "metric_value": 0, - "metric_unit": "km", - "first_total_mileage_km": 53490.9, - "latest_total_mileage_km": 53490.9, - "sample_count": 3, - "calculation_method": "TOTAL_MILEAGE_DIFF", - "created_at": "2026-07-01 22:14:13", - "updated_at": "2026-07-01 22:28:25" - }, - { - "vin": "LB9A32A21R0LS1707", - "stat_date": "2020-07-01", - "protocol": "GB32960", - "metric_key": "daily_total_mileage_km", - "metric_value": 53490.9, - "metric_unit": "km", - "first_total_mileage_km": 53490.9, - "latest_total_mileage_km": 53490.9, - "sample_count": 3, - "calculation_method": "TOTAL_MILEAGE_DIFF", - "created_at": "2026-07-01 22:14:13", - "updated_at": "2026-07-01 22:28:25" - } - ], - "limit": 10, - "offset": 0, - "total": 2 -} -``` - -JT808 真实生产查询： - -```text -http://115.29.187.205:20210/api/stats/daily-metrics?protocol=JT808&dateFrom=2026-07-01&dateTo=2026-07-01&limit=10 -``` - -返回 `total=0`。原因是此前合成回放产生的测试指标已清理，当前真实 808 已绑定 VIN 的车辆仍上报 `total_mileage_km=0`，stat-writer 按规则不写非正总里程指标。 - -实时接口回归： - -```text -http://115.29.187.205:20210/api/realtime/vehicles/LKLG7C4E3NA774736/online -``` - -返回 `online=true`，说明新增 stats API 没有影响 Redis 实时查询。 - -发布后 spool 观察： - -```text -发布后立即：spool_count=1101，new_spool_1min=102 -等待 60 秒：spool_count=1033，new_spool_1min=0 -``` - -说明：本次发布重启窗口产生的 spool 正在回放下降，且最近 1 分钟没有新增；gateway/realtime-api 最近日志未见 error、failed、Kafka publish error。 - -## 2026-07-01 23:50 生产端口接管与 RAW 查询 API 复验 - -部署版本： - -- Git commit：`5937132` -- 镜像：`crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com/oneos/vehicle-gateway-go:go-5937132-20260701234220` - -生产端口状态： - -```text -go-vehicle-gateway 0.0.0.0:808->808/tcp -go-vehicle-gateway 0.0.0.0:32960->32960/tcp -``` - -旧 Java 容器状态： - -```text -gb32960-ingest-app restart=no status=exited -jt808-ingest-app restart=no status=exited -yutong-mqtt-app restart=no status=exited -vehicle-history-app restart=no status=exited -vehicle-analytics-app restart=no status=exited -vehicle-state-app restart=no status=exited -telemetry-field-parser-app restart=no status=exited -``` - -生产连接确认： - -- JT808 端口 `808` 已收到外部连接：`222.66.200.68`、`115.231.168.135`。 -- GB32960 端口 `32960` 已收到外部连接：`8.134.95.166`、`117.160.0.65`。 -- ECS 当前没有 `8089` 监听。 - -RAW 查询 API： - -```text -GET /api/history/raw-frames -``` - -生产可访问样例： - -```text -http://115.29.187.205:20210/api/history/raw-frames?protocol=JT808&vin=LKLG7C4E3NA774736&messageId=0x0200&limit=1 -http://115.29.187.205:20210/api/history/raw-frames?protocol=GB32960&vin=LB9A32A21R0LS1707&limit=1 -``` - -接口复验结果： - -```text -JT808_RAW: -total=1 -protocol=JT808 -vin=LKLG7C4E3NA774736 -phone=013079963379 -message_id=512 -message_id_hex=0x0200 -parse_status=OK -raw_hex_len=94 -parsed_json_len=651 -fields_json_len=186 - -GB32960_RAW: -total=1 -protocol=GB32960 -vin=LB9A32A21R0LS1707 -message_id=2 -message_id_hex=0x0002 -parse_status=OK -raw_hex_len=1548 -parsed_json_len=3698 -fields_json_len=563 -``` - -同轮回归验证： - -```text -GET /api/realtime/vehicles/LKLG7C4E3NA774736/online -online=true -protocols=["JT808"] - -GET /api/stats/daily-metrics?vin=LB9A32A21R0LS1707&protocol=GB32960&dateFrom=2020-07-01&dateTo=2026-07-01&limit=1 -total=1 -protocol=GB32960 -metric_key=daily_mileage_km -calculation_method=TOTAL_MILEAGE_DIFF -``` - -说明： - -- RAW 查询已返回 `raw_hex`、`parsed_json`、`fields_json`，可以用于查看完整解析结果。 -- 日统计和实时查询在生产端口接管后仍可用。 - -## 2026-07-02 00:00 原生 systemd 部署与宇通 MQTT 复验 - -根据本次 goal 的新要求，Go 接入链路后续不再使用 Docker 部署。本轮已中断 Docker 镜像构建路径，改为 ECS 原生 Linux 二进制 + systemd。 - -部署版本： - -- Git commit：`594ab2d` -- 部署目录：`/opt/lingniu-go-native` -- 当前 release：`/opt/lingniu-go-native/releases/594ab2d` -- 当前指针：`/opt/lingniu-go-native/current -> /opt/lingniu-go-native/releases/594ab2d` - -systemd 服务： - -```text -lingniu-go-gateway.service active -lingniu-go-history-writer.service active -lingniu-go-stat-writer.service active -lingniu-go-realtime-api.service active -``` - -端口确认： - -```text -[::]:32960 gateway -[::]:808 gateway -[::]:20210 realtime-api -``` - -Go Docker 容器已停止： - -```text -go-vehicle-gateway Exited -go-history-writer Exited -go-stat-writer Exited -go-realtime-api Exited -``` - -宇通 MQTT： - -```text -yutong mqtt client started -mqtt subscribed broker=ssl://cpxlm.axxc.cn:38883 topic=/ytforward/shln/+ qos=1 -``` - -说明：MQTT broker 存在短连接 `EOF` 后自动重连现象，但订阅后已收到真实数据并落 RAW。 - -RAW 查询复验： - -```text -GET /api/history/raw-frames?protocol=YUTONG_MQTT&limit=3 - -total=3 -protocol=YUTONG_MQTT -vin=LMRKH9AC1R1004131 -vehicle_key=LMRKH9AC1R1004131 -parse_status=OK -parsed_json_len=1104 -fields_json_len=116 -``` - -同轮回归： - -```text -GET /api/history/raw-frames?protocol=JT808&vin=LKLG7C4E3NA774736&messageId=0x0200&limit=1 -total=1 -parse_status=OK - -GET /api/history/raw-frames?protocol=GB32960&vin=LB9A32A21R0LS1707&limit=1 -total=1 -parse_status=OK - -GET /api/realtime/vehicles/LKLG7C4E3NA774736/online -online=true -protocols=["JT808"] -``` - -本轮落地文件： - -- `deploy/systemd/README.md` -- `deploy/systemd/lingniu-go-gateway.service` -- `deploy/systemd/lingniu-go-history-writer.service` -- `deploy/systemd/lingniu-go-stat-writer.service` -- `deploy/systemd/lingniu-go-realtime-api.service` - -## 2026-07-02 00:04 宇通 MQTT EOF 诊断 - -现象： - -Go 原生部署后，`lingniu-go-gateway` 日志中出现多次： - -```text -mqtt connection lost error=EOF -mqtt connection lost error=write: broken pipe -mqtt subscribed topic=/ytforward/shln/+ qos=1 -``` - -10 分钟窗口统计： - -```text -mqtt subscribed = 33 -mqtt connection lost = 32 -``` - -对照旧 Java `yutong-mqtt-app` 历史日志，停止前也存在同类行为： - -```text -mqtt endpoint [yutong] received topic=/ytforward/shln/1 bytes=574 -mqtt endpoint [yutong] connection lost -Caused by: java.io.EOFException -mqtt endpoint [yutong] re-subscribed topic=/ytforward/shln/+ qos=1 -mqtt endpoint [yutong] received topic=/ytforward/shln/3 bytes=586 -``` - -判断： - -- EOF/短连接不是 Go TLS 支持或 systemd 原生部署新引入的问题。 -- 旧 Java 和新 Go 都表现为：订阅、收到数据、broker 断开、自动重连。 -- 当前链路可持续收到宇通 MQTT 数据并写入 TDengine RAW。 - -复验： - -```text -GET /api/history/raw-frames?protocol=YUTONG_MQTT&limit=1 - -total=1 -protocol=YUTONG_MQTT -vin=LMRKH9ACXR1004094 -parse_status=OK -parsed_json_len=1080 -fields_json_len=115 -``` - -运维建议： - -- 单条 EOF 不作为不可用判断。 -- 以 `YUTONG_MQTT` RAW 入库增长、最近成功 `mqtt subscribed` 时间、服务 `systemctl is-active` 作为可用性判断。 -- 如果 EOF 频率继续升高且 RAW 不再增长，再联系宇通侧确认 broker 长连接策略、同一 clientId 并发限制或网络出口策略。 diff --git a/docs/operations/go-vehicle-gateway-verification.md b/docs/operations/go-vehicle-gateway-verification.md deleted file mode 100644 index 0646e840..00000000 --- a/docs/operations/go-vehicle-gateway-verification.md +++ /dev/null @@ -1,158 +0,0 @@ -# Go Vehicle Gateway 验证手册 - -本文档用于验证 Go 重构运行面： - -- `go-vehicle-gateway`：GB32960 TCP、JT808 TCP、宇通 MQTT 接入。 -- `go-history-writer`：Kafka RAW 写 TDengine。 -- `go-stat-writer`：Kafka RAW 写 MySQL 每日指标。 -- `go-realtime-api`：Kafka unified event 写 Redis，并提供实时查询 API。 - -## 本地构建验证 - -```bash -cd go/vehicle-gateway -go test ./... -go build ./cmd/gateway -go build ./cmd/history-writer -go build ./cmd/stat-writer -go build ./cmd/realtime-api -``` - -## 生产原生部署验证 - -```bash -python3 tools/go_prod_acceptance.py --date 2026-07-02 -``` - -该命令会聚合执行： - -- `go_systemd_prod_smoke.py`：应用 ECS systemd 服务 active/enabled、端口归属和 gateway spool -- `go_kafka_prod_smoke.py`：Kafka topic 和消费组 lag -- `go_native_prod_smoke.py`：HTTP API、RAW `parsed_json`、历史核心表 `frame_id` 回链、每日指标公式、Redis 实时新鲜度、TDengine/MySQL/Redis 数据链路 - -如需单独检查 systemd： - -```bash -python3 tools/go_systemd_prod_smoke.py --host 115.29.187.205 --user root -``` - -脚本通过 SSH 检查： - -- `lingniu-go-gateway.service` -- `lingniu-go-history-writer.service` -- `lingniu-go-stat-writer.service` -- `lingniu-go-realtime-api.service` -- 上述服务均为 `active` 且 `enabled` -- `808`、`32960` 由 `gateway` 监听 -- `20210` 由 `realtime-api` 监听 -- `/opt/lingniu-go-native/spool/gateway` 无待回放文件 - -## 生产环境变量 - -当前 goal 的生产面使用 ECS 原生 systemd 部署，不再以 Docker/Portainer 作为生产运行方式。以下环境变量维护在 `/opt/lingniu-go-native/env/*.env`： - -```bash -KAFKA_BROKERS=172.17.111.56:9092 -TDENGINE_DSN=root:@ws(172.17.111.57:6041)/lingniu_vehicle_ts -MYSQL_DSN=lingniu_vehicle:@tcp(rm-bp179zbv481rnw3e2.mysql.rds.aliyuncs.com:3306)/lingniu_vehicle_data?parseTime=true&charset=utf8mb4,utf8&loc=Asia%2FShanghai -IDENTITY_MYSQL_DSN=${MYSQL_DSN} -VEHICLE_IDENTITY_TABLE=vehicle_identity_binding -REDIS_ADDR=r-bp1u741kij7e51i481.redis.rds.aliyuncs.com:6379 -REDIS_PASSWORD= -REDIS_DB=50 -``` - -宇通 MQTT 开启时再配置： - -```bash -YUTONG_MQTT_ENABLED=true -YUTONG_MQTT_URI= -YUTONG_MQTT_TOPICS=/ytforward/shln/+ -YUTONG_MQTT_CLIENT_ID=lingniu-go-yutong-mqtt -YUTONG_MQTT_USERNAME= -YUTONG_MQTT_PASSWORD= -``` - -## ECS 进程检查 - -```bash -systemctl is-active lingniu-go-gateway lingniu-go-history-writer lingniu-go-stat-writer lingniu-go-realtime-api - -ss -lntp | egrep ':(808|32960|20210)\b' -``` - -## Kafka 验证 - -优先使用 smoke 脚本做可重复检查： - -```bash -python3 tools/go_kafka_prod_smoke.py --host 114.55.58.251 --user root --max-lag 100 -``` - -脚本会检查 `vehicle.raw.go.gb32960.v1`、`vehicle.raw.go.jt808.v1`、`vehicle.raw.go.yutong-mqtt.v1`、`vehicle.event.go.unified.v1` 是否存在，并汇总 `go-history-writer`、`go-stat-writer`、`go-realtime-api` 的消费 lag。运行环境需要 SSH 免密或已建立可用的 SSH 认证方式。 - -手工核对命令： - -```bash -kafka-console-consumer --bootstrap-server 172.17.111.56:9092 \ - --topic vehicle.raw.go.jt808.v1 --max-messages 1 --timeout-ms 10000 - -kafka-console-consumer --bootstrap-server 172.17.111.56:9092 \ - --topic vehicle.raw.go.gb32960.v1 --max-messages 1 --timeout-ms 10000 - -kafka-console-consumer --bootstrap-server 172.17.111.56:9092 \ - --topic vehicle.raw.go.yutong-mqtt.v1 --max-messages 1 --timeout-ms 10000 - -kafka-console-consumer --bootstrap-server 172.17.111.56:9092 \ - --topic vehicle.event.go.unified.v1 --max-messages 1 --timeout-ms 10000 - -kafka-consumer-groups --bootstrap-server 172.17.111.56:9092 \ - --describe --group go-history-writer - -kafka-consumer-groups --bootstrap-server 172.17.111.56:9092 \ - --describe --group go-stat-writer - -kafka-consumer-groups --bootstrap-server 172.17.111.56:9092 \ - --describe --group go-realtime-api -``` - -## TDengine 验证 - -```sql -USE lingniu_vehicle_ts; -SHOW STABLES; -SELECT COUNT(*) FROM raw_frames; -SELECT COUNT(*) FROM vehicle_locations; -SELECT COUNT(*) FROM vehicle_mileage_points; -``` - -## MySQL 验证 - -```sql -SELECT protocol, metric_key, COUNT(*) -FROM vehicle_daily_metric -GROUP BY protocol, metric_key; - -SELECT * -FROM vehicle_daily_metric -WHERE metric_key IN ('daily_mileage_km', 'daily_total_mileage_km') -ORDER BY updated_at DESC -LIMIT 20; -``` - -## Redis 实时查询 - -```bash -curl -sS 'http://115.29.187.205:20210/api/realtime/vehicles/' -curl -sS 'http://115.29.187.205:20210/api/realtime/vehicles//online' -curl -sS 'http://115.29.187.205:20210/api/realtime/vehicles//protocols/JT808' -``` - -## 验收证据 - -正式切换前至少记录： - -- 一个真实 GB32960 VIN 的 RAW、location、mileage point、Redis snapshot。 -- 一个真实 JT808 VIN/phone 的 RAW、location、mileage point、daily metric、Redis snapshot。 -- 一个真实宇通 MQTT VIN 或 device_id 的 RAW、Redis snapshot。 -- `go-history-writer` 和 `go-stat-writer` 重启后 Kafka offset 能继续推进。 diff --git a/docs/operations/jt808-daily-mileage-runbook.md b/docs/operations/jt808-daily-mileage-runbook.md deleted file mode 100644 index 982d9e75..00000000 --- a/docs/operations/jt808-daily-mileage-runbook.md +++ /dev/null @@ -1,34 +0,0 @@ -# JT808 Daily Mileage Streaming - -## Scope - -The analytics app can calculate daily mileage from JT808 telemetry only. It consumes `vehicle.event.jt808.v1` and saves the current daily result into the common `vehicle-stat` metric repository. - -Only supported message backbone: Kafka. - -## Storage - -Runtime state: none outside `vehicle_stat_metric`. There is no separate JT808 daily-mileage table. The derived value is stored as one `daily_mileage_km` metric row in the common JDBC/MySQL table `vehicle_stat_metric`; the local-day minimum and maximum GPS total mileage values are kept on that same row as calculation source columns. - -The JT808 daily-mileage value is calculated from the GPS total mileage reported in location additional information: - -```text -daily_mileage_km = max_total_mileage_km - min_total_mileage_km -calculation_method = JT808_TOTAL_MILEAGE_DIFF -``` - -The first valid JT808 location point for a vehicle and local day stores both calculation endpoints on the metric row and writes `daily_mileage_km=0.0`. Later ordered or replayed points update the lower endpoint when the reported GPS total mileage is smaller, update the upper endpoint when it is larger, and rewrite the derived `daily_mileage_km` on the same row. - -## Runtime Settings - -Set these in Portainer or Nacos, without committing secrets: - -```text -KAFKA_TOPIC_JT808_EVENT=vehicle.event.jt808.v1 -VEHICLE_STAT_JT808_MILEAGE_ENABLED=true -MYSQL_JDBC_URL= -MYSQL_USERNAME= -MYSQL_PASSWORD= -``` - -Algorithm defaults use the Asia/Shanghai daily boundary. Restart recovery reads the same `daily_mileage_km` metric row. diff --git a/docs/operations/vehicle-ingest-tdengine-verification.md b/docs/operations/vehicle-ingest-tdengine-verification.md deleted file mode 100644 index 9d7f8cdd..00000000 --- a/docs/operations/vehicle-ingest-tdengine-verification.md +++ /dev/null @@ -1,351 +0,0 @@ -# Vehicle Ingest TDengine 链路验收手册 - -这份手册用于验收当前 GB32960/JT808/Yutong MQTT 接入重构链路： - -1. 协议 App 接收 TCP 报文。 -2. 协议 App 归档原始帧元数据，并把事件信封发布到 Kafka。 -3. `vehicle-history-app` 消费 Kafka。 -4. history 写入 raw、location 到 TDengine；不写 telemetry_fields。 -5. Swagger/API 通过 TDengine 做历史分页查询。 - -每一步都必须执行命令并检查输出后，才能标记为已验证。 - -## 必需运行参数 - -启动服务前先设置： - -```bash -export KAFKA_BROKERS=114.55.58.251:9092 -export TDENGINE_HISTORY_ENABLED=true -export TDENGINE_HISTORY_DATABASE=vehicle_ts -export TDENGINE_JDBC_URL='jdbc:TAOS-WS://:6041/vehicle_ts' -export TDENGINE_DRIVER_CLASS_NAME=com.taosdata.jdbc.ws.WebSocketDriver -export TDENGINE_USERNAME=root -export TDENGINE_PASSWORD='' -export TDENGINE_MIN_IDLE=0 -export TDENGINE_MAX_POOL_SIZE=32 -``` - -`TDENGINE_MIN_IDLE=0` 用于减少冷启动时 TDengine 连接重试日志。TDengine 地址稳定后，再按生产并发调大连接池。 - -高吞吐生产验收只验证 TDengine `raw_frames`、位置表和分页查询闭环。RAW 帧的 `parsedJson`、`metadataJson`、`rawUri` 直接进入 TDengine，GB32960 snapshot/fields 接口基于 `raw_frames` 与当前解析器即时返回结构化结果；接入服务写出的原始 `.bin` 只作为冷备复核材料。逐字段趋势宽表由独立字段解析服务消费 Kafka RAW/事件后维护，不由 history-app 写入。 - -## 构建 - -```bash -mvn -pl :gb32960-ingest-app,:jt808-ingest-app,:yutong-mqtt-app,:vehicle-history-app,:vehicle-analytics-app -am package -Dmaven.test.skip=true -``` - -预期产物： - -- `modules/apps/gb32960-ingest-app/target/gb32960-ingest-app.jar` -- `modules/apps/jt808-ingest-app/target/jt808-ingest-app.jar` -- `modules/apps/yutong-mqtt-app/target/yutong-mqtt-app.jar` -- `modules/apps/vehicle-history-app/target/vehicle-history-app.jar` -- `modules/apps/vehicle-analytics-app/target/vehicle-analytics-app.jar` - -## ECS 服务状态验证 - -生产服务只在 ECS 上运行，通过 Portainer/Docker 管理。不要在本机用 `java -jar`、plist 或其它守护进程方式启动 GB32960、JT808、Yutong MQTT、history、analytics。 - -```bash -docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}' \ - | egrep 'gb32960|jt808|yutong|vehicle-history|vehicle-analytics|NAMES' - -ss -lntp | egrep ':(808|32960|20100|20200|20310|20400|20500)\b' - -curl -sS http://127.0.0.1:20400/actuator/health -curl -sS http://127.0.0.1:20400/actuator/health/liveness -curl -sS http://127.0.0.1:20400/actuator/health/readiness -curl -sS http://127.0.0.1:20100/actuator/health -curl -sS http://127.0.0.1:20100/actuator/health/liveness -curl -sS http://127.0.0.1:20100/actuator/health/readiness -curl -sS http://127.0.0.1:20500/actuator/health -curl -sS http://127.0.0.1:20500/actuator/health/liveness -curl -sS http://127.0.0.1:20500/actuator/health/readiness -curl -sS http://127.0.0.1:20200/actuator/health -curl -sS http://127.0.0.1:20200/actuator/health/liveness -curl -sS http://127.0.0.1:20200/actuator/health/readiness -curl -sS http://127.0.0.1:20200/v3/api-docs \ - | grep -E '/api/event-history/locations|/api/event-history/raw-frames' -curl -sS http://127.0.0.1:20310/actuator/health -curl -sS http://127.0.0.1:20310/actuator/health/liveness -curl -sS http://127.0.0.1:20310/actuator/health/readiness -``` - -预期：五个容器均为运行状态；健康检查为 `UP`；JT808/GB32960 TCP 端口在 ECS 上监听；OpenAPI 中包含通用位置分页查询和 RAW 帧查询接口。history-app 不暴露 `/api/event-history/telemetry/fields`，也不持续写入逐字段宽表。JT808 位置帧中有 GPS 总里程时，`vehicle-analytics-app` 按差值法写入 MySQL `vehicle_stat_metric` 的 `daily_mileage_km` 指标。 - -## 运行位置约束 - -生产身份绑定固定使用 MySQL，不再提供 file/memory/sqlite 运行时切换入口。生产服务固定运行在 ECS/Portainer，不提供本机守护进程模板。 - -- `jt808-ingest-app` 监听 TCP `808`，HTTP `20400`。 -- `gb32960-ingest-app` 监听 TCP `32960`，HTTP `20100`。 -- `yutong-mqtt-app` 监听 HTTP `20500`。 -- `vehicle-history-app` 监听 HTTP `20200`。 -- `vehicle-analytics-app` 监听 HTTP `20310`。 -- history 热查询只依赖 Kafka 和 TDengine `raw_frames`；history 不需要共享 archive volume。 -- GB32960/JT808/Yutong MQTT 接入服务可以继续使用 `SINK_ARCHIVE_PATH` 保存原始冷备，但这不是 history API 的实时查询前置条件。 - -JT808 注册身份绑定： - -- 生产需要把 0x0100 注册信息维护到 MySQL 时，只需要配置 `VEHICLE_IDENTITY_MYSQL_*` 连接参数。 -- 需要提供 `VEHICLE_IDENTITY_MYSQL_JDBC_URL`、`VEHICLE_IDENTITY_MYSQL_USERNAME`、`VEHICLE_IDENTITY_MYSQL_PASSWORD`。 -- 服务启动时自动创建 `vehicle_identity_binding` 和 `vehicle_identity_binding_registration` 两张表。`vehicle_identity_binding` 只维护 `plate`、`vin` 两列；`vehicle_identity_binding_registration` 按 `protocol + phone` 保存 JT808 0x0100 注册字段。 -- 注册帧无真实 VIN 时也会写入 registration 表，`vin` 默认为 `unknown`。外部系统识别车辆后只需要维护车牌和 VIN，例如：`INSERT INTO vehicle_identity_binding (plate, vin) VALUES ('沪A61559F', 'LNVIN000000000001') ON DUPLICATE KEY UPDATE vin = VALUES(vin);` -- MySQL 绑定在启动时加载到内存索引，`plate/vin` 反写后会按 `VEHICLE_IDENTITY_MYSQL_REFRESH_INTERVAL` 周期刷新，默认 `60s`；帧解析热路径只查内存，不按帧访问 MySQL。 -- 后续同一终端上报 raw/event 时，服务会用 registration 表里的 `phone/device_id/plate` 加绑定表里的 `plate/vin` 解析成真实 VIN。 - -健康检查： - -```bash -curl -sS http://127.0.0.1:20400/actuator/health -curl -sS http://127.0.0.1:20400/actuator/health/liveness -curl -sS http://127.0.0.1:20400/actuator/health/readiness -curl -sS http://127.0.0.1:20100/actuator/health -curl -sS http://127.0.0.1:20100/actuator/health/liveness -curl -sS http://127.0.0.1:20100/actuator/health/readiness -curl -sS http://127.0.0.1:20500/actuator/health -curl -sS http://127.0.0.1:20500/actuator/health/liveness -curl -sS http://127.0.0.1:20500/actuator/health/readiness -curl -sS http://127.0.0.1:20200/actuator/health -curl -sS http://127.0.0.1:20200/actuator/health/liveness -curl -sS http://127.0.0.1:20200/actuator/health/readiness -curl -sS http://127.0.0.1:20310/actuator/health -curl -sS http://127.0.0.1:20310/actuator/health/liveness -curl -sS http://127.0.0.1:20310/actuator/health/readiness -``` - -API / Swagger： - -- JT808 ingest health: `http://127.0.0.1:20400/actuator/health` -- GB32960 ingest health: `http://127.0.0.1:20100/actuator/health` -- Yutong MQTT ingest health: `http://127.0.0.1:20500/actuator/health` -- History query: `http://127.0.0.1:20200/swagger-ui/index.html` -- Analytics metrics: `http://127.0.0.1:20310/swagger-ui/index.html` - -## Kafka Topic 验证 - -历史服务必须消费事件 topic 和 raw topic： - -- `vehicle.event.gb32960.v1` -- `vehicle.raw.gb32960.v1` -- `vehicle.event.jt808.v1` -- `vehicle.raw.jt808.v1` -- `vehicle.event.mqtt-yutong.v1` -- `vehicle.raw.mqtt-yutong.v1` - -使用 Kafka CLI 或管理工具检查 topic 是否存在，并确认 history consumer group 在测试流量后没有持续堆积。默认 `KAFKA_GROUP_HISTORY=vehicle-history` 时，实际 group 会拆成： - -- `vehicle-history-gb32960-event` -- `vehicle-history-gb32960-raw` -- `vehicle-history-jt808-event` -- `vehicle-history-jt808-raw` -- `vehicle-history-yutong-mqtt-event` -- `vehicle-history-yutong-mqtt-raw` - -## JT808 实时转发验收 - -如果外部平台已经把 JT808 报文转发到 ECS TCP `808`，至少观察 60 秒日志和 Kafka 数据。 - -必须拿到以下证据： - -- `jt808-ingest-app` 日志显示接入连接或解析到上游消息 ID。 -- Kafka 的 `vehicle.event.jt808.v1` 和 `vehicle.raw.jt808.v1` 有新增记录。 -- `vehicle-history-app` 日志没有持续出现 consumer 或 TDengine 写入失败。 - -消费完成后，使用已知终端手机号查询： - -```bash -curl -sS 'http://127.0.0.1:20200/api/event-history/jt808/locations?phone=&dateFrom=2026-06-29T00:00:00%2B08:00&dateTo=2026-06-29T23:59:59%2B08:00&limit=10' -``` - -预期：响应包含 `items`，并且每条记录包含 `eventTime`、`phone`、`longitude`、`latitude`、`speedKmh`、`rawUri`、`metadataJson`。 - -## GB32960 Snapshot / 字段投影验收 - -默认高吞吐模式下，GB32960 全字段查询不依赖 telemetry_fields 宽表。history 先查 TDengine `raw_frames`，直接使用 RAW 行中的 `parsedJson`/`parsedFields` 生成 snapshot 和字段投影： - -```bash -curl -sS 'http://127.0.0.1:20200/api/event-history/gb32960/snapshots?vin=&dateFrom=2026-06-29T00:00:00%2B08:00&dateTo=2026-06-29T23:59:59%2B08:00&limit=10' - -curl -sS 'http://127.0.0.1:20200/api/event-history/gb32960/snapshots/fields?vin=&fields=VEHICLE.speedKmh,VEHICLE.totalMileageKm,POSITION_V2016.longitude,POSITION_V2016.latitude&dateFrom=2026-06-29T00:00:00%2B08:00&dateTo=2026-06-29T23:59:59%2B08:00&limit=10' -``` - -预期：snapshot 返回 `sourceFrames.rawArchiveUri` 和解析后的 `blocks`；字段投影返回所选字段。新增协议字段后，先通过 replay 或专用解析任务补写 `raw_frames.parsedJson`，再基于历史 RAW 结构化结果查询。 - -## TDengine 直接检查 - -使用 TDengine CLI 或 JDBC 客户端检查超级表和子表是否创建： - -```sql -USE vehicle_ts; -SHOW STABLES; -SELECT COUNT(*) FROM raw_frames; -SELECT COUNT(*) FROM vehicle_locations; -SELECT COUNT(*) FROM jt808_locations; -``` - -预期： - -- `raw_frames`、`vehicle_locations`、`jt808_locations` 存在；有对应协议实时流量后，计数持续增长。 -- 字段趋势宽表不属于 history-app 验收范围，由独立字段解析服务负责。 -- 有实时流量后，计数持续增加。 - -## 失败语义 - -- TDengine 不可达时，TDengine 查询接口应该返回 HTTP `503`，消息类似 `tdengine history query failed`。 -- TDengine 未启用时，TDengine 专属接口不应该出现在 OpenAPI 中。 -- 路径不存在时，API 层应该返回 HTTP `404`，而不是泛化的存储失败。 - -## 2026-06-29 Superseded Local Live Verification Record - -该记录仅保留为解析和存储行为的历史证据；生产运行位置已经收敛到 ECS/Portainer。 - -当时参与验证的服务： - -- `jt808-ingest-app`：TCP `808`，HTTP `20400` -- `gb32960-ingest-app`：TCP `32960`，HTTP `20100` -- `vehicle-history-app`：HTTP `20200` - -当次健康检查均返回 `{"status":"UP"}`。 - -可重复运行 live 验收工具： - -```bash -python3 tools/vehicle_ingest_live_verify.py \ - --tdengine-rest-url 'http://115.29.185.82:6041/rest/sql/vehicle_ts' \ - --tdengine-username root \ - --tdengine-password '' \ - --history-base-url 'http://127.0.0.1:20200' \ - --date-from '2026-06-29 00:00:00' \ - --date-to '2026-06-29 23:59:59' \ - --jt808-peer-like '222.66.200.68:%' \ - --gb32960-peer-like '115.29.187.205:%' -``` - -该工具输出 `pass`、`warn` 或 `fail`：当前正式 GB32960 只有平台登录、没有车辆 `0x02` 时会输出 `warn`，避免把平台在线误判为车辆数据在线。正式车辆数据验收时增加 `--require-gb32960-vehicle-realtime`，缺少非测试 VIN 的 `0x02` 会直接返回非 0 退出码。 - -JT808 真实转发链路已验证： - -- ECS TCP `808` 有外部平台 `222.66.200.68` 持续发送 JT808 报文。 -- `vehicle.raw.jt808.v1` 和 `vehicle.event.jt808.v1` 被 history 消费。 -- `raw_frames` 中 `protocol='JT808'` 的真实行数已超过 `36000`。 -- 外部平台真实 0x0100 注册帧已超过 `347` 条，真实 0x0200 位置帧已超过 `12400` 条。 -- `jt808_locations` 中位置行数已超过 `8783`。 -- 当前高吞吐默认链路不维护 `telemetry_fields`；JT808 明细验证以 `raw_frames`、`vehicle_locations` 和 API 分页结果为准。 -- 分页 API 已用终端号 `13079963301` 验证第一页和 nextCursor 第二页，样例位置包含 `longitude=118.913846`、`latitude=31.927309`、`statusFlag=3` 和 `archive://2026/06/29/JT808/...` 原始帧引用。 -- 注册帧样例：终端号 `13079963320`、deviceId `9963320`、deviceType `SEG-9888G`、plate `沪A61559F`、maker `70112`、province `31`、city `113`、plateColor `2`。生产 VIN 反写依赖 MySQL identity store，后续 raw/event 的 VIN 解析也从 MySQL 绑定读取。 - -## JT808 ECS smoke / 轻压测工具 - -仓库提供 `tools/jt808_e2e_smoke.py`，用于重复验证 TCP `808` 到 TDengine history 的闭环： - -```bash -export TDENGINE_REST_URL='http://:6041/rest/sql/vehicle_ts' -export TDENGINE_USERNAME='root' -export TDENGINE_PASSWORD='' - -python3 tools/jt808_e2e_smoke.py \ - --connection-mode session \ - --start-phone 13079962000 \ - --frames 3 \ - --expect-history-count 3 \ - --verify-pagination \ - --archive-root "$PROJECT_ROOT/data/archive-jt808" \ - --tdengine-rest-url "$TDENGINE_REST_URL" \ - --tdengine-username "$TDENGINE_USERNAME" \ - --tdengine-password "$TDENGINE_PASSWORD" -``` - -默认 `--connection-mode per-frame`，适合单帧连接 smoke。生产验收建议使用 `--connection-mode session` 覆盖同一连接内连续位置帧。脚本会输出发送数量、history 可见数量、分页验证结果、raw archive 检查结果和 `tdengineRawFrames`。`tdengineRawFrames` 表示本次可见位置记录中的唯一 `rawUri` 有多少个已确认写入 TDengine `raw_frames`。 -- 合成 0x0100 注册帧终端号 `13079969999` 已验证：TCP `808` 返回 `0x8100` 注册 ACK，`raw_frames.metadata_json` 包含 `jt808.register.province`、`jt808.register.city`、`jt808.register.maker`、`jt808.register.deviceType`、`jt808.register.deviceId`、`jt808.register.plateColor`、`jt808.register.plate`，对应 raw archive 文件存在。 - -复查 SQL： - -```sql -USE vehicle_ts; -SELECT COUNT(*) FROM raw_frames WHERE protocol = 'JT808'; -SELECT COUNT(*) FROM jt808_locations WHERE protocol = 'JT808'; -SELECT event_time, vehicle_key, phone, message_id, raw_uri - FROM raw_frames - WHERE protocol = 'JT808' - ORDER BY event_time DESC - LIMIT 5; -SELECT ts, frame_id, phone, message_id, metadata_json, raw_uri - FROM raw_frames - WHERE protocol = 'JT808' AND phone = '13079969999' AND message_id = 256 - ORDER BY ts DESC - LIMIT 3; -``` - -复查 API： - -```bash -curl -sS 'http://127.0.0.1:20200/api/event-history/jt808/locations?phone=13079963301&dateFrom=2026-06-29T00:00:00%2B08:00&dateTo=2026-06-29T23:59:59%2B08:00&order=DESC&limit=3' -curl -sS 'http://127.0.0.1:20200/api/event-history/jt808/raw-frames?phone=13079963320&messageId=256&dateFrom=2026-06-29T00:00:00%2B08:00&dateTo=2026-06-29T23:59:59%2B08:00&order=DESC&limit=3' -``` - -`/api/event-history/jt808/raw-frames` 用于排查注册、鉴权、心跳、位置等原始帧索引。返回项会保留 `messageId`、`messageIdHex`、`rawUri`、`parseStatus`、`peer`、`vin`、`phone` 和完整 `metadataJson`；注册帧的 `metadataJson` 包含 `jt808.register.province`、`jt808.register.city`、`jt808.register.maker`、`jt808.register.deviceType`、`jt808.register.deviceId`、`jt808.register.plateColor`、`jt808.register.plate`、`identitySource`、`identityResolved` 等字段。 - -GB32960 链路已用正式平台登录和合成实时帧验证： - -- 正式平台 `115.29.187.205` 已连接 TCP `32960`，服务收到 `0x05 PLATFORM_LOGIN` 并返回 ACK；日志显示账号 `Hyundai`、策略 `ALLOW`。 -- 2026-06-29 正式平台登录 raw 已有 `3` 条，均落入 TDengine `raw_frames`，可通过 RAW 查询看到 `parsedJson` 中的 `PLATFORM_LOGIN` 解析结果。 -- 当前正式平台尚未发送车辆实时上报 `0x02`；非测试 VIN 的 `message_id=2` 计数为 `0`，因此不能把平台登录误判为车辆数据在线。 -- TCP `32960` 返回 GB32960 `2323` ACK。 -- `raw_frames` 中测试 VIN `LTEST202606290001` 已有 `2` 条 raw 记录。 -- 当前默认通过 GB32960 snapshot/fields 即时解析 `raw_frames` 中的 RAW JSON，不维护字段宽表增长。 -- latest raw 包含 `archive://2026/06/29/GB32960/LTEST202606290001/...`。 -- snapshots API 可查到 `speedKmh=62.4`、`totalMileageKm=223456.7`、`longitude=116.397128`、`latitude=39.916527`。 - -复查 SQL： - -```sql -USE vehicle_ts; -SELECT COUNT(*) FROM raw_frames - WHERE protocol = 'GB32960' AND vin = 'LTEST202606290001'; -SELECT event_time, vin, raw_uri, frame_id - FROM raw_frames - WHERE protocol = 'GB32960' AND vin = 'LTEST202606290001' - ORDER BY event_time DESC - LIMIT 5; -SELECT ts, vin, message_id, event_time, raw_uri, peer, metadata_json - FROM raw_frames - WHERE protocol = 'GB32960' AND peer LIKE '115.29.187.205:%' - ORDER BY ts DESC - LIMIT 5; -SELECT COUNT(*) FROM raw_frames - WHERE protocol = 'GB32960' AND message_id = 2 AND vin <> '' AND vin NOT LIKE 'LTEST%'; -``` - -也可以直接运行仓库 smoke 工具验证 TCP `32960`、GB32960 snapshot/fields、TDengine `raw_frames`。如果需要复核冷备 `.bin`，再传入接入服务 archive 根目录： - -```bash -export TDENGINE_REST_URL='http://:6041/rest/sql/vehicle_ts' -export TDENGINE_USERNAME='root' -export TDENGINE_PASSWORD='' - -python3 tools/gb32960_e2e_smoke.py \ - --archive-root "$PROJECT_ROOT/data/archive-jt808" \ - --tdengine-rest-url "$TDENGINE_REST_URL" \ - --tdengine-username "$TDENGINE_USERNAME" \ - --tdengine-password "$TDENGINE_PASSWORD" -``` - -预期输出包含 `records >= 1`、`fieldCounts` 中关键字段均大于 0、`tdengineRawFrames` 等于本次可见唯一 `rawUri` 数量。传入 `--archive-root` 时，`archiveChecked >= 1` 说明冷备文件也能复核；这里的 `records` 字段表示 snapshot 数量，用于兼容早期脚本输出名。 - -复查 API： - -```bash -curl -sS 'http://127.0.0.1:20200/api/event-history/gb32960/snapshots?vin=LTEST202606290001&dateFrom=2026-06-29T00:00:00%2B08:00&dateTo=2026-06-29T23:59:59%2B08:00&limit=10' -curl -sS 'http://127.0.0.1:20200/api/event-history/gb32960/snapshots/fields?vin=LTEST202606290001&fields=VEHICLE.speedKmh,VEHICLE.totalMileageKm,POSITION_V2016.longitude,POSITION_V2016.latitude&dateFrom=2026-06-29T00:00:00%2B08:00&dateTo=2026-06-29T23:59:59%2B08:00&limit=10' -``` - -## 当前注意事项 - -- 当前生产链路使用 TDengine WebSocket JDBC，例如 `jdbc:TAOS-WS://:6041/vehicle_ts`。 -- `KAFKA_CONSUMER_AUTO_OFFSET_RESET=latest` 适合生产接入新流量；如果要回放历史 Kafka 数据，需要切换 consumer group 或重置 offset。 -- `vehicle-history-app` raw 和 event 使用不同 consumer binding；raw consumer 必须开启，否则 `raw_frames` 不会持续增长。 -- JT808 设备没有 VIN 映射时，`vehicle_key` 使用 `jt808:`；启用 MySQL identity store 后，registration 表中已反写 VIN 的 phone/deviceId/plate 会用于后续 raw/event 的 VIN 解析。 -- GB32960 正式对端目前只验证到平台登录；等待真实车辆 `0x02` 到达后，再用非测试 VIN 复查 `vehicle_locations`、snapshot/fields API 和导出能力。 diff --git a/docs/superpowers/plans/2026-04-20-gb32960-body-parser-block-isolation.md b/docs/superpowers/plans/2026-04-20-gb32960-body-parser-block-isolation.md deleted file mode 100644 index ad1c2ff3..00000000 --- a/docs/superpowers/plans/2026-04-20-gb32960-body-parser-block-isolation.md +++ /dev/null @@ -1,600 +0,0 @@ -# GB32960 Body Parser — Per-Block Exception Isolation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** `Gb32960BodyParser` 主循环对单信息块的解析异常实现**单块隔离**：一块解析失败不再整帧放弃，改为把失败块兜成 `InfoBlock.Raw` 并继续解析后续信息块。 - -**Architecture:** 在主循环每次 `parser.parse(body)` 外面套 try/catch（仅捕获 `DecodeException` / `BufferUnderflowException` / `IndexOutOfBoundsException`，放行 RuntimeException 以免掩盖 parser bug）。失败时： -- 若 `fixedLen ≥ 0` 且剩余字节 ≥ fixedLen → 回滚 reader → 按 fixedLen 截取 Raw → position 前进 fixedLen → `continue` 循环 -- 否则（变长 parser 或剩余不足）→ 剩余字节全部兜成 Raw → `break` 循环（变长块无法安全找下一块边界） - -通过 `Gb32960Properties.Parse.lenientBlockFailure`（默认 true）控制开关，可以回退到严格模式。**不改 `InfoBlock.Raw` record 结构**（避免 downstream 兼容风险——`Gb32960EventMapper` 按 `findBlock(Class)` 类型查找，新增 Raw 不影响其行为）。 - -**Tech Stack:** Java 25, JUnit 5, AssertJ, Spring Boot ConfigurationProperties, Maven。仅改动 `protocol-gb32960` 模块。 - ---- - -## File Structure - -- **Modify** `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960Properties.java` — 新增 `Parse` 内嵌类 + `parse` 字段/getter/setter -- **Modify** `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParser.java` — 主循环 try/catch + recovery 逻辑 + `lenientBlockFailure` 字段 -- **Modify** `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfiguration.java` — 把配置穿给 BodyParser -- **Create** `protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParserIsolationTest.java` — 3 个隔离场景 + 1 个严格模式回退场景 -- **Modify** `bootstrap-all/src/main/resources/application.yml` — 添加 `parse:` 注释段示例 -- **Modify** `CHANGELOG.md` — 追加本次变更条目 - ---- - -## Task 1: 基线验证 —— 现有测试全绿 - -**Files:** 无 - -- [ ] **Step 1.1: 跑 protocol-gb32960 模块测试** - -Run: `mvn -pl protocol-gb32960 test -q` -Expected: BUILD SUCCESS，所有现有测试通过（`Gb32960DecoderGoldenTest`、`Gb32960FullBlocksTest`、`Gb32960DecoderTest`、`GuangdongFcEndToEndTest`、parser/* 各 Block Parser 单测、profile/* 选择器单测）。 - -如果基线红了，**先停下来修复基线**再进入 Task 2。 - ---- - -## Task 2: 添加 Parse 配置子节点 - -**Files:** -- Modify: `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960Properties.java` - -- [ ] **Step 2.1: 在 `Gb32960Properties` 类体中（在 `Auth` 字段后、`Tls` 字段前）新增 `parse` 字段** - -位置参考：现在 L24 `private Auth auth = new Auth();` 下面一行，在 L27 `private Tls tls = new Tls();` 之前插入： - -```java - /** - * 报文解析行为配置。 - * - *

默认 {@code lenientBlockFailure=true}：单个信息块解析异常时兜成 - * {@link com.lingniu.ingest.protocol.gb32960.model.InfoBlock.Raw} 继续解析， - * 不再整帧放弃。关闭后恢复旧行为（任意异常直接抛 {@code DecodeException} - * 放弃整帧），仅在灰度回滚时使用。 - */ - private Parse parse = new Parse(); -``` - -- [ ] **Step 2.2: 在类底部（`Tls` 静态类之后、`VendorExtension` 静态类之前）添加 `Parse` 静态内嵌类** - -```java - /** - * 报文解析容错策略。 - */ - public static class Parse { - /** - * 单块解析异常时是否兜底为 {@link com.lingniu.ingest.protocol.gb32960.model.InfoBlock.Raw} - * 继续解析。默认开启。 - * - *

    - *
  • {@code true}（默认）：parser 抛 DecodeException / BufferUnderflowException / - * IndexOutOfBoundsException 时，固定长度块按 fixedLen 截取 Raw 后 continue； - * 变长块或剩余字节不足时，剩余全部兜成 Raw 后 break 循环。 - *
  • {@code false}：保留旧行为，任意异常直接抛出 DecodeException 放弃整帧。 - *
- */ - private boolean lenientBlockFailure = true; - - public boolean isLenientBlockFailure() { return lenientBlockFailure; } - public void setLenientBlockFailure(boolean lenientBlockFailure) { - this.lenientBlockFailure = lenientBlockFailure; - } - } -``` - -- [ ] **Step 2.3: 在属性 getter/setter 区域（现有 `getTls` 之后）补充 `getParse` / `setParse`** - -```java - public Parse getParse() { return parse; } - public void setParse(Parse parse) { this.parse = parse; } -``` - -- [ ] **Step 2.4: 编译验证** - -Run: `mvn -pl protocol-gb32960 compile -q` -Expected: BUILD SUCCESS。 - -- [ ] **Step 2.5: 提交** - -```bash -git add protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960Properties.java -git commit -m "$(cat <<'EOF' -config(gb32960): add parse.lenientBlockFailure toggle - -新增 lingniu.ingest.gb32960.parse.lenientBlockFailure 配置开关（默认 true）， -为 Gb32960BodyParser 的单块异常隔离行为做回退开关。实现在后续 commit。 - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - ---- - -## Task 3: RED —— 写三个失败测试覆盖隔离场景 - -**Files:** -- Create: `protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParserIsolationTest.java` - -- [ ] **Step 3.1: 创建测试文件** - -完整内容： - -```java -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.api.spi.DecodeException; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.AlarmV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.PositionV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlockType; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayOutputStream; -import java.nio.ByteBuffer; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -/** - * 验证 {@link Gb32960BodyParser} 单块异常隔离行为。 - * - *

三个隔离场景： - *

    - *
  1. 固定长度块 parser 抛异常 → 失败块兜 Raw（按 fixedLen 截取）+ 后续块继续解析 - *
  2. 固定长度块剩余字节不足（截断帧） → 兜 Raw(剩余) + break 循环 - *
  3. 变长块 parser 抛异常 → 兜 Raw(从失败块起剩余全部) + break 循环 - *
- * - *

外加一个严格模式回退测试：{@code lenientBlockFailure=false} 时任意异常应抛 DecodeException。 - */ -class Gb32960BodyParserIsolationTest { - - /** 构造一个"看起来合法但中途 parser 失败"用来注入失败的 Position parser。 */ - private static final InfoBlockParser EXPLODING_FIXED_LEN_POSITION = new InfoBlockParser() { - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x05; } - @Override public int fixedLength() { return 9; } - @Override public InfoBlock parse(ByteBuffer buffer) { - // 模拟 parser 读了几字节后发现不合法就抛 - buffer.get(); buffer.get(); - throw new DecodeException("simulated parser failure in Position"); - } - }; - - /** 变长块（fixedLength=-1），parse 时消费若干字节后抛。模拟 Alarm/Voltage 类列表读越界。 */ - private static final InfoBlockParser EXPLODING_VAR_LEN_ALARM = new InfoBlockParser() { - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x07; } - @Override public int fixedLength() { return -1; } - @Override public InfoBlock parse(ByteBuffer buffer) { - // 假装读了个长度字段就爆 - buffer.get(); - throw new DecodeException("simulated parser failure in Alarm list-length read"); - } - }; - - @Test - void fixedLengthBlockFailure_isIsolated_subsequentBlocksStillParsed() { - // 帧布局：Vehicle(0x01, 20B) + Position(0x05, 9B 但 parser 爆) + Engine(0x04) 不构造， - // 简化为 Vehicle + FailingPosition + Vehicle 再次，验证"Position 之后还能继续"。 - InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of( - new VehicleV2016BlockParser(), - EXPLODING_FIXED_LEN_POSITION)); - - Gb32960BodyParser parser = new Gb32960BodyParser(registry); - - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeValidVehicle(os); // 1+20 = 21B - writePositionTypeAnd9ByteBody(os); // 1+9 = 10B (parser 会爆) - writeValidVehicle(os); // 1+20 = 21B - ByteBuffer body = ByteBuffer.wrap(os.toByteArray()); - - var result = parser.parse(ProtocolVersion.V2016, body); - - assertThat(result.blocks()).hasSize(3); - assertThat(result.blocks().get(0)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class); - assertThat(result.blocks().get(1)).isInstanceOfSatisfying(InfoBlock.Raw.class, raw -> { - assertThat(raw.typeCode()).isEqualTo(0x05); - assertThat(raw.type()).isEqualTo(InfoBlockType.RAW); - assertThat(raw.bytes()).hasSize(9); // 按 fixedLen 截取 - }); - assertThat(result.blocks().get(2)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class); - assertThat(body.hasRemaining()).isFalse(); - } - - @Test - void truncatedFixedLengthBlock_isWrappedAsRaw_loopTerminates() { - // Vehicle(21B) + Position(type 0x05)但 body 只给 3B —— 不足 9B - InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of( - new VehicleV2016BlockParser(), - new PositionV2016BlockParser())); - - Gb32960BodyParser parser = new Gb32960BodyParser(registry); - - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeValidVehicle(os); - os.write(0x05); // Position typeCode - os.write(0); os.write(0); os.write(0); // 只 3B，远远不够 9B - ByteBuffer body = ByteBuffer.wrap(os.toByteArray()); - - var result = parser.parse(ProtocolVersion.V2016, body); - - assertThat(result.blocks()).hasSize(2); - assertThat(result.blocks().get(0)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class); - assertThat(result.blocks().get(1)).isInstanceOfSatisfying(InfoBlock.Raw.class, raw -> { - assertThat(raw.typeCode()).isEqualTo(0x05); - assertThat(raw.bytes()).hasSize(3); // 剩余字节全兜成 Raw - }); - } - - @Test - void variableLengthBlockFailure_swallowsRemainderAsRaw_loopBreaks() { - // Vehicle(21B) + FailingAlarm(0x07)后面还有一个 Vehicle，但因 Alarm 变长无法安全跳过， - // 整段 Alarm 起的剩余字节都被兜成 Raw 后 break。 - InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of( - new VehicleV2016BlockParser(), - EXPLODING_VAR_LEN_ALARM)); - - Gb32960BodyParser parser = new Gb32960BodyParser(registry); - - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeValidVehicle(os); // 21B - os.write(0x07); // Alarm typeCode - for (int i = 0; i < 10; i++) os.write(0xAA); // 10B 的 alarm body（parser 爆） - writeValidVehicle(os); // 21B（应当不再被解析） - ByteBuffer body = ByteBuffer.wrap(os.toByteArray()); - - var result = parser.parse(ProtocolVersion.V2016, body); - - assertThat(result.blocks()).hasSize(2); - assertThat(result.blocks().get(0)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class); - assertThat(result.blocks().get(1)).isInstanceOfSatisfying(InfoBlock.Raw.class, raw -> { - assertThat(raw.typeCode()).isEqualTo(0x07); - // 剩余 10B alarm body + 1+20=21B 后续 Vehicle = 31B - assertThat(raw.bytes()).hasSize(31); - }); - } - - @Test - void strictMode_throwsOnAnyBlockFailure() { - InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of( - new VehicleV2016BlockParser(), - EXPLODING_FIXED_LEN_POSITION)); - - Gb32960BodyParser parser = new Gb32960BodyParser(registry); - parser.setLenientBlockFailure(false); - - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeValidVehicle(os); - writePositionTypeAnd9ByteBody(os); - ByteBuffer body = ByteBuffer.wrap(os.toByteArray()); - - assertThatThrownBy(() -> parser.parse(ProtocolVersion.V2016, body)) - .isInstanceOf(DecodeException.class); - } - - // ------------------------------------------------------------------------ - // helpers - // ------------------------------------------------------------------------ - - /** 写一个完整的合法 V2016 Vehicle 块（typeCode + 20B body）。 */ - private static void writeValidVehicle(ByteArrayOutputStream os) { - os.write(0x01); // typeCode Vehicle - os.write(0x01); // vehicleState=1 - os.write(0x01); // chargingState=1 - os.write(0x01); // runningMode=1 - os.write(0); os.write(0); // speed=0 - os.write(0); os.write(0); os.write(0); os.write(0); // mileage=0 - os.write(0); os.write(0); // totalVoltage=0 - os.write(0); os.write(0); // totalCurrent=0（offset 1000，原值0） - os.write(50); // soc=50% - os.write(0x01); // dcdc - os.write(0); // gear - os.write(0); os.write(0); // insulation - os.write(0); // accelerator - os.write(0); // brake - } - - /** 写 Position typeCode(0x05) + 9 字节 body（内容不重要，parser 替换成 EXPLODING 的）。 */ - private static void writePositionTypeAnd9ByteBody(ByteArrayOutputStream os) { - os.write(0x05); - for (int i = 0; i < 9; i++) os.write(0); - } -} -``` - -- [ ] **Step 3.2: 跑测试验证 RED** - -Run: `mvn -pl protocol-gb32960 test -Dtest=Gb32960BodyParserIsolationTest -q` -Expected: -- `fixedLengthBlockFailure_isIsolated_subsequentBlocksStillParsed` FAIL（当前会抛 DecodeException） -- `truncatedFixedLengthBlock_isWrappedAsRaw_loopTerminates` FAIL（当前 L132 会抛 DecodeException） -- `variableLengthBlockFailure_swallowsRemainderAsRaw_loopBreaks` FAIL（异常冒出整帧失败） -- `strictMode_throwsOnAnyBlockFailure` FAIL（没有 `setLenientBlockFailure` 方法，编译就红） - -**验证点**：编译失败（strictMode test 里 `setLenientBlockFailure` 尚未存在）是**预期** RED 信号——下一步实现。 - ---- - -## Task 4: GREEN —— 实现单块异常隔离 - -**Files:** -- Modify: `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParser.java` - -- [ ] **Step 4.1: 在类字段区新增 `lenientBlockFailure` 字段 + setter** - -在 `private final VendorExtensionSelector selector;`（现 L54）下面插入： - -```java - /** - * 单块解析异常时是否兜底为 Raw 后继续。由 - * {@link com.lingniu.ingest.protocol.gb32960.config.Gb32960Properties.Parse#isLenientBlockFailure()} - * 注入；默认 true。 - */ - private boolean lenientBlockFailure = true; - - public void setLenientBlockFailure(boolean lenientBlockFailure) { - this.lenientBlockFailure = lenientBlockFailure; - } -``` - -- [ ] **Step 4.2: 改造主循环：替换 L130~L150 的整个 "parse + 长度校验" 段** - -**完整替换块**：以下代码替换原来从 `int fixedLen = parser.fixedLength();`（L130）开始到 `blocks.add(block);`（L150）结束的整块。 - -```java - int fixedLen = parser.fixedLength(); - int posBefore = body.position(); - - // 旧行为：fixedLen 预检不足 → 直接抛。新行为（lenient 模式）：走统一恢复路径。 - if (fixedLen >= 0 && body.remaining() < fixedLen) { - if (!lenientBlockFailure) { - throw new DecodeException( - "info block 0x" + Integer.toHexString(typeCode) - + " needs " + fixedLen + " bytes but got " + body.remaining()); - } - // 剩余字节数不足 fixedLen —— 无法按块截取，整尾兜 Raw + break - int remaining = body.remaining(); - byte[] tail = new byte[remaining]; - body.get(tail); - log.warn("[gb32960] truncated block typeCode=0x{} declaredFixedLen={} remaining={} — wrapping tail as Raw", - Integer.toHexString(typeCode), fixedLen, remaining); - blocks.add(new InfoBlock.Raw(typeCode, InfoBlockType.RAW, tail)); - break; - } - - InfoBlock block; - try { - block = parser.parse(body); - } catch (DecodeException | java.nio.BufferUnderflowException | IndexOutOfBoundsException e) { - if (!lenientBlockFailure) { - if (e instanceof DecodeException de) throw de; - throw new DecodeException( - "parser " + parser.getClass().getSimpleName() + " failed: " + e.getMessage(), e); - } - // 恢复路径：先回滚 position 到 parse 入口 - body.position(posBefore); - if (fixedLen >= 0) { - // 固定长度块：按 fixedLen 截取 → 兜 Raw → continue 循环 - byte[] corrupt = new byte[fixedLen]; - body.get(corrupt); - log.warn("[gb32960] block parse failed typeCode=0x{} parser={} fixedLen={} — isolated as Raw, continuing", - Integer.toHexString(typeCode), parser.getClass().getSimpleName(), fixedLen, e); - blocks.add(new InfoBlock.Raw(typeCode, InfoBlockType.RAW, corrupt)); - continue; - } else { - // 变长块：无法安全找下一块边界 → 剩余整段兜 Raw + break - int remaining = body.remaining(); - byte[] tail = new byte[remaining]; - body.get(tail); - log.warn("[gb32960] variable-length block parse failed typeCode=0x{} parser={} remaining={} — wrapping remainder as Raw, stopping loop", - Integer.toHexString(typeCode), parser.getClass().getSimpleName(), remaining, e); - blocks.add(new InfoBlock.Raw(typeCode, InfoBlockType.RAW, tail)); - break; - } - } - - int consumed = body.position() - posBefore; - if (fixedLen >= 0 && consumed != fixedLen) { - // 这是 parser 契约违反（声明 fixedLen=X 但实际读了 Y），保持抛异常以暴露 parser bug。 - throw new DecodeException( - "parser " + parser.getClass().getSimpleName() - + " consumed " + consumed - + " bytes but declared " + fixedLen); - } - if (log.isDebugEnabled()) { - log.debug("[gb32960] block typeCode=0x{} parser={} pos {}→{} consumed={} declaredFixed={}", - Integer.toHexString(typeCode), parser.getClass().getSimpleName(), - typeStartPos, body.position(), consumed, fixedLen); - } - blocks.add(block); -``` - -- [ ] **Step 4.3: 运行隔离测试验证 GREEN** - -Run: `mvn -pl protocol-gb32960 test -Dtest=Gb32960BodyParserIsolationTest -q` -Expected: 4 个测试全部 PASS。 - -- [ ] **Step 4.4: 运行全模块测试检查回归** - -Run: `mvn -pl protocol-gb32960 test -q` -Expected: BUILD SUCCESS，所有原有测试（含 Golden、FullBlocks、GuangdongFcEndToEnd）依旧通过。 - -如果有原有测试红了：**停下来诊断**。最可能的原因是某个现有测试之前靠"抛异常"行为验证错误帧的，需要手动判断这是测试问题还是实现问题。 - -- [ ] **Step 4.5: 提交** - -```bash -git add protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParser.java \ - protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParserIsolationTest.java -git commit -m "$(cat <<'EOF' -feat(gb32960): isolate per-block parse failures in body parser - -主循环改造：单信息块 parser 抛 DecodeException / BufferUnderflowException / -IndexOutOfBoundsException 时不再放弃整帧。 - -- 固定长度块（fixedLen≥0）：回滚 reader，按 fixedLen 截取字节兜成 InfoBlock.Raw， - position 前进 fixedLen，continue 循环继续解析后续块 -- 变长块（fixedLen=-1）或剩余不足 fixedLen：剩余字节全部兜成 Raw 后 break -- parser 契约违反（consumed != declared fixedLen）仍抛异常，以暴露 parser bug -- 通过 lingniu.ingest.gb32960.parse.lenientBlockFailure=false 可回退严格模式 -- 新增 Gb32960BodyParserIsolationTest 覆盖 3 种失败场景 + 1 严格模式 - -不改 InfoBlock.Raw record 结构；下游 Gb32960EventMapper 按 findBlock(Class) 类型 -查找，新增 Raw 不影响其行为。 - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - ---- - -## Task 5: 把配置注入 BodyParser（AutoConfiguration 连线） - -**Files:** -- Modify: `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfiguration.java` - -- [ ] **Step 5.1: 定位 `Gb32960BodyParser` bean 定义** - -先查看：`grep -n "Gb32960BodyParser" protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfiguration.java` - -根据返回的行号，在构造 `Gb32960BodyParser` 的 @Bean 方法里，构造完成后立即调用： - -```java - Gb32960BodyParser parser = new Gb32960BodyParser(profileRegistry, selector); - parser.setLenientBlockFailure(properties.getParse().isLenientBlockFailure()); - return parser; -``` - -（如果目前的构造返回一行表达式，改成先赋给 local 变量再设 flag 再 return。） - -- [ ] **Step 5.2: 编译并运行全模块测试** - -Run: `mvn -pl protocol-gb32960 test -q` -Expected: BUILD SUCCESS。 - -- [ ] **Step 5.3: 提交** - -```bash -git add protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfiguration.java -git commit -m "$(cat <<'EOF' -config(gb32960): wire parse.lenientBlockFailure into BodyParser bean - -Gb32960AutoConfiguration 在创建 Gb32960BodyParser bean 后，将 -Gb32960Properties.Parse.lenientBlockFailure 通过 setter 注入，让运行期配置生效。 - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - ---- - -## Task 6: bootstrap-all 配置文档化 - -**Files:** -- Modify: `bootstrap-all/src/main/resources/application.yml` - -- [ ] **Step 6.1: 在 `lingniu.ingest.gb32960` 节点的 `vendor-extensions` 段之后、`jt808` 之前，插入 parse 配置示例** - -参考位置：现有 application.yml L57 末尾（`vendor-extensions` list 结束）。在 L58 `jt808:` 之前加入： - -```yaml - # 报文解析容错。默认启用单块异常隔离：某个信息块解析失败时兜成 Raw 后继续， - # 不再整帧丢弃。仅在需要严格失败语义（灰度回滚或抓虫）时设为 false。 - parse: - lenient-block-failure: true -``` - -- [ ] **Step 6.2: 启动一次 bootstrap-all 的 dry-run 编译** - -Run: `mvn -pl bootstrap-all compile -q` -Expected: BUILD SUCCESS（只是配置段注释变化，不会影响编译）。 - -- [ ] **Step 6.3: 提交** - -```bash -git add bootstrap-all/src/main/resources/application.yml -git commit -m "$(cat <<'EOF' -config(gb32960): document parse.lenient-block-failure in application.yml - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - ---- - -## Task 7: 全仓库回归测试 - -**Files:** 无（只是验证） - -- [ ] **Step 7.1: 跑完整项目测试** - -Run: `mvn test -q` -Expected: BUILD SUCCESS。重点关注： -- `protocol-gb32960` 模块全绿（Golden、FullBlocks、Isolation、GuangdongFcEndToEnd、Decoder、Mapper、所有 parser 单测、profile） -- 其它模块（`ingest-core`、`sink-kafka` 等）不受影响（无改动应自动通过） - -若有红，停下来诊断。 - ---- - -## Task 8: 更新 CHANGELOG - -**Files:** -- Modify: `CHANGELOG.md` - -- [ ] **Step 8.1: 在 CHANGELOG.md 文件顶部（在 `## [0.1.0] — 2026-04-15` 标题之前）新增一个 `## [Unreleased]` 段落；如已存在则追加** - -新增段落内容（如已存在 Unreleased 段则在其 `### Added` / `### Changed` 内部追加）： - -```markdown -## [Unreleased] - -### Changed —— GB/T 32960 Body Parser 单块异常隔离 -- `Gb32960BodyParser` 主循环对单信息块的 parser 异常不再放弃整帧： - - 固定长度块（`fixedLength ≥ 0`）：回滚 reader → 按 fixedLen 截取字节兜成 - `InfoBlock.Raw` → 继续解析后续块； - - 变长块或剩余字节不足：剩余字节全部兜成 `Raw` 后终止循环； - - `parser` 契约违反（声明 fixedLen=X 但实际读了 Y）仍抛 `DecodeException`， - 以暴露 parser bug，不被静默吞掉； - - 只捕获 `DecodeException` / `BufferUnderflowException` / `IndexOutOfBoundsException`， - `RuntimeException` 继续向上抛，保留 bug 可见性。 -- 新增配置 `lingniu.ingest.gb32960.parse.lenient-block-failure`（默认 `true`）； - 回退严格模式用 `false`。 -- `InfoBlock.Raw` record 结构**未变**，下游 `Gb32960EventMapper` 按 `findBlock(Class)` - 类型匹配，新增 Raw 不影响业务事件映射。 -- 覆盖测试：`Gb32960BodyParserIsolationTest` —— 3 种隔离场景 + 严格模式回退。 -``` - -- [ ] **Step 8.2: 提交** - -```bash -git add CHANGELOG.md -git commit -m "$(cat <<'EOF' -docs(changelog): record gb32960 body parser block isolation - -Co-Authored-By: Claude Opus 4.7 (1M context) -EOF -)" -``` - ---- - -## Self-Review Checklist - -- [x] **Spec coverage**: 原讨论的 A1~A7 全部覆盖——A1 状态机（Task 4 Step 4.2）、A2 三类错误策略（同）、A3 收窄异常种类（同，只 catch 三种）、A4 日志分类（warn 带 parser 名/typeCode/原因）、A5 配置开关（Task 2）、A6 合成帧测试（Task 3 三场景 + 严格回退）、A7 下游兼容（不改 Raw record，`findBlock` 类型匹配不受影响，plan 中已说明）。 -- [x] **Placeholder scan**: 无 TBD / TODO / "实现上面的" 等占位；每个 Step 含实际代码 or 命令。 -- [x] **Type consistency**: `lenientBlockFailure` 在 Properties / BodyParser 字段 / setter / 测试中拼写一致。`InfoBlockType.RAW` 枚举值依赖当前存在（已核对 InfoBlock.Raw record）。 -- [x] **测试命名一致**：`Gb32960BodyParserIsolationTest` 在 Task 3 创建、Task 4/7 引用一致。 -- [x] **未引入新枚举/proto 变更**：InfoBlock.Raw 沿用现有 `(int typeCode, InfoBlockType type, byte[] bytes)` 构造。 diff --git a/docs/superpowers/plans/2026-06-23-gb32960-history-duckdb-hot-store.md b/docs/superpowers/plans/2026-06-23-gb32960-history-duckdb-hot-store.md deleted file mode 100644 index 8e118f95..00000000 --- a/docs/superpowers/plans/2026-06-23-gb32960-history-duckdb-hot-store.md +++ /dev/null @@ -1,817 +0,0 @@ -> **Superseded:** This 2026-06-23 DuckDB hot-store plan is historical context. -> Use `docs/target-architecture.md` and -> `docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.md` for the -> current production architecture: TDengine stores hot history, `sink-archive` -> owns raw bytes, and `event-file-store` / DuckDB are no longer part of the -> current build surface. - -# GB32960 History DuckDB Hot Store Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Replace the current high-write-risk Parquet rewrite history path with a DuckDB hot history store that supports fast `vin + time` queries and full RAW frame replay. - -**Architecture:** Add a new `DuckDbHotEventFileStore` implementation behind the existing `EventFileStore` interface, backed by append/upsert DuckDB tables instead of rewriting per-vehicle Parquet files. Keep the current HTTP history API and RAW archive reader intact, then switch `vehicle-history-app` to the hot DuckDB store by configuration. - -**Tech Stack:** Java 26, Spring Boot, DuckDB JDBC, JUnit 5, AssertJ, existing Kafka `VehicleEnvelope`, existing `ArchiveStore`. - ---- - -## Scope - -This plan implements Phase 1 from `docs/superpowers/specs/2026-06-23-gb32960-production-readiness-design.md`. - -Included: - -- DuckDB hot event table for `EventFileRecord`. -- Idempotent writes by `event_id`. -- Query by protocol/date/VIN/eventType/eventTime/order/limit. -- Lookup by `rawArchiveUri`. -- Spring configuration to select the hot store. -- Tests that prove history endpoints still replay RAW frames. -- Local verification using the existing `vehicle-history-app`. - -Not included in this plan: - -- MySQL daily statistics. -- VIN-to-platform local mapping. -- Full telemetry point columnar table. -- Alarm timeline MySQL tables. -- Cold Parquet export. - -Those will be separate plans after this foundation lands. - -## Files - -- Create: `modules/sinks/event-file-store/src/main/java/com/lingniu/ingest/eventfilestore/DuckDbHotEventFileStore.java` -- Create: `modules/sinks/event-file-store/src/test/java/com/lingniu/ingest/eventfilestore/DuckDbHotEventFileStoreTest.java` -- Modify: `modules/sinks/event-file-store/src/main/java/com/lingniu/ingest/eventfilestore/config/EventFileStoreProperties.java` -- Modify: `modules/sinks/event-file-store/src/main/java/com/lingniu/ingest/eventfilestore/config/EventFileStoreAutoConfiguration.java` -- Modify: `modules/sinks/event-file-store/src/test/java/com/lingniu/ingest/eventfilestore/config/EventFileStoreAutoConfigurationTest.java` -- Modify: `modules/apps/vehicle-history-app/src/main/resources/application.yml` -- Modify: `modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppDefaultsTest.java` -- Modify: `modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Gb32960DecodedFrameServiceTest.java` - -## Task 1: Add Hot Store Configuration - -**Files:** -- Modify: `modules/sinks/event-file-store/src/main/java/com/lingniu/ingest/eventfilestore/config/EventFileStoreProperties.java` -- Modify: `modules/sinks/event-file-store/src/main/java/com/lingniu/ingest/eventfilestore/config/EventFileStoreAutoConfiguration.java` -- Test: `modules/sinks/event-file-store/src/test/java/com/lingniu/ingest/eventfilestore/config/EventFileStoreAutoConfigurationTest.java` - -- [ ] **Step 1: Write the failing auto-configuration test** - -Add a test that sets `lingniu.ingest.event-file-store.storage=duckdb-hot` and expects the bean class to be `DuckDbHotEventFileStore`. - -```java -@Test -void createsDuckDbHotStoreWhenStorageIsDuckDbHot() { - contextRunner - .withPropertyValues( - "lingniu.ingest.event-file-store.enabled=true", - "lingniu.ingest.event-file-store.storage=duckdb-hot", - "lingniu.ingest.event-file-store.path=" + tempDir.resolve("history")) - .run(context -> assertThat(context) - .hasSingleBean(EventFileStore.class) - .getBean(EventFileStore.class) - .isInstanceOf(DuckDbHotEventFileStore.class)); -} -``` - -- [ ] **Step 2: Run the new test and verify it fails** - -Run: - -```bash -mvn -pl :event-file-store -Dtest=EventFileStoreAutoConfigurationTest#createsDuckDbHotStoreWhenStorageIsDuckDbHot test -``` - -Expected: compilation failure because `DuckDbHotEventFileStore` and `storage` property do not exist. - -- [ ] **Step 3: Add the `storage` property** - -Add to `EventFileStoreProperties`: - -```java -/** - * Storage backend. `duckdb-hot` is the production backend; `parquet-sidecar` - * keeps the previous implementation available for compatibility tests. - */ -private String storage = "duckdb-hot"; - -public String getStorage() { - return storage; -} - -public void setStorage(String storage) { - this.storage = storage; -} -``` - -- [ ] **Step 4: Switch auto-configuration by storage mode** - -Change `eventFileStore(...)` to: - -```java -@Bean -@ConditionalOnMissingBean -public EventFileStore eventFileStore(EventFileStoreProperties properties, - ObjectProvider objectMapper) { - ObjectMapper mapper = mapper(objectMapper); - Path root = Path.of(properties.getPath()); - ZoneId zoneId = ZoneId.of(properties.getZoneId()); - String storage = properties.getStorage() == null ? "" : properties.getStorage().trim(); - return switch (storage) { - case "", "duckdb-hot" -> new DuckDbHotEventFileStore(root, zoneId, mapper); - case "parquet-sidecar" -> new DuckDbParquetEventFileStore(root, zoneId, mapper); - default -> throw new IllegalStateException( - "unsupported event-file-store storage: " + properties.getStorage()); - }; -} -``` - -Import `DuckDbHotEventFileStore`. - -- [ ] **Step 5: Run configuration tests** - -Run: - -```bash -mvn -pl :event-file-store -Dtest=EventFileStoreAutoConfigurationTest test -``` - -Expected: tests pass after the store class exists in Task 2. - -## Task 2: Implement DuckDB Hot Store Schema and Writes - -**Files:** -- Create: `modules/sinks/event-file-store/src/main/java/com/lingniu/ingest/eventfilestore/DuckDbHotEventFileStore.java` -- Create: `modules/sinks/event-file-store/src/test/java/com/lingniu/ingest/eventfilestore/DuckDbHotEventFileStoreTest.java` - -- [ ] **Step 1: Write failing append/query/idempotency tests** - -Create `DuckDbHotEventFileStoreTest`: - -```java -class DuckDbHotEventFileStoreTest { - - @TempDir - Path tempDir; - - @Test - void appendsRecordsAndQueriesByVinTypeAndTime() throws Exception { - EventFileStore store = store(); - store.append(rawRecord("raw-1", "VIN001", "2026-06-23T01:00:00Z")); - store.append(rawRecord("raw-2", "VIN002", "2026-06-23T01:00:01Z")); - store.append(rawRecord("raw-3", "VIN001", "2026-06-23T01:00:02Z")); - - EventFileQuery query = new EventFileQuery( - ProtocolId.GB32960, - LocalDate.parse("2026-06-23"), - LocalDate.parse("2026-06-23"), - Instant.parse("2026-06-23T01:00:00Z"), - Instant.parse("2026-06-23T01:00:03Z"), - EventFileQuery.Order.DESC, - 2, - "VIN001", - "RAW_ARCHIVE"); - - assertThat(store.query(query)) - .extracting(EventFileRecord::eventId) - .containsExactly("raw-3", "raw-1"); - } - - @Test - void appendAllIsIdempotentByEventId() throws Exception { - EventFileStore store = store(); - EventFileRecord original = rawRecord("same-id", "VIN001", "2026-06-23T01:00:00Z"); - EventFileRecord replacement = new EventFileRecord( - "same-id", - ProtocolId.GB32960, - "RAW_ARCHIVE", - "VIN001", - Instant.parse("2026-06-23T01:00:05Z"), - Instant.parse("2026-06-23T01:00:06Z"), - "archive://replacement.bin", - Map.of("source", "replacement"), - "{\"replacement\":true}"); - - store.appendAll(List.of(original, replacement)); - - assertThat(store.query(new EventFileQuery( - ProtocolId.GB32960, - LocalDate.parse("2026-06-23"), - LocalDate.parse("2026-06-23"), - EventFileQuery.Order.ASC, - 10, - "VIN001", - "RAW_ARCHIVE"))) - .singleElement() - .satisfies(record -> { - assertThat(record.eventId()).isEqualTo("same-id"); - assertThat(record.rawArchiveUri()).isEqualTo("archive://replacement.bin"); - }); - } - - @Test - void findsRecordByRawArchiveUri() throws Exception { - EventFileStore store = store(); - EventFileRecord record = rawRecord("raw-uri", "VIN001", "2026-06-23T01:00:00Z"); - store.append(record); - - EventFileRecord found = store.findByRawArchiveUri(record.rawArchiveUri()); - - assertThat(found).isNotNull(); - assertThat(found.eventId()).isEqualTo("raw-uri"); - } - - private EventFileStore store() { - return new DuckDbHotEventFileStore(tempDir, ZoneId.of("Asia/Shanghai"), new ObjectMapper()); - } - - private static EventFileRecord rawRecord(String id, String vin, String eventTime) { - return new EventFileRecord( - id, - ProtocolId.GB32960, - "RAW_ARCHIVE", - vin, - Instant.parse(eventTime), - Instant.parse(eventTime).plusMillis(100), - "archive://" + id + ".bin", - Map.of("platformAccount", "Hyundai", "command", "REALTIME_REPORT"), - "{\"eventId\":\"" + id + "\"}"); - } -} -``` - -- [ ] **Step 2: Run the test and verify it fails** - -Run: - -```bash -mvn -pl :event-file-store -Dtest=DuckDbHotEventFileStoreTest test -``` - -Expected: compilation failure because `DuckDbHotEventFileStore` does not exist. - -- [ ] **Step 3: Create the hot store class** - -Create `DuckDbHotEventFileStore` with this structure: - -```java -public final class DuckDbHotEventFileStore implements EventFileStore { - - private static final TypeReference> STRING_MAP = new TypeReference<>() {}; - - private final Path root; - private final Path dbPath; - private final ZoneId partitionZone; - private final ObjectMapper objectMapper; - private volatile boolean initialized; - - public DuckDbHotEventFileStore(Path root, ZoneId partitionZone) { - this(root, partitionZone, new ObjectMapper()); - } - - public DuckDbHotEventFileStore(Path root, ZoneId partitionZone, ObjectMapper objectMapper) { - if (root == null) { - throw new IllegalArgumentException("root must not be null"); - } - this.root = root.toAbsolutePath(); - this.dbPath = this.root.resolve("events.duckdb"); - this.partitionZone = partitionZone == null ? ZoneId.of("Asia/Shanghai") : partitionZone; - this.objectMapper = objectMapper == null ? new ObjectMapper() : objectMapper; - } - - @Override - public synchronized void appendAll(List records) throws IOException { - if (records == null || records.isEmpty()) { - return; - } - ensureInitialized(); - try (Connection connection = DriverManager.getConnection(jdbcUrl())) { - connection.setAutoCommit(false); - try { - upsertRecords(connection, records); - connection.commit(); - } catch (SQLException | IOException e) { - connection.rollback(); - throw e; - } finally { - connection.setAutoCommit(true); - } - } catch (SQLException e) { - throw new IOException("write duckdb hot event store failed", e); - } - } - - @Override - public List query(EventFileQuery query) throws IOException { - ensureInitialized(); - // Implement in Task 3. - return List.of(); - } - - @Override - public EventFileRecord findByRawArchiveUri(String rawArchiveUri) throws IOException { - ensureInitialized(); - // Implement in Task 3. - return null; - } -} -``` - -- [ ] **Step 4: Add schema initialization** - -Add: - -```java -private void ensureInitialized() throws IOException { - if (initialized) { - return; - } - synchronized (this) { - if (initialized) { - return; - } - Files.createDirectories(root); - try (Connection connection = DriverManager.getConnection(jdbcUrl()); - Statement statement = connection.createStatement()) { - statement.execute(""" - CREATE TABLE IF NOT EXISTS event_records ( - event_id VARCHAR PRIMARY KEY, - protocol VARCHAR NOT NULL, - event_type VARCHAR NOT NULL, - vin VARCHAR NOT NULL, - event_time_ms BIGINT NOT NULL, - ingest_time_ms BIGINT NOT NULL, - partition_date DATE NOT NULL, - raw_archive_uri VARCHAR NOT NULL, - metadata_json VARCHAR NOT NULL, - payload_json VARCHAR NOT NULL - ) - """); - statement.execute(""" - CREATE INDEX IF NOT EXISTS event_records_protocol_date_time_idx - ON event_records(protocol, partition_date, event_time_ms) - """); - statement.execute(""" - CREATE INDEX IF NOT EXISTS event_records_vin_date_time_idx - ON event_records(protocol, vin, partition_date, event_time_ms) - """); - statement.execute(""" - CREATE INDEX IF NOT EXISTS event_records_vin_type_date_time_idx - ON event_records(protocol, vin, event_type, partition_date, event_time_ms) - """); - statement.execute(""" - CREATE INDEX IF NOT EXISTS event_records_raw_archive_uri_idx - ON event_records(raw_archive_uri) - """); - initialized = true; - } catch (SQLException e) { - throw new IOException("initialize duckdb hot event store failed", e); - } - } -} -``` - -- [ ] **Step 5: Add idempotent batch upsert** - -Use DuckDB `INSERT OR REPLACE` inside one transaction: - -```java -private void upsertRecords(Connection connection, List records) - throws SQLException, IOException { - try (PreparedStatement ps = connection.prepareStatement(""" - INSERT OR REPLACE INTO event_records VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) - """)) { - for (EventFileRecord record : records) { - ps.setString(1, record.eventId()); - ps.setString(2, record.protocol().name()); - ps.setString(3, record.eventType()); - ps.setString(4, record.vin()); - ps.setLong(5, record.eventTime().toEpochMilli()); - ps.setLong(6, record.ingestTime().toEpochMilli()); - ps.setString(7, LocalDate.ofInstant(record.eventTime(), partitionZone).toString()); - ps.setString(8, record.rawArchiveUri()); - ps.setString(9, objectMapper.writeValueAsString(record.metadata())); - ps.setString(10, record.payloadJson()); - ps.addBatch(); - } - ps.executeBatch(); - } -} -``` - -- [ ] **Step 6: Run the hot store tests** - -Run: - -```bash -mvn -pl :event-file-store -Dtest=DuckDbHotEventFileStoreTest test -``` - -Expected: query tests still fail until Task 3 implements reads; append initialization should compile. - -## Task 3: Implement Hot Store Queries and Raw URI Lookup - -**Files:** -- Modify: `modules/sinks/event-file-store/src/main/java/com/lingniu/ingest/eventfilestore/DuckDbHotEventFileStore.java` -- Test: `modules/sinks/event-file-store/src/test/java/com/lingniu/ingest/eventfilestore/DuckDbHotEventFileStoreTest.java` - -- [ ] **Step 1: Implement `query(EventFileQuery)`** - -Use prepared statements for every external value: - -```java -@Override -public List query(EventFileQuery query) throws IOException { - ensureInitialized(); - String order = query.order() == EventFileQuery.Order.DESC ? "DESC" : "ASC"; - StringBuilder where = new StringBuilder(""" - WHERE protocol = ? - AND partition_date BETWEEN CAST(? AS DATE) AND CAST(? AS DATE) - """); - if (query.vin() != null) { - where.append(" AND vin = ?\n"); - } - if (query.eventType() != null) { - where.append(" AND event_type = ?\n"); - } - if (query.eventTimeFrom() != null) { - where.append(" AND event_time_ms >= ?\n"); - } - if (query.eventTimeTo() != null) { - where.append(" AND event_time_ms <= ?\n"); - } - String sql = """ - SELECT event_id, protocol, event_type, vin, event_time_ms, ingest_time_ms, - raw_archive_uri, metadata_json, payload_json - FROM event_records - %s - ORDER BY event_time_ms %s, ingest_time_ms %s, event_id %s - LIMIT ? - """.formatted(where, order, order, order); - try (Connection connection = DriverManager.getConnection(jdbcUrl()); - PreparedStatement ps = connection.prepareStatement(sql)) { - bindQuery(ps, query); - try (ResultSet rs = ps.executeQuery()) { - List out = new ArrayList<>(); - while (rs.next()) { - out.add(record(rs)); - } - return out; - } - } catch (SQLException e) { - throw new IOException("query duckdb hot event store failed", e); - } -} -``` - -- [ ] **Step 2: Add query binding helper** - -```java -private static void bindQuery(PreparedStatement ps, EventFileQuery query) throws SQLException { - int index = 1; - ps.setString(index++, query.protocol().name()); - ps.setString(index++, query.dateFrom().toString()); - ps.setString(index++, query.dateTo().toString()); - if (query.vin() != null) { - ps.setString(index++, query.vin()); - } - if (query.eventType() != null) { - ps.setString(index++, query.eventType()); - } - if (query.eventTimeFrom() != null) { - ps.setLong(index++, query.eventTimeFrom().toEpochMilli()); - } - if (query.eventTimeTo() != null) { - ps.setLong(index++, query.eventTimeTo().toEpochMilli()); - } - ps.setInt(index, query.limit()); -} -``` - -- [ ] **Step 3: Implement `findByRawArchiveUri`** - -```java -@Override -public EventFileRecord findByRawArchiveUri(String rawArchiveUri) throws IOException { - if (rawArchiveUri == null || rawArchiveUri.isBlank()) { - return null; - } - ensureInitialized(); - try (Connection connection = DriverManager.getConnection(jdbcUrl()); - PreparedStatement ps = connection.prepareStatement(""" - SELECT event_id, protocol, event_type, vin, event_time_ms, ingest_time_ms, - raw_archive_uri, metadata_json, payload_json - FROM event_records - WHERE raw_archive_uri = ? - ORDER BY event_type = 'RAW_ARCHIVE' DESC, ingest_time_ms DESC, event_id DESC - LIMIT 1 - """)) { - ps.setString(1, rawArchiveUri); - try (ResultSet rs = ps.executeQuery()) { - return rs.next() ? record(rs) : null; - } - } catch (SQLException e) { - throw new IOException("query duckdb hot store by raw archive uri failed", e); - } -} -``` - -- [ ] **Step 4: Add record mapper and helpers** - -```java -private EventFileRecord record(ResultSet rs) throws SQLException, IOException { - return new EventFileRecord( - rs.getString("event_id"), - protocol(rs.getString("protocol")), - rs.getString("event_type"), - rs.getString("vin"), - Instant.ofEpochMilli(rs.getLong("event_time_ms")), - Instant.ofEpochMilli(rs.getLong("ingest_time_ms")), - rs.getString("raw_archive_uri"), - readMetadata(rs.getString("metadata_json")), - rs.getString("payload_json")); -} - -private Map readMetadata(String json) throws IOException { - if (json == null || json.isBlank()) { - return Map.of(); - } - return objectMapper.readValue(json, STRING_MAP); -} - -private static ProtocolId protocol(String value) { - if (value == null || value.isBlank()) { - return ProtocolId.UNKNOWN; - } - try { - return ProtocolId.valueOf(value); - } catch (IllegalArgumentException ex) { - return ProtocolId.UNKNOWN; - } -} - -private String jdbcUrl() { - return "jdbc:duckdb:" + dbPath; -} -``` - -- [ ] **Step 5: Run hot store tests** - -Run: - -```bash -mvn -pl :event-file-store -Dtest=DuckDbHotEventFileStoreTest test -``` - -Expected: all `DuckDbHotEventFileStoreTest` tests pass. - -## Task 4: Preserve Legacy Store Tests and Update Defaults - -**Files:** -- Modify: `modules/sinks/event-file-store/src/test/java/com/lingniu/ingest/eventfilestore/DuckDbParquetEventFileStoreTest.java` -- Modify: `modules/apps/vehicle-history-app/src/main/resources/application.yml` -- Modify: `modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppDefaultsTest.java` - -- [ ] **Step 1: Keep Parquet tests explicitly legacy** - -No behavior change is needed in `DuckDbParquetEventFileStoreTest`; leave it instantiating `DuckDbParquetEventFileStore` directly. Add a class comment: - -```java -/** - * Compatibility coverage for the legacy Parquet sidecar backend. - * Production history uses DuckDbHotEventFileStore through auto-configuration. - */ -class DuckDbParquetEventFileStoreTest { -``` - -- [ ] **Step 2: Set vehicle-history-app storage default** - -Add to `modules/apps/vehicle-history-app/src/main/resources/application.yml`: - -```yaml - event-file-store: - enabled: ${EVENT_FILE_STORE_ENABLED:true} - storage: ${EVENT_FILE_STORE_STORAGE:duckdb-hot} - path: ${EVENT_FILE_STORE_PATH:./target/event-store/} - zone-id: ${EVENT_FILE_STORE_ZONE_ID:Asia/Shanghai} - batch-size: ${EVENT_FILE_STORE_BATCH_SIZE:1000} - flush-interval-millis: ${EVENT_FILE_STORE_FLUSH_INTERVAL_MILLIS:1000} -``` - -Keep existing indentation and only add `storage`; if `batch-size` already exists, update it to `1000`. - -- [ ] **Step 3: Update app default test** - -In `VehicleHistoryAppDefaultsTest`, assert: - -```java -assertThat(context.getEnvironment() - .getProperty("lingniu.ingest.event-file-store.storage")) - .isEqualTo("duckdb-hot"); -``` - -- [ ] **Step 4: Run app default tests** - -Run: - -```bash -mvn -pl :vehicle-history-app -Dtest=VehicleHistoryAppDefaultsTest test -``` - -Expected: default configuration test passes. - -## Task 5: Verify History Ingest Still Archives RAW Bytes - -**Files:** -- Modify: `modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/EventHistoryEnvelopeIngestorTest.java` -- Test: `modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Gb32960DecodedFrameServiceTest.java` - -- [ ] **Step 1: Add an integration-style test using the hot store** - -Add a test that writes a RAW envelope through `EventHistoryEnvelopeIngestor` into `DuckDbHotEventFileStore`, then finds it by URI. - -```java -@Test -void rawArchiveEnvelopeCanBeFoundFromDuckDbHotStoreByUri(@TempDir Path tempDir) throws Exception { - EventFileStore store = new DuckDbHotEventFileStore(tempDir, ZoneId.of("Asia/Shanghai"), OBJECT_MAPPER); - CapturingArchiveStore archive = new CapturingArchiveStore(); - EventHistoryEnvelopeIngestor ingestor = - new EventHistoryEnvelopeIngestor(store, new TelemetryEnvelopeRecordMapper(), archive); - String rawArchiveKey = "2026/06/23/GB32960/VINRAW001/raw-event-hot.bin"; - String rawArchiveUri = "archive://" + rawArchiveKey; - byte[] rawBytes = new byte[]{0x23, 0x23, 0x02, 0x01}; - - VehicleEnvelope envelope = VehicleEnvelope.newBuilder() - .setSchemaVersion("1.0") - .setEventId("raw-event-hot") - .setVin("VINRAW001") - .setSource("GB32960") - .setProtocolVersion("V2016") - .setEventTimeMs(1_782_112_400_000L) - .setIngestTimeMs(1_782_112_401_000L) - .putMetadata(RawArchiveKeys.META_KEY, rawArchiveKey) - .putMetadata(RawArchiveKeys.META_URI, rawArchiveUri) - .setRawArchive(RawArchiveRef.newBuilder() - .setUri(rawArchiveUri) - .setSizeBytes(rawBytes.length) - .setData(ByteString.copyFrom(rawBytes)) - .build()) - .build(); - - EnvelopeIngestResult result = ingestor.tryIngest(envelope.toByteArray()); - - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.STORED); - assertThat(store.findByRawArchiveUri(rawArchiveUri)) - .isNotNull() - .extracting(EventFileRecord::eventId) - .isEqualTo("raw-event-hot"); - assertThat(archive.bytesByKey).containsEntry(rawArchiveKey, rawBytes); -} -``` - -- [ ] **Step 2: Run the event history ingestor test** - -Run: - -```bash -mvn -pl :event-history-service -Dtest=EventHistoryEnvelopeIngestorTest test -``` - -Expected: test passes. - -- [ ] **Step 3: Run decoded frame service tests** - -Run: - -```bash -mvn -pl :event-history-service -Dtest=Gb32960DecodedFrameServiceTest test -``` - -Expected: existing replay/snapshot tests pass. If they use a fake store, no change is needed. - -## Task 6: Full Module Verification - -**Files:** -- No source changes unless failures expose missing imports or config assertions. - -- [ ] **Step 1: Run sink module tests** - -Run: - -```bash -mvn -pl :event-file-store test -``` - -Expected: all event-file-store tests pass, including both hot and legacy stores. - -- [ ] **Step 2: Run event-history-service tests** - -Run: - -```bash -mvn -pl :event-history-service test -``` - -Expected: all event-history-service tests pass. - -- [ ] **Step 3: Package vehicle-history-app** - -Run: - -```bash -mvn -pl :vehicle-history-app -am package -DskipTests -``` - -Expected: package succeeds. - -## Task 7: Local Runtime Verification - -**Files:** -- No committed source changes. - -- [ ] **Step 1: Stop any old history service on port 20200** - -Run: - -```bash -lsof -tiTCP:20200 -sTCP:LISTEN | xargs -r kill -``` - -Expected: no command output, or the old process exits. - -- [ ] **Step 2: Start vehicle-history-app with hot DuckDB store** - -Run: - -```bash -EVENT_FILE_STORE_STORAGE=duckdb-hot \ -EVENT_FILE_STORE_PATH=./target/live-history-event-store \ -SINK_ARCHIVE_PATH=./target/live-history-archive \ -KAFKA_BROKERS=114.55.58.251:9092 \ -KAFKA_CONSUMER_ENABLED=true \ -KAFKA_GROUP_HISTORY=vehicle-history-hot-$(date +%Y%m%d%H%M%S) \ -java --sun-misc-unsafe-memory-access=allow \ - -jar modules/apps/vehicle-history-app/target/vehicle-history-app.jar -``` - -Expected: - -- app starts on `http://127.0.0.1:20200`; -- logs show Kafka consumer subscribed; -- `target/live-history-event-store/events.duckdb` is created. - -- [ ] **Step 3: Verify health** - -Run: - -```bash -curl -sS http://127.0.0.1:20200/actuator/health -``` - -Expected: - -```json -{"status":"UP"} -``` - -- [ ] **Step 4: Verify a recent VIN query** - -Run with a VIN observed in live archive: - -```bash -curl -sS 'http://127.0.0.1:20200/api/event-history/gb32960/telemetry-snapshots?vin=LNXNEGRR9SR318194&platformAccount=Hyundai&dateFrom=2026-06-23&dateTo=2026-06-24&order=DESC&limit=3' -``` - -Expected: - -- response is a JSON array; -- if live traffic for that VIN exists after service start, at least one snapshot appears; -- `rawArchiveUris` point to existing files under `target/live-history-archive`. - -- [ ] **Step 5: Verify raw frame replay** - -Use one `rawArchiveUri` from Step 4: - -```bash -curl -sS 'http://127.0.0.1:20200/api/event-history/gb32960/frame?rawArchiveUri=archive://REPLACE_ME&platformAccount=Hyundai' -``` - -Expected: - -- response contains `vin`, `command`, `eventTime`, and parsed `blocks`; -- no `raw archive is missing` warning for the selected URI. - -## Self-Review Checklist - -- [ ] The new hot store does not rewrite Parquet files on every append. -- [ ] `appendAll` writes one batch in one transaction. -- [ ] Duplicate `event_id` replay is idempotent. -- [ ] Existing history HTTP APIs still use `EventFileStore`, so controller contracts are unchanged. -- [ ] RAW bytes still land in `ArchiveStore`. -- [ ] `findByRawArchiveUri` is backed by DuckDB index. -- [ ] No MySQL credential, RDS host, username, or password appears in the plan or source files. -- [ ] This plan does not claim the full production goal is complete; it only lands the history foundation. diff --git a/docs/superpowers/plans/2026-06-23-gb32960-service-split.md b/docs/superpowers/plans/2026-06-23-gb32960-service-split.md deleted file mode 100644 index eb1316f3..00000000 --- a/docs/superpowers/plans/2026-06-23-gb32960-service-split.md +++ /dev/null @@ -1,1649 +0,0 @@ -> **Superseded:** This 2026-06-23 service-split plan is historical context. -> Use `docs/target-architecture.md` and -> `docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.md` for the -> current production architecture: GB32960, JT808, and Yutong MQTT are the -> active ingest apps, history is backed by TDengine, and Xinda/event-file-store -> paths have been removed from the current build surface. - -# GB32960 Service Split Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Split the current all-in-one GB32960 production path into separate ingest, history, and analytics runnable apps with Kafka as the durability boundary and ACK after Kafka success. - -**Architecture:** Add three Spring Boot app modules while keeping `bootstrap-all` available for development and rollback. First isolate runtime composition, then introduce GB32960-specific Kafka topics, then change the GB32960 ACK path so realtime/report ACKs are written after required Kafka production succeeds. History and analytics become Kafka consumers in separate apps and no longer share a JVM with TCP ingest. - -**Tech Stack:** Java 25, Spring Boot 3.5, Maven multi-module build, Netty, Kafka clients, Protobuf envelope, DuckDB/Parquet event store, local archive sink. - ---- - -## Current-State Notes - -- Existing all-in-one entrypoint: `modules/apps/bootstrap-all/src/main/java/com/lingniu/ingest/bootstrap/IngestApplication.java`. -- Existing all-in-one module at the time of this plan imported protocols, - sinks, history, state, stat, command gateway, MQTT, and JT modules. Xinda was - later removed from source and build configuration. -- Existing GB32960 handler: `modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandler.java`. -- Existing handler currently writes report ACK before `dispatcher.dispatch(rf)`. -- Existing `DisruptorEventBus` publishes to sinks asynchronously and does not wait for sink completion. -- Existing `KafkaEventSink.accepts()` rejects `VehicleEvent.RawArchive`. -- Existing versioned topic names are not yet present; current topic names live under `lingniu.ingest.sink.kafka.topics`. -- Existing history consumer logic: `modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/EventHistoryEnvelopeIngestor.java`. -- Existing analytics consumers: `VehicleStateEnvelopeIngestor` and `VehicleStatEnvelopeIngestor`. -- There is an unrelated working-tree change in `modules/apps/bootstrap-all/src/main/resources/application.yml` for GB32960 diagnostics capacity. Do not revert it. Do not include it in service-split commits unless the user explicitly asks. - -## File Structure - -Create app modules: - -- `modules/apps/gb32960-ingest-app/pom.xml` -- `modules/apps/gb32960-ingest-app/src/main/java/com/lingniu/ingest/gb32960app/Gb32960IngestApplication.java` -- `modules/apps/gb32960-ingest-app/src/main/resources/application.yml` -- `modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppCompositionTest.java` -- `modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppDefaultsTest.java` - -- `modules/apps/vehicle-history-app/pom.xml` -- `modules/apps/vehicle-history-app/src/main/java/com/lingniu/ingest/historyapp/VehicleHistoryApplication.java` -- `modules/apps/vehicle-history-app/src/main/resources/application.yml` -- `modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppCompositionTest.java` -- `modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppDefaultsTest.java` - -- `modules/apps/vehicle-analytics-app/pom.xml` -- `modules/apps/vehicle-analytics-app/src/main/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsApplication.java` -- `modules/apps/vehicle-analytics-app/src/main/resources/application.yml` -- `modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppCompositionTest.java` -- `modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppDefaultsTest.java` - -Modify shared build and Kafka contract files: - -- `pom.xml` -- `modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkProperties.java` -- `modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/TopicRouter.java` -- `modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEventSink.java` -- `modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/TopicRouterTest.java` -- `modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEventSinkTest.java` - -Modify ACK/Kafka durability boundary files: - -- `modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/sink/EventSink.java` -- `modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/Dispatcher.java` -- `modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/DisruptorEventBus.java` -- `modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandler.java` -- `modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandlerAckBoundaryTest.java` - -Optional later follow-up files after the first split is running: - -- `docs/operations/gb32960-service-split-runbook.md` -- Deployment manifests or service manager files used by the user's environment. - ---- - -### Task 1: Register the Three New App Modules - -**Files:** -- Modify: `pom.xml` -- Create: `modules/apps/gb32960-ingest-app/pom.xml` -- Create: `modules/apps/vehicle-history-app/pom.xml` -- Create: `modules/apps/vehicle-analytics-app/pom.xml` - -- [ ] **Step 1: Add failing module-presence test by running Maven before modules exist** - -Run: - -```bash -export JAVA_HOME=/opt/homebrew/opt/openjdk@25/libexec/openjdk.jdk/Contents/Home -export PATH="$JAVA_HOME/bin:/opt/homebrew/bin:$PATH" -mvn -pl :gb32960-ingest-app -am validate -``` - -Expected: FAIL with a message that Maven cannot find project `:gb32960-ingest-app`. - -- [ ] **Step 2: Add modules to the root reactor** - -In `pom.xml`, add these module entries after `modules/apps/command-gateway` and before `modules/apps/bootstrap-all`: - -```xml - modules/apps/gb32960-ingest-app - modules/apps/vehicle-history-app - modules/apps/vehicle-analytics-app -``` - -Also add dependency management entries near the existing `command-gateway` entry: - -```xml - - com.lingniu.ingest - gb32960-ingest-app - ${project.version} - - - com.lingniu.ingest - vehicle-history-app - ${project.version} - - - com.lingniu.ingest - vehicle-analytics-app - ${project.version} - -``` - -- [ ] **Step 3: Create `gb32960-ingest-app` POM** - -Create `modules/apps/gb32960-ingest-app/pom.xml`: - -```xml - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - - gb32960-ingest-app - gb32960-ingest-app - GB32960 protocol ingress runtime: TCP decode, auth, Kafka production, and ACK. - - - - com.lingniu.ingest - ingest-core - - - com.lingniu.ingest - session-core - - - com.lingniu.ingest - observability - - - com.lingniu.ingest - protocol-gb32960 - - - com.lingniu.ingest - sink-kafka - - - org.springframework.boot - spring-boot-starter - - - org.springframework.boot - spring-boot-starter-test - test - - - - - gb32960-ingest-app - - - org.springframework.boot - spring-boot-maven-plugin - - --sun-misc-unsafe-memory-access=allow - - - - - repackage - - - - - - - -``` - -- [ ] **Step 4: Create `vehicle-history-app` POM** - -Create `modules/apps/vehicle-history-app/pom.xml`: - -```xml - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - - vehicle-history-app - vehicle-history-app - Vehicle raw archive, event history, and GB32960 frame query runtime. - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - observability - - - com.lingniu.ingest - sink-kafka - - - com.lingniu.ingest - sink-archive - - - com.lingniu.ingest - event-file-store - - - com.lingniu.ingest - event-history-service - - - com.lingniu.ingest - protocol-gb32960 - - - org.springframework.boot - spring-boot-starter-web - - - org.springframework.boot - spring-boot-starter-test - test - - - - - vehicle-history-app - - - org.springframework.boot - spring-boot-maven-plugin - - --sun-misc-unsafe-memory-access=allow - - - - - repackage - - - - - - - -``` - -- [ ] **Step 5: Create `vehicle-analytics-app` POM** - -Create `modules/apps/vehicle-analytics-app/pom.xml`: - -```xml - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - - vehicle-analytics-app - vehicle-analytics-app - Vehicle state, daily statistics, alarms, and analytics runtime. - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - observability - - - com.lingniu.ingest - sink-kafka - - - com.lingniu.ingest - vehicle-state-service - - - com.lingniu.ingest - vehicle-stat-service - - - org.springframework.boot - spring-boot-starter-web - - - org.springframework.boot - spring-boot-starter-test - test - - - - - vehicle-analytics-app - - - org.springframework.boot - spring-boot-maven-plugin - - --sun-misc-unsafe-memory-access=allow - - - - - repackage - - - - - - - -``` - -- [ ] **Step 6: Verify the modules are visible to Maven** - -Run: - -```bash -export JAVA_HOME=/opt/homebrew/opt/openjdk@25/libexec/openjdk.jdk/Contents/Home -export PATH="$JAVA_HOME/bin:/opt/homebrew/bin:$PATH" -mvn -pl :gb32960-ingest-app,:vehicle-history-app,:vehicle-analytics-app -am validate -``` - -Expected: BUILD SUCCESS. - -- [ ] **Step 7: Commit** - -```bash -git add pom.xml modules/apps/gb32960-ingest-app/pom.xml modules/apps/vehicle-history-app/pom.xml modules/apps/vehicle-analytics-app/pom.xml -git commit -m "build: add split service app modules" -``` - ---- - -### Task 2: Add App Entrypoints and Runtime Defaults - -**Files:** -- Create: `modules/apps/gb32960-ingest-app/src/main/java/com/lingniu/ingest/gb32960app/Gb32960IngestApplication.java` -- Create: `modules/apps/gb32960-ingest-app/src/main/resources/application.yml` -- Create: `modules/apps/vehicle-history-app/src/main/java/com/lingniu/ingest/historyapp/VehicleHistoryApplication.java` -- Create: `modules/apps/vehicle-history-app/src/main/resources/application.yml` -- Create: `modules/apps/vehicle-analytics-app/src/main/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsApplication.java` -- Create: `modules/apps/vehicle-analytics-app/src/main/resources/application.yml` - -- [ ] **Step 1: Create the GB32960 ingest main class** - -Create `modules/apps/gb32960-ingest-app/src/main/java/com/lingniu/ingest/gb32960app/Gb32960IngestApplication.java`: - -```java -package com.lingniu.ingest.gb32960app; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.boot.SpringApplication; -import org.springframework.boot.autoconfigure.SpringBootApplication; - -@SpringBootApplication(scanBasePackages = "com.lingniu.ingest") -public class Gb32960IngestApplication { - - private static final Logger log = LoggerFactory.getLogger(Gb32960IngestApplication.class); - - public static void main(String[] args) { - try { - SpringApplication.run(Gb32960IngestApplication.class, args); - } catch (Throwable t) { - log.error("GB32960 ingest application failed to start, forcing JVM exit", t); - System.exit(1); - } - } -} -``` - -- [ ] **Step 2: Create the GB32960 ingest config** - -Create `modules/apps/gb32960-ingest-app/src/main/resources/application.yml`: - -```yaml -spring: - application: - name: gb32960-ingest-app - threads: - virtual: - enabled: true - -server: - port: ${HTTP_PORT:20100} - -lingniu: - ingest: - gb32960: - enabled: true - port: ${GB32960_PORT:32960} - boss-threads: ${GB32960_BOSS_THREADS:1} - worker-threads: ${GB32960_WORKER_THREADS:0} - auth: - enabled: ${GB32960_AUTH_ENABLED:false} - case-sensitive: false - whitelist: [] - platforms: - - username: ${GB32960_PLATFORM_USER_HYUNDAI:Hyundai} - password: ${GB32960_PLATFORM_PWD_HYUNDAI:f2e3445d7cda409fb4f278f6fb890734} - allowed-ips: - - ${GB32960_PLATFORM_IP_HYUNDAI:8.134.95.166} - description: 外部下级平台 - Hyundai - tls: - enabled: ${GB32960_TLS_ENABLED:false} - cert-path: ${GB32960_TLS_CERT:} - key-path: ${GB32960_TLS_KEY:} - trust-cert-path: ${GB32960_TLS_CA:} - require-client-auth: true - vendor-extensions: - - name: guangdong-fc - match: - platform-accounts: - - Hyundai - vin-prefixes: [] - vins: [] - parse: - lenient-block-failure: true - diagnostics: - max-logged-frame-keys-per-channel: ${GB32960_DIAGNOSTICS_MAX_LOGGED_FRAME_KEYS_PER_CHANNEL:4096} - pipeline: - disruptor: - ring-buffer-size: ${PIPELINE_RING_BUFFER_SIZE:131072} - wait-strategy: ${PIPELINE_WAIT_STRATEGY:yielding} - producer-type: multi - dedup: - enabled: true - cache-size: 200000 - ttl-seconds: 600 - rate-limit: - per-vin-qps: 50 - session: - store: ${SESSION_STORE:redis} - ttl: ${SESSION_TTL:30m} - identity: - store: ${VEHICLE_IDENTITY_STORE:mysql} - mysql: - jdbc-url: ${VEHICLE_IDENTITY_MYSQL_JDBC_URL:jdbc:mysql://127.0.0.1:3306/lingniu_vehicle?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai} - username: ${VEHICLE_IDENTITY_MYSQL_USERNAME:root} - password: ${VEHICLE_IDENTITY_MYSQL_PASSWORD:} - table-name: ${VEHICLE_IDENTITY_MYSQL_TABLE:vehicle_identity_bindings} - initialize-schema: ${VEHICLE_IDENTITY_MYSQL_INITIALIZE_SCHEMA:true} - sink: - kafka: - enabled: ${KAFKA_ENABLED:true} - bootstrap-servers: ${KAFKA_BROKERS:114.55.58.251:9092} - compression-type: zstd - linger-ms: 20 - batch-size: 65536 - acks: all - enable-idempotence: true - node-id: ${KAFKA_NODE_ID:gb32960-ingest-local} - topics: - realtime: ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - location: ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - alarm: ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - session: ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - media-meta: ${KAFKA_TOPIC_MEDIA_META:vehicle.media.meta.v1} - raw-archive: ${KAFKA_TOPIC_GB32960_RAW:vehicle.raw.gb32960.v1} - dlq: ${KAFKA_TOPIC_GB32960_DLQ:vehicle.dlq.gb32960.v1} - consumer: - enabled: false - event-file-store: - enabled: false - event-history: - enabled: false - vehicle-state: - enabled: false - vehicle-stat: - enabled: false - -management: - endpoints: - web: - exposure: - include: health,info,metrics,prometheus,env - health: - redis: - enabled: ${MANAGEMENT_HEALTH_REDIS_ENABLED:false} - metrics: - tags: - application: gb32960-ingest-app - -logging: - level: - root: INFO - com.lingniu.ingest: INFO -``` - -- [ ] **Step 3: Create the history app main class** - -Create `modules/apps/vehicle-history-app/src/main/java/com/lingniu/ingest/historyapp/VehicleHistoryApplication.java`: - -```java -package com.lingniu.ingest.historyapp; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.boot.SpringApplication; -import org.springframework.boot.autoconfigure.SpringBootApplication; - -@SpringBootApplication(scanBasePackages = "com.lingniu.ingest") -public class VehicleHistoryApplication { - - private static final Logger log = LoggerFactory.getLogger(VehicleHistoryApplication.class); - - public static void main(String[] args) { - try { - SpringApplication.run(VehicleHistoryApplication.class, args); - } catch (Throwable t) { - log.error("Vehicle history application failed to start, forcing JVM exit", t); - System.exit(1); - } - } -} -``` - -- [ ] **Step 4: Create the history app config** - -Create `modules/apps/vehicle-history-app/src/main/resources/application.yml`: - -```yaml -spring: - application: - name: vehicle-history-app - threads: - virtual: - enabled: true - -server: - port: ${HTTP_PORT:20200} - -lingniu: - ingest: - gb32960: - enabled: false - sink: - kafka: - enabled: ${KAFKA_ENABLED:true} - bootstrap-servers: ${KAFKA_BROKERS:114.55.58.251:9092} - topics: - realtime: ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - raw-archive: ${KAFKA_TOPIC_GB32960_RAW:vehicle.raw.gb32960.v1} - dlq: ${KAFKA_TOPIC_GB32960_DLQ:vehicle.dlq.gb32960.v1} - consumer: - enabled: ${KAFKA_CONSUMER_ENABLED:true} - client-id-prefix: ${KAFKA_CONSUMER_CLIENT_ID_PREFIX:vehicle-history} - auto-offset-reset: ${KAFKA_CONSUMER_AUTO_OFFSET_RESET:earliest} - max-poll-records: ${KAFKA_CONSUMER_MAX_POLL_RECORDS:500} - bindings: - eventHistoryEnvelopeConsumerProcessor: - enabled: true - group-id: ${KAFKA_GROUP_HISTORY:vehicle-history} - topics: - - ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - - ${KAFKA_TOPIC_GB32960_RAW:vehicle.raw.gb32960.v1} - archive: - enabled: ${SINK_ARCHIVE_ENABLED:true} - type: local - path: ${SINK_ARCHIVE_PATH:./archive/} - event-file-store: - enabled: ${EVENT_FILE_STORE_ENABLED:true} - path: ${EVENT_FILE_STORE_PATH:./target/event-store/} - zone-id: ${EVENT_FILE_STORE_ZONE_ID:Asia/Shanghai} - batch-size: ${EVENT_FILE_STORE_BATCH_SIZE:500} - flush-interval-millis: ${EVENT_FILE_STORE_FLUSH_INTERVAL_MILLIS:1000} - event-history: - enabled: true - vehicle-state: - enabled: false - vehicle-stat: - enabled: false - -management: - endpoints: - web: - exposure: - include: health,info,metrics,prometheus,env - health: - redis: - enabled: ${MANAGEMENT_HEALTH_REDIS_ENABLED:false} - metrics: - tags: - application: vehicle-history-app -``` - -- [ ] **Step 5: Create the analytics app main class** - -Create `modules/apps/vehicle-analytics-app/src/main/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsApplication.java`: - -```java -package com.lingniu.ingest.analyticsapp; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.boot.SpringApplication; -import org.springframework.boot.autoconfigure.SpringBootApplication; - -@SpringBootApplication(scanBasePackages = "com.lingniu.ingest") -public class VehicleAnalyticsApplication { - - private static final Logger log = LoggerFactory.getLogger(VehicleAnalyticsApplication.class); - - public static void main(String[] args) { - try { - SpringApplication.run(VehicleAnalyticsApplication.class, args); - } catch (Throwable t) { - log.error("Vehicle analytics application failed to start, forcing JVM exit", t); - System.exit(1); - } - } -} -``` - -- [ ] **Step 6: Create the analytics app config** - -Create `modules/apps/vehicle-analytics-app/src/main/resources/application.yml`: - -```yaml -spring: - application: - name: vehicle-analytics-app - threads: - virtual: - enabled: true - -server: - port: ${HTTP_PORT:20300} - -lingniu: - ingest: - gb32960: - enabled: false - sink: - kafka: - enabled: ${KAFKA_ENABLED:true} - bootstrap-servers: ${KAFKA_BROKERS:114.55.58.251:9092} - topics: - realtime: ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - dlq: ${KAFKA_TOPIC_GB32960_DLQ:vehicle.dlq.gb32960.v1} - consumer: - enabled: ${KAFKA_CONSUMER_ENABLED:true} - client-id-prefix: ${KAFKA_CONSUMER_CLIENT_ID_PREFIX:vehicle-analytics} - auto-offset-reset: ${KAFKA_CONSUMER_AUTO_OFFSET_RESET:earliest} - max-poll-records: ${KAFKA_CONSUMER_MAX_POLL_RECORDS:500} - bindings: - vehicleStateEnvelopeConsumerProcessor: - enabled: ${VEHICLE_STATE_ENABLED:false} - group-id: ${KAFKA_GROUP_STATE:vehicle-state} - topics: - - ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - vehicleStatEnvelopeConsumerProcessor: - enabled: ${VEHICLE_STAT_ENABLED:true} - group-id: ${KAFKA_GROUP_STAT:vehicle-stat} - topics: - - ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - archive: - enabled: false - event-file-store: - enabled: false - event-history: - enabled: false - vehicle-state: - enabled: ${VEHICLE_STATE_ENABLED:false} - vehicle-stat: - enabled: ${VEHICLE_STAT_ENABLED:true} - file-path: ${VEHICLE_STAT_FILE_PATH:./target/vehicle-stat/} - zone-id: ${VEHICLE_STAT_ZONE_ID:Asia/Shanghai} - -management: - endpoints: - web: - exposure: - include: health,info,metrics,prometheus,env - health: - redis: - enabled: ${MANAGEMENT_HEALTH_REDIS_ENABLED:false} - metrics: - tags: - application: vehicle-analytics-app -``` - -- [ ] **Step 7: Build all three app jars** - -Run: - -```bash -export JAVA_HOME=/opt/homebrew/opt/openjdk@25/libexec/openjdk.jdk/Contents/Home -export PATH="$JAVA_HOME/bin:/opt/homebrew/bin:$PATH" -mvn -pl :gb32960-ingest-app,:vehicle-history-app,:vehicle-analytics-app -am package -Dmaven.test.skip=true -``` - -Expected: BUILD SUCCESS and jars under each app module's `target/`. - -- [ ] **Step 8: Commit** - -```bash -git add modules/apps/gb32960-ingest-app modules/apps/vehicle-history-app modules/apps/vehicle-analytics-app -git commit -m "feat: add split service entrypoints" -``` - ---- - -### Task 3: Add Composition Tests for Service Boundaries - -**Files:** -- Create: `modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppCompositionTest.java` -- Create: `modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppCompositionTest.java` -- Create: `modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppCompositionTest.java` - -- [ ] **Step 1: Write GB32960 ingest composition test** - -Create `modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppCompositionTest.java`: - -```java -package com.lingniu.ingest.gb32960app; - -import com.lingniu.ingest.eventfilestore.EventFileStore; -import com.lingniu.ingest.protocol.gb32960.config.Gb32960AutoConfiguration; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960NettyServer; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import com.lingniu.ingest.sink.kafka.KafkaEventSink; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; - -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960IngestAppCompositionTest { - - @Test - void gb32960IngestRuntimeStartsProtocolAndKafkaWithoutHistoryStorage() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(Gb32960AutoConfiguration.class)) - .withPropertyValues( - "lingniu.ingest.gb32960.enabled=true", - "lingniu.ingest.gb32960.port=0", - "lingniu.ingest.sink.kafka.enabled=false", - "lingniu.ingest.sink.archive.enabled=false", - "lingniu.ingest.event-file-store.enabled=false", - "lingniu.ingest.event-history.enabled=false", - "lingniu.ingest.vehicle-state.enabled=false", - "lingniu.ingest.vehicle-stat.enabled=false") - .run(context -> { - assertThat(context).hasSingleBean(Gb32960NettyServer.class); - assertThat(context).doesNotHaveBean(ArchiveStore.class); - assertThat(context).doesNotHaveBean(EventFileStore.class); - assertThat(context).doesNotHaveBean(KafkaEventSink.class); - }); - } -} -``` - -If this test does not create required supporting beans, add the same auto-config classes used by existing bootstrap tests: `IngestCoreAutoConfiguration`, `SessionCoreAutoConfiguration`, and `VehicleIdentityAutoConfiguration`. - -- [ ] **Step 2: Run GB32960 ingest composition test** - -Run: - -```bash -mvn -pl :gb32960-ingest-app -Dtest=Gb32960IngestAppCompositionTest test -``` - -Expected after adding required auto-config classes: PASS. - -- [ ] **Step 3: Write history composition test** - -Create `modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppCompositionTest.java`: - -```java -package com.lingniu.ingest.historyapp; - -import com.lingniu.ingest.eventfilestore.EventFileStore; -import com.lingniu.ingest.eventhistory.EventHistoryController; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960NettyServer; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; - -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; - -class VehicleHistoryAppCompositionTest { - - @TempDir - Path tempDir; - - @Test - void historyRuntimeCreatesStorageAndQueryBeansWithoutProtocolListener() { - new ApplicationContextRunner() - .withPropertyValues( - "lingniu.ingest.gb32960.enabled=false", - "lingniu.ingest.sink.archive.enabled=true", - "lingniu.ingest.sink.archive.path=" + tempDir.resolve("archive"), - "lingniu.ingest.event-file-store.enabled=true", - "lingniu.ingest.event-file-store.path=" + tempDir.resolve("event-store"), - "lingniu.ingest.event-history.enabled=true", - "lingniu.ingest.vehicle-state.enabled=false", - "lingniu.ingest.vehicle-stat.enabled=false", - "lingniu.ingest.sink.kafka.enabled=false") - .withConfiguration(AutoConfigurations.of( - com.lingniu.ingest.sink.archive.config.SinkArchiveAutoConfiguration.class, - com.lingniu.ingest.eventfilestore.config.EventFileStoreAutoConfiguration.class, - com.lingniu.ingest.eventhistory.config.EventHistoryAutoConfiguration.class)) - .run(context -> { - assertThat(context).hasSingleBean(ArchiveStore.class); - assertThat(context).hasSingleBean(EventFileStore.class); - assertThat(context).hasSingleBean(EventHistoryController.class); - assertThat(context).doesNotHaveBean(Gb32960NettyServer.class); - }); - } -} -``` - -- [ ] **Step 4: Run history composition test** - -Run: - -```bash -mvn -pl :vehicle-history-app -Dtest=VehicleHistoryAppCompositionTest test -``` - -Expected: PASS. - -- [ ] **Step 5: Write analytics composition test** - -Create `modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppCompositionTest.java`: - -```java -package com.lingniu.ingest.analyticsapp; - -import com.lingniu.ingest.eventfilestore.EventFileStore; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960NettyServer; -import com.lingniu.ingest.vehiclestat.DailyVehicleStatService; -import com.lingniu.ingest.vehiclestate.VehicleStateRepository; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; - -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; - -class VehicleAnalyticsAppCompositionTest { - - @TempDir - Path tempDir; - - @Test - void analyticsRuntimeCreatesStatisticsWithoutProtocolOrHistoryStorage() { - new ApplicationContextRunner() - .withPropertyValues( - "lingniu.ingest.gb32960.enabled=false", - "lingniu.ingest.sink.archive.enabled=false", - "lingniu.ingest.event-file-store.enabled=false", - "lingniu.ingest.event-history.enabled=false", - "lingniu.ingest.vehicle-state.enabled=false", - "lingniu.ingest.vehicle-stat.enabled=true", - "lingniu.ingest.vehicle-stat.file-path=" + tempDir.resolve("vehicle-stat"), - "lingniu.ingest.sink.kafka.enabled=false") - .withConfiguration(AutoConfigurations.of( - com.lingniu.ingest.vehiclestat.config.VehicleStatAutoConfiguration.class, - com.lingniu.ingest.vehiclestate.config.VehicleStateAutoConfiguration.class)) - .run(context -> { - assertThat(context).hasSingleBean(DailyVehicleStatService.class); - assertThat(context).doesNotHaveBean(VehicleStateRepository.class); - assertThat(context).doesNotHaveBean(Gb32960NettyServer.class); - assertThat(context).doesNotHaveBean(EventFileStore.class); - }); - } -} -``` - -- [ ] **Step 6: Run analytics composition test** - -Run: - -```bash -mvn -pl :vehicle-analytics-app -Dtest=VehicleAnalyticsAppCompositionTest test -``` - -Expected: PASS. - -- [ ] **Step 7: Commit** - -```bash -git add modules/apps/gb32960-ingest-app/src/test modules/apps/vehicle-history-app/src/test modules/apps/vehicle-analytics-app/src/test -git commit -m "test: cover split service composition" -``` - ---- - -### Task 4: Version Kafka Topics for GB32960 Raw/Event/DLQ - -**Files:** -- Modify: `modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkProperties.java` -- Modify: `modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/TopicRouter.java` -- Modify: `modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEventSink.java` -- Create or modify: `modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/TopicRouterTest.java` -- Create or modify: `modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEventSinkTest.java` - -- [ ] **Step 1: Write topic routing tests** - -Create `modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/TopicRouterTest.java` if it does not exist. Add tests proving GB32960 raw and normalized events can route to versioned topics. Use the existing `VehicleEvent` constructors from `EventFileStoreSinkTest` and `EnvelopeMapperTelemetrySnapshotTest` as examples. - -Expected test names: - -```java -@Test -void rawArchiveRoutesToVersionedGb32960RawTopic() { - KafkaSinkProperties.Topics topics = new KafkaSinkProperties.Topics(); - topics.setRawArchive("vehicle.raw.gb32960.v1"); - TopicRouter router = new TopicRouter(topics); - - VehicleEvent.RawArchive raw = new VehicleEvent.RawArchive( - "event-1", - "trace-1", - ProtocolId.GB32960, - "VIN001", - Instant.parse("2026-06-23T07:00:00Z"), - 0x02, - new byte[] {0x23, 0x23}, - Map.of("platformAccount", "Hyundai")); - - assertThat(router.route(raw)).isEqualTo("vehicle.raw.gb32960.v1"); -} -``` - -Also add a normalized realtime event test: - -```java -@Test -void realtimeRoutesToVersionedGb32960EventTopic() { - KafkaSinkProperties.Topics topics = new KafkaSinkProperties.Topics(); - topics.setRealtime("vehicle.event.gb32960.v1"); - TopicRouter router = new TopicRouter(topics); - - VehicleEvent.Realtime realtime = new VehicleEvent.Realtime( - "event-2", - "trace-2", - ProtocolId.GB32960, - "VIN001", - Instant.parse("2026-06-23T07:00:01Z"), - 0x02, - Map.of("soc", "80"), - Map.of("platformAccount", "Hyundai")); - - assertThat(router.route(realtime)).isEqualTo("vehicle.event.gb32960.v1"); -} -``` - -Adjust constructor arguments to match the current sealed `VehicleEvent` definitions in `modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/VehicleEvent.java`. - -- [ ] **Step 2: Run topic router test and verify failure if topic defaults are still old** - -Run: - -```bash -mvn -pl :sink-kafka -Dtest=TopicRouterTest test -``` - -Expected before implementation: compile failure if constructors are wrong or assertion failure if defaults are old. Fix constructors first; keep routing assertions. - -- [ ] **Step 3: Update `KafkaSinkProperties.Topics` defaults** - -In `KafkaSinkProperties.Topics`, change defaults: - -```java -private String realtime = "vehicle.event.gb32960.v1"; -private String location = "vehicle.event.gb32960.v1"; -private String alarm = "vehicle.event.gb32960.v1"; -private String session = "vehicle.event.gb32960.v1"; -private String mediaMeta = "vehicle.media.meta.v1"; -private String rawArchive = "vehicle.raw.gb32960.v1"; -private String dlq = "vehicle.dlq.gb32960.v1"; -``` - -- [ ] **Step 4: Allow raw archive records in KafkaEventSink** - -Change `KafkaEventSink.accepts()` to: - -```java -@Override -public boolean accepts(VehicleEvent event) { - return true; -} -``` - -Update the JavaDoc above it to say Kafka carries both normalized events and raw archive records in split-service mode. - -- [ ] **Step 5: Run sink-kafka tests** - -Run: - -```bash -mvn -pl :sink-kafka test -``` - -Expected: PASS. - -- [ ] **Step 6: Commit** - -```bash -git add modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkProperties.java \ - modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/TopicRouter.java \ - modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEventSink.java \ - modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/TopicRouterTest.java \ - modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEventSinkTest.java -git commit -m "feat: route gb32960 kafka topics" -``` - ---- - -### Task 5: Introduce an ACK-Aware GB32960 Dispatch Boundary - -**Files:** -- Modify: `modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/Dispatcher.java` -- Modify: `modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/DisruptorEventBus.java` -- Modify: `modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandler.java` -- Create: `modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandlerAckBoundaryTest.java` - -Design choice for first implementation: - -- Keep `Dispatcher.dispatch(RawFrame)` for existing async protocols and `bootstrap-all`. -- Add `Dispatcher.dispatchAndAwait(RawFrame, Duration)` returning `CompletableFuture` or throwing a checked runtime exception on timeout/failure. -- Implement `DisruptorEventBus.publishAndAwait(VehicleEvent, Duration)` by synchronously calling accepted sinks' `publish()` futures and waiting for all to complete. -- Use the synchronous path only from GB32960 report/heartbeat/session commands that require ACK after Kafka success. - -- [ ] **Step 1: Write failing ACK boundary test** - -Create `Gb32960ChannelHandlerAckBoundaryTest` using `EmbeddedChannel` and a fake dispatcher that records whether ACK happens before the future completes. - -Core assertion: - -```java -@Test -void realtimeReportAckWaitsForKafkaDurabilityBoundary() { - CompletableFuture durability = new CompletableFuture<>(); - RecordingAckService ackService = new RecordingAckService(); - AwaitingDispatcher dispatcher = new AwaitingDispatcher(durability); - Gb32960ChannelHandler handler = handlerWith(dispatcher, ackService); - - EmbeddedChannel channel = new EmbeddedChannel(handler); - channel.writeInbound(validRealtimeFrameBytes()); - - assertThat(ackService.writtenTags()).isEmpty(); - - durability.complete(null); - channel.runPendingTasks(); - - assertThat(ackService.writtenTags()).contains("report-ack-0x2"); -} -``` - -Implementation notes: - -- Reuse valid frame fixture builders from existing GB32960 codec tests. -- If `Gb32960AckService` is hard to spy because methods are concrete, subclass it in test and override `writeResponse`/`writeAndClose` to record tags instead of writing bytes. -- If `Dispatcher` is not subclassable, extract a small interface such as `FrameDispatcher` in `ingest-core` and adapt `Dispatcher` to implement it. - -- [ ] **Step 2: Run the failing test** - -Run: - -```bash -mvn -pl :protocol-gb32960 -Dtest=Gb32960ChannelHandlerAckBoundaryTest test -``` - -Expected before implementation: FAIL because current handler writes ACK before dispatch completion. - -- [ ] **Step 3: Add await API to event bus** - -In `DisruptorEventBus`, add: - -```java -public CompletableFuture publishAndAwait(VehicleEvent event) { - List> futures = sinks.stream() - .filter(sink -> sink.accepts(event)) - .map(sink -> sink.publish(event)) - .toList(); - if (futures.isEmpty()) { - return CompletableFuture.completedFuture(null); - } - return CompletableFuture.allOf(futures.toArray(CompletableFuture[]::new)); -} -``` - -This requires promoting `sinks` from constructor-local variable to a field: - -```java -private final List sinks; -``` - -Set it in the constructor: - -```java -this.sinks = List.copyOf(sinks); -``` - -- [ ] **Step 4: Add await API to Dispatcher** - -In `Dispatcher`, add: - -```java -public CompletableFuture dispatchAndAwait(RawFrame frame) { - List events = mapFrameToEvents(frame); - CompletableFuture[] futures = events.stream() - .map(eventBus::publishAndAwait) - .toArray(CompletableFuture[]::new); - return CompletableFuture.allOf(futures); -} -``` - -Refactor existing `dispatch(RawFrame)` to reuse `mapFrameToEvents(frame)`: - -```java -public void dispatch(RawFrame frame) { - for (VehicleEvent e : mapFrameToEvents(frame)) { - eventBus.publish(e); - } -} -``` - -If `Dispatcher` currently interleaves raw archive event publication and handler mapping, move that logic into a private method: - -```java -private List mapFrameToEvents(RawFrame frame) { - List events = new ArrayList<>(); - events.add(toRawArchive(frame)); - events.addAll(handlerEvents(frame)); - return events; -} -``` - -Use exact existing helper names from `Dispatcher.java`; do not duplicate mapping logic. - -- [ ] **Step 5: Move GB32960 ACK after dispatch completion** - -In `Gb32960ChannelHandler.channelRead0`, for commands that currently ACK before `dispatcher.dispatch(rf)`, build `RawFrame` first, call `dispatcher.dispatchAndAwait(rf)`, and write ACK in the completion callback: - -```java -dispatcher.dispatchAndAwait(rf).whenComplete((ignored, ex) -> { - if (ex != null) { - log.warn("[gb32960] dispatch failed before ack peer={} vin={} cmd=0x{}", - addr(ctx), vin, Integer.toHexString(cmd.code()), ex); - ctx.close(); - return; - } - writeAckForCommand(ctx, msg, rawVin); -}); -``` - -Extract current ACK logic into command-specific method: - -```java -private void writeAckForCommand(ChannelHandlerContext ctx, Gb32960Message msg, byte[] rawVin) { - switch (msg.header().command()) { - case VEHICLE_LOGIN -> writeVehicleLoginAck(ctx, msg, rawVin); - case VEHICLE_LOGOUT -> writeVehicleLogoutAck(ctx, msg, rawVin); - case PLATFORM_LOGIN -> writePlatformLoginAck(ctx, msg, rawVin); - case PLATFORM_LOGOUT -> writePlatformLogoutAck(ctx, msg, rawVin); - case REALTIME_REPORT, RESEND_REPORT, HEARTBEAT -> writeReportOrHeartbeatAck(ctx, msg, rawVin); - case TIME_CALIBRATION -> writeTimeCalibrationAck(ctx, msg, rawVin); - default -> { } - } -} -``` - -Keep authorization failure ACKs immediate because they are not accepted frames and do not cross the Kafka durability boundary. - -- [ ] **Step 6: Run ACK boundary test** - -Run: - -```bash -mvn -pl :protocol-gb32960 -Dtest=Gb32960ChannelHandlerAckBoundaryTest test -``` - -Expected: PASS. - -- [ ] **Step 7: Run core dispatcher tests** - -Run: - -```bash -mvn -pl :ingest-core test -``` - -Expected: PASS. - -- [ ] **Step 8: Commit** - -```bash -git add modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/Dispatcher.java \ - modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/DisruptorEventBus.java \ - modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandler.java \ - modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandlerAckBoundaryTest.java -git commit -m "feat: ack gb32960 after kafka boundary" -``` - ---- - -### Task 6: Wire History and Analytics Consumer Bindings - -**Files:** -- Modify: `modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/EventHistoryEnvelopeIngestor.java` -- Modify: `modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleStatEnvelopeIngestor.java` -- Modify: `modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateEnvelopeIngestor.java` -- Modify tests under the same service modules. - -- [ ] **Step 1: Verify event-history accepts raw and normalized envelopes** - -Run: - -```bash -rg -n "RawArchive|raw_archive|TelemetrySnapshot|Envelope" modules/services/event-history-service/src/main/java modules/services/event-history-service/src/test/java -``` - -Expected: identify whether `EventHistoryEnvelopeIngestor` already persists raw archive envelopes. If raw archive envelopes are ignored, continue with Step 2. - -- [ ] **Step 2: Add event-history raw archive ingestion test** - -Add a test to the existing event-history test suite that creates an envelope with `raw_archive` populated and asserts an `EventFileRecord` with `eventType = RAW_ARCHIVE` is written with `rawArchiveUri`. - -Expected assertion shape: - -```java -assertThat(record.eventType()).isEqualTo("RAW_ARCHIVE"); -assertThat(record.rawArchiveUri()).isEqualTo("archive://2026/06/23/GB32960/VIN001/event-1.bin"); -``` - -- [ ] **Step 3: Implement raw archive ingestion if missing** - -In `EventHistoryEnvelopeIngestor`, map raw archive envelopes to `EventFileRecord` using the same conventions as `EventFileStoreSink`. - -Implementation rule: - -- Do not embed raw bytes into Parquet history records. -- Store `rawArchiveUri`, event id, protocol, VIN, command, event time, and metadata. - -- [ ] **Step 4: Verify analytics consumers ignore raw topic records** - -Add or update tests for `VehicleStatEnvelopeIngestor` and `VehicleStateEnvelopeIngestor`: - -```java -@Test -void ignoresRawArchiveEnvelopeForAnalytics() { - ConsumerProcessResult result = ingestor.ingest(rawArchiveEnvelopeRecord()); - assertThat(result.success()).isTrue(); - assertThat(repositoryWrites()).isEmpty(); -} -``` - -Use the actual result type currently returned by the ingestor. - -- [ ] **Step 5: Run service consumer tests** - -Run: - -```bash -mvn -pl :event-history-service,:vehicle-state-service,:vehicle-stat-service test -``` - -Expected: PASS. - -- [ ] **Step 6: Commit** - -```bash -git add modules/services/event-history-service modules/services/vehicle-state-service modules/services/vehicle-stat-service -git commit -m "feat: split history and analytics consumers" -``` - ---- - -### Task 7: Add App Default Tests - -**Files:** -- Create: `modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppDefaultsTest.java` -- Create: `modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppDefaultsTest.java` -- Create: `modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppDefaultsTest.java` - -- [ ] **Step 1: Add YAML property loader helper to each test** - -Use this helper in each defaults test: - -```java -private static Properties loadApplicationProperties() { - YamlPropertiesFactoryBean yaml = new YamlPropertiesFactoryBean(); - yaml.setResources(new ClassPathResource("application.yml")); - Properties props = yaml.getObject(); - assertThat(props).isNotNull(); - return props; -} -``` - -Imports: - -```java -import org.springframework.beans.factory.config.YamlPropertiesFactoryBean; -import org.springframework.core.io.ClassPathResource; -import java.util.Properties; -import static org.assertj.core.api.Assertions.assertThat; -``` - -- [ ] **Step 2: Assert GB32960 ingest defaults** - -Create `Gb32960IngestAppDefaultsTest`: - -```java -@Test -void gb32960IngestDefaultsOnlyEnableProtocolAndKafkaProducer() { - Properties props = loadApplicationProperties(); - - assertThat(props.getProperty("spring.application.name")).isEqualTo("gb32960-ingest-app"); - assertThat(props.getProperty("lingniu.ingest.gb32960.enabled")).isEqualTo("true"); - assertThat(props.getProperty("lingniu.ingest.gb32960.port")).isEqualTo("${GB32960_PORT:32960}"); - assertThat(props.getProperty("lingniu.ingest.sink.kafka.enabled")).isEqualTo("${KAFKA_ENABLED:true}"); - assertThat(props.getProperty("lingniu.ingest.sink.kafka.consumer.enabled")).isEqualTo("false"); - assertThat(props.getProperty("lingniu.ingest.event-file-store.enabled")).isEqualTo("false"); - assertThat(props.getProperty("lingniu.ingest.event-history.enabled")).isEqualTo("false"); - assertThat(props.getProperty("lingniu.ingest.vehicle-state.enabled")).isEqualTo("false"); - assertThat(props.getProperty("lingniu.ingest.vehicle-stat.enabled")).isEqualTo("false"); -} -``` - -- [ ] **Step 3: Assert history defaults** - -Create `VehicleHistoryAppDefaultsTest`: - -```java -@Test -void historyDefaultsEnableStorageAndHistoryConsumerOnly() { - Properties props = loadApplicationProperties(); - - assertThat(props.getProperty("spring.application.name")).isEqualTo("vehicle-history-app"); - assertThat(props.getProperty("lingniu.ingest.gb32960.enabled")).isEqualTo("false"); - assertThat(props.getProperty("lingniu.ingest.sink.archive.enabled")).isEqualTo("${SINK_ARCHIVE_ENABLED:true}"); - assertThat(props.getProperty("lingniu.ingest.event-file-store.enabled")).isEqualTo("${EVENT_FILE_STORE_ENABLED:true}"); - assertThat(props.getProperty("lingniu.ingest.event-history.enabled")).isEqualTo("true"); - assertThat(props.getProperty("lingniu.ingest.vehicle-state.enabled")).isEqualTo("false"); - assertThat(props.getProperty("lingniu.ingest.vehicle-stat.enabled")).isEqualTo("false"); - assertThat(props.getProperty("lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryEnvelopeConsumerProcessor.group-id")) - .isEqualTo("${KAFKA_GROUP_HISTORY:vehicle-history}"); -} -``` - -- [ ] **Step 4: Assert analytics defaults** - -Create `VehicleAnalyticsAppDefaultsTest`: - -```java -@Test -void analyticsDefaultsEnableStatConsumerAndDisableProtocolAndHistoryStorage() { - Properties props = loadApplicationProperties(); - - assertThat(props.getProperty("spring.application.name")).isEqualTo("vehicle-analytics-app"); - assertThat(props.getProperty("lingniu.ingest.gb32960.enabled")).isEqualTo("false"); - assertThat(props.getProperty("lingniu.ingest.sink.archive.enabled")).isEqualTo("false"); - assertThat(props.getProperty("lingniu.ingest.event-file-store.enabled")).isEqualTo("false"); - assertThat(props.getProperty("lingniu.ingest.event-history.enabled")).isEqualTo("false"); - assertThat(props.getProperty("lingniu.ingest.vehicle-stat.enabled")).isEqualTo("${VEHICLE_STAT_ENABLED:true}"); - assertThat(props.getProperty("lingniu.ingest.sink.kafka.consumer.bindings.vehicleStatEnvelopeConsumerProcessor.group-id")) - .isEqualTo("${KAFKA_GROUP_STAT:vehicle-stat}"); -} -``` - -- [ ] **Step 5: Run app default tests** - -Run: - -```bash -mvn -pl :gb32960-ingest-app,:vehicle-history-app,:vehicle-analytics-app test -``` - -Expected: PASS. - -- [ ] **Step 6: Commit** - -```bash -git add modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppDefaultsTest.java \ - modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppDefaultsTest.java \ - modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppDefaultsTest.java -git commit -m "test: assert split app defaults" -``` - ---- - -### Task 8: End-to-End Verification With Local Kafka - -**Files:** -- Create: `docs/operations/gb32960-service-split-runbook.md` - -- [ ] **Step 1: Build all modules** - -Run: - -```bash -export JAVA_HOME=/opt/homebrew/opt/openjdk@25/libexec/openjdk.jdk/Contents/Home -export PATH="$JAVA_HOME/bin:/opt/homebrew/bin:$PATH" -mvn -pl :gb32960-ingest-app,:vehicle-history-app,:vehicle-analytics-app -am package -Dmaven.test.skip=true -``` - -Expected: BUILD SUCCESS. - -- [ ] **Step 2: Start Kafka test environment** - -Use the repository's existing Kafka setup if present. If there is no repository-local Kafka script, use a local Kafka compatible with `KAFKA_BROKERS=127.0.0.1:9092`. - -Create topics: - -```bash -kafka-topics --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic vehicle.raw.gb32960.v1 --partitions 12 --replication-factor 1 -kafka-topics --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic vehicle.event.gb32960.v1 --partitions 12 --replication-factor 1 -kafka-topics --bootstrap-server 127.0.0.1:9092 --create --if-not-exists --topic vehicle.dlq.gb32960.v1 --partitions 3 --replication-factor 1 -``` - -- [ ] **Step 3: Start split services on local ports** - -Run each command in a separate terminal: - -```bash -KAFKA_BROKERS=127.0.0.1:9092 \ -GB32960_PORT=32960 \ -HTTP_PORT=20100 \ -java --sun-misc-unsafe-memory-access=allow \ - -jar modules/apps/gb32960-ingest-app/target/gb32960-ingest-app.jar -``` - -```bash -KAFKA_BROKERS=127.0.0.1:9092 \ -HTTP_PORT=20200 \ -EVENT_FILE_STORE_PATH=./target/split-event-store \ -SINK_ARCHIVE_PATH=./target/split-archive \ -java --sun-misc-unsafe-memory-access=allow \ - -jar modules/apps/vehicle-history-app/target/vehicle-history-app.jar -``` - -```bash -KAFKA_BROKERS=127.0.0.1:9092 \ -HTTP_PORT=20300 \ -VEHICLE_STAT_FILE_PATH=./target/split-vehicle-stat \ -java --sun-misc-unsafe-memory-access=allow \ - -jar modules/apps/vehicle-analytics-app/target/vehicle-analytics-app.jar -``` - -- [ ] **Step 4: Verify health endpoints** - -Run: - -```bash -curl -sS http://127.0.0.1:20100/actuator/health -curl -sS http://127.0.0.1:20200/actuator/health -curl -sS http://127.0.0.1:20300/actuator/health -``` - -Expected: each returns `{"status":"UP"}`. - -- [ ] **Step 5: Send a known-valid GB32960 frame** - -Use an existing GB32960 test fixture from `modules/protocols/protocol-gb32960/src/test/java`. If no command-line sender exists, add a small test-only utility or use `nc` with a hex-decoded fixture: - -```bash -xxd -r -p /tmp/gb32960-valid-realtime.hex | nc 127.0.0.1 32960 -``` - -Expected: - -- Kafka topic `vehicle.raw.gb32960.v1` receives one raw record. -- Kafka topic `vehicle.event.gb32960.v1` receives one or more normalized records. -- Client receives a GB32960 success ACK. -- `target/split-archive` contains a `.bin` file. -- `target/split-event-store` contains `events.duckdb` or Parquet files after flush. -- `target/split-vehicle-stat` contains updated stat files after analytics consumes the event. - -- [ ] **Step 6: Write runbook** - -Create `docs/operations/gb32960-service-split-runbook.md` with: - -```markdown -# GB32960 Split Service Runbook - -## Services - -- `gb32960-ingest-app`: TCP 32960, HTTP 20100 -- `vehicle-history-app`: HTTP 20200 -- `vehicle-analytics-app`: HTTP 20300 - -## Kafka Topics - -- `vehicle.raw.gb32960.v1` -- `vehicle.event.gb32960.v1` -- `vehicle.dlq.gb32960.v1` - -## ACK Semantics - -GB32960 success ACK is sent after required Kafka production succeeds. History and analytics failures do not block ACK. - -## Local Start - -Use the commands from Task 8 Step 3. - -## Verification - -Use health endpoints, Kafka topic consumption, archive file count, event-store query APIs, analytics output files, and ACK latency metrics. - -## Rollback - -Stop split services and start `bootstrap-all` with the previous environment. Keep `bootstrap-all` available until split service counts and sample queries match production expectations. -``` - -- [ ] **Step 7: Commit** - -```bash -git add docs/operations/gb32960-service-split-runbook.md -git commit -m "docs: add gb32960 split service runbook" -``` - ---- - -### Task 9: Final Verification and Cutover Checklist - -**Files:** -- Modify: `docs/operations/gb32960-service-split-runbook.md` - -- [ ] **Step 1: Run targeted tests** - -Run: - -```bash -mvn -pl :sink-kafka,:ingest-core,:protocol-gb32960,:event-history-service,:vehicle-state-service,:vehicle-stat-service test -``` - -Expected: BUILD SUCCESS. - -- [ ] **Step 2: Run split app tests** - -Run: - -```bash -mvn -pl :gb32960-ingest-app,:vehicle-history-app,:vehicle-analytics-app test -``` - -Expected: BUILD SUCCESS. - -- [ ] **Step 3: Run package verification** - -Run: - -```bash -mvn -pl :gb32960-ingest-app,:vehicle-history-app,:vehicle-analytics-app -am package -Dmaven.test.skip=true -``` - -Expected: BUILD SUCCESS. - -- [ ] **Step 4: Update runbook with measured results** - -Append a section: - -```markdown -## Latest Local Verification - -- Build: -- Unit tests: -- Split app startup: -- Kafka raw records: -- Kafka event records: -- Archive files: -- Event-store files: -- Analytics output: -- ACK behavior: -``` - -Fill the fields with the actual command outputs and counts from the run. - -- [ ] **Step 5: Commit final verification docs** - -```bash -git add docs/operations/gb32960-service-split-runbook.md -git commit -m "docs: record gb32960 split verification" -``` - ---- - -## Self-Review - -Spec coverage: - -- Three services are covered by Tasks 1, 2, 3, and 7. -- Kafka topic and routing contract is covered by Task 4. -- ACK after Kafka success is covered by Task 5. -- History consumption and storage is covered by Task 6 and Task 8. -- Analytics consumption and processing is covered by Task 6 and Task 8. -- `bootstrap-all` remains untouched except for pre-existing unrelated worktree changes. -- Verification and runbook are covered by Tasks 8 and 9. - -Implementation risk notes: - -- Task 5 is the most invasive task because it changes protocol ACK ordering. Implement it behind the smallest possible new API and keep `Dispatcher.dispatch(RawFrame)` behavior for other protocols. -- If Kafka transactions are needed to atomically produce raw and normalized records, add that as a follow-up after the split app boundary is working. The first implementation can use idempotent producer plus shared `eventId`/`sourceRawEventId`. -- If composition tests need additional auto-config classes, copy the exact pattern from `ProtocolAutoConfigurationCompositionTest` rather than widening app dependencies. diff --git a/docs/superpowers/plans/2026-06-29-vehicle-ingest-redesign-phase1.md b/docs/superpowers/plans/2026-06-29-vehicle-ingest-redesign-phase1.md deleted file mode 100644 index 598ab42d..00000000 --- a/docs/superpowers/plans/2026-06-29-vehicle-ingest-redesign-phase1.md +++ /dev/null @@ -1,1519 +0,0 @@ -> **Superseded:** This 2026-06-29 phase-1 implementation plan is historical -> context. Use `docs/target-architecture.md` and -> `docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.md` for the -> current production architecture: raw bytes are owned by `sink-archive`, -> history reads come from TDengine, and the former `raw-archive-store` -> prototype has been removed from the current build surface. - -# Vehicle Ingest Redesign Phase 1 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Build the foundation for the new GB32960/JT808 architecture: shared fact model, raw archive contract, and byte-light Kafka schema. - -**Architecture:** Phase 1 adds new modules beside existing runtime paths. It does not change current Netty handlers, ACK behavior, or TDengine writes yet. The output is a tested set of immutable facts and contracts that later phases will wire into protocol apps and the history writer. - -**Tech Stack:** Java 25, Maven, JUnit 5, AssertJ, Protobuf 4, existing `ProtocolId`. - ---- - -## Scope - -This plan implements only the first slice of the redesign: - -- `modules/core/ingest-facts`: protocol-neutral fact records and helpers. -- `modules/sinks/raw-archive-store`: raw archive write/read contract and local URI-safe implementation skeleton. -- `modules/sinks/sink-kafka`: protobuf messages for raw frame facts and decoded facts. -- Parent Maven wiring and focused tests. - -It intentionally does not yet modify: - -- `gb32960-ingest-app` -- `jt808-ingest-app` -- existing `Dispatcher` -- existing `VehicleEvent` -- existing `vehicle-history-app` TDengine writer - -Those changes belong to later implementation plans. - -## File Structure - -Create: - -- `modules/core/ingest-facts/pom.xml`: Maven module for fact model. -- `modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/ParseStatus.java`: parse status enum. -- `modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/VehicleKey.java`: stable vehicle key derivation. -- `modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/RawFrameFact.java`: raw frame fact record. -- `modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/DecodedFact.java`: sealed decoded fact hierarchy. -- `modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/FactIds.java`: deterministic id helpers. -- `modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/VehicleKeyTest.java` -- `modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/RawFrameFactTest.java` -- `modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/FactIdsTest.java` -- `modules/sinks/raw-archive-store/pom.xml`: Maven module for raw archive contracts. -- `modules/sinks/raw-archive-store/src/main/java/com/lingniu/ingest/rawarchive/RawArchiveWriteRequest.java` -- `modules/sinks/raw-archive-store/src/main/java/com/lingniu/ingest/rawarchive/RawArchiveReceipt.java` -- `modules/sinks/raw-archive-store/src/main/java/com/lingniu/ingest/rawarchive/RawArchiveWriter.java` -- `modules/sinks/raw-archive-store/src/main/java/com/lingniu/ingest/rawarchive/RawArchiveReader.java` -- `modules/sinks/raw-archive-store/src/main/java/com/lingniu/ingest/rawarchive/LocalRawArchiveStore.java` -- `modules/sinks/raw-archive-store/src/test/java/com/lingniu/ingest/rawarchive/LocalRawArchiveStoreTest.java` - -Modify: - -- `pom.xml`: add modules and dependency management entries. -- `modules/sinks/sink-kafka/pom.xml`: add dependency on `ingest-facts`. -- `modules/sinks/sink-kafka/src/main/proto/vehicle_envelope.proto`: add `RawFrameFactPayload`, `DecodedFactPayload`, and related enums/messages. -- `modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/VehicleEnvelopeProtoCompatibilityTest.java`: verify new proto messages can round-trip. - ---- - -### Task 1: Add `ingest-facts` Maven Module - -**Files:** - -- Modify: `pom.xml` -- Create: `modules/core/ingest-facts/pom.xml` -- Test later in Task 2 through Task 4. - -- [ ] **Step 1: Add failing Maven target check** - -Run: - -```bash -mvn -pl modules/core/ingest-facts test -``` - -Expected: FAIL because Maven cannot find the selected project. - -- [ ] **Step 2: Add module to root `pom.xml`** - -Add this module after `modules/core/ingest-api`: - -```xml -modules/core/ingest-facts -``` - -Add this dependency management entry after `ingest-api`: - -```xml - - com.lingniu.ingest - ingest-facts - ${project.version} - -``` - -- [ ] **Step 3: Create module POM** - -Create `modules/core/ingest-facts/pom.xml`: - -```xml - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - ingest-facts - ingest-facts - Protocol-neutral raw frame and decoded fact model for vehicle ingestion. - - - - com.lingniu.ingest - ingest-api - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - -``` - -- [ ] **Step 4: Run Maven target check** - -Run: - -```bash -mvn -pl modules/core/ingest-facts test -``` - -Expected: PASS with no tests. - -- [ ] **Step 5: Commit** - -```bash -git add pom.xml modules/core/ingest-facts/pom.xml -git commit -m "build: add ingest facts module" -``` - ---- - -### Task 2: Implement Vehicle Key Derivation - -**Files:** - -- Create: `modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/VehicleKey.java` -- Create: `modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/VehicleKeyTest.java` - -- [ ] **Step 1: Write failing tests** - -Create `VehicleKeyTest.java`: - -```java -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class VehicleKeyTest { - - @Test - void usesVinForGb32960() { - assertThat(VehicleKey.derive(ProtocolId.GB32960, " LB9A32A20P0LS1257 ", "", "abc")) - .isEqualTo("LB9A32A20P0LS1257"); - } - - @Test - void usesResolvedVinForJt808WhenPresent() { - assertThat(VehicleKey.derive(ProtocolId.JT808, "VIN001", "013912345678", "abc")) - .isEqualTo("VIN001"); - } - - @Test - void usesPhoneForJt808WhenVinIsMissing() { - assertThat(VehicleKey.derive(ProtocolId.JT808, "", "013912345678", "abc")) - .isEqualTo("jt808:013912345678"); - } - - @Test - void usesStableUnknownHashWhenVehicleIdentifiersAreMissing() { - assertThat(VehicleKey.derive(ProtocolId.JT808, "", "", "abc")) - .isEqualTo(VehicleKey.derive(ProtocolId.JT808, null, null, "abc")) - .startsWith("unknown:JT808:"); - } - - @Test - void rejectsMissingProtocol() { - assertThatThrownBy(() -> VehicleKey.derive(null, "VIN001", "", "abc")) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("protocol"); - } -} -``` - -- [ ] **Step 2: Run test and verify RED** - -Run: - -```bash -mvn -pl modules/core/ingest-facts -Dtest=VehicleKeyTest test -``` - -Expected: FAIL because `VehicleKey` does not exist. - -- [ ] **Step 3: Implement `VehicleKey`** - -Create `VehicleKey.java`: - -```java -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; - -import java.nio.charset.StandardCharsets; -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.util.HexFormat; - -public final class VehicleKey { - - private VehicleKey() { - } - - public static String derive(ProtocolId protocol, String vin, String phone, String rawIdentitySeed) { - if (protocol == null) { - throw new IllegalArgumentException("protocol must not be null"); - } - String normalizedVin = clean(vin); - if (!normalizedVin.isBlank()) { - return normalizedVin; - } - String normalizedPhone = clean(phone); - if (protocol == ProtocolId.JT808 && !normalizedPhone.isBlank()) { - return "jt808:" + normalizedPhone; - } - String seed = clean(rawIdentitySeed); - if (seed.isBlank()) { - seed = protocol.name(); - } - return "unknown:" + protocol.name() + ":" + sha256(seed).substring(0, 16); - } - - private static String clean(String value) { - return value == null ? "" : value.trim(); - } - - private static String sha256(String value) { - try { - MessageDigest digest = MessageDigest.getInstance("SHA-256"); - return HexFormat.of().formatHex(digest.digest(value.getBytes(StandardCharsets.UTF_8))); - } catch (NoSuchAlgorithmException e) { - throw new IllegalStateException("SHA-256 is unavailable", e); - } - } -} -``` - -- [ ] **Step 4: Run test and verify GREEN** - -Run: - -```bash -mvn -pl modules/core/ingest-facts -Dtest=VehicleKeyTest test -``` - -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/VehicleKey.java \ - modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/VehicleKeyTest.java -git commit -m "feat: add vehicle key derivation" -``` - ---- - -### Task 3: Implement Raw Frame Fact and Id Helpers - -**Files:** - -- Create: `modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/ParseStatus.java` -- Create: `modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/FactIds.java` -- Create: `modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/RawFrameFact.java` -- Create: `modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/FactIdsTest.java` -- Create: `modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/RawFrameFactTest.java` - -- [ ] **Step 1: Write failing tests** - -Create `FactIdsTest.java`: - -```java -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; - -import static org.assertj.core.api.Assertions.assertThat; - -class FactIdsTest { - - @Test - void rawFrameIdIsStableForSameInputs() { - String first = FactIds.rawFrameId( - ProtocolId.GB32960, - "VIN001", - 2, - 7, - Instant.parse("2026-06-29T00:00:00Z"), - "sha256:abc"); - String second = FactIds.rawFrameId( - ProtocolId.GB32960, - "VIN001", - 2, - 7, - Instant.parse("2026-06-29T00:00:00Z"), - "sha256:abc"); - - assertThat(first).isEqualTo(second).startsWith("rf_"); - } - - @Test - void decodedFactIdIncludesKind() { - String id = FactIds.decodedFactId("rf_123", "location", 0); - - assertThat(id).isEqualTo("df_rf_123_location_0"); - } -} -``` - -Create `RawFrameFactTest.java`: - -```java -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class RawFrameFactTest { - - @Test - void normalizesOptionalStringsAndMetadata() { - RawFrameFact fact = new RawFrameFact( - "frame-1", - ProtocolId.JT808, - "jt808:013912345678", - null, - "013912345678", - 0x0200, - 0, - null, - Instant.parse("2026-06-29T00:00:00Z"), - null, - "archive://2026/06/29/JT808/jt808-013912345678/frame-1.bin", - "sha256:abc", - 128, - ParseStatus.SUCCEEDED, - null, - Map.of("messageName", "LOCATION")); - - assertThat(fact.vin()).isEmpty(); - assertThat(fact.eventTime()).isEqualTo(fact.receivedAt()); - assertThat(fact.peer()).isEmpty(); - assertThat(fact.parseError()).isEmpty(); - assertThat(fact.metadata()).containsEntry("messageName", "LOCATION"); - } - - @Test - void rejectsMissingRequiredFields() { - assertThatThrownBy(() -> new RawFrameFact( - "", - ProtocolId.GB32960, - "VIN001", - "VIN001", - "", - 2, - 0, - Instant.parse("2026-06-29T00:00:00Z"), - Instant.parse("2026-06-29T00:00:00Z"), - "", - "archive://x", - "sha256:abc", - 1, - ParseStatus.SUCCEEDED, - "", - Map.of())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("frameId"); - } -} -``` - -- [ ] **Step 2: Run tests and verify RED** - -Run: - -```bash -mvn -pl modules/core/ingest-facts -Dtest=FactIdsTest,RawFrameFactTest test -``` - -Expected: FAIL because `FactIds`, `RawFrameFact`, and `ParseStatus` do not exist. - -- [ ] **Step 3: Implement `ParseStatus`** - -Create `ParseStatus.java`: - -```java -package com.lingniu.ingest.facts; - -public enum ParseStatus { - NOT_PARSED, - SUCCEEDED, - FAILED -} -``` - -- [ ] **Step 4: Implement `FactIds`** - -Create `FactIds.java`: - -```java -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; - -import java.nio.charset.StandardCharsets; -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.time.Instant; -import java.util.HexFormat; - -public final class FactIds { - - private FactIds() { - } - - public static String rawFrameId(ProtocolId protocol, - String vehicleKey, - int messageId, - int subType, - Instant receivedAt, - String checksum) { - if (protocol == null) { - throw new IllegalArgumentException("protocol must not be null"); - } - String input = protocol.name() + "|" - + clean(vehicleKey) + "|" - + messageId + "|" - + subType + "|" - + (receivedAt == null ? "" : receivedAt.toString()) + "|" - + clean(checksum); - return "rf_" + sha256(input).substring(0, 32); - } - - public static String decodedFactId(String frameId, String kind, int ordinal) { - return "df_" + clean(frameId) + "_" + clean(kind) + "_" + ordinal; - } - - private static String clean(String value) { - return value == null ? "" : value.trim(); - } - - private static String sha256(String value) { - try { - MessageDigest digest = MessageDigest.getInstance("SHA-256"); - return HexFormat.of().formatHex(digest.digest(value.getBytes(StandardCharsets.UTF_8))); - } catch (NoSuchAlgorithmException e) { - throw new IllegalStateException("SHA-256 is unavailable", e); - } - } -} -``` - -- [ ] **Step 5: Implement `RawFrameFact`** - -Create `RawFrameFact.java`: - -```java -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; - -import java.time.Instant; -import java.util.Map; - -public record RawFrameFact( - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - int messageId, - int subType, - Instant eventTime, - Instant receivedAt, - String peer, - String rawUri, - String checksum, - long rawSizeBytes, - ParseStatus parseStatus, - String parseError, - Map metadata -) { - public RawFrameFact { - frameId = required(frameId, "frameId"); - if (protocol == null) { - throw new IllegalArgumentException("protocol must not be null"); - } - vehicleKey = required(vehicleKey, "vehicleKey"); - receivedAt = required(receivedAt, "receivedAt"); - eventTime = eventTime == null ? receivedAt : eventTime; - rawUri = required(rawUri, "rawUri"); - checksum = required(checksum, "checksum"); - if (rawSizeBytes < 0) { - throw new IllegalArgumentException("rawSizeBytes must not be negative"); - } - parseStatus = parseStatus == null ? ParseStatus.NOT_PARSED : parseStatus; - vin = clean(vin); - phone = clean(phone); - peer = clean(peer); - parseError = clean(parseError); - metadata = metadata == null ? Map.of() : Map.copyOf(metadata); - } - - private static String required(String value, String field) { - String cleaned = clean(value); - if (cleaned.isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - return cleaned; - } - - private static Instant required(Instant value, String field) { - if (value == null) { - throw new IllegalArgumentException(field + " must not be null"); - } - return value; - } - - private static String clean(String value) { - return value == null ? "" : value.trim(); - } -} -``` - -- [ ] **Step 6: Run tests and verify GREEN** - -Run: - -```bash -mvn -pl modules/core/ingest-facts -Dtest=FactIdsTest,RawFrameFactTest test -``` - -Expected: PASS. - -- [ ] **Step 7: Commit** - -```bash -git add modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts \ - modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts -git commit -m "feat: add raw frame fact model" -``` - ---- - -### Task 4: Implement Decoded Fact Records - -**Files:** - -- Create: `modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/DecodedFact.java` -- Create: `modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/DecodedFactTest.java` - -- [ ] **Step 1: Write failing test** - -Create `DecodedFactTest.java`: - -```java -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class DecodedFactTest { - - @Test - void locationFactCarriesRawReferenceAndMetadata() { - DecodedFact.Location location = new DecodedFact.Location( - "fact-1", - "frame-1", - ProtocolId.JT808, - "jt808:013912345678", - "", - "013912345678", - Instant.parse("2026-06-29T00:00:00Z"), - Instant.parse("2026-06-29T00:00:01Z"), - "archive://jt808/frame-1.bin", - Map.of("messageId", "0x0200"), - 120.123456, - 30.123456, - 10.0, - 42.0, - 180.0, - 0L, - 2L, - 1000.5); - - assertThat(location.factId()).isEqualTo("fact-1"); - assertThat(location.rawUri()).isEqualTo("archive://jt808/frame-1.bin"); - assertThat(location.metadata()).containsEntry("messageId", "0x0200"); - } -} -``` - -- [ ] **Step 2: Run test and verify RED** - -Run: - -```bash -mvn -pl modules/core/ingest-facts -Dtest=DecodedFactTest test -``` - -Expected: FAIL because `DecodedFact` does not exist. - -- [ ] **Step 3: Implement `DecodedFact`** - -Create `DecodedFact.java`: - -```java -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; - -import java.time.Instant; -import java.util.Map; - -public sealed interface DecodedFact - permits DecodedFact.Location, - DecodedFact.Realtime, - DecodedFact.Alarm, - DecodedFact.Session, - DecodedFact.Register, - DecodedFact.Extension { - - String factId(); - String frameId(); - ProtocolId protocol(); - String vehicleKey(); - String vin(); - String phone(); - Instant eventTime(); - Instant receivedAt(); - String rawUri(); - Map metadata(); - - record Location( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - Double longitude, - Double latitude, - Double altitudeM, - Double speedKmh, - Double directionDeg, - Long alarmFlag, - Long statusFlag, - Double totalMileageKm - ) implements DecodedFact { - public Location { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - metadata = BaseFields.metadata(metadata); - } - } - - record Realtime( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - Map fields - ) implements DecodedFact { - public Realtime { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - metadata = BaseFields.metadata(metadata); - fields = fields == null ? Map.of() : Map.copyOf(fields); - } - } - - record Alarm( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - String alarmLevel, - Long alarmCode, - String alarmName, - Map fields - ) implements DecodedFact { - public Alarm { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - alarmLevel = BaseFields.clean(alarmLevel); - alarmName = BaseFields.clean(alarmName); - metadata = BaseFields.metadata(metadata); - fields = fields == null ? Map.of() : Map.copyOf(fields); - } - } - - record Session( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - String sessionType, - Integer resultCode, - Map fields - ) implements DecodedFact { - public Session { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - sessionType = BaseFields.clean(sessionType); - metadata = BaseFields.metadata(metadata); - fields = fields == null ? Map.of() : Map.copyOf(fields); - } - } - - record Register( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - Map registrationFields - ) implements DecodedFact { - public Register { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - metadata = BaseFields.metadata(metadata); - registrationFields = registrationFields == null ? Map.of() : Map.copyOf(registrationFields); - } - } - - record Extension( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - String extensionType, - Map fields - ) implements DecodedFact { - public Extension { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - extensionType = BaseFields.clean(extensionType); - metadata = BaseFields.metadata(metadata); - fields = fields == null ? Map.of() : Map.copyOf(fields); - } - } - - final class BaseFields { - private BaseFields() { - } - - static void validate(String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - Instant eventTime, - Instant receivedAt, - String rawUri) { - required(factId, "factId"); - required(frameId, "frameId"); - if (protocol == null) { - throw new IllegalArgumentException("protocol must not be null"); - } - required(vehicleKey, "vehicleKey"); - if (eventTime == null) { - throw new IllegalArgumentException("eventTime must not be null"); - } - if (receivedAt == null) { - throw new IllegalArgumentException("receivedAt must not be null"); - } - required(rawUri, "rawUri"); - } - - static String clean(String value) { - return value == null ? "" : value.trim(); - } - - static Map metadata(Map metadata) { - return metadata == null ? Map.of() : Map.copyOf(metadata); - } - - private static void required(String value, String field) { - if (clean(value).isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - } - } -} -``` - -- [ ] **Step 4: Run test and verify GREEN** - -Run: - -```bash -mvn -pl modules/core/ingest-facts -Dtest=DecodedFactTest test -``` - -Expected: PASS. - -- [ ] **Step 5: Commit** - -```bash -git add modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/DecodedFact.java \ - modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/DecodedFactTest.java -git commit -m "feat: add decoded fact model" -``` - ---- - -### Task 5: Add Raw Archive Store Contract and Local Implementation - -**Files:** - -- Modify: `pom.xml` -- Create: `modules/sinks/raw-archive-store/pom.xml` -- Create: `modules/sinks/raw-archive-store/src/main/java/com/lingniu/ingest/rawarchive/RawArchiveWriteRequest.java` -- Create: `modules/sinks/raw-archive-store/src/main/java/com/lingniu/ingest/rawarchive/RawArchiveReceipt.java` -- Create: `modules/sinks/raw-archive-store/src/main/java/com/lingniu/ingest/rawarchive/RawArchiveWriter.java` -- Create: `modules/sinks/raw-archive-store/src/main/java/com/lingniu/ingest/rawarchive/RawArchiveReader.java` -- Create: `modules/sinks/raw-archive-store/src/main/java/com/lingniu/ingest/rawarchive/LocalRawArchiveStore.java` -- Create: `modules/sinks/raw-archive-store/src/test/java/com/lingniu/ingest/rawarchive/LocalRawArchiveStoreTest.java` - -- [ ] **Step 1: Write failing local archive test** - -Create `LocalRawArchiveStoreTest.java`: - -```java -package com.lingniu.ingest.rawarchive; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.nio.file.Path; -import java.time.Instant; -import java.util.HexFormat; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class LocalRawArchiveStoreTest { - - @TempDir - Path tempDir; - - @Test - void writesReadableRawObjectWithArchiveUriAndChecksum() throws Exception { - LocalRawArchiveStore store = new LocalRawArchiveStore(tempDir, "archive://raw"); - byte[] bytes = HexFormat.of().parseHex("7e020000007e"); - - RawArchiveReceipt receipt = store.write(new RawArchiveWriteRequest( - ProtocolId.JT808, - "jt808:013912345678", - "frame-1", - Instant.parse("2026-06-29T00:00:00Z"), - bytes)); - - assertThat(receipt.rawUri()).isEqualTo("archive://raw/2026/06/29/JT808/jt808_013912345678/frame-1.bin"); - assertThat(receipt.checksum()).startsWith("sha256:"); - assertThat(receipt.sizeBytes()).isEqualTo(bytes.length); - assertThat(store.read(receipt.rawUri())).containsExactly(bytes); - } - - @Test - void rejectsBlankFrameId() { - LocalRawArchiveStore store = new LocalRawArchiveStore(tempDir, "archive://raw"); - - assertThatThrownBy(() -> store.write(new RawArchiveWriteRequest( - ProtocolId.GB32960, - "VIN001", - "", - Instant.parse("2026-06-29T00:00:00Z"), - new byte[]{1}))) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("frameId"); - } -} -``` - -- [ ] **Step 2: Run test and verify RED** - -Run: - -```bash -mvn -pl modules/sinks/raw-archive-store test -``` - -Expected: FAIL because the module does not exist. - -- [ ] **Step 3: Add module and POM wiring** - -Add this root module after `modules/sinks/sink-archive`: - -```xml -modules/sinks/raw-archive-store -``` - -Add dependency management after `sink-archive`: - -```xml - - com.lingniu.ingest - raw-archive-store - ${project.version} - -``` - -Create `modules/sinks/raw-archive-store/pom.xml`: - -```xml - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - raw-archive-store - raw-archive-store - Raw frame archive writer and reader contracts. - - - - com.lingniu.ingest - ingest-api - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - -``` - -- [ ] **Step 4: Implement contracts** - -Create `RawArchiveWriteRequest.java`: - -```java -package com.lingniu.ingest.rawarchive; - -import com.lingniu.ingest.api.ProtocolId; - -import java.time.Instant; - -public record RawArchiveWriteRequest( - ProtocolId protocol, - String vehicleKey, - String frameId, - Instant receivedAt, - byte[] rawBytes -) { - public RawArchiveWriteRequest { - if (protocol == null) { - throw new IllegalArgumentException("protocol must not be null"); - } - vehicleKey = required(vehicleKey, "vehicleKey"); - frameId = required(frameId, "frameId"); - if (receivedAt == null) { - throw new IllegalArgumentException("receivedAt must not be null"); - } - if (rawBytes == null || rawBytes.length == 0) { - throw new IllegalArgumentException("rawBytes must not be empty"); - } - rawBytes = rawBytes.clone(); - } - - @Override - public byte[] rawBytes() { - return rawBytes.clone(); - } - - private static String required(String value, String field) { - String cleaned = value == null ? "" : value.trim(); - if (cleaned.isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - return cleaned; - } -} -``` - -Create `RawArchiveReceipt.java`: - -```java -package com.lingniu.ingest.rawarchive; - -public record RawArchiveReceipt(String rawUri, String checksum, long sizeBytes) { - public RawArchiveReceipt { - rawUri = required(rawUri, "rawUri"); - checksum = required(checksum, "checksum"); - if (sizeBytes < 0) { - throw new IllegalArgumentException("sizeBytes must not be negative"); - } - } - - private static String required(String value, String field) { - String cleaned = value == null ? "" : value.trim(); - if (cleaned.isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - return cleaned; - } -} -``` - -Create `RawArchiveWriter.java`: - -```java -package com.lingniu.ingest.rawarchive; - -import java.io.IOException; - -public interface RawArchiveWriter { - RawArchiveReceipt write(RawArchiveWriteRequest request) throws IOException; -} -``` - -Create `RawArchiveReader.java`: - -```java -package com.lingniu.ingest.rawarchive; - -import java.io.IOException; - -public interface RawArchiveReader { - byte[] read(String rawUri) throws IOException; -} -``` - -- [ ] **Step 5: Implement local store** - -Create `LocalRawArchiveStore.java`: - -```java -package com.lingniu.ingest.rawarchive; - -import java.io.IOException; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.LinkOption; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.time.ZoneId; -import java.util.HexFormat; - -public final class LocalRawArchiveStore implements RawArchiveWriter, RawArchiveReader { - - private static final ZoneId PARTITION_ZONE = ZoneId.of("Asia/Shanghai"); - - private final Path root; - private final String uriPrefix; - - public LocalRawArchiveStore(Path root, String uriPrefix) { - if (root == null) { - throw new IllegalArgumentException("root must not be null"); - } - this.root = root.toAbsolutePath().normalize(); - String prefix = uriPrefix == null || uriPrefix.isBlank() ? "archive://raw" : uriPrefix.trim(); - this.uriPrefix = prefix.endsWith("/") ? prefix.substring(0, prefix.length() - 1) : prefix; - } - - @Override - public RawArchiveReceipt write(RawArchiveWriteRequest request) throws IOException { - String key = key(request); - Path target = resolveKey(key); - Files.createDirectories(target.getParent()); - rejectSymlinkPath(target); - byte[] bytes = request.rawBytes(); - Files.write(target, bytes, StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE); - return new RawArchiveReceipt(uriPrefix + "/" + key, checksum(bytes), bytes.length); - } - - @Override - public byte[] read(String rawUri) throws IOException { - String key = keyFromUri(rawUri); - Path target = resolveKey(key); - rejectSymlinkPath(target); - return Files.readAllBytes(target); - } - - private String key(RawArchiveWriteRequest request) { - var date = request.receivedAt().atZone(PARTITION_ZONE).toLocalDate(); - return "%04d/%02d/%02d/%s/%s/%s.bin".formatted( - date.getYear(), - date.getMonthValue(), - date.getDayOfMonth(), - request.protocol().name(), - safeSegment(request.vehicleKey()), - safeSegment(request.frameId())); - } - - private String keyFromUri(String rawUri) { - if (rawUri == null || rawUri.isBlank()) { - throw new IllegalArgumentException("rawUri must not be blank"); - } - String normalized = rawUri.trim(); - String expected = uriPrefix + "/"; - if (!normalized.startsWith(expected)) { - throw new IllegalArgumentException("rawUri does not use expected prefix: " + rawUri); - } - return normalized.substring(expected.length()); - } - - private Path resolveKey(String key) { - Path target = root.resolve(key).normalize(); - if (!target.startsWith(root)) { - throw new IllegalArgumentException("raw archive key escapes root: " + key); - } - return target; - } - - private void rejectSymlinkPath(Path target) throws IOException { - Path current = root; - Path relative = root.relativize(target); - for (Path segment : relative) { - current = current.resolve(segment); - if (Files.isSymbolicLink(current)) { - throw new IOException("raw archive path traverses symlink: " + current); - } - if (Files.notExists(current, LinkOption.NOFOLLOW_LINKS)) { - return; - } - } - } - - private static String safeSegment(String value) { - String cleaned = value == null ? "" : value.trim(); - if (cleaned.isBlank()) { - return "unknown"; - } - return cleaned.replaceAll("[^A-Za-z0-9._-]", "_"); - } - - private static String checksum(byte[] bytes) { - try { - MessageDigest digest = MessageDigest.getInstance("SHA-256"); - return "sha256:" + HexFormat.of().formatHex(digest.digest(bytes)); - } catch (NoSuchAlgorithmException e) { - throw new IllegalStateException("SHA-256 is unavailable", e); - } - } -} -``` - -- [ ] **Step 6: Run tests and verify GREEN** - -Run: - -```bash -mvn -pl modules/sinks/raw-archive-store -Dtest=LocalRawArchiveStoreTest test -``` - -Expected: PASS. - -- [ ] **Step 7: Commit** - -```bash -git add pom.xml modules/sinks/raw-archive-store -git commit -m "feat: add raw archive store contract" -``` - ---- - -### Task 6: Add Kafka Proto Messages for Facts - -**Files:** - -- Modify: `modules/sinks/sink-kafka/pom.xml` -- Modify: `modules/sinks/sink-kafka/src/main/proto/vehicle_envelope.proto` -- Create: `modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/VehicleEnvelopeProtoCompatibilityTest.java` - -- [ ] **Step 1: Write failing proto compatibility test** - -Create `VehicleEnvelopeProtoCompatibilityTest.java`: - -```java -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.sink.kafka.proto.DecodedFactPayload; -import com.lingniu.ingest.sink.kafka.proto.ParseStatusProto; -import com.lingniu.ingest.sink.kafka.proto.RawFrameFactPayload; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; - -class VehicleEnvelopeProtoCompatibilityTest { - - @Test - void rawFrameFactPayloadRoundTripsThroughEnvelope() throws Exception { - VehicleEnvelope envelope = VehicleEnvelope.newBuilder() - .setSchemaVersion("2.0") - .setEventId("frame-1") - .setVin("VIN001") - .setSource("GB32960") - .setEventTimeMs(1782662400000L) - .setIngestTimeMs(1782662400001L) - .setRawFrameFact(RawFrameFactPayload.newBuilder() - .setFrameId("frame-1") - .setVehicleKey("VIN001") - .setMessageId(2) - .setRawUri("archive://raw/2026/06/29/GB32960/VIN001/frame-1.bin") - .setChecksum("sha256:abc") - .setRawSizeBytes(128) - .setParseStatus(ParseStatusProto.PARSE_STATUS_SUCCEEDED) - .build()) - .build(); - - VehicleEnvelope parsed = VehicleEnvelope.parseFrom(envelope.toByteArray()); - - assertThat(parsed.hasRawFrameFact()).isTrue(); - assertThat(parsed.getRawFrameFact().getFrameId()).isEqualTo("frame-1"); - assertThat(parsed.getRawFrameFact().getParseStatus()).isEqualTo(ParseStatusProto.PARSE_STATUS_SUCCEEDED); - } - - @Test - void decodedFactPayloadRoundTripsThroughEnvelope() throws Exception { - VehicleEnvelope envelope = VehicleEnvelope.newBuilder() - .setSchemaVersion("2.0") - .setEventId("fact-1") - .setVin("VIN001") - .setSource("JT808") - .setEventTimeMs(1782662400000L) - .setIngestTimeMs(1782662400001L) - .setDecodedFact(DecodedFactPayload.newBuilder() - .setFactId("fact-1") - .setFrameId("frame-1") - .setFactType("location") - .setVehicleKey("jt808:013912345678") - .setRawUri("archive://raw/2026/06/29/JT808/jt808_013912345678/frame-1.bin") - .putFields("longitude", "120.1") - .putFields("latitude", "30.1") - .build()) - .build(); - - VehicleEnvelope parsed = VehicleEnvelope.parseFrom(envelope.toByteArray()); - - assertThat(parsed.hasDecodedFact()).isTrue(); - assertThat(parsed.getDecodedFact().getFieldsMap()).containsEntry("longitude", "120.1"); - } -} -``` - -- [ ] **Step 2: Run test and verify RED** - -Run: - -```bash -mvn -pl modules/sinks/sink-kafka -Dtest=VehicleEnvelopeProtoCompatibilityTest test -``` - -Expected: FAIL because proto generated classes do not exist. - -- [ ] **Step 3: Add `ingest-facts` dependency to sink-kafka** - -Add this dependency to `modules/sinks/sink-kafka/pom.xml` after `ingest-api`: - -```xml - - com.lingniu.ingest - ingest-facts - -``` - -- [ ] **Step 4: Extend proto schema** - -Modify `vehicle_envelope.proto`. - -Add to `VehicleEnvelope` after `TelemetrySnapshot telemetry_snapshot = 50;`: - -```proto - RawFrameFactPayload raw_frame_fact = 60; - DecodedFactPayload decoded_fact = 61; -``` - -Add these messages after `TelemetryField`: - -```proto -enum ParseStatusProto { - PARSE_STATUS_UNSPECIFIED = 0; - PARSE_STATUS_NOT_PARSED = 1; - PARSE_STATUS_SUCCEEDED = 2; - PARSE_STATUS_FAILED = 3; -} - -message RawFrameFactPayload { - string frame_id = 1; - string vehicle_key = 2; - string vin = 3; - string phone = 4; - int32 message_id = 5; - int32 sub_type = 6; - string raw_uri = 7; - string checksum = 8; - int64 raw_size_bytes = 9; - ParseStatusProto parse_status = 10; - string parse_error = 11; - string peer = 12; - map metadata = 13; -} - -message DecodedFactPayload { - string fact_id = 1; - string frame_id = 2; - string fact_type = 3; - string vehicle_key = 4; - string vin = 5; - string phone = 6; - string raw_uri = 7; - map fields = 8; - map metadata = 9; -} -``` - -- [ ] **Step 5: Run test and verify GREEN** - -Run: - -```bash -mvn -pl modules/sinks/sink-kafka -Dtest=VehicleEnvelopeProtoCompatibilityTest test -``` - -Expected: PASS. - -- [ ] **Step 6: Run existing sink-kafka tests** - -Run: - -```bash -mvn -pl modules/sinks/sink-kafka test -``` - -Expected: PASS. Existing envelope tests must continue to pass because new proto fields are additive. - -- [ ] **Step 7: Commit** - -```bash -git add modules/sinks/sink-kafka/pom.xml \ - modules/sinks/sink-kafka/src/main/proto/vehicle_envelope.proto \ - modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/VehicleEnvelopeProtoCompatibilityTest.java -git commit -m "feat: add fact payloads to kafka envelope" -``` - ---- - -### Task 7: Verify Phase 1 Build - -**Files:** - -- No new files. - -- [ ] **Step 1: Run focused module tests** - -Run: - -```bash -mvn -pl modules/core/ingest-facts,modules/sinks/raw-archive-store,modules/sinks/sink-kafka -am test -``` - -Expected: PASS. - -- [ ] **Step 2: Run dependency tree smoke check** - -Run: - -```bash -mvn -pl modules/sinks/sink-kafka -am -DskipTests package -``` - -Expected: PASS. Generated protobuf Java sources include `RawFrameFactPayload`, `DecodedFactPayload`, and `ParseStatusProto`. - -- [ ] **Step 3: Search for accidental runtime wiring** - -Run: - -```bash -rg -n "RawFrameFact|DecodedFact|RawArchiveWriter|vehicle.raw-frame.v1|vehicle.decoded-fact.v1" \ - modules/apps modules/protocols modules/services modules/core modules/sinks -``` - -Expected: Matches only in the new modules, sink-kafka proto/test, and planned references. No existing handler behavior should be changed in Phase 1. - -- [ ] **Step 4: Commit final verification note if any docs changed** - -If the implementation updated this plan with checked boxes, commit it: - -```bash -git add docs/superpowers/plans/2026-06-29-vehicle-ingest-redesign-phase1.md -git commit -m "docs: update phase 1 implementation checklist" -``` - -If no docs changed, no commit is needed. - ---- - -## Self-Review - -Spec coverage: - -- Raw original frame preservation is covered by `RawFrameFact`, raw archive contracts, and local archive tests. -- Historical query storage is prepared by stable fact model and raw URI references; TDengine writer is intentionally deferred to Phase 3. -- Extension is covered by `DecodedFact.Extension` and byte-light proto maps. -- Performance foundations are covered by immutable facts, Kafka key design, and no handler wiring in Phase 1. - -Placeholder scan: - -- This plan contains no unfinished markers or unspecified implementation steps. - -Type consistency: - -- `RawFrameFact`, `ParseStatus`, `DecodedFact`, `RawArchiveWriteRequest`, `RawArchiveReceipt`, and proto payload names are used consistently across tasks. - -Next plans: - -- Phase 2: wire GB32960/JT808 durable raw boundary and fact publication. -- Phase 3: implement TDengine schema manager and history writer. -- Phase 4: implement history query APIs and raw replay endpoints. diff --git a/docs/superpowers/plans/2026-06-30-kafka-streaming-mileage.md b/docs/superpowers/plans/2026-06-30-kafka-streaming-mileage.md deleted file mode 100644 index 1cfd6828..00000000 --- a/docs/superpowers/plans/2026-06-30-kafka-streaming-mileage.md +++ /dev/null @@ -1,46 +0,0 @@ -# JT808 Kafka Streaming Mileage Plan - -## Current Goal - -Consume JT808 Kafka location events and write the derived daily mileage into the common vehicle-stat metric repository. - -## Current Design - -- Only supported message backbone: Kafka. -- Source topic: `vehicle.event.jt808.v1` -- Runtime app: `vehicle-analytics-app` -- Runtime state: none outside `vehicle_stat_metric` -- Metric output: `VehicleStatRepository.recordDailyMileageSample(...)` -- Production metric storage: JDBC/MySQL `vehicle_stat_metric` -- Date boundary: `Asia/Shanghai` -- Calculation method: `JT808_TOTAL_MILEAGE_DIFF` - -JT808 daily mileage is calculated only from the GPS total mileage reported by JT808 location additional information: - -```text -daily_mileage_km = max_total_mileage_km - min_total_mileage_km -``` - -The first valid local-day sample writes `daily_mileage_km=0.0`. Later ordered or replayed samples update the same metric row with: - -```text -metric_value = max_total_mileage_km - min_total_mileage_km -metric_key = daily_mileage_km -``` - -The local-day minimum and maximum GPS total mileage values stay on that same metric row as calculation source columns so restarts recover from MySQL without Redis or another state table. - -## Runtime Properties - -```text -KAFKA_TOPIC_JT808_EVENT=vehicle.event.jt808.v1 -VEHICLE_STAT_ENABLED=true -VEHICLE_STAT_JT808_MILEAGE_ENABLED=true -MYSQL_JDBC_URL= -MYSQL_USERNAME= -MYSQL_PASSWORD= -``` - -## Notes - -Do not create or write a protocol-specific JT808 daily-mileage table. Do not add another message backbone, distance accumulation, integral calculation, Redis state, or memory state back into this path unless the mileage definition changes again. diff --git a/docs/superpowers/plans/2026-07-01-go-ingest-redesign-phase1.md b/docs/superpowers/plans/2026-07-01-go-ingest-redesign-phase1.md deleted file mode 100644 index 20314115..00000000 --- a/docs/superpowers/plans/2026-07-01-go-ingest-redesign-phase1.md +++ /dev/null @@ -1,720 +0,0 @@ -# Go Vehicle Ingest Redesign Phase 1 Implementation Plan - -> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Build the first production-capable Go runtime for GB32960, JT808, and Yutong MQTT ingestion with unified Kafka, TDengine, MySQL statistics, and Redis realtime state boundaries. - -**Architecture:** Create a new single Go module at `go/vehicle-gateway` and migrate only the useful pieces from the existing `go/ingest-edge` and `go/vehicle-state` prototypes. The gateway produces unified envelopes to Kafka; independent consumers write TDengine history, MySQL daily metrics, and Redis realtime state. - -**Tech Stack:** Go 1.26+, Kafka (`segmentio/kafka-go`), Redis (`redis/go-redis`), MySQL (`go-sql-driver/mysql`), TDengine official Go connector, MQTT (`eclipse/paho.mqtt.golang`), standard library TCP. - ---- - -## File Structure - -- Create: `go/vehicle-gateway/go.mod` -- Create: `go/vehicle-gateway/cmd/gateway/main.go` -- Create: `go/vehicle-gateway/cmd/history-writer/main.go` -- Create: `go/vehicle-gateway/cmd/stat-writer/main.go` -- Create: `go/vehicle-gateway/cmd/realtime-api/main.go` -- Create: `go/vehicle-gateway/internal/envelope/envelope.go` -- Create: `go/vehicle-gateway/internal/envelope/envelope_test.go` -- Create: `go/vehicle-gateway/internal/protocol/jt808/*` -- Create: `go/vehicle-gateway/internal/protocol/gb32960/*` -- Create: `go/vehicle-gateway/internal/protocol/yutongmqtt/*` -- Create: `go/vehicle-gateway/internal/gateway/*` -- Create: `go/vehicle-gateway/internal/identity/*` -- Create: `go/vehicle-gateway/internal/eventbus/*` -- Create: `go/vehicle-gateway/internal/history/*` -- Create: `go/vehicle-gateway/internal/stats/*` -- Create: `go/vehicle-gateway/internal/realtime/*` -- Create: `go/vehicle-gateway/internal/observability/*` -- Modify: `README.md` -- Modify: `docs/target-architecture.md` -- Create: `deploy/portainer/docker-compose-go.yml` - -The existing `go/ingest-edge` and `go/vehicle-state` directories are migration sources only. After phase 1 is verified, remove or mark them superseded. - ---- - -### Task 1: Create Single Go Module - -**Files:** -- Create: `go/vehicle-gateway/go.mod` -- Create: `go/vehicle-gateway/internal/observability/logger.go` -- Create: `go/vehicle-gateway/cmd/gateway/main.go` - -- [ ] **Step 1: Create module manifest** - -Add `go/vehicle-gateway/go.mod`: - -```go -module lingniu-vehicle-ingest/go/vehicle-gateway - -go 1.26 - -require ( - github.com/eclipse/paho.mqtt.golang v1.5.1 - github.com/go-sql-driver/mysql v1.9.3 - github.com/redis/go-redis/v9 v9.17.2 - github.com/segmentio/kafka-go v0.4.49 - github.com/taosdata/driver-go/v3 v3.8.1 -) -``` - -- [ ] **Step 2: Add logger helper** - -Add `go/vehicle-gateway/internal/observability/logger.go`: - -```go -package observability - -import ( - "log/slog" - "os" -) - -func NewLogger(service string) *slog.Logger { - handler := slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{AddSource: true}) - return slog.New(handler).With("service", service) -} -``` - -- [ ] **Step 3: Add temporary gateway entrypoint** - -Add `go/vehicle-gateway/cmd/gateway/main.go`: - -```go -package main - -import "lingniu-vehicle-ingest/go/vehicle-gateway/internal/observability" - -func main() { - logger := observability.NewLogger("vehicle-gateway") - logger.Info("vehicle gateway scaffold started") -} -``` - -- [ ] **Step 4: Verify scaffold builds** - -Run: - -```bash -cd go/vehicle-gateway -go mod tidy -go test ./... -go build ./cmd/gateway -``` - -Expected: all commands exit `0`. - ---- - -### Task 2: Define Unified Envelope - -**Files:** -- Create: `go/vehicle-gateway/internal/envelope/envelope.go` -- Create: `go/vehicle-gateway/internal/envelope/envelope_test.go` - -- [ ] **Step 1: Write envelope tests** - -Add `go/vehicle-gateway/internal/envelope/envelope_test.go`: - -```go -package envelope - -import "testing" - -func TestFrameEnvelopeVehicleKeyPrefersVIN(t *testing.T) { - e := FrameEnvelope{Protocol: ProtocolJT808, VIN: "LNBVIN00000000001", Phone: "013307795425"} - if got := e.VehicleKey(); got != "LNBVIN00000000001" { - t.Fatalf("VehicleKey() = %q", got) - } -} - -func TestFrameEnvelopeVehicleKeyFallsBackToPhone(t *testing.T) { - e := FrameEnvelope{Protocol: ProtocolJT808, Phone: "013307795425"} - if got := e.VehicleKey(); got != "JT808:013307795425" { - t.Fatalf("VehicleKey() = %q", got) - } -} - -func TestFrameEnvelopeEventIDStable(t *testing.T) { - e := FrameEnvelope{ - Protocol: ProtocolJT808, - MessageID: "0x0200", - Phone: "013307795425", - Sequence: 1, - EventTimeMS: 1782745114000, - ReceivedAtMS: 1782745114999, - RawHex: "7e02000000ff7e", - } - a := e.StableEventID() - b := e.StableEventID() - if a == "" || a != b { - t.Fatalf("event id must be non-empty and stable: %q %q", a, b) - } -} -``` - -- [ ] **Step 2: Run tests and confirm failure** - -Run: - -```bash -cd go/vehicle-gateway -go test ./internal/envelope -``` - -Expected: fail because `FrameEnvelope` is not defined. - -- [ ] **Step 3: Implement envelope** - -Add `go/vehicle-gateway/internal/envelope/envelope.go`: - -```go -package envelope - -import ( - "crypto/sha256" - "encoding/hex" - "encoding/json" - "fmt" - "strings" -) - -type Protocol string - -const ( - ProtocolGB32960 Protocol = "GB32960" - ProtocolJT808 Protocol = "JT808" - ProtocolYutongMQTT Protocol = "YUTONG_MQTT" -) - -type ParseStatus string - -const ( - ParseOK ParseStatus = "OK" - ParsePartial ParseStatus = "PARTIAL" - ParseBadFrame ParseStatus = "BAD_FRAME" -) - -type FrameEnvelope struct { - EventID string `json:"event_id"` - TraceID string `json:"trace_id"` - Protocol Protocol `json:"protocol"` - MessageID string `json:"message_id"` - Sequence uint16 `json:"sequence"` - VIN string `json:"vin,omitempty"` - VehicleKeyHint string `json:"vehicle_key,omitempty"` - Phone string `json:"phone,omitempty"` - DeviceID string `json:"device_id,omitempty"` - Plate string `json:"plate,omitempty"` - SourceEndpoint string `json:"source_endpoint,omitempty"` - EventTimeMS int64 `json:"event_time_ms"` - ReceivedAtMS int64 `json:"received_at_ms"` - RawHex string `json:"raw_hex,omitempty"` - RawText string `json:"raw_text,omitempty"` - Parsed map[string]any `json:"parsed,omitempty"` - Fields map[string]any `json:"fields,omitempty"` - ParseStatus ParseStatus `json:"parse_status"` - ParseError string `json:"parse_error,omitempty"` -} - -func (e FrameEnvelope) VehicleKey() string { - if key := strings.TrimSpace(e.VIN); key != "" { - return key - } - if key := strings.TrimSpace(e.VehicleKeyHint); key != "" { - return key - } - if key := strings.TrimSpace(e.Phone); key != "" { - return string(e.Protocol) + ":" + key - } - if key := strings.TrimSpace(e.DeviceID); key != "" { - return string(e.Protocol) + ":" + key - } - return string(e.Protocol) + ":unknown" -} - -func (e FrameEnvelope) StableEventID() string { - if strings.TrimSpace(e.EventID) != "" { - return e.EventID - } - input := fmt.Sprintf("%s|%s|%s|%d|%d|%s", - e.Protocol, e.MessageID, e.VehicleKey(), e.Sequence, e.EventTimeMS, e.RawHex) - sum := sha256.Sum256([]byte(input)) - return hex.EncodeToString(sum[:16]) -} - -func (e FrameEnvelope) MarshalJSONBytes() ([]byte, error) { - if e.EventID == "" { - e.EventID = e.StableEventID() - } - if e.ParseStatus == "" { - e.ParseStatus = ParseOK - } - return json.Marshal(e) -} -``` - -- [ ] **Step 4: Verify tests pass** - -Run: - -```bash -cd go/vehicle-gateway -go test ./internal/envelope -``` - -Expected: pass. - ---- - -### Task 3: Implement JT808 Frame and 0200 Parser - -**Files:** -- Create: `go/vehicle-gateway/internal/protocol/jt808/parser.go` -- Create: `go/vehicle-gateway/internal/protocol/jt808/parser_test.go` - -- [ ] **Step 1: Add sample frame tests** - -Use the production sample provided in the thread: - -```text -7E020000320133077954250001000000000048000301D2C4C707376139000A00E6004F26063016235701040001900C2504000000000202000030011F31010F867E -``` - -Expected assertions: - -- message id is `0x0200` -- phone keeps normalized BCD string `013307795425` -- sequence is `1` -- `fields.total_mileage_km` is parsed from additional item `0x01` -- latitude, longitude, speed, direction, alarm, status, and device time are present - -- [ ] **Step 2: Implement parser** - -Implementation rules: - -- strip `0x7e` start/end delimiters -- unescape `0x7d 0x02 -> 0x7e` -- unescape `0x7d 0x01 -> 0x7d` -- verify XOR checksum -- parse 2011/2013 common header -- parse BCD phone from 6 bytes and keep both raw and normalized values in `parsed.header` -- parse 0200 fixed body -- parse additional item list as `id,length,value_hex` -- parse additional `0x01` as `total_mileage_km = uint32 / 10` - -- [ ] **Step 3: Verify** - -Run: - -```bash -cd go/vehicle-gateway -go test ./internal/protocol/jt808 -``` - -Expected: pass. - ---- - -### Task 4: Implement GB32960 Frame and Data Unit Parser - -**Files:** -- Create: `go/vehicle-gateway/internal/protocol/gb32960/parser.go` -- Create: `go/vehicle-gateway/internal/protocol/gb32960/parser_test.go` - -- [ ] **Step 1: Add parser tests** - -Tests must cover: - -- `##` frame boundary -- command id -- response flag -- 17-byte VIN -- encryption flag -- payload length -- BCC verification -- realtime data command `0x02` -- reissue data command `0x03` -- data unit `0x01` vehicle status fields -- data unit `0x05` position fields - -- [ ] **Step 2: Implement parser** - -Implementation rules: - -- parse header without allocating large temporary buffers -- keep RAW as hex in envelope -- write all recognized data units to `Parsed` -- write core fields to `Fields` -- unsupported data unit stays in `Parsed["unknown_units"]` - -- [ ] **Step 3: Verify** - -Run: - -```bash -cd go/vehicle-gateway -go test ./internal/protocol/gb32960 -``` - -Expected: pass. - ---- - -### Task 5: Implement Kafka Sink - -**Files:** -- Create: `go/vehicle-gateway/internal/eventbus/kafka_sink.go` -- Create: `go/vehicle-gateway/internal/eventbus/kafka_sink_test.go` - -- [ ] **Step 1: Add sink interface** - -Create a sink interface: - -```go -package eventbus - -import ( - "context" - "lingniu-vehicle-ingest/go/vehicle-gateway/internal/envelope" -) - -type Sink interface { - PublishRaw(context.Context, envelope.FrameEnvelope) error - PublishUnified(context.Context, envelope.FrameEnvelope) error - Close() error -} -``` - -- [ ] **Step 2: Implement Kafka topic routing** - -Rules: - -- `GB32960 -> vehicle.raw.gb32960.v1` -- `JT808 -> vehicle.raw.jt808.v1` -- `YUTONG_MQTT -> vehicle.raw.yutong-mqtt.v1` -- unified topic is `vehicle.event.unified.v1` -- message key is `env.VehicleKey()` - -- [ ] **Step 3: Verify routing with unit tests** - -Run: - -```bash -cd go/vehicle-gateway -go test ./internal/eventbus -``` - -Expected: pass. - ---- - -### Task 6: Implement TDengine History Writer - -**Files:** -- Create: `go/vehicle-gateway/internal/history/schema.go` -- Create: `go/vehicle-gateway/internal/history/writer.go` -- Create: `go/vehicle-gateway/internal/history/writer_test.go` -- Create: `go/vehicle-gateway/cmd/history-writer/main.go` - -- [ ] **Step 1: Add schema bootstrap SQL** - -Implement schema strings for: - -- database `lingniu_vehicle_ts` -- stable `raw_frames` -- stable `vehicle_locations` -- stable `vehicle_mileage_points` - -- [ ] **Step 2: Implement writer** - -Rules: - -- `AppendRawFrame` always writes one raw row. -- `AppendLocation` writes only when longitude and latitude exist. -- `AppendMileagePoint` writes only when `total_mileage_km` exists. -- child table name is deterministic hash of `protocol + vehicle_key`. -- escape tag values. - -- [ ] **Step 3: Use TDengine official driver** - -Import WebSocket driver in command: - -```go -import _ "github.com/taosdata/driver-go/v3/taosWS" -``` - -Default driver name: - -```text -TDENGINE_DRIVER=taosWS -``` - -Short-term compatibility: - -```text -TDENGINE_DRIVER=taosSql -``` - -Only use `taosSql` when ECS TDengine WebSocket is not available. - -- [ ] **Step 4: Verify** - -Run: - -```bash -cd go/vehicle-gateway -go test ./internal/history -go build ./cmd/history-writer -``` - -Expected: pass. - ---- - -### Task 7: Implement MySQL Daily Metric Writer - -**Files:** -- Create: `go/vehicle-gateway/internal/stats/schema.go` -- Create: `go/vehicle-gateway/internal/stats/daily_metric.go` -- Create: `go/vehicle-gateway/internal/stats/daily_metric_test.go` -- Create: `go/vehicle-gateway/cmd/stat-writer/main.go` - -- [ ] **Step 1: Add schema bootstrap** - -Implement `vehicle_daily_metric` schema from the design spec. - -- [ ] **Step 2: Add metric derivation tests** - -Test cases: - -- no `total_mileage_km` produces no metric -- one sample produces: - - `daily_mileage_km = 0` - - `daily_total_mileage_km = sample` -- later larger sample updates: - - `latest_total_mileage_km` - - `daily_mileage_km` - - `daily_total_mileage_km` -- out-of-order smaller sample updates: - - `first_total_mileage_km` - - `daily_mileage_km` - -- [ ] **Step 3: Implement MySQL upsert** - -Use one table and one idempotent upsert: - -```sql -INSERT INTO vehicle_daily_metric - (vin, stat_date, protocol, metric_key, metric_value, metric_unit, - first_total_mileage_km, latest_total_mileage_km, sample_count, calculation_method) -VALUES (?, ?, ?, ?, ?, 'km', ?, ?, 1, 'TOTAL_MILEAGE_DIFF') -ON DUPLICATE KEY UPDATE - first_total_mileage_km = LEAST(first_total_mileage_km, VALUES(first_total_mileage_km)), - latest_total_mileage_km = GREATEST(latest_total_mileage_km, VALUES(latest_total_mileage_km)), - metric_value = CASE - WHEN metric_key = 'daily_mileage_km' - THEN GREATEST(latest_total_mileage_km, VALUES(latest_total_mileage_km)) - - LEAST(first_total_mileage_km, VALUES(first_total_mileage_km)) - WHEN metric_key = 'daily_total_mileage_km' - THEN GREATEST(latest_total_mileage_km, VALUES(latest_total_mileage_km)) - ELSE VALUES(metric_value) - END, - sample_count = sample_count + 1, - updated_at = CURRENT_TIMESTAMP -``` - -- [ ] **Step 4: Verify** - -Run: - -```bash -cd go/vehicle-gateway -go test ./internal/stats -go build ./cmd/stat-writer -``` - -Expected: pass. - ---- - -### Task 8: Implement Redis Realtime API - -**Files:** -- Create: `go/vehicle-gateway/internal/realtime/repository.go` -- Create: `go/vehicle-gateway/internal/realtime/repository_test.go` -- Create: `go/vehicle-gateway/internal/realtime/http.go` -- Create: `go/vehicle-gateway/cmd/realtime-api/main.go` - -- [ ] **Step 1: Add repository contract** - -Methods: - -- `Update(ctx, envelope.FrameEnvelope) error` -- `GetMerged(ctx, vin string) (Snapshot, error)` -- `GetProtocol(ctx, vin string, protocol envelope.Protocol) (Snapshot, error)` -- `IsOnline(ctx, vin string) (OnlineStatus, error)` - -- [ ] **Step 2: Implement merge logic** - -Rules: - -- update `vehicle:latest:{vin}:{protocol}` -- update `vehicle:latest:{vin}` with newest fields -- update `vehicle:online:{vin}` -- update sorted set `vehicle:last_seen` - -- [ ] **Step 3: Implement HTTP API** - -Routes: - -- `GET /api/realtime/vehicles/{vin}` -- `GET /api/realtime/vehicles/{vin}/online` -- `GET /api/realtime/vehicles/{vin}/protocols/{protocol}` - -- [ ] **Step 4: Verify** - -Run: - -```bash -cd go/vehicle-gateway -go test ./internal/realtime -go build ./cmd/realtime-api -``` - -Expected: pass. - ---- - -### Task 9: Wire Gateway Runtime - -**Files:** -- Create: `go/vehicle-gateway/internal/gateway/tcp_server.go` -- Create: `go/vehicle-gateway/internal/gateway/mqtt_client.go` -- Modify: `go/vehicle-gateway/cmd/gateway/main.go` - -- [ ] **Step 1: Implement TCP server** - -Rules: - -- one goroutine per accepted connection -- bounded max connections -- read timeout and idle timeout -- protocol-specific frame extractor -- structured peer endpoint -- graceful shutdown on SIGTERM - -- [ ] **Step 2: Implement MQTT client** - -Rules: - -- connect with official production config from environment -- subscribe configured topic list -- convert each message into envelope -- publish raw and unified events -- reconnect with backoff - -- [ ] **Step 3: Verify local JSON mode** - -Run without Kafka: - -```bash -cd go/vehicle-gateway -GB32960_TCP_ADDR=:132960 JT808_TCP_ADDR=:18080 go run ./cmd/gateway -``` - -Expected: service starts and logs configured listeners. - ---- - -### Task 10: Docker and ECS Deployment - -**Files:** -- Create: `go/vehicle-gateway/Dockerfile` -- Create: `deploy/portainer/docker-compose-go.yml` -- Modify: `docs/operations/current-ecs-deployment.md` - -- [ ] **Step 1: Add multi-stage Dockerfile** - -Build all commands: - -- `gateway` -- `history-writer` -- `stat-writer` -- `realtime-api` - -- [ ] **Step 2: Add Portainer compose** - -Services: - -- `go-vehicle-gateway` -- `go-history-writer` -- `go-stat-writer` -- `go-realtime-api` - -Each service must include: - -- restart policy -- memory limit -- Kafka env -- MySQL/TDengine/Redis env as needed -- logging options - -- [ ] **Step 3: Verify Linux build** - -Run: - -```bash -cd go/vehicle-gateway -GOOS=linux GOARCH=amd64 go build ./cmd/gateway -GOOS=linux GOARCH=amd64 go build ./cmd/history-writer -GOOS=linux GOARCH=amd64 go build ./cmd/stat-writer -GOOS=linux GOARCH=amd64 go build ./cmd/realtime-api -``` - -Expected: all commands exit `0`. - ---- - -### Task 11: Production Verification - -**Files:** -- Create: `docs/operations/go-vehicle-gateway-verification.md` - -- [ ] **Step 1: Record test commands** - -Document commands to verify: - -- gateway process health -- Kafka topic consumption -- TDengine row counts -- MySQL daily metric rows -- Redis realtime lookup - -- [ ] **Step 2: Validate real traffic** - -Evidence required: - -- one real 32960 VIN with RAW, location, mileage point, daily metric, Redis snapshot -- one real JT808 phone/VIN with RAW, location, mileage point, daily metric, Redis snapshot -- one real Yutong MQTT VIN with RAW and Redis snapshot - -- [ ] **Step 3: Keep old Java services until evidence is captured** - -Only disable Java equivalents after the evidence file contains successful command outputs and timestamps. - ---- - -## Self-Review Checklist - -- The plan creates one new Go module instead of extending scattered prototypes. -- The plan covers GB32960, JT808, and Yutong MQTT ingress. -- The plan covers Kafka, TDengine, MySQL, and Redis. -- The plan includes 32960 and 808 daily mileage and daily total mileage. -- The plan includes local and ECS verification. -- The plan does not restore Xinda Push. -- The plan keeps Java services untouched until Go evidence exists. diff --git a/docs/superpowers/specs/2026-06-23-gb32960-production-readiness-design.md b/docs/superpowers/specs/2026-06-23-gb32960-production-readiness-design.md deleted file mode 100644 index 0f023d5f..00000000 --- a/docs/superpowers/specs/2026-06-23-gb32960-production-readiness-design.md +++ /dev/null @@ -1,481 +0,0 @@ -> **Superseded:** This 2026-06-23 DuckDB-based design is historical context. -> Use `docs/target-architecture.md` and -> `docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.md` for the -> current production architecture: TDengine is the default hot history store, -> `event-file-store` and Xinda Push have been removed from the production codebase. - -# GB32960 Production Readiness Design - -## Goal - -Make the GB32960 ingestion, history, and analytics services production-ready: - -- receive and parse GB32960 traffic quickly and accurately; -- maximize VIN-to-platform/vendor-profile association through platform login and local mappings; -- store complete RAW frames and queryable telemetry history with DuckDB; -- support full frame replay from persisted RAW bytes; -- compute real-time daily vehicle metrics and alarm timelines into MySQL; -- reduce service coupling and improve cohesion inside each module. - -## Non-Goals - -- Do not put database credentials in code, YAML files, tests, docs, or commits. -- Do not make MySQL the source for full telemetry history. MySQL is for derived daily metrics and alarm timelines. -- Do not require every private GB32960 extension to be perfect before storing data. RAW bytes must always be kept when a valid frame is received. - -## Production Architecture - -```mermaid -flowchart LR - tcp["GB32960 TCP:32960"] --> ingest["gb32960-ingest-app"] - mapping["local vin-platform-profile.jsonl"] --> ingest - ingest --> kafkaEvent["Kafka vehicle.event.gb32960.v1"] - ingest --> kafkaRaw["Kafka vehicle.raw.gb32960.v1"] - kafkaEvent --> history["vehicle-history-app"] - kafkaRaw --> history - history --> duckdb["DuckDB hot history"] - history --> raw["RAW frame archive"] - kafkaEvent --> analytics["vehicle-analytics-app"] - analytics --> mysql["MySQL daily stats + alarms"] -``` - -## Service Boundaries - -### 1. `gb32960-ingest-app` - -Responsibilities: - -- Netty TCP accept, frame splitting, GB32960 decode, auth, ACK, and Kafka production. -- Platform login handling and connection-local `platformAccount`. -- VIN-to-platform/vendor-profile resolution. -- Produce structured telemetry, session, alarm, diagnostics, and RAW envelope messages. - -It must not: - -- write DuckDB, MySQL, or local archive files; -- run historical queries; -- run daily statistics. - -ACK policy: - -- login/report-like GB32960 commands use durable ACK: ACK only after Kafka dispatch succeeds; -- malformed bytes that cannot form a valid frame are dropped with diagnostics; -- valid frames with partial private-block parse failures still go to Kafka with `parseStatus=PARTIAL`. - -VIN/profile resolution priority: - -1. active platform login on the current TCP channel; -2. exact VIN entry from local mapping file; -3. VIN prefix entry from local mapping file; -4. configured platform extension rule; -5. standard GB32960 profile. - -Local mapping file format: - -```jsonl -{"vin":"LNXNEGRR2SR321390","platformAccount":"Hyundai","vendorProfile":"guangdong-fc","enabled":true} -{"vinPrefix":"LNXNEGRR","platformAccount":"Hyundai","vendorProfile":"guangdong-fc","enabled":true} -``` - -The mapping file is loaded at startup and can be reloaded by an actuator endpoint or file watcher in a later phase. - -### 2. `vehicle-history-app` - -Responsibilities: - -- Consume Kafka event and RAW topics. -- Persist RAW bytes to an append-only archive. -- Persist query indexes and compact telemetry points to DuckDB. -- Serve high-performance query APIs: - - full frame list by `vin + eventTime`; - - telemetry fields by `vin + eventTime`; - - one-frame replay by `rawArchiveUri`; - - parse diagnostics and missing-profile troubleshooting. - -It must not: - -- listen on GB32960 TCP; -- run MySQL statistics; -- mutate daily business metrics. - -### 3. `vehicle-analytics-app` - -Responsibilities: - -- Consume Kafka event topic. -- Maintain real-time daily stats in MySQL. -- Maintain full alarm timelines in MySQL. -- Expose metrics query APIs and operational health. - -It must not: - -- parse RAW frames for normal operation; -- write DuckDB history; -- own GB32960 TCP connectivity. - -## Kafka Contract - -Keep Protobuf `VehicleEnvelope`, but make the producer contract stricter: - -- `eventId`: stable idempotency key, deterministic from protocol, VIN, command, event time, raw checksum, and sequence when available. -- `vin`: required for vehicle frames; platform-only commands use `_platform` or empty plus `platformAccount`. -- `eventTimeMs`: GB32960 data collection time when available. -- `ingestTimeMs`: service receive time. -- `metadata.platformAccount`: resolved platform account. -- `metadata.vendorProfile`: selected parser profile. -- `metadata.parseStatus`: `OK`, `PARTIAL`, or `FAILED`. -- `metadata.parseErrorCode`: stable code for failures. -- `raw_archive.data`: present on RAW topic only; event topic keeps `rawArchiveUri` pointer. -- `telemetry_snapshot`: standard field projection for downstream services. - -Consumer rule: - -- History consumes both event and RAW topics. -- Analytics consumes only event topic. -- Consumer offset is committed only after the target store commit succeeds. -- Reprocessing must be safe through deterministic ids and database upsert/ignore semantics. - -## DuckDB History Design - -Use DuckDB as the hot historical database, not a Parquet-rewrite sidecar. - -DuckDB official guidance says row-by-row insert loops are inefficient for bulk loading, and Appender/batched writes are the intended high-throughput path. Therefore history writes use a single writer worker with bounded batches and one DuckDB connection. - -### Tables - -`gb32960_frame_index` - -```sql -CREATE TABLE IF NOT EXISTS gb32960_frame_index ( - event_id VARCHAR PRIMARY KEY, - vin VARCHAR NOT NULL, - platform_account VARCHAR, - vendor_profile VARCHAR, - command VARCHAR NOT NULL, - command_code INTEGER NOT NULL, - event_time TIMESTAMP NOT NULL, - event_time_ms BIGINT NOT NULL, - ingest_time TIMESTAMP NOT NULL, - ingest_time_ms BIGINT NOT NULL, - partition_date DATE NOT NULL, - raw_archive_uri VARCHAR NOT NULL, - raw_checksum VARCHAR, - raw_size_bytes BIGINT NOT NULL, - parse_status VARCHAR NOT NULL, - parse_error_code VARCHAR, - block_count INTEGER NOT NULL, - metadata_json VARCHAR NOT NULL -); - -CREATE INDEX IF NOT EXISTS gb32960_frame_vin_time_idx - ON gb32960_frame_index(vin, event_time_ms); - -CREATE INDEX IF NOT EXISTS gb32960_frame_raw_uri_idx - ON gb32960_frame_index(raw_archive_uri); - -CREATE INDEX IF NOT EXISTS gb32960_frame_partition_time_idx - ON gb32960_frame_index(partition_date, event_time_ms); -``` - -`gb32960_telemetry_point` - -```sql -CREATE TABLE IF NOT EXISTS gb32960_telemetry_point ( - event_id VARCHAR NOT NULL, - vin VARCHAR NOT NULL, - event_time TIMESTAMP NOT NULL, - event_time_ms BIGINT NOT NULL, - ingest_time TIMESTAMP NOT NULL, - platform_account VARCHAR, - vendor_profile VARCHAR, - field_key VARCHAR NOT NULL, - value_type VARCHAR NOT NULL, - value_text VARCHAR NOT NULL, - value_num DOUBLE, - unit VARCHAR, - quality VARCHAR NOT NULL, - raw_archive_uri VARCHAR NOT NULL, - PRIMARY KEY(event_id, field_key) -); - -CREATE INDEX IF NOT EXISTS gb32960_point_vin_field_time_idx - ON gb32960_telemetry_point(vin, field_key, event_time_ms); -``` - -`gb32960_alarm_frame` - -```sql -CREATE TABLE IF NOT EXISTS gb32960_alarm_frame ( - event_id VARCHAR PRIMARY KEY, - vin VARCHAR NOT NULL, - event_time TIMESTAMP NOT NULL, - event_time_ms BIGINT NOT NULL, - level VARCHAR NOT NULL, - active_bits_json VARCHAR NOT NULL, - fault_codes_json VARCHAR NOT NULL, - raw_archive_uri VARCHAR NOT NULL -); - -CREATE INDEX IF NOT EXISTS gb32960_alarm_vin_time_idx - ON gb32960_alarm_frame(vin, event_time_ms); -``` - -### RAW Archive - -RAW archive remains file/object based: - -```text -archive://yyyy/MM/dd/GB32960/{vin}/{event_id}.bin -``` - -The archive writer verifies: - -- stored byte length equals envelope `raw_archive.size_bytes`; -- checksum matches; -- URI is unique for the deterministic event id. - -Frame replay path: - -1. query `gb32960_frame_index` by `rawArchiveUri`; -2. read RAW bytes from archive; -3. resolve `platformAccount/vendorProfile` from index metadata; -4. decode using current parser; -5. return decoded frame plus original indexed parse diagnostics. - -## Query APIs - -Keep existing endpoints but back them by the new DuckDB tables: - -- `GET /api/event-history/gb32960/frames` - - full frame query by VIN/time. -- `GET /api/event-history/gb32960/frame` - - one RAW frame replay by `rawArchiveUri`. -- `GET /api/event-history/gb32960/telemetry-snapshots` - - compact telemetry snapshot by VIN/time. -- `GET /api/event-history/gb32960/telemetry-fields` - - field projection optimized by `gb32960_telemetry_point`. -- `GET /api/event-history/gb32960/diagnostics` - - parse status and missing-profile investigation by VIN/time. - -Time semantics: - -- `eventTime` is business query time. -- `ingestTime` is kept for operational replay, late-arrival detection, and Kafka lag investigation. -- Dates without timezone are interpreted in `Asia/Shanghai`. - -## MySQL Analytics Design - -Credentials are supplied only by environment variables: - -```text -MYSQL_HOST -MYSQL_PORT -MYSQL_DATABASE -MYSQL_USERNAME -MYSQL_PASSWORD -``` - -Daily derived metrics are stored in the common metric table, not in a -protocol-specific daily-mileage table. - -```sql -CREATE TABLE IF NOT EXISTS vehicle_stat_metric ( - vin VARCHAR(64) NOT NULL, - stat_date DATE NOT NULL, - metric_key VARCHAR(64) NOT NULL, - metric_value DECIMAL(18,6) NULL, - metric_unit VARCHAR(16) NOT NULL, - calculation_method VARCHAR(64) NOT NULL, - created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, - updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, - PRIMARY KEY (vin, stat_date, metric_key) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; -``` - -`vehicle_alarm_timeline` - -```sql -CREATE TABLE IF NOT EXISTS vehicle_alarm_timeline ( - alarm_id VARCHAR(128) NOT NULL, - vin VARCHAR(32) NOT NULL, - alarm_key VARCHAR(128) NOT NULL, - level VARCHAR(32) NOT NULL, - start_time DATETIME(3) NOT NULL, - end_time DATETIME(3), - last_seen_time DATETIME(3) NOT NULL, - start_raw_archive_uri VARCHAR(512), - last_raw_archive_uri VARCHAR(512), - status VARCHAR(16) NOT NULL, - details_json JSON, - updated_at DATETIME(3) NOT NULL, - PRIMARY KEY (alarm_id), - KEY vehicle_alarm_vin_time_idx (vin, start_time), - KEY vehicle_alarm_open_idx (vin, status, alarm_key) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; -``` - -`vehicle_stat_event_dedup` - -```sql -CREATE TABLE IF NOT EXISTS vehicle_stat_event_dedup ( - event_id VARCHAR(128) NOT NULL, - consumer_group VARCHAR(128) NOT NULL, - processed_at DATETIME(3) NOT NULL, - PRIMARY KEY (event_id, consumer_group) -) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4; -``` - -Writes use `INSERT ... ON DUPLICATE KEY UPDATE` so replayed Kafka messages update the same daily row instead of creating duplicates. - -## Daily Metric Algorithms - -All daily statistics are keyed by `eventTime` in `Asia/Shanghai`. - -### Total Mileage - -Use the latest valid `totalMileageKm` seen for the day. - -Validation: - -- reject negative values; -- reject impossible jumps based on configurable max speed and elapsed time; -- keep the sample as diagnostic if rejected. - -### Daily Mileage - -JT808 daily mileage uses the GPS total mileage reported in location additional -information. The local-day minimum and maximum GPS total mileage values rewrite -the same metric row, so ordered streaming and historical replay use one -calculation path. - -```text -dailyMileage = today.maxTotalMileage - today.minTotalMileage -calculationMethod = JT808_TOTAL_MILEAGE_DIFF -``` - -The result is stored in `vehicle_stat_metric` with -`metric_key=daily_mileage_km`. There is no JT808-specific daily mileage table, -no Redis mileage state, and no previous-day baseline calculation. - -### Daily Power Consumption - -Preferred: - -- vendor cumulative power-consumption field if available. - -Fallback: - -```text -deltaKwh = sum(max(0, totalVoltageV * totalCurrentA) * deltaSeconds / 3600000) -``` - -The sign convention is configurable per vendor profile because GB32960 vehicle current may be positive or negative depending on charging/discharging representation. - -### Daily Hydrogen Consumption - -Use a tiered market-practical algorithm instead of a single fixed formula: - -1. `DIRECT_COUNTER`: vendor cumulative hydrogen consumption or remaining hydrogen mass delta. -2. `PVT_MASS_DELTA`: pressure/temperature/tank-volume mass estimate when storage data is available. -3. `FUEL_CELL_ENERGY_MODEL`: integrate fuel-cell electric output and divide by calibrated efficiency. -4. `UNKNOWN`: keep null when required signals are missing. - -The PVT approach follows the industry pattern behind compressed hydrogen consumption tests, where pressure, temperature, and tank volume can be used to estimate hydrogen mass. The implementation must expose calibration constants per vehicle model/profile: - -```json -{ - "vinPrefix": "LNXNEGRR", - "tankVolumeLiter": 140, - "hydrogenModel": "PVT_MASS_DELTA", - "fuelCellEfficiency": 0.52, - "currentDischargeSign": "POSITIVE" -} -``` - -### Daily Hydrogen Consumption Rate - -```text -dailyHydrogenKgPer100km = dailyHydrogenKg / dailyMileageKm * 100 -``` - -If mileage is missing or less than the configured minimum distance, keep the rate null and set quality to `INSUFFICIENT_DISTANCE`. - -### Alarm Timeline - -For each telemetry frame: - -- derive active alarm keys from alarm bits and fault code arrays; -- open a timeline row when a key first appears; -- update `last_seen_time` while it remains active; -- close open alarms when the key disappears after a configurable grace window; -- keep start/end raw archive URIs for replay. - -## Reliability and Observability - -Required metrics: - -- TCP active connections; -- frames received per command; -- decode failures by code; -- missing platform login rejections; -- vendor profile resolution source; -- Kafka send latency and failures; -- history write batch size, latency, failure count; -- DuckDB query latency by endpoint; -- MySQL upsert latency and failure count; -- consumer lag by group/topic/partition; -- late sample count by VIN/date. - -Required logs: - -- one structured line for every dropped or rejected valid frame; -- no per-frame INFO spam for successful reports after first frame per VIN/profile unless debug is enabled; -- include `vin`, `platformAccount`, `command`, `eventId`, `reasonCode`, and `peer`. - -## Migration Plan - -Phase 1: History foundation - -- Replace Parquet rewrite store with DuckDB hot tables and batch writer. -- Keep RAW bytes archive. -- Move existing history endpoints onto the new store. -- Add diagnostic endpoint for missing VIN/profile cases. - -Phase 2: Ingest association and diagnostics - -- Add local `vin-platform-profile.jsonl` loader. -- Use resolved profile before private block parsing. -- Add structured parse/reject diagnostics to Kafka metadata and logs. -- Add tests for platform login, mapping fallback, and partial private parsing. - -Phase 3: MySQL analytics - -- Add MySQL repository and migrations. -- Implement daily stat accumulator with idempotent upsert. -- Implement alarm timeline open/update/close logic. -- Add integration tests using Testcontainers or a local MySQL profile. - -Phase 4: Production hardening - -- Add backfill/replay commands from DuckDB RAW archive to analytics. -- Add retention/export from DuckDB to Parquet for cold storage. -- Add dashboards and operational runbook. - -## Acceptance Criteria - -- A valid GB32960 frame received on port 32960 appears in Kafka with RAW bytes or RAW URI. -- A valid frame is queryable by `vin + eventTime` from history. -- A returned frame can be replayed from `rawArchiveUri` and decoded with the stored profile. -- A VIN with only local mapping, and no current platform login, can still use the correct private parser profile. -- A private block parse failure does not drop the frame. -- Daily MySQL rows update in real time for mileage, total mileage, power consumption, hydrogen consumption, hydrogen rate, and sample metadata. -- Alarm timeline rows preserve full alarm start, update, and close history. -- Replaying the same Kafka event does not duplicate daily stats or alarm rows. -- No secret appears in committed files. - -## References - -- DuckDB Appender: https://duckdb.org/docs/current/data/appender -- DuckDB INSERT performance guidance: https://duckdb.org/docs/current/data/insert -- MySQL `INSERT ... ON DUPLICATE KEY UPDATE`: https://dev.mysql.com/doc/refman/9.7/en/insert-on-duplicate.html -- SAE J2572 overview: https://h2tools.org/fuel-cell-codes-and-standards/sae-j2572-measuring-exhaust-emissions-energy-consumption-and-range diff --git a/docs/superpowers/specs/2026-06-23-gb32960-service-split-design.md b/docs/superpowers/specs/2026-06-23-gb32960-service-split-design.md deleted file mode 100644 index a3f84303..00000000 --- a/docs/superpowers/specs/2026-06-23-gb32960-service-split-design.md +++ /dev/null @@ -1,411 +0,0 @@ -> **Superseded:** This 2026-06-23 split design is historical context. -> Use `docs/target-architecture.md` and -> `docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.md` for the -> current production architecture: TDengine is the default hot history store, -> `sink-archive` owns raw bytes, and `event-file-store` is optional -> compatibility only. - -# GB32960 Service Split Design - -Date: 2026-06-23 - -## Goal - -Split the current all-in-one vehicle ingest runtime into lower-coupled, higher-cohesion services. The first production slice focuses on GB/T 32960 because it is the active high-pressure path: persistent TCP connections, protocol ACKs, raw frame volume, history queries, and downstream analytics are currently assembled in one `bootstrap-all` process. - -The target outcome is: - -- Keep GB32960 ingest reliable and fast. -- Move disk writes and history queries out of the protocol process. -- Move statistics and analysis out of the protocol process. -- Use Kafka as the durable boundary between ingest, history, and analytics. -- Preserve `bootstrap-all` during migration as a local development and rollback entry point. - -## Decision - -Use ACK semantics A: - -> A GB32960 frame is ACKed successfully after the protocol service parses/accepts it and writes the required Kafka message(s) successfully. - -The protocol service must not wait for raw archive, DuckDB/Parquet writes, snapshot generation, or statistics computation before sending the GB32960 response frame. - -## Service Boundaries - -### `gb32960-ingest-app` - -Purpose: own GB32960 connection handling and Kafka production. - -Responsibilities: - -- Listen on the GB32960 TCP port. -- Handle platform login, VIN authorization, TLS if enabled, and session state required for protocol responses. -- Decode GB32960 frames and apply vendor extension parsing. -- Produce raw frame records and normalized event records to Kafka. -- ACK successful reports only after Kafka production succeeds. -- Publish malformed or rejected frames to a DLQ when possible. -- Expose operational health and metrics for connections, decode rate, Kafka latency, ACK latency, and DLQ count. - -Non-responsibilities: - -- No local raw archive writes. -- No DuckDB or Parquet history writes. -- No snapshot query API. -- No vehicle statistics or analysis. -- No long-running analytical calculations. - -Initial module dependencies: - -- `ingest-api` -- `ingest-core` -- `ingest-codec-common` -- `session-core` -- `protocol-gb32960` -- `sink-kafka` -- `observability` - -### `vehicle-history-app` - -Purpose: own raw storage, historical index storage, and history query APIs. - -Responsibilities: - -- Consume GB32960 raw records and normalized event records from Kafka. -- Write raw `.bin` payloads through `sink-archive`. -- Write queryable event indexes through `event-file-store`. -- Provide APIs for raw frame lookup, decoded frame lookup, history query, and snapshot/source-frame lookup. -- Rebuild or backfill the DuckDB sidecar index from stored Parquet files when needed. -- Track consumer lag, write latency, archive failures, index failures, and query latency. - -Non-responsibilities: - -- No TCP protocol listener. -- No GB32960 ACK decisions. -- No online statistics computation. -- No direct coupling to `gb32960-ingest-app` internals. - -Initial module dependencies: - -- `ingest-api` -- `sink-kafka` -- `sink-archive` -- `event-file-store` -- `event-history-service` -- `protocol-gb32960` only for raw-frame decode/query support -- `observability` - -### `vehicle-analytics-app` - -Purpose: own vehicle state, daily statistics, alarms, and later analytics. - -Responsibilities: - -- Consume normalized vehicle event records from Kafka. -- Maintain latest vehicle state if enabled. -- Compute daily vehicle statistics and alarm-derived metrics. -- Persist analytical outputs to the selected backend. -- Support reprocessing from Kafka offsets when analytical logic changes. -- Track consumer lag, per-VIN processing latency, aggregation failures, and output write latency. - -Non-responsibilities: - -- No protocol listener. -- No GB32960 ACK decisions. -- No raw archive writes. -- No normal-path dependency on raw archive or history APIs. - -Initial module dependencies: - -- `ingest-api` -- `sink-kafka` -- `vehicle-state-service` -- `vehicle-stat-service` -- `observability` - -## Kafka Contract - -Kafka is the service boundary. Messages must be stable enough that `vehicle-history-app` and `vehicle-analytics-app` do not depend on protocol service internals. - -### Topics - -Use versioned topics: - -- `vehicle.raw.gb32960.v1` -- `vehicle.event.gb32960.v1` -- `vehicle.dlq.gb32960.v1` - -The current topic names under `lingniu.ingest.sink.kafka.topics` can remain temporarily for compatibility, but the split apps should converge on versioned topic names. - -### Keys - -Use VIN as the Kafka key whenever a VIN is available. - -Benefits: - -- Preserves per-vehicle ordering within a partition. -- Lets history and analytics scale by partition. -- Keeps stateful analytics simpler. - -When VIN is unavailable, use a stable fallback key such as platform account plus peer address plus receive time bucket. - -### Raw Record - -Topic: `vehicle.raw.gb32960.v1` - -Minimum fields: - -- `schemaVersion` -- `protocol = GB32960` -- `eventId` -- `receiveTime` -- `eventTime` if decoded -- `peer` -- `platformAccount` -- `vin` if decoded -- `command` -- `protocolVersion` -- `rawBytes` -- `checksumStatus` -- `decodeStatus` -- `metadata` - -Consumers: - -- `vehicle-history-app` writes archive `.bin` files and raw-frame query indexes. -- Future replay tools can regenerate normalized events from raw records. - -### Normalized Event Record - -Topic: `vehicle.event.gb32960.v1` - -Minimum fields: - -- `schemaVersion` -- `protocol = GB32960` -- `eventId` -- `sourceRawEventId` -- `receiveTime` -- `eventTime` -- `platformAccount` -- `vin` -- `command` -- `eventType` -- `normalizedFields` -- `vendorFields` -- `alarmFields` -- `metadata` - -Consumers: - -- `vehicle-history-app` writes queryable history records. -- `vehicle-analytics-app` updates state and statistics. - -### DLQ Record - -Topic: `vehicle.dlq.gb32960.v1` - -Minimum fields: - -- `schemaVersion` -- `stage` -- `errorCode` -- `errorMessage` -- `receiveTime` -- `peer` -- `platformAccount` if known -- `vin` if known -- `rawBytes` when available -- `metadata` - -Typical stages: - -- `FRAME_DECODE` -- `BODY_PARSE` -- `AUTH` -- `KAFKA_PRODUCE` -- `ACK_WRITE` - -## ACK and Failure Semantics - -### Success Path - -1. Receive TCP bytes. -2. Decode a complete GB32960 frame. -3. Validate checksum/auth/session rules. -4. Build raw and normalized Kafka records. -5. Produce required Kafka records successfully. -6. Send GB32960 success ACK. - -### Kafka Produce Failure - -If required Kafka production fails, the service must not send a success ACK. - -Allowed behavior: - -- Retry within a bounded timeout. -- Return a GB32960 failure response when the protocol allows it. -- Close the channel after repeated produce failures. -- Emit local error logs and metrics. - -Do not silently ACK frames that have not crossed the Kafka durability boundary. - -### History or Analytics Failure - -Failures in `vehicle-history-app` or `vehicle-analytics-app` must not affect GB32960 ACKs. They are handled by Kafka offset retry, DLQ, operational alerting, and backfill/replay. - -## Current Coupling to Remove - -`bootstrap-all` currently assembles protocol modules, Kafka producer/consumer, archive sink, event file store, history API, vehicle state, vehicle statistics, command gateway, and multiple inbound protocols in one application. - -The split should remove these production couplings: - -- Protocol runtime directly containing archive and event-file-store writers. -- Protocol runtime directly containing statistics processors. -- History query API sharing the same process as TCP ingest. -- Kafka consumer workers living in the same all-in-one app as protocol listeners. -- Operational scaling tied to one JVM for ingest, storage, queries, and analytics. - -## Migration Plan - -### Phase 1: Add Split App Entrypoints - -Add new app modules: - -- `modules/apps/gb32960-ingest-app` -- `modules/apps/vehicle-history-app` -- `modules/apps/vehicle-analytics-app` - -Keep `modules/apps/bootstrap-all` unchanged as the fallback runtime. - -### Phase 2: GB32960 Ingest App - -Create a production profile for `gb32960-ingest-app`: - -- Enable `protocol-gb32960`. -- Enable Kafka producer. -- Disable local archive. -- Disable event-file-store. -- Disable history API. -- Disable vehicle state/stat services. -- Disable unrelated protocols. - -Acceptance: - -- GB32960 TCP port starts. -- Platform login works. -- Realtime report frames are parsed. -- Kafka records are produced with VIN keys. -- ACK is sent only after Kafka success. - -### Phase 3: History App - -Create `vehicle-history-app`: - -- Enable Kafka consumer. -- Enable archive sink. -- Enable event-file-store. -- Enable event-history HTTP API. -- Disable protocol listeners. -- Disable analytics processors. - -Acceptance: - -- Consumes `vehicle.raw.gb32960.v1`. -- Writes raw `.bin` files. -- Consumes `vehicle.event.gb32960.v1`. -- Writes Parquet/DuckDB indexes. -- Queries can locate raw frames and decoded snapshots. - -### Phase 4: Analytics App - -Create `vehicle-analytics-app`: - -- Enable Kafka consumer. -- Enable vehicle state if a state backend is configured. -- Enable vehicle statistics. -- Disable protocol listeners. -- Disable archive and event-file-store. - -Acceptance: - -- Consumes `vehicle.event.gb32960.v1`. -- Maintains expected per-VIN state/stat outputs. -- Can be restarted and resume from Kafka offsets. -- Does not affect ingest ACK latency. - -### Phase 5: Parallel Run and Cutover - -Run old and new paths in parallel for a bounded validation window. - -Compare: - -- Received frame count. -- Kafka produced record count. -- Raw archive file count. -- Unique VIN count. -- History query count and sample query correctness. -- Analytics output count and sample values. -- GB32960 ACK latency before and after split. - -Cut over only after counts and sample queries match within the agreed tolerance. - -## Operational Guidance - -Scale services independently: - -- Scale `gb32960-ingest-app` by connection count and Kafka produce latency. -- Scale `vehicle-history-app` by disk throughput, consumer lag, and query latency. -- Scale `vehicle-analytics-app` by consumer lag and computation latency. - -Monitor: - -- Kafka producer error rate and p99 produce latency. -- ACK p99 latency. -- TCP active connections. -- Decode failures and DLQ counts. -- Consumer lag per group. -- Archive write latency and failure rate. -- Event-file-store flush latency and failure rate. -- Analytics processing latency. - -## Risks and Mitigations - -Risk: Kafka outage blocks GB32960 ACKs. - -Mitigation: bounded producer retry, clear failure ACK/close behavior, Kafka cluster monitoring, and optional local emergency spool only as a later explicit design. - -Risk: Two Kafka topics for raw and normalized events can diverge. - -Mitigation: include `eventId` and `sourceRawEventId`, produce records transactionally if required, or publish a single envelope containing both raw and normalized sections in the first implementation if transaction support is not ready. - -Risk: History app reprocessing duplicates archive/index records. - -Mitigation: deterministic archive keys and idempotent index writes keyed by `eventId` or `rawArchiveUri`. - -Risk: Analytics logic changes require backfill. - -Mitigation: keep raw and normalized topics with sufficient retention; allow analytics consumers to reset offsets or run a backfill group. - -Risk: Query APIs accidentally depend on ingest internals. - -Mitigation: history APIs should depend on `event-file-store`, `sink-archive`, and decode libraries only, not `gb32960-ingest-app`. - -## Open Decisions - -These are intentionally left for implementation planning: - -- Whether Kafka production uses two independent sends or a transaction for raw + normalized records. -- Exact protobuf/JSON schema shape for `vehicle.raw.gb32960.v1` and `vehicle.event.gb32960.v1`. -- Whether latest snapshot state belongs only to `vehicle-analytics-app` or is also materialized by `vehicle-history-app` for query convenience. -- Initial Kafka partition count and retention duration. -- Whether command downlink stays with `command-gateway` or gets a separate command service later. - -## Completion Criteria - -The split is complete when: - -- `gb32960-ingest-app`, `vehicle-history-app`, and `vehicle-analytics-app` are separate runnable app modules. -- `gb32960-ingest-app` can receive live GB32960 data and ACK after Kafka success. -- `vehicle-history-app` can consume Kafka and provide raw/history/snapshot query APIs. -- `vehicle-analytics-app` can consume Kafka and compute state/stat outputs. -- `bootstrap-all` remains available for development and rollback. -- Verification covers build, app startup, Kafka production/consumption, raw archive writes, history queries, analytics outputs, and ACK latency. diff --git a/docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.md b/docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.md deleted file mode 100644 index 531c4f37..00000000 --- a/docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.md +++ /dev/null @@ -1,621 +0,0 @@ -# Vehicle Ingest Redesign Design - -## Background - -Current GB32960 and JT808 ingestion can receive, decode, publish, and query data, but the responsibilities are split across several partially overlapping paths: - -- Netty handlers produce `RawFrame` and the dispatcher emits both raw archive events and normalized vehicle events. -- Kafka envelopes carry raw archive references, while raw bytes are stored separately by the archive sink. -- GB32960 historical queries use raw archive records plus file replay decoding. -- JT808 location history is materialized separately into TDengine. -- DuckDB/Parquet, local raw archive, Kafka, and TDengine each own part of the story, so query and durability semantics are hard to reason about. - -The new design intentionally keeps useful protocol parsers and app deployment knowledge, but replaces the ingestion, durability, storage, and query boundaries with a first-principles model. - -## Goals - -1. Store every original inbound protocol frame, including malformed frames when bytes are available. -2. Query historical uploaded data for GB32960 and JT808 by vehicle, protocol, message type, and time range. -3. Extend to new fields, protocol messages, and vendor profiles without rewriting the core pipeline. -4. Sustain production load for 10,000 vehicles and 1,000 concurrent historical query requests per second. -5. Make ACK and durability rules explicit enough that an acknowledged frame is not silently lost. - -## Non-Goals - -- Do not make Kafka the long-term historical query database. -- Do not store large raw byte arrays directly in Kafka topics. -- Do not keep DuckDB/Parquet as the production hot query store for GB32960 and JT808. -- Do not build one generic API that hides protocol-specific semantics. Shared internals are preferred; external APIs may remain protocol-aware. - -## Recommended Architecture - -Use `Raw Archive + Kafka + TDengine` as the core architecture. - -```mermaid -flowchart LR - GB["GB32960 TCP"] --> Edge["Protocol Edge"] - JT["JT808 TCP"] --> Edge - Edge --> RawStore["Raw Archive"] - RawStore --> RawTopic["Kafka vehicle.raw-frame.v1"] - Edge --> Decode["Protocol Decoder"] - Decode --> FactTopic["Kafka vehicle.decoded-fact.v1"] - RawTopic --> Writer["History Writer"] - FactTopic --> Writer - Writer --> TD["TDengine"] - Writer --> Redis["Realtime State"] - API["History API"] --> TD - API --> RawStore - RealtimeAPI["Realtime API"] --> Redis -``` - -The architecture has three facts: - -- `RawFrameFact`: one row per inbound frame, whether parsing succeeds or fails. -- `DecodedFact`: zero or more normalized facts derived from a raw frame. -- `RawArchiveObject`: immutable raw bytes addressed by `raw_uri`. - -The raw archive is the forensic source of truth. TDengine is the query source of truth. Kafka is the decoupling and replay pipe. - -## Module Boundaries - -### Protocol Edge Apps - -Apps: - -- `gb32960-ingest-app` -- `jt808-ingest-app` - -Responsibilities: - -- Accept TCP connections. -- Split frames. -- Perform protocol-level validation, authentication, session binding, and ACK. -- Create `RawFrameFact`. -- Persist raw bytes through `RawArchiveWriter`. -- Publish raw frame fact to Kafka. -- Decode frames and publish decoded facts. - -They do not query history, export data, or write business tables directly. - -### Ingest Core - -New module: - -- `modules/core/ingest-facts` - -Core types: - -```java -public record RawFrameFact( - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - int messageId, - int subType, - Instant eventTime, - Instant receivedAt, - String peer, - String rawUri, - String checksum, - long rawSizeBytes, - ParseStatus parseStatus, - String parseError, - Map metadata -) {} -``` - -```java -public sealed interface DecodedFact permits - DecodedFact.Location, - DecodedFact.Realtime, - DecodedFact.Alarm, - DecodedFact.Session, - DecodedFact.Register, - DecodedFact.Extension { - - String factId(); - String frameId(); - ProtocolId protocol(); - String vehicleKey(); - String vin(); - Instant eventTime(); - Instant receivedAt(); - String rawUri(); - Map metadata(); -} -``` - -`vehicleKey` is the stable partition key: - -- GB32960: VIN. -- JT808: resolved VIN when known, otherwise `jt808:`. -- Unknown/malformed: `unknown::`. - -### Raw Archive - -New or refactored module: - -- `modules/sinks/raw-archive-store` - -Interface: - -```java -public interface RawArchiveWriter { - RawArchiveReceipt write(RawArchiveWriteRequest request) throws IOException; -} -``` - -Receipt: - -```java -public record RawArchiveReceipt( - String rawUri, - String checksum, - long sizeBytes -) {} -``` - -Rules: - -- Raw bytes are immutable once written. -- Write path is content-addressed or deterministic by `protocol/date/vehicleKey/frameId.bin`. -- Write must be fsync-capable for local storage and pluggable for object storage. -- `rawUri` must be resolvable by history API for single-frame replay. -- Malformed frames are written with `parseStatus=FAILED` when bytes are available. - -Deployment constraints: - -- Raw archive, TDengine, and the history service may run on different ECS instances, but should use private-network access in the same VPC and availability zone. -- High-frequency historical queries only read TDengine and must not read raw archive. -- Raw archive is only used by single-frame detail, troubleshooting, replay, and asynchronous export paths. -- `rawUri` must be globally resolvable by services and must not be a private local absolute path on one ECS instance. -- The first local raw archive implementation must stay behind a replaceable interface so it can later move to OSS, MinIO, or another object store. - -### Kafka Topics - -Use stable versioned topics: - -- `vehicle.raw-frame.v1` -- `vehicle.decoded-fact.v1` -- `vehicle.dlq.v1` - -Kafka key: - -```text -: -``` - -`vehicle.raw-frame.v1` payload contains `RawFrameFact` without raw bytes. It carries `rawUri`, checksum, byte size, message id, parse status, and metadata. - -`vehicle.decoded-fact.v1` payload contains one `DecodedFact`. It always references `frameId` and `rawUri`. - -`vehicle.dlq.v1` carries failed store, publish, decode, and write attempts with enough metadata to replay from raw archive when possible. - -### History Writer - -New app or refactored app: - -- `vehicle-history-app` - -Responsibilities: - -- Consume `vehicle.raw-frame.v1`. -- Consume `vehicle.decoded-fact.v1`. -- Batch write TDengine. -- Update realtime state store for latest vehicle state. -- Write DLQ on invalid payloads or storage failures. - -The writer is idempotent by `(frame_id)` for raw facts and `(fact_id)` for decoded facts. - -## TDengine Model - -TDengine is the production historical query store. - -### Raw Frames Super Table - -```sql -CREATE STABLE IF NOT EXISTS raw_frames ( - ts TIMESTAMP, - frame_id NCHAR(64), - received_at TIMESTAMP, - message_id INT, - sub_type INT, - event_time TIMESTAMP, - raw_uri NCHAR(512), - checksum NCHAR(128), - raw_size_bytes BIGINT, - parse_status NCHAR(16), - parse_error NCHAR(512), - peer NCHAR(128), - metadata_json NCHAR(4096) -) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) -); -``` - -Child table: - -```text -raw__ -``` - -Primary query patterns: - -- Raw frames by vehicle and time. -- Raw frames by protocol, message id, parse status, and time. -- Single raw frame by `frame_id` or `raw_uri`. - -### Location Super Table - -```sql -CREATE STABLE IF NOT EXISTS vehicle_locations ( - ts TIMESTAMP, - fact_id NCHAR(64), - frame_id NCHAR(64), - received_at TIMESTAMP, - longitude DOUBLE, - latitude DOUBLE, - altitude_m DOUBLE, - speed_kmh DOUBLE, - direction_deg DOUBLE, - alarm_flag BIGINT, - status_flag BIGINT, - total_mileage_km DOUBLE, - raw_uri NCHAR(512), - metadata_json NCHAR(4096) -) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) -); -``` - -### Realtime Super Table - -```sql -CREATE STABLE IF NOT EXISTS vehicle_realtime ( - ts TIMESTAMP, - fact_id NCHAR(64), - frame_id NCHAR(64), - received_at TIMESTAMP, - speed_kmh DOUBLE, - total_mileage_km DOUBLE, - battery_soc DOUBLE, - total_voltage_v DOUBLE, - total_current_a DOUBLE, - running_mode NCHAR(64), - vehicle_state NCHAR(64), - charging_state NCHAR(64), - raw_uri NCHAR(512), - fields_json NCHAR(8192), - metadata_json NCHAR(4096) -) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) -); -``` - -`fields_json` stores sparse full-field values for low-frequency inspection. High-frequency queried fields should be promoted to explicit columns or a dedicated extension table. - -### Alarm Super Table - -```sql -CREATE STABLE IF NOT EXISTS vehicle_alarms ( - ts TIMESTAMP, - fact_id NCHAR(64), - frame_id NCHAR(64), - received_at TIMESTAMP, - alarm_level NCHAR(32), - alarm_code BIGINT, - alarm_name NCHAR(128), - longitude DOUBLE, - latitude DOUBLE, - raw_uri NCHAR(512), - fields_json NCHAR(8192), - metadata_json NCHAR(4096) -) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) -); -``` - -### Session and Registration Tables - -```sql -CREATE STABLE IF NOT EXISTS vehicle_sessions ( - ts TIMESTAMP, - fact_id NCHAR(64), - frame_id NCHAR(64), - received_at TIMESTAMP, - session_type NCHAR(32), - result_code INT, - raw_uri NCHAR(512), - fields_json NCHAR(8192), - metadata_json NCHAR(4096) -) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) -); -``` - -JT808 registration details go into `vehicle_sessions` first and may later be projected into MySQL identity bindings for durable identity lookup. - -## ACK and Durability Rules - -For frames that require a protocol ACK: - -1. Frame bytes are accepted from Netty. -2. Raw bytes are written to raw archive. -3. `RawFrameFact` is published to Kafka. -4. Protocol-specific ACK is sent. -5. Decoding and `DecodedFact` publication may run before or after ACK, but failures must emit a DLQ event and update raw frame parse status. - -For GB32960 realtime/report frames, this prevents acknowledging a frame whose original bytes cannot be recovered. - -For JT808 register/auth/heartbeat, ACK follows the same raw archive and raw fact boundary. Register ACK can still include token generation before raw fact publish, but must not be flushed to the channel until the durability boundary succeeds. - -If raw archive write fails, close or keep the channel according to protocol tolerance, but do not ACK success. - -If Kafka raw fact publish fails, do not ACK success for durable-ACK frames. Non-durable frames go to local DLQ when configured. - -## Protocol Decoding and Extension - -Use a plugin contract: - -```java -public interface ProtocolFrameDecoder { - ProtocolId protocol(); - DecodeResult decode(RawFrameFact rawFact, byte[] rawBytes); -} -``` - -```java -public record DecodeResult( - ParseStatus status, - List facts, - String errorMessage, - Map metadata -) {} -``` - -GB32960: - -- Frame command maps to session, realtime, location, alarm, and extension facts. -- Vendor extensions are selected by platform account, VIN, or configured profile. -- Full raw replay remains possible by reading `rawUri` and running the current decoder. - -JT808: - -- Message id maps to register, auth, heartbeat, location, batch location, media metadata, and extension facts. -- Identity resolution updates `vehicleKey`. -- Registration facts carry province, city, maker, device id, plate, plate color, and auth token metadata. - -New protocol messages do not require TDengine schema changes unless they become high-frequency query dimensions. Unknown fields are stored as extension facts with `fields_json`. - -## Query APIs - -Keep API surfaces explicit. - -Raw frame APIs: - -- `GET /api/history/raw-frames` -- Filters: `protocol`, `vin`, `phone`, `vehicleKey`, `messageId`, `parseStatus`, `dateFrom`, `dateTo`, `pageSize`, `cursor`. -- Returns frame metadata, raw URI, checksum, parse status, and compact decoded summary when available. - -Single raw frame API: - -- `GET /api/history/raw-frames/{frameId}` -- Returns raw metadata and decoded detail. -- Supports `includeRawHex=true` with a strict size limit. - -Location APIs: - -- `GET /api/history/locations` -- Filters: `protocol`, `vin`, `phone`, `vehicleKey`, time range, cursor. - -Realtime APIs: - -- `GET /api/history/realtime` -- Historical time-series query. - -Realtime state APIs: - -- `GET /api/realtime/vehicles/{vehicleKey}` -- Reads Redis latest state, not TDengine. - -Export: - -- Export is asynchronous. -- Request creates an export job. -- Worker streams TDengine query results to object storage or local export files. -- API returns job status and download URI. - -## Pagination - -Use cursor pagination, not deep offset, for high-QPS history queries. - -Cursor fields: - -```text -ts, received_at, fact_id -``` - -Descending query predicate: - -```sql -WHERE ( - ts < :cursorTs - OR (ts = :cursorTs AND received_at < :cursorReceivedAt) - OR (ts = :cursorTs AND received_at = :cursorReceivedAt AND fact_id < :cursorFactId) -) -``` - -Page size defaults to 100 and is capped at 1000. - -## Capacity Design - -Assumptions: - -- 10,000 vehicles. -- Normal report interval: 5 to 10 seconds. -- Expected writes: 1,000 to 2,000 frames per second. -- Peak burst factor: 3x. -- Historical query target: 1,000 requests per second. - -Write path: - -- Netty worker threads stay CPU-light. -- Raw archive writes use bounded async workers. -- Kafka producer uses compression, batching, idempotence, and vehicle-key partitioning. -- History writer batches TDengine inserts by super table and child table. -- TDengine child tables are partitioned by vehicle key to make single-vehicle queries cheap. - -Read path: - -- Most high-QPS queries require vehicle key and time range. -- Unbounded cross-vehicle queries are admin-only and capped. -- Realtime latest state is served from Redis latest-state cache. -- Raw detail replay reads one raw object and decodes on demand. -- Common dictionary and metadata responses are cached in memory. - -Backpressure: - -- Raw archive worker queue has a fixed bound. -- Kafka publish futures are tracked. -- If durability boundary cannot complete within timeout, durable ACK fails and the channel is closed or throttled. -- History writer lag is observable by Kafka consumer lag and TDengine batch latency. - -## Migration Plan - -Phase 1: Build new fact model beside existing code. - -- Add `ingest-facts`. -- Add raw archive receipt contract. -- Add serializers for raw frame and decoded fact Kafka messages. -- Add tests for stable ids, vehicle key, and raw URI generation. - -Phase 2: Harden raw durability boundary. - -- Refactor GB32960 and JT808 handlers so ACK waits for raw archive and raw fact publish. -- Keep existing parsers. -- Emit decoded facts to the new Kafka topic. - -Phase 3: Build TDengine writer. - -- Create TDengine schema manager. -- Write raw frame facts. -- Write location, realtime, alarm, session facts. -- Add idempotent insert behavior by stable ids. - -Phase 4: Build query APIs. - -- Raw frame query. -- Single frame replay. -- Location history query. -- Realtime history query. -- Latest realtime state query. - -Phase 5: Cut over and retire old hot paths. - -- Stop using DuckDB/Parquet as production hot history for GB32960/JT808. -- Keep raw replay compatibility while historical data migrates. -- Remove protocol-specific ad hoc history storage once TDengine coverage is verified. - -## Testing Strategy - -Unit tests: - -- `RawFrameFact` validation and vehicle key derivation. -- Raw archive path generation and checksum. -- GB32960 decoder facts from golden frames. -- JT808 decoder facts from sample frames. -- ACK boundary success and failure cases. -- TDengine SQL generation. - -Integration tests: - -- Start app with embedded or mocked Kafka producer and fake raw archive. -- Send GB32960 sample frame and verify raw fact, decoded fact, raw archive receipt. -- Send JT808 register/auth/location frames and verify registration/session/location facts. -- Simulate raw archive failure and verify no success ACK. -- Simulate Kafka publish failure and verify durable ACK failure. - -Storage tests: - -- TDengine schema initialization. -- Batch insert raw frames and decoded facts. -- Query by vehicle key and time range. -- Cursor pagination correctness. -- Single raw frame replay by `rawUri`. - -Load tests: - -- 2,000 frames/s sustained write for at least 30 minutes. -- 6,000 frames/s burst for 5 minutes. -- 1,000 QPS location/realtime history queries with bounded p95 latency. -- Verify no raw frame count gap between raw archive receipts, Kafka raw facts, and TDengine raw frame rows. - -## Operational Requirements - -Metrics: - -- TCP connections. -- Frame receive rate. -- Raw archive write latency and failure count. -- Kafka publish latency and failure count. -- Decoder success/failure count by protocol and message id. -- TDengine batch size, latency, and failure count. -- Query QPS and latency by endpoint. -- Kafka consumer lag. - -Logs: - -- One structured log per failed frame with `frameId`, protocol, vehicle key, message id, raw URI, and error. -- No full raw bytes in normal logs. -- Raw hex only in explicit debug tools with size limits. - -Alerts: - -- Raw archive write failure. -- Durable ACK timeout. -- Kafka producer failure. -- History writer lag. -- TDengine write failure. -- Raw archive count and TDengine raw frame row count divergence. - -## Acceptance Criteria - -The redesign is complete only when all of the following are true: - -1. Every accepted GB32960 and JT808 frame creates a raw archive object and a TDengine `raw_frames` row. -2. Malformed frames with bytes are queryable from raw history with parse status and error reason. -3. GB32960 and JT808 location history can be queried by vehicle and time range with cursor pagination. -4. GB32960 realtime history can be queried by vehicle and time range. -5. JT808 registration/session data is queryable and can update identity binding. -6. Single-frame detail can reload raw bytes by `rawUri` and decode with the current decoder. -7. ACK-required frames do not receive success ACK before raw archive and raw fact publication succeed. -8. New protocol fields can be added by implementing a decoder/projector and, only when needed, a TDengine table migration. -9. Load tests demonstrate sustained 2,000 frames/s writes and 1,000 QPS bounded historical queries. -10. Old DuckDB/Parquet hot-history paths are no longer required for GB32960/JT808 production queries. - -## Open Decisions - -The following choices are intentionally fixed for the first implementation: - -- Use local raw archive first, with the interface shaped for object storage later. -- Use TDengine as the only production hot historical query store for GB32960/JT808. -- Use explicit protocol-aware query APIs instead of a single generic query API. -- Use cursor pagination for all high-QPS historical queries. -- Keep Kafka payloads byte-light by sending raw references instead of raw bytes. diff --git a/docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.zh.md b/docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.zh.md deleted file mode 100644 index b3b457a7..00000000 --- a/docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.zh.md +++ /dev/null @@ -1,624 +0,0 @@ -# 车辆接入重设计方案 - -## 背景 - -当前 GB32960 和 JT808 链路已经能够完成接收、解析、发布和查询，但职责边界被拆散在多条路径里： - -- Netty Handler 生成 `RawFrame`，Dispatcher 同时发出原始归档事件和标准化车辆事件。 -- Kafka Envelope 只携带 raw archive 引用，原始 bytes 由 archive sink 另行存储。 -- GB32960 历史查询依赖 raw archive 记录，再回读文件重新解析。 -- JT808 位置历史又单独物化到 TDengine。 -- DuckDB/Parquet、本地 raw archive、Kafka、TDengine 分别承担一部分职责，导致查询语义和持久化边界不好推理。 - -新设计保留已有协议解析器和部署经验，但重新定义接入、持久化、存储和查询边界。 - -## 目标 - -1. 保存每一条入站协议原始帧，包括可拿到 bytes 的异常帧。 -2. 支持按车辆、协议、消息类型、时间范围查询 GB32960 和 JT808 历史上报数据。 -3. 新增字段、协议消息、厂商扩展时，不需要重写核心管线。 -4. 支撑 10,000 辆车生产写入，以及 1,000 QPS 历史查询。 -5. 明确 ACK 和持久化规则，避免“已经 ACK 但数据悄悄丢失”。 - -## 非目标 - -- 不把 Kafka 作为长期历史查询数据库。 -- 不把大块原始 bytes 直接塞进 Kafka topic。 -- 不继续把 DuckDB/Parquet 作为 GB32960/JT808 的生产热查询库。 -- 不做一个掩盖协议差异的万能接口。内部可以统一，外部 API 保持协议语义清晰。 - -## 推荐架构 - -采用 `Raw Archive + Kafka + TDengine` 作为核心架构。 - -```mermaid -flowchart LR - GB["GB32960 TCP"] --> Edge["协议接入边界"] - JT["JT808 TCP"] --> Edge - Edge --> RawStore["Raw Archive"] - RawStore --> RawTopic["Kafka vehicle.raw-frame.v1"] - Edge --> Decode["协议解析器"] - Decode --> FactTopic["Kafka vehicle.decoded-fact.v1"] - RawTopic --> Writer["History Writer"] - FactTopic --> Writer - Writer --> TD["TDengine"] - Writer --> Redis["实时状态"] - API["历史 API"] --> TD - API --> RawStore - RealtimeAPI["实时 API"] --> Redis -``` - -架构里只有三类事实： - -- `RawFrameFact`：每一条入站帧一条记录，不管解析成功还是失败。 -- `DecodedFact`：从 raw frame 派生出来的零条或多条标准事实。 -- `RawArchiveObject`：不可变原始 bytes，通过 `raw_uri` 寻址。 - -raw archive 是排障和回放的事实源。TDengine 是查询事实源。Kafka 是解耦和重放管道。 - -## 模块边界 - -### 协议接入 APP - -APP： - -- `gb32960-ingest-app` -- `jt808-ingest-app` - -职责： - -- 接收 TCP 连接。 -- 完成帧切分。 -- 执行协议层校验、鉴权、会话绑定和 ACK。 -- 创建 `RawFrameFact`。 -- 通过 `RawArchiveWriter` 持久化原始 bytes。 -- 发布 raw frame fact 到 Kafka。 -- 解析协议帧并发布 decoded facts。 - -接入 APP 不负责历史查询、导出，也不直接写业务查询表。 - -### Ingest Core - -新增模块： - -- `modules/core/ingest-facts` - -核心类型： - -```java -public record RawFrameFact( - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - int messageId, - int subType, - Instant eventTime, - Instant receivedAt, - String peer, - String rawUri, - String checksum, - long rawSizeBytes, - ParseStatus parseStatus, - String parseError, - Map metadata -) {} -``` - -```java -public sealed interface DecodedFact permits - DecodedFact.Location, - DecodedFact.Realtime, - DecodedFact.Alarm, - DecodedFact.Session, - DecodedFact.Register, - DecodedFact.Extension { - - String factId(); - String frameId(); - ProtocolId protocol(); - String vehicleKey(); - String vin(); - Instant eventTime(); - Instant receivedAt(); - String rawUri(); - Map metadata(); -} -``` - -`vehicleKey` 是稳定分区键： - -- GB32960：VIN。 -- JT808：已解析到 VIN 时使用 VIN，否则使用 `jt808:`。 -- 未知或异常帧：使用 `unknown::`。 - -### Raw Archive - -新增或重构模块： - -- `modules/sinks/raw-archive-store` - -接口： - -```java -public interface RawArchiveWriter { - RawArchiveReceipt write(RawArchiveWriteRequest request) throws IOException; -} -``` - -写入回执： - -```java -public record RawArchiveReceipt( - String rawUri, - String checksum, - long sizeBytes -) {} -``` - -规则： - -- 原始 bytes 一旦写入，不允许原地修改。 -- 写入路径可以按内容寻址，也可以按 `protocol/date/vehicleKey/frameId.bin` 生成。 -- 本地存储需要支持 fsync 能力，接口要预留对象存储实现。 -- `rawUri` 必须能被历史 API 解析，用于单帧回放。 -- 异常帧只要有 bytes，也必须写入，并记录 `parseStatus=FAILED`。 - -部署约束： - -- Raw archive、TDengine、history 服务可以不在同一台 ECS，但应在同一 VPC 和同一可用区内通过内网访问。 -- 高频历史查询只访问 TDengine，不访问 raw archive。 -- Raw archive 只进入单帧详情、问题排查、回放和异步导出路径。 -- `rawUri` 必须是全局可解析地址，不能是某台 ECS 私有的本地绝对路径。 -- 第一版本地 raw archive 需要通过可替换接口封装，后续可以切换到 OSS、MinIO 或其它对象存储。 - -### Kafka Topic - -使用稳定的版本化 topic： - -- `vehicle.raw-frame.v1` -- `vehicle.decoded-fact.v1` -- `vehicle.dlq.v1` - -Kafka key： - -```text -: -``` - -`vehicle.raw-frame.v1` 的 payload 是不含 raw bytes 的 `RawFrameFact`，携带 `rawUri`、checksum、byte size、message id、parse status 和 metadata。 - -`vehicle.decoded-fact.v1` 的 payload 是单条 `DecodedFact`，必须引用 `frameId` 和 `rawUri`。 - -`vehicle.dlq.v1` 保存存储失败、发布失败、解析失败和写入失败事件，并携带足够元数据，尽量支持从 raw archive 重放。 - -### History Writer - -新增或重构 APP： - -- `vehicle-history-app` - -职责： - -- 消费 `vehicle.raw-frame.v1`。 -- 消费 `vehicle.decoded-fact.v1`。 -- 批量写 TDengine。 -- 更新实时状态存储。 -- 对非法 payload 或存储失败写 DLQ。 - -Writer 写入需要幂等： - -- raw fact 以 `(frame_id)` 幂等。 -- decoded fact 以 `(fact_id)` 幂等。 - -## TDengine 模型 - -TDengine 是生产历史查询主库。 - -### 原始帧超级表 - -```sql -CREATE STABLE IF NOT EXISTS raw_frames ( - ts TIMESTAMP, - frame_id NCHAR(64), - received_at TIMESTAMP, - message_id INT, - sub_type INT, - event_time TIMESTAMP, - raw_uri NCHAR(512), - checksum NCHAR(128), - raw_size_bytes BIGINT, - parse_status NCHAR(16), - parse_error NCHAR(512), - peer NCHAR(128), - metadata_json NCHAR(4096) -) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) -); -``` - -子表命名： - -```text -raw__ -``` - -主要查询模式： - -- 按车辆和时间查询原始帧。 -- 按协议、消息 ID、解析状态和时间查询原始帧。 -- 按 `frame_id` 或 `raw_uri` 查询单帧。 - -### 位置超级表 - -```sql -CREATE STABLE IF NOT EXISTS vehicle_locations ( - ts TIMESTAMP, - fact_id NCHAR(64), - frame_id NCHAR(64), - received_at TIMESTAMP, - longitude DOUBLE, - latitude DOUBLE, - altitude_m DOUBLE, - speed_kmh DOUBLE, - direction_deg DOUBLE, - alarm_flag BIGINT, - status_flag BIGINT, - total_mileage_km DOUBLE, - raw_uri NCHAR(512), - metadata_json NCHAR(4096) -) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) -); -``` - -### 实时数据超级表 - -```sql -CREATE STABLE IF NOT EXISTS vehicle_realtime ( - ts TIMESTAMP, - fact_id NCHAR(64), - frame_id NCHAR(64), - received_at TIMESTAMP, - speed_kmh DOUBLE, - total_mileage_km DOUBLE, - battery_soc DOUBLE, - total_voltage_v DOUBLE, - total_current_a DOUBLE, - running_mode NCHAR(64), - vehicle_state NCHAR(64), - charging_state NCHAR(64), - raw_uri NCHAR(512), - fields_json NCHAR(8192), - metadata_json NCHAR(4096) -) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) -); -``` - -`fields_json` 存放稀疏全字段值，用于低频排查。高频查询字段应提升为明确列，或放入专用扩展表。 - -### 报警超级表 - -```sql -CREATE STABLE IF NOT EXISTS vehicle_alarms ( - ts TIMESTAMP, - fact_id NCHAR(64), - frame_id NCHAR(64), - received_at TIMESTAMP, - alarm_level NCHAR(32), - alarm_code BIGINT, - alarm_name NCHAR(128), - longitude DOUBLE, - latitude DOUBLE, - raw_uri NCHAR(512), - fields_json NCHAR(8192), - metadata_json NCHAR(4096) -) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) -); -``` - -### 会话和注册表 - -```sql -CREATE STABLE IF NOT EXISTS vehicle_sessions ( - ts TIMESTAMP, - fact_id NCHAR(64), - frame_id NCHAR(64), - received_at TIMESTAMP, - session_type NCHAR(32), - result_code INT, - raw_uri NCHAR(512), - fields_json NCHAR(8192), - metadata_json NCHAR(4096) -) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) -); -``` - -JT808 注册详情先进入 `vehicle_sessions`，后续可以投影到 MySQL 车辆身份绑定表，用于持久身份解析。 - -## ACK 和持久化规则 - -对需要协议 ACK 的帧： - -1. Netty 接收到 frame bytes。 -2. 原始 bytes 写入 raw archive。 -3. `RawFrameFact` 发布到 Kafka。 -4. 发送协议 ACK。 -5. 解码和 `DecodedFact` 发布可以在 ACK 前或 ACK 后执行，但失败必须发 DLQ，并更新 raw frame 解析状态。 - -对 GB32960 实时上报和补发帧，这条规则保证不会 ACK 一条无法恢复原始 bytes 的帧。 - -对 JT808 注册、鉴权、心跳，也遵循相同 raw archive 和 raw fact 边界。注册 ACK 可以先生成 token，但在 durability boundary 成功前不能 flush 到 channel。 - -如果 raw archive 写入失败，不返回成功 ACK。具体是关闭连接还是保留连接，由协议容忍度决定。 - -如果 Kafka raw fact 发布失败，durable ACK 帧不返回成功 ACK。非 durable 帧可以写本地 DLQ。 - -## 协议解析和扩展 - -使用插件契约： - -```java -public interface ProtocolFrameDecoder { - ProtocolId protocol(); - DecodeResult decode(RawFrameFact rawFact, byte[] rawBytes); -} -``` - -```java -public record DecodeResult( - ParseStatus status, - List facts, - String errorMessage, - Map metadata -) {} -``` - -GB32960： - -- 命令帧映射为会话、实时、位置、报警和扩展事实。 -- 厂商扩展按平台账号、VIN 或配置 profile 选择。 -- 全量 raw replay 仍通过读取 `rawUri` 并运行当前解析器完成。 - -JT808： - -- 消息 ID 映射为注册、鉴权、心跳、位置、批量位置、多媒体元数据和扩展事实。 -- 身份解析会更新 `vehicleKey`。 -- 注册事实携带省、市、厂商、设备 ID、车牌、车牌颜色和 auth token metadata。 - -新增协议消息通常不需要改 TDengine schema，除非它成为高频查询维度。未知字段先作为 extension fact 存入 `fields_json`。 - -## 查询 API - -API 保持明确，不做一个含糊的统一接口。 - -原始帧查询： - -- `GET /api/history/raw-frames` -- 过滤条件：`protocol`、`vin`、`phone`、`vehicleKey`、`messageId`、`parseStatus`、`dateFrom`、`dateTo`、`pageSize`、`cursor`。 -- 返回 frame metadata、raw URI、checksum、parse status，以及可选的紧凑解析摘要。 - -单帧详情： - -- `GET /api/history/raw-frames/{frameId}` -- 返回 raw metadata 和解码详情。 -- 支持 `includeRawHex=true`，但必须有严格大小限制。 - -位置历史： - -- `GET /api/history/locations` -- 过滤条件：`protocol`、`vin`、`phone`、`vehicleKey`、时间范围、cursor。 - -实时历史： - -- `GET /api/history/realtime` -- 查询历史时序数据。 - -实时状态： - -- `GET /api/realtime/vehicles/{vehicleKey}` -- 从 Redis 或内存 latest state 读取，不扫 TDengine。 - -导出： - -- 导出必须异步。 -- 请求创建 export job。 -- Worker 将 TDengine 查询结果流式写入对象存储或本地导出文件。 -- API 返回 job 状态和下载 URI。 - -## 分页 - -高 QPS 历史查询使用 cursor pagination，不使用深 offset。 - -Cursor 字段： - -```text -ts, received_at, fact_id -``` - -倒序查询谓词： - -```sql -WHERE ( - ts < :cursorTs - OR (ts = :cursorTs AND received_at < :cursorReceivedAt) - OR (ts = :cursorTs AND received_at = :cursorReceivedAt AND fact_id < :cursorFactId) -) -``` - -默认 page size 为 100，上限为 1000。 - -## 容量设计 - -假设： - -- 10,000 辆车。 -- 常规上报间隔 5 到 10 秒。 -- 预计写入 1,000 到 2,000 frames/s。 -- 峰值突发系数 3 倍。 -- 历史查询目标 1,000 requests/s。 - -写路径： - -- Netty worker 保持 CPU-light。 -- Raw archive 写入使用有界异步 worker。 -- Kafka producer 开启压缩、批量、幂等，并按 vehicle key 分区。 -- History writer 按超级表和子表批量写 TDengine。 -- TDengine 子表按 vehicle key 分区，使单车查询足够便宜。 - -读路径： - -- 高频查询必须带 vehicle key 和时间范围。 -- 无边界跨车查询仅限管理接口，并强制限流和限制返回量。 -- 实时 latest state 从 Redis 或内存状态读取。 -- Raw 详情只读取单个 raw object 并按需解析。 -- 字典和元数据接口使用内存缓存。 - -背压： - -- Raw archive worker queue 固定上限。 -- Kafka publish future 必须被跟踪。 -- Durability boundary 超时时，durable ACK 失败，channel 关闭或被限流。 -- History writer lag 通过 Kafka consumer lag 和 TDengine batch latency 观察。 - -## 迁移计划 - -阶段 1：在现有代码旁边建立新的 fact model。 - -- 新增 `ingest-facts`。 -- 新增 raw archive receipt contract。 -- 新增 raw frame 和 decoded fact 的 Kafka 序列化。 -- 为 stable id、vehicle key、raw URI 生成规则加测试。 - -阶段 2：强化 raw durability boundary。 - -- 重构 GB32960 和 JT808 handler，让 ACK 等待 raw archive 和 raw fact publish。 -- 保留现有解析器。 -- 向新 Kafka topic 发 decoded facts。 - -阶段 3：构建 TDengine writer。 - -- 创建 TDengine schema manager。 -- 写入 raw frame facts。 -- 写入 location、realtime、alarm、session facts。 -- 按 stable id 实现幂等写入。 - -阶段 4：构建查询 API。 - -- Raw frame 查询。 -- 单帧回放。 -- 位置历史查询。 -- 实时历史查询。 -- 最新实时状态查询。 - -阶段 5：切换并下线旧热路径。 - -- GB32960/JT808 生产热历史不再依赖 DuckDB/Parquet。 -- 历史迁移期间保留 raw replay 兼容。 -- TDengine 覆盖验证完成后，移除协议特定的临时历史存储。 - -## 测试策略 - -单元测试： - -- `RawFrameFact` 校验和 vehicle key 派生。 -- Raw archive 路径生成和 checksum。 -- GB32960 golden frame 到 decoded facts。 -- JT808 sample frame 到 decoded facts。 -- ACK boundary 成功和失败场景。 -- TDengine SQL 生成。 - -集成测试： - -- 使用 mocked Kafka producer 和 fake raw archive 启动 APP。 -- 发送 GB32960 sample frame，验证 raw fact、decoded fact 和 raw archive receipt。 -- 发送 JT808 注册、鉴权、位置帧，验证 registration、session、location facts。 -- 模拟 raw archive 失败，验证不发送成功 ACK。 -- 模拟 Kafka publish 失败，验证 durable ACK 失败。 - -存储测试： - -- TDengine schema 初始化。 -- 批量写入 raw frames 和 decoded facts。 -- 按 vehicle key 和时间范围查询。 -- Cursor 分页正确性。 -- 按 `rawUri` 单帧回放。 - -压测： - -- 持续 30 分钟 2,000 frames/s 写入。 -- 持续 5 分钟 6,000 frames/s 突发写入。 -- 1,000 QPS location/realtime 历史查询，p95 延迟受控。 -- 校验 raw archive receipt、Kafka raw fact、TDengine raw frame row 三者数量无缺口。 - -## 运维要求 - -指标： - -- TCP 连接数。 -- Frame 接收速率。 -- Raw archive 写入延迟和失败数。 -- Kafka 发布延迟和失败数。 -- 按协议和 message id 统计 decoder 成功/失败数。 -- TDengine batch size、延迟和失败数。 -- 各查询 endpoint 的 QPS 和延迟。 -- Kafka consumer lag。 - -日志： - -- 每个失败帧输出一条结构化日志，包含 `frameId`、protocol、vehicle key、message id、raw URI 和 error。 -- 正常日志不输出完整 raw bytes。 -- Raw hex 只允许在显式 debug 工具里输出，并限制大小。 - -告警： - -- Raw archive 写入失败。 -- Durable ACK 超时。 -- Kafka producer 失败。 -- History writer lag。 -- TDengine 写入失败。 -- Raw archive 数量和 TDengine raw frame row 数量不一致。 - -## 验收标准 - -重设计只有满足以下条件才算完成： - -1. 每一条被接收的 GB32960 和 JT808 帧都会创建 raw archive object 和 TDengine `raw_frames` 行。 -2. 有 bytes 的异常帧可以从 raw history 查询到，包含 parse status 和错误原因。 -3. GB32960 和 JT808 位置历史可以按车辆和时间范围 cursor 分页查询。 -4. GB32960 实时历史可以按车辆和时间范围查询。 -5. JT808 注册和会话数据可查询，并且可以更新 identity binding。 -6. 单帧详情可以通过 `rawUri` 重新读取 raw bytes，并使用当前解析器解码。 -7. 需要 ACK 的帧，在 raw archive 和 raw fact publish 成功前，不会发送成功 ACK。 -8. 新协议字段可以通过新增 decoder/projector 扩展；只有成为高频查询字段时才需要 TDengine 表迁移。 -9. 压测证明可以持续 2,000 frames/s 写入，并支持 1,000 QPS 有边界历史查询。 -10. GB32960/JT808 生产查询不再依赖旧 DuckDB/Parquet 热历史路径。 - -## 已固定决策 - -第一版实现固定以下选择： - -- 先使用本地 raw archive，实现接口时预留对象存储。 -- TDengine 是 GB32960/JT808 唯一生产热历史查询库。 -- 使用协议明确的查询 API，不做一个泛化万能查询 API。 -- 高频历史查询统一使用 cursor pagination。 -- Kafka payload 保持轻量，只传 raw 引用，不传 raw bytes。 diff --git a/docs/superpowers/specs/2026-07-01-go-ingest-redesign-design.md b/docs/superpowers/specs/2026-07-01-go-ingest-redesign-design.md deleted file mode 100644 index bf84986d..00000000 --- a/docs/superpowers/specs/2026-07-01-go-ingest-redesign-design.md +++ /dev/null @@ -1,498 +0,0 @@ -# Go 车辆接入重构设计规格 - -## 目标 - -使用 Go 重新设计车辆数据接入运行面，让系统回到第一性原则： - -- 接入层只负责可靠收包、协议解析、应答、身份解析和投递。 -- Kafka 是唯一在线消息通道。 -- TDengine 保存高吞吐时序明细和完整 RAW 解析 JSON。 -- MySQL 保存低频配置、身份映射和每日统计窄表。 -- Redis 保存可重建的准实时状态。 -- 32960、JT808、宇通 MQTT 使用同一套数据层接口，避免每个协议各自落库。 - -第一阶段只做生产链路的骨架和核心数据闭环：接收、解析、Kafka、TDengine、MySQL 统计、Redis 实时查询、ECS 验证。历史查询 API 和业务侧复杂分析放在后续阶段。 - -## 参考原则 - -本设计只吸收开源项目的边界思想，不复制外部实现代码。 - -- GB32960 参考 `DarkInno/gb32960-go-sdk`：连接管理、Handler、Forwarder、鉴权接口分离。 -- JT808 参考 `cuteLittleDevil/go-jt808`：高并发 Go 原生 TCP、协议核心小而可扩展、事件/适配器机制。 -- JT808 参考 `fakeyanss/jt808-server-go`：`FramePayload -> PacketData -> JT808Msg -> Reply` 的分层处理、Gateway 模式、保活和版本兼容。 -- TDengine 参考官方数据模型：按数据采集点建立 supertable，静态维度做 tags，明细数据按子表写入；Go 连接优先使用 WebSocket 驱动。 - -## 非目标 - -- 第一阶段不继续扩展 Java 服务。 -- 第一阶段不做统一大查询 API。 -- 第一阶段不恢复信达 Push。 -- 第一阶段不把所有协议字段展开成一张超宽 TDengine 表。 -- Redis 不作为历史事实源。 -- MySQL 不保存高频原始明细。 - -## 目标目录 - -新的 Go 运行面放在 `go/vehicle-gateway`，不继续沿用实验性的多 module 分散结构。现有 `go/ingest-edge` 和 `go/vehicle-state` 可以作为迁移素材，最终归并或删除。 - -```text -go/vehicle-gateway/ - cmd/ - gateway/ # 协议接入进程：32960 TCP、JT808 TCP、宇通 MQTT - history-writer/ # Kafka -> TDengine RAW/location/mileage points - stat-writer/ # Kafka -> MySQL 每日统计窄表 - realtime-api/ # Redis 准实时查询 API - internal/ - config/ # 环境变量、配置文件、校验 - gateway/ # TCP/MQTT 生命周期、连接限制、优雅停机 - protocol/ - gb32960/ # 32960 编解码 - jt808/ # 808 编解码 - yutongmqtt/ # 宇通 MQTT payload 解析 - identity/ # VIN、phone、device_id、plate 绑定 - envelope/ # 统一事件模型 - eventbus/ # Kafka producer/consumer - history/ # TDengine adapter - stats/ # 每日里程聚合 - realtime/ # Redis adapter 与快照合并 - observability/ # 日志、metrics、健康检查 -``` - -## 协议接入设计 - -### GB32960 - -职责： - -- 监听 TCP `32960`。 -- 支持车辆登入、实时信息上报、补发信息上报、车辆登出、平台登入/登出、心跳、校时。 -- 正确返回协议应答。 -- 完整解析 2016 数据单元： - - 整车数据 - - 驱动电机数据 - - 燃料电池数据 - - 发动机数据 - - 车辆位置数据 - - 极值数据 - - 报警数据 - - 可充电储能装置电压数据 - - 可充电储能装置温度数据 -- RAW 中保存完整 parsed JSON。 -- 关键字段标准化为统一 field： - - `speed_kmh` - - `total_mileage_km` - - `soc_percent` - - `longitude` - - `latitude` - - `vehicle_status` - - `charge_status` - -### JT808 - -职责： - -- 监听 TCP `808`。 -- 支持 2011/2013 基础兼容，预留 2019 版本标记。 -- 分层处理： - - Frame：`0x7e` 包边界、转义、校验。 - - Packet：消息头、手机号 BCD、流水号、分包。 - - Message：注册、鉴权、心跳、注销、位置、批量位置、未知上行。 - - Reply：注册应答、平台通用应答。 -- 终端手机号按协议头 BCD 解析，内部同时保留原始 BCD、规范化 phone、去前导零 phone。 -- 注册帧和鉴权帧写入 808 registration 表。 -- 如果设备直接上报位置帧，也要走 identity 解析，尝试用 phone、device_id、plate 找到 VIN。 -- 0200 附加信息必须解析表 27： - - `0x01` GPS 总里程，DWORD，单位 0.1 km。 - - 其他已知附加项结构化放入 `parsed_json.additional`。 -- 标准化字段： - - `speed_kmh` - - `total_mileage_km` - - `longitude` - - `latitude` - - `altitude_m` - - `direction_deg` - - `alarm_flag` - - `status_flag` - -### 宇通 MQTT - -职责： - -- 使用正式 MQTT 配置订阅生产 topic。 -- 接收 payload 后解析为统一 envelope。 -- 按 payload 中的车辆标识解析 VIN、车牌、设备号。 -- 保存完整 payload 和 parsed JSON。 -- 标准化字段向 32960/808 对齐： - - `speed_kmh` - - `total_mileage_km` - - `longitude` - - `latitude` - - `soc_percent` - - `vehicle_status` - -## 统一 Envelope - -所有协议接入后输出同一结构。 - -```json -{ - "event_id": "string", - "trace_id": "string", - "protocol": "GB32960|JT808|YUTONG_MQTT", - "message_id": "string", - "vin": "string", - "vehicle_key": "string", - "phone": "string", - "device_id": "string", - "plate": "string", - "source_endpoint": "ip:port or mqtt://broker/topic", - "event_time_ms": 0, - "received_at_ms": 0, - "raw_hex": "string", - "raw_text": "string", - "parsed": {}, - "fields": { - "speed_kmh": 0, - "total_mileage_km": 0, - "longitude": 0, - "latitude": 0 - }, - "parse_status": "OK|PARTIAL|BAD_FRAME", - "parse_error": "" -} -``` - -规则： - -- `event_id` 使用协议、设备标识、消息流水、事件时间、RAW hash 生成，保证幂等。 -- Kafka partition key 优先使用 VIN；没有 VIN 时使用 `protocol:phone/device_id`。 -- `parsed` 保存协议完整结构化字段。 -- `fields` 只保存跨协议核心字段，用于统计、位置和实时快照。 - -## Kafka Topic - -生产只支持 Kafka。 - -| Topic | 内容 | Key | -|---|---|---| -| `vehicle.raw.gb32960.v1` | 32960 完整 RAW envelope | VIN 或 vehicle_key | -| `vehicle.raw.jt808.v1` | 808 完整 RAW envelope | VIN 或 phone | -| `vehicle.raw.yutong-mqtt.v1` | 宇通 MQTT 完整 RAW envelope | VIN 或 vehicle_key | -| `vehicle.event.unified.v1` | 统一轻量事件，用于业务消费 | VIN 或 vehicle_key | - -接入层必须先写 RAW topic，再写 unified event。后续消费者只依赖 Kafka，不直接依赖接入进程内存。 - -发布可靠性： - -- Kafka 生产端必须开启同步写入，RAW 写成功后才允许写 unified event。 -- Kafka 写入必须支持可配置重试、单次写超时和短退避，默认值为 `KAFKA_PUBLISH_ATTEMPTS=3`、`KAFKA_PUBLISH_TIMEOUT_MS=3000`、`KAFKA_PUBLISH_BACKOFF_MS=100`。 -- gateway 支持本地磁盘 spool/WAL，配置 `KAFKA_SPOOL_DIR` 后启用；Kafka 长时间不可用时，发布失败的 envelope 先原子写入本地 JSON 文件。 -- spool 补发按文件名顺序执行，补发成功后删除文件；默认补发间隔由 `KAFKA_SPOOL_REPLAY_INTERVAL_MS=1000` 控制。 -- 如果 RAW 写 Kafka 失败并已落盘，同一个 event 的 unified event 不允许抢先写 Kafka，也必须进入 spool，确保恢复后按 RAW -> unified 顺序补发。 -- Kafka topic 不允许自动创建，topic 和分区数由部署脚本或运维初始化，避免生产拼写错误造成隐性分流。 - -## TDengine 数据库设计 - -如果现有 TDengine 库不符合目标，可以新建库并迁移。建议库名： - -```sql -CREATE DATABASE IF NOT EXISTS lingniu_vehicle_ts KEEP 7300 DURATION 10 BUFFER 256; -``` - -### raw_frames - -保存完整 RAW 和完整 parsed JSON。 - -```sql -CREATE STABLE IF NOT EXISTS raw_frames ( - ts TIMESTAMP, - frame_id NCHAR(64), - event_id NCHAR(64), - message_id INT, - event_time TIMESTAMP, - received_at TIMESTAMP, - raw_size_bytes INT, - raw_hex BINARY(16374), - raw_text BINARY(16374), - parsed_json BINARY(16374), - fields_json BINARY(4096), - parse_status NCHAR(16), - parse_error BINARY(1024), - source_endpoint NCHAR(128) -) TAGS ( - protocol NCHAR(32), - vehicle_key NCHAR(64), - vin NCHAR(32), - phone NCHAR(32), - device_id NCHAR(64) -); -``` - -说明： - -- 子表按 `protocol + vehicle_key hash` 创建。 -- RAW 查询优先按 tags 和时间范围过滤。 -- `parsed_json` 是完整协议字段；查询 API 可选择是否返回，避免默认大字段拖慢。 - -### vehicle_locations - -只保存位置查询核心字段。 - -```sql -CREATE STABLE IF NOT EXISTS vehicle_locations ( - ts TIMESTAMP, - event_id NCHAR(64), - frame_id NCHAR(64), - received_at TIMESTAMP, - longitude DOUBLE, - latitude DOUBLE, - altitude_m DOUBLE, - speed_kmh DOUBLE, - direction_deg INT, - alarm_flag BIGINT, - status_flag BIGINT, - total_mileage_km DOUBLE -) TAGS ( - protocol NCHAR(32), - vehicle_key NCHAR(64), - vin NCHAR(32), - phone NCHAR(32), - device_id NCHAR(64) -); -``` - -### vehicle_mileage_points - -保存总里程采样点，用于回放重算和查询。 - -```sql -CREATE STABLE IF NOT EXISTS vehicle_mileage_points ( - ts TIMESTAMP, - event_id NCHAR(64), - frame_id NCHAR(64), - received_at TIMESTAMP, - total_mileage_km DOUBLE, - speed_kmh DOUBLE, - longitude DOUBLE, - latitude DOUBLE -) TAGS ( - protocol NCHAR(32), - vehicle_key NCHAR(64), - vin NCHAR(32), - phone NCHAR(32), - device_id NCHAR(64) -); -``` - -## MySQL 数据库设计 - -MySQL 只保存配置、身份映射、808 注册鉴权和每日统计窄表。 - -### vehicle_identity_binding - -用于把外部标识定位到 VIN。 - -```sql -CREATE TABLE vehicle_identity_binding ( - id BIGINT PRIMARY KEY AUTO_INCREMENT, - vin VARCHAR(32) NOT NULL, - plate VARCHAR(32) NULL, - phone VARCHAR(32) NULL, - device_id VARCHAR(64) NULL, - remark VARCHAR(255) NULL, - created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, - updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, - UNIQUE KEY uk_vin (vin), - KEY idx_plate (plate), - KEY idx_phone (phone), - KEY idx_device_id (device_id) -); -``` - -### jt808_registration - -只记录 808 独有的注册、鉴权和在线身份信息。 - -```sql -CREATE TABLE jt808_registration ( - phone VARCHAR(32) PRIMARY KEY, - vin VARCHAR(32) NULL, - plate VARCHAR(32) NULL, - device_id VARCHAR(64) NULL, - manufacturer_id VARCHAR(32) NULL, - terminal_model VARCHAR(64) NULL, - terminal_id VARCHAR(64) NULL, - auth_code VARCHAR(128) NULL, - source_endpoint VARCHAR(128) NULL, - first_registered_at DATETIME NULL, - latest_registered_at DATETIME NULL, - latest_authed_at DATETIME NULL, - latest_seen_at DATETIME NULL, - created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, - updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, - KEY idx_vin (vin), - KEY idx_plate (plate), - KEY idx_device_id (device_id) -); -``` - -### vehicle_daily_metric - -32960 和 808 的每日里程、每日总里程都进入同一张窄表。 - -```sql -CREATE TABLE vehicle_daily_metric ( - id BIGINT PRIMARY KEY AUTO_INCREMENT, - vin VARCHAR(32) NOT NULL, - stat_date DATE NOT NULL, - protocol VARCHAR(32) NOT NULL, - metric_key VARCHAR(64) NOT NULL, - metric_value DECIMAL(18,3) NOT NULL, - metric_unit VARCHAR(16) NOT NULL, - first_total_mileage_km DECIMAL(18,3) NULL, - latest_total_mileage_km DECIMAL(18,3) NULL, - sample_count BIGINT NOT NULL DEFAULT 0, - calculation_method VARCHAR(64) NOT NULL, - created_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP, - updated_at DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, - UNIQUE KEY uk_daily_metric (vin, stat_date, protocol, metric_key), - KEY idx_stat_date (stat_date), - KEY idx_protocol_metric (protocol, metric_key) -); -``` - -指标： - -- `daily_mileage_km = max(total_mileage_km) - min(total_mileage_km)` -- `daily_total_mileage_km = max(total_mileage_km)` - -统计消费者可幂等重放。每个样本 upsert 时只维护当天 `min/max total_mileage_km`，不依赖进程内状态。 - -## Redis 实时数据设计 - -Redis 保存可从 Kafka 重建的数据。 - -| Key | 内容 | -|---|---| -| `vehicle:online:{vin}` | 在线状态、最后活跃时间、协议列表 | -| `vehicle:latest:{vin}` | 合并后的 VIN 最新快照 | -| `vehicle:latest:{vin}:{protocol}` | 单协议最新快照 | -| `vehicle:last_seen` | VIN 到最后接收时间的 sorted set | - -合并规则： - -- 位置以最新 event_time 为准。 -- 总里程取最新上报值，不做递增假设。 -- 32960 多帧字段按 field timestamp 合并。 -- 808 和 MQTT 不覆盖其他协议独有字段。 -- Redis TTL 用于在线判断，不作为数据删除依据。 - -## 数据流 - -```mermaid -flowchart LR - source["车辆 / 平台"] --> gateway["Go gateway"] - gateway --> parser["protocol parser"] - parser --> identity["identity resolver"] - identity --> kafkaRaw["Kafka RAW topics"] - identity --> kafkaEvent["Kafka unified event"] - kafkaRaw --> history["history-writer"] - kafkaRaw --> stats["stat-writer"] - kafkaEvent --> realtime["realtime consumer"] - history --> td["TDengine raw_frames / locations / mileage_points"] - stats --> mysql["MySQL vehicle_daily_metric"] - realtime --> redis["Redis latest state"] -``` - -## 错误处理 - -- 协议坏帧仍写 RAW topic，`parse_status=BAD_FRAME`。 -- 可部分解析的帧写 `parse_status=PARTIAL`，保留 parse error。 -- Kafka 写失败时接入层先按配置重试；重试耗尽后不写 unified event，必须打错误日志和 metrics。 -- 启用 `KAFKA_SPOOL_DIR` 后，Kafka 重试耗尽会落本地 spool；如果 spool 写失败，才视为接入层最终失败。 -- TDengine 写失败不提交 Kafka offset。 -- MySQL 统计写失败不提交 Kafka offset。 -- Redis 写失败不影响历史和统计，但要通过 metrics 暴露。 -- 808 相同 phone 新连接时，旧连接应被替换并记录 latest_seen。 - -## 部署设计 - -生产 ECS 运行 Go 二进制或容器： - -- `vehicle-gateway` -- `vehicle-history-writer` -- `vehicle-stat-writer` -- `vehicle-realtime-api` - -配置来源： - -- 环境变量优先。 -- Nacos 可作为后续配置中心，但 Go 第一阶段必须能只靠环境变量启动，便于 Portainer 部署和故障恢复。 - -TDengine 连接： - -- 优先 WebSocket `taosWS`。 -- 若当前 ECS 只开放原生端口，可短期兼容 `taosSql`，但代码配置必须显式标记为兼容模式。 - -## 验证计划 - -### 本地验证 - -- 用样例 808 报文验证： - - phone BCD 解析。 - - 0200 经纬度、速度、方向、时间。 - - 表 27 `0x01` GPS 总里程。 -- 用真实 32960 报文验证： - - 登录应答正确。 - - 实时帧和补发帧都能写 RAW。 - - 位置和总里程字段进入统一 fields。 -- 用宇通 MQTT payload 验证： - - payload 完整保存。 - - 核心字段归一化。 -- 不连 Kafka 时可输出 JSON log。 -- 连接 Kafka 时 RAW topic 可消费到 envelope。 - -### ECS 验证 - -- 部署前先旁路接收或短窗口切流。 -- 验证端口： - - `32960` - - `808` - - MQTT 订阅连接。 -- 验证 Kafka topic 有持续写入。 -- 验证 TDengine： - - `raw_frames` 有三种协议数据。 - - `vehicle_locations` 能按 VIN、协议、时间分页查询。 - - `vehicle_mileage_points` 有 32960 和 808 总里程采样。 -- 验证 MySQL： - - `vehicle_daily_metric` 有 32960 和 808 的 `daily_mileage_km`、`daily_total_mileage_km`。 -- 验证 Redis： - - 查询 VIN 在线。 - - 查询 VIN 合并实时快照。 - - 查询 VIN 分协议实时快照。 - -## 验收标准 - -- Go gateway 能在 ECS 上接收真实 32960、808、宇通 MQTT 数据。 -- 三种协议 RAW 都进入 Kafka。 -- Kafka 短暂写失败时，gateway 按配置重试；单元测试覆盖首次失败后成功和重试耗尽返回错误。 -- Kafka 长时间不可用时，gateway 可把 RAW/unified 写入本地 spool；单元测试覆盖 RAW 已落盘时 unified 也落盘、恢复后按顺序补发并删除文件。 -- 三种协议 RAW 和 parsed JSON 都进入 TDengine。 -- 32960 和 808 的位置进入 `vehicle_locations`。 -- 32960 和 808 的总里程采样进入 `vehicle_mileage_points`。 -- 32960 和 808 的每日里程、每日总里程进入 MySQL 窄表。 -- Redis 可以查询 VIN 在线和实时数据。 -- 任一消费者宕机后可通过 Kafka offset 继续恢复。 -- 查询和统计验证使用 ECS 上的真实数据完成。 - -## 实施顺序 - -1. 创建 `go/vehicle-gateway` 单 module，迁移现有 Go 原型中可用的解析、Kafka、TDengine、MySQL、Redis 代码。 -2. 先写协议解析单元测试，覆盖 808 样例报文和 32960 核心字段。 -3. 完成统一 envelope 和 Kafka sink。 -4. 完成 TDengine schema bootstrap 与 history writer。 -5. 完成 MySQL schema bootstrap 与 stat writer。 -6. 完成 Redis realtime writer/API。 -7. 本地启动 gateway，用 ECS 转发流量验证。 -8. 构建 Linux 镜像，部署 ECS。 -9. 按验收标准逐项验证并记录证据。 diff --git a/docs/target-architecture.md b/docs/target-architecture.md deleted file mode 100644 index 2f636c4a..00000000 --- a/docs/target-architecture.md +++ /dev/null @@ -1,202 +0,0 @@ -# Target Architecture - -This document records the production target for `lingniu-vehicle-ingest`. -The project should stay small at runtime: protocol apps ingest and publish, -history consumes and indexes, analytics derives metrics, and business systems -read through explicit APIs or Kafka. - -Only supported message backbone: Kafka. - -## Active Scope - -Active production protocols: - -| Protocol | Runtime app | Event topic | Raw topic | -|---|---|---|---| -| GB/T 32960 | `gb32960-ingest-app` | `vehicle.event.gb32960.v1` | `vehicle.raw.gb32960.v1` | -| JT/T 808 | `jt808-ingest-app` | `vehicle.event.jt808.v1` | `vehicle.raw.jt808.v1` | -| Yutong MQTT | `yutong-mqtt-app` | `vehicle.event.mqtt-yutong.v1` | `vehicle.raw.mqtt-yutong.v1` | - -Xinda Push source, Maven profile, deployment bindings, and history consumers -are removed. New optimization work should target the active protocols above. - -JSATL12 attachment upload is also outside the default production reactor. Keep -it available through the `optional-attachments` profile until an attachment app -is explicitly deployed. - -## Goals - -- Keep protocol IO, raw archive, history, latest-state, and statistics as - separate responsibilities. -- Publish normalized Kafka envelopes with VIN as the partition key whenever a - VIN is known. -- Store complete raw payload metadata and parsed JSON in TDengine `raw_frames`. -- Store query-friendly location rows separately from raw payload JSON. -- Keep derived statistics in metric tables, not protocol-specific daily tables. -- Use Redis only for optional latest-state APIs that can be rebuilt from Kafka. -- Keep optional compatibility modules outside the production hot path. - -## Non-Goals - -- Ingest apps do not write business tables. -- Ingest apps do not serve history query APIs. -- Redis is not the long-term historical store. -- File-based event indexes are not part of the current build surface. -- `telemetry_fields` parsing and field-trend APIs are owned by a separate - field parsing service. This project stores RAW parsed JSON plus compact - location rows and does not expose a telemetry-fields history API. - -## Runtime Responsibilities - -### Protocol Apps - -Apps: - -- `gb32960-ingest-app` -- `jt808-ingest-app` -- `yutong-mqtt-app` - -Responsibilities: - -- Accept protocol traffic. -- Decode, authenticate, acknowledge, and maintain sessions where the protocol - requires it. -- Archive raw bytes through `sink-archive` when bytes are available. -- Publish parsed event envelopes to Kafka. -- Publish raw envelopes to Kafka. -- Resolve vehicle identity through the shared MySQL identity binding table. - -Protocol apps must stay independent from TDengine history readers, Redis state -repositories, and statistic calculators. - -### Event Contract - -The Kafka sink owns the protobuf envelope and consumer plumbing. -Consumers should use `EnvelopeConsumerProcessor` so invalid protobuf, skipped -envelopes, and downstream failures are converted to structured results and DLQ -records instead of blocking a partition. - -Core identity fields: - -| Field | Meaning | -|---|---| -| `event_id` | Parsed event idempotency key | -| `trace_id` | Cross-service trace id | -| `vin` | Vehicle id and Kafka partition key | -| `source` | Source protocol, for example `GB32960`, `JT808`, `YUTONG_MQTT` | -| `event_time_ms` | Device event time | -| `ingest_time_ms` | Platform receive time | -| `raw_uri` | `archive://...` reference to raw bytes when available | - -Full-field telemetry uses `TelemetrySnapshot` fields with stable keys such as -`total_mileage_km`, `speed_kmh`, `longitude`, and `latitude`. - -### History App - -App: `vehicle-history-app` - -Responsibilities: - -- Consume active protocol event topics. -- Consume active protocol raw topics. -- Write raw frame rows into TDengine `raw_frames`. -- Write compact location rows into TDengine location tables. -- Keep `payloadJson.parsed` on raw rows for full raw inspection. -- Expose paged history APIs for raw frames and locations. -- Keep specialized APIs off by default unless explicitly enabled. - -Production default: - -- `TDENGINE_HISTORY_ENABLED=true` in deployment. -- `EVENT_FILE_STORE_ENABLED=false`. - -File-based event indexes have been removed; this app should keep history -queries on TDengine and raw replay on `archive://...` references. - -### Analytics App - -App: `vehicle-analytics-app` - -Responsibilities: - -- Consume JT808 event envelopes. -- Calculate 808 daily mileage from the reported GPS total mileage only: - -```text -daily_mileage_km = max_total_mileage_km - min_total_mileage_km -calculation_method = JT808_TOTAL_MILEAGE_DIFF -``` - -- Store the metric in `vehicle_stat_metric` with - `metric_key = daily_mileage_km`; the local-day minimum and maximum GPS total - mileage used for the subtraction stay on the same metric row as calculation - source columns. - -Runtime state: none outside `vehicle_stat_metric`. There is no JT808-specific -daily mileage table. Restart recovery reads the same metric row. - -### Latest State - -Module: `vehicle-state-service` - -This module is outside the default production reactor. Build it with -`-Poptional-latest-state` when Redis latest-state APIs are explicitly needed. - -Responsibilities: - -- Consume normalized Kafka envelopes. -- Maintain latest state, location, and safety snapshots in Redis. -- Serve latest-state APIs for operational screens. - -Redis state must be rebuildable from Kafka replay and should not be used as the -source of truth for historical queries. - -## Data Flow - -Default production flow: - -```mermaid -flowchart LR - vehicle["Vehicles / Platforms"] --> ingest["Protocol apps"] - ingest --> archive["sink-archive
raw bytes"] - ingest --> kafka["Kafka
event + raw envelopes"] - kafka --> history["vehicle-history-app"] - kafka --> analytics["vehicle-analytics-app"] - history --> tdengine["TDengine
raw_frames + locations"] - analytics --> mysql["MySQL
vehicle_stat_metric"] -``` - -Optional latest-state flow, enabled only with `-Poptional-latest-state`, consumes -the same Kafka envelopes and rebuilds Redis snapshots outside the default -production deployment. - -## Failure Handling - -- Protocol apps continue accepting traffic when history, analytics, or Redis - consumers are unavailable. -- Kafka producer failures are handled by sink retry/circuit-breaker behavior and - DLQ topics where configured. -- History consumers should commit offsets only after TDengine writes succeed. -- Analytics daily mileage state can recover from `vehicle_stat_metric` or Kafka - replay. -- Raw archive failures must be observable, but event publication and history - indexing remain separate concerns. - -## Acceptance Criteria - -- Active production deployment contains GB32960, JT808, Yutong MQTT, - vehicle-history, and vehicle-analytics apps. -- Xinda Push source and deployment bindings are removed. -- JSATL12 is absent from the default Maven reactor and available through - `optional-attachments`. -- vehicle-state-service is absent from the default Maven reactor and available - through `optional-latest-state`. -- raw-archive-store prototype has been removed; raw bytes are written through - `sink-archive`. -- History APIs read from TDengine; file-based event indexes are not maintained. -- Raw frame queries can return complete parsed JSON through `payloadJson.parsed`. -- Location queries page over compact rows and reference raw records instead of - duplicating full raw JSON. -- JT808 daily mileage is stored only in `vehicle_stat_metric`. -- Runtime state needed for 808 mileage is also in `vehicle_stat_metric`. -- Build validation covers the active modules and their composition tests. diff --git a/docs/vehicle-telemetry-internal-fields.md b/docs/vehicle-telemetry-internal-fields.md deleted file mode 100644 index 3ecc50ad..00000000 --- a/docs/vehicle-telemetry-internal-fields.md +++ /dev/null @@ -1,146 +0,0 @@ -# Vehicle Telemetry Internal Fields - -This document defines the internal telemetry field model for Lingniu hydrogen -vehicle operations. Protocol fields such as GB/T 32960, JT808, MQTT, and vendor -extensions must be mapped into these fields before storage, statistics, or -business reporting. - -## Design Rules - -- Business statistics use internal fields, not protocol field names. -- Every internal field has one stable unit and one business meaning. -- Protocol-specific raw values remain traceable through `source_protocol`, - `protocol_version`, `metadata`, and raw archive references. -- Core operational fields are stored as typed columns or typed payload fields. -- Unstable vendor-specific fields can stay in extension metadata until promoted. -- Hydrogen leak safety is a first-class critical safety event, not a generic - alarm count. - -## Identity And Time - -| Internal field | Type | Unit | Meaning | GB/T 32960 source | -|---|---:|---|---|---| -| `vin` | string | - | Vehicle VIN and partition key | header VIN | -| `event_time` | instant | - | Device collection time | command body timestamp | -| `ingest_time` | instant | - | Platform receive time | ingest server clock | -| `source_protocol` | enum | - | Source adapter | `GB32960` | -| `protocol_version` | string | - | Source protocol version | `V2016` / `V2025` | - -## Real-Time Vehicle State - -| Internal field | Type | Unit | Meaning | GB/T 32960 source | -|---|---:|---|---|---| -| `vehicle_state` | enum | - | Started, shutdown, other, invalid | vehicle block `vehicleState` | -| `charging_state` | enum | - | Charging state | vehicle block `chargingState` | -| `running_mode` | enum | - | Electric, hybrid, fuel, other | vehicle block `runningMode` | -| `speed_kmh` | double | km/h | Current speed | vehicle block `speedKmh` | -| `total_mileage_km` | double | km | Odometer mileage | vehicle block `totalMileageKm` | -| `gear_level` | integer | - | Gear value | vehicle block `gearRaw` low 4 bits | -| `accelerator_pedal` | double | % | Accelerator pedal opening | V2016 vehicle block | -| `brake_pedal` | double | % | Brake pedal opening | V2016 vehicle block | - -## Location - -| Internal field | Type | Unit | Meaning | GB/T 32960 source | -|---|---:|---|---|---| -| `longitude` | double | deg | Longitude | position block | -| `latitude` | double | deg | Latitude | position block | -| `altitude_m` | double | m | Altitude when available | extension or other protocol | -| `direction_deg` | double | deg | Direction when available | extension or other protocol | -| `location_status_raw` | long | - | Raw location status bits | position block `statusFlag` | - -## Battery And Electricity - -| Internal field | Type | Unit | Meaning | GB/T 32960 source | -|---|---:|---|---|---| -| `battery_soc` | double | % | Battery SOC | vehicle block `socPercent` | -| `battery_voltage_v` | double | V | Battery total voltage | vehicle block `totalVoltageV` | -| `battery_current_a` | double | A | Battery total current | vehicle block `totalCurrentA` | -| `battery_power_kw` | double | kW | Derived instantaneous battery power | `voltage * current / 1000` in statistics | -| `battery_temperature_max_c` | integer | C | Max battery temperature | temperature or extreme blocks | -| `battery_temperature_min_c` | integer | C | Min battery temperature | temperature or extreme blocks | -| `daily_electricity_kwh` | double | kWh | Daily electricity usage | statistics time integral | - -Daily electricity must be computed in the statistics layer from typed telemetry -points. Do not accumulate protocol raw values directly. - -## Hydrogen And Fuel Cell - -| Internal field | Type | Unit | Meaning | GB/T 32960 source | -|---|---:|---|---|---| -| `fc_voltage_v` | double | V | Fuel cell voltage | fuel cell block | -| `fc_current_a` | double | A | Fuel cell current | fuel cell block | -| `fc_temp_c` | double | C | Fuel cell or hydrogen system temperature | fuel cell block | -| `hydrogen_remaining_kg` | double | kg | Remaining hydrogen mass | vendor extension or capacity conversion | -| `hydrogen_remaining_percent` | double | % | Remaining hydrogen percent | V2025 fuel cell block | -| `hydrogen_high_pressure_mpa` | double | MPa | Hydrogen high pressure | fuel cell block | -| `hydrogen_low_pressure_mpa` | double | MPa | Hydrogen low pressure when available | extension or other protocol | -| `daily_hydrogen_kg` | double | kg | Daily hydrogen consumed | statistics layer | -| `hydrogen_mileage_efficiency` | double | km/kg | Mileage per kg hydrogen | statistics layer | - -Daily hydrogen usage should be calculated from adjacent hydrogen remaining -values. A rising value indicates refueling and must not be counted as negative -consumption. - -## Hydrogen Tank Safety - -| Internal field | Type | Unit | Meaning | GB/T 32960 source | -|---|---:|---|---|---| -| `safety_category` | enum | - | `GENERAL`, `TANK_PRESSURE`, `TANK_TEMPERATURE`, `HYDROGEN_LEAK` | alarm bit classification | -| `hydrogen_leak_detected` | boolean | - | Whether hydrogen leak is detected | general alarm bit `HYDROGEN_LEAK` | -| `hydrogen_leak_level` | enum | - | `NONE`, `WARNING`, `CRITICAL`, `UNKNOWN` | internal rule | -| `hydrogen_leak_action_required` | boolean | - | Whether immediate handling is required | internal rule | -| `tank_pressure_status` | enum | - | Normal, warning, critical | hydrogen pressure bit or threshold rule | -| `tank_temperature_status` | enum | - | Normal, warning, critical | hydrogen temp bit or threshold rule | - -Safety rule: - -```text -If HYDROGEN_LEAK is present: - alarm.level = CRITICAL - safety_category = HYDROGEN_LEAK - hydrogen_leak_detected = true - hydrogen_leak_level = CRITICAL - hydrogen_leak_action_required = true -``` - -Hydrogen leak overrides otherwise normal vehicle, speed, battery, or hydrogen -state. It must be counted independently from generic alarms. - -## Daily Operation Statistics - -Daily statistics are generated by `stat_date + vin`. - -| Internal field | Type | Unit | Meaning | -|---|---:|---|---| -| `stat_date` | date | - | Statistics date | -| `vin` | string | - | Vehicle VIN | -| `first_event_time` | instant | - | First valid telemetry time | -| `last_event_time` | instant | - | Last valid telemetry time | -| `online_minutes` | double | min | Online duration | -| `running_minutes` | double | min | Running duration | -| `daily_mileage_km` | double | km | Daily mileage | -| `daily_electricity_kwh` | double | kWh | Daily electricity usage | -| `daily_hydrogen_kg` | double | kg | Daily hydrogen usage | -| `hydrogen_added_kg` | double | kg | Refueled hydrogen | -| `km_per_kg_hydrogen` | double | km/kg | Hydrogen efficiency | -| `kwh_per_100km` | double | kWh/100km | Electricity efficiency | -| `alarm_count` | integer | - | Generic alarm count | -| `tank_safety_alarm_count` | integer | - | Tank safety alarm count | -| `hydrogen_leak_alarm_count` | integer | - | Hydrogen leak alarm count | -| `hydrogen_leak_duration_seconds` | long | s | Hydrogen leak duration | -| `data_quality_level` | enum | - | Good, partial, bad | - -## Implementation Status - -- `RealtimePayload` already uses internal operational field names for state, - location, battery, fuel cell, and hydrogen fields. -- `AlarmPayload` carries internal hydrogen safety fields. -- `Gb32960EventMapper` maps GB/T 32960 alarm bits into internal safety fields. -- `TelemetrySnapshot` publishes full-field internal telemetry values through the - Kafka protobuf envelope. -- `event-history-service`, `vehicle-state-service`, and `vehicle-stat-service` - consume the same full-field telemetry snapshot contract. -- `event-history-service` CSV export can flatten selected internal fields with - `fields=field_a,field_b`; media archive references are exposed through - `rawArchiveUri` by falling back to `MediaMeta.archiveRef`. diff --git a/modules/apps/command-gateway/pom.xml b/modules/apps/command-gateway/pom.xml deleted file mode 100644 index 10eeedae..00000000 --- a/modules/apps/command-gateway/pom.xml +++ /dev/null @@ -1,58 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - command-gateway - command-gateway - - 下行命令网关：HTTP → 设备。复用 session-core 的 CommandDispatcher， - 覆盖原 JT808Controller / JT1078Controller 的 43 个端点（PoC 阶段先落核心子集）。 - - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - session-core - - - com.lingniu.ingest - protocol-jt808 - - - com.lingniu.ingest - protocol-jt1078 - - - org.springframework.boot - spring-boot-starter-web - - - org.springframework.boot - spring-boot-starter-validation - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.springframework.boot - spring-boot-test - test - - - diff --git a/modules/apps/command-gateway/src/main/java/com/lingniu/ingest/gateway/CommandGatewayAutoConfiguration.java b/modules/apps/command-gateway/src/main/java/com/lingniu/ingest/gateway/CommandGatewayAutoConfiguration.java deleted file mode 100644 index c987f13b..00000000 --- a/modules/apps/command-gateway/src/main/java/com/lingniu/ingest/gateway/CommandGatewayAutoConfiguration.java +++ /dev/null @@ -1,15 +0,0 @@ -package com.lingniu.ingest.gateway; - -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnClass; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.context.annotation.ComponentScan; -import org.springframework.web.servlet.DispatcherServlet; - -@AutoConfiguration -@ConditionalOnClass(DispatcherServlet.class) -@ConditionalOnProperty(prefix = "lingniu.ingest.command-gateway", name = "enabled", havingValue = "true") -@ComponentScan(basePackageClasses = TerminalCommandController.class) -public class CommandGatewayAutoConfiguration { - // 只扫描 REST 下行命令网关；不参与 32960 接收、RAW 冷存或历史查询链路。 -} diff --git a/modules/apps/command-gateway/src/main/java/com/lingniu/ingest/gateway/CommandResult.java b/modules/apps/command-gateway/src/main/java/com/lingniu/ingest/gateway/CommandResult.java deleted file mode 100644 index d9ed98e0..00000000 --- a/modules/apps/command-gateway/src/main/java/com/lingniu/ingest/gateway/CommandResult.java +++ /dev/null @@ -1,24 +0,0 @@ -package com.lingniu.ingest.gateway; - -/** - * 统一命令响应 envelope。 - * - * @param success 是否成功 - * @param code 业务状态码（0=成功，其它=失败原因） - * @param message 人类可读消息 - * @param data 可选负载（设备应答对象、JSON 节点等） - */ -public record CommandResult(boolean success, int code, String message, Object data) { - - public static CommandResult ok(Object data) { - return new CommandResult(true, 0, "ok", data); - } - - public static CommandResult notImplemented(String hint) { - return new CommandResult(false, 501, "not_implemented: " + hint, null); - } - - public static CommandResult failure(int code, String message) { - return new CommandResult(false, code, message, null); - } -} diff --git a/modules/apps/command-gateway/src/main/java/com/lingniu/ingest/gateway/TerminalCommandController.java b/modules/apps/command-gateway/src/main/java/com/lingniu/ingest/gateway/TerminalCommandController.java deleted file mode 100644 index 5215c4d1..00000000 --- a/modules/apps/command-gateway/src/main/java/com/lingniu/ingest/gateway/TerminalCommandController.java +++ /dev/null @@ -1,281 +0,0 @@ -package com.lingniu.ingest.gateway; - -import com.lingniu.ingest.protocol.jt808.downlink.Jt808Commands; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import com.lingniu.ingest.protocol.jt1078.downlink.Jt1078Commands; -import com.lingniu.ingest.session.CommandDispatcher; -import com.lingniu.ingest.session.DeviceSession; -import com.lingniu.ingest.session.SessionStore; -import org.springframework.web.bind.annotation.DeleteMapping; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.PostMapping; -import org.springframework.web.bind.annotation.PutMapping; -import org.springframework.web.bind.annotation.RequestBody; -import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.RequestParam; -import org.springframework.web.bind.annotation.RestController; - -import java.nio.charset.StandardCharsets; -import java.time.Duration; -import java.util.List; -import java.util.Map; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionException; -import java.util.stream.Collectors; - -/** - * 下行命令 REST 入口。命令通过 {@link CommandDispatcher} 发往协议模块， - * 由 protocol-jt808 的 {@code Jt808CommandDispatcher} 实际发送到终端。 - * - *

路径参数 {@code clientId} 可以是 sessionId / phone / vin， - * 由 {@link SessionStore} 三索引解析。 - */ -@RestController -@RequestMapping("/terminal") -public class TerminalCommandController { - - private static final Duration DEFAULT_TIMEOUT = Duration.ofSeconds(20); - - private final SessionStore sessions; - private final CommandDispatcher dispatcher; - - public TerminalCommandController(SessionStore sessions, CommandDispatcher dispatcher) { - this.sessions = sessions; - this.dispatcher = dispatcher; - } - - @GetMapping("/session") - public CommandResult session(@RequestParam String clientId) { - Optional s = resolveSession(clientId); - return s.map(CommandResult::ok) - .orElseGet(() -> CommandResult.failure(404, "session not found: " + clientId)); - } - - @GetMapping("/location") - public CommandResult queryLocation(@RequestParam String clientId) { - return awaitSync(clientId, Jt808Commands.queryLocation(), "query-location"); - } - - @GetMapping("/parameters") - public CommandResult queryParameters(@RequestParam String clientId) { - return awaitSync(clientId, Jt808Commands.queryParameters(), "query-parameters"); - } - - @PutMapping("/parameters") - public CommandResult setParameters(@RequestParam String clientId, - @RequestBody Map body) { - List items = body.entrySet().stream() - .map(e -> new Jt808Commands.ParamItem( - parseId(e.getKey()), - e.getValue() == null ? new byte[0] : e.getValue().getBytes(StandardCharsets.UTF_8))) - .collect(Collectors.toList()); - return awaitSync(clientId, Jt808Commands.setParameters(items), "set-parameters"); - } - - @PostMapping("/control") - public CommandResult terminalControl(@RequestParam String clientId, - @RequestBody Map body) { - int word = Integer.parseInt(body.getOrDefault("command", "0").toString()); - String params = body.getOrDefault("params", "").toString(); - return awaitSync(clientId, Jt808Commands.terminalControl(word, params), "terminal-control"); - } - - @PostMapping("/ack") - public CommandResult ack(@RequestParam String clientId, @RequestBody Map body) { - int ackSerial = body.getOrDefault("ackSerial", 0); - int ackMsgId = body.getOrDefault("ackMsgId", 0); - int result = body.getOrDefault("result", 0); - return fireAndForget(clientId, Jt808Commands.platformAck(ackSerial, ackMsgId, result), "platform-ack"); - } - - @GetMapping("/attributes") - public CommandResult attributes(@RequestParam String clientId) { - return awaitSync(clientId, Jt808Commands.queryAttributes(), "query-attributes"); - } - - @DeleteMapping("/area") - public CommandResult deleteArea(@RequestParam String clientId, @RequestParam String ids) { - return awaitSync(clientId, Jt808Commands.deleteArea(parseIds(ids)), "delete-area"); - } - - @PostMapping("/alarm_ack") - public CommandResult alarmAck(@RequestParam String clientId, @RequestBody Map body) { - int responseSerialNo = Integer.parseInt(body.getOrDefault("responseSerialNo", "0").toString()); - long type = parseLong(body.getOrDefault("type", "0").toString()); - return fireAndForget(clientId, Jt808Commands.alarmAck(responseSerialNo, type), "alarm-ack"); - } - - @GetMapping("/jt1078/attributes") - public CommandResult jt1078Attributes(@RequestParam String clientId) { - return awaitSync(clientId, Jt1078Commands.queryMediaAttributes(), "jt1078-query-media-attributes"); - } - - @PostMapping("/jt1078/realtime") - public CommandResult jt1078RealtimePlay(@RequestParam String clientId, - @RequestBody Map body) { - return awaitSync(clientId, Jt1078Commands.realtimePlay( - stringValue(body, "ip", ""), - intValue(body, "tcpPort", 0), - intValue(body, "udpPort", 0), - intValue(body, "channelNo", 0), - intValue(body, "mediaType", 0), - intValue(body, "streamType", 0)), "jt1078-realtime-play"); - } - - @PostMapping("/jt1078/realtime_control") - public CommandResult jt1078RealtimeControl(@RequestParam String clientId, - @RequestBody Map body) { - return awaitSync(clientId, Jt1078Commands.realtimeControl( - intValue(body, "channelNo", 0), - intValue(body, "command", 0), - intValue(body, "closeType", 0), - intValue(body, "streamType", 0)), "jt1078-realtime-control"); - } - - @PostMapping("/jt1078/history") - public CommandResult jt1078HistoryPlay(@RequestParam String clientId, - @RequestBody Map body) { - return awaitSync(clientId, Jt1078Commands.historyPlay( - stringValue(body, "ip", ""), - intValue(body, "tcpPort", 0), - intValue(body, "udpPort", 0), - intValue(body, "channelNo", 0), - intValue(body, "mediaType", 0), - intValue(body, "streamType", 0), - intValue(body, "storageType", 0), - intValue(body, "playbackMode", 0), - intValue(body, "playbackSpeed", 0), - stringValue(body, "startTime", ""), - stringValue(body, "endTime", "")), "jt1078-history-play"); - } - - @PostMapping("/jt1078/history_control") - public CommandResult jt1078HistoryControl(@RequestParam String clientId, - @RequestBody Map body) { - return awaitSync(clientId, Jt1078Commands.historyControl( - intValue(body, "channelNo", 0), - intValue(body, "playbackMode", 0), - intValue(body, "playbackSpeed", 0), - stringValue(body, "playbackTime", "")), "jt1078-history-control"); - } - - @PostMapping("/jt1078/resource_search") - public CommandResult jt1078ResourceSearch(@RequestParam String clientId, - @RequestBody Map body) { - return awaitSync(clientId, Jt1078Commands.resourceSearch( - intValue(body, "channelNo", 0), - stringValue(body, "startTime", ""), - stringValue(body, "endTime", ""), - longValue(body, "warnBit1", 0), - longValue(body, "warnBit2", 0), - intValue(body, "mediaType", 0), - intValue(body, "streamType", 0), - intValue(body, "storageType", 0)), "jt1078-resource-search"); - } - - @PostMapping("/jt1078/file_upload") - public CommandResult jt1078FileUpload(@RequestParam String clientId, - @RequestBody Map body) { - return awaitSync(clientId, Jt1078Commands.fileUpload( - stringValue(body, "ip", ""), - intValue(body, "port", 0), - stringValue(body, "username", ""), - stringValue(body, "password", ""), - stringValue(body, "path", ""), - intValue(body, "channelNo", 0), - stringValue(body, "startTime", ""), - stringValue(body, "endTime", ""), - longValue(body, "warnBit1", 0), - longValue(body, "warnBit2", 0), - intValue(body, "mediaType", 0), - intValue(body, "streamType", 0), - intValue(body, "storageType", 0), - intValue(body, "condition", 0)), "jt1078-file-upload"); - } - - @PostMapping("/jt1078/file_upload_control") - public CommandResult jt1078FileUploadControl(@RequestParam String clientId, - @RequestBody Map body) { - return awaitSync(clientId, Jt1078Commands.fileUploadControl( - intValue(body, "responseSerialNo", 0), - intValue(body, "command", 0)), "jt1078-file-upload-control"); - } - - // ===== helpers ===== - - private Optional resolveSession(String clientId) { - // 运维侧常拿到 VIN、手机号或 sessionId 中任意一个；这里按三索引兜底解析到在线连接。 - return sessions.findBySessionId(clientId) - .or(() -> sessions.findByPhone(clientId)) - .or(() -> sessions.findByVin(clientId)); - } - - private CommandResult awaitSync(String clientId, Jt808Commands.DownlinkCommand cmd, String op) { - // 需要终端应答的命令走 request-response，超时时间由网关统一兜底，避免 HTTP 线程无限等待。 - CompletableFuture future = - dispatcher.request(clientId, cmd, Jt808Message.class, DEFAULT_TIMEOUT); - try { - Jt808Message resp = future.join(); - return CommandResult.ok(Map.of( - "messageId", "0x" + Integer.toHexString(resp.header().messageId()), - "serial", resp.header().serialNo(), - "phone", resp.header().phone())); - } catch (CompletionException e) { - Throwable cause = e.getCause() != null ? e.getCause() : e; - return CommandResult.failure(500, op + " failed: " + cause.getMessage()); - } catch (Exception e) { - return CommandResult.failure(500, op + " failed: " + e.getMessage()); - } - } - - private CommandResult fireAndForget(String clientId, Jt808Commands.DownlinkCommand cmd, String op) { - try { - // ACK 类命令不等待终端回包；只确认写入在线 session 对应的 channel。 - dispatcher.notify(clientId, cmd).join(); - return CommandResult.ok(Map.of("op", op)); - } catch (Exception e) { - return CommandResult.failure(500, op + " failed: " + e.getMessage()); - } - } - - private static long parseId(String key) { - if (key == null || key.isBlank()) return 0; - String k = key.startsWith("0x") ? key.substring(2) : key; - return Long.parseLong(k, 16); - } - - private static List parseIds(String ids) { - if (ids == null || ids.isBlank()) { - return List.of(); - } - return java.util.Arrays.stream(ids.split(",")) - .map(String::trim) - .filter(s -> !s.isBlank()) - .map(TerminalCommandController::parseLong) - .toList(); - } - - private static long parseLong(String value) { - if (value == null || value.isBlank()) return 0; - String v = value.startsWith("0x") || value.startsWith("0X") ? value.substring(2) : value; - return value.startsWith("0x") || value.startsWith("0X") - ? Long.parseLong(v, 16) - : Long.parseLong(v); - } - - private static String stringValue(Map body, String key, String defaultValue) { - Object value = body.get(key); - return value == null ? defaultValue : value.toString(); - } - - private static int intValue(Map body, String key, int defaultValue) { - Object value = body.get(key); - return value == null ? defaultValue : (int) parseLong(value.toString()); - } - - private static long longValue(Map body, String key, long defaultValue) { - Object value = body.get(key); - return value == null ? defaultValue : parseLong(value.toString()); - } -} diff --git a/modules/apps/command-gateway/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/apps/command-gateway/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index ac81ba27..00000000 --- a/modules/apps/command-gateway/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.gateway.CommandGatewayAutoConfiguration diff --git a/modules/apps/command-gateway/src/test/java/com/lingniu/ingest/gateway/CommandGatewayAutoConfigurationTest.java b/modules/apps/command-gateway/src/test/java/com/lingniu/ingest/gateway/CommandGatewayAutoConfigurationTest.java deleted file mode 100644 index 9f162bbe..00000000 --- a/modules/apps/command-gateway/src/test/java/com/lingniu/ingest/gateway/CommandGatewayAutoConfigurationTest.java +++ /dev/null @@ -1,62 +0,0 @@ -package com.lingniu.ingest.gateway; - -import com.lingniu.ingest.session.CommandDispatcher; -import com.lingniu.ingest.session.DeviceSession; -import com.lingniu.ingest.session.SessionStore; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; - -import java.time.Duration; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.function.UnaryOperator; - -import static org.assertj.core.api.Assertions.assertThat; - -class CommandGatewayAutoConfigurationTest { - - private final ApplicationContextRunner contextRunner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(CommandGatewayAutoConfiguration.class)) - .withBean(SessionStore.class, EmptySessionStore::new) - .withBean(CommandDispatcher.class, NoopCommandDispatcher::new); - - @Test - void backsOffByDefault() { - contextRunner.run(context -> assertThat(context).doesNotHaveBean(TerminalCommandController.class)); - } - - @Test - void createsControllerWhenExplicitlyEnabled() { - contextRunner - .withPropertyValues("lingniu.ingest.command-gateway.enabled=true") - .run(context -> assertThat(context).hasSingleBean(TerminalCommandController.class)); - } - - private static final class NoopCommandDispatcher implements CommandDispatcher { - @Override - public CompletableFuture notify(String sessionId, Object command) { - return CompletableFuture.completedFuture(null); - } - - @Override - public CompletableFuture request(String sessionId, - Object command, - Class responseType, - Duration timeout) { - return CompletableFuture.failedFuture(new UnsupportedOperationException("not used")); - } - } - - private static final class EmptySessionStore implements SessionStore { - @Override public void put(DeviceSession session) {} - @Override public Optional findBySessionId(String sessionId) { return Optional.empty(); } - @Override public Optional findByVin(String vin) { return Optional.empty(); } - @Override public Optional findByPhone(String phone) { return Optional.empty(); } - @Override public Optional update(String sessionId, UnaryOperator updater) { - return Optional.empty(); - } - @Override public void remove(String sessionId) {} - @Override public int size() { return 0; } - } -} diff --git a/modules/apps/command-gateway/src/test/java/com/lingniu/ingest/gateway/TerminalCommandControllerTest.java b/modules/apps/command-gateway/src/test/java/com/lingniu/ingest/gateway/TerminalCommandControllerTest.java deleted file mode 100644 index 7f18f127..00000000 --- a/modules/apps/command-gateway/src/test/java/com/lingniu/ingest/gateway/TerminalCommandControllerTest.java +++ /dev/null @@ -1,147 +0,0 @@ -package com.lingniu.ingest.gateway; - -import com.lingniu.ingest.protocol.jt808.downlink.Jt808Commands; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import com.lingniu.ingest.session.CommandDispatcher; -import com.lingniu.ingest.session.DeviceSession; -import com.lingniu.ingest.session.SessionStore; -import org.junit.jupiter.api.Test; - -import java.time.Duration; -import java.util.Map; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.function.UnaryOperator; - -import static org.assertj.core.api.Assertions.assertThat; - -class TerminalCommandControllerTest { - - @Test - void queryAttributesDispatchesJt8088107InsteadOfNotImplemented() { - RecordingDispatcher dispatcher = new RecordingDispatcher(); - TerminalCommandController controller = new TerminalCommandController(new EmptySessionStore(), dispatcher); - - CommandResult result = controller.attributes("13800138000"); - - assertThat(result.success()).isTrue(); - assertThat(dispatcher.lastRequestCommand.messageId()).isEqualTo(0x8107); - assertThat(dispatcher.lastRequestCommand.body()).isEmpty(); - } - - @Test - void alarmAckDispatchesJt8088203WithResponseSerialAndAlarmType() { - RecordingDispatcher dispatcher = new RecordingDispatcher(); - TerminalCommandController controller = new TerminalCommandController(new EmptySessionStore(), dispatcher); - - CommandResult result = controller.alarmAck("13800138000", Map.of( - "responseSerialNo", 0x1234, - "type", 0x0000_0001)); - - assertThat(result.success()).isTrue(); - assertThat(dispatcher.lastNotifyCommand.messageId()).isEqualTo(0x8203); - assertThat(dispatcher.lastNotifyCommand.body()).containsExactly( - 0x12, 0x34, - 0x00, 0x00, 0x00, 0x01); - } - - @Test - void deleteAreaDispatchesJt8088601WithCommaSeparatedAreaIds() { - RecordingDispatcher dispatcher = new RecordingDispatcher(); - TerminalCommandController controller = new TerminalCommandController(new EmptySessionStore(), dispatcher); - - CommandResult result = controller.deleteArea("13800138000", "1,0x01020304"); - - assertThat(result.success()).isTrue(); - assertThat(dispatcher.lastRequestCommand.messageId()).isEqualTo(0x8601); - assertThat(dispatcher.lastRequestCommand.body()).containsExactly( - 0x02, - 0x00, 0x00, 0x00, 0x01, - 0x01, 0x02, 0x03, 0x04); - } - - @Test - void jt1078RealtimePlayDispatches9101ThroughSharedDispatcher() { - RecordingDispatcher dispatcher = new RecordingDispatcher(); - TerminalCommandController controller = new TerminalCommandController(new EmptySessionStore(), dispatcher); - - CommandResult result = controller.jt1078RealtimePlay("13800138000", Map.of( - "ip", "10.1.2.3", - "tcpPort", 11078, - "udpPort", 11079, - "channelNo", 2, - "mediaType", 1, - "streamType", 0)); - - assertThat(result.success()).isTrue(); - assertThat(dispatcher.lastRequestCommand.messageId()).isEqualTo(0x9101); - assertThat(dispatcher.lastRequestCommand.body()).containsExactly( - 0x08, '1', '0', '.', '1', '.', '2', '.', '3', - 0x2B, 0x46, - 0x2B, 0x47, - 0x02, - 0x01, - 0x00); - } - - @Test - void jt1078ResourceSearchDispatches9205AndAwaitsFileListResponse() { - RecordingDispatcher dispatcher = new RecordingDispatcher(); - TerminalCommandController controller = new TerminalCommandController(new EmptySessionStore(), dispatcher); - - CommandResult result = controller.jt1078ResourceSearch("13800138000", Map.of( - "channelNo", 3, - "startTime", "2026-06-22 08:09:10", - "endTime", "260622180000", - "warnBit1", "0x01020304", - "warnBit2", "0x05060708", - "mediaType", 3, - "streamType", 2, - "storageType", 1)); - - assertThat(result.success()).isTrue(); - assertThat(dispatcher.lastRequestCommand.messageId()).isEqualTo(0x9205); - assertThat(dispatcher.lastRequestCommand.body()).containsExactly( - 0x03, - 0x26, 0x06, 0x22, 0x08, 0x09, 0x10, - 0x26, 0x06, 0x22, 0x18, 0x00, 0x00, - 0x01, 0x02, 0x03, 0x04, - 0x05, 0x06, 0x07, 0x08, - 0x03, - 0x02, - 0x01); - } - - private static final class RecordingDispatcher implements CommandDispatcher { - private Jt808Commands.DownlinkCommand lastNotifyCommand; - private Jt808Commands.DownlinkCommand lastRequestCommand; - - @Override - public CompletableFuture notify(String sessionId, Object command) { - this.lastNotifyCommand = (Jt808Commands.DownlinkCommand) command; - return CompletableFuture.completedFuture(null); - } - - @Override - @SuppressWarnings("unchecked") - public CompletableFuture request(String sessionId, Object command, Class responseType, Duration timeout) { - this.lastRequestCommand = (Jt808Commands.DownlinkCommand) command; - Jt808Message response = new Jt808Message( - new Jt808Header(0x0107, 0, 0, false, Jt808Header.ProtocolVersion.V2013, - "13800138000", 1, 0, 0), - new com.lingniu.ingest.protocol.jt808.model.Jt808Body.Raw(0x0107, new byte[0])); - return CompletableFuture.completedFuture((T) response); - } - } - - private static final class EmptySessionStore implements SessionStore { - @Override public void put(DeviceSession session) {} - @Override public Optional findBySessionId(String sessionId) { return Optional.empty(); } - @Override public Optional findByVin(String vin) { return Optional.empty(); } - @Override public Optional findByPhone(String phone) { return Optional.empty(); } - @Override public Optional update(String sessionId, UnaryOperator updater) { return Optional.empty(); } - @Override public void remove(String sessionId) {} - @Override public int size() { return 0; } - } -} diff --git a/modules/apps/gb32960-ingest-app/pom.xml b/modules/apps/gb32960-ingest-app/pom.xml deleted file mode 100644 index 337adec8..00000000 --- a/modules/apps/gb32960-ingest-app/pom.xml +++ /dev/null @@ -1,68 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - - gb32960-ingest-app - gb32960-ingest-app - GB32960 protocol ingress runtime: TCP decode, auth, Kafka production, and ACK. - - - - com.lingniu.ingest - ingest-core - - - com.lingniu.ingest - session-core - - - com.lingniu.ingest - observability - - - com.lingniu.ingest - vehicle-identity - - - com.lingniu.ingest - protocol-gb32960 - - - com.lingniu.ingest - sink-kafka - - - com.lingniu.ingest - sink-archive - - - org.springframework.boot - spring-boot-starter-web - - - com.alibaba.cloud - spring-cloud-starter-alibaba-nacos-config - - - org.springframework.boot - spring-boot-starter-test - test - - - - - gb32960-ingest-app - - - org.springframework.boot - spring-boot-maven-plugin - - - - diff --git a/modules/apps/gb32960-ingest-app/src/main/java/com/lingniu/ingest/gb32960app/Gb32960IngestApplication.java b/modules/apps/gb32960-ingest-app/src/main/java/com/lingniu/ingest/gb32960app/Gb32960IngestApplication.java deleted file mode 100644 index 35019bbb..00000000 --- a/modules/apps/gb32960-ingest-app/src/main/java/com/lingniu/ingest/gb32960app/Gb32960IngestApplication.java +++ /dev/null @@ -1,21 +0,0 @@ -package com.lingniu.ingest.gb32960app; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.boot.SpringApplication; -import org.springframework.boot.autoconfigure.SpringBootApplication; - -@SpringBootApplication -public class Gb32960IngestApplication { - - private static final Logger log = LoggerFactory.getLogger(Gb32960IngestApplication.class); - - public static void main(String[] args) { - try { - SpringApplication.run(Gb32960IngestApplication.class, args); - } catch (Throwable t) { - log.error("GB32960 ingest application failed to start, forcing JVM exit", t); - System.exit(1); - } - } -} diff --git a/modules/apps/gb32960-ingest-app/src/main/java/com/lingniu/ingest/gb32960app/actuator/Gb32960SessionsEndpoint.java b/modules/apps/gb32960-ingest-app/src/main/java/com/lingniu/ingest/gb32960app/actuator/Gb32960SessionsEndpoint.java deleted file mode 100644 index 218e08c8..00000000 --- a/modules/apps/gb32960-ingest-app/src/main/java/com/lingniu/ingest/gb32960app/actuator/Gb32960SessionsEndpoint.java +++ /dev/null @@ -1,27 +0,0 @@ -package com.lingniu.ingest.gb32960app.actuator; - -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960ConnectionDiagnostics; -import org.springframework.boot.actuate.endpoint.annotation.Endpoint; -import org.springframework.boot.actuate.endpoint.annotation.ReadOperation; -import org.springframework.stereotype.Component; - -import java.util.Map; - -@Component -@Endpoint(id = "gb32960sessions") -public class Gb32960SessionsEndpoint { - - private final Gb32960ConnectionDiagnostics diagnostics; - - public Gb32960SessionsEndpoint(Gb32960ConnectionDiagnostics diagnostics) { - this.diagnostics = diagnostics; - } - - @ReadOperation - public Map sessions() { - var sessions = diagnostics.activeSessions(); - return Map.of( - "activeCount", sessions.size(), - "sessions", sessions); - } -} diff --git a/modules/apps/gb32960-ingest-app/src/main/resources/application.yml b/modules/apps/gb32960-ingest-app/src/main/resources/application.yml deleted file mode 100644 index ea6c320c..00000000 --- a/modules/apps/gb32960-ingest-app/src/main/resources/application.yml +++ /dev/null @@ -1,137 +0,0 @@ -spring: - application: - name: gb32960-ingest-app - config: - import: - - optional:nacos:${spring.application.name}.${NACOS_CONFIG_FILE_EXTENSION:yml}?group=${NACOS_GROUP:DEFAULT_GROUP}&refreshEnabled=${NACOS_REFRESH_ENABLED:true} - cloud: - nacos: - config: - enabled: ${NACOS_CONFIG_ENABLED:true} - server-addr: ${NACOS_SERVER_ADDR:127.0.0.1:8848} - namespace: ${NACOS_NAMESPACE:} - group: ${NACOS_GROUP:DEFAULT_GROUP} - file-extension: ${NACOS_CONFIG_FILE_EXTENSION:yml} - username: ${NACOS_USERNAME:} - password: ${NACOS_PASSWORD:} - threads: - virtual: - enabled: true - data: - redis: - host: ${REDIS_HOST:127.0.0.1} - port: ${REDIS_PORT:6379} - username: ${REDIS_USERNAME:} - password: ${REDIS_PASSWORD:} - database: ${REDIS_DATABASE:50} - -server: - port: ${HTTP_PORT:20100} - -lingniu: - ingest: - gb32960: - enabled: true - server: - enabled: true - port: ${GB32960_PORT:32960} - boss-threads: ${GB32960_BOSS_THREADS:1} - worker-threads: ${GB32960_WORKER_THREADS:0} - auth: - enabled: ${GB32960_AUTH_ENABLED:false} - case-sensitive: false - whitelist: [] - platforms: - - username: ${GB32960_PLATFORM_USER_HYUNDAI:Hyundai} - password: ${GB32960_PLATFORM_PWD_HYUNDAI:} - allowed-ips: - - ${GB32960_PLATFORM_IP_HYUNDAI:} - description: 外部下级平台 - Hyundai - - username: ${GB32960_PLATFORM_USER_YUEJIN:YueJin} - password: ${GB32960_PLATFORM_PWD_YUEJIN:} - allowed-ips: - - ${GB32960_PLATFORM_IP_YUEJIN:} - description: 外部下级平台 - YueJin - tls: - enabled: ${GB32960_TLS_ENABLED:false} - cert-path: ${GB32960_TLS_CERT:} - key-path: ${GB32960_TLS_KEY:} - trust-cert-path: ${GB32960_TLS_CA:} - require-client-auth: true - vendor-extensions: - - name: guangdong-fc - match: - platform-accounts: - - Hyundai - - YueJin - vin-prefixes: [] - vins: [] - parse: - lenient-block-failure: true - diagnostics: - max-logged-frame-keys-per-channel: ${GB32960_DIAGNOSTICS_MAX_LOGGED_FRAME_KEYS_PER_CHANNEL:4096} - pipeline: - disruptor: - ring-buffer-size: ${PIPELINE_RING_BUFFER_SIZE:131072} - wait-strategy: ${PIPELINE_WAIT_STRATEGY:yielding} - producer-type: multi - dedup: - enabled: true - cache-size: 200000 - ttl-seconds: 600 - rate-limit: - per-vin-qps: 50 - session: - ttl: ${SESSION_TTL:30m} - identity: - mysql: - jdbc-url: ${VEHICLE_IDENTITY_MYSQL_JDBC_URL:jdbc:mysql://127.0.0.1:3306/lingniu_vehicle?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai} - username: ${VEHICLE_IDENTITY_MYSQL_USERNAME:root} - password: ${VEHICLE_IDENTITY_MYSQL_PASSWORD:} - table-name: ${VEHICLE_IDENTITY_MYSQL_TABLE:vehicle_identity_bindings} - initialize-schema: ${VEHICLE_IDENTITY_MYSQL_INITIALIZE_SCHEMA:true} - sink: - kafka: - enabled: true - bootstrap-servers: ${KAFKA_BROKERS:114.55.58.251:9092} - compression-type: zstd - linger-ms: 20 - batch-size: 65536 - acks: all - enable-idempotence: true - node-id: ${KAFKA_NODE_ID:gb32960-ingest-local} - topics: - realtime: ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - location: ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - alarm: ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - session: ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - media-meta: ${KAFKA_TOPIC_MEDIA_META:vehicle.media.meta.v1} - raw-archive: ${KAFKA_TOPIC_GB32960_RAW:vehicle.raw.gb32960.v1} - dlq: ${KAFKA_TOPIC_GB32960_DLQ:vehicle.dlq.gb32960.v1} - consumer: - enabled: false - archive: - enabled: ${SINK_ARCHIVE_ENABLED:true} - type: local - path: ${SINK_ARCHIVE_PATH:./archive/} - -management: - endpoints: - web: - exposure: - include: health,info,metrics,prometheus,gb32960sessions - endpoint: - health: - probes: - enabled: ${MANAGEMENT_HEALTH_PROBES_ENABLED:true} - health: - redis: - enabled: ${MANAGEMENT_HEALTH_REDIS_ENABLED:false} - metrics: - tags: - application: gb32960-ingest-app - -logging: - level: - root: INFO - com.lingniu.ingest: INFO diff --git a/modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppCompositionTest.java b/modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppCompositionTest.java deleted file mode 100644 index c863bda9..00000000 --- a/modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppCompositionTest.java +++ /dev/null @@ -1,78 +0,0 @@ -package com.lingniu.ingest.gb32960app; - -import com.lingniu.ingest.core.config.IngestCoreAutoConfiguration; -import com.lingniu.ingest.identity.MySqlVehicleIdentityService; -import com.lingniu.ingest.identity.config.VehicleIdentityAutoConfiguration; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.config.Gb32960AutoConfiguration; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960NettyServer; -import com.lingniu.ingest.session.SessionStore; -import com.lingniu.ingest.session.config.SessionCoreAutoConfiguration; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import com.lingniu.ingest.sink.archive.RawArchiveEventSink; -import com.lingniu.ingest.sink.archive.config.SinkArchiveAutoConfiguration; -import com.lingniu.ingest.sink.kafka.KafkaEventSink; -import com.lingniu.ingest.sink.kafka.KafkaEnvelopeDeadLetterSink; -import com.lingniu.ingest.sink.kafka.KafkaSinkAutoConfiguration; -import org.apache.kafka.clients.producer.KafkaProducer; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.context.ApplicationContext; -import org.springframework.util.ClassUtils; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.mock; - -class Gb32960IngestAppCompositionTest { - - private final ApplicationContextRunner contextRunner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of( - IngestCoreAutoConfiguration.class, - SessionCoreAutoConfiguration.class, - VehicleIdentityAutoConfiguration.class, - KafkaSinkAutoConfiguration.class, - SinkArchiveAutoConfiguration.class, - Gb32960AutoConfiguration.class)) - .withAllowBeanDefinitionOverriding(true) - .withBean("kafkaProducer", KafkaProducer.class, Gb32960IngestAppCompositionTest::kafkaProducer) - .withBean(SessionStore.class, () -> mock(SessionStore.class)) - .withPropertyValues( - "lingniu.ingest.gb32960.enabled=true", - "lingniu.ingest.gb32960.server.enabled=true", - "lingniu.ingest.gb32960.port=0", - "lingniu.ingest.identity.mysql.initialize-schema=false", - "lingniu.ingest.sink.kafka.enabled=true", - "lingniu.ingest.sink.kafka.bootstrap-servers=localhost:9092", - "lingniu.ingest.sink.kafka.consumer.enabled=false", - "lingniu.ingest.sink.archive.enabled=true", - "lingniu.ingest.sink.archive.path=target/test-archive-gb32960", - "lingniu.ingest.event-history.enabled=false"); - - @Test - void createsGb32960ListenerWithoutHistoryStorageBoundaries() { - contextRunner.run(context -> { - assertThat(context).hasSingleBean(Gb32960MessageDecoder.class); - assertThat(context).hasSingleBean(Gb32960NettyServer.class); - assertThat(context).hasSingleBean(MySqlVehicleIdentityService.class); - assertThat(context).hasSingleBean(KafkaEventSink.class); - assertThat(context).hasSingleBean(KafkaEnvelopeDeadLetterSink.class); - assertThat(context).hasSingleBean(ArchiveStore.class); - assertThat(context).hasSingleBean(RawArchiveEventSink.class); - - assertTypeNotPresent(context, "com.lingniu.ingest.eventfilestore.DuckDbParquetEventFileStore"); - assertTypeNotPresent(context, "com.lingniu.ingest.eventfilestore.config.EventFileStoreAutoConfiguration"); - }); - } - - @SuppressWarnings("unchecked") - private static KafkaProducer kafkaProducer() { - return mock(KafkaProducer.class); - } - - private static void assertTypeNotPresent(ApplicationContext context, String className) { - assertThat(ClassUtils.isPresent(className, context.getClassLoader())) - .as("%s must stay outside the GB32960 ingest runtime", className) - .isFalse(); - } -} diff --git a/modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppDefaultsTest.java b/modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppDefaultsTest.java deleted file mode 100644 index 30f285dc..00000000 --- a/modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/Gb32960IngestAppDefaultsTest.java +++ /dev/null @@ -1,91 +0,0 @@ -package com.lingniu.ingest.gb32960app; - -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.config.YamlPropertiesFactoryBean; -import org.springframework.core.io.ClassPathResource; - -import java.io.IOException; -import java.nio.charset.StandardCharsets; -import java.util.Properties; - -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960IngestAppDefaultsTest { - - @Test - void applicationDefaultsKeepGb32960IngestAsTcpKafkaProducerOnly() throws IOException { - Properties properties = applicationProperties(); - String legacyKafkaPrefix = "lingniu.ingest.sink." + "mq."; - String kafkaTypeProperty = "lingniu.ingest.sink.kafka." + "type"; - - assertThat(properties) - .containsEntry("spring.application.name", "gb32960-ingest-app") - .containsEntry( - "spring.config.import[0]", - "optional:nacos:${spring.application.name}.${NACOS_CONFIG_FILE_EXTENSION:yml}?group=${NACOS_GROUP:DEFAULT_GROUP}&refreshEnabled=${NACOS_REFRESH_ENABLED:true}") - .containsEntry("spring.cloud.nacos.config.server-addr", "${NACOS_SERVER_ADDR:127.0.0.1:8848}") - .containsEntry("spring.data.redis.host", "${REDIS_HOST:127.0.0.1}") - .containsEntry("spring.data.redis.port", "${REDIS_PORT:6379}") - .containsEntry("spring.data.redis.database", "${REDIS_DATABASE:50}") - .containsEntry("lingniu.ingest.gb32960.enabled", true) - .containsEntry("lingniu.ingest.gb32960.server.enabled", true) - .containsEntry("lingniu.ingest.gb32960.port", "${GB32960_PORT:32960}") - .containsEntry("lingniu.ingest.gb32960.auth.platforms[0].username", - "${GB32960_PLATFORM_USER_HYUNDAI:Hyundai}") - .containsEntry("lingniu.ingest.gb32960.auth.platforms[0].password", - "${GB32960_PLATFORM_PWD_HYUNDAI:}") - .containsEntry("lingniu.ingest.gb32960.auth.platforms[1].username", - "${GB32960_PLATFORM_USER_YUEJIN:YueJin}") - .containsEntry("lingniu.ingest.gb32960.auth.platforms[1].password", - "${GB32960_PLATFORM_PWD_YUEJIN:}") - .containsEntry("lingniu.ingest.gb32960.vendor-extensions[0].match.platform-accounts[0]", - "Hyundai") - .containsEntry("lingniu.ingest.gb32960.vendor-extensions[0].match.platform-accounts[1]", - "YueJin") - .containsEntry("lingniu.ingest.sink.kafka.enabled", true) - .containsEntry("lingniu.ingest.sink.kafka.consumer.enabled", false) - .containsEntry("lingniu.ingest.sink.archive.enabled", "${SINK_ARCHIVE_ENABLED:true}") - .containsEntry("lingniu.ingest.sink.archive.path", "${SINK_ARCHIVE_PATH:./archive/}") - .containsEntry("lingniu.ingest.identity.mysql.table-name", - "${VEHICLE_IDENTITY_MYSQL_TABLE:vehicle_identity_bindings}") - .containsEntry("management.endpoints.web.exposure.include", - "health,info,metrics,prometheus,gb32960sessions"); - - assertThat(properties.stringPropertyNames()) - .noneMatch(name -> name.startsWith("lingniu.ingest.event-history.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.event-file-store.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.vehicle-state.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.vehicle-stat.")) - .noneMatch(name -> name.startsWith(legacyKafkaPrefix)) - .noneMatch(name -> name.equals(kafkaTypeProperty)) - .noneMatch(name -> name.equals("lingniu.ingest.session.store")) - .noneMatch(name -> name.equals("lingniu.ingest.identity.store")); - assertThat(applicationYaml()) - .doesNotContain("event-history:") - .doesNotContain("event-file-store:") - .doesNotContain("vehicle-state:") - .doesNotContain("vehicle-stat:") - .doesNotContain("springdoc:"); - assertThat(applicationYaml()) - .contains("password: ${GB32960_PLATFORM_PWD_HYUNDAI:}") - .contains("password: ${GB32960_PLATFORM_PWD_YUEJIN:}") - .doesNotContain("SESSION_STORE") - .doesNotContain("VEHICLE_IDENTITY_STORE") - .doesNotContain("KAFKA_ENABLED"); - } - - private static Properties applicationProperties() { - YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean(); - factory.setResources(new ClassPathResource("application.yml")); - - Properties properties = factory.getObject(); - - assertThat(properties).isNotNull(); - return properties; - } - - private static String applicationYaml() throws IOException { - return new String(new ClassPathResource("application.yml").getInputStream().readAllBytes(), - StandardCharsets.UTF_8); - } -} diff --git a/modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/actuator/Gb32960SessionsEndpointTest.java b/modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/actuator/Gb32960SessionsEndpointTest.java deleted file mode 100644 index 72d475b8..00000000 --- a/modules/apps/gb32960-ingest-app/src/test/java/com/lingniu/ingest/gb32960app/actuator/Gb32960SessionsEndpointTest.java +++ /dev/null @@ -1,33 +0,0 @@ -package com.lingniu.ingest.gb32960app.actuator; - -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960ConnectionDiagnostics; -import com.lingniu.ingest.protocol.gb32960.model.CommandType; -import org.junit.jupiter.api.Test; - -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960SessionsEndpointTest { - - @Test - void returnsActiveGb32960Sessions() { - Gb32960ConnectionDiagnostics diagnostics = new Gb32960ConnectionDiagnostics(); - diagnostics.opened("session-1", "127.0.0.1:65469"); - diagnostics.received("session-1", CommandType.PLATFORM_LOGIN, ""); - diagnostics.platformLoginAccepted("session-1", "Hyundai"); - - Gb32960SessionsEndpoint endpoint = new Gb32960SessionsEndpoint(diagnostics); - - Map result = endpoint.sessions(); - - assertThat(result).containsEntry("activeCount", 1); - assertThat(result.get("sessions")).isInstanceOf(List.class); - assertThat((List) result.get("sessions")).singleElement() - .hasFieldOrPropertyWithValue("sessionId", "session-1") - .hasFieldOrPropertyWithValue("peer", "127.0.0.1:65469") - .hasFieldOrPropertyWithValue("lastCommand", "PLATFORM_LOGIN") - .hasFieldOrPropertyWithValue("platformAccount", "Hyundai"); - } -} diff --git a/modules/apps/jt808-ingest-app/pom.xml b/modules/apps/jt808-ingest-app/pom.xml deleted file mode 100644 index fa39adda..00000000 --- a/modules/apps/jt808-ingest-app/pom.xml +++ /dev/null @@ -1,68 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - - jt808-ingest-app - jt808-ingest-app - JT808 protocol ingress runtime: TCP decode, identity mapping, Kafka production, and downlink session routing. - - - - com.lingniu.ingest - ingest-core - - - com.lingniu.ingest - session-core - - - com.lingniu.ingest - observability - - - com.lingniu.ingest - vehicle-identity - - - com.lingniu.ingest - protocol-jt808 - - - com.lingniu.ingest - sink-kafka - - - com.lingniu.ingest - sink-archive - - - org.springframework.boot - spring-boot-starter-web - - - com.alibaba.cloud - spring-cloud-starter-alibaba-nacos-config - - - org.springframework.boot - spring-boot-starter-test - test - - - - - jt808-ingest-app - - - org.springframework.boot - spring-boot-maven-plugin - - - - diff --git a/modules/apps/jt808-ingest-app/src/main/java/com/lingniu/ingest/jt808app/Jt808IngestApplication.java b/modules/apps/jt808-ingest-app/src/main/java/com/lingniu/ingest/jt808app/Jt808IngestApplication.java deleted file mode 100644 index 5a833540..00000000 --- a/modules/apps/jt808-ingest-app/src/main/java/com/lingniu/ingest/jt808app/Jt808IngestApplication.java +++ /dev/null @@ -1,21 +0,0 @@ -package com.lingniu.ingest.jt808app; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.boot.SpringApplication; -import org.springframework.boot.autoconfigure.SpringBootApplication; - -@SpringBootApplication -public class Jt808IngestApplication { - - private static final Logger log = LoggerFactory.getLogger(Jt808IngestApplication.class); - - public static void main(String[] args) { - try { - SpringApplication.run(Jt808IngestApplication.class, args); - } catch (Throwable t) { - log.error("JT808 ingest application failed to start, forcing JVM exit", t); - System.exit(1); - } - } -} diff --git a/modules/apps/jt808-ingest-app/src/main/resources/application.yml b/modules/apps/jt808-ingest-app/src/main/resources/application.yml deleted file mode 100644 index 546e2bf6..00000000 --- a/modules/apps/jt808-ingest-app/src/main/resources/application.yml +++ /dev/null @@ -1,97 +0,0 @@ -spring: - application: - name: jt808-ingest-app - config: - import: - - optional:nacos:${spring.application.name}.${NACOS_CONFIG_FILE_EXTENSION:yml}?group=${NACOS_GROUP:DEFAULT_GROUP}&refreshEnabled=${NACOS_REFRESH_ENABLED:true} - cloud: - nacos: - config: - enabled: ${NACOS_CONFIG_ENABLED:true} - server-addr: ${NACOS_SERVER_ADDR:127.0.0.1:8848} - namespace: ${NACOS_NAMESPACE:} - group: ${NACOS_GROUP:DEFAULT_GROUP} - file-extension: ${NACOS_CONFIG_FILE_EXTENSION:yml} - username: ${NACOS_USERNAME:} - password: ${NACOS_PASSWORD:} - threads: - virtual: - enabled: true - data: - redis: - host: ${REDIS_HOST:127.0.0.1} - port: ${REDIS_PORT:6379} - username: ${REDIS_USERNAME:} - password: ${REDIS_PASSWORD:} - database: ${REDIS_DATABASE:50} - -server: - port: ${HTTP_PORT:20400} - -lingniu: - ingest: - jt808: - enabled: true - port: ${JT808_PORT:808} - worker-threads: ${JT808_WORKER_THREADS:0} - pipeline: - disruptor: - ring-buffer-size: ${PIPELINE_RING_BUFFER_SIZE:131072} - wait-strategy: ${PIPELINE_WAIT_STRATEGY:yielding} - producer-type: multi - dedup: - enabled: true - cache-size: 200000 - ttl-seconds: 600 - rate-limit: - per-vin-qps: 50 - session: - ttl: ${SESSION_TTL:30m} - identity: - mysql: - jdbc-url: ${VEHICLE_IDENTITY_MYSQL_JDBC_URL:jdbc:mysql://127.0.0.1:3306/lingniu_vehicle?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai} - username: ${VEHICLE_IDENTITY_MYSQL_USERNAME:root} - password: ${VEHICLE_IDENTITY_MYSQL_PASSWORD:} - table-name: ${VEHICLE_IDENTITY_MYSQL_TABLE:vehicle_identity_bindings} - initialize-schema: ${VEHICLE_IDENTITY_MYSQL_INITIALIZE_SCHEMA:true} - sink: - kafka: - enabled: true - bootstrap-servers: ${KAFKA_BROKERS:114.55.58.251:9092} - compression-type: zstd - linger-ms: 20 - batch-size: 65536 - acks: all - enable-idempotence: true - node-id: ${KAFKA_NODE_ID:jt808-ingest-local} - topics: - realtime: ${KAFKA_TOPIC_JT808_EVENT:vehicle.event.jt808.v1} - location: ${KAFKA_TOPIC_JT808_EVENT:vehicle.event.jt808.v1} - alarm: ${KAFKA_TOPIC_JT808_EVENT:vehicle.event.jt808.v1} - session: ${KAFKA_TOPIC_JT808_EVENT:vehicle.event.jt808.v1} - media-meta: ${KAFKA_TOPIC_MEDIA_META:vehicle.media.meta.v1} - raw-archive: ${KAFKA_TOPIC_JT808_RAW:vehicle.raw.jt808.v1} - dlq: ${KAFKA_TOPIC_JT808_DLQ:vehicle.dlq.jt808.v1} - consumer: - enabled: false - -management: - endpoints: - web: - exposure: - include: health,info,metrics,prometheus - endpoint: - health: - probes: - enabled: ${MANAGEMENT_HEALTH_PROBES_ENABLED:true} - health: - redis: - enabled: ${MANAGEMENT_HEALTH_REDIS_ENABLED:false} - metrics: - tags: - application: jt808-ingest-app - -logging: - level: - root: INFO - com.lingniu.ingest: INFO diff --git a/modules/apps/jt808-ingest-app/src/test/java/com/lingniu/ingest/jt808app/Jt808IngestAppCompositionTest.java b/modules/apps/jt808-ingest-app/src/test/java/com/lingniu/ingest/jt808app/Jt808IngestAppCompositionTest.java deleted file mode 100644 index 0fd1a278..00000000 --- a/modules/apps/jt808-ingest-app/src/test/java/com/lingniu/ingest/jt808app/Jt808IngestAppCompositionTest.java +++ /dev/null @@ -1,77 +0,0 @@ -package com.lingniu.ingest.jt808app; - -import com.lingniu.ingest.core.config.IngestCoreAutoConfiguration; -import com.lingniu.ingest.identity.MySqlVehicleIdentityService; -import com.lingniu.ingest.identity.config.VehicleIdentityAutoConfiguration; -import com.lingniu.ingest.protocol.jt808.codec.Jt808MessageDecoder; -import com.lingniu.ingest.protocol.jt808.config.Jt808AutoConfiguration; -import com.lingniu.ingest.protocol.jt808.inbound.Jt808NettyServer; -import com.lingniu.ingest.session.SessionStore; -import com.lingniu.ingest.session.config.SessionCoreAutoConfiguration; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import com.lingniu.ingest.sink.archive.RawArchiveEventSink; -import com.lingniu.ingest.sink.archive.config.SinkArchiveAutoConfiguration; -import com.lingniu.ingest.sink.kafka.KafkaEnvelopeDeadLetterSink; -import com.lingniu.ingest.sink.kafka.KafkaEventSink; -import com.lingniu.ingest.sink.kafka.KafkaSinkAutoConfiguration; -import org.apache.kafka.clients.producer.KafkaProducer; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.context.ApplicationContext; -import org.springframework.util.ClassUtils; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.mock; - -class Jt808IngestAppCompositionTest { - - private final ApplicationContextRunner contextRunner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of( - IngestCoreAutoConfiguration.class, - SessionCoreAutoConfiguration.class, - VehicleIdentityAutoConfiguration.class, - KafkaSinkAutoConfiguration.class, - SinkArchiveAutoConfiguration.class, - Jt808AutoConfiguration.class)) - .withAllowBeanDefinitionOverriding(true) - .withBean("kafkaProducer", KafkaProducer.class, Jt808IngestAppCompositionTest::kafkaProducer) - .withBean(SessionStore.class, () -> mock(SessionStore.class)) - .withPropertyValues( - "lingniu.ingest.jt808.enabled=true", - "lingniu.ingest.jt808.port=0", - "lingniu.ingest.identity.mysql.initialize-schema=false", - "lingniu.ingest.sink.kafka.enabled=true", - "lingniu.ingest.sink.kafka.bootstrap-servers=localhost:9092", - "lingniu.ingest.sink.kafka.consumer.enabled=false", - "lingniu.ingest.sink.archive.enabled=true", - "lingniu.ingest.sink.archive.path=target/test-archive-jt808", - "lingniu.ingest.event-history.enabled=false"); - - @Test - void createsJt808ListenerWithoutHistoryStorageBoundaries() { - contextRunner.run(context -> { - assertThat(context).hasSingleBean(Jt808MessageDecoder.class); - assertThat(context).hasSingleBean(Jt808NettyServer.class); - assertThat(context).hasSingleBean(MySqlVehicleIdentityService.class); - assertThat(context).hasSingleBean(KafkaEventSink.class); - assertThat(context).hasSingleBean(KafkaEnvelopeDeadLetterSink.class); - assertThat(context).hasSingleBean(ArchiveStore.class); - assertThat(context).hasSingleBean(RawArchiveEventSink.class); - - assertTypeNotPresent(context, "com.lingniu.ingest.eventfilestore.DuckDbParquetEventFileStore"); - assertTypeNotPresent(context, "com.lingniu.ingest.eventfilestore.config.EventFileStoreAutoConfiguration"); - }); - } - - @SuppressWarnings("unchecked") - private static KafkaProducer kafkaProducer() { - return mock(KafkaProducer.class); - } - - private static void assertTypeNotPresent(ApplicationContext context, String className) { - assertThat(ClassUtils.isPresent(className, context.getClassLoader())) - .as("%s must stay outside the JT808 ingest runtime", className) - .isFalse(); - } -} diff --git a/modules/apps/jt808-ingest-app/src/test/java/com/lingniu/ingest/jt808app/Jt808IngestAppDefaultsTest.java b/modules/apps/jt808-ingest-app/src/test/java/com/lingniu/ingest/jt808app/Jt808IngestAppDefaultsTest.java deleted file mode 100644 index a896cefb..00000000 --- a/modules/apps/jt808-ingest-app/src/test/java/com/lingniu/ingest/jt808app/Jt808IngestAppDefaultsTest.java +++ /dev/null @@ -1,78 +0,0 @@ -package com.lingniu.ingest.jt808app; - -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.config.YamlPropertiesFactoryBean; -import org.springframework.core.io.ClassPathResource; - -import java.io.IOException; -import java.nio.charset.StandardCharsets; -import java.util.Properties; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt808IngestAppDefaultsTest { - - @Test - void applicationDefaultsKeepJt808IngestAsTcpKafkaProducerOnly() throws IOException { - Properties properties = applicationProperties(); - - assertThat(properties) - .containsEntry("spring.application.name", "jt808-ingest-app") - .containsEntry( - "spring.config.import[0]", - "optional:nacos:${spring.application.name}.${NACOS_CONFIG_FILE_EXTENSION:yml}?group=${NACOS_GROUP:DEFAULT_GROUP}&refreshEnabled=${NACOS_REFRESH_ENABLED:true}") - .containsEntry("spring.cloud.nacos.config.server-addr", "${NACOS_SERVER_ADDR:127.0.0.1:8848}") - .containsEntry("spring.data.redis.host", "${REDIS_HOST:127.0.0.1}") - .containsEntry("spring.data.redis.port", "${REDIS_PORT:6379}") - .containsEntry("spring.data.redis.database", "${REDIS_DATABASE:50}") - .containsEntry("server.port", "${HTTP_PORT:20400}") - .containsEntry("lingniu.ingest.jt808.enabled", true) - .containsEntry("lingniu.ingest.jt808.port", "${JT808_PORT:808}") - .containsEntry("lingniu.ingest.sink.kafka.enabled", true) - .containsEntry("lingniu.ingest.sink.kafka.consumer.enabled", false) - .containsEntry("lingniu.ingest.sink.kafka.topics.realtime", "${KAFKA_TOPIC_JT808_EVENT:vehicle.event.jt808.v1}") - .containsEntry("lingniu.ingest.sink.kafka.topics.raw-archive", "${KAFKA_TOPIC_JT808_RAW:vehicle.raw.jt808.v1}") - .containsEntry("lingniu.ingest.sink.kafka.topics.dlq", "${KAFKA_TOPIC_JT808_DLQ:vehicle.dlq.jt808.v1}") - .containsEntry("lingniu.ingest.identity.mysql.table-name", - "${VEHICLE_IDENTITY_MYSQL_TABLE:vehicle_identity_bindings}") - .containsEntry("lingniu.ingest.identity.mysql.jdbc-url", - "${VEHICLE_IDENTITY_MYSQL_JDBC_URL:jdbc:mysql://127.0.0.1:3306/lingniu_vehicle?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai}") - .containsEntry("lingniu.ingest.identity.mysql.username", "${VEHICLE_IDENTITY_MYSQL_USERNAME:root}") - .containsEntry("lingniu.ingest.identity.mysql.password", "${VEHICLE_IDENTITY_MYSQL_PASSWORD:}") - .containsEntry("lingniu.ingest.identity.mysql.initialize-schema", - "${VEHICLE_IDENTITY_MYSQL_INITIALIZE_SCHEMA:true}") - .containsEntry("management.endpoints.web.exposure.include", "health,info,metrics,prometheus"); - - assertThat(properties.stringPropertyNames()) - .noneMatch(name -> name.startsWith("lingniu.ingest.event-history.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.event-file-store.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.vehicle-state.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.vehicle-stat.")) - .noneMatch(name -> name.equals("lingniu.ingest.session.store")) - .noneMatch(name -> name.equals("lingniu.ingest.identity.store")); - assertThat(applicationYaml()) - .doesNotContain("event-history:") - .doesNotContain("event-file-store:") - .doesNotContain("vehicle-state:") - .doesNotContain("vehicle-stat:") - .doesNotContain("springdoc:") - .doesNotContain("SESSION_STORE") - .doesNotContain("VEHICLE_IDENTITY_STORE") - .doesNotContain("KAFKA_ENABLED"); - } - - private static Properties applicationProperties() { - YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean(); - factory.setResources(new ClassPathResource("application.yml")); - - Properties properties = factory.getObject(); - - assertThat(properties).isNotNull(); - return properties; - } - - private static String applicationYaml() throws IOException { - return new String(new ClassPathResource("application.yml").getInputStream().readAllBytes(), - StandardCharsets.UTF_8); - } -} diff --git a/modules/apps/vehicle-analytics-app/pom.xml b/modules/apps/vehicle-analytics-app/pom.xml deleted file mode 100644 index f5668637..00000000 --- a/modules/apps/vehicle-analytics-app/pom.xml +++ /dev/null @@ -1,64 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - - vehicle-analytics-app - vehicle-analytics-app - Vehicle daily statistics and analytics runtime. - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - observability - - - com.lingniu.ingest - sink-kafka - - - com.lingniu.ingest - vehicle-stat-service - - - org.springframework.boot - spring-boot-starter-jdbc - - - org.springframework.boot - spring-boot-starter-web - - - com.alibaba.cloud - spring-cloud-starter-alibaba-nacos-config - - - org.springdoc - springdoc-openapi-starter-webmvc-ui - - - org.springframework.boot - spring-boot-starter-test - test - - - - - vehicle-analytics-app - - - org.springframework.boot - spring-boot-maven-plugin - - - - diff --git a/modules/apps/vehicle-analytics-app/src/main/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsApplication.java b/modules/apps/vehicle-analytics-app/src/main/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsApplication.java deleted file mode 100644 index 3bf0d5a0..00000000 --- a/modules/apps/vehicle-analytics-app/src/main/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsApplication.java +++ /dev/null @@ -1,21 +0,0 @@ -package com.lingniu.ingest.analyticsapp; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.boot.SpringApplication; -import org.springframework.boot.autoconfigure.SpringBootApplication; - -@SpringBootApplication -public class VehicleAnalyticsApplication { - - private static final Logger log = LoggerFactory.getLogger(VehicleAnalyticsApplication.class); - - public static void main(String[] args) { - try { - SpringApplication.run(VehicleAnalyticsApplication.class, args); - } catch (Throwable t) { - log.error("Vehicle analytics application failed to start, forcing JVM exit", t); - System.exit(1); - } - } -} diff --git a/modules/apps/vehicle-analytics-app/src/main/resources/application.yml b/modules/apps/vehicle-analytics-app/src/main/resources/application.yml deleted file mode 100644 index aeefc92c..00000000 --- a/modules/apps/vehicle-analytics-app/src/main/resources/application.yml +++ /dev/null @@ -1,71 +0,0 @@ -spring: - application: - name: vehicle-analytics-app - config: - import: - - optional:nacos:${spring.application.name}.${NACOS_CONFIG_FILE_EXTENSION:yml}?group=${NACOS_GROUP:DEFAULT_GROUP}&refreshEnabled=${NACOS_REFRESH_ENABLED:true} - cloud: - nacos: - config: - enabled: ${NACOS_CONFIG_ENABLED:true} - server-addr: ${NACOS_SERVER_ADDR:127.0.0.1:8848} - namespace: ${NACOS_NAMESPACE:} - group: ${NACOS_GROUP:DEFAULT_GROUP} - file-extension: ${NACOS_CONFIG_FILE_EXTENSION:yml} - username: ${NACOS_USERNAME:} - password: ${NACOS_PASSWORD:} - threads: - virtual: - enabled: true - datasource: - url: ${MYSQL_JDBC_URL:} - username: ${MYSQL_USERNAME:} - password: ${MYSQL_PASSWORD:} - driver-class-name: com.mysql.cj.jdbc.Driver - -server: - port: ${HTTP_PORT:20310} - -springdoc: - swagger-ui: - path: /swagger-ui.html - api-docs: - path: /v3/api-docs - -lingniu: - ingest: - sink: - kafka: - enabled: true - bootstrap-servers: ${KAFKA_BROKERS:114.55.58.251:9092} - consumer: - dlq-topic: ${KAFKA_TOPIC_JT808_DLQ:vehicle.dlq.jt808.v1} - enabled: ${KAFKA_CONSUMER_ENABLED:true} - client-id-prefix: ${KAFKA_CONSUMER_CLIENT_ID_PREFIX:vehicle-analytics} - auto-offset-reset: ${KAFKA_CONSUMER_AUTO_OFFSET_RESET:earliest} - max-poll-records: ${KAFKA_CONSUMER_MAX_POLL_RECORDS:500} - max-poll-interval-millis: ${KAFKA_CONSUMER_MAX_POLL_INTERVAL_MILLIS:900000} - bindings: - vehicleStatEnvelopeConsumerProcessor: - enabled: true - group-id: ${KAFKA_GROUP_STAT:vehicle-stat} - topics: - - ${KAFKA_TOPIC_JT808_EVENT:vehicle.event.jt808.v1} - vehicle-stat: - enabled: ${VEHICLE_STAT_ENABLED:true} - zone-id: ${VEHICLE_STAT_ZONE_ID:Asia/Shanghai} - jt808: - enabled: ${VEHICLE_STAT_JT808_MILEAGE_ENABLED:true} - -management: - endpoints: - web: - exposure: - include: health,info,metrics,prometheus - endpoint: - health: - probes: - enabled: ${MANAGEMENT_HEALTH_PROBES_ENABLED:true} - metrics: - tags: - application: vehicle-analytics-app diff --git a/modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppCompositionTest.java b/modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppCompositionTest.java deleted file mode 100644 index fb2f3a73..00000000 --- a/modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppCompositionTest.java +++ /dev/null @@ -1,74 +0,0 @@ -package com.lingniu.ingest.analyticsapp; - -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.sink.kafka.KafkaEnvelopeDeadLetterSink; -import com.lingniu.ingest.sink.kafka.KafkaEventSink; -import com.lingniu.ingest.sink.kafka.KafkaSinkAutoConfiguration; -import com.lingniu.ingest.vehiclestat.JdbcVehicleStatMetricRepository; -import com.lingniu.ingest.vehiclestat.VehicleStatController; -import com.lingniu.ingest.vehiclestat.VehicleStatEnvelopeIngestor; -import com.lingniu.ingest.vehiclestat.VehicleStatRepository; -import com.lingniu.ingest.vehiclestat.config.VehicleStatAutoConfiguration; -import com.lingniu.ingest.vehiclestat.jt808.Jt808MileageStreamProcessor; -import org.apache.kafka.clients.producer.KafkaProducer; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.context.ApplicationContext; -import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.util.ClassUtils; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.mock; - -class VehicleAnalyticsAppCompositionTest { - - @Test - void createsStatsBeansWithoutProtocolListenerOrFileHistoryStore() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of( - KafkaSinkAutoConfiguration.class, - VehicleStatAutoConfiguration.class)) - .withAllowBeanDefinitionOverriding(true) - .withBean("kafkaProducer", KafkaProducer.class, VehicleAnalyticsAppCompositionTest::kafkaProducer) - .withBean(JdbcTemplate.class, () -> mock(JdbcTemplate.class)) - .withBean(ObjectMapper.class, ObjectMapper::new) - .withPropertyValues( - "lingniu.ingest.sink.kafka.enabled=true", - "lingniu.ingest.sink.kafka.bootstrap-servers=localhost:9092", - "lingniu.ingest.sink.kafka.consumer.enabled=false", - "lingniu.ingest.vehicle-stat.enabled=true", - "lingniu.ingest.vehicle-stat.jt808.enabled=true") - .run(context -> { - assertThat(context).hasSingleBean(VehicleStatRepository.class); - assertThat(context).hasSingleBean(JdbcVehicleStatMetricRepository.class); - assertThat(context).hasSingleBean(Jt808MileageStreamProcessor.class); - assertThat(context).hasSingleBean(VehicleStatEnvelopeIngestor.class); - assertThat(context).hasSingleBean(VehicleStatController.class); - assertThat(context).hasSingleBean(KafkaEventSink.class); - assertThat(context).hasSingleBean(KafkaEnvelopeDeadLetterSink.class); - assertThat(context).hasBean("vehicleStatEnvelopeConsumerProcessor"); - assertThat(context.getBean("vehicleStatEnvelopeConsumerProcessor")) - .isInstanceOf(EnvelopeConsumerProcessor.class); - - assertTypeNotPresent(context, "com.lingniu.ingest.vehiclestate.VehicleStateEnvelopeIngestor"); - assertTypeNotPresent(context, "com.lingniu.ingest.vehiclestate.config.VehicleStateAutoConfiguration"); - assertTypeNotPresent(context, "com.lingniu.ingest.protocol.gb32960.inbound.Gb32960NettyServer"); - assertTypeNotPresent(context, "com.lingniu.ingest.eventfilestore.DuckDbParquetEventFileStore"); - assertTypeNotPresent(context, - "com.lingniu.ingest.eventfilestore.config.EventFileStoreAutoConfiguration"); - }); - } - - @SuppressWarnings("unchecked") - private static KafkaProducer kafkaProducer() { - return mock(KafkaProducer.class); - } - - private static void assertTypeNotPresent(ApplicationContext context, String className) { - assertThat(ClassUtils.isPresent(className, context.getClassLoader())) - .as("%s must stay outside the vehicle analytics runtime", className) - .isFalse(); - } -} diff --git a/modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppDefaultsTest.java b/modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppDefaultsTest.java deleted file mode 100644 index d8d14143..00000000 --- a/modules/apps/vehicle-analytics-app/src/test/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsAppDefaultsTest.java +++ /dev/null @@ -1,77 +0,0 @@ -package com.lingniu.ingest.analyticsapp; - -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.config.YamlPropertiesFactoryBean; -import org.springframework.core.io.ClassPathResource; - -import java.io.IOException; -import java.nio.charset.StandardCharsets; -import java.util.Properties; - -import static org.assertj.core.api.Assertions.assertThat; - -class VehicleAnalyticsAppDefaultsTest { - - @Test - void applicationDefaultsKeepAnalyticsAsStatConsumerRuntime() throws IOException { - Properties properties = applicationProperties(); - - assertThat(properties) - .containsEntry("spring.application.name", "vehicle-analytics-app") - .containsEntry( - "spring.config.import[0]", - "optional:nacos:${spring.application.name}.${NACOS_CONFIG_FILE_EXTENSION:yml}?group=${NACOS_GROUP:DEFAULT_GROUP}&refreshEnabled=${NACOS_REFRESH_ENABLED:true}") - .containsEntry("spring.cloud.nacos.config.server-addr", "${NACOS_SERVER_ADDR:127.0.0.1:8848}") - .containsEntry("server.port", "${HTTP_PORT:20310}") - .containsEntry("lingniu.ingest.vehicle-stat.enabled", "${VEHICLE_STAT_ENABLED:true}") - .containsEntry( - "lingniu.ingest.vehicle-stat.jt808.enabled", - "${VEHICLE_STAT_JT808_MILEAGE_ENABLED:true}") - .containsEntry("lingniu.ingest.sink.kafka.enabled", true) - .containsEntry("lingniu.ingest.sink.kafka.consumer.enabled", "${KAFKA_CONSUMER_ENABLED:true}") - .containsEntry("lingniu.ingest.sink.kafka.consumer.dlq-topic", - "${KAFKA_TOPIC_JT808_DLQ:vehicle.dlq.jt808.v1}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.vehicleStatEnvelopeConsumerProcessor.enabled", - true) - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.vehicleStatEnvelopeConsumerProcessor.group-id", - "${KAFKA_GROUP_STAT:vehicle-stat}") - .containsEntry("management.endpoints.web.exposure.include", "health,info,metrics,prometheus"); - - assertThat(properties.stringPropertyNames()) - .noneMatch(name -> name.startsWith("lingniu.ingest.gb32960.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.event-history.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.sink.archive.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.sink.kafka.topics.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.event-file-store.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.vehicle-state.")) - .noneMatch(name -> name.contains("vehicleStateEnvelopeConsumerProcessor")) - .noneMatch(name -> name.startsWith("spring.data.redis.")) - .noneMatch(name -> name.startsWith("management.health.redis.")); - assertThat(applicationYaml()) - .doesNotContain("gb32960:") - .doesNotContain("event-history:") - .doesNotContain("archive:") - .doesNotContain("event-file-store:") - .doesNotContain("vehicle-state:") - .doesNotContain("vehicleStateEnvelopeConsumerProcessor") - .doesNotContain("redis:") - .doesNotContain("KAFKA_ENABLED"); - } - - private static Properties applicationProperties() { - YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean(); - factory.setResources(new ClassPathResource("application.yml")); - - Properties properties = factory.getObject(); - - assertThat(properties).isNotNull(); - return properties; - } - - private static String applicationYaml() throws IOException { - return new String(new ClassPathResource("application.yml").getInputStream().readAllBytes(), - StandardCharsets.UTF_8); - } -} diff --git a/modules/apps/vehicle-history-app/pom.xml b/modules/apps/vehicle-history-app/pom.xml deleted file mode 100644 index 7a9719d2..00000000 --- a/modules/apps/vehicle-history-app/pom.xml +++ /dev/null @@ -1,68 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - - vehicle-history-app - vehicle-history-app - Vehicle raw archive, event history, and GB32960 frame query runtime. - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - observability - - - com.lingniu.ingest - sink-kafka - - - com.lingniu.ingest - tdengine-history-store - - - com.lingniu.ingest - event-history-service - - - com.lingniu.ingest - protocol-gb32960 - - - org.springframework.boot - spring-boot-starter-web - - - com.alibaba.cloud - spring-cloud-starter-alibaba-nacos-config - - - org.springdoc - springdoc-openapi-starter-webmvc-ui - - - org.springframework.boot - spring-boot-starter-test - test - - - - - vehicle-history-app - - - org.springframework.boot - spring-boot-maven-plugin - - - - diff --git a/modules/apps/vehicle-history-app/src/main/java/com/lingniu/ingest/historyapp/VehicleHistoryApplication.java b/modules/apps/vehicle-history-app/src/main/java/com/lingniu/ingest/historyapp/VehicleHistoryApplication.java deleted file mode 100644 index 48ac2363..00000000 --- a/modules/apps/vehicle-history-app/src/main/java/com/lingniu/ingest/historyapp/VehicleHistoryApplication.java +++ /dev/null @@ -1,21 +0,0 @@ -package com.lingniu.ingest.historyapp; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.boot.SpringApplication; -import org.springframework.boot.autoconfigure.SpringBootApplication; - -@SpringBootApplication -public class VehicleHistoryApplication { - - private static final Logger log = LoggerFactory.getLogger(VehicleHistoryApplication.class); - - public static void main(String[] args) { - try { - SpringApplication.run(VehicleHistoryApplication.class, args); - } catch (Throwable t) { - log.error("Vehicle history application failed to start, forcing JVM exit", t); - System.exit(1); - } - } -} diff --git a/modules/apps/vehicle-history-app/src/main/java/com/lingniu/ingest/historyapp/VehicleHistoryKafkaConsumerConfiguration.java b/modules/apps/vehicle-history-app/src/main/java/com/lingniu/ingest/historyapp/VehicleHistoryKafkaConsumerConfiguration.java deleted file mode 100644 index 5884c23f..00000000 --- a/modules/apps/vehicle-history-app/src/main/java/com/lingniu/ingest/historyapp/VehicleHistoryKafkaConsumerConfiguration.java +++ /dev/null @@ -1,46 +0,0 @@ -package com.lingniu.ingest.historyapp; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.sink.kafka.KafkaEnvelopeConsumerFactory; -import com.lingniu.ingest.sink.kafka.KafkaEnvelopeConsumerRunner; -import com.lingniu.ingest.sink.kafka.KafkaEnvelopeConsumerWorker; -import com.lingniu.ingest.sink.kafka.KafkaSinkProperties; -import org.springframework.beans.factory.annotation.Qualifier; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; - -import java.time.Duration; -import java.util.List; -import java.util.Map; - -@Configuration(proxyBeanMethods = false) -public class VehicleHistoryKafkaConsumerConfiguration { - - @Bean - @ConditionalOnMissingBean - @ConditionalOnProperty(prefix = "lingniu.ingest.sink.kafka.consumer", name = "enabled", havingValue = "true") - public KafkaEnvelopeConsumerRunner vehicleHistoryKafkaEnvelopeConsumerRunner( - @Qualifier("eventHistoryEnvelopeConsumerProcessor") EnvelopeConsumerProcessor processor, - @Qualifier("eventHistoryRawEnvelopeConsumerProcessor") EnvelopeConsumerProcessor rawProcessor, - KafkaSinkProperties props) { - List workers = new KafkaEnvelopeConsumerFactory().createWorkers( - Map.of( - "eventHistoryGb32960EnvelopeConsumerProcessor", processor, - "eventHistoryJt808EnvelopeConsumerProcessor", processor, - "eventHistoryYutongMqttEnvelopeConsumerProcessor", processor, - "eventHistoryGb32960RawEnvelopeConsumerProcessor", rawProcessor, - "eventHistoryJt808RawEnvelopeConsumerProcessor", rawProcessor, - "eventHistoryYutongMqttRawEnvelopeConsumerProcessor", rawProcessor), - props); - if (workers.isEmpty()) { - throw new IllegalStateException("no vehicle history kafka consumer workers created; check consumer bindings"); - } - return new KafkaEnvelopeConsumerRunner( - workers, - Duration.ofMillis(props.getConsumer().getPollTimeoutMillis()), - Duration.ofMillis(props.getConsumer().getLoopBackoffMillis()), - props.getConsumer().isAutoStartup()); - } -} diff --git a/modules/apps/vehicle-history-app/src/main/resources/application.yml b/modules/apps/vehicle-history-app/src/main/resources/application.yml deleted file mode 100644 index 0b08b0de..00000000 --- a/modules/apps/vehicle-history-app/src/main/resources/application.yml +++ /dev/null @@ -1,103 +0,0 @@ -spring: - application: - name: vehicle-history-app - config: - import: - - optional:nacos:${spring.application.name}.${NACOS_CONFIG_FILE_EXTENSION:yml}?group=${NACOS_GROUP:DEFAULT_GROUP}&refreshEnabled=${NACOS_REFRESH_ENABLED:true} - cloud: - nacos: - config: - enabled: ${NACOS_CONFIG_ENABLED:true} - server-addr: ${NACOS_SERVER_ADDR:127.0.0.1:8848} - namespace: ${NACOS_NAMESPACE:} - group: ${NACOS_GROUP:DEFAULT_GROUP} - file-extension: ${NACOS_CONFIG_FILE_EXTENSION:yml} - username: ${NACOS_USERNAME:} - password: ${NACOS_PASSWORD:} - threads: - virtual: - enabled: true - -server: - port: ${HTTP_PORT:20200} - -springdoc: - swagger-ui: - path: /swagger-ui.html - api-docs: - path: /v3/api-docs - -lingniu: - ingest: - sink: - kafka: - enabled: true - bootstrap-servers: ${KAFKA_BROKERS:114.55.58.251:9092} - consumer: - dlq-topic: ${KAFKA_TOPIC_HISTORY_DLQ:vehicle.dlq.history.v1} - enabled: ${KAFKA_CONSUMER_ENABLED:true} - client-id-prefix: ${KAFKA_CONSUMER_CLIENT_ID_PREFIX:vehicle-history} - auto-offset-reset: ${KAFKA_CONSUMER_AUTO_OFFSET_RESET:earliest} - max-poll-records: ${KAFKA_CONSUMER_MAX_POLL_RECORDS:2000} - max-poll-interval-millis: ${KAFKA_CONSUMER_MAX_POLL_INTERVAL_MILLIS:1800000} - concurrency: ${KAFKA_CONSUMER_CONCURRENCY:3} - bindings: - eventHistoryGb32960EnvelopeConsumerProcessor: - enabled: true - group-id: ${KAFKA_GROUP_HISTORY_GB32960_EVENT:${KAFKA_GROUP_HISTORY:vehicle-history}-gb32960-event} - topics: - - ${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1} - eventHistoryJt808EnvelopeConsumerProcessor: - enabled: true - group-id: ${KAFKA_GROUP_HISTORY_JT808_EVENT:${KAFKA_GROUP_HISTORY:vehicle-history}-jt808-event} - topics: - - ${KAFKA_TOPIC_JT808_EVENT:vehicle.event.jt808.v1} - eventHistoryYutongMqttEnvelopeConsumerProcessor: - enabled: true - group-id: ${KAFKA_GROUP_HISTORY_YUTONG_MQTT_EVENT:${KAFKA_GROUP_HISTORY:vehicle-history}-yutong-mqtt-event} - topics: - - ${KAFKA_TOPIC_YUTONG_MQTT_EVENT:vehicle.event.mqtt-yutong.v1} - eventHistoryGb32960RawEnvelopeConsumerProcessor: - enabled: true - group-id: ${KAFKA_GROUP_HISTORY_GB32960_RAW:${KAFKA_GROUP_HISTORY:vehicle-history}-gb32960-raw} - topics: - - ${KAFKA_TOPIC_GB32960_RAW:vehicle.raw.gb32960.v1} - eventHistoryJt808RawEnvelopeConsumerProcessor: - enabled: true - group-id: ${KAFKA_GROUP_HISTORY_JT808_RAW:${KAFKA_GROUP_HISTORY:vehicle-history}-jt808-raw} - topics: - - ${KAFKA_TOPIC_JT808_RAW:vehicle.raw.jt808.v1} - eventHistoryYutongMqttRawEnvelopeConsumerProcessor: - enabled: true - group-id: ${KAFKA_GROUP_HISTORY_YUTONG_MQTT_RAW:${KAFKA_GROUP_HISTORY:vehicle-history}-yutong-mqtt-raw} - topics: - - ${KAFKA_TOPIC_YUTONG_MQTT_RAW:vehicle.raw.mqtt-yutong.v1} - tdengine-history: - enabled: ${TDENGINE_HISTORY_ENABLED:false} - database: ${TDENGINE_HISTORY_DATABASE:vehicle_history} - jdbc-url: ${TDENGINE_JDBC_URL:jdbc:TAOS-RS://${TDENGINE_HOST:127.0.0.1}:${TDENGINE_PORT:6041}/${TDENGINE_HISTORY_DATABASE:vehicle_history}} - username: ${TDENGINE_USERNAME:root} - password: ${TDENGINE_PASSWORD:taosdata} - driver-class-name: ${TDENGINE_DRIVER_CLASS_NAME:com.taosdata.jdbc.rs.RestfulDriver} - maximum-pool-size: ${TDENGINE_MAX_POOL_SIZE:16} - minimum-idle: ${TDENGINE_MIN_IDLE:0} - connection-timeout-millis: ${TDENGINE_CONNECTION_TIMEOUT_MILLIS:5000} - initialization-fail-timeout-millis: ${TDENGINE_INITIALIZATION_FAIL_TIMEOUT_MILLIS:-1} - event-history: - enabled: true - -management: - endpoints: - web: - exposure: - include: health,info,metrics,prometheus - endpoint: - health: - probes: - enabled: ${MANAGEMENT_HEALTH_PROBES_ENABLED:true} - health: - redis: - enabled: ${MANAGEMENT_HEALTH_REDIS_ENABLED:false} - metrics: - tags: - application: vehicle-history-app diff --git a/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/MavenModuleProfileTest.java b/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/MavenModuleProfileTest.java deleted file mode 100644 index 2b1127ce..00000000 --- a/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/MavenModuleProfileTest.java +++ /dev/null @@ -1,878 +0,0 @@ -package com.lingniu.ingest.historyapp; - -import org.junit.jupiter.api.Test; -import org.w3c.dom.Document; -import org.w3c.dom.Element; -import org.w3c.dom.NodeList; - -import javax.xml.parsers.DocumentBuilderFactory; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.ArrayList; -import java.util.List; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -class MavenModuleProfileTest { - - private static final List ACTIVE_APP_POMS = List.of( - "modules/apps/gb32960-ingest-app/pom.xml", - "modules/apps/jt808-ingest-app/pom.xml", - "modules/apps/yutong-mqtt-app/pom.xml", - "modules/apps/vehicle-history-app/pom.xml", - "modules/apps/vehicle-analytics-app/pom.xml"); - - @Test - void commandGatewayIsOutsideDefaultProductionReactor() throws Exception { - Document pom = rootPom(); - - assertThat(defaultModules(pom)) - .doesNotContain("modules/apps/command-gateway") - .doesNotContain("modules/protocols/protocol-jt1078"); - - assertThat(profileModules(pom, "optional-command-gateway")) - .containsExactly( - "modules/protocols/protocol-jt1078", - "modules/apps/command-gateway"); - } - - @Test - void jsatl12AttachmentProtocolIsOptionalAndOutsideDefaultProductionReactor() throws Exception { - Document pom = rootPom(); - - assertThat(defaultModules(pom)) - .doesNotContain("modules/protocols/protocol-jsatl12"); - - assertThat(profileModules(pom, "optional-attachments")) - .containsExactly("modules/protocols/protocol-jsatl12"); - } - - @Test - void latestStateServiceIsOptionalAndOutsideDefaultProductionReactor() throws Exception { - Document pom = rootPom(); - - assertThat(defaultModules(pom)) - .doesNotContain("modules/services/vehicle-state-service"); - - assertThat(profileModules(pom, "optional-latest-state")) - .containsExactly("modules/services/vehicle-state-service"); - } - - @Test - void rawArchiveStorePrototypeIsRemovedFromBuildSurface() throws Exception { - Document pom = rootPom(); - - assertThat(defaultModules(pom)) - .doesNotContain("modules/sinks/raw-archive-store"); - - assertThat(profileModules(pom, "optional-raw-archive-store")) - .as("raw bytes are written by sink-archive; raw-archive-store was an unused prototype") - .isEmpty(); - assertThat(Files.exists(repositoryRoot().resolve("modules/sinks/raw-archive-store"))) - .isFalse(); - } - - @Test - void eventFileStoreCompatibilityPathIsRemovedFromProductionStack() throws Exception { - Document pom = rootPom(); - String readme = Files.readString(repositoryRoot().resolve("README.md")); - - assertThat(defaultModules(pom)) - .doesNotContain("modules/sinks/event-file-store"); - assertThat(profileModules(pom, "optional-event-file-store")) - .as("DuckDB/Parquet event-file-store has been replaced by TDengine raw history") - .isEmpty(); - assertThat(Files.exists(repositoryRoot().resolve("modules/sinks/event-file-store"))) - .isFalse(); - assertThat(rootDependencyManagementArtifacts(pom)) - .doesNotContain("event-file-store"); - assertThat(hasProjectProperty(pom, "duckdb.version")) - .isFalse(); - assertThat(hasDependency(pom, "org.duckdb", "duckdb_jdbc")) - .isFalse(); - assertThat(readme) - .doesNotContain("event-file-store") - .doesNotContain("DuckDB") - .doesNotContain("Parquet"); - } - - @Test - void kafkaSinkModuleUsesKafkaNamingInsteadOfMqNaming() throws Exception { - Document pom = rootPom(); - Document kafkaSinkPom = modulePom("modules/sinks/sink-kafka/pom.xml"); - String readme = Files.readString(repositoryRoot().resolve("README.md")); - String legacyModule = "modules/sinks/sink-" + "mq"; - String legacyArtifact = "sink-" + "mq"; - - assertThat(defaultModules(pom)) - .contains("modules/sinks/sink-kafka") - .doesNotContain(legacyModule); - assertThat(rootDependencyManagementArtifacts(pom)) - .contains("sink-kafka") - .doesNotContain(legacyArtifact); - assertThat(firstDirectChild(kafkaSinkPom.getDocumentElement(), "artifactId").getTextContent()) - .isEqualTo("sink-kafka"); - assertThat(readme) - .contains("sink-kafka/") - .doesNotContain(legacyArtifact); - } - - @Test - void xindaModulesAreRemovedFromBuildSurface() throws Exception { - Document pom = rootPom(); - String rootPomText = Files.readString(repositoryRoot().resolve("pom.xml")); - String readme = Files.readString(repositoryRoot().resolve("README.md")); - - assertThat(defaultModules(pom)) - .doesNotContain("modules/inbound/inbound-xinda-push") - .doesNotContain("modules/apps/xinda-push-app"); - assertThat(profileModules(pom, "legacy-xinda")) - .as("xinda is deleted, not hidden behind an opt-in profile") - .isEmpty(); - assertThat(Files.exists(repositoryRoot().resolve("modules/inbound/inbound-xinda-push"))) - .isFalse(); - assertThat(Files.exists(repositoryRoot().resolve("modules/apps/xinda-push-app"))) - .isFalse(); - assertThat(rootPomText) - .doesNotContain("legacy-xinda") - .doesNotContain("inbound-xinda-push") - .doesNotContain("xinda-push-app"); - assertThat(readme) - .contains("信达 Push 已废弃并删除源码") - .doesNotContain("信达 Push 仅保留") - .doesNotContain("废弃兼容模块"); - } - - @Test - void privateLingniuRepositoryIsRemovedWithXinda() throws Exception { - Document pom = rootPom(); - - assertThat(repositoryIds(pom.getDocumentElement())) - .doesNotContain("lingniu-nexus"); - assertThat(profileRepositoryIds(pom, "legacy-xinda")) - .isEmpty(); - } - - @Test - void optionalModuleVersionsAreScopedToTheirProfiles() throws Exception { - Document pom = rootPom(); - - assertThat(rootDependencyManagementArtifacts(pom)) - .doesNotContain( - "protocol-jt1078", - "protocol-jsatl12", - "vehicle-state-service", - "raw-archive-store", - "command-gateway", - "inbound-xinda-push", - "xinda-push-app"); - - assertThat(profileDependencyManagementArtifacts(pom, "optional-command-gateway")) - .containsExactly("protocol-jt1078", "command-gateway"); - assertThat(profileDependencyManagementArtifacts(pom, "optional-attachments")) - .containsExactly("protocol-jsatl12"); - assertThat(profileDependencyManagementArtifacts(pom, "optional-latest-state")) - .containsExactly("vehicle-state-service"); - assertThat(profileDependencyManagementArtifacts(pom, "optional-raw-archive-store")) - .isEmpty(); - assertThat(profileDependencyManagementArtifacts(pom, "optional-event-file-store")) - .isEmpty(); - assertThat(profileDependencyManagementArtifacts(pom, "legacy-xinda")) - .isEmpty(); - } - - @Test - void appsDirectoryContainsOnlyRealMavenModules() throws Exception { - try (Stream apps = Files.list(repositoryRoot().resolve("modules/apps"))) { - assertThat(apps.filter(Files::isDirectory) - .filter(path -> !Files.exists(path.resolve("pom.xml"))) - .map(path -> path.getFileName().toString())) - .isEmpty(); - } - } - - @Test - void vehicleIdentityTestSupportIsCentralizedInsteadOfCopiedIntoProtocolTests() throws Exception { - Document pom = rootPom(); - List testSupportFixtures = javaFilesNamed("InMemoryVehicleIdentityService.java"); - - assertThat(defaultModules(pom)) - .contains("modules/testing/vehicle-identity-test-support"); - assertThat(rootDependencyManagementArtifacts(pom)) - .contains("vehicle-identity-test-support"); - assertThat(testSupportFixtures) - .containsExactly("modules/testing/vehicle-identity-test-support/src/main/java/com/lingniu/ingest/identity/InMemoryVehicleIdentityService.java"); - - for (String modulePom : List.of( - "modules/protocols/protocol-jt808/pom.xml", - "modules/inbound/inbound-mqtt/pom.xml", - "modules/protocols/protocol-jt1078/pom.xml", - "modules/protocols/protocol-jsatl12/pom.xml")) { - assertThat(hasDependency(modulePom(modulePom), "com.lingniu.ingest", "vehicle-identity-test-support")) - .as(modulePom) - .isTrue(); - } - } - - @Test - void readmeMatchesCurrentRuntimeVersionsAndSplitAppResponsibilities() throws Exception { - Document pom = rootPom(); - String readme = Files.readString(repositoryRoot().resolve("README.md")); - - assertThat(readme) - .contains("Spring Boot " + projectProperty(pom, "spring-boot.version")) - .contains("Netty " + projectProperty(pom, "netty.version")) - .contains("Eclipse Paho " + projectProperty(pom, "paho-mqtt.version")) - .contains("session-core/ 设备会话 + 鉴权 + Token + Redis SessionStore") - .contains("| 并发 | Disruptor 4 + 虚拟线程 |") - .contains("| 事件流 | **Kafka**（唯一生产消息通道，vin 分区，保证单车有序） |") - .contains("| 冷存 | 本地文件系统 |") - .contains("| 构建 | Maven |") - .contains("protocol-jt1078/ JT/T 1078（optional-command-gateway profile") - .contains("protocol-jsatl12/ 苏标主动安全报警附件（optional-attachments profile）") - .contains("vehicle-state-service/ Kafka 全字段事件消费 + Redis 热状态查询（optional-latest-state profile）") - .contains("command-gateway/ 可选 HTTP → 设备下行命令（optional-command-gateway profile）") - .contains("vehicle-history-app/ TDengine 历史查询 + RAW JSON") - .contains("vehicle-analytics-app/ JT808 每日里程指标消费") - .contains("协议接入应用只负责收、解析、校验、规整和投递 Kafka") - .contains("生产链路只支持 Kafka") - .contains("`vehicle-analytics-app` 将 JT808 `daily_mileage_km` 写入 MySQL `vehicle_stat_metric`") - .contains("mvn -pl :gb32960-ingest-app,:jt808-ingest-app,:yutong-mqtt-app,:vehicle-history-app,:vehicle-analytics-app -am package -Dmaven.test.skip=true") - .contains("生产服务只在 ECS/Portainer 上运行") - .doesNotContain("Spring Boot 3.4") - .doesNotContain("Netty 4.1") - .doesNotContain("HiveMQ MQTT Client") - .doesNotContain("RabbitMQ") - .doesNotContain("RocketMQ") - .doesNotContain("其他 MQ 后端") - .doesNotContain("Spotless") - .doesNotContain("ArchUnit") - .doesNotContain("S3 / OSS / 本地") - .doesNotContain("StructuredTaskScope") - .doesNotContain("mvn clean install -DskipTests") - .doesNotContain("-DskipTests") - .doesNotContain("spring-boot:run") - .doesNotContain("Caffeine 内存降级") - .doesNotContain("Redis/Memory SessionStore") - .doesNotContain("event-file-store") - .doesNotContain("DuckDB") - .doesNotContain("Parquet") - .doesNotContain("Parquet + DuckDB 文件型明细库") - .doesNotContain("车辆状态/日统计消费") - .doesNotContain("本服务不写业务库") - .doesNotContain("里程统计全部由 Kafka 下游消费者实现") - .doesNotContain("| 消息队列 |"); - } - - @Test - void archiveBuildSurfaceDocumentsLocalFilesystemOnly() throws Exception { - String archivePom = Files.readString(repositoryRoot().resolve("modules/sinks/sink-archive/pom.xml")); - String moduleDataFlow = Files.readString(repositoryRoot().resolve("docs/module-data-flow.html")); - - assertThat(archivePom) - .contains("原始报文冷存：本地文件系统实现。") - .doesNotContain("S3/OSS") - .doesNotContain("MinIO") - .doesNotContain("software.amazon.awssdk") - .doesNotContain("com.aliyun.oss"); - assertThat(moduleDataFlow) - .contains("sink-archive
RawArchive 到本地文件系统冷存") - .contains("归档文件的真实本地路径只属于 ArchiveStore") - .contains("原始报文冷存，当前为本地文件系统实现") - .doesNotContain("本地/S3/OSS") - .doesNotContain("S3/OSS") - .doesNotContain("MinIO"); - } - - @Test - void yutongMqttBuildSurfaceUsesOnlyPahoClient() throws Exception { - Document pom = rootPom(); - - assertThat(hasProjectProperty(pom, "paho-mqtt.version")) - .isTrue(); - assertThat(hasDependency(pom, "org.eclipse.paho", "org.eclipse.paho.client.mqttv3")) - .isTrue(); - assertThat(hasProjectProperty(pom, "fusesource-mqtt.version")) - .isFalse(); - assertThat(hasDependency(pom, "org.fusesource.mqtt-client", "mqtt-client")) - .isFalse(); - } - - @Test - void activeAppBootPackagingConfigurationIsManagedOnceInRootPom() throws Exception { - Document pom = rootPom(); - Element managedBootPlugin = managedPlugin(pom, "org.springframework.boot", "spring-boot-maven-plugin"); - - assertThat(managedBootPlugin).isNotNull(); - assertThat(managedBootPlugin.getTextContent()) - .contains("${spring-boot.version}") - .contains("--sun-misc-unsafe-memory-access=allow") - .contains("-Dio.netty.transport.noNative=true") - .contains("repackage") - .contains("build-info"); - - for (String appPom : ACTIVE_APP_POMS) { - Document app = modulePom(appPom); - Element bootPlugin = buildPlugin(app, "org.springframework.boot", "spring-boot-maven-plugin"); - - assertThat(bootPlugin) - .as(appPom) - .isNotNull(); - assertThat(firstDirectChild(bootPlugin, "configuration")) - .as(appPom + " should inherit shared JVM arguments from root pluginManagement") - .isNull(); - assertThat(firstDirectChild(bootPlugin, "executions")) - .as(appPom + " should inherit repackage/build-info executions from root pluginManagement") - .isNull(); - } - } - - @Test - void observabilityBuildSurfaceUsesMicrometerWithoutUnusedOpenTelemetryBom() throws Exception { - Document pom = rootPom(); - Document observabilityPom = modulePom("modules/core/observability/pom.xml"); - String readme = Files.readString(repositoryRoot().resolve("README.md")); - String moduleDataFlow = Files.readString(repositoryRoot().resolve("docs/module-data-flow.html")); - String traceSpan = Files.readString(repositoryRoot() - .resolve("modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/TraceSpan.java")); - - assertThat(hasProjectProperty(pom, "micrometer.version")) - .isTrue(); - assertThat(hasDependency(pom, "io.micrometer", "micrometer-bom")) - .isTrue(); - assertThat(hasProjectProperty(pom, "otel.version")) - .isFalse(); - assertThat(hasDependency(pom, "io.opentelemetry", "opentelemetry-bom")) - .isFalse(); - assertThat(hasDependency(observabilityPom, "com.lingniu.ingest", "ingest-core")) - .as("observability is an interceptor API module and should not depend on core implementation") - .isFalse(); - assertThat(productionJavaContains("io.opentelemetry")) - .isFalse(); - assertThat(readme) - .contains("| 可观测 | Micrometer + Prometheus |") - .doesNotContain("OpenTelemetry"); - assertThat(moduleDataFlow) - .contains("observability
metrics / health") - .doesNotContain("observability
metrics / tracing / health"); - assertThat(traceSpan) - .doesNotContain("OpenTelemetry") - .doesNotContain("span"); - } - - @Test - void activeAppsDependOnSharedObservabilityForHealthAndMetrics() throws Exception { - for (String appPom : ACTIVE_APP_POMS) { - assertThat(hasDependency(modulePom(appPom), "com.lingniu.ingest", "observability")) - .as(appPom + " exposes actuator health/readiness and Prometheus metrics") - .isTrue(); - } - } - - @Test - void springdocUiIsOnlyPackagedWithAppsThatExposeRestApis() throws Exception { - for (String appPom : List.of( - "modules/apps/gb32960-ingest-app/pom.xml", - "modules/apps/jt808-ingest-app/pom.xml", - "modules/apps/yutong-mqtt-app/pom.xml")) { - assertThat(hasDependency(modulePom(appPom), "org.springdoc", "springdoc-openapi-starter-webmvc-ui")) - .as(appPom + " exposes Actuator only; Swagger UI belongs to query/API apps") - .isFalse(); - } - for (String appPom : List.of( - "modules/apps/vehicle-history-app/pom.xml", - "modules/apps/vehicle-analytics-app/pom.xml")) { - assertThat(hasDependency(modulePom(appPom), "org.springdoc", "springdoc-openapi-starter-webmvc-ui")) - .as(appPom + " exposes REST APIs") - .isTrue(); - } - } - - @Test - void buildSurfaceDoesNotManageLombokWhenProductionSourcesDoNotUseIt() throws Exception { - Document pom = rootPom(); - - assertThat(hasProjectProperty(pom, "lombok.version")) - .isFalse(); - assertThat(hasDependency(pom, "org.projectlombok", "lombok")) - .isFalse(); - assertThat(productionJavaContains("lombok.")) - .isFalse(); - } - - @Test - void buildSurfaceDoesNotManageArchUnitWhenThereAreNoArchitectureTests() throws Exception { - Document pom = rootPom(); - - assertThat(hasProjectProperty(pom, "archunit.version")) - .isFalse(); - assertThat(hasDependency(pom, "com.tngtech.archunit", "archunit-junit5")) - .isFalse(); - assertThat(javaSourcesContain("import com.tngtech." + "archunit")) - .isFalse(); - } - - @Test - void buildSurfaceDoesNotManageTestcontainersWhenKafkaTestsUseInMemoryFakes() throws Exception { - Document pom = rootPom(); - Document kafkaSinkPom = modulePom("modules/sinks/sink-kafka/pom.xml"); - - assertThat(hasProjectProperty(pom, "testcontainers.version")) - .isFalse(); - assertThat(hasDependency(pom, "org.testcontainers", "testcontainers-bom")) - .isFalse(); - assertThat(hasDependency(kafkaSinkPom, "org.testcontainers", "kafka")) - .isFalse(); - assertThat(javaSourcesContain("import org." + "testcontainers")) - .isFalse(); - } - - @Test - void productionDocsDoNotAdvertiseMemorySessionStore() throws Exception { - String runbook = Files.readString(repositoryRoot() - .resolve("docs/operations/gb32960-service-split-runbook.md")); - String moduleDataFlow = Files.readString(repositoryRoot().resolve("docs/module-data-flow.html")); - String redesignSpec = Files.readString(repositoryRoot() - .resolve("docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.md")); - - assertThat(runbook) - .contains("Session state requires Redis") - .contains("REDIS_HOST") - .contains("REDIS_DATABASE=50") - .doesNotContain("SESSION_STORE=memory") - .doesNotContain("without Redis"); - assertThat(moduleDataFlow) - .contains("SessionStore 仅保留 Redis 后端") - .doesNotContain("SessionStore 支持 memory") - .doesNotContain("无 Redis 降级"); - assertThat(redesignSpec) - .contains("Reads Redis latest state") - .doesNotContain("Redis or in-memory") - .doesNotContain("Redis/in-memory"); - } - - @Test - void sessionCorePomDescriptionDoesNotAdvertiseRedisAsFallback() throws Exception { - Document sessionCorePom = modulePom("modules/core/session-core/pom.xml"); - String description = firstDirectChild(sessionCorePom.getDocumentElement(), "description").getTextContent(); - - assertThat(description) - .contains("Redis SessionStore") - .doesNotContain("兜底") - .doesNotContain("fallback") - .doesNotContain("memory"); - } - - @Test - void rootPomDescriptionMatchesCurrentRuntimeStack() throws Exception { - Document pom = rootPom(); - String description = firstDirectChild(pom.getDocumentElement(), "description").getTextContent(); - - assertThat(description) - .contains("只支持 Kafka") - .contains("Java 25") - .contains("Spring Boot " + projectProperty(pom, "spring-boot.version")) - .contains("Netty " + projectProperty(pom, "netty.version")) - .doesNotContain("Spring Boot 3.4"); - } - - @Test - void defaultProductionSurfaceOnlySupportsKafkaAsMessageBackbone() throws Exception { - List productionSurface = List.of( - repositoryRoot().resolve("README.md"), - repositoryRoot().resolve("pom.xml"), - repositoryRoot().resolve("deploy/portainer/docker-compose.yml"), - repositoryRoot().resolve("docs/operations/gb32960-service-split-runbook.md"), - repositoryRoot().resolve("docs/operations/vehicle-ingest-tdengine-verification.md"), - repositoryRoot().resolve("docs/operations/jt808-daily-mileage-runbook.md"), - repositoryRoot().resolve("docs/target-architecture.md")); - - for (Path path : productionSurface) { - String text = Files.readString(path); - - assertThat(text) - .as(path.toString()) - .doesNotContain("sink-mq") - .doesNotContain("RabbitMQ") - .doesNotContain("RocketMQ") - .doesNotContain("Pulsar") - .doesNotContain("AMQP") - .doesNotContain("其他 MQ 后端"); - } - } - - @Test - void architectureDecisionsMatchCurrentDefaultProductionSurface() throws Exception { - String decisions = Files.readString(repositoryRoot().resolve("DECISIONS.md")); - - assertThat(decisions) - .contains("## ADR-004 信达 Push：Removed") - .contains("## ADR-006 部署形态：GB32960 三应用拆分\n- **Status**: Superseded by ADR-011") - .contains("## ADR-011 默认生产面：三协议接入 + 历史 + 统计") - .contains("## ADR-012 JSATL12 附件上传：Optional only") - .contains("GB32960、JT808、Yutong MQTT、vehicle-history-app、vehicle-analytics-app") - .contains("Xinda Push 源码、Maven profile、Woodpecker 镜像发布和历史消费绑定全部移除") - .contains("JSATL12 仅保留在 `optional-attachments` profile") - .contains("vehicle-state-service 仅保留在 `optional-latest-state` profile") - .contains("raw-archive-store 原型已删除") - .contains("协议接入应用不直接写业务库") - .contains("`vehicle-analytics-app` 写入 MySQL `vehicle_stat_metric`") - .doesNotContain("event-file-store") - .doesNotContain("optional-event-file-store") - .doesNotContain("optional-raw-archive-store") - .doesNotContain("legacy-xinda") - .doesNotContain("## ADR-004 信达 Push：保留模块但彻底重写\n- **Status**: Accepted") - .doesNotContain("CI/CD 分别构建三个镜像") - .doesNotContain("Spring Boot 3.4.x"); - } - - @Test - void productionDocsMatchCompactHistoryAndJt808OnlyAnalyticsSurface() throws Exception { - String moduleDataFlow = Files.readString(repositoryRoot().resolve("docs/module-data-flow.html")); - String targetArchitecture = Files.readString(repositoryRoot().resolve("docs/target-architecture.md")); - String tdengineVerification = Files.readString(repositoryRoot() - .resolve("docs/operations/vehicle-ingest-tdengine-verification.md")); - String runbook = Files.readString(repositoryRoot() - .resolve("docs/operations/gb32960-service-split-runbook.md")); - - assertThat(moduleDataFlow) - .contains("TDengine raw_frames + locations") - .contains("TDengine raw_frames 和位置表") - .doesNotContain("telemetry field 行映射") - .doesNotContain("可选 telemetry field 表"); - assertThat(tdengineVerification) - .contains("history 写入 raw、location 到 TDengine；不写 telemetry_fields") - .doesNotContain("按需开启 telemetry field"); - assertThat(runbook) - .contains("| `vehicle.event.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app` |") - .contains("| `vehicle.event.jt808.v1` | `jt808-ingest-app` | `vehicle-history-app`, `vehicle-analytics-app` |") - .doesNotContain("| `vehicle.event.gb32960.v1` | `gb32960-ingest-app` | `vehicle-history-app`, `vehicle-analytics-app` |"); - assertThat(targetArchitecture) - .contains("vehicle-state-service is absent from the default Maven reactor") - .doesNotContain("kafka --> state[\"vehicle-state-service\"]") - .doesNotContain("state --> redisState[\"Redis
latest state\"]"); - } - - @Test - void supersededDesignDocsPointToCurrentArchitecture() throws Exception { - for (String path : List.of( - "docs/superpowers/specs/2026-06-23-gb32960-production-readiness-design.md", - "docs/superpowers/specs/2026-06-23-gb32960-service-split-design.md")) { - String doc = Files.readString(repositoryRoot().resolve(path)); - - assertThat(doc) - .as(path) - .startsWith("> **Superseded:**") - .contains("docs/target-architecture.md") - .contains("docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.md"); - } - } - - @Test - void supersededImplementationPlansPointToCurrentArchitecture() throws Exception { - for (String path : List.of( - "docs/superpowers/plans/2026-06-23-gb32960-history-duckdb-hot-store.md", - "docs/superpowers/plans/2026-06-23-gb32960-service-split.md", - "docs/superpowers/plans/2026-06-29-vehicle-ingest-redesign-phase1.md")) { - String doc = Files.readString(repositoryRoot().resolve(path)); - - assertThat(doc) - .as(path) - .startsWith("> **Superseded:**") - .contains("docs/target-architecture.md") - .contains("docs/superpowers/specs/2026-06-29-vehicle-ingest-redesign.md"); - } - } - - @Test - void duckDbAndParquetAreNotPartOfTheCurrentBuildSurface() throws Exception { - Document pom = rootPom(); - - assertThat(hasProjectProperty(pom, "duckdb.version")) - .isFalse(); - assertThat(hasDependency(pom, "org.duckdb", "duckdb_jdbc")) - .isFalse(); - assertThat(Files.exists(repositoryRoot().resolve("modules/sinks/event-file-store"))) - .isFalse(); - } - - @Test - void legacyEventFileStoreContractsAreRemovedFromProductionSources() throws Exception { - Path root = repositoryRoot(); - - assertThat(Files.exists(root.resolve("modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/history"))) - .isFalse(); - assertThat(Files.exists(root.resolve( - "modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/EventHistoryController.java"))) - .isFalse(); - assertThat(Files.exists(root.resolve( - "modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/TelemetryEnvelopeRecordMapper.java"))) - .isFalse(); - assertThat(productionJavaContains("EventFileStore")) - .isFalse(); - assertThat(productionJavaContains("EventFileRecord")) - .isFalse(); - assertThat(productionJavaContains("EventFileQuery")) - .isFalse(); - } - - private static Document rootPom() throws Exception { - return modulePom("pom.xml"); - } - - private static Document modulePom(String path) throws Exception { - DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance(); - factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true); - return factory.newDocumentBuilder().parse(Files.newInputStream(repositoryRoot().resolve(path))); - } - - private static String projectProperty(Document pom, String name) { - Element properties = firstDirectChild(pom.getDocumentElement(), "properties"); - Element property = firstDirectChild(properties, name); - if (property == null) { - throw new IllegalArgumentException("project property not found: " + name); - } - return property.getTextContent().trim(); - } - - private static boolean hasProjectProperty(Document pom, String name) { - Element properties = firstDirectChild(pom.getDocumentElement(), "properties"); - return properties != null && firstDirectChild(properties, name) != null; - } - - private static List defaultModules(Document pom) { - Element project = pom.getDocumentElement(); - return directModules(firstDirectChild(project, "modules")); - } - - private static List profileModules(Document pom, String profileId) { - Element profile = profile(pom, profileId); - if (profile == null) { - return List.of(); - } - return directModules(firstDirectChild(profile, "modules")); - } - - private static List rootDependencyManagementArtifacts(Document pom) { - return dependencyManagementArtifacts(pom.getDocumentElement()); - } - - private static List profileDependencyManagementArtifacts(Document pom, String profileId) { - Element profile = profile(pom, profileId); - if (profile == null) { - return List.of(); - } - return dependencyManagementArtifacts(profile); - } - - private static List profileRepositoryIds(Document pom, String profileId) { - Element profile = profile(pom, profileId); - if (profile == null) { - return List.of(); - } - return repositoryIds(profile); - } - - private static List repositoryIds(Element owner) { - Element repositories = firstDirectChild(owner, "repositories"); - if (repositories == null) { - return List.of(); - } - List ids = new ArrayList<>(); - NodeList children = repositories.getChildNodes(); - for (int i = 0; i < children.getLength(); i++) { - if (!(children.item(i) instanceof Element repository) || !"repository".equals(repository.getTagName())) { - continue; - } - Element id = firstDirectChild(repository, "id"); - if (id != null) { - ids.add(id.getTextContent().trim()); - } - } - return List.copyOf(ids); - } - - private static Element profile(Document pom, String profileId) { - Element profiles = firstDirectChild(pom.getDocumentElement(), "profiles"); - if (profiles == null) { - return null; - } - NodeList children = profiles.getChildNodes(); - for (int i = 0; i < children.getLength(); i++) { - if (!(children.item(i) instanceof Element profile) || !"profile".equals(profile.getTagName())) { - continue; - } - Element id = firstDirectChild(profile, "id"); - if (id != null && profileId.equals(id.getTextContent().trim())) { - return profile; - } - } - return null; - } - - private static List directModules(Element modules) { - if (modules == null) { - return List.of(); - } - List out = new ArrayList<>(); - NodeList children = modules.getChildNodes(); - for (int i = 0; i < children.getLength(); i++) { - if (children.item(i) instanceof Element child && "module".equals(child.getTagName())) { - out.add(child.getTextContent().trim()); - } - } - return List.copyOf(out); - } - - private static List dependencyManagementArtifacts(Element owner) { - Element dependencyManagement = firstDirectChild(owner, "dependencyManagement"); - if (dependencyManagement == null) { - return List.of(); - } - Element dependencies = firstDirectChild(dependencyManagement, "dependencies"); - if (dependencies == null) { - return List.of(); - } - List artifacts = new ArrayList<>(); - NodeList children = dependencies.getChildNodes(); - for (int i = 0; i < children.getLength(); i++) { - if (!(children.item(i) instanceof Element dependency) || !"dependency".equals(dependency.getTagName())) { - continue; - } - Element groupId = firstDirectChild(dependency, "groupId"); - Element artifactId = firstDirectChild(dependency, "artifactId"); - if (groupId != null - && artifactId != null - && "com.lingniu.ingest".equals(groupId.getTextContent().trim())) { - artifacts.add(artifactId.getTextContent().trim()); - } - } - return List.copyOf(artifacts); - } - - private static Element managedPlugin(Document pom, String groupId, String artifactId) { - Element build = firstDirectChild(pom.getDocumentElement(), "build"); - Element pluginManagement = build == null ? null : firstDirectChild(build, "pluginManagement"); - return plugin(pluginManagement, groupId, artifactId); - } - - private static Element buildPlugin(Document pom, String groupId, String artifactId) { - Element build = firstDirectChild(pom.getDocumentElement(), "build"); - return plugin(build, groupId, artifactId); - } - - private static Element plugin(Element owner, String groupId, String artifactId) { - Element plugins = owner == null ? null : firstDirectChild(owner, "plugins"); - if (plugins == null) { - return null; - } - NodeList children = plugins.getChildNodes(); - for (int i = 0; i < children.getLength(); i++) { - if (!(children.item(i) instanceof Element plugin) || !"plugin".equals(plugin.getTagName())) { - continue; - } - Element currentGroupId = firstDirectChild(plugin, "groupId"); - Element currentArtifactId = firstDirectChild(plugin, "artifactId"); - if (currentGroupId != null - && currentArtifactId != null - && groupId.equals(currentGroupId.getTextContent().trim()) - && artifactId.equals(currentArtifactId.getTextContent().trim())) { - return plugin; - } - } - return null; - } - - private static boolean hasDependency(Document pom, String groupId, String artifactId) { - Element owner = pom.getDocumentElement(); - Element dependencyManagement = firstDirectChild(owner, "dependencyManagement"); - Element dependencies = firstDirectChild(owner, "dependencies"); - if (hasDependencyIn(dependencies, groupId, artifactId)) { - return true; - } - if (dependencyManagement == null) { - return false; - } - return hasDependencyIn(firstDirectChild(dependencyManagement, "dependencies"), groupId, artifactId); - } - - private static boolean hasDependencyIn(Element dependencies, String groupId, String artifactId) { - if (dependencies == null) { - return false; - } - NodeList children = dependencies.getChildNodes(); - for (int i = 0; i < children.getLength(); i++) { - if (!(children.item(i) instanceof Element dependency) || !"dependency".equals(dependency.getTagName())) { - continue; - } - Element currentGroupId = firstDirectChild(dependency, "groupId"); - Element currentArtifactId = firstDirectChild(dependency, "artifactId"); - if (currentGroupId != null - && currentArtifactId != null - && groupId.equals(currentGroupId.getTextContent().trim()) - && artifactId.equals(currentArtifactId.getTextContent().trim())) { - return true; - } - } - return false; - } - - private static boolean productionJavaContains(String token) throws Exception { - return javaSourcesContain(token, false); - } - - private static boolean javaSourcesContain(String token) throws Exception { - return javaSourcesContain(token, true); - } - - private static boolean javaSourcesContain(String token, boolean includeTests) throws Exception { - try (Stream files = Files.walk(repositoryRoot().resolve("modules"))) { - return files - .filter(Files::isRegularFile) - .filter(path -> path.toString().endsWith(".java")) - .filter(path -> includeTests || !path.toString().contains("/src/test/")) - .anyMatch(path -> { - try { - return Files.readString(path).contains(token); - } catch (Exception e) { - throw new IllegalStateException("failed to read " + path, e); - } - }); - } - } - - private static List javaFilesNamed(String fileName) throws Exception { - Path root = repositoryRoot(); - try (Stream files = Files.walk(root.resolve("modules"))) { - return files - .filter(Files::isRegularFile) - .filter(path -> fileName.equals(path.getFileName().toString())) - .map(root::relativize) - .map(Path::toString) - .sorted() - .toList(); - } - } - - private static Element firstDirectChild(Element parent, String tagName) { - NodeList children = parent.getChildNodes(); - for (int i = 0; i < children.getLength(); i++) { - if (children.item(i) instanceof Element child && tagName.equals(child.getTagName())) { - return child; - } - } - return null; - } - - private static Path repositoryRoot() { - Path current = Path.of(System.getProperty("user.dir")).toAbsolutePath(); - while (current != null) { - if (Files.exists(current.resolve("pom.xml")) && Files.exists(current.resolve("modules"))) { - return current; - } - current = current.getParent(); - } - throw new IllegalStateException("repository root not found"); - } -} diff --git a/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/PortainerComposeResourceLimitsTest.java b/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/PortainerComposeResourceLimitsTest.java deleted file mode 100644 index 98fa48a7..00000000 --- a/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/PortainerComposeResourceLimitsTest.java +++ /dev/null @@ -1,679 +0,0 @@ -package com.lingniu.ingest.historyapp; - -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.config.YamlPropertiesFactoryBean; -import org.springframework.core.io.FileSystemResource; - -import java.io.IOException; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.List; -import java.util.Properties; - -import static org.assertj.core.api.Assertions.assertThat; - -class PortainerComposeResourceLimitsTest { - - @Test - void portainerStackDocumentsDefaultProductionSurfaceOnly() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - - assertThat(compose) - .contains("Default production stack: GB32960, JT808, Yutong MQTT, history, analytics.") - .contains("Legacy and optional services stay outside this compose file.") - .doesNotContain("KAFKA_ENABLED") - .doesNotContain("\n xinda-push-app:\n") - .doesNotContain("\n command-gateway:\n") - .doesNotContain("\n vehicle-state-service:\n") - .doesNotContain("\n raw-archive-store:\n") - .doesNotContain("\n event-file-store:\n"); - } - - @Test - void productionServicesHaveOverridableMemoryLimits() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - - assertServiceMemoryLimit(compose, "gb32960-ingest-app", "GB32960_MEM_LIMIT", "768m"); - assertServiceMemoryLimit(compose, "jt808-ingest-app", "JT808_MEM_LIMIT", "768m"); - assertServiceMemoryLimit(compose, "yutong-mqtt-app", "YUTONG_MQTT_MEM_LIMIT", "512m"); - assertServiceMemoryLimit(compose, "vehicle-history-app", "VEHICLE_HISTORY_MEM_LIMIT", "1536m"); - assertServiceMemoryLimit(compose, "vehicle-analytics-app", "VEHICLE_ANALYTICS_MEM_LIMIT", "1024m"); - } - - @Test - void productionImagesShareOverridableRegistryAndNamespace() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String registry = "${LINGNIU_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com}"; - String namespace = "${LINGNIU_IMAGE_NAMESPACE:-oneos}"; - - for (String serviceName : List.of( - "gb32960-ingest-app", - "jt808-ingest-app", - "yutong-mqtt-app", - "vehicle-history-app", - "vehicle-analytics-app")) { - assertThat(serviceBlock(compose, serviceName)) - .contains("image: " + registry + "/" + namespace + "/" + serviceName - + ":${LINGNIU_IMAGE_VERSION:?set LINGNIU_IMAGE_VERSION}"); - } - assertThat(countOccurrences(compose, "image: crpi-85r4m0ackrm3qpje")) - .isZero(); - assertThat(countOccurrences(compose, "/oneos/")).isZero(); - } - - @Test - void portainerServicesUseContainerReadinessHealthchecks() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String dockerfile = Files.readString(repositoryRoot().resolve("Dockerfile")); - - assertThat(dockerfile).contains("apt-get install -y --no-install-recommends curl"); - assertThat(compose) - .contains("x-readiness-healthcheck: &readiness-healthcheck\n" - + " interval: 30s\n" - + " timeout: 5s\n" - + " retries: 3\n" - + " start_period: 60s"); - assertServiceReadinessHealthcheck(compose, "gb32960-ingest-app", "20100"); - assertServiceReadinessHealthcheck(compose, "jt808-ingest-app", "20400"); - assertServiceReadinessHealthcheck(compose, "yutong-mqtt-app", "20500"); - assertServiceReadinessHealthcheck(compose, "vehicle-history-app", "20200"); - assertServiceReadinessHealthcheck(compose, "vehicle-analytics-app", "20310"); - } - - @Test - void containerDefaultsIncludeJava25UnsafeCompatibilityFlag() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String dockerfile = Files.readString(repositoryRoot().resolve("Dockerfile")); - - assertThat(dockerfile) - .contains("ENV JAVA_OPTS=\"--sun-misc-unsafe-memory-access=allow -Dio.netty.transport.noNative=true\""); - assertThat(commonEnvBlock(compose)) - .contains("JAVA_OPTS: ${JAVA_OPTS:---sun-misc-unsafe-memory-access=allow -Dio.netty.transport.noNative=true -XX:+UseZGC -XX:MaxRAMPercentage=75}"); - } - - @Test - void redisSessionDefaultsUseEcsPrivateEndpoint() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - - assertThat(compose) - .contains("REDIS_HOST: ${REDIS_HOST:-r-bp1u741kij7e51i481.redis.rds.aliyuncs.com}") - .doesNotContain("r-bp1u741kij7e51i481pd.redis.rds.aliyuncs.com"); - } - - @Test - void activeAppsExposeHealthProbeGroupsByDefault() { - Path root = repositoryRoot(); - List activeAppConfigs = List.of( - root.resolve("modules/apps/gb32960-ingest-app/src/main/resources/application.yml"), - root.resolve("modules/apps/jt808-ingest-app/src/main/resources/application.yml"), - root.resolve("modules/apps/yutong-mqtt-app/src/main/resources/application.yml"), - root.resolve("modules/apps/vehicle-history-app/src/main/resources/application.yml"), - root.resolve("modules/apps/vehicle-analytics-app/src/main/resources/application.yml")); - - for (Path config : activeAppConfigs) { - assertThat(yamlProperties(config)) - .as(config.toString()) - .containsEntry("management.endpoint.health.probes.enabled", - "${MANAGEMENT_HEALTH_PROBES_ENABLED:true}"); - } - } - - @Test - void productionRunbookChecksLivenessAndReadinessForActiveApps() throws IOException { - String runbook = Files.readString(repositoryRoot() - .resolve("docs/operations/gb32960-service-split-runbook.md")); - - for (String httpPort : List.of("20100", "20400", "20500", "20200", "20310")) { - assertThat(runbook) - .contains("curl -sS http://127.0.0.1:" + httpPort + "/actuator/health/liveness") - .contains("curl -sS http://127.0.0.1:" + httpPort + "/actuator/health/readiness"); - } - } - - @Test - void defaultProductionSurfaceDoesNotShipLocalLaunchctlRunners() throws IOException { - Path root = repositoryRoot(); - String runbook = Files.readString(root.resolve("docs/operations/gb32960-service-split-runbook.md")); - - assertThat(root.resolve("deploy/local/launchctl")).doesNotExist(); - assertThat(runbook) - .contains("Production services run on ECS through Portainer/Docker.") - .doesNotContain("Local launchctl Deployment") - .doesNotContain("deploy/local/launchctl") - .doesNotContain("launchctl bootstrap") - .doesNotContain("launchctl bootout"); - } - - @Test - void tdengineVerificationRunbookChecksLivenessAndReadinessForManagedApps() throws IOException { - String runbook = Files.readString(repositoryRoot() - .resolve("docs/operations/vehicle-ingest-tdengine-verification.md")); - - for (String httpPort : List.of("20100", "20400", "20500", "20200", "20310")) { - assertThat(runbook) - .contains("curl -sS http://127.0.0.1:" + httpPort + "/actuator/health/liveness") - .contains("curl -sS http://127.0.0.1:" + httpPort + "/actuator/health/readiness"); - } - } - - @Test - void operationRunbooksUseBuildFlagThatSkipsTestCompilation() throws IOException { - Path operations = repositoryRoot().resolve("docs/operations"); - String splitRunbook = Files.readString(operations.resolve("gb32960-service-split-runbook.md")); - String tdengineVerification = Files.readString(operations.resolve("vehicle-ingest-tdengine-verification.md")); - - assertThat(splitRunbook) - .contains("-Dmaven.test.skip=true") - .doesNotContain("-DskipTests"); - assertThat(tdengineVerification) - .contains("-Dmaven.test.skip=true") - .doesNotContain("-DskipTests"); - } - - @Test - void dockerEntrypointExecsJavaForSignalHandling() throws IOException { - String dockerfile = Files.readString(repositoryRoot().resolve("Dockerfile")); - - assertThat(dockerfile) - .contains("ENTRYPOINT [\"sh\", \"-c\", \"exec java $JAVA_OPTS -jar /app/app.jar\"]") - .doesNotContain("\"java $JAVA_OPTS -jar /app/app.jar\""); - } - - @Test - void gb32960ComposeExposesBothPlatformAccountsWithoutSecrets() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String gb32960Service = serviceBlock(compose, "gb32960-ingest-app"); - - assertThat(gb32960Service) - .contains("GB32960_PLATFORM_USER_HYUNDAI: ${GB32960_PLATFORM_USER_HYUNDAI:-Hyundai}") - .contains("GB32960_PLATFORM_PWD_HYUNDAI: ${GB32960_PLATFORM_PWD_HYUNDAI:-}") - .contains("GB32960_PLATFORM_IP_HYUNDAI: ${GB32960_PLATFORM_IP_HYUNDAI:-}") - .contains("GB32960_PLATFORM_USER_YUEJIN: ${GB32960_PLATFORM_USER_YUEJIN:-YueJin}") - .contains("GB32960_PLATFORM_PWD_YUEJIN: ${GB32960_PLATFORM_PWD_YUEJIN:-}") - .contains("GB32960_PLATFORM_IP_YUEJIN: ${GB32960_PLATFORM_IP_YUEJIN:-}"); - } - - @Test - void vehicleHistoryOnlyExposesBaseConsumerGroupEnvironmentName() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String historyService = serviceBlock(compose, "vehicle-history-app"); - - assertThat(historyService) - .contains("KAFKA_GROUP_HISTORY: ${KAFKA_GROUP_HISTORY:-vehicle-history}") - .doesNotContain("KAFKA_GROUP_HISTORY_GB32960_EVENT") - .doesNotContain("KAFKA_GROUP_HISTORY_JT808_EVENT") - .doesNotContain("KAFKA_GROUP_HISTORY_YUTONG_MQTT_EVENT") - .doesNotContain("KAFKA_GROUP_HISTORY_GB32960_RAW") - .doesNotContain("KAFKA_GROUP_HISTORY_JT808_RAW") - .doesNotContain("KAFKA_GROUP_HISTORY_YUTONG_MQTT_RAW"); - } - - @Test - void kafkaConsumerEnvironmentIsScopedToConsumerAppsOnly() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String commonEnv = commonEnvBlock(compose); - String ingestServices = serviceBlock(compose, "gb32960-ingest-app") - + serviceBlock(compose, "jt808-ingest-app") - + serviceBlock(compose, "yutong-mqtt-app"); - String historyService = serviceBlock(compose, "vehicle-history-app"); - String analyticsService = serviceBlock(compose, "vehicle-analytics-app"); - - assertThat(commonEnv) - .doesNotContain("KAFKA_CONSUMER_ENABLED") - .doesNotContain("KAFKA_CONSUMER_MAX_POLL_INTERVAL"); - assertThat(ingestServices) - .doesNotContain("KAFKA_CONSUMER_"); - assertThat(historyService) - .contains("KAFKA_CONSUMER_ENABLED: ${KAFKA_CONSUMER_ENABLED_HISTORY:-true}") - .contains("KAFKA_CONSUMER_MAX_POLL_INTERVAL_MILLIS: ${KAFKA_CONSUMER_MAX_POLL_INTERVAL_MILLIS_HISTORY:-1800000}") - .doesNotContain("KAFKA_CONSUMER_MAX_POLL_INTERVAL_MS"); - assertThat(analyticsService) - .contains("KAFKA_CONSUMER_ENABLED: ${KAFKA_CONSUMER_ENABLED_ANALYTICS:-true}") - .contains("KAFKA_CONSUMER_MAX_POLL_INTERVAL_MILLIS: ${KAFKA_CONSUMER_MAX_POLL_INTERVAL_MILLIS_ANALYTICS:-900000}"); - } - - @Test - void kafkaTopicEnvironmentIsScopedToServicesThatUseIt() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String commonEnv = commonEnvBlock(compose); - String gb32960Service = serviceBlock(compose, "gb32960-ingest-app"); - String jt808Service = serviceBlock(compose, "jt808-ingest-app"); - String mqttService = serviceBlock(compose, "yutong-mqtt-app"); - String historyService = serviceBlock(compose, "vehicle-history-app"); - String analyticsService = serviceBlock(compose, "vehicle-analytics-app"); - - assertThat(commonEnv).doesNotContain("KAFKA_TOPIC_"); - assertThat(gb32960Service) - .contains("KAFKA_TOPIC_GB32960_EVENT: ${KAFKA_TOPIC_GB32960_EVENT:-vehicle.event.gb32960.v1}") - .contains("KAFKA_TOPIC_GB32960_RAW: ${KAFKA_TOPIC_GB32960_RAW:-vehicle.raw.gb32960.v1}") - .contains("KAFKA_TOPIC_GB32960_DLQ: ${KAFKA_TOPIC_GB32960_DLQ:-vehicle.dlq.gb32960.v1}") - .doesNotContain("KAFKA_TOPIC_JT808") - .doesNotContain("KAFKA_TOPIC_YUTONG_MQTT"); - assertThat(jt808Service) - .contains("KAFKA_TOPIC_JT808_EVENT: ${KAFKA_TOPIC_JT808_EVENT:-vehicle.event.jt808.v1}") - .contains("KAFKA_TOPIC_JT808_RAW: ${KAFKA_TOPIC_JT808_RAW:-vehicle.raw.jt808.v1}") - .contains("KAFKA_TOPIC_JT808_DLQ: ${KAFKA_TOPIC_JT808_DLQ:-vehicle.dlq.jt808.v1}") - .doesNotContain("KAFKA_TOPIC_GB32960") - .doesNotContain("KAFKA_TOPIC_YUTONG_MQTT"); - assertThat(mqttService) - .contains("KAFKA_TOPIC_YUTONG_MQTT_EVENT: ${KAFKA_TOPIC_YUTONG_MQTT_EVENT:-vehicle.event.mqtt-yutong.v1}") - .contains("KAFKA_TOPIC_YUTONG_MQTT_RAW: ${KAFKA_TOPIC_YUTONG_MQTT_RAW:-vehicle.raw.mqtt-yutong.v1}") - .contains("KAFKA_TOPIC_YUTONG_MQTT_DLQ: ${KAFKA_TOPIC_YUTONG_MQTT_DLQ:-vehicle.dlq.mqtt-yutong.v1}") - .doesNotContain("KAFKA_TOPIC_GB32960") - .doesNotContain("KAFKA_TOPIC_JT808"); - assertThat(historyService) - .contains("KAFKA_TOPIC_GB32960_EVENT: ${KAFKA_TOPIC_GB32960_EVENT:-vehicle.event.gb32960.v1}") - .contains("KAFKA_TOPIC_GB32960_RAW: ${KAFKA_TOPIC_GB32960_RAW:-vehicle.raw.gb32960.v1}") - .contains("KAFKA_TOPIC_JT808_EVENT: ${KAFKA_TOPIC_JT808_EVENT:-vehicle.event.jt808.v1}") - .contains("KAFKA_TOPIC_JT808_RAW: ${KAFKA_TOPIC_JT808_RAW:-vehicle.raw.jt808.v1}") - .contains("KAFKA_TOPIC_YUTONG_MQTT_EVENT: ${KAFKA_TOPIC_YUTONG_MQTT_EVENT:-vehicle.event.mqtt-yutong.v1}") - .contains("KAFKA_TOPIC_YUTONG_MQTT_RAW: ${KAFKA_TOPIC_YUTONG_MQTT_RAW:-vehicle.raw.mqtt-yutong.v1}") - .contains("KAFKA_TOPIC_HISTORY_DLQ: ${KAFKA_TOPIC_HISTORY_DLQ:-vehicle.dlq.history.v1}") - .doesNotContain("KAFKA_TOPIC_GB32960_DLQ") - .doesNotContain("KAFKA_TOPIC_JT808_DLQ") - .doesNotContain("KAFKA_TOPIC_YUTONG_MQTT_DLQ"); - assertThat(analyticsService) - .contains("KAFKA_TOPIC_JT808_EVENT: ${KAFKA_TOPIC_JT808_EVENT:-vehicle.event.jt808.v1}") - .contains("KAFKA_TOPIC_JT808_DLQ: ${KAFKA_TOPIC_JT808_DLQ:-vehicle.dlq.jt808.v1}") - .doesNotContain("KAFKA_TOPIC_GB32960") - .doesNotContain("KAFKA_TOPIC_YUTONG_MQTT") - .doesNotContain("KAFKA_TOPIC_JT808_RAW"); - } - - @Test - void vehicleHistoryUsesTdengineHistoryEnvironmentNames() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - - assertThat(compose) - .contains("TDENGINE_HISTORY_ENABLED: ${TDENGINE_HISTORY_ENABLED:-true}") - .contains("TDENGINE_HISTORY_DATABASE: ${TDENGINE_HISTORY_DATABASE:-vehicle_ts}") - .doesNotContain("TDENGINE_TELEMETRY_FIELDS_ENABLED") - .doesNotContain("\n TDENGINE_ENABLED:") - .doesNotContain("\n TDENGINE_DATABASE:"); - } - - @Test - void vehicleHistoryDoesNotExposeUnusedArchiveSinkSettings() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String historyService = serviceBlock(compose, "vehicle-history-app"); - - assertThat(historyService) - .doesNotContain("SINK_ARCHIVE_ENABLED") - .doesNotContain("SINK_ARCHIVE_PATH") - .doesNotContain("/archive"); - assertThat(compose) - .doesNotContain("vehicle-history-archive:"); - } - - @Test - void portainerIngestAppsWriteColdArchiveToMountedDataVolume() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String gb32960Service = serviceBlock(compose, "gb32960-ingest-app"); - String jt808Service = serviceBlock(compose, "jt808-ingest-app"); - String mqttService = serviceBlock(compose, "yutong-mqtt-app"); - - assertThat(gb32960Service) - .contains("SINK_ARCHIVE_PATH: ${GB32960_ARCHIVE_PATH:-/data/archive}") - .contains("- gb32960-ingest-data:/data"); - assertThat(jt808Service) - .contains("SINK_ARCHIVE_PATH: ${JT808_ARCHIVE_PATH:-/data/archive}") - .contains("- jt808-ingest-data:/data"); - assertThat(mqttService) - .contains("SINK_ARCHIVE_PATH: ${YUTONG_MQTT_ARCHIVE_PATH:-/data/archive}") - .contains("- yutong-mqtt-data:/data"); - assertThat(compose).contains("\n yutong-mqtt-data:\n"); - } - - @Test - void operationRunbooksDescribeTdengineRawJsonHotPath() throws IOException { - Path operations = repositoryRoot().resolve("docs/operations"); - String tdengineVerification = Files.readString(operations.resolve("vehicle-ingest-tdengine-verification.md")); - String splitRunbook = Files.readString(operations.resolve("gb32960-service-split-runbook.md")); - - assertThat(tdengineVerification) - .contains("TDengine `raw_frames`") - .doesNotContain("20482") - .doesNotContain("三个服务必须共用同一个 `SINK_ARCHIVE_PATH`") - .doesNotContain("再回读共享 archive") - .doesNotContain("读取共享 archive 中的原始 `.bin`") - .doesNotContain("EVENT_FILE_STORE_ENABLED") - .doesNotContain("EVENT_FILE_STORE_PATH") - .doesNotContain("TDENGINE_TELEMETRY_FIELDS_ENABLED") - .doesNotContain("/api/event-history/telemetry/fields' 会存在") - .doesNotContain("/api/event-history/records"); - assertThat(splitRunbook) - .contains("TDengine `raw_frames`") - .doesNotContain("must share the same `SINK_ARCHIVE_PATH`") - .doesNotContain("history can only read raw frames") - .doesNotContain("shared volume mounted to all three services"); - } - - @Test - void splitRunbookDocumentsAllActiveProductionAppsAndTopics() throws IOException { - String splitRunbook = Files.readString(repositoryRoot() - .resolve("docs/operations/gb32960-service-split-runbook.md")); - String activeBuildCommand = - "mvn -pl :gb32960-ingest-app,:jt808-ingest-app,:yutong-mqtt-app,:vehicle-history-app,:vehicle-analytics-app -am package -Dmaven.test.skip=true"; - - assertThat(splitRunbook) - .contains("| Yutong MQTT ingest | `:yutong-mqtt-app`") - .contains("`vehicle.raw.mqtt-yutong.v1`") - .contains("`vehicle.event.mqtt-yutong.v1`") - .contains("`vehicle.dlq.mqtt-yutong.v1`") - .contains("`vehicle.dlq.history.v1`") - .contains(activeBuildCommand) - .contains("modules/apps/yutong-mqtt-app/target/yutong-mqtt-app.jar") - .doesNotContain("20482") - .doesNotContain("bootstrap-all") - .doesNotContain("mvn -pl :gb32960-ingest-app,:vehicle-history-app,:vehicle-analytics-app") - .doesNotContain("mvn -pl :gb32960-ingest-app,:jt808-ingest-app,:vehicle-history-app,:vehicle-analytics-app") - .doesNotContain("repackaged all three app jars"); - } - - @Test - void tdengineVerificationRunbookDocumentsAllActiveProductionApps() throws IOException { - String runbook = Files.readString(repositoryRoot() - .resolve("docs/operations/vehicle-ingest-tdengine-verification.md")); - String activeBuildCommand = - "mvn -pl :gb32960-ingest-app,:jt808-ingest-app,:yutong-mqtt-app,:vehicle-history-app,:vehicle-analytics-app -am package -Dmaven.test.skip=true"; - - assertThat(runbook) - .contains(activeBuildCommand) - .contains("modules/apps/gb32960-ingest-app/target/gb32960-ingest-app.jar") - .contains("modules/apps/jt808-ingest-app/target/jt808-ingest-app.jar") - .contains("modules/apps/yutong-mqtt-app/target/yutong-mqtt-app.jar") - .contains("modules/apps/vehicle-history-app/target/vehicle-history-app.jar") - .contains("modules/apps/vehicle-analytics-app/target/vehicle-analytics-app.jar") - .contains("vehicle.raw.mqtt-yutong.v1") - .contains("vehicle.event.mqtt-yutong.v1") - .contains("vehicle-history-yutong-mqtt-event") - .contains("vehicle-history-yutong-mqtt-raw") - .contains("vehicle-analytics-app") - .doesNotContain("mvn -pl modules/apps/jt808-ingest-app,modules/apps/gb32960-ingest-app,modules/apps/vehicle-history-app") - .doesNotContain("健康检查已确认三个服务"); - } - - @Test - void architectureDocsNameKafkaInsteadOfGenericMq() throws IOException { - String decisions = Files.readString(repositoryRoot().resolve("DECISIONS.md")); - String dataFlow = Files.readString(repositoryRoot().resolve("docs/module-data-flow.html")); - - assertThat(decisions) - .contains("生产链路只支持 Kafka") - .doesNotContain("MQ 选型") - .doesNotContain("RocketMQ"); - assertThat(dataFlow) - .contains("participant Kafka as sink-kafka") - .doesNotContain("participant MQ") - .doesNotContain("->>MQ") - .doesNotContain("MQ->>"); - } - - @Test - void activeSpringBootAppsDoNotScanWholeIngestPackage() throws IOException { - Path root = repositoryRoot(); - List activeApps = List.of( - root.resolve("modules/apps/gb32960-ingest-app/src/main/java/com/lingniu/ingest/gb32960app/Gb32960IngestApplication.java"), - root.resolve("modules/apps/jt808-ingest-app/src/main/java/com/lingniu/ingest/jt808app/Jt808IngestApplication.java"), - root.resolve("modules/apps/yutong-mqtt-app/src/main/java/com/lingniu/ingest/yutongmqttapp/YutongMqttApplication.java"), - root.resolve("modules/apps/vehicle-history-app/src/main/java/com/lingniu/ingest/historyapp/VehicleHistoryApplication.java"), - root.resolve("modules/apps/vehicle-analytics-app/src/main/java/com/lingniu/ingest/analyticsapp/VehicleAnalyticsApplication.java")); - - for (Path app : activeApps) { - assertThat(Files.readString(app)) - .contains("@SpringBootApplication") - .doesNotContain("scanBasePackages = \"com.lingniu.ingest\""); - } - } - - @Test - void splitRunbookKeepsAnalyticsAsJt808MetricRuntimeOnly() throws IOException { - String splitRunbook = Files.readString(repositoryRoot() - .resolve("docs/operations/gb32960-service-split-runbook.md")); - - assertThat(splitRunbook) - .contains("Consume JT808 event Kafka records") - .contains("vehicle_stat_metric") - .doesNotContain("Analytics state:") - .doesNotContain("VEHICLE_STATE_ENABLED") - .doesNotContain("latest-state API in analytics") - .doesNotContain("Vehicle state can also be enabled from this runtime"); - } - - @Test - void splitRunbookUsesTdengineHistoryVerificationInsteadOfLegacyRecordApi() throws IOException { - String splitRunbook = Files.readString(repositoryRoot() - .resolve("docs/operations/gb32960-service-split-runbook.md")); - - assertThat(splitRunbook) - .contains("TDENGINE_HISTORY_ENABLED=true") - .contains("SELECT COUNT(*) FROM raw_frames") - .contains("SELECT COUNT(*) FROM vehicle_locations") - .doesNotContain("EVENT_FILE_STORE_PATH") - .doesNotContain("target/split-event-store") - .doesNotContain("event-store files") - .doesNotContain("/api/event-history/records?"); - } - - @Test - void vehicleAnalyticsComposeOnlyExposesStatConsumerEnvironment() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String analyticsService = serviceBlock(compose, "vehicle-analytics-app"); - - assertThat(analyticsService) - .contains("KAFKA_GROUP_STAT: ${KAFKA_GROUP_STAT:-vehicle-stat}") - .contains("VEHICLE_STAT_ENABLED: ${VEHICLE_STAT_ENABLED:-true}") - .contains("VEHICLE_STAT_JT808_MILEAGE_ENABLED: ${VEHICLE_STAT_JT808_MILEAGE_ENABLED:-true}") - .doesNotContain("KAFKA_GROUP_STATE") - .doesNotContain("VEHICLE_STATE_ENABLED"); - } - - @Test - void kafkaConsumerAppsDoNotDependOnProtocolIngestContainers() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String historyService = serviceBlock(compose, "vehicle-history-app"); - String analyticsService = serviceBlock(compose, "vehicle-analytics-app"); - - assertThat(historyService) - .doesNotContain(" depends_on:") - .doesNotContain("- gb32960-ingest-app") - .doesNotContain("- jt808-ingest-app") - .doesNotContain("- yutong-mqtt-app"); - assertThat(analyticsService) - .doesNotContain(" depends_on:") - .doesNotContain("- gb32960-ingest-app") - .doesNotContain("- jt808-ingest-app") - .doesNotContain("- yutong-mqtt-app"); - } - - @Test - void vehicleAnalyticsDoesNotInheritRedisStateEnvironment() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String commonEnv = commonEnvBlock(compose); - String analyticsService = serviceBlock(compose, "vehicle-analytics-app"); - - assertThat(analyticsService).contains("<<: *common-env"); - assertThat(commonEnv) - .doesNotContain("REDIS_HOST") - .doesNotContain("REDIS_PORT") - .doesNotContain("REDIS_DATABASE") - .doesNotContain("REDIS_USERNAME") - .doesNotContain("REDIS_PASSWORD"); - } - - @Test - void mysqlEnvironmentIsScopedToIdentityAndAnalyticsOnly() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String commonEnv = commonEnvBlock(compose); - String identityEnv = blockBetween(compose, "x-identity-mysql-env: &identity-mysql-env\n", - "\nx-redis-session-env:"); - String ingestServices = serviceBlock(compose, "gb32960-ingest-app") - + serviceBlock(compose, "jt808-ingest-app") - + serviceBlock(compose, "yutong-mqtt-app"); - String analyticsService = serviceBlock(compose, "vehicle-analytics-app"); - - assertThat(commonEnv) - .doesNotContain("MYSQL_JDBC_URL") - .doesNotContain("MYSQL_USERNAME") - .doesNotContain("MYSQL_PASSWORD"); - assertThat(identityEnv) - .contains("VEHICLE_IDENTITY_MYSQL_JDBC_URL: ${MYSQL_JDBC_URL:-}") - .contains("VEHICLE_IDENTITY_MYSQL_USERNAME: ${MYSQL_USERNAME:-}") - .contains("VEHICLE_IDENTITY_MYSQL_PASSWORD: ${MYSQL_PASSWORD:-}"); - assertThat(ingestServices) - .doesNotContain("\n MYSQL_JDBC_URL:") - .doesNotContain("\n MYSQL_USERNAME:") - .doesNotContain("\n MYSQL_PASSWORD:"); - assertThat(analyticsService) - .contains("MYSQL_JDBC_URL: ${MYSQL_JDBC_URL:-}") - .contains("MYSQL_USERNAME: ${MYSQL_USERNAME:-}") - .contains("MYSQL_PASSWORD: ${MYSQL_PASSWORD:-}"); - } - - @Test - void ingestAppsReuseSharedIdentityAndRedisEnvironmentAnchors() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - String gb32960Service = serviceBlock(compose, "gb32960-ingest-app"); - String jt808Service = serviceBlock(compose, "jt808-ingest-app"); - String mqttService = serviceBlock(compose, "yutong-mqtt-app"); - - assertThat(compose) - .contains("x-identity-mysql-env: &identity-mysql-env") - .contains("x-redis-session-env: &redis-session-env") - .doesNotContain("VEHICLE_IDENTITY_STORE"); - assertThat(gb32960Service) - .contains("- *identity-mysql-env") - .contains("- *redis-session-env"); - assertThat(jt808Service) - .contains("- *identity-mysql-env") - .contains("- *redis-session-env"); - assertThat(mqttService) - .contains("- *identity-mysql-env") - .doesNotContain("- *redis-session-env"); - - assertThat(gb32960Service + jt808Service + mqttService) - .doesNotContain("VEHICLE_IDENTITY_MYSQL_JDBC_URL") - .doesNotContain("VEHICLE_IDENTITY_MYSQL_USERNAME") - .doesNotContain("VEHICLE_IDENTITY_MYSQL_PASSWORD") - .doesNotContain("REDIS_HOST") - .doesNotContain("REDIS_PORT") - .doesNotContain("REDIS_DATABASE") - .doesNotContain("REDIS_USERNAME") - .doesNotContain("REDIS_PASSWORD"); - } - - @Test - void moduleDataFlowShowsSplitProductionAppsInsteadOfDeletedBootstrapAll() throws IOException { - String moduleDataFlow = Files.readString(repositoryRoot().resolve("docs/module-data-flow.html")); - - assertThat(moduleDataFlow) - .contains("gb32960-ingest-app") - .contains("jt808-ingest-app") - .contains("yutong-mqtt-app") - .contains("vehicle-history-app") - .contains("vehicle-analytics-app") - .doesNotContain("bootstrap-all"); - } - - private static void assertServiceMemoryLimit(String compose, - String serviceName, - String envName, - String defaultLimit) { - assertThat(compose) - .contains(serviceName + ":\n") - .contains("\n " + serviceName + ":\n") - .contains(" mem_limit: ${" + envName + ":-" + defaultLimit + "}"); - } - - private static void assertServiceReadinessHealthcheck(String compose, String serviceName, String httpPort) { - assertThat(serviceBlock(compose, serviceName)) - .contains(" healthcheck:\n") - .contains(" <<: *readiness-healthcheck\n") - .contains(" test: [\"CMD-SHELL\", \"curl -fsS http://127.0.0.1:" - + httpPort + "/actuator/health/readiness >/dev/null\"]") - .doesNotContain(" interval: 30s\n") - .doesNotContain(" timeout: 5s\n") - .doesNotContain(" retries: 3\n") - .doesNotContain(" start_period: 60s\n"); - } - - private static String serviceBlock(String compose, String serviceName) { - String marker = "\n " + serviceName + ":\n"; - int start = compose.indexOf(marker); - if (start < 0) { - throw new IllegalArgumentException("service not found: " + serviceName); - } - for (int cursor = start + marker.length(); cursor < compose.length(); ) { - int nextLine = compose.indexOf('\n', cursor); - if (nextLine < 0) { - return compose.substring(start); - } - int lineStart = nextLine + 1; - if (lineStart + 2 < compose.length() - && compose.charAt(lineStart) == ' ' - && compose.charAt(lineStart + 1) == ' ' - && compose.charAt(lineStart + 2) != ' ') { - return compose.substring(start, nextLine); - } - cursor = lineStart; - } - return compose.substring(start); - } - - private static String commonEnvBlock(String compose) { - String marker = "x-common-env: &common-env\n"; - int start = compose.indexOf(marker); - if (start < 0) { - throw new IllegalArgumentException("common env block not found"); - } - int end = compose.indexOf("\nx-identity-mysql-env:", start); - if (end < 0) { - end = compose.indexOf("\nservices:\n", start); - } - if (end < 0) { - throw new IllegalArgumentException("services block not found"); - } - return compose.substring(start, end); - } - - private static String blockBetween(String text, String startMarker, String endMarker) { - int start = text.indexOf(startMarker); - if (start < 0) { - throw new IllegalArgumentException("block start not found: " + startMarker); - } - int end = text.indexOf(endMarker, start + startMarker.length()); - if (end < 0) { - throw new IllegalArgumentException("block end not found: " + endMarker); - } - return text.substring(start, end); - } - - private static int countOccurrences(String text, String needle) { - int count = 0; - int index = 0; - while ((index = text.indexOf(needle, index)) >= 0) { - count++; - index += needle.length(); - } - return count; - } - - private static Properties yamlProperties(Path path) { - YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean(); - factory.setResources(new FileSystemResource(path)); - Properties properties = factory.getObject(); - assertThat(properties).isNotNull(); - return properties; - } - - private static Path repositoryRoot() { - Path current = Path.of(System.getProperty("user.dir")).toAbsolutePath(); - while (current != null) { - if (Files.exists(current.resolve("deploy/portainer/docker-compose.yml"))) { - return current; - } - current = current.getParent(); - } - throw new IllegalStateException("repository root not found"); - } -} diff --git a/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppCompositionTest.java b/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppCompositionTest.java deleted file mode 100644 index a7140237..00000000 --- a/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppCompositionTest.java +++ /dev/null @@ -1,208 +0,0 @@ -package com.lingniu.ingest.historyapp; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.api.consumer.EnvelopeDeadLetterSink; -import com.lingniu.ingest.api.consumer.EnvelopeIngestResult; -import com.lingniu.ingest.eventhistory.EventHistoryEnvelopeIngestor; -import com.lingniu.ingest.eventhistory.Gb32960DecodedFrameService; -import com.lingniu.ingest.eventhistory.Gb32960FrameController; -import com.lingniu.ingest.eventhistory.config.EventHistoryAutoConfiguration; -import com.lingniu.ingest.eventhistory.Jt808LocationHistoryController; -import com.lingniu.ingest.eventhistory.LocationHistoryController; -import com.lingniu.ingest.eventhistory.RawFrameHistoryController; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960NettyServer; -import com.lingniu.ingest.sink.kafka.KafkaEnvelopeDeadLetterSink; -import com.lingniu.ingest.sink.kafka.KafkaEventSink; -import com.lingniu.ingest.sink.kafka.KafkaEnvelopeConsumerRunner; -import com.lingniu.ingest.sink.kafka.KafkaSinkProperties; -import com.lingniu.ingest.sink.kafka.KafkaSinkAutoConfiguration; -import com.lingniu.ingest.tdenginehistory.TdengineHistorySchema; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryWriter; -import com.lingniu.ingest.tdenginehistory.config.TdengineHistoryAutoConfiguration; -import org.apache.kafka.clients.producer.KafkaProducer; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.context.annotation.Bean; - -import java.lang.reflect.Method; -import java.nio.file.Path; -import java.util.Arrays; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.mock; - -class VehicleHistoryAppCompositionTest { - - @TempDir - Path tempDir; - - @Test - void kafkaConsumerConfigurationDoesNotRedeclareServiceOwnedHistoryBeans() { - List beanMethods = Arrays.stream(VehicleHistoryKafkaConsumerConfiguration.class.getDeclaredMethods()) - .filter(method -> method.isAnnotationPresent(Bean.class)) - .map(Method::getName) - .toList(); - - assertThat(beanMethods) - .doesNotContain("telemetryEnvelopeRecordMapper") - .doesNotContain("eventHistoryEnvelopeIngestor") - .doesNotContain("eventHistoryEnvelopeConsumerProcessor") - .doesNotContain("eventHistoryRawEnvelopeConsumerProcessor"); - } - - @Test - void kafkaConsumerRunnerRejectsGenericHistoryBindings() { - new ApplicationContextRunner() - .withUserConfiguration(VehicleHistoryKafkaConsumerConfiguration.class) - .withBean("eventHistoryEnvelopeConsumerProcessor", EnvelopeConsumerProcessor.class, - () -> processor("event-history")) - .withBean("eventHistoryRawEnvelopeConsumerProcessor", EnvelopeConsumerProcessor.class, - () -> processor("event-history-raw")) - .withBean(KafkaSinkProperties.class, VehicleHistoryAppCompositionTest::genericOnlyHistoryConsumerProps) - .withPropertyValues("lingniu.ingest.sink.kafka.consumer.enabled=true") - .run(context -> { - assertThat(context).hasFailed(); - assertThat(context.getStartupFailure()) - .hasRootCauseMessage("no vehicle history kafka consumer workers created; check consumer bindings"); - }); - } - - @Test - void createsTdengineHistoryStorageAndQueryBeansWithoutGb32960TcpServerOrLocalArchiveSink() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of( - TdengineHistoryAutoConfiguration.class, - KafkaSinkAutoConfiguration.class, - EventHistoryAutoConfiguration.class)) - .withUserConfiguration(VehicleHistoryKafkaConsumerConfiguration.class) - .withAllowBeanDefinitionOverriding(true) - .withBean("kafkaProducer", KafkaProducer.class, VehicleHistoryAppCompositionTest::kafkaProducer) - .withPropertyValues( - "lingniu.ingest.tdengine-history.enabled=true", - "lingniu.ingest.tdengine-history.database=vehicle_history_test", - "lingniu.ingest.event-history.enabled=true", - "lingniu.ingest.sink.kafka.enabled=true", - "lingniu.ingest.sink.kafka.bootstrap-servers=localhost:9092", - "lingniu.ingest.sink.kafka.consumer.enabled=false", - "lingniu.ingest.vehicle-state.enabled=false", - "lingniu.ingest.vehicle-stat.enabled=false") - .run(context -> { - assertThat(context).doesNotHaveBean("archiveStore"); - assertThat(context).doesNotHaveBean("rawArchiveEventSink"); - assertThat(context).doesNotHaveBean("eventFileStore"); - assertThat(context).doesNotHaveBean("eventFileStoreSink"); - assertThat(context).hasSingleBean(TdengineHistorySchema.class); - assertThat(context).hasSingleBean(KafkaEventSink.class); - assertThat(context).hasSingleBean(KafkaEnvelopeDeadLetterSink.class); - - assertThat(context).hasSingleBean(EventHistoryEnvelopeIngestor.class); - assertThat(context).hasSingleBean(Gb32960MessageDecoder.class); - assertThat(context).hasSingleBean(Gb32960DecodedFrameService.class); - assertThat(context).hasSingleBean(LocationHistoryController.class); - assertThat(context).hasSingleBean(RawFrameHistoryController.class); - assertThat(context).doesNotHaveBean("telemetryFieldHistoryController"); - assertThat(context).doesNotHaveBean(Gb32960FrameController.class); - assertThat(context).doesNotHaveBean(Jt808LocationHistoryController.class); - assertThat(context).doesNotHaveBean(Gb32960NettyServer.class); - }); - } - - @Test - void createsHistoryIngestorForTdengineOnlyRuntime() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(EventHistoryAutoConfiguration.class)) - .withUserConfiguration(VehicleHistoryKafkaConsumerConfiguration.class) - .withBean(TdengineHistoryWriter.class, () -> mock(TdengineHistoryWriter.class)) - .withBean(EnvelopeDeadLetterSink.class, () -> mock(EnvelopeDeadLetterSink.class)) - .withPropertyValues( - "lingniu.ingest.event-history.enabled=true", - "lingniu.ingest.sink.kafka.consumer.enabled=false") - .run(context -> { - assertThat(context).hasSingleBean(EventHistoryEnvelopeIngestor.class); - assertThat(context).doesNotHaveBean("eventFileStore"); - }); - } - - @Test - void applicationScanDoesNotExposeTelemetryFieldControllerByDefault() { - new ApplicationContextRunner() - .withUserConfiguration(VehicleHistoryApplication.class) - .withBean(TdengineHistoryReader.class, () -> mock(TdengineHistoryReader.class)) - .withPropertyValues( - "spring.cloud.nacos.config.enabled=false", - "spring.config.import=", - "lingniu.ingest.event-history.enabled=true", - "lingniu.ingest.tdengine-history.enabled=false", - "lingniu.ingest.sink.kafka.enabled=false", - "lingniu.ingest.sink.kafka.consumer.enabled=false") - .run(context -> { - assertThat(context).hasSingleBean(LocationHistoryController.class); - assertThat(context).hasSingleBean(RawFrameHistoryController.class); - assertThat(context).doesNotHaveBean("telemetryFieldHistoryController"); - }); - } - - @Test - void createsKafkaConsumerRunnerWhenTdengineHistoryRuntimeConsumesKafka() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of( - TdengineHistoryAutoConfiguration.class, - KafkaSinkAutoConfiguration.class, - EventHistoryAutoConfiguration.class)) - .withUserConfiguration(VehicleHistoryKafkaConsumerConfiguration.class) - .withAllowBeanDefinitionOverriding(true) - .withBean("kafkaProducer", KafkaProducer.class, VehicleHistoryAppCompositionTest::kafkaProducer) - .withPropertyValues( - "lingniu.ingest.tdengine-history.enabled=true", - "lingniu.ingest.tdengine-history.database=vehicle_history_test", - "lingniu.ingest.event-history.enabled=true", - "lingniu.ingest.sink.kafka.enabled=true", - "lingniu.ingest.sink.kafka.bootstrap-servers=localhost:9092", - "lingniu.ingest.sink.kafka.consumer.enabled=true", - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryGb32960EnvelopeConsumerProcessor.enabled=true", - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryGb32960EnvelopeConsumerProcessor.group-id=history-gb32960", - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryGb32960EnvelopeConsumerProcessor.topics[0]=vehicle.event.gb32960.v1") - .run(context -> { - assertThat(context).hasSingleBean(EventHistoryEnvelopeIngestor.class); - assertThat(context.getBeanNamesForType(EnvelopeConsumerProcessor.class)) - .contains("eventHistoryEnvelopeConsumerProcessor"); - assertThat(context).hasSingleBean(KafkaEnvelopeConsumerRunner.class); - }); - } - - @SuppressWarnings("unchecked") - private static KafkaProducer kafkaProducer() { - return mock(KafkaProducer.class); - } - - private static EnvelopeConsumerProcessor processor(String service) { - return new EnvelopeConsumerProcessor( - service, - ignored -> EnvelopeIngestResult.processed("event-1", "VIN001"), - ignored -> {}); - } - - private static KafkaSinkProperties genericOnlyHistoryConsumerProps() { - KafkaSinkProperties props = new KafkaSinkProperties(); - props.setBootstrapServers("localhost:9092"); - props.getConsumer().setEnabled(true); - props.getConsumer().setBindings(Map.of( - "eventHistoryEnvelopeConsumerProcessor", binding("history-event", "vehicle.event.generic.v1"), - "eventHistoryRawEnvelopeConsumerProcessor", binding("history-raw", "vehicle.raw.generic.v1"))); - return props; - } - - private static KafkaSinkProperties.Binding binding(String groupId, String topic) { - KafkaSinkProperties.Binding binding = new KafkaSinkProperties.Binding(); - binding.setGroupId(groupId); - binding.setTopics(List.of(topic)); - return binding; - } - -} diff --git a/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppDefaultsTest.java b/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppDefaultsTest.java deleted file mode 100644 index 313bd745..00000000 --- a/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/VehicleHistoryAppDefaultsTest.java +++ /dev/null @@ -1,151 +0,0 @@ -package com.lingniu.ingest.historyapp; - -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.config.YamlPropertiesFactoryBean; -import org.springframework.core.io.ClassPathResource; - -import java.io.IOException; -import java.net.URL; -import java.nio.charset.StandardCharsets; -import java.util.Collections; -import java.util.List; -import java.util.Properties; - -import static org.assertj.core.api.Assertions.assertThat; - -class VehicleHistoryAppDefaultsTest { - - @Test - void applicationDefaultsKeepHistoryAsDecoderConsumerAndStorageRuntime() throws IOException { - Properties properties = applicationProperties(); - - assertThat(properties) - .containsEntry("spring.application.name", "vehicle-history-app") - .containsEntry( - "spring.config.import[0]", - "optional:nacos:${spring.application.name}.${NACOS_CONFIG_FILE_EXTENSION:yml}?group=${NACOS_GROUP:DEFAULT_GROUP}&refreshEnabled=${NACOS_REFRESH_ENABLED:true}") - .containsEntry("spring.cloud.nacos.config.server-addr", "${NACOS_SERVER_ADDR:127.0.0.1:8848}") - .containsEntry("lingniu.ingest.tdengine-history.enabled", "${TDENGINE_HISTORY_ENABLED:false}") - .containsEntry("lingniu.ingest.tdengine-history.database", "${TDENGINE_HISTORY_DATABASE:vehicle_history}") - .containsEntry( - "lingniu.ingest.tdengine-history.jdbc-url", - "${TDENGINE_JDBC_URL:jdbc:TAOS-RS://${TDENGINE_HOST:127.0.0.1}:${TDENGINE_PORT:6041}/${TDENGINE_HISTORY_DATABASE:vehicle_history}}") - .containsEntry("lingniu.ingest.tdengine-history.username", "${TDENGINE_USERNAME:root}") - .containsEntry("lingniu.ingest.tdengine-history.password", "${TDENGINE_PASSWORD:taosdata}") - .containsEntry( - "lingniu.ingest.tdengine-history.driver-class-name", - "${TDENGINE_DRIVER_CLASS_NAME:com.taosdata.jdbc.rs.RestfulDriver}") - .containsEntry("lingniu.ingest.tdengine-history.maximum-pool-size", "${TDENGINE_MAX_POOL_SIZE:16}") - .containsEntry("lingniu.ingest.tdengine-history.minimum-idle", "${TDENGINE_MIN_IDLE:0}") - .containsEntry("lingniu.ingest.event-history.enabled", true) - .containsEntry("lingniu.ingest.sink.kafka.enabled", true) - .containsEntry("lingniu.ingest.sink.kafka.consumer.enabled", "${KAFKA_CONSUMER_ENABLED:true}") - .containsEntry("lingniu.ingest.sink.kafka.consumer.dlq-topic", - "${KAFKA_TOPIC_HISTORY_DLQ:vehicle.dlq.history.v1}") - .containsEntry("lingniu.ingest.sink.kafka.consumer.max-poll-records", "${KAFKA_CONSUMER_MAX_POLL_RECORDS:2000}") - .containsEntry("lingniu.ingest.sink.kafka.consumer.concurrency", "${KAFKA_CONSUMER_CONCURRENCY:3}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.max-poll-interval-millis", - "${KAFKA_CONSUMER_MAX_POLL_INTERVAL_MILLIS:1800000}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryGb32960EnvelopeConsumerProcessor.enabled", - true) - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryGb32960EnvelopeConsumerProcessor.group-id", - "${KAFKA_GROUP_HISTORY_GB32960_EVENT:${KAFKA_GROUP_HISTORY:vehicle-history}-gb32960-event}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryGb32960EnvelopeConsumerProcessor.topics[0]", - "${KAFKA_TOPIC_GB32960_EVENT:vehicle.event.gb32960.v1}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryJt808EnvelopeConsumerProcessor.enabled", - true) - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryJt808EnvelopeConsumerProcessor.group-id", - "${KAFKA_GROUP_HISTORY_JT808_EVENT:${KAFKA_GROUP_HISTORY:vehicle-history}-jt808-event}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryJt808EnvelopeConsumerProcessor.topics[0]", - "${KAFKA_TOPIC_JT808_EVENT:vehicle.event.jt808.v1}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryYutongMqttEnvelopeConsumerProcessor.enabled", - true) - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryYutongMqttEnvelopeConsumerProcessor.group-id", - "${KAFKA_GROUP_HISTORY_YUTONG_MQTT_EVENT:${KAFKA_GROUP_HISTORY:vehicle-history}-yutong-mqtt-event}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryYutongMqttEnvelopeConsumerProcessor.topics[0]", - "${KAFKA_TOPIC_YUTONG_MQTT_EVENT:vehicle.event.mqtt-yutong.v1}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryGb32960RawEnvelopeConsumerProcessor.enabled", - true) - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryGb32960RawEnvelopeConsumerProcessor.group-id", - "${KAFKA_GROUP_HISTORY_GB32960_RAW:${KAFKA_GROUP_HISTORY:vehicle-history}-gb32960-raw}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryGb32960RawEnvelopeConsumerProcessor.topics[0]", - "${KAFKA_TOPIC_GB32960_RAW:vehicle.raw.gb32960.v1}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryJt808RawEnvelopeConsumerProcessor.enabled", - true) - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryJt808RawEnvelopeConsumerProcessor.group-id", - "${KAFKA_GROUP_HISTORY_JT808_RAW:${KAFKA_GROUP_HISTORY:vehicle-history}-jt808-raw}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryJt808RawEnvelopeConsumerProcessor.topics[0]", - "${KAFKA_TOPIC_JT808_RAW:vehicle.raw.jt808.v1}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryYutongMqttRawEnvelopeConsumerProcessor.enabled", - true) - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryYutongMqttRawEnvelopeConsumerProcessor.group-id", - "${KAFKA_GROUP_HISTORY_YUTONG_MQTT_RAW:${KAFKA_GROUP_HISTORY:vehicle-history}-yutong-mqtt-raw}") - .containsEntry( - "lingniu.ingest.sink.kafka.consumer.bindings.eventHistoryYutongMqttRawEnvelopeConsumerProcessor.topics[0]", - "${KAFKA_TOPIC_YUTONG_MQTT_RAW:vehicle.raw.mqtt-yutong.v1}") - .containsEntry("management.endpoints.web.exposure.include", "health,info,metrics,prometheus"); - - assertThat(properties.stringPropertyNames()) - .noneMatch(name -> name.startsWith("lingniu.ingest.gb32960.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.vehicle-state.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.vehicle-stat.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.sink.archive.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.sink.kafka.topics.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.event-file-store.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.event-history.api.")); - assertThat(properties.stringPropertyNames()) - .noneMatch(name -> name.equals("lingniu.ingest.tdengine-history.telemetry-fields-enabled")); - assertThat(applicationYaml()) - .doesNotContain("\n archive:\n") - .doesNotContain("sink.archive") - .doesNotContain("raw-archive: ${KAFKA_TOPIC_GB32960_RAW") - .doesNotContain("EVENT_HISTORY_SPECIALIZED_API_ENABLED") - .doesNotContain("KAFKA_ENABLED"); - } - - @Test - void runtimeClasspathDoesNotContainLegacyCommonsLoggingJar() throws Exception { - List logFactoryResources = Collections.list(Thread.currentThread() - .getContextClassLoader() - .getResources("org/apache/commons/logging/LogFactory.class")) - .stream() - .map(URL::toExternalForm) - .toList(); - - assertThat(logFactoryResources) - .noneMatch(resource -> resource.contains("/commons-logging-")); - } - - private static Properties applicationProperties() { - YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean(); - factory.setResources(new ClassPathResource("application.yml")); - - Properties properties = factory.getObject(); - - assertThat(properties).isNotNull(); - return properties; - } - - private static String applicationYaml() throws IOException { - return new String(new ClassPathResource("application.yml").getInputStream().readAllBytes(), - StandardCharsets.UTF_8); - } -} diff --git a/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/WoodpeckerPipelineTest.java b/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/WoodpeckerPipelineTest.java deleted file mode 100644 index 7da865b4..00000000 --- a/modules/apps/vehicle-history-app/src/test/java/com/lingniu/ingest/historyapp/WoodpeckerPipelineTest.java +++ /dev/null @@ -1,94 +0,0 @@ -package com.lingniu.ingest.historyapp; - -import org.junit.jupiter.api.Test; - -import java.io.IOException; -import java.nio.file.Files; -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; - -class WoodpeckerPipelineTest { - - @Test - void dockerPublishUsesOneProductionAppList() throws IOException { - String pipeline = Files.readString(repositoryRoot().resolve("woodpecker.yml")); - - assertThat(pipeline) - .contains("ACTIVE_APPS: &active_apps \"gb32960-ingest-app jt808-ingest-app yutong-mqtt-app vehicle-history-app vehicle-analytics-app\"") - .contains("ACTIVE_APPS: *active_apps") - .contains("- name: docker-build-apps") - .doesNotContain("xinda-push-app") - .doesNotContain("KAFKA_TOPIC_XINDA_PUSH"); - - assertThat(countOccurrences(pipeline, "- name: docker-build-")).isEqualTo(1); - } - - @Test - void activeAppListIsDefinedOnceAndReusedAcrossPipelineSteps() throws IOException { - String pipeline = Files.readString(repositoryRoot().resolve("woodpecker.yml")); - String activeApps = "gb32960-ingest-app jt808-ingest-app yutong-mqtt-app vehicle-history-app vehicle-analytics-app"; - - assertThat(countOccurrences(pipeline, activeApps)).isEqualTo(1); - assertThat(pipeline) - .contains("ACTIVE_APPS: &active_apps \"" + activeApps + "\"") - .contains("ACTIVE_APPS: *active_apps") - .doesNotContain("ACTIVE_APPS=\"" + activeApps + "\""); - } - - @Test - void mavenBuildSkipsTestExecutionAndCompilationInCi() throws IOException { - String pipeline = Files.readString(repositoryRoot().resolve("woodpecker.yml")); - - assertThat(pipeline) - .contains("mvn -B -ntp -am -pl \"$ACTIVE_MODULES\" package -Dmaven.test.skip=true") - .doesNotContain("-DskipTests"); - } - - @Test - void dockerPublishUsesOverridableImagePrefix() throws IOException { - String pipeline = Files.readString(repositoryRoot().resolve("woodpecker.yml")); - - assertThat(pipeline) - .contains("REGISTRY=${LINGNIU_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com}") - .contains("NAMESPACE=${LINGNIU_IMAGE_NAMESPACE:-oneos}") - .contains("IMAGE=$REGISTRY/$NAMESPACE/$APP_NAME:$APP_VERSION") - .doesNotContain("\n REGISTRY=crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com") - .doesNotContain("\n NAMESPACE=oneos"); - } - - @Test - void dockerImageExposesOnlyCurrentProductionPorts() throws IOException { - String dockerfile = Files.readString(repositoryRoot().resolve("Dockerfile")); - - assertThat(dockerfile) - .contains("EXPOSE 20100 20200 20310 20400 20500 32960 808") - .doesNotContain("20300") - .doesNotContain("10808") - .doesNotContain("8089"); - } - - private static int countOccurrences(String value, String needle) { - int count = 0; - int from = 0; - while (true) { - int next = value.indexOf(needle, from); - if (next < 0) { - return count; - } - count++; - from = next + needle.length(); - } - } - - private static Path repositoryRoot() { - Path current = Path.of(System.getProperty("user.dir")).toAbsolutePath(); - while (current != null) { - if (Files.exists(current.resolve("woodpecker.yml"))) { - return current; - } - current = current.getParent(); - } - throw new IllegalStateException("repository root not found"); - } -} diff --git a/modules/apps/yutong-mqtt-app/pom.xml b/modules/apps/yutong-mqtt-app/pom.xml deleted file mode 100644 index e2896cc3..00000000 --- a/modules/apps/yutong-mqtt-app/pom.xml +++ /dev/null @@ -1,64 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - - yutong-mqtt-app - yutong-mqtt-app - Yutong MQTT ingress runtime: MQTT subscribe, JSON parse, identity mapping, Kafka production, and raw archive. - - - - com.lingniu.ingest - ingest-core - - - com.lingniu.ingest - observability - - - com.lingniu.ingest - vehicle-identity - - - com.lingniu.ingest - inbound-mqtt - - - com.lingniu.ingest - sink-kafka - - - com.lingniu.ingest - sink-archive - - - org.springframework.boot - spring-boot-starter-web - - - com.alibaba.cloud - spring-cloud-starter-alibaba-nacos-config - - - org.springframework.boot - spring-boot-starter-test - test - - - - - yutong-mqtt-app - - - org.springframework.boot - spring-boot-maven-plugin - - - - diff --git a/modules/apps/yutong-mqtt-app/src/main/java/com/lingniu/ingest/yutongmqttapp/YutongMqttApplication.java b/modules/apps/yutong-mqtt-app/src/main/java/com/lingniu/ingest/yutongmqttapp/YutongMqttApplication.java deleted file mode 100644 index 274fa344..00000000 --- a/modules/apps/yutong-mqtt-app/src/main/java/com/lingniu/ingest/yutongmqttapp/YutongMqttApplication.java +++ /dev/null @@ -1,21 +0,0 @@ -package com.lingniu.ingest.yutongmqttapp; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.boot.SpringApplication; -import org.springframework.boot.autoconfigure.SpringBootApplication; - -@SpringBootApplication -public class YutongMqttApplication { - - private static final Logger log = LoggerFactory.getLogger(YutongMqttApplication.class); - - public static void main(String[] args) { - try { - SpringApplication.run(YutongMqttApplication.class, args); - } catch (Throwable t) { - log.error("Yutong MQTT ingest application failed to start, forcing JVM exit", t); - System.exit(1); - } - } -} diff --git a/modules/apps/yutong-mqtt-app/src/main/resources/application.yml b/modules/apps/yutong-mqtt-app/src/main/resources/application.yml deleted file mode 100644 index 172326e0..00000000 --- a/modules/apps/yutong-mqtt-app/src/main/resources/application.yml +++ /dev/null @@ -1,108 +0,0 @@ -spring: - application: - name: yutong-mqtt-app - config: - import: - - optional:nacos:${spring.application.name}.${NACOS_CONFIG_FILE_EXTENSION:yml}?group=${NACOS_GROUP:DEFAULT_GROUP}&refreshEnabled=${NACOS_REFRESH_ENABLED:true} - cloud: - nacos: - config: - enabled: ${NACOS_CONFIG_ENABLED:true} - server-addr: ${NACOS_SERVER_ADDR:127.0.0.1:8848} - namespace: ${NACOS_NAMESPACE:} - group: ${NACOS_GROUP:DEFAULT_GROUP} - file-extension: ${NACOS_CONFIG_FILE_EXTENSION:yml} - username: ${NACOS_USERNAME:} - password: ${NACOS_PASSWORD:} - threads: - virtual: - enabled: true - -server: - port: ${HTTP_PORT:20500} - -lingniu: - ingest: - mqtt: - enabled: ${YUTONG_MQTT_ENABLED:false} - auto-startup: ${YUTONG_MQTT_AUTO_STARTUP:true} - endpoints: - - name: ${YUTONG_MQTT_ENDPOINT_NAME:yutong} - uri: ${YUTONG_MQTT_URI:} - topic: ${YUTONG_MQTT_TOPIC:#} - qos: ${YUTONG_MQTT_QOS:1} - client-id: ${YUTONG_MQTT_CLIENT_ID:lingniu-yutong-mqtt} - username: ${YUTONG_MQTT_USERNAME:} - password: ${YUTONG_MQTT_PASSWORD:} - clean-session: ${YUTONG_MQTT_CLEAN_SESSION:false} - keep-alive-seconds: ${YUTONG_MQTT_KEEP_ALIVE_SECONDS:20} - connection-timeout-seconds: ${YUTONG_MQTT_CONNECTION_TIMEOUT_SECONDS:10} - profile: yutong - tls: - ca-pem: ${YUTONG_MQTT_TLS_CA_PEM:} - client-pem: ${YUTONG_MQTT_TLS_CLIENT_PEM:} - client-key: ${YUTONG_MQTT_TLS_CLIENT_KEY:} - hostname-verification-enabled: ${YUTONG_MQTT_TLS_HOSTNAME_VERIFICATION_ENABLED:true} - pipeline: - disruptor: - ring-buffer-size: ${PIPELINE_RING_BUFFER_SIZE:131072} - wait-strategy: ${PIPELINE_WAIT_STRATEGY:yielding} - producer-type: multi - dedup: - enabled: true - cache-size: 200000 - ttl-seconds: 600 - rate-limit: - per-vin-qps: 50 - identity: - mysql: - jdbc-url: ${VEHICLE_IDENTITY_MYSQL_JDBC_URL:jdbc:mysql://127.0.0.1:3306/lingniu_vehicle?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai} - username: ${VEHICLE_IDENTITY_MYSQL_USERNAME:root} - password: ${VEHICLE_IDENTITY_MYSQL_PASSWORD:} - table-name: ${VEHICLE_IDENTITY_MYSQL_TABLE:vehicle_identity_bindings} - initialize-schema: ${VEHICLE_IDENTITY_MYSQL_INITIALIZE_SCHEMA:true} - sink: - kafka: - enabled: true - bootstrap-servers: ${KAFKA_BROKERS:114.55.58.251:9092} - compression-type: zstd - linger-ms: 20 - batch-size: 65536 - acks: all - enable-idempotence: true - node-id: ${KAFKA_NODE_ID:yutong-mqtt-local} - topics: - realtime: ${KAFKA_TOPIC_YUTONG_MQTT_EVENT:vehicle.event.mqtt-yutong.v1} - location: ${KAFKA_TOPIC_YUTONG_MQTT_EVENT:vehicle.event.mqtt-yutong.v1} - alarm: ${KAFKA_TOPIC_YUTONG_MQTT_EVENT:vehicle.event.mqtt-yutong.v1} - session: ${KAFKA_TOPIC_YUTONG_MQTT_EVENT:vehicle.event.mqtt-yutong.v1} - media-meta: ${KAFKA_TOPIC_MEDIA_META:vehicle.media.meta.v1} - raw-archive: ${KAFKA_TOPIC_YUTONG_MQTT_RAW:vehicle.raw.mqtt-yutong.v1} - dlq: ${KAFKA_TOPIC_YUTONG_MQTT_DLQ:vehicle.dlq.mqtt-yutong.v1} - consumer: - enabled: false - archive: - enabled: ${SINK_ARCHIVE_ENABLED:true} - type: local - path: ${SINK_ARCHIVE_PATH:./archive/} - -management: - endpoints: - web: - exposure: - include: health,info,metrics,prometheus - endpoint: - health: - probes: - enabled: ${MANAGEMENT_HEALTH_PROBES_ENABLED:true} - health: - redis: - enabled: ${MANAGEMENT_HEALTH_REDIS_ENABLED:false} - metrics: - tags: - application: yutong-mqtt-app - -logging: - level: - root: INFO - com.lingniu.ingest: INFO diff --git a/modules/apps/yutong-mqtt-app/src/test/java/com/lingniu/ingest/yutongmqttapp/YutongMqttAppCompositionTest.java b/modules/apps/yutong-mqtt-app/src/test/java/com/lingniu/ingest/yutongmqttapp/YutongMqttAppCompositionTest.java deleted file mode 100644 index c6da8a66..00000000 --- a/modules/apps/yutong-mqtt-app/src/test/java/com/lingniu/ingest/yutongmqttapp/YutongMqttAppCompositionTest.java +++ /dev/null @@ -1,83 +0,0 @@ -package com.lingniu.ingest.yutongmqttapp; - -import com.lingniu.ingest.core.config.IngestCoreAutoConfiguration; -import com.lingniu.ingest.identity.MySqlVehicleIdentityService; -import com.lingniu.ingest.identity.config.VehicleIdentityAutoConfiguration; -import com.lingniu.ingest.inbound.mqtt.client.MqttEndpointManager; -import com.lingniu.ingest.inbound.mqtt.config.MqttInboundAutoConfiguration; -import com.lingniu.ingest.inbound.mqtt.handler.MqttRealtimeHandler; -import com.lingniu.ingest.inbound.mqtt.profile.MqttProfileRegistry; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import com.lingniu.ingest.sink.archive.RawArchiveEventSink; -import com.lingniu.ingest.sink.archive.config.SinkArchiveAutoConfiguration; -import com.lingniu.ingest.sink.kafka.KafkaEnvelopeDeadLetterSink; -import com.lingniu.ingest.sink.kafka.KafkaEventSink; -import com.lingniu.ingest.sink.kafka.KafkaSinkAutoConfiguration; -import org.apache.kafka.clients.producer.KafkaProducer; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.extension.ExtendWith; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.boot.test.system.CapturedOutput; -import org.springframework.boot.test.system.OutputCaptureExtension; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.mock; - -@ExtendWith(OutputCaptureExtension.class) -class YutongMqttAppCompositionTest { - - private final ApplicationContextRunner contextRunner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of( - IngestCoreAutoConfiguration.class, - VehicleIdentityAutoConfiguration.class, - KafkaSinkAutoConfiguration.class, - SinkArchiveAutoConfiguration.class, - MqttInboundAutoConfiguration.class)) - .withAllowBeanDefinitionOverriding(true) - .withBean("kafkaProducer", KafkaProducer.class, YutongMqttAppCompositionTest::kafkaProducer) - .withPropertyValues( - "lingniu.ingest.mqtt.enabled=true", - "lingniu.ingest.mqtt.auto-startup=false", - "lingniu.ingest.mqtt.endpoints[0].name=yutong-test", - "lingniu.ingest.mqtt.endpoints[0].uri=tcp://127.0.0.1:1883", - "lingniu.ingest.mqtt.endpoints[0].topic=/yutong/#", - "lingniu.ingest.mqtt.endpoints[0].profile=yutong", - "lingniu.ingest.identity.mysql.initialize-schema=false", - "lingniu.ingest.sink.kafka.enabled=true", - "lingniu.ingest.sink.kafka.bootstrap-servers=localhost:9092", - "lingniu.ingest.sink.kafka.consumer.enabled=false", - "lingniu.ingest.sink.archive.enabled=true", - "lingniu.ingest.sink.archive.path=target/test-archive-yutong", - "lingniu.ingest.event-history.enabled=false"); - - @Test - void createsMqttIngressAndKafkaSinksWithoutHistoryStorage(CapturedOutput output) { - contextRunner.run(context -> { - assertThat(context).hasSingleBean(MqttEndpointManager.class); - assertThat(context).hasSingleBean(MqttProfileRegistry.class); - assertThat(context).hasSingleBean(MqttRealtimeHandler.class); - assertThat(context).hasSingleBean(MySqlVehicleIdentityService.class); - assertThat(context).hasSingleBean(KafkaEventSink.class); - assertThat(context).hasSingleBean(KafkaEnvelopeDeadLetterSink.class); - assertThat(context).hasSingleBean(ArchiveStore.class); - assertThat(context).hasSingleBean(RawArchiveEventSink.class); - assertThat(context).doesNotHaveBean("mqttEndpointStartupRunner"); - }); - assertThat(output).doesNotContain("mqtt endpoint [yutong-test] initializing"); - assertThat(output).doesNotContain("mqtt endpoint [yutong-test] init failed"); - } - - @Test - void createsStartupRunnerWhenAutoStartupIsEnabled() { - contextRunner.withPropertyValues("lingniu.ingest.mqtt.auto-startup=true").run(context -> { - assertThat(context).hasSingleBean(MqttEndpointManager.class); - assertThat(context).hasBean("mqttEndpointStartupRunner"); - }); - } - - @SuppressWarnings("unchecked") - private static KafkaProducer kafkaProducer() { - return mock(KafkaProducer.class); - } -} diff --git a/modules/apps/yutong-mqtt-app/src/test/java/com/lingniu/ingest/yutongmqttapp/YutongMqttAppDefaultsTest.java b/modules/apps/yutong-mqtt-app/src/test/java/com/lingniu/ingest/yutongmqttapp/YutongMqttAppDefaultsTest.java deleted file mode 100644 index 08d02d65..00000000 --- a/modules/apps/yutong-mqtt-app/src/test/java/com/lingniu/ingest/yutongmqttapp/YutongMqttAppDefaultsTest.java +++ /dev/null @@ -1,71 +0,0 @@ -package com.lingniu.ingest.yutongmqttapp; - -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.config.YamlPropertiesFactoryBean; -import org.springframework.core.io.ClassPathResource; - -import java.io.IOException; -import java.nio.charset.StandardCharsets; -import java.util.Properties; - -import static org.assertj.core.api.Assertions.assertThat; - -class YutongMqttAppDefaultsTest { - - @Test - void applicationDefaultsKeepYutongMqttAsKafkaProducerOnly() throws IOException { - Properties properties = applicationProperties(); - - assertThat(properties) - .containsEntry("spring.application.name", "yutong-mqtt-app") - .containsEntry( - "spring.config.import[0]", - "optional:nacos:${spring.application.name}.${NACOS_CONFIG_FILE_EXTENSION:yml}?group=${NACOS_GROUP:DEFAULT_GROUP}&refreshEnabled=${NACOS_REFRESH_ENABLED:true}") - .containsEntry("spring.cloud.nacos.config.server-addr", "${NACOS_SERVER_ADDR:127.0.0.1:8848}") - .containsEntry("server.port", "${HTTP_PORT:20500}") - .containsEntry("lingniu.ingest.mqtt.enabled", "${YUTONG_MQTT_ENABLED:false}") - .containsEntry("lingniu.ingest.mqtt.auto-startup", "${YUTONG_MQTT_AUTO_STARTUP:true}") - .containsEntry("lingniu.ingest.mqtt.endpoints[0].uri", "${YUTONG_MQTT_URI:}") - .containsEntry("lingniu.ingest.mqtt.endpoints[0].topic", "${YUTONG_MQTT_TOPIC:#}") - .containsEntry("lingniu.ingest.mqtt.endpoints[0].profile", "yutong") - .containsEntry("lingniu.ingest.sink.kafka.enabled", true) - .containsEntry("lingniu.ingest.sink.kafka.consumer.enabled", false) - .containsEntry("lingniu.ingest.sink.kafka.topics.realtime", "${KAFKA_TOPIC_YUTONG_MQTT_EVENT:vehicle.event.mqtt-yutong.v1}") - .containsEntry("lingniu.ingest.sink.kafka.topics.raw-archive", "${KAFKA_TOPIC_YUTONG_MQTT_RAW:vehicle.raw.mqtt-yutong.v1}") - .containsEntry("lingniu.ingest.sink.kafka.topics.dlq", "${KAFKA_TOPIC_YUTONG_MQTT_DLQ:vehicle.dlq.mqtt-yutong.v1}") - .containsEntry("lingniu.ingest.identity.mysql.table-name", - "${VEHICLE_IDENTITY_MYSQL_TABLE:vehicle_identity_bindings}") - .containsEntry("lingniu.ingest.sink.archive.enabled", "${SINK_ARCHIVE_ENABLED:true}") - .containsEntry("management.endpoints.web.exposure.include", "health,info,metrics,prometheus"); - - assertThat(properties.stringPropertyNames()) - .noneMatch(name -> name.startsWith("lingniu.ingest.event-history.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.event-file-store.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.vehicle-state.")) - .noneMatch(name -> name.startsWith("lingniu.ingest.vehicle-stat.")) - .noneMatch(name -> name.equals("lingniu.ingest.identity.store")); - assertThat(applicationYaml()) - .doesNotContain("event-history:") - .doesNotContain("event-file-store:") - .doesNotContain("vehicle-state:") - .doesNotContain("vehicle-stat:") - .doesNotContain("springdoc:") - .doesNotContain("VEHICLE_IDENTITY_STORE") - .doesNotContain("KAFKA_ENABLED"); - } - - private static Properties applicationProperties() { - YamlPropertiesFactoryBean factory = new YamlPropertiesFactoryBean(); - factory.setResources(new ClassPathResource("application.yml")); - - Properties properties = factory.getObject(); - - assertThat(properties).isNotNull(); - return properties; - } - - private static String applicationYaml() throws IOException { - return new String(new ClassPathResource("application.yml").getInputStream().readAllBytes(), - StandardCharsets.UTF_8); - } -} diff --git a/modules/apps/yutong-mqtt-app/src/test/java/com/lingniu/ingest/yutongmqttapp/YutongMqttPortainerComposeTest.java b/modules/apps/yutong-mqtt-app/src/test/java/com/lingniu/ingest/yutongmqttapp/YutongMqttPortainerComposeTest.java deleted file mode 100644 index a67c2d66..00000000 --- a/modules/apps/yutong-mqtt-app/src/test/java/com/lingniu/ingest/yutongmqttapp/YutongMqttPortainerComposeTest.java +++ /dev/null @@ -1,50 +0,0 @@ -package com.lingniu.ingest.yutongmqttapp; - -import org.junit.jupiter.api.Test; - -import java.io.IOException; -import java.nio.file.Files; -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; - -class YutongMqttPortainerComposeTest { - - @Test - void portainerComposeUsesYutongMqttApplicationEnvironmentNames() throws IOException { - String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml")); - - assertThat(compose) - .contains("KAFKA_TOPIC_YUTONG_MQTT_EVENT:") - .contains("KAFKA_TOPIC_YUTONG_MQTT_RAW:") - .contains("KAFKA_TOPIC_YUTONG_MQTT_DLQ:") - .contains("YUTONG_MQTT_ENABLED: ${YUTONG_MQTT_ENABLED:-false}") - .contains("YUTONG_MQTT_AUTO_STARTUP: ${YUTONG_MQTT_AUTO_STARTUP:-true}") - .contains("YUTONG_MQTT_URI: ${YUTONG_MQTT_URI:-}") - .contains("YUTONG_MQTT_TOPIC: \"${YUTONG_MQTT_TOPIC:-#}\"") - .contains("YUTONG_MQTT_KEEP_ALIVE_SECONDS: ${YUTONG_MQTT_KEEP_ALIVE_SECONDS:-20}") - .contains("YUTONG_MQTT_CONNECTION_TIMEOUT_SECONDS: ${YUTONG_MQTT_CONNECTION_TIMEOUT_SECONDS:-10}") - .contains("YUTONG_MQTT_TLS_CA_PEM: ${YUTONG_MQTT_TLS_CA_PEM:-}") - .contains("YUTONG_MQTT_TLS_CLIENT_PEM: ${YUTONG_MQTT_TLS_CLIENT_PEM:-}") - .contains("YUTONG_MQTT_TLS_CLIENT_KEY: ${YUTONG_MQTT_TLS_CLIENT_KEY:-}") - .contains("YUTONG_MQTT_TLS_HOSTNAME_VERIFICATION_ENABLED: ${YUTONG_MQTT_TLS_HOSTNAME_VERIFICATION_ENABLED:-true}") - .doesNotContain("YUTONG_MQTT_URI: ${YUTONG_MQTT_URI:-tcp://127.0.0.1:1883}") - .doesNotContain("KAFKA_TOPIC_PARTNER_EVENT") - .doesNotContain("KAFKA_TOPIC_PARTNER_RAW") - .doesNotContain("KAFKA_TOPIC_PARTNER_DLQ") - .doesNotContain("\n MQTT_ENABLED:") - .doesNotContain("\n MQTT_URI:") - .doesNotContain("\n MQTT_TOPIC:"); - } - - private static Path repositoryRoot() { - Path current = Path.of(System.getProperty("user.dir")).toAbsolutePath(); - while (current != null) { - if (Files.exists(current.resolve("deploy/portainer/docker-compose.yml"))) { - return current; - } - current = current.getParent(); - } - throw new IllegalStateException("repository root not found"); - } -} diff --git a/modules/core/ingest-api/pom.xml b/modules/core/ingest-api/pom.xml deleted file mode 100644 index 88cf89fb..00000000 --- a/modules/core/ingest-api/pom.xml +++ /dev/null @@ -1,30 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - ingest-api - ingest-api - SPI、sealed 领域事件、注解定义。零 Spring 依赖。 - - - - org.slf4j - slf4j-api - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/ProtocolId.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/ProtocolId.java deleted file mode 100644 index 8814ac33..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/ProtocolId.java +++ /dev/null @@ -1,14 +0,0 @@ -package com.lingniu.ingest.api; - -/** - * 协议标识，作为所有 SPI / 注解 / 路由 key 的统一枚举。 - * 新增协议需要在此处登记，避免散落的字符串常量。 - */ -public enum ProtocolId { - UNKNOWN, - GB32960, - JT808, - JT1078, - JSATL12, - MQTT_YUTONG -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/AsyncBatch.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/AsyncBatch.java deleted file mode 100644 index 6cea65df..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/AsyncBatch.java +++ /dev/null @@ -1,30 +0,0 @@ -package com.lingniu.ingest.api.annotation; - -import java.lang.annotation.ElementType; -import java.lang.annotation.Retention; -import java.lang.annotation.RetentionPolicy; -import java.lang.annotation.Target; - -/** - * 批量异步处理：框架把多次同类调用聚合成 {@code List} 交给方法。 - * - *

处理方法签名需接收 {@code List}，返回 {@code List} 或单个 - * {@code VehicleEvent}。Dispatcher 不直接调用目标方法，而是交给 - * {@code AsyncBatchExecutor} 按 size/waitMs 聚合后调用。 - * - *

注意：当调用方需要给每条事件补不同的 rawArchiveUri 等逐条元数据时，执行器会按单条 flush， - * 以保证事件与原始帧一一对应。 - */ -@Retention(RetentionPolicy.RUNTIME) -@Target(ElementType.METHOD) -public @interface AsyncBatch { - - /** 达到该批量大小后立即 flush。 */ - int size() default 1000; - - /** 第一条消息入队后最多等待毫秒数，避免低流量车辆长期不出批。 */ - long waitMs() default 500; - - /** 每个 Handler 方法对应的批处理 worker 数。 */ - int poolSize() default 2; -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/EventEmit.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/EventEmit.java deleted file mode 100644 index 48c81478..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/EventEmit.java +++ /dev/null @@ -1,17 +0,0 @@ -package com.lingniu.ingest.api.annotation; - -import com.lingniu.ingest.api.event.VehicleEvent; - -import java.lang.annotation.ElementType; -import java.lang.annotation.Retention; -import java.lang.annotation.RetentionPolicy; -import java.lang.annotation.Target; - -/** - * 声明当前 Handler 方法产出的事件类型。用于文档化和静态校验。 - */ -@Retention(RetentionPolicy.RUNTIME) -@Target(ElementType.METHOD) -public @interface EventEmit { - Class[] value(); -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/IdempotentKey.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/IdempotentKey.java deleted file mode 100644 index 24badc4d..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/IdempotentKey.java +++ /dev/null @@ -1,19 +0,0 @@ -package com.lingniu.ingest.api.annotation; - -import java.lang.annotation.ElementType; -import java.lang.annotation.Retention; -import java.lang.annotation.RetentionPolicy; -import java.lang.annotation.Target; - -/** - * 声明幂等键，由 Dedup 拦截器使用。支持 SpEL 表达式，上下文根对象为 Handler 参数。 - * - *

示例：{@code @IdempotentKey("#msg.vin + ':' + #msg.seq + ':' + #msg.eventTime")} - * - *

当前内置去重主要基于 RawFrame sourceMeta/raw bytes；该注解保留给更细粒度 Handler 级去重。 - */ -@Retention(RetentionPolicy.RUNTIME) -@Target(ElementType.METHOD) -public @interface IdempotentKey { - String value(); -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/MessageMapping.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/MessageMapping.java deleted file mode 100644 index aa53a873..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/MessageMapping.java +++ /dev/null @@ -1,33 +0,0 @@ -package com.lingniu.ingest.api.annotation; - -import java.lang.annotation.ElementType; -import java.lang.annotation.Retention; -import java.lang.annotation.RetentionPolicy; -import java.lang.annotation.Target; - -/** - * 方法级路由注解：声明此方法处理哪些协议消息。 - * - *

{@link #command} 和 {@link #infoType} 的含义由各协议自行解释，Dispatcher 只负责匹配： - *

    - *
  • GB/T 32960：{@code command} = 0x01 车辆登入 / 0x02 实时上报 / ...；{@code infoType} = 信息体 ID - *
  • JT/T 808：{@code command} = 消息 ID（0x0100 / 0x0200 / ...）；{@code infoType} 留空 - *
  • MQTT：{@code command} 可为 topic hash 或忽略 - *
- * - *

同一个方法可以声明多个 command/infoType。Dispatcher 会按协议、command、infoType 精确路由； - * 32960 实时上报解析后通常由 0x02/0x03 主命令加信息体 ID 决定落到哪个 Handler。 - */ -@Retention(RetentionPolicy.RUNTIME) -@Target(ElementType.METHOD) -public @interface MessageMapping { - - /** 协议主命令或消息 ID，空数组表示该维度不限制。 */ - int[] command() default {}; - - /** 协议子类型，GB32960 中对应信息体 ID；空数组表示该维度不限制。 */ - int[] infoType() default {}; - - /** 仅用于日志/Swagger/排障展示，不参与路由。 */ - String desc() default ""; -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/ProtocolHandler.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/ProtocolHandler.java deleted file mode 100644 index 6e308444..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/ProtocolHandler.java +++ /dev/null @@ -1,30 +0,0 @@ -package com.lingniu.ingest.api.annotation; - -import com.lingniu.ingest.api.ProtocolId; - -import java.lang.annotation.ElementType; -import java.lang.annotation.Retention; -import java.lang.annotation.RetentionPolicy; -import java.lang.annotation.Target; - -/** - * 标记一个类为协议 Handler Bean，由 {@code AnnotationHandlerProcessor} 自动扫描注册。 - * - *

典型用法： - * 

{@code
- * @ProtocolHandler(protocol = ProtocolId.GB32960)
- * public class Gb32960RealtimeHandler {
- *     @MessageMapping(command = 0x02)
- *     public VehicleEvent.Realtime handle(Gb32960Message msg) { ... }
- * }
- * }
- */ -@Retention(RetentionPolicy.RUNTIME) -@Target(ElementType.TYPE) -public @interface ProtocolHandler { - - ProtocolId protocol(); - - /** 可选：子协议版本（如 32960 的 2011 / 2017）。 */ - String version() default ""; -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/RateLimited.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/RateLimited.java deleted file mode 100644 index 77855f91..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/RateLimited.java +++ /dev/null @@ -1,23 +0,0 @@ -package com.lingniu.ingest.api.annotation; - -import java.lang.annotation.ElementType; -import java.lang.annotation.Retention; -import java.lang.annotation.RetentionPolicy; -import java.lang.annotation.Target; - -/** - * 单 VIN 速率限制声明。 - * - *

当前内置限流器在 Pipeline before 阶段按 VIN 中止处理，超限帧不会进入 Handler 和下游存储。 - * 如果需要把超限帧写入 DLQ，应在拦截器或入口层显式增加对应 sink。 - */ -@Retention(RetentionPolicy.RUNTIME) -@Target(ElementType.METHOD) -public @interface RateLimited { - - /** 单 VIN 每秒最大消息数。 */ - int perVin() default 50; - - /** 全局 QPS 上限；<=0 表示不设限。 */ - int global() default -1; -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/TraceSpan.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/TraceSpan.java deleted file mode 100644 index 1dbbb521..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/annotation/TraceSpan.java +++ /dev/null @@ -1,15 +0,0 @@ -package com.lingniu.ingest.api.annotation; - -import java.lang.annotation.ElementType; -import java.lang.annotation.Retention; -import java.lang.annotation.RetentionPolicy; -import java.lang.annotation.Target; - -/** - * 为 Handler 方法标记轻量 traceId 诊断名称。 - */ -@Retention(RetentionPolicy.RUNTIME) -@Target(ElementType.METHOD) -public @interface TraceSpan { - String value(); -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeBatchIngestor.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeBatchIngestor.java deleted file mode 100644 index 03aaafe7..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeBatchIngestor.java +++ /dev/null @@ -1,12 +0,0 @@ -package com.lingniu.ingest.api.consumer; - -import java.util.List; - -/** - * Optional extension for consumers that can persist a Kafka poll batch in one - * storage operation. - */ -public interface EnvelopeBatchIngestor extends EnvelopeIngestor { - - List tryIngestAll(List kafkaValues); -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeConsumerProcessor.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeConsumerProcessor.java deleted file mode 100644 index f5b51abb..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeConsumerProcessor.java +++ /dev/null @@ -1,113 +0,0 @@ -package com.lingniu.ingest.api.consumer; - -import java.time.Instant; -import java.util.ArrayList; -import java.util.EnumSet; -import java.util.List; -import java.util.Set; - -public final class EnvelopeConsumerProcessor { - - private static final Set DEAD_LETTER_STATUSES = EnumSet.of( - EnvelopeIngestResult.Status.SKIPPED, - EnvelopeIngestResult.Status.INVALID_ENVELOPE, - EnvelopeIngestResult.Status.FAILED - ); - - private final String service; - private final EnvelopeIngestor ingestor; - private final EnvelopeDeadLetterSink deadLetterSink; - - public EnvelopeConsumerProcessor(String service, EnvelopeIngestor ingestor, EnvelopeDeadLetterSink deadLetterSink) { - if (ingestor == null) { - throw new IllegalArgumentException("ingestor must not be null"); - } - if (deadLetterSink == null) { - throw new IllegalArgumentException("deadLetterSink must not be null"); - } - this.service = service == null || service.isBlank() ? "unknown" : service; - this.ingestor = ingestor; - this.deadLetterSink = deadLetterSink; - } - - public EnvelopeIngestResult process(EnvelopeConsumerRecord record) { - if (record == null) { - throw new IllegalArgumentException("record must not be null"); - } - EnvelopeIngestResult result = processAll(List.of(record)).getFirst(); - return result; - } - - public List processAll(List records) { - if (records == null) { - throw new IllegalArgumentException("records must not be null"); - } - if (records.isEmpty()) { - return List.of(); - } - List payloads = new ArrayList<>(records.size()); - for (EnvelopeConsumerRecord record : records) { - if (record == null) { - throw new IllegalArgumentException("record must not be null"); - } - payloads.add(record.payload()); - } - List results = ingestor.tryIngestAll(payloads); - if (results.size() != records.size()) { - throw new IllegalStateException("ingestor result count does not match record count"); - } - for (int i = 0; i < records.size(); i++) { - publishDeadLetterIfNeeded(records.get(i), results.get(i)); - } - return results; - } - - private void publishDeadLetterIfNeeded(EnvelopeConsumerRecord record, EnvelopeIngestResult result) { - // 派生消费者不在这里抛出业务异常给 Kafka worker；坏消息统一进入 DLQ， - // worker 看到 process 正常返回后才能提交 offset，避免同一坏消息无限阻塞消费组。 - if (DEAD_LETTER_STATUSES.contains(result.status())) { - deadLetterSink.publish(toDeadLetter(record, result)); - } - } - - public List processBatch(List records) { - if (records == null || records.isEmpty()) { - return List.of(); - } - if (!(ingestor instanceof EnvelopeBatchIngestor batchIngestor)) { - return records.stream().map(this::process).toList(); - } - List payloads = records.stream() - .map(EnvelopeConsumerRecord::payload) - .toList(); - List results = batchIngestor.tryIngestAll(payloads); - if (results.size() != records.size()) { - throw new IllegalStateException("batch ingestor returned " + results.size() - + " results for " + records.size() + " records"); - } - List copy = new ArrayList<>(results.size()); - for (int i = 0; i < records.size(); i++) { - EnvelopeIngestResult result = results.get(i); - if (DEAD_LETTER_STATUSES.contains(result.status())) { - deadLetterSink.publish(toDeadLetter(records.get(i), result)); - } - copy.add(result); - } - return List.copyOf(copy); - } - - private EnvelopeDeadLetterRecord toDeadLetter(EnvelopeConsumerRecord record, EnvelopeIngestResult result) { - return new EnvelopeDeadLetterRecord( - service, - record.topic(), - record.partition(), - record.offset(), - record.key(), - result.status(), - result.eventId(), - result.vin(), - result.message(), - record.payload(), - Instant.now()); - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeConsumerRecord.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeConsumerRecord.java deleted file mode 100644 index 40bb948e..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeConsumerRecord.java +++ /dev/null @@ -1,20 +0,0 @@ -package com.lingniu.ingest.api.consumer; - -public record EnvelopeConsumerRecord( - String topic, - int partition, - long offset, - String key, - byte[] payload -) { - public EnvelopeConsumerRecord { - topic = topic == null ? "" : topic; - key = key == null ? "" : key; - payload = payload == null ? new byte[0] : payload.clone(); - } - - @Override - public byte[] payload() { - return payload.clone(); - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeDeadLetterRecord.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeDeadLetterRecord.java deleted file mode 100644 index 45cb7901..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeDeadLetterRecord.java +++ /dev/null @@ -1,36 +0,0 @@ -package com.lingniu.ingest.api.consumer; - -import java.time.Instant; - -public record EnvelopeDeadLetterRecord( - String service, - String topic, - int partition, - long offset, - String key, - EnvelopeIngestResult.Status status, - String eventId, - String vin, - String message, - byte[] payload, - Instant createdAt -) { - public EnvelopeDeadLetterRecord { - service = service == null ? "" : service; - topic = topic == null ? "" : topic; - key = key == null ? "" : key; - if (status == null) { - throw new IllegalArgumentException("status must not be null"); - } - eventId = eventId == null ? "" : eventId; - vin = vin == null ? "" : vin; - message = message == null ? "" : message; - payload = payload == null ? new byte[0] : payload.clone(); - createdAt = createdAt == null ? Instant.now() : createdAt; - } - - @Override - public byte[] payload() { - return payload.clone(); - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeDeadLetterSink.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeDeadLetterSink.java deleted file mode 100644 index 5de699af..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeDeadLetterSink.java +++ /dev/null @@ -1,6 +0,0 @@ -package com.lingniu.ingest.api.consumer; - -@FunctionalInterface -public interface EnvelopeDeadLetterSink { - void publish(EnvelopeDeadLetterRecord record); -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeIngestResult.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeIngestResult.java deleted file mode 100644 index c1a7f43b..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeIngestResult.java +++ /dev/null @@ -1,45 +0,0 @@ -package com.lingniu.ingest.api.consumer; - -/** - * Non-throwing result for Kafka envelope consumers. It lets production - * listeners isolate bad records without blocking the partition. - */ -public record EnvelopeIngestResult(Status status, String eventId, String vin, String message) { - - public enum Status { - STORED, - PROCESSED, - SKIPPED, - INVALID_ENVELOPE, - FAILED - } - - public EnvelopeIngestResult { - if (status == null) { - throw new IllegalArgumentException("status must not be null"); - } - eventId = eventId == null ? "" : eventId; - vin = vin == null ? "" : vin; - message = message == null ? "" : message; - } - - public static EnvelopeIngestResult stored(String eventId, String vin) { - return new EnvelopeIngestResult(Status.STORED, eventId, vin, ""); - } - - public static EnvelopeIngestResult processed(String eventId, String vin) { - return new EnvelopeIngestResult(Status.PROCESSED, eventId, vin, ""); - } - - public static EnvelopeIngestResult skipped(String eventId, String vin, String message) { - return new EnvelopeIngestResult(Status.SKIPPED, eventId, vin, message); - } - - public static EnvelopeIngestResult invalid(String message) { - return new EnvelopeIngestResult(Status.INVALID_ENVELOPE, "", "", message); - } - - public static EnvelopeIngestResult failed(String eventId, String vin, String message) { - return new EnvelopeIngestResult(Status.FAILED, eventId, vin, message); - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeIngestor.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeIngestor.java deleted file mode 100644 index 2d200465..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/consumer/EnvelopeIngestor.java +++ /dev/null @@ -1,20 +0,0 @@ -package com.lingniu.ingest.api.consumer; - -import java.util.ArrayList; -import java.util.List; - -@FunctionalInterface -public interface EnvelopeIngestor { - EnvelopeIngestResult tryIngest(byte[] kafkaValue); - - default List tryIngestAll(List kafkaValues) { - if (kafkaValues == null || kafkaValues.isEmpty()) { - return List.of(); - } - List results = new ArrayList<>(kafkaValues.size()); - for (byte[] kafkaValue : kafkaValues) { - results.add(tryIngest(kafkaValue)); - } - return results; - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/AlarmPayload.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/AlarmPayload.java deleted file mode 100644 index 48c9e6bf..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/AlarmPayload.java +++ /dev/null @@ -1,65 +0,0 @@ -package com.lingniu.ingest.api.event; - -import java.util.List; -import java.util.Set; - -/** - * 报警事件载荷，跨协议归一。 - * - * @param level 报警最高等级 - * @param alarmTypeCode 协议原始 alarm type 编码（按协议自定义），可用于联查协议规范 - * @param alarmTypeName 报警类型名称，如 {@code GB32960_ALARM} - * @param faultCodes 故障代码列表（协议私有编码字符串化，便于下游去重/聚合） - * @param activeBits 结构化的通用报警位集合（2016 版：0~15；2025 版：0~27），对照 GB/T 32960.3 表 24 - * @param longitude 经度（可选，若事件伴随位置） - * @param latitude 纬度（可选） - * @param safetyCategory 内部安全分类。氢气泄露独立成类，不与普通故障混用 - * @param hydrogenLeakDetected 是否检测到氢气泄露 - * @param hydrogenLeakLevel 氢气泄露等级 - * @param hydrogenLeakActionRequired 是否需要立即处置 - */ -public record AlarmPayload( - AlarmLevel level, - int alarmTypeCode, - String alarmTypeName, - List faultCodes, - Set activeBits, - Double longitude, - Double latitude, - SafetyCategory safetyCategory, - boolean hydrogenLeakDetected, - HydrogenLeakLevel hydrogenLeakLevel, - boolean hydrogenLeakActionRequired -) { - /** - * 报警等级（归一化的跨协议分类）。 - * - *

    - *
  • {@link #INFO}：提示级（对应 GB/T 32960.3 0/无故障） - *
  • {@link #MINOR}：1 级 —— 不影响车辆正常行驶 - *
  • {@link #MAJOR}：2 级 —— 影响车辆性能，需驾驶员限制行驶 - *
  • {@link #CRITICAL}：3 级（驾驶员应立即停车）或 4 级（热事件最高级） - *
- */ - public enum AlarmLevel { INFO, MINOR, MAJOR, CRITICAL } - - /** - * 面向运营统计的内部安全分类。 - */ - public enum SafetyCategory { - GENERAL, - TANK_PRESSURE, - TANK_TEMPERATURE, - HYDROGEN_LEAK - } - - /** - * 氢气泄露内部等级。明确泄露信号一律视为 CRITICAL。 - */ - public enum HydrogenLeakLevel { - NONE, - WARNING, - CRITICAL, - UNKNOWN - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/LocationPayload.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/LocationPayload.java deleted file mode 100644 index f786fde9..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/LocationPayload.java +++ /dev/null @@ -1,23 +0,0 @@ -package com.lingniu.ingest.api.event; - -/** 位置事件载荷。坐标已统一转换为 WGS84 十进制度。 */ -public record LocationPayload( - double longitude, - double latitude, - double altitudeM, - double speedKmh, - double directionDeg, - long alarmFlag, - long statusFlag, - Double totalMileageKm -) { - public LocationPayload(double longitude, - double latitude, - double altitudeM, - double speedKmh, - double directionDeg, - long alarmFlag, - long statusFlag) { - this(longitude, latitude, altitudeM, speedKmh, directionDeg, alarmFlag, statusFlag, null); - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/RawArchiveKeys.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/RawArchiveKeys.java deleted file mode 100644 index 014098d9..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/RawArchiveKeys.java +++ /dev/null @@ -1,59 +0,0 @@ -package com.lingniu.ingest.api.event; - -import com.lingniu.ingest.api.ProtocolId; - -import java.time.Instant; -import java.time.ZoneId; -import java.time.format.DateTimeFormatter; -import java.util.Map; - -/** - * Shared raw-archive key conventions used by Dispatcher, archive sink, and query services. - * - *

32960 生产链路里，RAW .bin 文件本体按 {@code 日期/协议/VIN/eventId.bin} 落在 archive 根目录， - * TDengine raw_frames 或兼容索引只保存 {@code archive://...} 引用。这个类集中维护 key 规则， - * 避免接收端、落盘端、snapshot 查询端对目录分区的理解不一致。

- */ -public final class RawArchiveKeys { - - public static final String META_EVENT_ID = "rawArchiveEventId"; - public static final String META_KEY = "rawArchiveKey"; - public static final String META_URI = "rawArchiveUri"; - public static final String META_PARSED_JSON = "rawArchiveParsedJson"; - - private static final DateTimeFormatter DATE_KEY = - DateTimeFormatter.ofPattern("yyyy/MM/dd").withZone(ZoneId.of("Asia/Shanghai")); - - private RawArchiveKeys() { - } - - public static String key(Instant ingestTime, ProtocolId source, String vin, String eventId) { - // 分区日期使用 ingestTime 的东八区自然日，和磁盘目录保持一致；eventTime 只代表车端上报时间。 - String dateKey = DATE_KEY.format(ingestTime == null ? Instant.EPOCH : ingestTime); - String safeVin = vin == null || vin.isBlank() ? "unknown-vin" : vin; - String safeEventId = eventId == null || eventId.isBlank() - ? Long.toHexString(System.nanoTime()) : eventId; - String safeSource = source == null ? "UNKNOWN" : source.name(); - return dateKey + "/" + safeSource + "/" + safeVin + "/" + safeEventId + ".bin"; - } - - public static String key(VehicleEvent.RawArchive raw) { - String key = metadataValue(raw.metadata(), META_KEY); - if (!key.isBlank()) { - return key; - } - return key(raw.ingestTime(), raw.source(), raw.vin(), raw.eventId()); - } - - public static String logicalUri(String key) { - return key == null || key.isBlank() ? "" : "archive://" + key; - } - - private static String metadataValue(Map metadata, String key) { - if (metadata == null || key == null) { - return ""; - } - String value = metadata.get(key); - return value == null ? "" : value; - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/RealtimePayload.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/RealtimePayload.java deleted file mode 100644 index 76ceefa1..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/RealtimePayload.java +++ /dev/null @@ -1,49 +0,0 @@ -package com.lingniu.ingest.api.event; - -/** - * 实时数据载荷（扁平 record，字段用 {@code Double} / {@code Integer} 允许 null，表达"未上报")。 - * - *

不再复刻旧服务 {@code VehicleDataActual} 的 273 字段巨型类 —— 协议特有细节通过 - * {@link VehicleEvent#metadata()} 传递。此处只保留跨协议通用字段。 - */ -public record RealtimePayload( - // ===== 动力 & 能源 ===== - Double speedKmh, - Double totalMileageKm, - Double batterySoc, - Double batteryVoltageV, - Double batteryCurrentA, - - // ===== 燃料电池 / 氢能 ===== - Double fcVoltageV, - Double fcCurrentA, - Double fcTempC, - Double hydrogenRemainingKg, - Double hydrogenHighPressureMpa, - Double hydrogenLowPressureMpa, - - // ===== 车辆状态 ===== - VehicleState vehicleState, - ChargingState chargingState, - RunningMode runningMode, - Integer gearLevel, - Double acceleratorPedal, - Double brakePedal, - - // ===== 位置（冗余一份便于单主题消费）===== - Double longitude, - Double latitude, - Double altitudeM, - Double directionDeg, - - // ===== 环境 ===== - Double ambientTempC, - Double coolantTempC -) { - - public enum VehicleState { STARTED, SHUTDOWN, OTHER, INVALID } - - public enum ChargingState { UNCHARGED, PARKED_CHARGING, DRIVING_CHARGING, CHARGED, OTHER, INVALID } - - public enum RunningMode { ELECTRIC, HYBRID, FUEL, OTHER, INVALID } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/TelemetryFieldValue.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/TelemetryFieldValue.java deleted file mode 100644 index d101ad25..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/TelemetryFieldValue.java +++ /dev/null @@ -1,79 +0,0 @@ -package com.lingniu.ingest.api.event; - -/** - * One normalized telemetry field value. - * - *

Protocol mappers use stable internal field keys so Kafka, TDengine, Redis, - * and statistics do not depend on protocol-specific names. - * - *

{@code value} 统一用字符串承载，真实类型由 {@code valueType} 表达。这样 Kafka - * protobuf、CSV 导出、JSON 字段和前端展示可以共用同一份字段结构；数值精度/格式化 - * 由查询层或前端按 {@code valueType + unit} 决定。 - */ -public record TelemetryFieldValue( - String key, - ValueType valueType, - String value, - String unit, - Quality quality, - String sourcePath -) { - - public TelemetryFieldValue { - if (key == null || key.isBlank()) { - throw new IllegalArgumentException("key must not be blank"); - } - if (valueType == null) { - throw new IllegalArgumentException("valueType must not be null"); - } - value = value == null ? "" : value; - unit = unit == null ? "" : unit; - // 默认 GOOD，只有解析器明确知道异常/无效/缺失时才降级，避免调用方到处补质量字段。 - quality = quality == null ? Quality.GOOD : quality; - sourcePath = sourcePath == null ? "" : sourcePath; - } - - public static TelemetryFieldValue stringValue(String key, String value, String unit, String sourcePath) { - return new TelemetryFieldValue(key, ValueType.STRING, value, unit, Quality.GOOD, sourcePath); - } - - public static TelemetryFieldValue doubleValue(String key, double value, String unit, String sourcePath) { - return new TelemetryFieldValue(key, ValueType.DOUBLE, Double.toString(value), unit, Quality.GOOD, sourcePath); - } - - public static TelemetryFieldValue longValue(String key, long value, String unit, String sourcePath) { - return new TelemetryFieldValue(key, ValueType.LONG, Long.toString(value), unit, Quality.GOOD, sourcePath); - } - - public static TelemetryFieldValue booleanValue(String key, boolean value, String unit, String sourcePath) { - return new TelemetryFieldValue(key, ValueType.BOOLEAN, Boolean.toString(value), unit, Quality.GOOD, sourcePath); - } - - public static TelemetryFieldValue instantValue(String key, String value, String unit, String sourcePath) { - return new TelemetryFieldValue(key, ValueType.INSTANT, value, unit, Quality.GOOD, sourcePath); - } - - public static TelemetryFieldValue jsonValue(String key, String value, String unit, String sourcePath) { - return new TelemetryFieldValue(key, ValueType.JSON, value, unit, Quality.GOOD, sourcePath); - } - - public static TelemetryFieldValue missing(String key, String unit, String sourcePath) { - return new TelemetryFieldValue(key, ValueType.STRING, "", unit, Quality.MISSING, sourcePath); - } - - public enum ValueType { - STRING, - DOUBLE, - LONG, - BOOLEAN, - INSTANT, - JSON - } - - public enum Quality { - GOOD, - ESTIMATED, - INVALID, - MISSING - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/TelemetryMetadataValues.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/TelemetryMetadataValues.java deleted file mode 100644 index 48f85bba..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/TelemetryMetadataValues.java +++ /dev/null @@ -1,22 +0,0 @@ -package com.lingniu.ingest.api.event; - -import java.util.Locale; - -/** - * Stable string formatting for metadata values shared across protocol mappers. - */ -public final class TelemetryMetadataValues { - - private TelemetryMetadataValues() { - } - - public static String doubleValue(double value) { - if (!Double.isFinite(value)) { - return ""; - } - String formatted = String.format(Locale.ROOT, "%.6f", value); - return formatted.indexOf('.') < 0 - ? formatted - : formatted.replaceAll("0+$", "").replaceAll("\\.$", ""); - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/TelemetrySnapshot.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/TelemetrySnapshot.java deleted file mode 100644 index 2035eb55..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/TelemetrySnapshot.java +++ /dev/null @@ -1,147 +0,0 @@ -package com.lingniu.ingest.api.event; - -import com.lingniu.ingest.api.ProtocolId; - -import java.time.Instant; -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; -import java.util.Optional; - -/** - * Normalized full-field telemetry snapshot for one parsed vehicle event. - * - *

This is the shared contract for Kafka, TDengine history, Redis hot state, - * and statistics. It intentionally lives in {@code ingest-api} and has no - * Spring, Kafka, Redis, or TDengine dependency. - */ -public record TelemetrySnapshot( - String eventId, - String vin, - ProtocolId protocol, - String eventType, - Instant eventTime, - Instant ingestTime, - String rawArchiveUri, - Map metadata, - List fields -) { - - public TelemetrySnapshot { - if (eventId == null || eventId.isBlank()) { - throw new IllegalArgumentException("eventId must not be blank"); - } - if (vin == null || vin.isBlank()) { - throw new IllegalArgumentException("vin must not be blank"); - } - if (protocol == null) { - throw new IllegalArgumentException("protocol must not be null"); - } - if (eventType == null || eventType.isBlank()) { - throw new IllegalArgumentException("eventType must not be blank"); - } - if (eventTime == null) { - throw new IllegalArgumentException("eventTime must not be null"); - } - if (ingestTime == null) { - throw new IllegalArgumentException("ingestTime must not be null"); - } - rawArchiveUri = rawArchiveUri == null ? "" : rawArchiveUri; - metadata = Map.copyOf(metadata == null ? Map.of() : metadata); - fields = List.copyOf(fields == null ? List.of() : fields); - // snapshot/fields 查询按 key 投影和导出，重复 key 会导致前端列和 CSV 表头不稳定，构造期直接拒绝。 - validateUniqueKeys(fields); - } - - public Optional field(String key) { - if (key == null || key.isBlank()) { - return Optional.empty(); - } - // fields 保持插入顺序，单字段查询走线性查找；批量查询应使用 fieldsByKey 建索引。 - for (TelemetryFieldValue field : fields) { - if (field.key().equals(key)) { - return Optional.of(field); - } - } - return Optional.empty(); - } - - public Map fieldsByKey() { - Map index = new LinkedHashMap<>(); - for (TelemetryFieldValue field : fields) { - index.put(field.key(), field); - } - // LinkedHashMap 保留原始字段顺序，CSV 导出和 Swagger 示例能稳定复现解析器顺序。 - return Map.copyOf(index); - } - - public static Builder builder(String eventId, - String vin, - ProtocolId protocol, - String eventType, - Instant eventTime, - Instant ingestTime) { - return new Builder(eventId, vin, protocol, eventType, eventTime, ingestTime); - } - - private static void validateUniqueKeys(List fields) { - Map seen = new LinkedHashMap<>(); - for (TelemetryFieldValue field : fields) { - if (field == null) { - throw new IllegalArgumentException("field must not be null"); - } - TelemetryFieldValue previous = seen.put(field.key(), field); - if (previous != null) { - throw new IllegalArgumentException("duplicate telemetry field key: " + field.key()); - } - } - } - - public static final class Builder { - private final String eventId; - private final String vin; - private final ProtocolId protocol; - private final String eventType; - private final Instant eventTime; - private final Instant ingestTime; - private String rawArchiveUri = ""; - private Map metadata = Map.of(); - private final List fields = new ArrayList<>(); - - private Builder(String eventId, - String vin, - ProtocolId protocol, - String eventType, - Instant eventTime, - Instant ingestTime) { - this.eventId = eventId; - this.vin = vin; - this.protocol = protocol; - this.eventType = eventType; - this.eventTime = eventTime; - this.ingestTime = ingestTime; - } - - public Builder rawArchiveUri(String rawArchiveUri) { - this.rawArchiveUri = rawArchiveUri; - return this; - } - - public Builder metadata(Map metadata) { - this.metadata = metadata == null ? Map.of() : Map.copyOf(metadata); - return this; - } - - public Builder addField(TelemetryFieldValue field) { - this.fields.add(field); - return this; - } - - public TelemetrySnapshot build() { - return new TelemetrySnapshot( - eventId, vin, protocol, eventType, eventTime, ingestTime, - rawArchiveUri, metadata, fields); - } - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/VehicleEvent.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/VehicleEvent.java deleted file mode 100644 index d4f4e90a..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/VehicleEvent.java +++ /dev/null @@ -1,186 +0,0 @@ -package com.lingniu.ingest.api.event; - -import com.lingniu.ingest.api.ProtocolId; - -import java.time.Instant; -import java.util.Map; - -/** - * 领域事件根类型。所有协议归一到这里。 - * - *

使用 sealed + record 表达有界闭合的事件集合，下游消费方可用 {@code switch} 模式匹配穷尽处理。 - * - *

字段约定： - *

    - *
  • {@link #vin} 作为 Kafka 分区 key，保证单车顺序 - *
  • {@link #eventTime} 设备上报时间（而非服务器接收时间） - *
  • {@link #ingestTime} 服务器接收时间，用于延迟监控 - *
  • {@link #traceId} W3C traceparent 上下文 - *
- */ -public sealed interface VehicleEvent - permits VehicleEvent.Realtime, - VehicleEvent.Location, - VehicleEvent.Alarm, - VehicleEvent.Login, - VehicleEvent.Logout, - VehicleEvent.Heartbeat, - VehicleEvent.MediaMeta, - VehicleEvent.Passthrough, - VehicleEvent.RawArchive { - - String eventId(); - String vin(); - ProtocolId source(); - Instant eventTime(); - Instant ingestTime(); - String traceId(); - - /** 通用元数据：协议版本、终端手机号、网关 IP 等。 */ - Map metadata(); - - // ===== 具体事件类型 ===== - - /** 整车实时数据（32960 实时上报 / MQTT 宇通 / JT808 位置扩展等都归一到这里）。 */ - record Realtime( - String eventId, - String vin, - ProtocolId source, - Instant eventTime, - Instant ingestTime, - String traceId, - Map metadata, - RealtimePayload payload - ) implements VehicleEvent {} - - /** 纯位置事件（808 的 0x0200 等映射到此）。 */ - record Location( - String eventId, - String vin, - ProtocolId source, - Instant eventTime, - Instant ingestTime, - String traceId, - Map metadata, - LocationPayload payload - ) implements VehicleEvent {} - - /** 报警事件。 */ - record Alarm( - String eventId, - String vin, - ProtocolId source, - Instant eventTime, - Instant ingestTime, - String traceId, - Map metadata, - AlarmPayload payload - ) implements VehicleEvent {} - - /** 设备登入 / 车辆登入。 */ - record Login( - String eventId, - String vin, - ProtocolId source, - Instant eventTime, - Instant ingestTime, - String traceId, - Map metadata, - String iccid, - String protocolVersion - ) implements VehicleEvent {} - - /** 设备登出。 */ - record Logout( - String eventId, - String vin, - ProtocolId source, - Instant eventTime, - Instant ingestTime, - String traceId, - Map metadata - ) implements VehicleEvent {} - - /** 心跳 / 链路保活。 */ - record Heartbeat( - String eventId, - String vin, - ProtocolId source, - Instant eventTime, - Instant ingestTime, - String traceId, - Map metadata - ) implements VehicleEvent {} - - /** 多媒体元数据（大文件本体走对象存储，通过 archiveRef 引用）。 */ - record MediaMeta( - String eventId, - String vin, - ProtocolId source, - Instant eventTime, - Instant ingestTime, - String traceId, - Map metadata, - String mediaId, - String mediaType, - long sizeBytes, - String archiveRef - ) implements VehicleEvent {} - - /** 透传 / 自定义扩展消息。 */ - record Passthrough( - String eventId, - String vin, - ProtocolId source, - Instant eventTime, - Instant ingestTime, - String traceId, - Map metadata, - int passthroughType, - byte[] data - ) implements VehicleEvent {} - - /** - * 原始报文归档事件：每条成功解码的入站帧由 Dispatcher 产出一条，携带原始字节和归档索引信息。 - * 32960 历史全字段查询依赖它对应的 {@code archive://...} 文件能够被回读。 - * - *

当前 {@code KafkaEventSink} 会发送本类型的索引信息，RAW bytes 由 archive 落盘保存； - * 因此启用 raw 落盘/转发实现时，必须同时保证 {@link RawArchiveKeys} 的 key 规则一致。 - * - * @param command 协议主命令码（如 32960 0x02/0x03 等），冗余在事件里便于按命令分片归档 - * @param infoType 协议子类型（可为 0），同上 - * @param rawBytes 原始入站字节，不可为 null - * @param parsedJson 已解析结构化 JSON。只用于 RAW 明细查询，不应复制到位置/字段等派生表。 - */ - record RawArchive( - String eventId, - String vin, - ProtocolId source, - Instant eventTime, - Instant ingestTime, - String traceId, - Map metadata, - int command, - int infoType, - byte[] rawBytes, - String parsedJson - ) implements VehicleEvent { - public RawArchive(String eventId, - String vin, - ProtocolId source, - Instant eventTime, - Instant ingestTime, - String traceId, - Map metadata, - int command, - int infoType, - byte[] rawBytes) { - this(eventId, vin, source, eventTime, ingestTime, traceId, metadata, - command, infoType, rawBytes, ""); - } - - public RawArchive { - parsedJson = parsedJson == null ? "" : parsedJson; - } - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/VehicleEventTelemetrySnapshotMapper.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/VehicleEventTelemetrySnapshotMapper.java deleted file mode 100644 index d4392d43..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/event/VehicleEventTelemetrySnapshotMapper.java +++ /dev/null @@ -1,206 +0,0 @@ -package com.lingniu.ingest.api.event; - -import java.util.Map; -import java.util.Optional; - -/** - * Converts current {@link VehicleEvent} variants into normalized full-field - * telemetry snapshots. - * - *

Protocol-specific mappers can add richer fields later, but all downstream - * modules should consume {@link TelemetrySnapshot} instead of protocol objects. - */ -public final class VehicleEventTelemetrySnapshotMapper { - - private VehicleEventTelemetrySnapshotMapper() { - } - - public static Optional toSnapshot(VehicleEvent event) { - if (event == null || event instanceof VehicleEvent.RawArchive) { - return Optional.empty(); - } - TelemetrySnapshot.Builder builder = TelemetrySnapshot.builder( - event.eventId(), - event.vin(), - event.source(), - eventType(event), - event.eventTime(), - event.ingestTime()) - .metadata(event.metadata()) - .rawArchiveUri(rawArchiveUri(event)); - - switch (event) { - case VehicleEvent.Realtime realtime -> addRealtimeFields(builder, realtime.payload()); - case VehicleEvent.Location location -> addLocationFields(builder, location.payload()); - case VehicleEvent.Alarm alarm -> addAlarmFields(builder, alarm.payload()); - case VehicleEvent.Login login -> { - addString(builder, "iccid", login.iccid(), "", "event.login.iccid"); - addString(builder, "protocol_version", login.protocolVersion(), "", "event.login.protocolVersion"); - } - case VehicleEvent.Logout ignored -> { - } - case VehicleEvent.Heartbeat ignored -> { - } - case VehicleEvent.MediaMeta media -> { - addString(builder, "media_id", media.mediaId(), "", "event.media.mediaId"); - addString(builder, "media_type", media.mediaType(), "", "event.media.mediaType"); - builder.addField(TelemetryFieldValue.longValue( - "media_size_bytes", media.sizeBytes(), "bytes", "event.media.sizeBytes")); - addString(builder, "media_archive_ref", media.archiveRef(), "", "event.media.archiveRef"); - } - case VehicleEvent.Passthrough passthrough -> { - builder.addField(TelemetryFieldValue.longValue( - "passthrough_type", passthrough.passthroughType(), "", "event.passthrough.type")); - builder.addField(TelemetryFieldValue.longValue( - "passthrough_size_bytes", - passthrough.data() == null ? 0 : passthrough.data().length, - "bytes", - "event.passthrough.data")); - } - case VehicleEvent.RawArchive ignored -> { - return Optional.empty(); - } - } - return Optional.of(builder.build()); - } - - private static void addRealtimeFields(TelemetrySnapshot.Builder builder, RealtimePayload p) { - if (p == null) return; - addDouble(builder, "speed_kmh", p.speedKmh(), "km/h", "event.realtime.speedKmh"); - addDouble(builder, "total_mileage_km", p.totalMileageKm(), "km", "event.realtime.totalMileageKm"); - addDouble(builder, "battery_soc", p.batterySoc(), "%", "event.realtime.batterySoc"); - addDouble(builder, "battery_voltage_v", p.batteryVoltageV(), "V", "event.realtime.batteryVoltageV"); - addDouble(builder, "battery_current_a", p.batteryCurrentA(), "A", "event.realtime.batteryCurrentA"); - addDouble(builder, "fc_voltage_v", p.fcVoltageV(), "V", "event.realtime.fcVoltageV"); - addDouble(builder, "fc_current_a", p.fcCurrentA(), "A", "event.realtime.fcCurrentA"); - addDouble(builder, "fc_temp_c", p.fcTempC(), "C", "event.realtime.fcTempC"); - addDouble(builder, "hydrogen_remaining_kg", p.hydrogenRemainingKg(), "kg", "event.realtime.hydrogenRemainingKg"); - addDouble(builder, "hydrogen_high_pressure_mpa", p.hydrogenHighPressureMpa(), "MPa", "event.realtime.hydrogenHighPressureMpa"); - addDouble(builder, "hydrogen_low_pressure_mpa", p.hydrogenLowPressureMpa(), "MPa", "event.realtime.hydrogenLowPressureMpa"); - addEnum(builder, "vehicle_state", p.vehicleState(), "", "event.realtime.vehicleState"); - addEnum(builder, "charging_state", p.chargingState(), "", "event.realtime.chargingState"); - addEnum(builder, "running_mode", p.runningMode(), "", "event.realtime.runningMode"); - addLong(builder, "gear_level", p.gearLevel(), "", "event.realtime.gearLevel"); - addDouble(builder, "accelerator_pedal", p.acceleratorPedal(), "%", "event.realtime.acceleratorPedal"); - addDouble(builder, "brake_pedal", p.brakePedal(), "%", "event.realtime.brakePedal"); - addDouble(builder, "longitude", p.longitude(), "deg", "event.realtime.longitude"); - addDouble(builder, "latitude", p.latitude(), "deg", "event.realtime.latitude"); - addDouble(builder, "altitude_m", p.altitudeM(), "m", "event.realtime.altitudeM"); - addDouble(builder, "direction_deg", p.directionDeg(), "deg", "event.realtime.directionDeg"); - addDouble(builder, "ambient_temp_c", p.ambientTempC(), "C", "event.realtime.ambientTempC"); - addDouble(builder, "coolant_temp_c", p.coolantTempC(), "C", "event.realtime.coolantTempC"); - } - - private static void addLocationFields(TelemetrySnapshot.Builder builder, LocationPayload p) { - if (p == null) return; - builder.addField(TelemetryFieldValue.doubleValue("longitude", p.longitude(), "deg", "event.location.longitude")); - builder.addField(TelemetryFieldValue.doubleValue("latitude", p.latitude(), "deg", "event.location.latitude")); - builder.addField(TelemetryFieldValue.doubleValue("altitude_m", p.altitudeM(), "m", "event.location.altitudeM")); - builder.addField(TelemetryFieldValue.doubleValue("speed_kmh", p.speedKmh(), "km/h", "event.location.speedKmh")); - addDouble(builder, "total_mileage_km", p.totalMileageKm(), "km", "event.location.totalMileageKm"); - builder.addField(TelemetryFieldValue.doubleValue("direction_deg", p.directionDeg(), "deg", "event.location.directionDeg")); - builder.addField(TelemetryFieldValue.longValue("location_alarm_flag", p.alarmFlag(), "", "event.location.alarmFlag")); - builder.addField(TelemetryFieldValue.longValue("location_status_raw", p.statusFlag(), "", "event.location.statusFlag")); - } - - private static void addAlarmFields(TelemetrySnapshot.Builder builder, AlarmPayload p) { - if (p == null) return; - addEnum(builder, "alarm_level", p.level(), "", "event.alarm.level"); - builder.addField(TelemetryFieldValue.longValue("alarm_type_code", p.alarmTypeCode(), "", "event.alarm.alarmTypeCode")); - addString(builder, "alarm_type_name", p.alarmTypeName(), "", "event.alarm.alarmTypeName"); - if (p.faultCodes() != null && !p.faultCodes().isEmpty()) { - builder.addField(TelemetryFieldValue.stringValue( - "fault_codes", String.join(",", p.faultCodes()), "", "event.alarm.faultCodes")); - } - if (p.activeBits() != null && !p.activeBits().isEmpty()) { - builder.addField(TelemetryFieldValue.stringValue( - "active_alarm_bits", String.join(",", p.activeBits()), "", "event.alarm.activeBits")); - } - addDouble(builder, "longitude", p.longitude(), "deg", "event.alarm.longitude"); - addDouble(builder, "latitude", p.latitude(), "deg", "event.alarm.latitude"); - addEnum(builder, "safety_category", p.safetyCategory(), "", "event.alarm.safetyCategory"); - builder.addField(TelemetryFieldValue.booleanValue( - "hydrogen_leak_detected", p.hydrogenLeakDetected(), "", "event.alarm.hydrogenLeakDetected")); - addEnum(builder, "hydrogen_leak_level", p.hydrogenLeakLevel(), "", "event.alarm.hydrogenLeakLevel"); - builder.addField(TelemetryFieldValue.booleanValue( - "hydrogen_leak_action_required", - p.hydrogenLeakActionRequired(), - "", - "event.alarm.hydrogenLeakActionRequired")); - } - - private static String eventType(VehicleEvent event) { - if (event instanceof VehicleEvent.Realtime) return "REALTIME"; - if (event instanceof VehicleEvent.Location) return "LOCATION"; - if (event instanceof VehicleEvent.Alarm) return "ALARM"; - if (event instanceof VehicleEvent.Login) return "LOGIN"; - if (event instanceof VehicleEvent.Logout) return "LOGOUT"; - if (event instanceof VehicleEvent.Heartbeat) return "HEARTBEAT"; - if (event instanceof VehicleEvent.MediaMeta) return "MEDIA_META"; - if (event instanceof VehicleEvent.Passthrough) return "PASSTHROUGH"; - return "UNKNOWN"; - } - - private static String rawArchiveUri(VehicleEvent event) { - Map metadata = event.metadata(); - if (metadata == null) { - return mediaArchiveRef(event); - } - String uri = metadata.getOrDefault(RawArchiveKeys.META_URI, ""); - if (uri != null && !uri.isBlank()) { - return uri; - } - String key = metadata.getOrDefault(RawArchiveKeys.META_KEY, ""); - if (key != null && !key.isBlank()) { - return RawArchiveKeys.logicalUri(key); - } - return mediaArchiveRef(event); - } - - private static String mediaArchiveRef(VehicleEvent event) { - if (event instanceof VehicleEvent.MediaMeta media && media.archiveRef() != null) { - return media.archiveRef(); - } - return ""; - } - - private static void addDouble(TelemetrySnapshot.Builder builder, - String key, - Double value, - String unit, - String sourcePath) { - if (value != null) { - builder.addField(TelemetryFieldValue.doubleValue(key, value, unit, sourcePath)); - } - } - - private static void addLong(TelemetrySnapshot.Builder builder, - String key, - Integer value, - String unit, - String sourcePath) { - if (value != null) { - builder.addField(TelemetryFieldValue.longValue(key, value, unit, sourcePath)); - } - } - - private static void addString(TelemetrySnapshot.Builder builder, - String key, - String value, - String unit, - String sourcePath) { - if (value != null && !value.isBlank()) { - builder.addField(TelemetryFieldValue.stringValue(key, value, unit, sourcePath)); - } - } - - private static void addEnum(TelemetrySnapshot.Builder builder, - String key, - Enum value, - String unit, - String sourcePath) { - if (value != null) { - builder.addField(TelemetryFieldValue.stringValue(key, value.name(), unit, sourcePath)); - } - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/package-info.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/package-info.java deleted file mode 100644 index c93d2a45..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/package-info.java +++ /dev/null @@ -1,13 +0,0 @@ -/** - * lingniu-vehicle-ingest 公共 API 模块。零 Spring 依赖，只定义 SPI、sealed 领域事件、注解。 - * - *

分包： - *

    - *
  • {@code annotation} - Handler 注解体系 - *
  • {@code event} - sealed {@code VehicleEvent} + payload records - *
  • {@code pipeline} - RawFrame / IngestContext / IngestInterceptor - *
  • {@code sink} - EventSink SPI - *
  • {@code spi} - ProtocolPlugin / FrameDecoder / FrameEncoder / EventMapper - *
- */ -package com.lingniu.ingest.api; diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/pipeline/IngestContext.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/pipeline/IngestContext.java deleted file mode 100644 index 6b54d829..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/pipeline/IngestContext.java +++ /dev/null @@ -1,50 +0,0 @@ -package com.lingniu.ingest.api.pipeline; - -import java.util.HashMap; -import java.util.Map; - -/** - * 一次消息处理的上下文，贯穿整个拦截链与 Handler。线程不共享，不需要同步。 - * - *

attributes 用于在入口、拦截器、Dispatcher 之间传递轻量元数据，例如 traceId、归档 URI、 - * 鉴权结果等；不要放大对象或长期缓存，避免高频上报时放大内存占用。 - */ -public final class IngestContext { - - private final String traceId; - private final Map attributes = new HashMap<>(8); - private volatile boolean aborted; - private volatile String abortReason; - - public IngestContext(String traceId) { - this.traceId = traceId; - } - - public String traceId() { - return traceId; - } - - public T attr(String key) { - @SuppressWarnings("unchecked") - T v = (T) attributes.get(key); - return v; - } - - public void attr(String key, Object value) { - attributes.put(key, value); - } - - public void abort(String reason) { - // reason 会进入日志/诊断信息，保持短小、机器可读，方便定位是去重、限流还是鉴权失败。 - this.aborted = true; - this.abortReason = reason; - } - - public boolean aborted() { - return aborted; - } - - public String abortReason() { - return abortReason; - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/pipeline/IngestInterceptor.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/pipeline/IngestInterceptor.java deleted file mode 100644 index e0574fa8..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/pipeline/IngestInterceptor.java +++ /dev/null @@ -1,21 +0,0 @@ -package com.lingniu.ingest.api.pipeline; - -import com.lingniu.ingest.api.event.VehicleEvent; - -/** - * 拦截器 SPI。内置实现：Auth / Dedup / RateLimit / CoordTransform / Tracing。 - * 实现 {@link org.springframework.core.Ordered}（或使用 {@code @Order}）控制顺序。 - */ -public interface IngestInterceptor { - - /** 解码后、分发到 Handler 之前触发。返回 false 表示终止后续处理。 */ - default boolean before(RawFrame frame, IngestContext ctx) { - return true; - } - - /** Handler 成功产出事件后触发，允许修饰事件或取消投递。 */ - default void after(VehicleEvent event, IngestContext ctx) {} - - /** 处理链任一环节抛出异常时触发。 */ - default void onError(Throwable error, IngestContext ctx) {} -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/pipeline/RawFrame.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/pipeline/RawFrame.java deleted file mode 100644 index d292ce19..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/pipeline/RawFrame.java +++ /dev/null @@ -1,30 +0,0 @@ -package com.lingniu.ingest.api.pipeline; - -import com.lingniu.ingest.api.ProtocolId; - -import java.time.Instant; -import java.util.Map; - -/** - * 进入 Pipeline 的原始帧描述。 - * - * @param protocolId 来源协议 - * @param command 协议主命令 / 消息 ID（由各协议自行定义） - * @param infoType 可选子类型（如 32960 的信息体 ID） - * @param payload 已经被 {@link com.lingniu.ingest.api.spi.FrameDecoder} 解析出的协议对象 - * @param rawBytes 原始字节，用于冷存与排障（可能为 null，按配置开关） - * @param sourceMeta 来源元数据：peer ip、端口、session id、topic 等 - * @param receivedAt 服务器接收时刻 - * - *

RawFrame 是所有协议进入统一 Pipeline 的边界对象。32960 TCP 入口会把 VIN、seq、 - * rawArchiveUri 等关键索引放进 sourceMeta，后续去重、快照和历史查询都依赖这些字段保持稳定。 - */ -public record RawFrame( - ProtocolId protocolId, - int command, - int infoType, - Object payload, - byte[] rawBytes, - Map sourceMeta, - Instant receivedAt -) {} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/sink/EventSink.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/sink/EventSink.java deleted file mode 100644 index dcc13451..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/sink/EventSink.java +++ /dev/null @@ -1,28 +0,0 @@ -package com.lingniu.ingest.api.sink; - -import com.lingniu.ingest.api.event.VehicleEvent; - -import java.util.List; -import java.util.concurrent.CompletableFuture; - -/** - * 事件出口 SPI。Kafka、本地归档、指标打点都是一种 Sink。 - * - *

实现应当线程安全、非阻塞（内部异步 IO）。由 Disruptor EventBus 扇出调用。 - */ -public interface EventSink { - - String name(); - - CompletableFuture publish(VehicleEvent event); - - default CompletableFuture publishBatch(List batch) { - CompletableFuture[] all = batch.stream().map(this::publish).toArray(CompletableFuture[]::new); - return CompletableFuture.allOf(all); - } - - /** 是否接受此事件类型。默认全部接受。 */ - default boolean accepts(VehicleEvent event) { - return true; - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/DecodeException.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/DecodeException.java deleted file mode 100644 index 0c49d960..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/DecodeException.java +++ /dev/null @@ -1,12 +0,0 @@ -package com.lingniu.ingest.api.spi; - -/** 解码阶段异常，指示数据体本身违反协议。应被 Dispatcher 路由到 DLQ。 */ -public class DecodeException extends RuntimeException { - public DecodeException(String message) { - super(message); - } - - public DecodeException(String message, Throwable cause) { - super(message, cause); - } -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/EventMapper.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/EventMapper.java deleted file mode 100644 index ecceef3c..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/EventMapper.java +++ /dev/null @@ -1,16 +0,0 @@ -package com.lingniu.ingest.api.spi; - -import com.lingniu.ingest.api.event.VehicleEvent; - -import java.util.List; - -/** - * 协议消息 → 领域事件的映射器（Adapter 模式）。 - * - *

实现类负责把"我这个协议特有的 Java 对象"翻译成统一的 {@link VehicleEvent}。 - * 允许一条协议消息展开为多条领域事件（例如 32960 的一次实时上报同时产出 Realtime + Location）。 - */ -@FunctionalInterface -public interface EventMapper { - List toEvents(I message); -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/FrameDecoder.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/FrameDecoder.java deleted file mode 100644 index 9aa9eb90..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/FrameDecoder.java +++ /dev/null @@ -1,13 +0,0 @@ -package com.lingniu.ingest.api.spi; - -import java.nio.ByteBuffer; - -/** - * 协议帧解码器：字节流 → 协议消息对象。 - * - *

实现类应是无状态的（拆包/粘包由上游 Netty 的帧解码器处理），此处只负责一条完整帧的解析。 - */ -@FunctionalInterface -public interface FrameDecoder { - I decode(ByteBuffer buffer) throws DecodeException; -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/FrameEncoder.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/FrameEncoder.java deleted file mode 100644 index ded9040d..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/FrameEncoder.java +++ /dev/null @@ -1,9 +0,0 @@ -package com.lingniu.ingest.api.spi; - -import java.nio.ByteBuffer; - -/** 下行帧编码器。无下行的协议返回 null 即可。 */ -@FunctionalInterface -public interface FrameEncoder { - ByteBuffer encode(O message); -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/MessageHandler.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/MessageHandler.java deleted file mode 100644 index 13b5278d..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/MessageHandler.java +++ /dev/null @@ -1,16 +0,0 @@ -package com.lingniu.ingest.api.spi; - -import com.lingniu.ingest.api.event.VehicleEvent; - -import java.util.List; - -/** - * 编程式 Handler 接口。推荐使用 {@code @ProtocolHandler + @MessageMapping} 注解方式代替。 - * 此接口保留是为了 SPI 场景下的脚本式扩展（例如动态加载的 groovy handler）。 - */ -public interface MessageHandler { - - boolean supports(I message); - - List handle(I message); -} diff --git a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/ProtocolPlugin.java b/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/ProtocolPlugin.java deleted file mode 100644 index 49947910..00000000 --- a/modules/core/ingest-api/src/main/java/com/lingniu/ingest/api/spi/ProtocolPlugin.java +++ /dev/null @@ -1,31 +0,0 @@ -package com.lingniu.ingest.api.spi; - -import com.lingniu.ingest.api.ProtocolId; - -import java.util.List; - -/** - * 协议插件 SPI。每个协议模块提供一个实现，由 {@code ProtocolPluginRegistry} 通过 ServiceLoader - * 或 Spring Bean 方式发现并装配。 - * - * @param 协议解码后的消息类型 - * @param 下行消息类型（无下行的协议用 {@link Void}） - */ -public interface ProtocolPlugin { - - ProtocolId id(); - - FrameDecoder decoder(); - - FrameEncoder encoder(); - - EventMapper eventMapper(); - - /** - * 插件自带的协议特定 Handler 列表（可选）。 - * 通常推荐使用 {@code @ProtocolHandler} + Spring Bean 的方式注册，此处返回空列表。 - */ - default List> builtinHandlers() { - return List.of(); - } -} diff --git a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/consumer/EnvelopeConsumerProcessorTest.java b/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/consumer/EnvelopeConsumerProcessorTest.java deleted file mode 100644 index 80f142f2..00000000 --- a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/consumer/EnvelopeConsumerProcessorTest.java +++ /dev/null @@ -1,55 +0,0 @@ -package com.lingniu.ingest.api.consumer; - -import org.junit.jupiter.api.Test; - -import java.util.ArrayList; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class EnvelopeConsumerProcessorTest { - - @Test - void invalidEnvelopePublishesDeadLetterRecordWithKafkaPosition() { - CapturingDeadLetterSink deadLetters = new CapturingDeadLetterSink(); - EnvelopeConsumerProcessor processor = new EnvelopeConsumerProcessor( - "event-history", ignored -> EnvelopeIngestResult.invalid("VehicleEnvelope bytes are invalid"), deadLetters); - - EnvelopeIngestResult result = processor.process( - new EnvelopeConsumerRecord("vehicle.realtime", 2, 42L, "VIN001", new byte[]{0x01, 0x02})); - - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.INVALID_ENVELOPE); - assertThat(deadLetters.records).singleElement().satisfies(record -> { - assertThat(record.service()).isEqualTo("event-history"); - assertThat(record.topic()).isEqualTo("vehicle.realtime"); - assertThat(record.partition()).isEqualTo(2); - assertThat(record.offset()).isEqualTo(42L); - assertThat(record.key()).isEqualTo("VIN001"); - assertThat(record.status()).isEqualTo(EnvelopeIngestResult.Status.INVALID_ENVELOPE); - assertThat(record.message()).contains("VehicleEnvelope"); - assertThat(record.payload()).containsExactly(0x01, 0x02); - }); - } - - @Test - void processedEnvelopeDoesNotPublishDeadLetter() { - CapturingDeadLetterSink deadLetters = new CapturingDeadLetterSink(); - EnvelopeConsumerProcessor processor = new EnvelopeConsumerProcessor( - "vehicle-state", ignored -> EnvelopeIngestResult.processed("event-1", "VIN001"), deadLetters); - - EnvelopeIngestResult result = processor.process( - new EnvelopeConsumerRecord("vehicle.realtime", 0, 7L, "VIN001", new byte[]{0x0a})); - - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.PROCESSED); - assertThat(deadLetters.records).isEmpty(); - } - - private static final class CapturingDeadLetterSink implements EnvelopeDeadLetterSink { - private final List records = new ArrayList<>(); - - @Override - public void publish(EnvelopeDeadLetterRecord record) { - records.add(record); - } - } -} diff --git a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/RawArchiveKeysTest.java b/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/RawArchiveKeysTest.java deleted file mode 100644 index a8e92eab..00000000 --- a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/RawArchiveKeysTest.java +++ /dev/null @@ -1,40 +0,0 @@ -package com.lingniu.ingest.api.event; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class RawArchiveKeysTest { - - @Test - void keyDateUsesShanghaiBusinessDayInsteadOfUtcDay() { - String key = RawArchiveKeys.key( - Instant.parse("2026-06-22T16:30:00Z"), - ProtocolId.GB32960, - "VIN001", - "event-1"); - - assertThat(key).isEqualTo("2026/06/23/GB32960/VIN001/event-1.bin"); - } - - @Test - void keyForRawArchiveUsesPrecomputedMetadataKey() { - VehicleEvent.RawArchive raw = new VehicleEvent.RawArchive( - "event-1", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-22T16:30:00Z"), - Instant.parse("2026-06-22T16:30:00Z"), - "trace-1", - Map.of(RawArchiveKeys.META_KEY, "precomputed/key.bin"), - 0x02, - 0, - new byte[]{0x23, 0x23}); - - assertThat(RawArchiveKeys.key(raw)).isEqualTo("precomputed/key.bin"); - } -} diff --git a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/TelemetryFieldValueTest.java b/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/TelemetryFieldValueTest.java deleted file mode 100644 index e38170ec..00000000 --- a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/TelemetryFieldValueTest.java +++ /dev/null @@ -1,39 +0,0 @@ -package com.lingniu.ingest.api.event; - -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class TelemetryFieldValueTest { - - @Test - void createsTypedDoubleFieldWithStableMetadata() { - TelemetryFieldValue field = TelemetryFieldValue.doubleValue( - "total_mileage_km", 12345.6, "km", "gb32960.vehicle.totalMileageKm"); - - assertThat(field.key()).isEqualTo("total_mileage_km"); - assertThat(field.valueType()).isEqualTo(TelemetryFieldValue.ValueType.DOUBLE); - assertThat(field.value()).isEqualTo("12345.6"); - assertThat(field.unit()).isEqualTo("km"); - assertThat(field.quality()).isEqualTo(TelemetryFieldValue.Quality.GOOD); - assertThat(field.sourcePath()).isEqualTo("gb32960.vehicle.totalMileageKm"); - } - - @Test - void rejectsBlankFieldKey() { - assertThatThrownBy(() -> TelemetryFieldValue.stringValue(" ", "STARTED", "", "gb32960.vehicle.state")) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("key"); - } - - @Test - void marksMissingValueExplicitly() { - TelemetryFieldValue field = TelemetryFieldValue.missing( - "hydrogen_remaining_kg", "kg", "gb32960.vendor.hydrogenRemaining"); - - assertThat(field.valueType()).isEqualTo(TelemetryFieldValue.ValueType.STRING); - assertThat(field.value()).isEmpty(); - assertThat(field.quality()).isEqualTo(TelemetryFieldValue.Quality.MISSING); - } -} diff --git a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/TelemetryMetadataValuesTest.java b/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/TelemetryMetadataValuesTest.java deleted file mode 100644 index d19abfc6..00000000 --- a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/TelemetryMetadataValuesTest.java +++ /dev/null @@ -1,23 +0,0 @@ -package com.lingniu.ingest.api.event; - -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; - -class TelemetryMetadataValuesTest { - - @Test - void formatsDoubleWithoutFloatingPointNoise() { - assertThat(TelemetryMetadataValues.doubleValue(123456.70000000001)).isEqualTo("123456.7"); - assertThat(TelemetryMetadataValues.doubleValue(1234.5)).isEqualTo("1234.5"); - assertThat(TelemetryMetadataValues.doubleValue(1234.0)).isEqualTo("1234"); - assertThat(TelemetryMetadataValues.doubleValue(1.23456789)).isEqualTo("1.234568"); - } - - @Test - void rejectsNonFiniteDoubleValuesAsBlankMetadata() { - assertThat(TelemetryMetadataValues.doubleValue(Double.NaN)).isEmpty(); - assertThat(TelemetryMetadataValues.doubleValue(Double.POSITIVE_INFINITY)).isEmpty(); - assertThat(TelemetryMetadataValues.doubleValue(Double.NEGATIVE_INFINITY)).isEmpty(); - } -} diff --git a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/TelemetrySnapshotTest.java b/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/TelemetrySnapshotTest.java deleted file mode 100644 index 3b22ccf7..00000000 --- a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/TelemetrySnapshotTest.java +++ /dev/null @@ -1,58 +0,0 @@ -package com.lingniu.ingest.api.event; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class TelemetrySnapshotTest { - - @Test - void indexesFieldsByInternalKey() { - TelemetrySnapshot snapshot = new TelemetrySnapshot( - "event-1", - "VIN001", - ProtocolId.GB32960, - "REALTIME", - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "archive://raw/event-1.bin", - Map.of("protocolVersion", "V2016"), - List.of( - TelemetryFieldValue.doubleValue("total_mileage_km", 120.5, "km", "gb32960.vehicle.totalMileageKm"), - TelemetryFieldValue.booleanValue("hydrogen_leak_detected", true, "", "gb32960.alarm.HYDROGEN_LEAK") - ) - ); - - assertThat(snapshot.field("total_mileage_km")).isPresent(); - assertThat(snapshot.field("total_mileage_km").orElseThrow().value()).isEqualTo("120.5"); - assertThat(snapshot.field("hydrogen_leak_detected").orElseThrow().value()).isEqualTo("true"); - assertThat(snapshot.field("unknown")).isEmpty(); - assertThat(snapshot.metadata()).containsEntry("protocolVersion", "V2016"); - } - - @Test - void rejectsDuplicateFieldKeys() { - assertThatThrownBy(() -> new TelemetrySnapshot( - "event-1", - "VIN001", - ProtocolId.GB32960, - "REALTIME", - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "", - Map.of(), - List.of( - TelemetryFieldValue.doubleValue("speed_kmh", 1.0, "km/h", "a"), - TelemetryFieldValue.doubleValue("speed_kmh", 2.0, "km/h", "b") - ) - )) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("duplicate"); - } -} diff --git a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/VehicleEventTelemetrySnapshotMapperTest.java b/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/VehicleEventTelemetrySnapshotMapperTest.java deleted file mode 100644 index ab3caeba..00000000 --- a/modules/core/ingest-api/src/test/java/com/lingniu/ingest/api/event/VehicleEventTelemetrySnapshotMapperTest.java +++ /dev/null @@ -1,175 +0,0 @@ -package com.lingniu.ingest.api.event; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class VehicleEventTelemetrySnapshotMapperTest { - - @Test - void mapsRealtimeEventToInternalTelemetryFields() { - VehicleEvent.Realtime event = new VehicleEvent.Realtime( - "event-1", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "trace-1", - Map.of("protocolVersion", "V2016", "rawArchiveUri", "archive://raw/event-1.bin"), - new RealtimePayload( - 80.5, - 12345.6, - 55.0, - 610.0, - -20.5, - 220.0, - 100.0, - 65.0, - 8.2, - 35.0, - 1.1, - RealtimePayload.VehicleState.STARTED, - RealtimePayload.ChargingState.UNCHARGED, - RealtimePayload.RunningMode.FUEL, - 3, - 20.0, - 0.0, - 113.12, - 23.45, - 10.0, - 180.0, - 30.0, - 70.0 - ) - ); - - TelemetrySnapshot snapshot = VehicleEventTelemetrySnapshotMapper.toSnapshot(event).orElseThrow(); - - assertThat(snapshot.eventId()).isEqualTo("event-1"); - assertThat(snapshot.rawArchiveUri()).isEqualTo("archive://raw/event-1.bin"); - assertThat(snapshot.field("speed_kmh").orElseThrow().value()).isEqualTo("80.5"); - assertThat(snapshot.field("total_mileage_km").orElseThrow().value()).isEqualTo("12345.6"); - assertThat(snapshot.field("vehicle_state").orElseThrow().value()).isEqualTo("STARTED"); - assertThat(snapshot.field("hydrogen_high_pressure_mpa").orElseThrow().value()).isEqualTo("35.0"); - assertThat(snapshot.field("longitude").orElseThrow().value()).isEqualTo("113.12"); - } - - @Test - void mapsHydrogenLeakAlarmToInternalSafetyFields() { - VehicleEvent.Alarm event = new VehicleEvent.Alarm( - "event-2", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "trace-2", - Map.of("protocolVersion", "V2016"), - new AlarmPayload( - AlarmPayload.AlarmLevel.CRITICAL, - 1, - "GB32960_ALARM", - java.util.List.of("OTH-1"), - java.util.Set.of("HYDROGEN_LEAK"), - 113.12, - 23.45, - AlarmPayload.SafetyCategory.HYDROGEN_LEAK, - true, - AlarmPayload.HydrogenLeakLevel.CRITICAL, - true - ) - ); - - TelemetrySnapshot snapshot = VehicleEventTelemetrySnapshotMapper.toSnapshot(event).orElseThrow(); - - assertThat(snapshot.eventType()).isEqualTo("ALARM"); - assertThat(snapshot.field("hydrogen_leak_detected").orElseThrow().value()).isEqualTo("true"); - assertThat(snapshot.field("hydrogen_leak_level").orElseThrow().value()).isEqualTo("CRITICAL"); - assertThat(snapshot.field("hydrogen_leak_action_required").orElseThrow().value()).isEqualTo("true"); - assertThat(snapshot.field("safety_category").orElseThrow().value()).isEqualTo("HYDROGEN_LEAK"); - } - - @Test - void mapsLocationMileageToInternalTelemetryField() { - VehicleEvent.Location event = new VehicleEvent.Location( - "event-location-1", - "VIN001", - ProtocolId.JT808, - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "trace-location-1", - Map.of("messageId", "0x200"), - new LocationPayload(113.12, 23.45, 10.0, 60.0, 90.0, 0, 1, 1234.5) - ); - - TelemetrySnapshot snapshot = VehicleEventTelemetrySnapshotMapper.toSnapshot(event).orElseThrow(); - - assertThat(snapshot.field("total_mileage_km").orElseThrow().value()).isEqualTo("1234.5"); - assertThat(snapshot.field("total_mileage_km").orElseThrow().sourcePath()) - .isEqualTo("event.location.totalMileageKm"); - } - - @Test - void mediaMetaUsesArchiveRefAsQueryableRawArchiveUriFallback() { - VehicleEvent.MediaMeta event = new VehicleEvent.MediaMeta( - "media-1", - "VIN001", - ProtocolId.JT1078, - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "trace-4", - Map.of("channel", "1"), - "media-001", - "jt1078-video-h264-channel-1", - 1024, - "file:///archive/jt1078/2026/06/22/VIN001/media-001.rtp" - ); - - TelemetrySnapshot snapshot = VehicleEventTelemetrySnapshotMapper.toSnapshot(event).orElseThrow(); - - assertThat(snapshot.eventType()).isEqualTo("MEDIA_META"); - assertThat(snapshot.rawArchiveUri()) - .isEqualTo("file:///archive/jt1078/2026/06/22/VIN001/media-001.rtp"); - assertThat(snapshot.field("media_archive_ref").orElseThrow().value()) - .isEqualTo("file:///archive/jt1078/2026/06/22/VIN001/media-001.rtp"); - } - - @Test - void derivesRawArchiveUriFromRawArchiveKeyMetadata() { - VehicleEvent.Heartbeat event = new VehicleEvent.Heartbeat( - "heartbeat-1", - "VIN001", - ProtocolId.JT808, - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "trace-5", - Map.of("rawArchiveKey", "2026/06/22/JT808/VIN001/raw-1.bin") - ); - - TelemetrySnapshot snapshot = VehicleEventTelemetrySnapshotMapper.toSnapshot(event).orElseThrow(); - - assertThat(snapshot.rawArchiveUri()) - .isEqualTo("archive://2026/06/22/JT808/VIN001/raw-1.bin"); - } - - @Test - void skipsRawArchiveBecauseItIsStoredByArchiveSink() { - VehicleEvent.RawArchive raw = new VehicleEvent.RawArchive( - "raw-1", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "trace-3", - Map.of(), - 0x02, - 0, - new byte[]{0x23, 0x23} - ); - - assertThat(VehicleEventTelemetrySnapshotMapper.toSnapshot(raw)).isEmpty(); - } -} diff --git a/modules/core/ingest-codec-common/pom.xml b/modules/core/ingest-codec-common/pom.xml deleted file mode 100644 index 80d9e6a3..00000000 --- a/modules/core/ingest-codec-common/pom.xml +++ /dev/null @@ -1,30 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - ingest-codec-common - ingest-codec-common - BCD / CRC / BCC / bit utils 等公共编解码工具。 - - - - io.netty - netty-buffer - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - diff --git a/modules/core/ingest-codec-common/src/main/java/com/lingniu/ingest/codec/BccChecksum.java b/modules/core/ingest-codec-common/src/main/java/com/lingniu/ingest/codec/BccChecksum.java deleted file mode 100644 index 5dc49274..00000000 --- a/modules/core/ingest-codec-common/src/main/java/com/lingniu/ingest/codec/BccChecksum.java +++ /dev/null @@ -1,19 +0,0 @@ -package com.lingniu.ingest.codec; - -/** BCC（异或校验）。GB/T 32960 与 JT/T 808 都使用此方式做帧尾校验。 */ -public final class BccChecksum { - - private BccChecksum() {} - - public static byte compute(byte[] data, int offset, int length) { - byte sum = 0; - for (int i = offset; i < offset + length; i++) { - sum ^= data[i]; - } - return sum; - } - - public static boolean verify(byte[] data, int offset, int length, byte expected) { - return compute(data, offset, length) == expected; - } -} diff --git a/modules/core/ingest-codec-common/src/main/java/com/lingniu/ingest/codec/BcdCodec.java b/modules/core/ingest-codec-common/src/main/java/com/lingniu/ingest/codec/BcdCodec.java deleted file mode 100644 index 32fdddbd..00000000 --- a/modules/core/ingest-codec-common/src/main/java/com/lingniu/ingest/codec/BcdCodec.java +++ /dev/null @@ -1,30 +0,0 @@ -package com.lingniu.ingest.codec; - -/** BCD (Binary-Coded Decimal) 编解码。JT/T 808 的手机号、时间戳常用。 */ -public final class BcdCodec { - - private BcdCodec() {} - - public static byte[] encode(String decimal) { - int len = (decimal.length() + 1) / 2; - byte[] out = new byte[len]; - int idx = decimal.length() % 2; - if (idx == 1) { - out[0] = (byte) Character.digit(decimal.charAt(0), 10); - } - for (int i = idx; i < decimal.length(); i += 2) { - int hi = Character.digit(decimal.charAt(i), 10); - int lo = Character.digit(decimal.charAt(i + 1), 10); - out[(i + idx) / 2] = (byte) ((hi << 4) | lo); - } - return out; - } - - public static String decode(byte[] bcd) { - StringBuilder sb = new StringBuilder(bcd.length * 2); - for (byte b : bcd) { - sb.append((b >> 4) & 0x0F).append(b & 0x0F); - } - return sb.toString(); - } -} diff --git a/modules/core/ingest-codec-common/src/main/java/com/lingniu/ingest/codec/BitUtils.java b/modules/core/ingest-codec-common/src/main/java/com/lingniu/ingest/codec/BitUtils.java deleted file mode 100644 index cf3034fd..00000000 --- a/modules/core/ingest-codec-common/src/main/java/com/lingniu/ingest/codec/BitUtils.java +++ /dev/null @@ -1,34 +0,0 @@ -package com.lingniu.ingest.codec; - -/** 位运算工具：从字节数组读写无符号整数。 */ -public final class BitUtils { - - private BitUtils() {} - - public static int readUint8(byte[] buf, int offset) { - return buf[offset] & 0xFF; - } - - public static int readUint16BE(byte[] buf, int offset) { - return ((buf[offset] & 0xFF) << 8) | (buf[offset + 1] & 0xFF); - } - - public static long readUint32BE(byte[] buf, int offset) { - return ((long) (buf[offset] & 0xFF) << 24) - | ((long) (buf[offset + 1] & 0xFF) << 16) - | ((long) (buf[offset + 2] & 0xFF) << 8) - | (buf[offset + 3] & 0xFF); - } - - public static void writeUint16BE(byte[] buf, int offset, int value) { - buf[offset] = (byte) ((value >> 8) & 0xFF); - buf[offset + 1] = (byte) (value & 0xFF); - } - - public static void writeUint32BE(byte[] buf, int offset, long value) { - buf[offset] = (byte) ((value >> 24) & 0xFF); - buf[offset + 1] = (byte) ((value >> 16) & 0xFF); - buf[offset + 2] = (byte) ((value >> 8) & 0xFF); - buf[offset + 3] = (byte) (value & 0xFF); - } -} diff --git a/modules/core/ingest-codec-common/src/test/java/com/lingniu/ingest/codec/BcdCodecTest.java b/modules/core/ingest-codec-common/src/test/java/com/lingniu/ingest/codec/BcdCodecTest.java deleted file mode 100644 index 7ee9f94f..00000000 --- a/modules/core/ingest-codec-common/src/test/java/com/lingniu/ingest/codec/BcdCodecTest.java +++ /dev/null @@ -1,28 +0,0 @@ -package com.lingniu.ingest.codec; - -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; - -class BcdCodecTest { - - @Test - void roundTripEvenLength() { - // 偶数位直接等值 - byte[] bcd = BcdCodec.encode("1380013800"); - assertThat(BcdCodec.decode(bcd)).isEqualTo("1380013800"); - } - - @Test - void roundTripOddLengthPadsLeadingZero() { - // 奇数位会在高位补 0 - byte[] bcd = BcdCodec.encode("13800138000"); - assertThat(BcdCodec.decode(bcd)).isEqualTo("013800138000"); - } - - @Test - void checksumMatches() { - byte[] data = {0x01, 0x02, 0x03, 0x04}; - assertThat(BccChecksum.compute(data, 0, data.length)).isEqualTo((byte) 0x04); - } -} diff --git a/modules/core/ingest-core/pom.xml b/modules/core/ingest-core/pom.xml deleted file mode 100644 index 00dbde8f..00000000 --- a/modules/core/ingest-core/pom.xml +++ /dev/null @@ -1,74 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - ingest-core - ingest-core - Dispatcher / Interceptor Chain / Disruptor Event Bus / 注解扫描与自动注册。 - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - ingest-codec-common - - - com.lingniu.ingest - ingest-facts - - - org.springframework.boot - spring-boot-starter - - - com.fasterxml.jackson.core - jackson-databind - - - com.fasterxml.jackson.datatype - jackson-datatype-jsr310 - - - org.springframework.boot - spring-boot-autoconfigure - - - org.springframework.boot - spring-boot-configuration-processor - true - - - com.lmax - disruptor - - - com.github.ben-manes.caffeine - caffeine - - - io.github.resilience4j - resilience4j-ratelimiter - - - io.github.resilience4j - resilience4j-circuitbreaker - - - io.netty - netty-all - - - org.springframework.boot - spring-boot-starter-test - test - - - diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/AsyncBatchExecutor.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/AsyncBatchExecutor.java deleted file mode 100644 index 0399e506..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/AsyncBatchExecutor.java +++ /dev/null @@ -1,206 +0,0 @@ -package com.lingniu.ingest.core.concurrency; - -import com.lingniu.ingest.api.annotation.AsyncBatch; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.core.dispatcher.HandlerDefinition; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.lang.reflect.InvocationTargetException; -import java.lang.reflect.Method; -import java.util.ArrayList; -import java.util.List; -import java.util.concurrent.ArrayBlockingQueue; -import java.util.concurrent.BlockingQueue; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ConcurrentMap; -import java.util.concurrent.TimeUnit; -import java.util.function.Consumer; -import java.util.function.UnaryOperator; - -/** - * {@link AsyncBatch} 注解的执行器：把 Handler 方法从"单条调用"变成"批量调用"。 - * - *

每个 Handler 方法对应一个 {@link Batcher}，背后是一个固定容量的 {@link BlockingQueue} - * 和 {@link AsyncBatch#poolSize()} 条守护虚拟线程。批量触发条件： - *

    - *
  • 累积 {@link AsyncBatch#size()} 条立即 flush - *
  • 从第一条入队起等待 {@link AsyncBatch#waitMs()} 毫秒后强制 flush - *
- * - *

目标 Handler 方法签名约定： - * 

{@code
- *   @MessageMapping(...)
- *   @AsyncBatch(size = 4000, waitMs = 1000, poolSize = 2)
- *   public List onBatch(List batch) { ... }
- * }
- * - *

flush 产出的事件由构造器注入的 {@code eventPublisher} 异步投递（通常是 - * {@link DisruptorEventBus#publish}），不阻塞 Netty EventLoop。 - */ -public final class AsyncBatchExecutor implements AutoCloseable { - - private static final Logger log = LoggerFactory.getLogger(AsyncBatchExecutor.class); - - private final Consumer eventPublisher; - private final ConcurrentMap batchers = new ConcurrentHashMap<>(); - - public AsyncBatchExecutor(Consumer eventPublisher) { - this.eventPublisher = eventPublisher; - } - - /** 供 Dispatcher 调用：把单条消息交给目标 Handler 的 batcher。 */ - public void submit(HandlerDefinition def, Object message) { - Batcher b = batchers.computeIfAbsent(def.method(), m -> new Batcher(def, eventPublisher)); - b.offer(new BatchItem(message, UnaryOperator.identity(), false)); - } - - public void submit(HandlerDefinition def, Object message, UnaryOperator eventTransformer) { - Batcher b = batchers.computeIfAbsent(def.method(), m -> new Batcher(def, eventPublisher)); - b.offer(new BatchItem(message, eventTransformer == null ? UnaryOperator.identity() : eventTransformer, true)); - } - - @Override - public void close() { - batchers.values().forEach(Batcher::close); - batchers.clear(); - log.info("AsyncBatchExecutor closed"); - } - - // ===== internals ===== - - private static final class Batcher implements AutoCloseable { - private final HandlerDefinition def; - private final Consumer publisher; - private final int batchSize; - private final long waitMs; - private final BlockingQueue queue; - private final Thread[] workers; - private volatile boolean running = true; - - Batcher(HandlerDefinition def, Consumer publisher) { - AsyncBatch cfg = def.asyncBatch(); - this.def = def; - this.publisher = publisher; - this.batchSize = Math.max(1, cfg.size()); - this.waitMs = Math.max(1, cfg.waitMs()); - this.queue = new ArrayBlockingQueue<>(Math.max(batchSize * 4, 16)); - int pool = Math.max(1, cfg.poolSize()); - this.workers = new Thread[pool]; - for (int i = 0; i < pool; i++) { - Thread t = Thread.ofVirtual() - .name("batcher-" + def.method().getName() + "-" + i) - .unstarted(this::loop); - workers[i] = t; - t.start(); - } - log.info("batcher started method={} size={} waitMs={} pool={}", - def.method().getName(), batchSize, waitMs, pool); - } - - void offer(BatchItem item) { - try { - if (!queue.offer(item, 100, TimeUnit.MILLISECONDS)) { - log.warn("batcher queue full, dropping item for {}", def.method().getName()); - } - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); - } - } - - private void loop() { - while (running) { - try { - BatchItem first = queue.poll(200, TimeUnit.MILLISECONDS); - if (first == null) continue; - List buf = new ArrayList<>(batchSize); - buf.add(first); - long deadlineNanos = System.nanoTime() + waitMs * 1_000_000L; - while (buf.size() < batchSize) { - long remain = deadlineNanos - System.nanoTime(); - if (remain <= 0) break; - BatchItem next = queue.poll(remain, TimeUnit.NANOSECONDS); - if (next == null) break; - buf.add(next); - } - flush(buf); - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); - return; - } catch (Exception e) { - log.error("batcher loop error method={}", def.method().getName(), e); - } - } - } - - private void flush(List buf) { - if (requiresPerItemTransform(buf)) { - // rawArchiveUri/rawSize 等元数据是每帧唯一的。为了不把第一条帧的元数据错误复制给整批事件， - // 一旦存在逐条 transformer，就按单条调用 Handler，再逐条发布。 - for (BatchItem item : buf) { - flushTransformedItem(item); - } - return; - } - - publishEvents(invoke(buf.stream().map(BatchItem::message).toList()), UnaryOperator.identity()); - } - - private boolean requiresPerItemTransform(List buf) { - for (BatchItem item : buf) { - if (item.transformRequired()) { - return true; - } - } - return false; - } - - private void flushTransformedItem(BatchItem item) { - publishEvents(invoke(List.of(item.message())), item.eventTransformer()); - } - - @SuppressWarnings("unchecked") - private List invoke(List messages) { - List events; - try { - // Handler 仍然只暴露一个批量签名；是否批满或超时由 Batcher 统一处理。 - Object result = def.method().invoke(def.bean(), messages); - events = switch (result) { - case null -> List.of(); - case List list -> (List) list; - case VehicleEvent e -> List.of(e); - default -> throw new IllegalStateException( - "@AsyncBatch method must return List: " + def.method()); - }; - } catch (InvocationTargetException e) { - log.error("batcher invoke failed method={}", def.method().getName(), e.getCause()); - return List.of(); - } catch (IllegalAccessException e) { - log.error("batcher access denied method={}", def.method().getName(), e); - return List.of(); - } - return events; - } - - private void publishEvents(List events, UnaryOperator eventTransformer) { - for (VehicleEvent e : events) { - try { - publisher.accept(eventTransformer.apply(e)); - } catch (Exception ex) { - log.warn("batcher publish failed eventId={}", e.eventId(), ex); - } - } - } - - @Override - public void close() { - running = false; - for (Thread t : workers) t.interrupt(); - } - } - - private record BatchItem(Object message, - UnaryOperator eventTransformer, - boolean transformRequired) { - } -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/DisruptorEventBus.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/DisruptorEventBus.java deleted file mode 100644 index 8e084cd6..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/DisruptorEventBus.java +++ /dev/null @@ -1,136 +0,0 @@ -package com.lingniu.ingest.core.concurrency; - -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.sink.EventSink; -import com.lmax.disruptor.BlockingWaitStrategy; -import com.lmax.disruptor.BusySpinWaitStrategy; -import com.lmax.disruptor.EventHandler; -import com.lmax.disruptor.SleepingWaitStrategy; -import com.lmax.disruptor.WaitStrategy; -import com.lmax.disruptor.YieldingWaitStrategy; -import com.lmax.disruptor.dsl.Disruptor; -import com.lmax.disruptor.dsl.ProducerType; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.util.ArrayList; -import java.util.List; -import java.util.Objects; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.ExecutorService; -import java.util.concurrent.Executors; -import java.util.concurrent.ThreadFactory; -import java.util.concurrent.atomic.AtomicLong; - -/** - * 基于 Disruptor 的事件总线：Dispatcher 投递事件 → RingBuffer → Sink 扇出。 - * - *

使用虚拟线程作为 Sink 消费线程，避免阻塞 IO 占用平台线程。 - * 单 VIN 有序性由上游 Netty + 分区投递保证，RingBuffer 不做 per-vin 排序。 - */ -public final class DisruptorEventBus implements AutoCloseable { - - private static final Logger log = LoggerFactory.getLogger(DisruptorEventBus.class); - - private final Disruptor disruptor; - private final List sinks; - private final ExecutorService awaitExecutor; - private final AtomicLong published = new AtomicLong(); - private final AtomicLong failed = new AtomicLong(); - - public DisruptorEventBus(int ringBufferSize, String waitStrategyName, List sinks) { - this.sinks = List.copyOf(sinks); - this.awaitExecutor = Executors.newVirtualThreadPerTaskExecutor(); - ThreadFactory tf = Thread.ofVirtual().name("ingest-bus-", 0).factory(); - this.disruptor = new Disruptor<>( - VehicleEventSlot::new, - ringBufferSize, - tf, - ProducerType.MULTI, - waitStrategy(waitStrategyName)); - - EventHandler[] handlers = this.sinks.stream() - .map(this::toHandler) - .toArray(EventHandler[]::new); - // 每个 sink 一个独立 handler，事件按扇出模式同时写 Kafka、archive、历史索引等目标。 - disruptor.handleEventsWith(handlers); - disruptor.start(); - log.info("DisruptorEventBus started ringBuffer={} wait={} sinks={}", - ringBufferSize, waitStrategyName, this.sinks.size()); - } - - public void publish(VehicleEvent event) { - disruptor.getRingBuffer().publishEvent((slot, seq, e) -> slot.event = e, event); - published.incrementAndGet(); - } - - public CompletableFuture publishAndAwait(VehicleEvent event, String requiredSinkName) { - String requiredSink = Objects.requireNonNull(requiredSinkName, "requiredSinkName"); - published.incrementAndGet(); - List acceptingSinks = sinks.stream() - .filter(sink -> sink.accepts(event)) - .toList(); - List requiredSinks = acceptingSinks.stream() - .filter(sink -> requiredSink.equals(sink.name())) - .toList(); - if (requiredSinks.isEmpty()) { - return CompletableFuture.failedFuture(new IllegalStateException( - "required sink '" + requiredSink + "' did not accept event " + event.eventId())); - } - - List> requiredFutures = new ArrayList<>(); - for (EventSink sink : acceptingSinks) { - CompletableFuture future = CompletableFuture - .supplyAsync(() -> publishToSinkAndTrack(sink, event), awaitExecutor) - .thenCompose(f -> f); - if (requiredSink.equals(sink.name())) { - requiredFutures.add(future); - } - } - return CompletableFuture.allOf(requiredFutures.toArray(CompletableFuture[]::new)); - } - - @Override - public void close() { - disruptor.shutdown(); - awaitExecutor.shutdown(); - log.info("DisruptorEventBus stopped published={} failed={}", published.get(), failed.get()); - } - - private EventHandler toHandler(EventSink sink) { - return (slot, seq, endOfBatch) -> { - VehicleEvent e = slot.event; - if (e == null || !sink.accepts(e)) return; - try { - publishToSinkAndTrack(sink, e); - } catch (Throwable t) { - failed.incrementAndGet(); - log.warn("sink {} publish failed", sink.name(), t); - } - }; - } - - private CompletableFuture publishToSinkAndTrack(EventSink sink, VehicleEvent event) { - try { - return sink.publish(event).whenComplete((ignored, ex) -> { - if (ex == null) return; - // sink 异步失败只计数和打日志；await 调用方仍会收到异常并决定是否 ACK。 - failed.incrementAndGet(); - log.warn("sink {} publish failed", sink.name(), ex); - }); - } catch (Throwable t) { - failed.incrementAndGet(); - log.warn("sink {} publish failed", sink.name(), t); - return CompletableFuture.failedFuture(t); - } - } - - private static WaitStrategy waitStrategy(String name) { - return switch (name == null ? "yielding" : name.toLowerCase()) { - case "blocking" -> new BlockingWaitStrategy(); - case "sleeping" -> new SleepingWaitStrategy(); - case "busy-spin" -> new BusySpinWaitStrategy(); - default -> new YieldingWaitStrategy(); - }; - } -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/VehicleEventSlot.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/VehicleEventSlot.java deleted file mode 100644 index 5a3b0ec2..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/concurrency/VehicleEventSlot.java +++ /dev/null @@ -1,12 +0,0 @@ -package com.lingniu.ingest.core.concurrency; - -import com.lingniu.ingest.api.event.VehicleEvent; - -/** Disruptor RingBuffer 槽位：持有可变引用，避免每次发布都分配新对象。 */ -public final class VehicleEventSlot { - public VehicleEvent event; - - public void clear() { - this.event = null; - } -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/config/IngestCoreAutoConfiguration.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/config/IngestCoreAutoConfiguration.java deleted file mode 100644 index debe4687..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/config/IngestCoreAutoConfiguration.java +++ /dev/null @@ -1,92 +0,0 @@ -package com.lingniu.ingest.core.config; - -import com.lingniu.ingest.api.pipeline.IngestInterceptor; -import com.lingniu.ingest.api.sink.EventSink; -import com.lingniu.ingest.core.concurrency.AsyncBatchExecutor; -import com.lingniu.ingest.core.concurrency.DisruptorEventBus; -import com.lingniu.ingest.core.dispatcher.AnnotationHandlerBeanPostProcessor; -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.core.dispatcher.HandlerInvoker; -import com.lingniu.ingest.core.dispatcher.HandlerRegistry; -import com.lingniu.ingest.core.pipeline.InterceptorChain; -import com.lingniu.ingest.core.pipeline.builtin.DedupInterceptor; -import com.lingniu.ingest.core.pipeline.builtin.RateLimitInterceptor; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; - -import java.util.List; - -/** - * ingest-core 的自动装配入口。所有 Bean 都是 {@code @ConditionalOnMissingBean}，便于下游替换。 - * - *

生产链路顺序：入口收到 {@code RawFrame} → {@link InterceptorChain} 做准入 → - * {@link Dispatcher} 定位协议 Handler → Handler 产出 {@code VehicleEvent} → - * {@link DisruptorEventBus} 扇出到 Archive、Kafka、历史索引等 {@code EventSink}。 - */ -@AutoConfiguration -@EnableConfigurationProperties(IngestCoreProperties.class) -public class IngestCoreAutoConfiguration { - - @Bean - @ConditionalOnMissingBean - public HandlerRegistry handlerRegistry() { - return new HandlerRegistry(); - } - - @Bean - @ConditionalOnMissingBean - public HandlerInvoker handlerInvoker() { - return new HandlerInvoker(); - } - - @Bean - public static AnnotationHandlerBeanPostProcessor annotationHandlerBeanPostProcessor() { - return new AnnotationHandlerBeanPostProcessor(); - } - - @Bean - @ConditionalOnProperty(prefix = "lingniu.ingest.pipeline.dedup", name = "enabled", havingValue = "true", matchIfMissing = true) - public DedupInterceptor dedupInterceptor(IngestCoreProperties props) { - // 去重尽量放在限流前，重复包不应消耗后续每 VIN QPS 配额。 - return new DedupInterceptor(props.getDedup().getCacheSize(), props.getDedup().getTtlSeconds()); - } - - @Bean - public RateLimitInterceptor rateLimitInterceptor(IngestCoreProperties props) { - return new RateLimitInterceptor(props.getRateLimit().getPerVinQps(), props.getRateLimit().getMaxVins()); - } - - @Bean - @ConditionalOnMissingBean - public InterceptorChain interceptorChain(List interceptors) { - return new InterceptorChain(interceptors); - } - - @Bean(destroyMethod = "close") - @ConditionalOnMissingBean - public DisruptorEventBus disruptorEventBus(IngestCoreProperties props, List sinks) { - return new DisruptorEventBus( - props.getDisruptor().getRingBufferSize(), - props.getDisruptor().getWaitStrategy(), - sinks); - } - - @Bean(destroyMethod = "close") - @ConditionalOnMissingBean - public AsyncBatchExecutor asyncBatchExecutor(DisruptorEventBus eventBus) { - return new AsyncBatchExecutor(eventBus::publish); - } - - @Bean - @ConditionalOnMissingBean - public Dispatcher dispatcher(HandlerRegistry registry, - InterceptorChain chain, - HandlerInvoker invoker, - DisruptorEventBus eventBus, - AsyncBatchExecutor batchExecutor) { - return new Dispatcher(registry, chain, invoker, eventBus, batchExecutor); - } -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/config/IngestCoreProperties.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/config/IngestCoreProperties.java deleted file mode 100644 index dfec3e99..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/config/IngestCoreProperties.java +++ /dev/null @@ -1,62 +0,0 @@ -package com.lingniu.ingest.core.config; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -@ConfigurationProperties(prefix = "lingniu.ingest.pipeline") -public class IngestCoreProperties { - - private Disruptor disruptor = new Disruptor(); - private Dedup dedup = new Dedup(); - private RateLimit rateLimit = new RateLimit(); - - public Disruptor getDisruptor() { return disruptor; } - public void setDisruptor(Disruptor disruptor) { this.disruptor = disruptor; } - - public Dedup getDedup() { return dedup; } - public void setDedup(Dedup dedup) { this.dedup = dedup; } - - public RateLimit getRateLimit() { return rateLimit; } - public void setRateLimit(RateLimit rateLimit) { this.rateLimit = rateLimit; } - - public static class Disruptor { - /** 事件总线 ring buffer 大小；生产环境应保持 2 的幂，避免 Disruptor 初始化失败。 */ - private int ringBufferSize = 131072; - /** waiting 策略直接影响延迟和 CPU 占用，默认 yielding 偏低延迟。 */ - private String waitStrategy = "yielding"; - private String producerType = "multi"; - - public int getRingBufferSize() { return ringBufferSize; } - public void setRingBufferSize(int ringBufferSize) { this.ringBufferSize = ringBufferSize; } - public String getWaitStrategy() { return waitStrategy; } - public void setWaitStrategy(String waitStrategy) { this.waitStrategy = waitStrategy; } - public String getProducerType() { return producerType; } - public void setProducerType(String producerType) { this.producerType = producerType; } - } - - public static class Dedup { - private boolean enabled = true; - /** 单实例缓存的幂等键数量上限，按 VIN 数和重发窗口估算。 */ - private int cacheSize = 200000; - /** 幂等键保留窗口；过短会放过重发包，过长会增加内存压力。 */ - private long ttlSeconds = 600; - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public int getCacheSize() { return cacheSize; } - public void setCacheSize(int cacheSize) { this.cacheSize = cacheSize; } - public long getTtlSeconds() { return ttlSeconds; } - public void setTtlSeconds(long ttlSeconds) { this.ttlSeconds = ttlSeconds; } - } - - public static class RateLimit { - /** 每 VIN 每秒进入 Handler 的最大帧数，默认按 32960 高频实时上报预留。 */ - private int perVinQps = 50; - /** VIN limiter 缓存上限；覆盖活跃车辆数即可，淘汰后会重新创建 limiter。 */ - private int maxVins = 100000; - - public int getPerVinQps() { return perVinQps; } - public void setPerVinQps(int perVinQps) { this.perVinQps = perVinQps; } - public int getMaxVins() { return maxVins; } - public void setMaxVins(int maxVins) { this.maxVins = maxVins; } - } -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/AnnotationHandlerBeanPostProcessor.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/AnnotationHandlerBeanPostProcessor.java deleted file mode 100644 index 0812558e..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/AnnotationHandlerBeanPostProcessor.java +++ /dev/null @@ -1,86 +0,0 @@ -package com.lingniu.ingest.core.dispatcher; - -import com.lingniu.ingest.api.annotation.AsyncBatch; -import com.lingniu.ingest.api.annotation.IdempotentKey; -import com.lingniu.ingest.api.annotation.MessageMapping; -import com.lingniu.ingest.api.annotation.ProtocolHandler; -import com.lingniu.ingest.api.annotation.RateLimited; -import org.springframework.beans.BeansException; -import org.springframework.beans.factory.BeanFactory; -import org.springframework.beans.factory.BeanFactoryAware; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.beans.factory.config.BeanPostProcessor; -import org.springframework.core.annotation.AnnotatedElementUtils; - -import java.lang.reflect.Method; - -/** - * Spring BeanPostProcessor：扫描带 {@link ProtocolHandler} 注解的 Bean， - * 把每个带 {@link MessageMapping} 的方法注册到 {@link HandlerRegistry}。 - * - *

替代旧代码里遍布的 {@code if (msgId == 0x0100) ... else if (msgId == 0x0102) ...} 风格。 - */ -public class AnnotationHandlerBeanPostProcessor implements BeanPostProcessor, BeanFactoryAware { - - private static final Logger log = LoggerFactory.getLogger(AnnotationHandlerBeanPostProcessor.class); - - private HandlerRegistry registry; - private BeanFactory beanFactory; - - public AnnotationHandlerBeanPostProcessor() { - } - - public AnnotationHandlerBeanPostProcessor(HandlerRegistry registry) { - this.registry = registry; - } - - @Override - public void setBeanFactory(BeanFactory beanFactory) throws BeansException { - this.beanFactory = beanFactory; - } - - @Override - public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException { - Class type = bean.getClass(); - ProtocolHandler classAnno = AnnotatedElementUtils.findMergedAnnotation(type, ProtocolHandler.class); - if (classAnno == null) return bean; - - for (Method method : type.getMethods()) { - MessageMapping mapping = AnnotatedElementUtils.findMergedAnnotation(method, MessageMapping.class); - if (mapping == null) continue; - - Class paramType = method.getParameterCount() > 0 ? method.getParameterTypes()[0] : Object.class; - int[] commands = mapping.command().length == 0 ? new int[]{0} : mapping.command(); - int[] infoTypes = mapping.infoType().length == 0 ? new int[]{0} : mapping.infoType(); - - // 这些注解只声明策略，实际限流、幂等、异步批处理由 Dispatcher/Interceptor 层解释执行。 - RateLimited rl = AnnotatedElementUtils.findMergedAnnotation(method, RateLimited.class); - IdempotentKey ik = AnnotatedElementUtils.findMergedAnnotation(method, IdempotentKey.class); - AsyncBatch ab = AnnotatedElementUtils.findMergedAnnotation(method, AsyncBatch.class); - - for (int cmd : commands) { - for (int info : infoTypes) { - HandlerDefinition def = new HandlerDefinition( - classAnno.protocol(), cmd, info, mapping.desc(), - bean, method, paramType, rl, ik, ab); - registry().register(def); - log.info("registered handler {} protocol={} command=0x{} info=0x{}", - type.getSimpleName() + "#" + method.getName(), - classAnno.protocol(), Integer.toHexString(cmd), Integer.toHexString(info)); - } - } - } - return bean; - } - - private HandlerRegistry registry() { - if (registry == null) { - if (beanFactory == null) { - throw new IllegalStateException("BeanFactory must be set before registering protocol handlers"); - } - registry = beanFactory.getBean(HandlerRegistry.class); - } - return registry; - } -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/Dispatcher.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/Dispatcher.java deleted file mode 100644 index e066ee67..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/Dispatcher.java +++ /dev/null @@ -1,304 +0,0 @@ -package com.lingniu.ingest.core.dispatcher; - -import com.fasterxml.jackson.core.JsonProcessingException; -import com.fasterxml.jackson.databind.ObjectMapper; -import com.fasterxml.jackson.databind.json.JsonMapper; -import com.lingniu.ingest.api.event.RawArchiveKeys; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.core.concurrency.AsyncBatchExecutor; -import com.lingniu.ingest.core.concurrency.DisruptorEventBus; -import com.lingniu.ingest.core.pipeline.InterceptorChain; -import com.lingniu.ingest.facts.RawFrameIdentity; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.time.Duration; -import java.time.Instant; -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; -import java.util.UUID; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionException; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicLong; - -/** - * 核心分发器：RawFrame → 拦截链 → Handler → 事件 → Disruptor EventBus。 - * - *

所有 Inbound Adapter（Netty / MQTT / PushClient）都调用 {@link #dispatch(RawFrame)}， - * 协议差异在这里被彻底抹平。 - * - *

对于带 {@code @AsyncBatch} 的 Handler，Dispatcher 不直接反射调用，而是把消息交给 - * {@link AsyncBatchExecutor} 进行聚合 → 批量调用 → 异步发布事件。 - */ -public final class Dispatcher { - - private static final Logger log = LoggerFactory.getLogger(Dispatcher.class); - private static final AtomicLong RAW_ARCHIVE_SEQUENCE = new AtomicLong(); - private static final String REQUIRED_DURABLE_SINK = "kafka"; - private static final ObjectMapper RAW_ARCHIVE_JSON = JsonMapper.builder() - .findAndAddModules() - .build(); - - private final HandlerRegistry registry; - private final InterceptorChain interceptors; - private final HandlerInvoker invoker; - private final DisruptorEventBus eventBus; - private final AsyncBatchExecutor batchExecutor; - - public Dispatcher(HandlerRegistry registry, - InterceptorChain interceptors, - HandlerInvoker invoker, - DisruptorEventBus eventBus, - AsyncBatchExecutor batchExecutor) { - this.registry = registry; - this.interceptors = interceptors; - this.invoker = invoker; - this.eventBus = eventBus; - this.batchExecutor = batchExecutor; - } - - public void dispatch(RawFrame frame) { - IngestContext ctx = new IngestContext(UUID.randomUUID().toString()); - try { - DispatchPlan plan = mapFrameToEvents(frame, ctx, false); - for (VehicleEvent e : plan.events()) { - eventBus.publish(e); - } - if (plan.failure() != null) { - throw plan.failure(); - } - } catch (Throwable t) { - log.error("dispatch failure traceId={}", ctx.traceId(), t); - interceptors.onError(t, ctx); - } - } - - public CompletableFuture dispatchAndAwait(RawFrame frame) { - IngestContext ctx = new IngestContext(UUID.randomUUID().toString()); - try { - DispatchPlan plan = mapFrameToEvents(frame, ctx, true); - if (plan.events().isEmpty() && plan.failure() == null) { - return CompletableFuture.failedFuture(new IllegalStateException( - "awaited dispatch produced no events; required sink '" + REQUIRED_DURABLE_SINK + "' was not reached")); - } - CompletableFuture[] futures = plan.events().stream() - .map(event -> eventBus.publishAndAwait(event, REQUIRED_DURABLE_SINK)) - .toArray(CompletableFuture[]::new); - CompletableFuture boundary = CompletableFuture.allOf(futures); - if (plan.failure() == null) { - return boundary; - } - return boundary.handle((ignored, publishFailure) -> { - log.error("dispatch failure traceId={}", ctx.traceId(), plan.failure()); - interceptors.onError(plan.failure(), ctx); - if (publishFailure != null) { - throw new CompletionException(publishFailure); - } - throw new CompletionException(plan.failure()); - }); - } catch (Throwable t) { - log.error("dispatch failure traceId={}", ctx.traceId(), t); - interceptors.onError(t, ctx); - return CompletableFuture.failedFuture(t); - } - } - - public CompletableFuture dispatchAndAwait(RawFrame frame, Duration timeout) { - CompletableFuture future = dispatchAndAwait(frame); - if (timeout == null || timeout.isZero() || timeout.isNegative()) { - return future; - } - return future.orTimeout(timeout.toMillis(), TimeUnit.MILLISECONDS); - } - - private DispatchPlan mapFrameToEvents(RawFrame frame, IngestContext ctx, boolean awaitDurability) { - List out = new ArrayList<>(); - // 在 interceptor 之前发 RawArchive，保证原始字节被无条件落盘（dedup/rate-limit - // 不会过滤它），满足"原始可回放"目标。archive 的写盘靠下游 Sink 消费。 - RawArchiveLookup rawArchive = appendRawArchive(frame, ctx, out); - - try { - if (!interceptors.before(frame, ctx)) { - log.debug("frame aborted: {}", ctx.abortReason()); - return new DispatchPlan(out, null); - } - - List handlers = registry.resolve( - frame.protocolId(), frame.command(), frame.infoType()); - if (handlers.isEmpty()) { - log.debug("no handler for {} cmd=0x{} info=0x{}", - frame.protocolId(), Integer.toHexString(frame.command()), - Integer.toHexString(frame.infoType())); - return new DispatchPlan(out, null); - } - - for (HandlerDefinition def : handlers) { - if (def.asyncBatch() != null) { - if (awaitDurability) { - throw new AwaitedAsyncBatchUnsupportedException( - "@AsyncBatch handler cannot be used with dispatchAndAwait until batch completion is awaited: " - + def.method()); - } - batchExecutor.submit(def, frame.payload(), event -> enrichWithRawArchive(event, rawArchive)); - continue; - } - List events = invoker.invoke(def, frame, ctx); - for (VehicleEvent e : events) { - e = enrichWithRawArchive(e, rawArchive); - interceptors.after(e, ctx); - out.add(e); - } - } - return new DispatchPlan(out, null); - } catch (Throwable t) { - if (t instanceof AwaitedAsyncBatchUnsupportedException) { - return new DispatchPlan(List.of(), t); - } - return new DispatchPlan(out, t); - } - } - - /** - * 从 {@link RawFrame} 构造一条 {@link VehicleEvent.RawArchive} 发到 EventBus。 - * 仅在 {@code rawBytes} 非空时发——有些入站适配器（未来）可能只传解析后的对象。 - * - *

VIN 取自 sourceMeta 里的 {@code vin} key（由各入站适配器负责填充）；缺失时 - * 留空字符串，archive sink 会用 "unknown-vin" 占位保证 key 可解析。 - */ - private RawArchiveLookup appendRawArchive(RawFrame frame, IngestContext ctx, List out) { - byte[] bytes = frame.rawBytes(); - if (bytes == null || bytes.length == 0) return RawArchiveLookup.empty(); - - Map meta = frame.sourceMeta() == null - ? Map.of() - : new LinkedHashMap<>(frame.sourceMeta()); - String parsedJson = parsedJson(frame.payload(), meta.remove(RawArchiveKeys.META_PARSED_JSON)); - String vin = meta.getOrDefault("vin", ""); - String phone = meta.getOrDefault("phone", ""); - Instant ingestTime = frame.receivedAt() != null ? frame.receivedAt() : Instant.now(); - String eventId = nextRawArchiveEventId(ingestTime); - String key = RawArchiveKeys.key(ingestTime, frame.protocolId(), vin, eventId); - RawFrameIdentity identity = RawFrameIdentity.derive( - frame.protocolId(), vin, phone, eventId, frame.command(), frame.infoType(), ingestTime, bytes); - Map archiveMeta = addRawArchiveMetadata(meta, eventId, key, identity.frameId()); - - VehicleEvent.RawArchive raw = new VehicleEvent.RawArchive( - eventId, - vin, - frame.protocolId(), - ingestTime, - ingestTime, - ctx.traceId(), - archiveMeta, - frame.command(), - frame.infoType(), - bytes, - parsedJson); - out.add(raw); - return new RawArchiveLookup(eventId, key, RawArchiveKeys.logicalUri(key), identity.frameId()); - } - - private static String parsedJson(Object payload, String explicitParsedJson) { - if (explicitParsedJson != null && !explicitParsedJson.isBlank()) { - return explicitParsedJson; - } - if (payload == null) { - return "{}"; - } - try { - return RAW_ARCHIVE_JSON.writeValueAsString(payload); - } catch (JsonProcessingException ex) { - return "{\"serializationError\":\"" + escapeJson(ex.getMessage()) + "\",\"payloadType\":\"" - + escapeJson(payload.getClass().getName()) + "\"}"; - } - } - - private static String escapeJson(String value) { - if (value == null) { - return ""; - } - return value.replace("\\", "\\\\").replace("\"", "\\\""); - } - - private static String nextRawArchiveEventId(Instant ingestTime) { - // 文件名需要可排序且单进程内唯一：毫秒时间放大到微秒量级，再用 AtomicLong 递增兜底。 - long base = (ingestTime == null ? Instant.now() : ingestTime).toEpochMilli() * 1000L; - long next = RAW_ARCHIVE_SEQUENCE.updateAndGet(previous -> Math.max(previous + 1, base)); - return Long.toString(next); - } - - private static Map addRawArchiveMetadata(Map metadata, - String eventId, - String key, - String frameId) { - Map out = new LinkedHashMap<>(metadata == null ? Map.of() : metadata); - out.putIfAbsent(RawArchiveKeys.META_EVENT_ID, eventId); - out.putIfAbsent(RawArchiveKeys.META_KEY, key); - out.putIfAbsent(RawArchiveKeys.META_URI, RawArchiveKeys.logicalUri(key)); - if (frameId != null && !frameId.isBlank()) { - out.putIfAbsent("rawFrameId", frameId); - out.putIfAbsent("frame_id", frameId); - } - return Map.copyOf(out); - } - - private static VehicleEvent enrichWithRawArchive(VehicleEvent event, RawArchiveLookup rawArchive) { - if (event == null || rawArchive.isEmpty() || event instanceof VehicleEvent.RawArchive) { - return event; - } - // 派生事件携带 rawArchiveUri，便于后续服务从 Kafka 事件回查原始报文。 - Map metadata = addRawArchiveMetadata( - event.metadata(), rawArchive.eventId(), rawArchive.key(), rawArchive.frameId()); - return switch (event) { - case VehicleEvent.Realtime e -> new VehicleEvent.Realtime( - e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(), - e.traceId(), metadata, e.payload()); - case VehicleEvent.Location e -> new VehicleEvent.Location( - e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(), - e.traceId(), metadata, e.payload()); - case VehicleEvent.Alarm e -> new VehicleEvent.Alarm( - e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(), - e.traceId(), metadata, e.payload()); - case VehicleEvent.Login e -> new VehicleEvent.Login( - e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(), - e.traceId(), metadata, e.iccid(), e.protocolVersion()); - case VehicleEvent.Logout e -> new VehicleEvent.Logout( - e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(), - e.traceId(), metadata); - case VehicleEvent.Heartbeat e -> new VehicleEvent.Heartbeat( - e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(), - e.traceId(), metadata); - case VehicleEvent.MediaMeta e -> new VehicleEvent.MediaMeta( - e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(), - e.traceId(), metadata, e.mediaId(), e.mediaType(), e.sizeBytes(), e.archiveRef()); - case VehicleEvent.Passthrough e -> new VehicleEvent.Passthrough( - e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(), - e.traceId(), metadata, e.passthroughType(), e.data()); - case VehicleEvent.RawArchive e -> e; - }; - } - - private record RawArchiveLookup(String eventId, String key, String uri, String frameId) { - private static RawArchiveLookup empty() { - return new RawArchiveLookup("", "", "", ""); - } - - private boolean isEmpty() { - return key == null || key.isBlank(); - } - } - - private record DispatchPlan(List events, Throwable failure) { - } - - private static final class AwaitedAsyncBatchUnsupportedException extends IllegalStateException { - private AwaitedAsyncBatchUnsupportedException(String message) { - super(message); - } - } -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/HandlerDefinition.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/HandlerDefinition.java deleted file mode 100644 index 71bd179a..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/HandlerDefinition.java +++ /dev/null @@ -1,31 +0,0 @@ -package com.lingniu.ingest.core.dispatcher; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.annotation.AsyncBatch; -import com.lingniu.ingest.api.annotation.IdempotentKey; -import com.lingniu.ingest.api.annotation.RateLimited; - -import java.lang.reflect.Method; - -/** - * 一个被注解扫描到的 Handler 方法的静态描述。不可变。 - */ -public record HandlerDefinition( - ProtocolId protocol, - int command, - int infoType, - String desc, - Object bean, - Method method, - Class parameterType, - RateLimited rateLimited, - IdempotentKey idempotentKey, - AsyncBatch asyncBatch -) { - public boolean matches(ProtocolId p, int cmd, int info) { - if (p != protocol) return false; - if (command != 0 && command != cmd) return false; - if (infoType != 0 && infoType != info) return false; - return true; - } -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/HandlerInvoker.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/HandlerInvoker.java deleted file mode 100644 index f6ec962c..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/HandlerInvoker.java +++ /dev/null @@ -1,34 +0,0 @@ -package com.lingniu.ingest.core.dispatcher; - -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.RawFrame; - -import java.lang.reflect.InvocationTargetException; -import java.util.Collections; -import java.util.List; - -/** - * 反射调用 Handler。单独抽出是为了后续可插拔：未来可替换为 MethodHandle 或字节码生成。 - */ -public class HandlerInvoker { - - @SuppressWarnings("unchecked") - public List invoke(HandlerDefinition def, RawFrame frame, IngestContext ctx) { - try { - Object result = def.method().invoke(def.bean(), frame.payload()); - return switch (result) { - case null -> Collections.emptyList(); - case VehicleEvent e -> List.of(e); - // Handler 可以一次产出多个领域事件，例如 GB32960 0x02 同时产出 Realtime/Location/Alarm。 - case List list -> (List) list; - default -> throw new IllegalStateException( - "Handler return type must be VehicleEvent or List: " + def.method()); - }; - } catch (InvocationTargetException e) { - throw new RuntimeException("Handler threw exception: " + def.method(), e.getCause()); - } catch (IllegalAccessException e) { - throw new RuntimeException("Handler not accessible: " + def.method(), e); - } - } -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/HandlerRegistry.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/HandlerRegistry.java deleted file mode 100644 index 90a7d158..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/dispatcher/HandlerRegistry.java +++ /dev/null @@ -1,51 +0,0 @@ -package com.lingniu.ingest.core.dispatcher; - -import com.lingniu.ingest.api.ProtocolId; - -import java.util.ArrayList; -import java.util.Collections; -import java.util.List; -import java.util.Map; -import java.util.concurrent.ConcurrentHashMap; - -/** - * Handler 注册中心。按 {@code (ProtocolId, command)} 索引，O(1) 查找；同 key 可注册多个 Handler，由 - * {@link HandlerDefinition#matches} 进一步精确匹配 {@code infoType}。 - */ -public final class HandlerRegistry { - - private final Map> byRoute = new ConcurrentHashMap<>(); - private final List wildcardHandlers = Collections.synchronizedList(new ArrayList<>()); - - public void register(HandlerDefinition def) { - if (def.command() == 0) { - wildcardHandlers.add(def); - return; - } - byRoute.computeIfAbsent(new RoutingKey(def.protocol(), def.command()), - k -> Collections.synchronizedList(new ArrayList<>())).add(def); - } - - public List resolve(ProtocolId protocol, int command, int infoType) { - List exact = byRoute.get(new RoutingKey(protocol, command)); - List result = new ArrayList<>(); - if (exact != null) { - for (HandlerDefinition d : exact) { - if (d.matches(protocol, command, infoType)) result.add(d); - } - } - if (!result.isEmpty()) { - return result; - } - for (HandlerDefinition d : wildcardHandlers) { - if (d.matches(protocol, command, infoType)) result.add(d); - } - return result; - } - - public int size() { - return byRoute.values().stream().mapToInt(List::size).sum() + wildcardHandlers.size(); - } - - private record RoutingKey(ProtocolId protocol, int command) {} -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/pipeline/InterceptorChain.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/pipeline/InterceptorChain.java deleted file mode 100644 index f3109787..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/pipeline/InterceptorChain.java +++ /dev/null @@ -1,51 +0,0 @@ -package com.lingniu.ingest.core.pipeline; - -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.IngestInterceptor; -import com.lingniu.ingest.api.pipeline.RawFrame; -import org.springframework.core.annotation.AnnotationAwareOrderComparator; - -import java.util.ArrayList; -import java.util.List; - -/** - * 顺序执行的拦截器链，通过 Spring {@code @Order} 或 {@code Ordered} 排序。 - * - *

{@link #before(RawFrame, IngestContext)} 是接入链路的准入闸门：任何拦截器返回 - * {@code false} 或调用 {@link IngestContext#abort(String)} 都会停止后续协议解析/Handler 调用。 - * {@link #after(VehicleEvent, IngestContext)} 与 {@link #onError(Throwable, IngestContext)} - * 则保持广播语义，让指标、审计等横切逻辑都能观察到同一条处理结果。 - */ -public final class InterceptorChain { - - private final List interceptors; - - public InterceptorChain(List interceptors) { - List sorted = new ArrayList<>(interceptors); - AnnotationAwareOrderComparator.sort(sorted); - this.interceptors = List.copyOf(sorted); - } - - public boolean before(RawFrame frame, IngestContext ctx) { - for (IngestInterceptor i : interceptors) { - // before 阶段短路是有意的：去重、限流失败后不应继续消耗解析和存储资源。 - if (!i.before(frame, ctx) || ctx.aborted()) { - return false; - } - } - return true; - } - - public void after(VehicleEvent event, IngestContext ctx) { - for (IngestInterceptor i : interceptors) { - i.after(event, ctx); - } - } - - public void onError(Throwable error, IngestContext ctx) { - for (IngestInterceptor i : interceptors) { - i.onError(error, ctx); - } - } -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/pipeline/builtin/DedupInterceptor.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/pipeline/builtin/DedupInterceptor.java deleted file mode 100644 index 7c4f72f4..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/pipeline/builtin/DedupInterceptor.java +++ /dev/null @@ -1,80 +0,0 @@ -package com.lingniu.ingest.core.pipeline.builtin; - -import com.github.benmanes.caffeine.cache.Cache; -import com.github.benmanes.caffeine.cache.Caffeine; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.IngestInterceptor; -import com.lingniu.ingest.api.pipeline.RawFrame; -import org.springframework.core.Ordered; - -import java.util.Arrays; -import java.time.Duration; -import java.util.Map; - -/** - * 基于 Caffeine 的本地幂等去重。 - * - *

Key 构造：{@code protocolId + vehicle identity + command + seq/fingerprint}。vehicle identity - * 优先使用已解析 VIN；VIN 未解析时使用 phone/deviceId/terminalId/plate，避免 JT808 多终端在 - * {@code vin=unknown} 时同流水号互相去重。如果上游已经在 {@link RawFrame#sourceMeta()} 里带了 - * {@code seq}，优先用真实流水号；否则退化为 raw bytes fingerprint，避免设备重连或 TCP 重发导致 - * 同一帧重复进入 Archive/Kafka/历史索引。 - * - *

当前缓存是进程内的，只能保证单实例去重。32960 若做多副本水平扩容，需要把这个位置替换成 - * Redis/集中式幂等键，否则不同实例仍可能各自接收一次相同原始包。 - */ -public class DedupInterceptor implements IngestInterceptor, Ordered { - - private final Cache seen; - - public DedupInterceptor(int maxSize, long ttlSeconds) { - this.seen = Caffeine.newBuilder() - .maximumSize(maxSize) - .expireAfterWrite(Duration.ofSeconds(ttlSeconds)) - .build(); - } - - @Override - public boolean before(RawFrame frame, IngestContext ctx) { - Map meta = frame.sourceMeta() == null ? Map.of() : frame.sourceMeta(); - String identity = identityKey(frame, meta); - String seq = meta.get("seq"); - if (seq == null || seq.isBlank()) { - // 某些入口没有硬件流水号，使用原始字节哈希作为保底键，至少能挡住同连接内的重复帧。 - seq = rawFingerprint(frame); - } - String key = frame.protocolId() + ":" + identity + ":" + frame.command() + ":" + seq; - if (seen.asMap().putIfAbsent(key, Boolean.TRUE) != null) { - ctx.abort("duplicate:" + key); - return false; - } - return true; - } - - @Override - public int getOrder() { - return 100; - } - - private static String identityKey(RawFrame frame, Map meta) { - String vin = meta.getOrDefault("vin", "unknown").trim(); - if (!vin.isBlank() && !"unknown".equalsIgnoreCase(vin)) { - return "vin:" + vin; - } - for (String key : new String[]{"phone", "deviceId", "terminalId", "plate"}) { - String value = meta.get(key); - if (value != null && !value.isBlank()) { - return key + ":" + value.trim(); - } - } - return "raw:" + rawFingerprint(frame); - } - - private static String rawFingerprint(RawFrame frame) { - byte[] rawBytes = frame.rawBytes(); - if (rawBytes != null && rawBytes.length > 0) { - return "raw:" + Integer.toHexString(Arrays.hashCode(rawBytes)); - } - return "receivedAt:" + frame.receivedAt(); - } -} diff --git a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/pipeline/builtin/RateLimitInterceptor.java b/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/pipeline/builtin/RateLimitInterceptor.java deleted file mode 100644 index 3a7da48d..00000000 --- a/modules/core/ingest-core/src/main/java/com/lingniu/ingest/core/pipeline/builtin/RateLimitInterceptor.java +++ /dev/null @@ -1,81 +0,0 @@ -package com.lingniu.ingest.core.pipeline.builtin; - -import com.github.benmanes.caffeine.cache.Cache; -import com.github.benmanes.caffeine.cache.Caffeine; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.IngestInterceptor; -import com.lingniu.ingest.api.pipeline.RawFrame; -import io.github.resilience4j.ratelimiter.RateLimiter; -import io.github.resilience4j.ratelimiter.RateLimiterConfig; -import org.springframework.core.Ordered; - -import java.time.Duration; - -/** - * 单 VIN 速率限制。每个 VIN 一个独立的 Resilience4j RateLimiter，由 Caffeine 按 LRU 管理。 - * - *

这里限的是进入业务 Handler 前的原始帧，不区分命令类型；32960 生产侧如需允许登入/登出穿透， - * 应在此处扩展白名单或改为按命令配置，而不是在存储层丢弃。 - */ -public class RateLimitInterceptor implements IngestInterceptor, Ordered { - - private final Cache limiters; - private final RateLimiterConfig defaultConfig; - - public RateLimitInterceptor(int perVinQps, int maxVins) { - this.defaultConfig = RateLimiterConfig.custom() - .limitForPeriod(perVinQps) - .limitRefreshPeriod(Duration.ofSeconds(1)) - .timeoutDuration(Duration.ZERO) - .build(); - this.limiters = Caffeine.newBuilder() - .maximumSize(maxVins) - .expireAfterAccess(Duration.ofMinutes(10)) - .build(); - } - - @Override - public boolean before(RawFrame frame, IngestContext ctx) { - String limiterKey = limiterKey(frame); - // 车辆粒度隔离：VIN 可用时按 VIN；JT808 等未映射 VIN 的协议按 vehicleKey/phone 兜底， - // 避免所有 unknown 终端挤在同一个限流桶里。 - RateLimiter rl = limiters.get(limiterKey, k -> RateLimiter.of("vehicle-" + k, defaultConfig)); - if (!rl.acquirePermission()) { - ctx.abort("rate-limited:" + limiterKey); - return false; - } - return true; - } - - private static String limiterKey(RawFrame frame) { - String vin = meta(frame, "vin"); - if (isKnown(vin)) { - return "vin:" + vin; - } - String vehicleKey = meta(frame, "vehicleKey"); - if (isKnown(vehicleKey)) { - return "vehicleKey:" + vehicleKey; - } - String phone = meta(frame, "phone"); - if (isKnown(phone)) { - return "phone:" + phone; - } - return "unknown"; - } - - private static String meta(RawFrame frame, String key) { - if (frame == null || frame.sourceMeta() == null) { - return ""; - } - return frame.sourceMeta().getOrDefault(key, "").trim(); - } - - private static boolean isKnown(String value) { - return value != null && !value.isBlank() && !"unknown".equalsIgnoreCase(value.trim()); - } - - @Override - public int getOrder() { - return 200; - } -} diff --git a/modules/core/ingest-core/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/core/ingest-core/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index c5adbd90..00000000 --- a/modules/core/ingest-core/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.core.config.IngestCoreAutoConfiguration diff --git a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/concurrency/DisruptorEventBusAwaitTest.java b/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/concurrency/DisruptorEventBusAwaitTest.java deleted file mode 100644 index fbbaea9e..00000000 --- a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/concurrency/DisruptorEventBusAwaitTest.java +++ /dev/null @@ -1,127 +0,0 @@ -package com.lingniu.ingest.core.concurrency; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.sink.EventSink; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.List; -import java.util.Map; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionException; -import java.util.concurrent.CountDownLatch; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicLong; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class DisruptorEventBusAwaitTest { - - @Test - void publishAndAwaitDoesNotInvokeSinkPublishOnCallerThread() throws Exception { - long callerThreadId = Thread.currentThread().threadId(); - CapturingSink sink = new CapturingSink("kafka"); - - try (DisruptorEventBus bus = new DisruptorEventBus(1024, "blocking", java.util.List.of(sink))) { - bus.publishAndAwait(event(), "kafka").get(3, TimeUnit.SECONDS); - } - - assertThat(sink.publishThreadId.get()).isNotEqualTo(callerThreadId); - } - - @Test - void publishAndAwaitDoesNotInvokeOptionalSinksWhenRequiredSinkIsAbsent() throws Exception { - CapturingSink optionalSink = new CapturingSink("raw-archive"); - - try (DisruptorEventBus bus = new DisruptorEventBus(1024, "blocking", List.of(optionalSink))) { - assertThatThrownBy(() -> bus.publishAndAwait(event(), "kafka").join()) - .isInstanceOf(CompletionException.class) - .hasRootCauseInstanceOf(IllegalStateException.class) - .hasMessageContaining("kafka"); - - assertThat(optionalSink.published.await(200, TimeUnit.MILLISECONDS)).isFalse(); - } - } - - @Test - void publishFansOutEveryEventToEveryAcceptingSink() throws Exception { - int count = 1000; - CountingSink rawArchive = new CountingSink("raw-archive", count); - CountingSink kafka = new CountingSink("kafka", count); - - try (DisruptorEventBus bus = new DisruptorEventBus(1024, "blocking", List.of(rawArchive, kafka))) { - for (int i = 0; i < count; i++) { - bus.publish(event("heartbeat-" + i)); - } - - assertThat(rawArchive.received.await(3, TimeUnit.SECONDS)).isTrue(); - assertThat(kafka.received.await(3, TimeUnit.SECONDS)).isTrue(); - } - - assertThat(rawArchive.count.get()).isEqualTo(count); - assertThat(kafka.count.get()).isEqualTo(count); - } - - private static VehicleEvent.Heartbeat event() { - return event("heartbeat-1"); - } - - private static VehicleEvent.Heartbeat event(String eventId) { - Instant now = Instant.parse("2026-06-23T10:00:00Z"); - return new VehicleEvent.Heartbeat( - eventId, - "VIN001", - ProtocolId.GB32960, - now, - now, - "trace-1", - Map.of()); - } - - private static final class CapturingSink implements EventSink { - private final String name; - private final AtomicLong publishThreadId = new AtomicLong(-1); - private final CountDownLatch published = new CountDownLatch(1); - - private CapturingSink(String name) { - this.name = name; - } - - @Override - public String name() { - return name; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - publishThreadId.set(Thread.currentThread().threadId()); - published.countDown(); - return CompletableFuture.completedFuture(null); - } - } - - private static final class CountingSink implements EventSink { - private final String name; - private final AtomicLong count = new AtomicLong(); - private final CountDownLatch received; - - private CountingSink(String name, int expected) { - this.name = name; - this.received = new CountDownLatch(expected); - } - - @Override - public String name() { - return name; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - count.incrementAndGet(); - received.countDown(); - return CompletableFuture.completedFuture(null); - } - } -} diff --git a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/config/IngestCoreAutoConfigurationTest.java b/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/config/IngestCoreAutoConfigurationTest.java deleted file mode 100644 index 3bf4c9cb..00000000 --- a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/config/IngestCoreAutoConfigurationTest.java +++ /dev/null @@ -1,20 +0,0 @@ -package com.lingniu.ingest.core.config; - -import org.junit.jupiter.api.Test; - -import java.lang.reflect.Method; -import java.lang.reflect.Modifier; - -import static org.assertj.core.api.Assertions.assertThat; - -class IngestCoreAutoConfigurationTest { - - @Test - void beanPostProcessorFactoryMethodIsStaticToAvoidEarlyConfigurationInstantiation() throws Exception { - Method method = IngestCoreAutoConfiguration.class.getDeclaredMethod( - "annotationHandlerBeanPostProcessor"); - - assertThat(Modifier.isStatic(method.getModifiers())).isTrue(); - assertThat(method.getParameterCount()).isZero(); - } -} diff --git a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/dispatcher/DispatcherDurableAckBoundaryTest.java b/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/dispatcher/DispatcherDurableAckBoundaryTest.java deleted file mode 100644 index 0512c463..00000000 --- a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/dispatcher/DispatcherDurableAckBoundaryTest.java +++ /dev/null @@ -1,217 +0,0 @@ -package com.lingniu.ingest.core.dispatcher; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.annotation.AsyncBatch; -import com.lingniu.ingest.api.annotation.MessageMapping; -import com.lingniu.ingest.api.annotation.ProtocolHandler; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.IngestInterceptor; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.api.sink.EventSink; -import com.lingniu.ingest.core.concurrency.AsyncBatchExecutor; -import com.lingniu.ingest.core.concurrency.DisruptorEventBus; -import com.lingniu.ingest.core.pipeline.InterceptorChain; -import org.junit.jupiter.api.Test; - -import java.lang.reflect.Method; -import java.time.Instant; -import java.util.List; -import java.util.Map; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionException; -import java.util.concurrent.CompletionStage; -import java.util.concurrent.atomic.AtomicInteger; -import java.util.concurrent.atomic.AtomicReference; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class DispatcherDurableAckBoundaryTest { - - @Test - void dispatchAndAwaitFailsWhenNoKafkaSinkAcceptsEvent() throws Exception { - EventSink rawArchiveSink = new NamedSink("raw-archive", true); - - try (Harness harness = new Harness(registryWithHeartbeatHandler(), List.of(rawArchiveSink), List.of())) { - CompletionStage future = harness.dispatcher.dispatchAndAwait(frame(0x07, "payload")); - - assertThatThrownBy(() -> future.toCompletableFuture().join()) - .isInstanceOf(CompletionException.class) - .hasRootCauseInstanceOf(IllegalStateException.class) - .hasMessageContaining("kafka"); - } - } - - @Test - void dispatchAndAwaitNotifiesInterceptorWhenPlanCapturesFailure() throws Exception { - CapturingInterceptor interceptor = new CapturingInterceptor(); - - try (Harness harness = new Harness(registryWithThrowingHandler(), List.of(new NamedSink("kafka", true)), - List.of(interceptor))) { - CompletionStage future = harness.dispatcher.dispatchAndAwait(frame(0x08, "payload")); - - assertThatThrownBy(() -> future.toCompletableFuture().join()) - .isInstanceOf(CompletionException.class) - .hasRootCauseInstanceOf(IllegalStateException.class) - .rootCause() - .hasMessageContaining("handler failed"); - assertThat(interceptor.error.get()) - .isInstanceOf(RuntimeException.class) - .hasMessageContaining("Handler threw exception"); - } - } - - @Test - void dispatchAndAwaitFailsFastForAsyncBatchHandler() throws Exception { - NamedSink kafka = new NamedSink("kafka", true); - try (Harness harness = new Harness(registryWithAsyncBatchHandler(), List.of(kafka), List.of())) { - CompletionStage future = harness.dispatcher.dispatchAndAwait( - frame(0x09, "payload", new byte[]{0x23, 0x23, 0x09})); - - assertThatThrownBy(() -> future.toCompletableFuture().join()) - .isInstanceOf(CompletionException.class) - .hasRootCauseInstanceOf(IllegalStateException.class) - .hasMessageContaining("@AsyncBatch"); - assertThat(kafka.published.get()).isZero(); - } - } - - private static HandlerRegistry registryWithHeartbeatHandler() throws NoSuchMethodException { - HeartbeatHandler bean = new HeartbeatHandler(); - Method method = HeartbeatHandler.class.getDeclaredMethod("onHeartbeat", Object.class); - return registry(bean, method, 0x07, method.getAnnotation(AsyncBatch.class)); - } - - private static HandlerRegistry registryWithThrowingHandler() throws NoSuchMethodException { - ThrowingHandler bean = new ThrowingHandler(); - Method method = ThrowingHandler.class.getDeclaredMethod("onFrame", Object.class); - return registry(bean, method, 0x08, method.getAnnotation(AsyncBatch.class)); - } - - private static HandlerRegistry registryWithAsyncBatchHandler() throws NoSuchMethodException { - AsyncHandler bean = new AsyncHandler(); - Method method = AsyncHandler.class.getDeclaredMethod("onBatch", List.class); - return registry(bean, method, 0x09, method.getAnnotation(AsyncBatch.class)); - } - - private static HandlerRegistry registry(Object bean, Method method, int command, AsyncBatch asyncBatch) { - HandlerRegistry registry = new HandlerRegistry(); - registry.register(new HandlerDefinition( - ProtocolId.GB32960, - command, - 0, - method.getName(), - bean, - method, - Object.class, - null, - null, - asyncBatch)); - return registry; - } - - private static RawFrame frame(int command, Object payload) { - return frame(command, payload, null); - } - - private static RawFrame frame(int command, Object payload, byte[] rawBytes) { - return new RawFrame( - ProtocolId.GB32960, - command, - 0, - payload, - rawBytes, - Map.of("vin", "VIN001"), - Instant.parse("2026-06-23T10:00:00Z")); - } - - private static VehicleEvent.Heartbeat event(String id) { - Instant now = Instant.parse("2026-06-23T10:00:00Z"); - return new VehicleEvent.Heartbeat(id, "VIN001", ProtocolId.GB32960, now, now, "trace-" + id, Map.of()); - } - - private static final class Harness implements AutoCloseable { - private final DisruptorEventBus eventBus; - private final AsyncBatchExecutor batchExecutor; - private final Dispatcher dispatcher; - - private Harness(HandlerRegistry registry, List sinks, List interceptors) { - this.eventBus = new DisruptorEventBus(1024, "blocking", sinks); - this.batchExecutor = new AsyncBatchExecutor(eventBus::publish); - this.dispatcher = new Dispatcher( - registry, - new InterceptorChain(interceptors), - new HandlerInvoker(), - eventBus, - batchExecutor); - } - - @Override - public void close() { - batchExecutor.close(); - eventBus.close(); - } - } - - private static final class NamedSink implements EventSink { - private final String name; - private final boolean accepts; - private final AtomicInteger published = new AtomicInteger(); - - private NamedSink(String name, boolean accepts) { - this.name = name; - this.accepts = accepts; - } - - @Override - public String name() { - return name; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - published.incrementAndGet(); - return CompletableFuture.completedFuture(null); - } - - @Override - public boolean accepts(VehicleEvent event) { - return accepts; - } - } - - private static final class CapturingInterceptor implements IngestInterceptor { - private final AtomicReference error = new AtomicReference<>(); - - @Override - public void onError(Throwable error, IngestContext ctx) { - this.error.set(error); - } - } - - @ProtocolHandler(protocol = ProtocolId.GB32960) - static final class HeartbeatHandler { - @MessageMapping(command = 0x07) - VehicleEvent.Heartbeat onHeartbeat(Object ignored) { - return event("heartbeat-1"); - } - } - - @ProtocolHandler(protocol = ProtocolId.GB32960) - static final class ThrowingHandler { - @MessageMapping(command = 0x08) - VehicleEvent.Heartbeat onFrame(Object ignored) { - throw new IllegalStateException("handler failed"); - } - } - - @ProtocolHandler(protocol = ProtocolId.GB32960) - public static final class AsyncHandler { - @MessageMapping(command = 0x09) - @AsyncBatch(size = 2, waitMs = 10, poolSize = 1) - public List onBatch(List ignored) { - return List.of(event("async-1")); - } - } -} diff --git a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/dispatcher/DispatcherRawArchiveMetadataTest.java b/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/dispatcher/DispatcherRawArchiveMetadataTest.java deleted file mode 100644 index 1b446ccd..00000000 --- a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/dispatcher/DispatcherRawArchiveMetadataTest.java +++ /dev/null @@ -1,232 +0,0 @@ -package com.lingniu.ingest.core.dispatcher; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.annotation.AsyncBatch; -import com.lingniu.ingest.api.annotation.MessageMapping; -import com.lingniu.ingest.api.annotation.ProtocolHandler; -import com.lingniu.ingest.api.event.LocationPayload; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.api.sink.EventSink; -import com.lingniu.ingest.core.concurrency.AsyncBatchExecutor; -import com.lingniu.ingest.core.concurrency.DisruptorEventBus; -import com.lingniu.ingest.core.pipeline.InterceptorChain; -import org.junit.jupiter.api.Test; - -import java.lang.reflect.Method; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; -import java.util.concurrent.CompletableFuture; - -import static org.assertj.core.api.Assertions.assertThat; - -class DispatcherRawArchiveMetadataTest { - - @Test - void enrichesBusinessEventsWithRawArchiveLookupMetadata() throws Exception { - CollectingSink sink = new CollectingSink(); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - try { - Dispatcher dispatcher = new Dispatcher( - registryWithLocationHandler(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - - dispatcher.dispatch(new RawFrame( - ProtocolId.JT808, - 0x0200, - 0, - new SamplePayload("payload-0200", 7), - new byte[]{0x7e, 0x01, 0x7e}, - Map.of("vin", "VIN001", "peer", "127.0.0.1:10001"), - Instant.parse("2026-06-22T08:00:00Z"))); - - VehicleEvent.RawArchive raw = awaitEvent(sink, VehicleEvent.RawArchive.class); - VehicleEvent.Location location = awaitEvent(sink, VehicleEvent.Location.class); - - String rawArchiveKey = raw.metadata().get("rawArchiveKey"); - String rawFrameId = raw.metadata().get("rawFrameId"); - assertThat(rawArchiveKey).isNotBlank(); - assertThat(rawFrameId).startsWith("rf_"); - assertThat(raw.parsedJson()).contains("\"name\":\"payload-0200\""); - assertThat(raw.metadata()).doesNotContainKey("rawArchiveParsedJson"); - assertThat(raw.eventId()).containsOnlyDigits(); - assertThat(rawArchiveKey).endsWith("/" + raw.eventId() + ".bin"); - assertThat(raw.metadata()).containsEntry("frame_id", rawFrameId); - assertThat(location.metadata()) - .containsEntry("rawArchiveEventId", raw.eventId()) - .containsEntry("rawArchiveKey", rawArchiveKey) - .containsEntry("rawArchiveUri", "archive://" + rawArchiveKey) - .containsEntry("rawFrameId", rawFrameId) - .containsEntry("frame_id", rawFrameId); - } finally { - batchExecutor.close(); - eventBus.close(); - } - } - - @Test - void enrichesAsyncBatchEventsWithTheirRawArchiveLookupMetadata() throws Exception { - CollectingSink sink = new CollectingSink(); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - try { - Dispatcher dispatcher = new Dispatcher( - registryWithAsyncLocationHandler(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - - dispatcher.dispatch(new RawFrame( - ProtocolId.JT808, - 0x0201, - 0, - "payload-1", - new byte[]{0x7e, 0x02, 0x7e}, - Map.of("vin", "VIN002"), - Instant.parse("2026-06-22T08:01:00Z"))); - - VehicleEvent.RawArchive raw = awaitEvent(sink, VehicleEvent.RawArchive.class); - VehicleEvent.Location location = awaitLocationEvent(sink, "async-location-1"); - - String rawArchiveKey = raw.metadata().get("rawArchiveKey"); - String rawFrameId = raw.metadata().get("rawFrameId"); - assertThat(rawFrameId).startsWith("rf_"); - assertThat(location.metadata()) - .containsEntry("rawArchiveEventId", raw.eventId()) - .containsEntry("rawArchiveKey", rawArchiveKey) - .containsEntry("rawArchiveUri", "archive://" + rawArchiveKey) - .containsEntry("rawFrameId", rawFrameId) - .containsEntry("frame_id", rawFrameId); - } finally { - batchExecutor.close(); - eventBus.close(); - } - } - - private static HandlerRegistry registryWithLocationHandler() throws NoSuchMethodException { - LocationHandler bean = new LocationHandler(); - Method method = LocationHandler.class.getDeclaredMethod("onLocation", SamplePayload.class); - HandlerRegistry registry = new HandlerRegistry(); - registry.register(new HandlerDefinition( - ProtocolId.JT808, - 0x0200, - 0, - "test-location", - bean, - method, - Object.class, - null, - null, - null)); - return registry; - } - - private static HandlerRegistry registryWithAsyncLocationHandler() throws NoSuchMethodException { - AsyncLocationHandler bean = new AsyncLocationHandler(); - Method method = AsyncLocationHandler.class.getDeclaredMethod("onLocationBatch", List.class); - HandlerRegistry registry = new HandlerRegistry(); - registry.register(new HandlerDefinition( - ProtocolId.JT808, - 0x0201, - 0, - "test-async-location", - bean, - method, - Object.class, - null, - null, - method.getAnnotation(AsyncBatch.class))); - return registry; - } - - private static T awaitEvent(CollectingSink sink, Class type) throws InterruptedException { - long deadline = System.currentTimeMillis() + 3000; - while (System.currentTimeMillis() < deadline) { - synchronized (sink.events) { - for (VehicleEvent event : sink.events) { - if (type.isInstance(event)) { - return type.cast(event); - } - } - } - Thread.sleep(25); - } - throw new AssertionError("event not published: " + type.getSimpleName() + ", actual=" + sink.events); - } - - private static VehicleEvent.Location awaitLocationEvent(CollectingSink sink, String eventId) throws InterruptedException { - long deadline = System.currentTimeMillis() + 3000; - while (System.currentTimeMillis() < deadline) { - synchronized (sink.events) { - for (VehicleEvent event : sink.events) { - if (event instanceof VehicleEvent.Location location && location.eventId().equals(eventId)) { - return location; - } - } - } - Thread.sleep(25); - } - throw new AssertionError("location event not published: " + eventId + ", actual=" + sink.events); - } - - @ProtocolHandler(protocol = ProtocolId.JT808) - static final class LocationHandler { - @MessageMapping(command = 0x0200) - VehicleEvent.Location onLocation(SamplePayload ignored) { - return new VehicleEvent.Location( - "location-1", - "VIN001", - ProtocolId.JT808, - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "trace-location", - Map.of(), - new LocationPayload(113.12, 23.45, 10.0, 60.0, 90.0, 0, 1)); - } - } - - record SamplePayload(String name, int value) { - } - - @ProtocolHandler(protocol = ProtocolId.JT808) - public static final class AsyncLocationHandler { - @MessageMapping(command = 0x0201) - @AsyncBatch(size = 2, waitMs = 10, poolSize = 1) - public List onLocationBatch(List batch) { - return List.of(new VehicleEvent.Location( - "async-location-1", - "VIN002", - ProtocolId.JT808, - Instant.parse("2026-06-22T08:01:00Z"), - Instant.parse("2026-06-22T08:01:01Z"), - "trace-async-location", - Map.of(), - new LocationPayload(113.12, 23.45, 10.0, 60.0, 90.0, 0, 1))); - } - } - - private static final class CollectingSink implements EventSink { - private final List events = new ArrayList<>(); - - @Override - public String name() { - return "collecting"; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - synchronized (events) { - events.add(event); - } - return CompletableFuture.completedFuture(null); - } - } -} diff --git a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/dispatcher/HandlerRegistryTest.java b/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/dispatcher/HandlerRegistryTest.java deleted file mode 100644 index 1eaa8f72..00000000 --- a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/dispatcher/HandlerRegistryTest.java +++ /dev/null @@ -1,51 +0,0 @@ -package com.lingniu.ingest.core.dispatcher; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import java.lang.reflect.Method; - -import static org.assertj.core.api.Assertions.assertThat; - -class HandlerRegistryTest { - - @Test - void exactCommandMatchTakesPrecedenceOverWildcardHandlers() throws Exception { - HandlerRegistry registry = new HandlerRegistry(); - Object bean = new SampleHandlers(); - Method exact = SampleHandlers.class.getMethod("exact", Object.class); - Method wildcard = SampleHandlers.class.getMethod("wildcard", Object.class); - HandlerDefinition exactDef = definition(0x0200, "exact", bean, exact); - HandlerDefinition wildcardDef = definition(0, "wildcard", bean, wildcard); - - registry.register(exactDef); - registry.register(wildcardDef); - - assertThat(registry.resolve(ProtocolId.JT808, 0x0200, 0)) - .containsExactly(exactDef); - assertThat(registry.resolve(ProtocolId.JT808, 0x0500, 0)) - .containsExactly(wildcardDef); - } - - private static HandlerDefinition definition(int command, String desc, Object bean, Method method) { - return new HandlerDefinition( - ProtocolId.JT808, - command, - 0, - desc, - bean, - method, - Object.class, - null, - null, - null); - } - - public static final class SampleHandlers { - public void exact(Object ignored) { - } - - public void wildcard(Object ignored) { - } - } -} diff --git a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/pipeline/builtin/DedupInterceptorTest.java b/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/pipeline/builtin/DedupInterceptorTest.java deleted file mode 100644 index 85e33e23..00000000 --- a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/pipeline/builtin/DedupInterceptorTest.java +++ /dev/null @@ -1,66 +0,0 @@ -package com.lingniu.ingest.core.pipeline.builtin; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.RawFrame; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class DedupInterceptorTest { - - @Test - void usesRawFingerprintWhenSequenceIsMissing() { - DedupInterceptor interceptor = new DedupInterceptor(100, 60); - IngestContext ctx = new IngestContext("trace-1"); - - RawFrame first = frame(new byte[]{0x23, 0x23, 0x02, 0x01}); - RawFrame differentPayload = frame(new byte[]{0x23, 0x23, 0x02, 0x02}); - RawFrame duplicate = frame(new byte[]{0x23, 0x23, 0x02, 0x01}); - - assertThat(interceptor.before(first, ctx)).isTrue(); - assertThat(interceptor.before(differentPayload, ctx)).isTrue(); - assertThat(interceptor.before(duplicate, ctx)).isFalse(); - assertThat(ctx.aborted()).isTrue(); - } - - @Test - void doesNotDeduplicateDifferentJt808PhonesWhenVinIsUnknownAndSequenceMatches() { - DedupInterceptor interceptor = new DedupInterceptor(100, 60); - - RawFrame firstPhone = jt808Frame("13079960001", "1000"); - RawFrame secondPhone = jt808Frame("13079960002", "1000"); - RawFrame duplicateFirstPhone = jt808Frame("13079960001", "1000"); - - assertThat(interceptor.before(firstPhone, new IngestContext("trace-1"))).isTrue(); - assertThat(interceptor.before(secondPhone, new IngestContext("trace-2"))).isTrue(); - IngestContext duplicateContext = new IngestContext("trace-3"); - assertThat(interceptor.before(duplicateFirstPhone, duplicateContext)).isFalse(); - assertThat(duplicateContext.abortReason()).contains("duplicate"); - } - - private static RawFrame frame(byte[] rawBytes) { - return new RawFrame( - ProtocolId.GB32960, - 0x02, - 0, - new Object(), - rawBytes, - Map.of("vin", "TESTSTATVIN000001"), - Instant.parse("2026-06-22T13:00:00Z")); - } - - private static RawFrame jt808Frame(String phone, String seq) { - return new RawFrame( - ProtocolId.JT808, - 0x0200, - 0, - new Object(), - new byte[]{0x02, 0x00, 0x00, 0x1c}, - Map.of("vin", "unknown", "phone", phone, "seq", seq), - Instant.parse("2026-06-22T13:00:00Z")); - } -} diff --git a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/pipeline/builtin/RateLimitInterceptorTest.java b/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/pipeline/builtin/RateLimitInterceptorTest.java deleted file mode 100644 index 45859a3e..00000000 --- a/modules/core/ingest-core/src/test/java/com/lingniu/ingest/core/pipeline/builtin/RateLimitInterceptorTest.java +++ /dev/null @@ -1,55 +0,0 @@ -package com.lingniu.ingest.core.pipeline.builtin; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.RawFrame; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class RateLimitInterceptorTest { - - @Test - void unknownJt808VinUsesPhoneAsIndependentLimiterKey() { - RateLimitInterceptor interceptor = new RateLimitInterceptor(1, 100); - IngestContext first = new IngestContext("trace-1"); - IngestContext second = new IngestContext("trace-2"); - - boolean firstAllowed = interceptor.before(frame("unknown", "13079970001"), first); - boolean secondAllowed = interceptor.before(frame("unknown", "13079970002"), second); - - assertThat(firstAllowed).isTrue(); - assertThat(secondAllowed).isTrue(); - assertThat(first.aborted()).isFalse(); - assertThat(second.aborted()).isFalse(); - } - - @Test - void sameJt808PhoneStillSharesLimiterWhenVinIsUnknown() { - RateLimitInterceptor interceptor = new RateLimitInterceptor(1, 100); - IngestContext first = new IngestContext("trace-1"); - IngestContext second = new IngestContext("trace-2"); - - boolean firstAllowed = interceptor.before(frame("unknown", "13079970001"), first); - boolean secondAllowed = interceptor.before(frame("unknown", "13079970001"), second); - - assertThat(firstAllowed).isTrue(); - assertThat(secondAllowed).isFalse(); - assertThat(second.aborted()).isTrue(); - assertThat(second.abortReason()).isEqualTo("rate-limited:phone:13079970001"); - } - - private static RawFrame frame(String vin, String phone) { - return new RawFrame( - ProtocolId.JT808, - 0x0200, - 0, - new Object(), - new byte[]{0x01}, - Map.of("vin", vin, "phone", phone), - Instant.parse("2026-06-29T00:00:00Z")); - } -} diff --git a/modules/core/ingest-facts/pom.xml b/modules/core/ingest-facts/pom.xml deleted file mode 100644 index a8d6f220..00000000 --- a/modules/core/ingest-facts/pom.xml +++ /dev/null @@ -1,30 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - ingest-facts - ingest-facts - Protocol-neutral raw frame and decoded fact model for vehicle ingestion. - - - - com.lingniu.ingest - ingest-api - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - diff --git a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/DecodedFact.java b/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/DecodedFact.java deleted file mode 100644 index f155d039..00000000 --- a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/DecodedFact.java +++ /dev/null @@ -1,215 +0,0 @@ -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; - -import java.time.Instant; -import java.util.Map; - -public sealed interface DecodedFact - permits DecodedFact.Location, - DecodedFact.Realtime, - DecodedFact.Alarm, - DecodedFact.Session, - DecodedFact.Register, - DecodedFact.Extension { - - String factId(); - String frameId(); - ProtocolId protocol(); - String vehicleKey(); - String vin(); - String phone(); - Instant eventTime(); - Instant receivedAt(); - String rawUri(); - Map metadata(); - - record Location( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - Double longitude, - Double latitude, - Double altitudeM, - Double speedKmh, - Double directionDeg, - Long alarmFlag, - Long statusFlag, - Double totalMileageKm - ) implements DecodedFact { - public Location { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - metadata = BaseFields.metadata(metadata); - } - } - - record Realtime( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - Map fields - ) implements DecodedFact { - public Realtime { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - metadata = BaseFields.metadata(metadata); - fields = fields == null ? Map.of() : Map.copyOf(fields); - } - } - - record Alarm( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - String alarmLevel, - Long alarmCode, - String alarmName, - Map fields - ) implements DecodedFact { - public Alarm { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - alarmLevel = BaseFields.clean(alarmLevel); - alarmName = BaseFields.clean(alarmName); - metadata = BaseFields.metadata(metadata); - fields = fields == null ? Map.of() : Map.copyOf(fields); - } - } - - record Session( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - String sessionType, - Integer resultCode, - Map fields - ) implements DecodedFact { - public Session { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - sessionType = BaseFields.clean(sessionType); - metadata = BaseFields.metadata(metadata); - fields = fields == null ? Map.of() : Map.copyOf(fields); - } - } - - record Register( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - Map registrationFields - ) implements DecodedFact { - public Register { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - metadata = BaseFields.metadata(metadata); - registrationFields = registrationFields == null ? Map.of() : Map.copyOf(registrationFields); - } - } - - record Extension( - String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - Instant eventTime, - Instant receivedAt, - String rawUri, - Map metadata, - String extensionType, - Map fields - ) implements DecodedFact { - public Extension { - BaseFields.validate(factId, frameId, protocol, vehicleKey, eventTime, receivedAt, rawUri); - vin = BaseFields.clean(vin); - phone = BaseFields.clean(phone); - extensionType = BaseFields.clean(extensionType); - metadata = BaseFields.metadata(metadata); - fields = fields == null ? Map.of() : Map.copyOf(fields); - } - } - - final class BaseFields { - private BaseFields() { - } - - static void validate(String factId, - String frameId, - ProtocolId protocol, - String vehicleKey, - Instant eventTime, - Instant receivedAt, - String rawUri) { - required(factId, "factId"); - required(frameId, "frameId"); - if (protocol == null) { - throw new IllegalArgumentException("protocol must not be null"); - } - required(vehicleKey, "vehicleKey"); - if (eventTime == null) { - throw new IllegalArgumentException("eventTime must not be null"); - } - if (receivedAt == null) { - throw new IllegalArgumentException("receivedAt must not be null"); - } - required(rawUri, "rawUri"); - } - - static String clean(String value) { - return value == null ? "" : value.trim(); - } - - static Map metadata(Map metadata) { - return metadata == null ? Map.of() : Map.copyOf(metadata); - } - - private static void required(String value, String field) { - if (clean(value).isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - } - } -} diff --git a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/FactIds.java b/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/FactIds.java deleted file mode 100644 index aa1cdfdd..00000000 --- a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/FactIds.java +++ /dev/null @@ -1,50 +0,0 @@ -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; - -import java.nio.charset.StandardCharsets; -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.time.Instant; -import java.util.HexFormat; - -public final class FactIds { - - private FactIds() { - } - - public static String rawFrameId(ProtocolId protocol, - String vehicleKey, - int messageId, - int subType, - Instant receivedAt, - String checksum) { - if (protocol == null) { - throw new IllegalArgumentException("protocol must not be null"); - } - String input = protocol.name() + "|" - + clean(vehicleKey) + "|" - + messageId + "|" - + subType + "|" - + (receivedAt == null ? "" : receivedAt.toString()) + "|" - + clean(checksum); - return "rf_" + sha256(input).substring(0, 32); - } - - public static String decodedFactId(String frameId, String kind, int ordinal) { - return "df_" + clean(frameId) + "_" + clean(kind) + "_" + ordinal; - } - - private static String clean(String value) { - return value == null ? "" : value.trim(); - } - - private static String sha256(String value) { - try { - MessageDigest digest = MessageDigest.getInstance("SHA-256"); - return HexFormat.of().formatHex(digest.digest(value.getBytes(StandardCharsets.UTF_8))); - } catch (NoSuchAlgorithmException e) { - throw new IllegalStateException("SHA-256 is unavailable", e); - } - } -} diff --git a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/ParseStatus.java b/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/ParseStatus.java deleted file mode 100644 index 1855f7f1..00000000 --- a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/ParseStatus.java +++ /dev/null @@ -1,7 +0,0 @@ -package com.lingniu.ingest.facts; - -public enum ParseStatus { - NOT_PARSED, - SUCCEEDED, - FAILED -} diff --git a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/RawFrameFact.java b/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/RawFrameFact.java deleted file mode 100644 index 17298499..00000000 --- a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/RawFrameFact.java +++ /dev/null @@ -1,65 +0,0 @@ -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; - -import java.time.Instant; -import java.util.Map; - -public record RawFrameFact( - String frameId, - ProtocolId protocol, - String vehicleKey, - String vin, - String phone, - int messageId, - int subType, - Instant eventTime, - Instant receivedAt, - String peer, - String rawUri, - String checksum, - long rawSizeBytes, - ParseStatus parseStatus, - String parseError, - Map metadata -) { - public RawFrameFact { - frameId = required(frameId, "frameId"); - if (protocol == null) { - throw new IllegalArgumentException("protocol must not be null"); - } - vehicleKey = required(vehicleKey, "vehicleKey"); - receivedAt = required(receivedAt, "receivedAt"); - eventTime = eventTime == null ? receivedAt : eventTime; - rawUri = required(rawUri, "rawUri"); - checksum = required(checksum, "checksum"); - if (rawSizeBytes < 0) { - throw new IllegalArgumentException("rawSizeBytes must not be negative"); - } - parseStatus = parseStatus == null ? ParseStatus.NOT_PARSED : parseStatus; - vin = clean(vin); - phone = clean(phone); - peer = clean(peer); - parseError = clean(parseError); - metadata = metadata == null ? Map.of() : Map.copyOf(metadata); - } - - private static String required(String value, String field) { - String cleaned = clean(value); - if (cleaned.isBlank()) { - throw new IllegalArgumentException(field + " must not be blank"); - } - return cleaned; - } - - private static Instant required(Instant value, String field) { - if (value == null) { - throw new IllegalArgumentException(field + " must not be null"); - } - return value; - } - - private static String clean(String value) { - return value == null ? "" : value.trim(); - } -} diff --git a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/RawFrameIdentity.java b/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/RawFrameIdentity.java deleted file mode 100644 index 58b9bfc5..00000000 --- a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/RawFrameIdentity.java +++ /dev/null @@ -1,43 +0,0 @@ -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; - -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.time.Instant; -import java.util.HexFormat; - -public record RawFrameIdentity( - String checksum, - String vehicleKey, - String frameId -) { - public RawFrameIdentity { - checksum = checksum == null ? "" : checksum; - vehicleKey = vehicleKey == null ? "" : vehicleKey; - frameId = frameId == null ? "" : frameId; - } - - public static RawFrameIdentity derive(ProtocolId protocol, - String vin, - String phone, - String rawIdentitySeed, - int messageId, - int subType, - Instant receivedAt, - byte[] rawBytes) { - String checksum = "sha256:" + sha256(rawBytes); - String vehicleKey = VehicleKey.derive(protocol, vin, phone, rawIdentitySeed); - String frameId = FactIds.rawFrameId(protocol, vehicleKey, messageId, subType, receivedAt, checksum); - return new RawFrameIdentity(checksum, vehicleKey, frameId); - } - - private static String sha256(byte[] value) { - try { - MessageDigest digest = MessageDigest.getInstance("SHA-256"); - return HexFormat.of().formatHex(digest.digest(value == null ? new byte[0] : value)); - } catch (NoSuchAlgorithmException e) { - throw new IllegalStateException("SHA-256 is unavailable", e); - } - } -} diff --git a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/VehicleKey.java b/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/VehicleKey.java deleted file mode 100644 index ae93a68f..00000000 --- a/modules/core/ingest-facts/src/main/java/com/lingniu/ingest/facts/VehicleKey.java +++ /dev/null @@ -1,46 +0,0 @@ -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; - -import java.nio.charset.StandardCharsets; -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.util.HexFormat; - -public final class VehicleKey { - - private VehicleKey() { - } - - public static String derive(ProtocolId protocol, String vin, String phone, String rawIdentitySeed) { - if (protocol == null) { - throw new IllegalArgumentException("protocol must not be null"); - } - String normalizedVin = clean(vin); - if (!normalizedVin.isBlank()) { - return normalizedVin; - } - String normalizedPhone = clean(phone); - if (protocol == ProtocolId.JT808 && !normalizedPhone.isBlank()) { - return "jt808:" + normalizedPhone; - } - String seed = clean(rawIdentitySeed); - if (seed.isBlank()) { - seed = protocol.name(); - } - return "unknown:" + protocol.name() + ":" + sha256(seed).substring(0, 16); - } - - private static String clean(String value) { - return value == null ? "" : value.trim(); - } - - private static String sha256(String value) { - try { - MessageDigest digest = MessageDigest.getInstance("SHA-256"); - return HexFormat.of().formatHex(digest.digest(value.getBytes(StandardCharsets.UTF_8))); - } catch (NoSuchAlgorithmException e) { - throw new IllegalStateException("SHA-256 is unavailable", e); - } - } -} diff --git a/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/DecodedFactTest.java b/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/DecodedFactTest.java deleted file mode 100644 index 06f71891..00000000 --- a/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/DecodedFactTest.java +++ /dev/null @@ -1,39 +0,0 @@ -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class DecodedFactTest { - - @Test - void locationFactCarriesRawReferenceAndMetadata() { - DecodedFact.Location location = new DecodedFact.Location( - "fact-1", - "frame-1", - ProtocolId.JT808, - "jt808:013912345678", - "", - "013912345678", - Instant.parse("2026-06-29T00:00:00Z"), - Instant.parse("2026-06-29T00:00:01Z"), - "archive://jt808/frame-1.bin", - Map.of("messageId", "0x0200"), - 120.123456, - 30.123456, - 10.0, - 42.0, - 180.0, - 0L, - 2L, - 1000.5); - - assertThat(location.factId()).isEqualTo("fact-1"); - assertThat(location.rawUri()).isEqualTo("archive://jt808/frame-1.bin"); - assertThat(location.metadata()).containsEntry("messageId", "0x0200"); - } -} diff --git a/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/FactIdsTest.java b/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/FactIdsTest.java deleted file mode 100644 index cf5ffa29..00000000 --- a/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/FactIdsTest.java +++ /dev/null @@ -1,38 +0,0 @@ -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; - -import static org.assertj.core.api.Assertions.assertThat; - -class FactIdsTest { - - @Test - void rawFrameIdIsStableForSameInputs() { - String first = FactIds.rawFrameId( - ProtocolId.GB32960, - "VIN001", - 2, - 7, - Instant.parse("2026-06-29T00:00:00Z"), - "sha256:abc"); - String second = FactIds.rawFrameId( - ProtocolId.GB32960, - "VIN001", - 2, - 7, - Instant.parse("2026-06-29T00:00:00Z"), - "sha256:abc"); - - assertThat(first).isEqualTo(second).startsWith("rf_"); - } - - @Test - void decodedFactIdIncludesKind() { - String id = FactIds.decodedFactId("rf_123", "location", 0); - - assertThat(id).isEqualTo("df_rf_123_location_0"); - } -} diff --git a/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/RawFrameFactTest.java b/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/RawFrameFactTest.java deleted file mode 100644 index 74299833..00000000 --- a/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/RawFrameFactTest.java +++ /dev/null @@ -1,63 +0,0 @@ -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class RawFrameFactTest { - - @Test - void normalizesOptionalStringsAndMetadata() { - RawFrameFact fact = new RawFrameFact( - "frame-1", - ProtocolId.JT808, - "jt808:013912345678", - null, - "013912345678", - 0x0200, - 0, - null, - Instant.parse("2026-06-29T00:00:00Z"), - null, - "archive://2026/06/29/JT808/jt808-013912345678/frame-1.bin", - "sha256:abc", - 128, - ParseStatus.SUCCEEDED, - null, - Map.of("messageName", "LOCATION")); - - assertThat(fact.vin()).isEmpty(); - assertThat(fact.eventTime()).isEqualTo(fact.receivedAt()); - assertThat(fact.peer()).isEmpty(); - assertThat(fact.parseError()).isEmpty(); - assertThat(fact.metadata()).containsEntry("messageName", "LOCATION"); - } - - @Test - void rejectsMissingRequiredFields() { - assertThatThrownBy(() -> new RawFrameFact( - "", - ProtocolId.GB32960, - "VIN001", - "VIN001", - "", - 2, - 0, - Instant.parse("2026-06-29T00:00:00Z"), - Instant.parse("2026-06-29T00:00:00Z"), - "", - "archive://x", - "sha256:abc", - 1, - ParseStatus.SUCCEEDED, - "", - Map.of())) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("frameId"); - } -} diff --git a/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/RawFrameIdentityTest.java b/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/RawFrameIdentityTest.java deleted file mode 100644 index c0a59f1d..00000000 --- a/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/RawFrameIdentityTest.java +++ /dev/null @@ -1,36 +0,0 @@ -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; - -import static org.assertj.core.api.Assertions.assertThat; - -class RawFrameIdentityTest { - - @Test - void derivesChecksumVehicleKeyAndFrameIdTogether() { - RawFrameIdentity identity = RawFrameIdentity.derive( - ProtocolId.JT808, - "", - "13307795425", - "1782900000000000", - 0x0200, - 0, - Instant.parse("2026-07-01T10:00:00Z"), - new byte[]{0x7e, 0x02, 0x00, 0x7e}); - - assertThat(identity.checksum()).startsWith("sha256:"); - assertThat(identity.vehicleKey()).isEqualTo("jt808:13307795425"); - assertThat(identity.frameId()) - .isEqualTo(FactIds.rawFrameId( - ProtocolId.JT808, - "jt808:13307795425", - 0x0200, - 0, - Instant.parse("2026-07-01T10:00:00Z"), - identity.checksum())) - .startsWith("rf_"); - } -} diff --git a/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/VehicleKeyTest.java b/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/VehicleKeyTest.java deleted file mode 100644 index 43820676..00000000 --- a/modules/core/ingest-facts/src/test/java/com/lingniu/ingest/facts/VehicleKeyTest.java +++ /dev/null @@ -1,42 +0,0 @@ -package com.lingniu.ingest.facts; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class VehicleKeyTest { - - @Test - void usesVinForGb32960() { - assertThat(VehicleKey.derive(ProtocolId.GB32960, " LB9A32A20P0LS1257 ", "", "abc")) - .isEqualTo("LB9A32A20P0LS1257"); - } - - @Test - void usesResolvedVinForJt808WhenPresent() { - assertThat(VehicleKey.derive(ProtocolId.JT808, "VIN001", "013912345678", "abc")) - .isEqualTo("VIN001"); - } - - @Test - void usesPhoneForJt808WhenVinIsMissing() { - assertThat(VehicleKey.derive(ProtocolId.JT808, "", "013912345678", "abc")) - .isEqualTo("jt808:013912345678"); - } - - @Test - void usesStableUnknownHashWhenVehicleIdentifiersAreMissing() { - assertThat(VehicleKey.derive(ProtocolId.JT808, "", "", "abc")) - .isEqualTo(VehicleKey.derive(ProtocolId.JT808, null, null, "abc")) - .startsWith("unknown:JT808:"); - } - - @Test - void rejectsMissingProtocol() { - assertThatThrownBy(() -> VehicleKey.derive(null, "VIN001", "", "abc")) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("protocol"); - } -} diff --git a/modules/core/observability/pom.xml b/modules/core/observability/pom.xml deleted file mode 100644 index e5e14c6e..00000000 --- a/modules/core/observability/pom.xml +++ /dev/null @@ -1,38 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - observability - observability - Micrometer + Prometheus + 业务指标封装（事件计数、Dispatcher 耗时、DLQ 计数等）。 - - - - com.lingniu.ingest - ingest-api - - - org.springframework.boot - spring-boot-starter-actuator - - - io.micrometer - micrometer-registry-prometheus - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - diff --git a/modules/core/observability/src/main/java/com/lingniu/ingest/observability/IngestMetrics.java b/modules/core/observability/src/main/java/com/lingniu/ingest/observability/IngestMetrics.java deleted file mode 100644 index c9fa09b9..00000000 --- a/modules/core/observability/src/main/java/com/lingniu/ingest/observability/IngestMetrics.java +++ /dev/null @@ -1,104 +0,0 @@ -package com.lingniu.ingest.observability; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.IngestInterceptor; -import com.lingniu.ingest.api.pipeline.RawFrame; -import io.micrometer.core.instrument.Counter; -import io.micrometer.core.instrument.MeterRegistry; -import io.micrometer.core.instrument.Tags; -import io.micrometer.core.instrument.Timer; -import org.springframework.core.Ordered; - -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ConcurrentMap; - -/** - * 指标拦截器：自动以 {@link IngestInterceptor} 形式接入拦截链，无侵入采集核心流量指标。 - * - *

暴露指标： - *

    - *
  • {@code ingest_frames_total{protocol}} - 入站帧计数 - *
  • {@code ingest_events_total{protocol,type}} - 事件产出计数 - *
  • {@code ingest_errors_total{protocol}} - 异常计数 - *
  • {@code ingest_frame_duration_seconds{protocol}} - 单帧处理耗时 - *
- */ -public class IngestMetrics implements IngestInterceptor, Ordered { - - private static final String ATTR_TIMER_SAMPLE = "metrics.timer.sample"; - private static final String ATTR_PROTOCOL = "metrics.protocol"; - - private final MeterRegistry registry; - private final ConcurrentMap frameCounters = new ConcurrentHashMap<>(); - private final ConcurrentMap errorCounters = new ConcurrentHashMap<>(); - private final ConcurrentMap frameTimers = new ConcurrentHashMap<>(); - private final ConcurrentMap eventCounters = new ConcurrentHashMap<>(); - - public IngestMetrics(MeterRegistry registry) { - this.registry = registry; - } - - @Override - public boolean before(RawFrame frame, IngestContext ctx) { - ProtocolId p = frame.protocolId() == null ? ProtocolId.UNKNOWN : frame.protocolId(); - ctx.attr(ATTR_PROTOCOL, p); - frameCounters.computeIfAbsent(p, k -> - Counter.builder("ingest_frames_total").tag("protocol", k.name()).register(registry)).increment(); - Timer.Sample sample = Timer.start(registry); - ctx.attr(ATTR_TIMER_SAMPLE, sample); - return true; - } - - @Override - public void after(VehicleEvent event, IngestContext ctx) { - String key = event.source().name() + ":" + event.getClass().getSimpleName(); - eventCounters.computeIfAbsent(key, k -> { - String[] parts = k.split(":"); - return Counter.builder("ingest_events_total") - .tags(Tags.of("protocol", parts[0], "type", parts[1])) - .register(registry); - }).increment(); - - Timer.Sample s = ctx.attr(ATTR_TIMER_SAMPLE); - if (s != null) { - stopTimer(ctx, protocolTag(ctx)); - } - } - - @Override - public void onError(Throwable error, IngestContext ctx) { - String protocol = protocolTag(ctx); - errorCounters.computeIfAbsent(protocol, p -> - Counter.builder("ingest_errors_total").tag("protocol", p).register(registry)).increment(); - stopTimer(ctx, protocol); - } - - @Override - public int getOrder() { - return 10; // 尽早介入，晚于 Tracing（order=0） - } - - private void stopTimer(IngestContext ctx, String protocol) { - if (ctx == null) { - return; - } - Timer.Sample sample = ctx.attr(ATTR_TIMER_SAMPLE); - if (sample == null) { - return; - } - Timer timer = frameTimers.computeIfAbsent(protocol, p -> - Timer.builder("ingest_frame_duration_seconds").tag("protocol", p).register(registry)); - sample.stop(timer); - ctx.attr(ATTR_TIMER_SAMPLE, null); - } - - private static String protocolTag(IngestContext ctx) { - if (ctx == null) { - return "unknown"; - } - ProtocolId protocol = ctx.attr(ATTR_PROTOCOL); - return protocol == null || protocol == ProtocolId.UNKNOWN ? "unknown" : protocol.name(); - } -} diff --git a/modules/core/observability/src/main/java/com/lingniu/ingest/observability/ObservabilityAutoConfiguration.java b/modules/core/observability/src/main/java/com/lingniu/ingest/observability/ObservabilityAutoConfiguration.java deleted file mode 100644 index a6a5f63f..00000000 --- a/modules/core/observability/src/main/java/com/lingniu/ingest/observability/ObservabilityAutoConfiguration.java +++ /dev/null @@ -1,18 +0,0 @@ -package com.lingniu.ingest.observability; - -import io.micrometer.core.instrument.MeterRegistry; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.context.annotation.Bean; - -@AutoConfiguration -@ConditionalOnBean(MeterRegistry.class) -public class ObservabilityAutoConfiguration { - - @Bean - @ConditionalOnMissingBean - public IngestMetrics ingestMetrics(MeterRegistry registry) { - return new IngestMetrics(registry); - } -} diff --git a/modules/core/observability/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/core/observability/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index b3367068..00000000 --- a/modules/core/observability/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.observability.ObservabilityAutoConfiguration diff --git a/modules/core/observability/src/test/java/com/lingniu/ingest/observability/IngestMetricsTest.java b/modules/core/observability/src/test/java/com/lingniu/ingest/observability/IngestMetricsTest.java deleted file mode 100644 index cb44aa7d..00000000 --- a/modules/core/observability/src/test/java/com/lingniu/ingest/observability/IngestMetricsTest.java +++ /dev/null @@ -1,88 +0,0 @@ -package com.lingniu.ingest.observability; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.RawFrame; -import io.micrometer.core.instrument.simple.SimpleMeterRegistry; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class IngestMetricsTest { - - @Test - void countsErrorsAgainstTheFrameProtocolCapturedInBefore() { - SimpleMeterRegistry registry = new SimpleMeterRegistry(); - IngestMetrics metrics = new IngestMetrics(registry); - IngestContext context = new IngestContext("trace-1"); - - metrics.before(frame(ProtocolId.JT808), context); - metrics.onError(new IllegalStateException("boom"), context); - - assertThat(registry.counter("ingest_errors_total", "protocol", "JT808").count()) - .isEqualTo(1.0); - assertThat(registry.find("ingest_errors_total").tag("protocol", "unknown").counter()) - .isNull(); - } - - @Test - void recordsFrameDurationWhenProcessingFailsAfterBefore() { - SimpleMeterRegistry registry = new SimpleMeterRegistry(); - IngestMetrics metrics = new IngestMetrics(registry); - IngestContext context = new IngestContext("trace-2"); - - metrics.before(frame(ProtocolId.MQTT_YUTONG), context); - metrics.onError(new RuntimeException("boom"), context); - - assertThat(registry.timer("ingest_frame_duration_seconds", "protocol", "MQTT_YUTONG").count()) - .isEqualTo(1); - } - - @Test - void recordsFrameDurationAgainstTheFrameProtocolCapturedInBefore() { - SimpleMeterRegistry registry = new SimpleMeterRegistry(); - IngestMetrics metrics = new IngestMetrics(registry); - IngestContext context = new IngestContext("trace-4"); - - metrics.before(frame(ProtocolId.JT808), context); - metrics.after(new VehicleEvent.Heartbeat( - "event-1", - "VIN001", - ProtocolId.UNKNOWN, - Instant.parse("2026-07-01T00:00:01Z"), - Instant.parse("2026-07-01T00:00:02Z"), - "trace-4", - Map.of()), context); - - assertThat(registry.timer("ingest_frame_duration_seconds", "protocol", "JT808").count()) - .isEqualTo(1); - assertThat(registry.find("ingest_frame_duration_seconds").tag("protocol", "unknown").timer()) - .isNull(); - } - - @Test - void countsErrorsAsUnknownWhenThereIsNoFrameProtocolContext() { - SimpleMeterRegistry registry = new SimpleMeterRegistry(); - IngestMetrics metrics = new IngestMetrics(registry); - - metrics.onError(new IllegalStateException("boom"), new IngestContext("trace-3")); - - assertThat(registry.counter("ingest_errors_total", "protocol", "unknown").count()) - .isEqualTo(1.0); - } - - private static RawFrame frame(ProtocolId protocolId) { - return new RawFrame( - protocolId, - 0x0200, - 0, - null, - new byte[]{0x01}, - Map.of(), - Instant.parse("2026-07-01T00:00:00Z")); - } -} diff --git a/modules/core/session-core/pom.xml b/modules/core/session-core/pom.xml deleted file mode 100644 index d02dbe36..00000000 --- a/modules/core/session-core/pom.xml +++ /dev/null @@ -1,52 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - session-core - session-core - 设备会话 + 鉴权 + Token + Redis SessionStore。 - - - - com.lingniu.ingest - ingest-api - - - org.springframework.boot - spring-boot-starter - - - org.springframework.boot - spring-boot-autoconfigure - - - org.springframework.boot - spring-boot-starter-data-redis - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.mockito - mockito-core - test - - - org.springframework.boot - spring-boot-test - test - - - diff --git a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/CommandDispatcher.java b/modules/core/session-core/src/main/java/com/lingniu/ingest/session/CommandDispatcher.java deleted file mode 100644 index 30958922..00000000 --- a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/CommandDispatcher.java +++ /dev/null @@ -1,23 +0,0 @@ -package com.lingniu.ingest.session; - -import java.time.Duration; -import java.util.concurrent.CompletableFuture; - -/** - * 下行命令调度器 SPI。 - * - *

用于 command-gateway 把 HTTP 请求转为设备命令，等待设备应答。真正的实现需要与协议 - * 模块（如 jt808）的 Netty 通道绑定：按 sessionId 找到 Channel → 发送命令字节 → 用 - * {@code CompletableFuture + 序列号 map} 实现同步请求/应答。 - * - *

当前阶段提供接口 + 占位实现，具体 Netty 绑定在 command-gateway / protocol-jt808 - * 的下一批次落地。 - */ -public interface CommandDispatcher { - - /** 异步下发命令，不等应答。 */ - CompletableFuture notify(String sessionId, Object command); - - /** 同步下发命令并等待应答。 */ - CompletableFuture request(String sessionId, Object command, Class responseType, Duration timeout); -} diff --git a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/DeviceSession.java b/modules/core/session-core/src/main/java/com/lingniu/ingest/session/DeviceSession.java deleted file mode 100644 index b2261608..00000000 --- a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/DeviceSession.java +++ /dev/null @@ -1,46 +0,0 @@ -package com.lingniu.ingest.session; - -import com.lingniu.ingest.api.ProtocolId; - -import java.time.Instant; -import java.util.Map; - -/** - * 设备会话领域模型。不可变；更新通过 {@link SessionStore#update} 原子替换。 - * - * @param sessionId 会话 ID（通常由 Netty channel id + 终端号生成） - * @param protocol 协议 ID - * @param vin 车架号（注册/鉴权后补齐，注册前为 null） - * @param phone 终端手机号 / 设备号 - * @param token 鉴权 Token（平台下发） - * @param peerAddress 对端 IP:port - * @param registeredAt 注册时间 - * @param lastSeenAt 最后活跃时间 - * @param attributes 协议特定属性（协议版本、型号等） - */ -public record DeviceSession( - String sessionId, - ProtocolId protocol, - String vin, - String phone, - String token, - String peerAddress, - Instant registeredAt, - Instant lastSeenAt, - Map attributes -) { - public DeviceSession touch(Instant now) { - return new DeviceSession(sessionId, protocol, vin, phone, token, peerAddress, - registeredAt, now, attributes); - } - - public DeviceSession withVin(String vin) { - return new DeviceSession(sessionId, protocol, vin, phone, token, peerAddress, - registeredAt, lastSeenAt, attributes); - } - - public DeviceSession withToken(String token) { - return new DeviceSession(sessionId, protocol, vin, phone, token, peerAddress, - registeredAt, lastSeenAt, attributes); - } -} diff --git a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/NoopCommandDispatcher.java b/modules/core/session-core/src/main/java/com/lingniu/ingest/session/NoopCommandDispatcher.java deleted file mode 100644 index 6539dabe..00000000 --- a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/NoopCommandDispatcher.java +++ /dev/null @@ -1,31 +0,0 @@ -package com.lingniu.ingest.session; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.time.Duration; -import java.util.concurrent.CompletableFuture; - -/** - * 占位命令调度器：返回 {@link UnsupportedOperationException}， - * 待 command-gateway 与协议模块的下行通道打通后替换为真实实现。 - */ -public final class NoopCommandDispatcher implements CommandDispatcher { - - private static final Logger log = LoggerFactory.getLogger(NoopCommandDispatcher.class); - - @Override - public CompletableFuture notify(String sessionId, Object command) { - log.warn("[noop] notify sessionId={} command={} dropped", sessionId, command); - return CompletableFuture.failedFuture(new UnsupportedOperationException( - "CommandDispatcher not yet implemented; configure a real one in protocol module.")); - } - - @Override - public CompletableFuture request(String sessionId, Object command, Class responseType, Duration timeout) { - log.warn("[noop] request sessionId={} command={} responseType={} dropped", - sessionId, command, responseType.getSimpleName()); - return CompletableFuture.failedFuture(new UnsupportedOperationException( - "CommandDispatcher not yet implemented; configure a real one in protocol module.")); - } -} diff --git a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/RedisSessionStore.java b/modules/core/session-core/src/main/java/com/lingniu/ingest/session/RedisSessionStore.java deleted file mode 100644 index 90a86457..00000000 --- a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/RedisSessionStore.java +++ /dev/null @@ -1,191 +0,0 @@ -package com.lingniu.ingest.session; - -import com.lingniu.ingest.api.ProtocolId; -import org.springframework.data.redis.core.StringRedisTemplate; - -import java.time.Duration; -import java.time.Instant; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Optional; -import java.util.Set; -import java.util.function.UnaryOperator; - -/** - * Redis-backed session store for multi-instance command routing metadata. - * - *

The Netty Channel itself stays local to the protocol instance. Redis keeps - * the stable terminal indexes so HTTP callers can resolve sessionId / VIN / - * phone consistently across restarts and replicas. - */ -public final class RedisSessionStore implements SessionStore { - - private static final String PREFIX = "vehicle:session:"; - private static final String ID_SET_KEY = PREFIX + "index:ids"; - - private final StringRedisTemplate redis; - private final Duration ttl; - - public RedisSessionStore(StringRedisTemplate redis, Duration ttl) { - this.redis = redis; - this.ttl = ttl; - } - - @Override - public void put(DeviceSession session) { - // 同一个 sessionId 重新写入时先清理旧 VIN/phone 索引，避免旧外部号继续指向新会话。 - findBySessionId(session.sessionId()).ifPresent(old -> deleteIndexes(old)); - redis.opsForHash().putAll(sessionKey(session.sessionId()), toHash(session)); - redis.expire(sessionKey(session.sessionId()), ttl); - if (hasText(session.vin())) { - redis.opsForValue().set(vinIndexKey(session.vin()), session.sessionId(), ttl); - } - if (hasText(session.phone())) { - redis.opsForValue().set(phoneIndexKey(session.phone()), session.sessionId(), ttl); - } - redis.opsForSet().add(ID_SET_KEY, session.sessionId()); - redis.expire(ID_SET_KEY, ttl); - } - - @Override - public Optional findBySessionId(String sessionId) { - if (!hasText(sessionId)) { - return Optional.empty(); - } - Map values = redis.opsForHash().entries(sessionKey(sessionId)); - if (values == null || values.isEmpty()) { - return Optional.empty(); - } - redis.expire(sessionKey(sessionId), ttl); - return Optional.of(fromHash(values)); - } - - @Override - public Optional findByVin(String vin) { - if (!hasText(vin)) { - return Optional.empty(); - } - String sessionId = redis.opsForValue().get(vinIndexKey(vin)); - Optional session = findBySessionId(sessionId); - if (session.isEmpty() && hasText(sessionId)) { - // 二级索引可能比主 hash 晚过期；发现悬挂索引时立即清理。 - redis.delete(vinIndexKey(vin)); - } - return session; - } - - @Override - public Optional findByPhone(String phone) { - if (!hasText(phone)) { - return Optional.empty(); - } - String sessionId = redis.opsForValue().get(phoneIndexKey(phone)); - Optional session = findBySessionId(sessionId); - if (session.isEmpty() && hasText(sessionId)) { - // 二级索引可能比主 hash 晚过期；发现悬挂索引时立即清理。 - redis.delete(phoneIndexKey(phone)); - } - return session; - } - - @Override - public synchronized Optional update(String sessionId, UnaryOperator updater) { - Optional current = findBySessionId(sessionId); - if (current.isEmpty()) { - return Optional.empty(); - } - DeviceSession next = updater.apply(current.orElseThrow()); - put(next); - return Optional.of(next); - } - - @Override - public void remove(String sessionId) { - Optional current = findBySessionId(sessionId); - redis.delete(sessionKey(sessionId)); - current.ifPresent(this::deleteIndexes); - redis.opsForSet().remove(ID_SET_KEY, sessionId); - } - - @Override - public int size() { - Long size = redis.opsForSet().size(ID_SET_KEY); - if (size == null || size <= 0) { - return 0; - } - return size > Integer.MAX_VALUE ? Integer.MAX_VALUE : size.intValue(); - } - - private void deleteIndexes(DeviceSession session) { - if (hasText(session.vin())) { - redis.delete(vinIndexKey(session.vin())); - } - if (hasText(session.phone())) { - redis.delete(phoneIndexKey(session.phone())); - } - } - - private static Map toHash(DeviceSession session) { - Map values = new LinkedHashMap<>(); - // attr.* 保留协议侧附加字段，后续扩展不需要改 Redis schema。 - put(values, "sessionId", session.sessionId()); - put(values, "protocol", session.protocol() == null ? null : session.protocol().name()); - put(values, "vin", session.vin()); - put(values, "phone", session.phone()); - put(values, "token", session.token()); - put(values, "peerAddress", session.peerAddress()); - put(values, "registeredAt", session.registeredAt() == null ? null : session.registeredAt().toString()); - put(values, "lastSeenAt", session.lastSeenAt() == null ? null : session.lastSeenAt().toString()); - if (session.attributes() != null) { - session.attributes().forEach((key, value) -> put(values, "attr." + key, value)); - } - return values; - } - - private static DeviceSession fromHash(Map values) { - Map attributes = new LinkedHashMap<>(); - values.forEach((key, value) -> { - String k = String.valueOf(key); - if (k.startsWith("attr.") && value != null) { - attributes.put(k.substring("attr.".length()), String.valueOf(value)); - } - }); - return new DeviceSession( - value(values, "sessionId"), - ProtocolId.valueOf(value(values, "protocol")), - value(values, "vin"), - value(values, "phone"), - value(values, "token"), - value(values, "peerAddress"), - Instant.parse(value(values, "registeredAt")), - Instant.parse(value(values, "lastSeenAt")), - Map.copyOf(attributes)); - } - - private static void put(Map values, String key, String value) { - if (value != null) { - values.put(key, value); - } - } - - private static String value(Map values, String key) { - Object value = values.get(key); - return value == null ? null : String.valueOf(value); - } - - private static String sessionKey(String sessionId) { - return PREFIX + sessionId; - } - - private static String vinIndexKey(String vin) { - return PREFIX + "index:vin:" + vin; - } - - private static String phoneIndexKey(String phone) { - return PREFIX + "index:phone:" + phone; - } - - private static boolean hasText(String value) { - return value != null && !value.isBlank(); - } -} diff --git a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/SessionProperties.java b/modules/core/session-core/src/main/java/com/lingniu/ingest/session/SessionProperties.java deleted file mode 100644 index d44f1abb..00000000 --- a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/SessionProperties.java +++ /dev/null @@ -1,35 +0,0 @@ -package com.lingniu.ingest.session; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -import java.time.Duration; -import java.util.Locale; - -@ConfigurationProperties(prefix = "lingniu.ingest.session") -public class SessionProperties { - - private String store = "redis"; - - private Duration ttl = Duration.ofMinutes(30); - - public String getStore() { - return store; - } - - public void setStore(String store) { - String value = store == null || store.isBlank() ? "redis" : store.trim().toLowerCase(Locale.ROOT); - if (!"redis".equals(value)) { - throw new IllegalStateException("session.store supports only redis; configured value: " + store); - } - this.store = value; - } - - public Duration getTtl() { - return ttl; - } - - public void setTtl(Duration ttl) { - this.ttl = ttl; - } - -} diff --git a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/SessionStore.java b/modules/core/session-core/src/main/java/com/lingniu/ingest/session/SessionStore.java deleted file mode 100644 index 38811206..00000000 --- a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/SessionStore.java +++ /dev/null @@ -1,25 +0,0 @@ -package com.lingniu.ingest.session; - -import java.util.Optional; -import java.util.function.UnaryOperator; - -/** - * 设备会话存储 SPI。生产自动配置只创建 Redis 实现；测试可显式提供轻量实现。 - */ -public interface SessionStore { - - void put(DeviceSession session); - - Optional findBySessionId(String sessionId); - - Optional findByVin(String vin); - - Optional findByPhone(String phone); - - /** 原子更新：读 → 修改 → 写。返回更新后的会话；若原会话不存在返回 empty。 */ - Optional update(String sessionId, UnaryOperator updater); - - void remove(String sessionId); - - int size(); -} diff --git a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/config/SessionCoreAutoConfiguration.java b/modules/core/session-core/src/main/java/com/lingniu/ingest/session/config/SessionCoreAutoConfiguration.java deleted file mode 100644 index 29f9497e..00000000 --- a/modules/core/session-core/src/main/java/com/lingniu/ingest/session/config/SessionCoreAutoConfiguration.java +++ /dev/null @@ -1,37 +0,0 @@ -package com.lingniu.ingest.session.config; - -import com.lingniu.ingest.session.CommandDispatcher; -import com.lingniu.ingest.session.NoopCommandDispatcher; -import com.lingniu.ingest.session.RedisSessionStore; -import com.lingniu.ingest.session.SessionProperties; -import com.lingniu.ingest.session.SessionStore; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.data.redis.RedisAutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; -import org.springframework.data.redis.core.StringRedisTemplate; - -@AutoConfiguration(after = RedisAutoConfiguration.class) -@EnableConfigurationProperties(SessionProperties.class) -public class SessionCoreAutoConfiguration { - - @Bean - @ConditionalOnBean(StringRedisTemplate.class) - @ConditionalOnProperty(prefix = "lingniu.ingest.session", name = "store", havingValue = "redis", - matchIfMissing = true) - @ConditionalOnMissingBean - public SessionStore redisSessionStore(StringRedisTemplate redis, SessionProperties properties) { - // Redis 只保存会话元数据索引；真实 Netty Channel 仍在各协议进程本地内存中。 - return new RedisSessionStore(redis, properties.getTtl()); - } - - @Bean - @ConditionalOnMissingBean - public CommandDispatcher commandDispatcher() { - // 没有协议模块提供真实下行实现时使用 Noop，避免仅接收/查询部署因为缺 Bean 启动失败。 - return new NoopCommandDispatcher(); - } -} diff --git a/modules/core/session-core/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/core/session-core/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index 0f53ea0f..00000000 --- a/modules/core/session-core/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.session.config.SessionCoreAutoConfiguration diff --git a/modules/core/session-core/src/test/java/com/lingniu/ingest/session/RedisSessionStoreTest.java b/modules/core/session-core/src/test/java/com/lingniu/ingest/session/RedisSessionStoreTest.java deleted file mode 100644 index 1b4df081..00000000 --- a/modules/core/session-core/src/test/java/com/lingniu/ingest/session/RedisSessionStoreTest.java +++ /dev/null @@ -1,137 +0,0 @@ -package com.lingniu.ingest.session; - -import com.lingniu.ingest.api.ProtocolId; -import org.junit.jupiter.api.Test; -import org.springframework.data.redis.core.HashOperations; -import org.springframework.data.redis.core.SetOperations; -import org.springframework.data.redis.core.StringRedisTemplate; -import org.springframework.data.redis.core.ValueOperations; - -import java.time.Duration; -import java.time.Instant; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Optional; -import java.util.Set; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.ArgumentMatchers.any; -import static org.mockito.ArgumentMatchers.eq; -import static org.mockito.Mockito.atLeastOnce; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.when; - -class RedisSessionStoreTest { - - @Test - void putStoresSessionHashAndThreeIndexesWithTtl() { - RedisFixture fixture = new RedisFixture(); - RedisSessionStore store = new RedisSessionStore(fixture.redis, Duration.ofMinutes(45)); - DeviceSession session = sampleSession("sid-1", "VIN001", "13800138000"); - - store.put(session); - - verify(fixture.hashOps).putAll(eq("vehicle:session:sid-1"), any(Map.class)); - verify(fixture.valueOps).set("vehicle:session:index:vin:VIN001", "sid-1", Duration.ofMinutes(45)); - verify(fixture.valueOps).set("vehicle:session:index:phone:13800138000", "sid-1", Duration.ofMinutes(45)); - verify(fixture.setOps).add("vehicle:session:index:ids", "sid-1"); - verify(fixture.redis).expire("vehicle:session:sid-1", Duration.ofMinutes(45)); - verify(fixture.redis).expire("vehicle:session:index:ids", Duration.ofMinutes(45)); - } - - @Test - void findsBySessionIdVinAndPhoneFromRedisIndexes() { - RedisFixture fixture = new RedisFixture(); - RedisSessionStore store = new RedisSessionStore(fixture.redis, Duration.ofMinutes(30)); - Map stored = storedSession("sid-1", "VIN001", "13800138000"); - when(fixture.hashOps.entries("vehicle:session:sid-1")).thenReturn(stored); - when(fixture.valueOps.get("vehicle:session:index:vin:VIN001")).thenReturn("sid-1"); - when(fixture.valueOps.get("vehicle:session:index:phone:13800138000")).thenReturn("sid-1"); - - Optional byId = store.findBySessionId("sid-1"); - Optional byVin = store.findByVin("VIN001"); - Optional byPhone = store.findByPhone("13800138000"); - - assertThat(byId).isPresent(); - assertThat(byVin).contains(byId.orElseThrow()); - assertThat(byPhone).contains(byId.orElseThrow()); - assertThat(byId.orElseThrow().attributes()).containsEntry("identitySource", "BINDING"); - verify(fixture.redis, atLeastOnce()).expire("vehicle:session:sid-1", Duration.ofMinutes(30)); - } - - @Test - void updateRewritesIndexesWhenVinChangesAndRemoveDeletesAllKeys() { - RedisFixture fixture = new RedisFixture(); - RedisSessionStore store = new RedisSessionStore(fixture.redis, Duration.ofMinutes(30)); - when(fixture.hashOps.entries("vehicle:session:sid-1")) - .thenReturn(storedSession("sid-1", "VIN001", "13800138000")) - .thenReturn(storedSession("sid-1", "VIN001", "13800138000")) - .thenReturn(storedSession("sid-1", "VIN002", "13800138000")); - - Optional updated = store.update("sid-1", session -> session.withVin("VIN002")); - store.remove("sid-1"); - - assertThat(updated).isPresent(); - verify(fixture.redis).delete("vehicle:session:index:vin:VIN001"); - verify(fixture.valueOps).set("vehicle:session:index:vin:VIN002", "sid-1", Duration.ofMinutes(30)); - verify(fixture.redis).delete("vehicle:session:sid-1"); - verify(fixture.redis).delete("vehicle:session:index:vin:VIN002"); - verify(fixture.redis, atLeastOnce()).delete("vehicle:session:index:phone:13800138000"); - verify(fixture.setOps).remove("vehicle:session:index:ids", "sid-1"); - } - - @Test - void sizeReturnsRedisIdSetCardinality() { - RedisFixture fixture = new RedisFixture(); - RedisSessionStore store = new RedisSessionStore(fixture.redis, Duration.ofMinutes(30)); - when(fixture.setOps.size("vehicle:session:index:ids")).thenReturn(12L); - - assertThat(store.size()).isEqualTo(12); - } - - private static DeviceSession sampleSession(String sid, String vin, String phone) { - return new DeviceSession( - sid, - ProtocolId.JT808, - vin, - phone, - "token-1", - "127.0.0.1:9000", - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:01:00Z"), - Map.of("identitySource", "BINDING")); - } - - private static Map storedSession(String sid, String vin, String phone) { - Map values = new LinkedHashMap<>(); - values.put("sessionId", sid); - values.put("protocol", "JT808"); - values.put("vin", vin); - values.put("phone", phone); - values.put("token", "token-1"); - values.put("peerAddress", "127.0.0.1:9000"); - values.put("registeredAt", "2026-06-22T08:00:00Z"); - values.put("lastSeenAt", "2026-06-22T08:01:00Z"); - values.put("attr.identitySource", "BINDING"); - return values; - } - - private static final class RedisFixture { - private final StringRedisTemplate redis = mock(StringRedisTemplate.class); - @SuppressWarnings("unchecked") - private final HashOperations hashOps = mock(HashOperations.class); - @SuppressWarnings("unchecked") - private final ValueOperations valueOps = mock(ValueOperations.class); - @SuppressWarnings("unchecked") - private final SetOperations setOps = mock(SetOperations.class); - - private RedisFixture() { - when(redis.opsForHash()).thenReturn(hashOps); - when(redis.opsForValue()).thenReturn(valueOps); - when(redis.opsForSet()).thenReturn(setOps); - when(hashOps.entries("vehicle:session:sid-1")).thenReturn(Map.of()); - when(setOps.members("vehicle:session:index:ids")).thenReturn(Set.of()); - } - } -} diff --git a/modules/core/session-core/src/test/java/com/lingniu/ingest/session/config/SessionCoreAutoConfigurationTest.java b/modules/core/session-core/src/test/java/com/lingniu/ingest/session/config/SessionCoreAutoConfigurationTest.java deleted file mode 100644 index 2d13d90d..00000000 --- a/modules/core/session-core/src/test/java/com/lingniu/ingest/session/config/SessionCoreAutoConfigurationTest.java +++ /dev/null @@ -1,84 +0,0 @@ -package com.lingniu.ingest.session.config; - -import com.lingniu.ingest.session.RedisSessionStore; -import com.lingniu.ingest.session.SessionStore; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.autoconfigure.data.redis.RedisAutoConfiguration; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.data.redis.core.StringRedisTemplate; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; -import static org.mockito.Mockito.mock; - -class SessionCoreAutoConfigurationTest { - - private final ApplicationContextRunner contextRunner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(SessionCoreAutoConfiguration.class)); - - @Test - void doesNotCreateImplicitInMemorySessionStoreByDefault() { - contextRunner.run(context -> { - assertThat(context).doesNotHaveBean(SessionStore.class); - }); - } - - @Test - void doesNotPublishInMemorySessionStoreImplementation() { - assertThatThrownBy(() -> Class.forName("com.lingniu.ingest.session.InMemorySessionStore")) - .isInstanceOf(ClassNotFoundException.class); - } - - @Test - void usesRedisSessionStoreWhenEnabledAndRedisTemplateExists() { - contextRunner - .withBean(StringRedisTemplate.class, () -> mock(StringRedisTemplate.class)) - .withPropertyValues( - "lingniu.ingest.session.store=redis", - "lingniu.ingest.session.ttl=45m") - .run(context -> { - assertThat(context).hasSingleBean(SessionStore.class); - assertThat(context).hasSingleBean(RedisSessionStore.class); - }); - } - - @Test - void usesRedisSessionStoreByDefaultWhenRedisTemplateExists() { - contextRunner - .withBean(StringRedisTemplate.class, () -> mock(StringRedisTemplate.class)) - .run(context -> { - assertThat(context).hasSingleBean(SessionStore.class); - assertThat(context).hasSingleBean(RedisSessionStore.class); - }); - } - - @Test - void waitsForRedisAutoConfigurationBeforeCheckingRedisTemplateBean() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of( - SessionCoreAutoConfiguration.class, - RedisAutoConfiguration.class)) - .run(context -> { - assertThat(context).hasSingleBean(StringRedisTemplate.class); - assertThat(context).hasSingleBean(SessionStore.class); - assertThat(context).hasSingleBean(RedisSessionStore.class); - }); - } - - @Test - void rejectsMemorySessionStoreModeWithRedisOnlyRule() { - contextRunner - .withPropertyValues("lingniu.ingest.session.store=memory") - .run(context -> assertThat(context.getStartupFailure()) - .hasRootCauseInstanceOf(IllegalStateException.class) - .hasStackTraceContaining("session.store supports only redis") - .hasStackTraceContaining("memory")); - } - - @Test - void sessionPropertiesDoNotExposeLegacyStoreEnum() { - assertThatThrownBy(() -> Class.forName("com.lingniu.ingest.session.SessionProperties$Store")) - .isInstanceOf(ClassNotFoundException.class); - } -} diff --git a/modules/core/vehicle-identity/pom.xml b/modules/core/vehicle-identity/pom.xml deleted file mode 100644 index 2c6b9ad2..00000000 --- a/modules/core/vehicle-identity/pom.xml +++ /dev/null @@ -1,61 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - vehicle-identity - vehicle-identity - 跨协议车辆身份解析与外部标识绑定。 - - - - com.lingniu.ingest - ingest-api - - - org.springframework.boot - spring-boot-starter - - - org.springframework.boot - spring-boot-autoconfigure - - - org.springframework - spring-jdbc - - - com.fasterxml.jackson.core - jackson-databind - - - com.mysql - mysql-connector-j - runtime - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.springframework.boot - spring-boot-test - test - - - com.h2database - h2 - test - - - diff --git a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/MySqlVehicleIdentityService.java b/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/MySqlVehicleIdentityService.java deleted file mode 100644 index 90336fec..00000000 --- a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/MySqlVehicleIdentityService.java +++ /dev/null @@ -1,232 +0,0 @@ -package com.lingniu.ingest.identity; - -import com.fasterxml.jackson.core.JsonProcessingException; -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.identity.config.VehicleIdentityProperties; -import org.springframework.jdbc.core.ConnectionCallback; -import org.springframework.jdbc.core.JdbcTemplate; - -import javax.sql.DataSource; -import java.sql.DatabaseMetaData; -import java.sql.ResultSet; -import java.sql.SQLException; -import java.util.List; -import java.util.Locale; -import java.util.Map; - -public final class MySqlVehicleIdentityService implements VehicleIdentityResolver, VehicleIdentityRegistry { - - private final JdbcTemplate jdbc; - private final ObjectMapper mapper; - private final String tableName; - - public MySqlVehicleIdentityService(DataSource dataSource, - VehicleIdentityProperties.Mysql properties, - ObjectMapper mapper) { - this.jdbc = new JdbcTemplate(dataSource); - this.mapper = mapper == null ? new ObjectMapper() : mapper; - VehicleIdentityProperties.Mysql mysql = properties == null ? new VehicleIdentityProperties.Mysql() : properties; - this.tableName = safeTableName(mysql.getTableName()); - if (mysql.isInitializeSchema()) { - initializeSchema(); - } - } - - @Override - public void bind(VehicleIdentityBinding binding) { - String metadataJson = metadataJson(binding.metadata()); - jdbc.update(""" - INSERT INTO %s - (protocol, vin, phone, device_id, plate, province, city, maker, device_type, plate_color, metadata_json, - created_at, updated_at) - VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP) - ON DUPLICATE KEY UPDATE - vin = VALUES(vin), - device_id = VALUES(device_id), - province = VALUES(province), - city = VALUES(city), - maker = VALUES(maker), - device_type = VALUES(device_type), - plate_color = VALUES(plate_color), - metadata_json = VALUES(metadata_json), - updated_at = CURRENT_TIMESTAMP - """.formatted(tableName), - protocol(binding.protocol()), binding.vin(), binding.phone(), binding.deviceId(), binding.plate(), - binding.metadata().getOrDefault("province", ""), - binding.metadata().getOrDefault("city", ""), - binding.metadata().getOrDefault("maker", ""), - binding.metadata().getOrDefault("deviceType", ""), - binding.metadata().getOrDefault("plateColor", ""), - metadataJson); - } - - @Override - public VehicleIdentity resolve(VehicleIdentityLookup lookup) { - if (lookup == null) { - return new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN); - } - if (!lookup.vin().isBlank()) { - return new VehicleIdentity(lookup.vin(), true, VehicleIdentitySource.EXPLICIT_VIN); - } - VehicleIdentity byPhone = resolveBy("phone", lookup.protocol(), lookup.phone(), VehicleIdentitySource.BOUND_PHONE); - if (byPhone != null) { - return byPhone; - } - VehicleIdentity byDevice = resolveBy("device_id", lookup.protocol(), lookup.deviceId(), VehicleIdentitySource.BOUND_DEVICE_ID); - if (byDevice != null) { - return byDevice; - } - VehicleIdentity byPlate = resolveBy("plate", lookup.protocol(), lookup.plate(), VehicleIdentitySource.BOUND_PLATE); - if (byPlate != null) { - return byPlate; - } - if (!lookup.deviceId().isBlank()) { - return new VehicleIdentity(lookup.deviceId(), false, VehicleIdentitySource.FALLBACK_DEVICE_ID); - } - if (!lookup.phone().isBlank()) { - return new VehicleIdentity(lookup.phone(), false, VehicleIdentitySource.FALLBACK_PHONE); - } - if (!lookup.plate().isBlank()) { - return new VehicleIdentity(lookup.plate(), false, VehicleIdentitySource.FALLBACK_PLATE); - } - return new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN); - } - - private VehicleIdentity resolveBy(String column, ProtocolId protocol, String externalId, VehicleIdentitySource source) { - if (externalId == null || externalId.isBlank()) { - return null; - } - String safeColumn = switch (column) { - case "phone", "device_id", "plate" -> column; - default -> throw new IllegalArgumentException("unsupported identity lookup column: " + column); - }; - List matches = jdbc.query(""" - SELECT vin - FROM %s - WHERE protocol = ? AND %s = ? - ORDER BY updated_at DESC - LIMIT 1 - """.formatted(tableName, safeColumn), - (rs, rowNum) -> identity(rs, source), - protocol(protocol), externalId.trim()); - return matches.isEmpty() ? null : matches.getFirst(); - } - - private VehicleIdentity identity(ResultSet rs, VehicleIdentitySource source) throws SQLException { - return new VehicleIdentity(rs.getString("vin"), true, source); - } - - private void initializeSchema() { - jdbc.execute(""" - CREATE TABLE IF NOT EXISTS %s ( - id BIGINT AUTO_INCREMENT PRIMARY KEY, - protocol VARCHAR(32) NOT NULL, - vin VARCHAR(32) NOT NULL, - phone VARCHAR(64) NOT NULL DEFAULT '', - device_id VARCHAR(64) NOT NULL DEFAULT '', - plate VARCHAR(64) NOT NULL DEFAULT '', - province VARCHAR(32) NOT NULL DEFAULT '', - city VARCHAR(32) NOT NULL DEFAULT '', - maker VARCHAR(64) NOT NULL DEFAULT '', - device_type VARCHAR(128) NOT NULL DEFAULT '', - plate_color VARCHAR(32) NOT NULL DEFAULT '', - metadata_json TEXT, - created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, - updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, - UNIQUE KEY uk_vehicle_identity_protocol_phone (protocol, phone), - KEY idx_vehicle_identity_protocol_device (protocol, device_id), - KEY idx_vehicle_identity_protocol_plate (protocol, plate) - ) - """.formatted(tableName)); - ensureColumn("phone", "phone VARCHAR(64) NOT NULL DEFAULT ''"); - ensureColumn("device_id", "device_id VARCHAR(64) NOT NULL DEFAULT ''"); - ensureColumn("plate", "plate VARCHAR(64) NOT NULL DEFAULT ''"); - ensureColumn("province", "province VARCHAR(32) NOT NULL DEFAULT ''"); - ensureColumn("city", "city VARCHAR(32) NOT NULL DEFAULT ''"); - ensureColumn("maker", "maker VARCHAR(64) NOT NULL DEFAULT ''"); - ensureColumn("device_type", "device_type VARCHAR(128) NOT NULL DEFAULT ''"); - ensureColumn("plate_color", "plate_color VARCHAR(32) NOT NULL DEFAULT ''"); - ensureColumn("metadata_json", "metadata_json TEXT"); - ensureColumn("created_at", "created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP"); - ensureColumn("updated_at", "updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP"); - ensureIndex("idx_vehicle_identity_protocol_device", "CREATE INDEX idx_vehicle_identity_protocol_device ON %s (protocol, device_id)"); - ensureIndex("idx_vehicle_identity_protocol_plate", "CREATE INDEX idx_vehicle_identity_protocol_plate ON %s (protocol, plate)"); - } - - private void ensureColumn(String columnName, String definition) { - if (!columnExists(columnName)) { - jdbc.execute("ALTER TABLE " + tableName + " ADD COLUMN " + definition); - } - } - - private boolean columnExists(String columnName) { - return Boolean.TRUE.equals(jdbc.execute((ConnectionCallback) connection -> { - DatabaseMetaData metaData = connection.getMetaData(); - for (String table : nameVariants(tableName)) { - for (String column : nameVariants(columnName)) { - try (ResultSet rs = metaData.getColumns(null, null, table, column)) { - if (rs.next()) { - return true; - } - } - } - } - return false; - })); - } - - private void ensureIndex(String indexName, String ddlTemplate) { - if (!indexExists(indexName)) { - jdbc.execute(ddlTemplate.formatted(tableName)); - } - } - - private boolean indexExists(String indexName) { - return Boolean.TRUE.equals(jdbc.execute((ConnectionCallback) connection -> { - DatabaseMetaData metaData = connection.getMetaData(); - for (String table : nameVariants(tableName)) { - try (ResultSet rs = metaData.getIndexInfo(null, null, table, false, false)) { - while (rs.next()) { - String existing = rs.getString("INDEX_NAME"); - if (existing != null && existing.equalsIgnoreCase(indexName)) { - return true; - } - } - } - } - return false; - })); - } - - private static String[] nameVariants(String name) { - return new String[]{ - name, - name.toLowerCase(Locale.ROOT), - name.toUpperCase(Locale.ROOT) - }; - } - - private String metadataJson(Map metadata) { - if (metadata == null || metadata.isEmpty()) { - return "{}"; - } - try { - return mapper.writeValueAsString(metadata); - } catch (JsonProcessingException e) { - throw new IllegalArgumentException("vehicle identity metadata serialize failed", e); - } - } - - private static String protocol(ProtocolId protocol) { - return protocol == null ? "UNKNOWN" : protocol.name(); - } - - private static String safeTableName(String value) { - String name = value == null || value.isBlank() ? "vehicle_identity_bindings" : value.trim(); - if (!name.matches("[A-Za-z0-9_]+")) { - throw new IllegalArgumentException("invalid vehicle identity mysql table name: " + value); - } - return name.toLowerCase(Locale.ROOT); - } -} diff --git a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentity.java b/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentity.java deleted file mode 100644 index 06449c86..00000000 --- a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentity.java +++ /dev/null @@ -1,16 +0,0 @@ -package com.lingniu.ingest.identity; - -public record VehicleIdentity( - String vin, - boolean resolved, - VehicleIdentitySource source -) { - public VehicleIdentity { - vin = normalize(vin); - source = source == null ? VehicleIdentitySource.UNKNOWN : source; - } - - private static String normalize(String value) { - return value == null ? "" : value.trim(); - } -} diff --git a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityBinding.java b/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityBinding.java deleted file mode 100644 index 3a31a26a..00000000 --- a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityBinding.java +++ /dev/null @@ -1,37 +0,0 @@ -package com.lingniu.ingest.identity; - -import com.lingniu.ingest.api.ProtocolId; - -import java.util.Map; - -public record VehicleIdentityBinding( - ProtocolId protocol, - String vin, - String phone, - String deviceId, - String plate, - Map metadata -) { - public VehicleIdentityBinding(ProtocolId protocol, String vin, String phone, String deviceId, String plate) { - this(protocol, vin, phone, deviceId, plate, Map.of()); - } - - public VehicleIdentityBinding { - vin = normalize(vin); - phone = normalize(phone); - deviceId = normalize(deviceId); - plate = normalize(plate); - metadata = metadata == null ? Map.of() : Map.copyOf(metadata); - if (vin.isBlank()) { - throw new IllegalArgumentException("vin must not be blank"); - } - } - - public boolean hasResolvedVin() { - return !"unknown".equalsIgnoreCase(vin); - } - - private static String normalize(String value) { - return value == null ? "" : value.trim(); - } -} diff --git a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityLookup.java b/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityLookup.java deleted file mode 100644 index 4d9ea38c..00000000 --- a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityLookup.java +++ /dev/null @@ -1,22 +0,0 @@ -package com.lingniu.ingest.identity; - -import com.lingniu.ingest.api.ProtocolId; - -public record VehicleIdentityLookup( - ProtocolId protocol, - String vin, - String phone, - String deviceId, - String plate -) { - public VehicleIdentityLookup { - vin = normalize(vin); - phone = normalize(phone); - deviceId = normalize(deviceId); - plate = normalize(plate); - } - - private static String normalize(String value) { - return value == null ? "" : value.trim(); - } -} diff --git a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityRegistry.java b/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityRegistry.java deleted file mode 100644 index d15808c5..00000000 --- a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityRegistry.java +++ /dev/null @@ -1,9 +0,0 @@ -package com.lingniu.ingest.identity; - -public interface VehicleIdentityRegistry { - - void bind(VehicleIdentityBinding binding); - - default void register(VehicleRegistrationBinding registration) { - } -} diff --git a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityResolver.java b/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityResolver.java deleted file mode 100644 index 00228696..00000000 --- a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentityResolver.java +++ /dev/null @@ -1,6 +0,0 @@ -package com.lingniu.ingest.identity; - -public interface VehicleIdentityResolver { - - VehicleIdentity resolve(VehicleIdentityLookup lookup); -} diff --git a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentitySource.java b/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentitySource.java deleted file mode 100644 index b35cfa69..00000000 --- a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleIdentitySource.java +++ /dev/null @@ -1,13 +0,0 @@ -package com.lingniu.ingest.identity; - -public enum VehicleIdentitySource { - EXPLICIT_VIN, - REGISTERED, - BOUND_PHONE, - BOUND_DEVICE_ID, - BOUND_PLATE, - FALLBACK_DEVICE_ID, - FALLBACK_PHONE, - FALLBACK_PLATE, - UNKNOWN -} diff --git a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleRegistrationBinding.java b/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleRegistrationBinding.java deleted file mode 100644 index 4761c61b..00000000 --- a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/VehicleRegistrationBinding.java +++ /dev/null @@ -1,36 +0,0 @@ -package com.lingniu.ingest.identity; - -import com.lingniu.ingest.api.ProtocolId; - -public record VehicleRegistrationBinding( - ProtocolId protocol, - String vin, - String phone, - String deviceId, - String plate, - Integer province, - Integer city, - String maker, - String deviceType, - Integer plateColor -) { - public VehicleRegistrationBinding { - vin = normalize(vin); - phone = normalize(phone); - deviceId = normalize(deviceId); - plate = normalize(plate); - maker = normalize(maker); - deviceType = normalize(deviceType); - if (phone.isBlank() && deviceId.isBlank() && plate.isBlank()) { - throw new IllegalArgumentException("at least one registration identifier must not be blank"); - } - } - - public boolean hasResolvedVin() { - return !vin.isBlank() && !"unknown".equalsIgnoreCase(vin); - } - - private static String normalize(String value) { - return value == null ? "" : value.trim(); - } -} diff --git a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/config/VehicleIdentityAutoConfiguration.java b/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/config/VehicleIdentityAutoConfiguration.java deleted file mode 100644 index c47f50c0..00000000 --- a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/config/VehicleIdentityAutoConfiguration.java +++ /dev/null @@ -1,43 +0,0 @@ -package com.lingniu.ingest.identity.config; - -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.identity.MySqlVehicleIdentityService; -import com.lingniu.ingest.identity.VehicleIdentityRegistry; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; -import org.springframework.jdbc.datasource.DriverManagerDataSource; - -import javax.sql.DataSource; -import java.util.Locale; - -@AutoConfiguration -@EnableConfigurationProperties(VehicleIdentityProperties.class) -public class VehicleIdentityAutoConfiguration { - - @Bean - @ConditionalOnMissingBean - public ObjectMapper vehicleIdentityObjectMapper() { - return new ObjectMapper(); - } - - @Bean - @ConditionalOnMissingBean({VehicleIdentityResolver.class, VehicleIdentityRegistry.class}) - public MySqlVehicleIdentityService mysqlVehicleIdentityService(VehicleIdentityProperties properties, - ObjectMapper objectMapper) { - requireMysqlStore(properties.getStore()); - VehicleIdentityProperties.Mysql mysql = properties.getMysql(); - DataSource dataSource = new DriverManagerDataSource( - mysql.getJdbcUrl(), mysql.getUsername(), mysql.getPassword()); - return new MySqlVehicleIdentityService(dataSource, mysql, objectMapper); - } - - private static void requireMysqlStore(String value) { - String store = value == null || value.isBlank() ? "mysql" : value.trim().toLowerCase(Locale.ROOT); - if (!"mysql".equals(store)) { - throw new IllegalStateException("identity.store supports only mysql; configured value: " + value); - } - } -} diff --git a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/config/VehicleIdentityProperties.java b/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/config/VehicleIdentityProperties.java deleted file mode 100644 index c47f7cc5..00000000 --- a/modules/core/vehicle-identity/src/main/java/com/lingniu/ingest/identity/config/VehicleIdentityProperties.java +++ /dev/null @@ -1,34 +0,0 @@ -package com.lingniu.ingest.identity.config; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -@ConfigurationProperties(prefix = "lingniu.ingest.identity") -public class VehicleIdentityProperties { - - private String store = "mysql"; - private Mysql mysql = new Mysql(); - - public String getStore() { return store; } - public void setStore(String store) { this.store = store; } - public Mysql getMysql() { return mysql; } - public void setMysql(Mysql mysql) { this.mysql = mysql; } - - public static class Mysql { - private String jdbcUrl = "jdbc:mysql://127.0.0.1:3306/lingniu_vehicle?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=Asia/Shanghai"; - private String username = "root"; - private String password = ""; - private String tableName = "vehicle_identity_bindings"; - private boolean initializeSchema = true; - - public String getJdbcUrl() { return jdbcUrl; } - public void setJdbcUrl(String jdbcUrl) { this.jdbcUrl = jdbcUrl; } - public String getUsername() { return username; } - public void setUsername(String username) { this.username = username; } - public String getPassword() { return password; } - public void setPassword(String password) { this.password = password; } - public String getTableName() { return tableName; } - public void setTableName(String tableName) { this.tableName = tableName; } - public boolean isInitializeSchema() { return initializeSchema; } - public void setInitializeSchema(boolean initializeSchema) { this.initializeSchema = initializeSchema; } - } -} diff --git a/modules/core/vehicle-identity/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/core/vehicle-identity/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index 2ad78acc..00000000 --- a/modules/core/vehicle-identity/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.identity.config.VehicleIdentityAutoConfiguration diff --git a/modules/core/vehicle-identity/src/test/java/com/lingniu/ingest/identity/MySqlVehicleIdentityServiceJdbcTest.java b/modules/core/vehicle-identity/src/test/java/com/lingniu/ingest/identity/MySqlVehicleIdentityServiceJdbcTest.java deleted file mode 100644 index 87eb5cfb..00000000 --- a/modules/core/vehicle-identity/src/test/java/com/lingniu/ingest/identity/MySqlVehicleIdentityServiceJdbcTest.java +++ /dev/null @@ -1,90 +0,0 @@ -package com.lingniu.ingest.identity; - -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.identity.config.VehicleIdentityProperties; -import org.h2.jdbcx.JdbcDataSource; -import org.junit.jupiter.api.Test; - -import java.sql.Connection; -import java.sql.ResultSet; -import java.sql.Statement; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; -import java.util.UUID; - -import static org.assertj.core.api.Assertions.assertThat; - -class MySqlVehicleIdentityServiceJdbcTest { - - @Test - void plateVinBindingIsQueryableThroughRealJdbc() throws Exception { - JdbcDataSource dataSource = new JdbcDataSource(); - dataSource.setURL("jdbc:h2:mem:" + UUID.randomUUID() - + ";MODE=MySQL;DATABASE_TO_UPPER=false;DB_CLOSE_DELAY=-1"); - - VehicleIdentityProperties.Mysql mysql = new VehicleIdentityProperties.Mysql(); - mysql.setTableName("vehicle_identity_binding"); - MySqlVehicleIdentityService service = new MySqlVehicleIdentityService( - dataSource, - mysql, - new ObjectMapper()); - - service.bind(new VehicleIdentityBinding( - ProtocolId.JT808, - "LNVIN00000061001", - "13079961001", - "dev61001", - "粤B61001", - Map.of( - "province", "44", - "city", "4401", - "maker", "maker-a", - "deviceType", "type-a", - "plateColor", "1"))); - - try (Connection connection = dataSource.getConnection(); - Statement statement = connection.createStatement()) { - ResultSet binding = statement.executeQuery(""" - SELECT vin, phone, device_id, plate, province, city, maker, device_type, plate_color - FROM vehicle_identity_binding - WHERE protocol = 'JT808' AND phone = '13079961001' - """); - assertThat(binding.next()).isTrue(); - assertThat(binding.getString("vin")).isEqualTo("LNVIN00000061001"); - assertThat(binding.getString("phone")).isEqualTo("13079961001"); - assertThat(binding.getString("device_id")).isEqualTo("dev61001"); - assertThat(binding.getString("plate")).isEqualTo("粤B61001"); - assertThat(binding.getString("province")).isEqualTo("44"); - assertThat(binding.getString("city")).isEqualTo("4401"); - assertThat(binding.getString("maker")).isEqualTo("maker-a"); - assertThat(binding.getString("device_type")).isEqualTo("type-a"); - assertThat(binding.getString("plate_color")).isEqualTo("1"); - - List columns = new ArrayList<>(); - ResultSet bindingColumns = connection.getMetaData() - .getColumns(null, null, "vehicle_identity_binding", null); - while (bindingColumns.next()) { - columns.add(bindingColumns.getString("COLUMN_NAME")); - } - assertThat(columns).contains( - "protocol", - "phone", - "device_id", - "plate", - "vin", - "metadata_json"); - } - - assertThat(service.resolve(new VehicleIdentityLookup( - ProtocolId.JT808, "", "13079961001", "", "")).vin()) - .isEqualTo("LNVIN00000061001"); - assertThat(service.resolve(new VehicleIdentityLookup( - ProtocolId.JT808, "", "", "dev61001", "")).vin()) - .isEqualTo("LNVIN00000061001"); - assertThat(service.resolve(new VehicleIdentityLookup( - ProtocolId.JT808, "", "", "", "粤B61001")).vin()) - .isEqualTo("LNVIN00000061001"); - } -} diff --git a/modules/core/vehicle-identity/src/test/java/com/lingniu/ingest/identity/MySqlVehicleIdentityServiceTest.java b/modules/core/vehicle-identity/src/test/java/com/lingniu/ingest/identity/MySqlVehicleIdentityServiceTest.java deleted file mode 100644 index 1088a3f3..00000000 --- a/modules/core/vehicle-identity/src/test/java/com/lingniu/ingest/identity/MySqlVehicleIdentityServiceTest.java +++ /dev/null @@ -1,157 +0,0 @@ -package com.lingniu.ingest.identity; - -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.identity.config.VehicleIdentityProperties; -import org.junit.jupiter.api.Test; -import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.jdbc.datasource.DriverManagerDataSource; - -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class MySqlVehicleIdentityServiceTest { - - @Test - void upsertsRegistrationBindingAndResolvesByExternalIdentifiers() { - DriverManagerDataSource dataSource = new DriverManagerDataSource( - "jdbc:h2:mem:mysql-identity;MODE=MySQL;DATABASE_TO_UPPER=false;DB_CLOSE_DELAY=-1", "sa", ""); - VehicleIdentityProperties.Mysql mysql = new VehicleIdentityProperties.Mysql(); - mysql.setTableName("vehicle_identity_bindings"); - MySqlVehicleIdentityService service = new MySqlVehicleIdentityService( - dataSource, - mysql, - new ObjectMapper()); - - service.bind(new VehicleIdentityBinding( - ProtocolId.JT808, - "JT808AABBCCDDEE1", - "13079963289", - "9963289", - "沪A00113F", - Map.of( - "province", "31", - "city", "113", - "maker", "70112", - "deviceType", "SEG-9888G", - "plateColor", "2"))); - - assertThat(service.resolve(new VehicleIdentityLookup(ProtocolId.JT808, "", "13079963289", "", ""))) - .extracting(VehicleIdentity::vin, VehicleIdentity::resolved, VehicleIdentity::source) - .containsExactly("JT808AABBCCDDEE1", true, VehicleIdentitySource.BOUND_PHONE); - assertThat(service.resolve(new VehicleIdentityLookup(ProtocolId.JT808, "", "", "9963289", ""))) - .extracting(VehicleIdentity::vin, VehicleIdentity::resolved, VehicleIdentity::source) - .containsExactly("JT808AABBCCDDEE1", true, VehicleIdentitySource.BOUND_DEVICE_ID); - assertThat(service.resolve(new VehicleIdentityLookup(ProtocolId.JT808, "", "", "", "沪A00113F"))) - .extracting(VehicleIdentity::vin, VehicleIdentity::resolved, VehicleIdentity::source) - .containsExactly("JT808AABBCCDDEE1", true, VehicleIdentitySource.BOUND_PLATE); - - JdbcTemplate jdbc = new JdbcTemplate(dataSource); - Map row = jdbc.queryForMap(""" - SELECT vin, phone, device_id, plate, province, city, maker, device_type, plate_color - FROM vehicle_identity_bindings - WHERE protocol = 'JT808' AND phone = '13079963289' - """); - assertThat(row) - .containsEntry("vin", "JT808AABBCCDDEE1") - .containsEntry("phone", "13079963289") - .containsEntry("device_id", "9963289") - .containsEntry("plate", "沪A00113F") - .containsEntry("province", "31") - .containsEntry("city", "113") - .containsEntry("maker", "70112") - .containsEntry("device_type", "SEG-9888G") - .containsEntry("plate_color", "2"); - } - - @Test - void initializesMissingColumnsOnExistingLegacyTable() { - DriverManagerDataSource dataSource = new DriverManagerDataSource( - "jdbc:h2:mem:mysql-identity-legacy;MODE=MySQL;DATABASE_TO_UPPER=false;DB_CLOSE_DELAY=-1", "sa", ""); - JdbcTemplate jdbc = new JdbcTemplate(dataSource); - jdbc.execute(""" - CREATE TABLE vehicle_identity_binding ( - id BIGINT AUTO_INCREMENT PRIMARY KEY, - protocol VARCHAR(32) NOT NULL, - vin VARCHAR(32) NOT NULL, - legacy_key VARCHAR(64) NOT NULL DEFAULT '' - ) - """); - VehicleIdentityProperties.Mysql mysql = new VehicleIdentityProperties.Mysql(); - mysql.setTableName("vehicle_identity_binding"); - - MySqlVehicleIdentityService service = new MySqlVehicleIdentityService( - dataSource, - mysql, - new ObjectMapper()); - service.bind(new VehicleIdentityBinding( - ProtocolId.JT808, - "JT808LEGACY00001", - "13079963289", - "9963289", - "沪A00113F", - Map.of("maker", "70112", "deviceType", "SEG-9888G"))); - - Map row = jdbc.queryForMap(""" - SELECT vin, maker, device_type, metadata_json - FROM vehicle_identity_binding - WHERE protocol = 'JT808' AND phone = '13079963289' - """); - assertThat(row) - .containsEntry("vin", "JT808LEGACY00001") - .containsEntry("maker", "70112") - .containsEntry("device_type", "SEG-9888G"); - assertThat((String) row.get("metadata_json")).contains("SEG-9888G"); - } - - @Test - void upsertsByCompositeProtocolPhonePlatePrimaryKey() { - DriverManagerDataSource dataSource = new DriverManagerDataSource( - "jdbc:h2:mem:mysql-identity-composite;MODE=MySQL;DATABASE_TO_UPPER=false;DB_CLOSE_DELAY=-1", "sa", ""); - JdbcTemplate jdbc = new JdbcTemplate(dataSource); - jdbc.execute(""" - CREATE TABLE vehicle_identity_binding ( - protocol VARCHAR(32) NOT NULL DEFAULT 'ALL', - phone VARCHAR(128) NOT NULL DEFAULT '', - plate VARCHAR(128) NOT NULL DEFAULT '', - vin VARCHAR(64) NOT NULL, - province VARCHAR(32) NOT NULL DEFAULT '', - city VARCHAR(32) NOT NULL DEFAULT '', - maker VARCHAR(64) NOT NULL DEFAULT '', - device_type VARCHAR(128) NOT NULL DEFAULT '', - plate_color VARCHAR(32) NOT NULL DEFAULT '', - metadata_json TEXT, - created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, - updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, - device_id VARCHAR(64) NOT NULL DEFAULT '', - PRIMARY KEY (protocol, phone, plate) - ) - """); - VehicleIdentityProperties.Mysql mysql = new VehicleIdentityProperties.Mysql(); - mysql.setTableName("vehicle_identity_binding"); - MySqlVehicleIdentityService service = new MySqlVehicleIdentityService( - dataSource, - mysql, - new ObjectMapper()); - service.bind(new VehicleIdentityBinding(ProtocolId.JT808, - "VIN-A", "13079963289", "DEV-A", "沪A00113F")); - - service.bind(new VehicleIdentityBinding(ProtocolId.JT808, - "VIN-B", "13079963289", "DEV-B", "沪A00114F")); - service.bind(new VehicleIdentityBinding(ProtocolId.JT808, - "VIN-B2", "13079963289", "DEV-B2", "沪A00114F")); - - Integer rows = jdbc.queryForObject(""" - SELECT COUNT(*) - FROM vehicle_identity_binding - WHERE protocol = 'JT808' AND phone = '13079963289' - """, Integer.class); - assertThat(rows).isEqualTo(2); - assertThat(jdbc.queryForObject(""" - SELECT vin - FROM vehicle_identity_binding - WHERE protocol = 'JT808' AND phone = '13079963289' AND plate = '沪A00114F' - """, String.class)).isEqualTo("VIN-B2"); - } -} diff --git a/modules/core/vehicle-identity/src/test/java/com/lingniu/ingest/identity/config/VehicleIdentityAutoConfigurationTest.java b/modules/core/vehicle-identity/src/test/java/com/lingniu/ingest/identity/config/VehicleIdentityAutoConfigurationTest.java deleted file mode 100644 index ca673fa4..00000000 --- a/modules/core/vehicle-identity/src/test/java/com/lingniu/ingest/identity/config/VehicleIdentityAutoConfigurationTest.java +++ /dev/null @@ -1,84 +0,0 @@ -package com.lingniu.ingest.identity.config; - -import com.lingniu.ingest.identity.VehicleIdentityRegistry; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; - -import java.nio.file.Files; -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; - -class VehicleIdentityAutoConfigurationTest { - - private final ApplicationContextRunner runner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(VehicleIdentityAutoConfiguration.class)); - - @Test - void createsMysqlIdentityServiceByDefault() { - runner - .withPropertyValues( - "lingniu.ingest.identity.mysql.jdbc-url=jdbc:h2:mem:auto-identity-default;MODE=MySQL;DATABASE_TO_UPPER=false;DB_CLOSE_DELAY=-1", - "lingniu.ingest.identity.mysql.username=sa", - "lingniu.ingest.identity.mysql.password=", - "lingniu.ingest.identity.mysql.table-name=vehicle_identity_bindings") - .run(context -> { - assertThat(context).hasSingleBean(VehicleIdentityResolver.class); - assertThat(context).hasSingleBean(VehicleIdentityRegistry.class); - assertThat(context.getBean(VehicleIdentityResolver.class)) - .isSameAs(context.getBean(VehicleIdentityRegistry.class)); - assertThat(context.getBean(VehicleIdentityResolver.class).getClass().getSimpleName()) - .isEqualTo("MySqlVehicleIdentityService"); - }); - } - - @Test - void createsMysqlIdentityServiceWhenConfigured() { - runner - .withPropertyValues( - "lingniu.ingest.identity.store=mysql", - "lingniu.ingest.identity.mysql.jdbc-url=jdbc:h2:mem:auto-identity;MODE=MySQL;DATABASE_TO_UPPER=false;DB_CLOSE_DELAY=-1", - "lingniu.ingest.identity.mysql.username=sa", - "lingniu.ingest.identity.mysql.password=", - "lingniu.ingest.identity.mysql.table-name=vehicle_identity_bindings") - .run(context -> { - assertThat(context).hasSingleBean(VehicleIdentityResolver.class); - assertThat(context).hasSingleBean(VehicleIdentityRegistry.class); - assertThat(context.getBean(VehicleIdentityResolver.class)) - .isSameAs(context.getBean(VehicleIdentityRegistry.class)); - assertThat(context.getBean(VehicleIdentityResolver.class).getClass().getSimpleName()) - .isEqualTo("MySqlVehicleIdentityService"); - }); - } - - @Test - void rejectsUnsupportedStoreMode() { - runner - .withPropertyValues("lingniu.ingest.identity.store=file") - .run(context -> assertThat(context.getStartupFailure()) - .hasRootCauseInstanceOf(IllegalStateException.class) - .hasMessageContaining("identity.store supports only mysql") - .hasMessageContaining("file")); - } - - @Test - void rejectsSqliteStoreModeWithSameMysqlOnlyRule() { - runner - .withPropertyValues("lingniu.ingest.identity.store=sqlite") - .run(context -> assertThat(context.getStartupFailure()) - .hasRootCauseInstanceOf(IllegalStateException.class) - .hasMessageContaining("identity.store supports only mysql") - .hasMessageContaining("sqlite")); - } - - @Test - void moduleDoesNotKeepAlternativeStoreImplementations() { - Path sourceRoot = Path.of("src/main/java/com/lingniu/ingest/identity"); - - assertThat(Files.exists(sourceRoot.resolve("InMemoryVehicleIdentityService.java"))) - .as("identity store is mysql-only; in-memory implementation must not be shipped") - .isFalse(); - } -} diff --git a/modules/inbound/inbound-mqtt/pom.xml b/modules/inbound/inbound-mqtt/pom.xml deleted file mode 100644 index d5efad2d..00000000 --- a/modules/inbound/inbound-mqtt/pom.xml +++ /dev/null @@ -1,60 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - inbound-mqtt - inbound-mqtt - MQTT 接入模块（支持多 endpoint，宇通等），Eclipse Paho MQTT Client + 双向 TLS。 - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - ingest-core - - - com.lingniu.ingest - vehicle-identity - - - com.lingniu.ingest - vehicle-identity-test-support - test - - - org.springframework.boot - spring-boot-starter - - - org.springframework.boot - spring-boot-configuration-processor - true - - - org.eclipse.paho - org.eclipse.paho.client.mqttv3 - - - com.fasterxml.jackson.core - jackson-databind - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/client/MqttEndpointManager.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/client/MqttEndpointManager.java deleted file mode 100644 index efe435bf..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/client/MqttEndpointManager.java +++ /dev/null @@ -1,380 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.client; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentityLookup; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.inbound.mqtt.config.MqttInboundProperties; -import com.lingniu.ingest.inbound.mqtt.model.MqttPayload; -import com.lingniu.ingest.inbound.mqtt.profile.MqttProfileRegistry; -import org.eclipse.paho.client.mqttv3.IMqttDeliveryToken; -import org.eclipse.paho.client.mqttv3.MqttAsyncClient; -import org.eclipse.paho.client.mqttv3.MqttCallbackExtended; -import org.eclipse.paho.client.mqttv3.MqttConnectOptions; -import org.eclipse.paho.client.mqttv3.MqttException; -import org.eclipse.paho.client.mqttv3.MqttMessage; -import org.eclipse.paho.client.mqttv3.persist.MemoryPersistence; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.net.URI; -import java.nio.charset.StandardCharsets; -import java.time.Instant; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.UUID; -import java.util.concurrent.TimeUnit; - -/** - * MQTT 多 endpoint 生命周期管理。 - * - *

为每个 endpoint 创建一个 Paho 异步客户端 → 连接 → 订阅 → 注册回调 → - * 回调里解析 payload → {@link Dispatcher#dispatch(RawFrame)}。 - * - *

MQTT 回调不做业务处理，虚拟线程由 Dispatcher 侧的 Disruptor 承接。 - */ -public final class MqttEndpointManager implements AutoCloseable { - - private static final Logger log = LoggerFactory.getLogger(MqttEndpointManager.class); - - private final MqttInboundProperties props; - private final MqttProfileRegistry profileRegistry; - private final VehicleIdentityResolver identityResolver; - private final Dispatcher dispatcher; - private final List clients = new ArrayList<>(); - private boolean started; - - public MqttEndpointManager(MqttInboundProperties props, - MqttProfileRegistry profileRegistry, - VehicleIdentityResolver identityResolver, - Dispatcher dispatcher) { - this.props = props; - this.profileRegistry = profileRegistry; - if (identityResolver == null) { - throw new IllegalArgumentException("identityResolver must not be null"); - } - this.identityResolver = identityResolver; - this.dispatcher = dispatcher; - } - - public synchronized void start() { - if (started) { - return; - } - started = true; - if (props.getEndpoints().isEmpty()) { - log.warn("mqtt inbound enabled but no endpoints configured"); - return; - } - log.info("mqtt inbound starting endpoints={}", props.getEndpoints().size()); - for (MqttInboundProperties.Endpoint ep : props.getEndpoints()) { - try { - // 多 endpoint 互不影响：单个连接初始化失败会派发 operationalError，但不会阻止其他 endpoint。 - log.info("mqtt endpoint [{}] initializing uri={} topic={} profile={} qos={} cleanSession={} tlsHostnameVerification={}", - ep.getName(), ep.getUri(), ep.getTopic(), ep.getProfile(), ep.getQos(), ep.isCleanSession(), - ep.getTls() == null || ep.getTls().isHostnameVerificationEnabled()); - MqttAsyncClient client = buildClient(ep); - clients.add(client); - connectAndSubscribe(client, ep); - } catch (Exception e) { - log.error("mqtt endpoint [{}] init failed, skipped", ep.getName(), e); - publishOperationalError(ep, "init", e.getMessage() == null ? e.getClass().getSimpleName() : e.getMessage()); - } - } - } - - private MqttAsyncClient buildClient(MqttInboundProperties.Endpoint ep) throws MqttException { - String clientId = ep.getClientId() == null || ep.getClientId().isBlank() - ? "lingniu-" + UUID.randomUUID() : ep.getClientId(); - return new MqttAsyncClient(pahoServerUri(ep.getUri()), clientId, new MemoryPersistence()); - } - - private void connectAndSubscribe(MqttAsyncClient client, MqttInboundProperties.Endpoint ep) throws MqttException { - client.setCallback(new MqttCallbackExtended() { - @Override - public void connectComplete(boolean reconnect, String serverURI) { - if (!reconnect) { - return; - } - try { - client.subscribe(ep.getTopic(), mapQos(ep.getQos())).waitForCompletion(timeoutMs(ep)); - log.info("mqtt endpoint [{}] re-subscribed topic={} qos={}", ep.getName(), ep.getTopic(), ep.getQos()); - } catch (Exception e) { - log.error("mqtt endpoint [{}] re-subscribe failed topic={}", ep.getName(), ep.getTopic(), e); - publishOperationalError(ep, "subscribe", - e.getMessage() == null ? e.getClass().getSimpleName() : e.getMessage()); - } - } - - @Override - public void connectionLost(Throwable cause) { - log.warn("mqtt endpoint [{}] connection lost", ep.getName(), cause); - } - - @Override - public void messageArrived(String topic, MqttMessage message) { - log.info("mqtt endpoint [{}] received topic={} bytes={}", ep.getName(), topic, message.getPayload().length); - onMessage(ep, topic, message.getPayload()); - } - - @Override - public void deliveryComplete(IMqttDeliveryToken token) { - // inbound only - } - }); - - MqttConnectOptions options = connectOptions(ep); - client.connect(options).waitForCompletion(timeoutMs(ep)); - log.info("mqtt endpoint [{}] connected serverUri={} clientId={}", - ep.getName(), client.getServerURI(), client.getClientId()); - client.subscribe(ep.getTopic(), mapQos(ep.getQos())).waitForCompletion(timeoutMs(ep)); - log.info("mqtt endpoint [{}] subscribed topic={} qos={}", ep.getName(), ep.getTopic(), ep.getQos()); - } - - private void onMessage(MqttInboundProperties.Endpoint ep, String topic, byte[] payload) { - try { - MqttPayload parsed = profileRegistry.parse(ep.getName(), ep.getProfile(), topic, payload); - String externalDeviceId = firstNonBlank(parsed.deviceId(), parsed.vin()); - IdentityResolution identity = resolveIdentity(parsed); - parsed = parsed.withIdentity(identity.identity(), identity.errorMessage()); - Map meta = new HashMap<>(4); - // metadata 同时保留外部设备号和内部 VIN，便于排查绑定表错误导致的车辆归属问题。 - meta.put("vin", identity.identity().vin()); - meta.put("externalVin", parsed.externalVin() == null ? "" : parsed.externalVin()); - meta.put("phone", parsed.phone() == null ? "" : parsed.phone()); - meta.put("mqttDeviceId", externalDeviceId); - meta.put("plateNo", parsed.plateNo() == null ? "" : parsed.plateNo()); - meta.put("identityResolved", Boolean.toString(identity.identity().resolved())); - meta.put("identitySource", identity.identity().source().name()); - if (identity.errorMessage() != null) { - meta.put("identityError", "true"); - meta.put("identityErrorMessage", identity.errorMessage()); - } - meta.put("endpoint", ep.getName()); - meta.put("topic", topic); - meta.put("profile", parsed.profile() == null ? "" : parsed.profile()); - if (parsed.data() != null && parsed.data().path("_parseError").asBoolean(false)) { - meta.put("parseError", "true"); - meta.put("parseErrorMessage", parsed.data().path("_parseErrorMessage").asText("")); - meta.put("unsupportedProfile", - Boolean.toString(parsed.data().path("_unsupportedProfile").asBoolean(false))); - } - RawFrame rf = new RawFrame( - ProtocolId.MQTT_YUTONG, 0, 0, - parsed, payload, meta, Instant.now()); - dispatcher.dispatch(rf); - } catch (Exception e) { - log.warn("mqtt endpoint [{}] message handling failed topic={} len={}", - ep.getName(), topic, payload.length, e); - // 单条消息异常也转成 RawFrame 错误事件，避免 MQTT 回调线程吞掉现场。 - publishMessageError(ep, topic, payload, e.getMessage() == null ? e.getClass().getSimpleName() : e.getMessage()); - } - } - - private IdentityResolution resolveIdentity(MqttPayload payload) { - String externalDeviceId = firstNonBlank(payload.deviceId(), payload.vin()); - String externalVin = payload.externalVin() == null ? "" : payload.externalVin(); - String phone = payload.phone() == null ? "" : payload.phone(); - String plateNo = payload.plateNo() == null ? "" : payload.plateNo(); - try { - VehicleIdentity bound = identityResolver.resolve(new VehicleIdentityLookup( - ProtocolId.MQTT_YUTONG, - externalVin, - phone, - externalDeviceId, - plateNo)); - if (bound.resolved() || !looksLikeVin(externalDeviceId)) { - return new IdentityResolution(bound, null); - } - return new IdentityResolution(identityResolver.resolve(new VehicleIdentityLookup( - ProtocolId.MQTT_YUTONG, - externalDeviceId, - phone, - externalDeviceId, - plateNo)), null); - } catch (RuntimeException e) { - return new IdentityResolution( - new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN), - e.getMessage() == null ? "" : e.getMessage()); - } - } - - private static String firstNonBlank(String... values) { - for (String value : values) { - if (value != null && !value.isBlank()) { - return value; - } - } - return ""; - } - - private void publishOperationalError(MqttInboundProperties.Endpoint ep, String phase, String reason) { - // 连接/订阅级别错误也进入 Dispatcher，这样监控和历史查询能看到接入端本身的问题。 - String endpointName = ep == null ? "" : ep.getName(); - String profile = ep == null ? "" : ep.getProfile(); - String topic = ep == null ? "" : ep.getTopic(); - String json = "{\"operationalError\":true" - + ",\"endpoint\":\"" + escape(endpointName) + "\"" - + ",\"profile\":\"" + escape(profile) + "\"" - + ",\"topic\":\"" + escape(topic) + "\"" - + ",\"phase\":\"" + escape(phase) + "\"" - + ",\"reason\":\"" + escape(reason) + "\"}"; - byte[] raw = json.getBytes(StandardCharsets.UTF_8); - MqttPayload payload = MqttProfileRegistry.operationalErrorPayload( - endpointName, profile, topic, raw, phase, reason); - Map meta = new HashMap<>(); - meta.put("vin", "unknown"); - meta.put("identityResolved", "false"); - meta.put("identitySource", "UNKNOWN"); - meta.put("endpoint", endpointName == null ? "" : endpointName); - meta.put("topic", topic == null ? "" : topic); - meta.put("profile", profile == null ? "" : profile); - meta.put("operationalError", "true"); - meta.put("phase", phase == null ? "" : phase); - meta.put("reason", reason == null ? "" : reason); - dispatcher.dispatch(new RawFrame( - ProtocolId.MQTT_YUTONG, - 0, - 0, - payload, - raw, - meta, - Instant.now())); - } - - private void publishMessageError(MqttInboundProperties.Endpoint ep, String topic, byte[] raw, String reason) { - String endpointName = ep == null ? "" : ep.getName(); - String profile = ep == null ? "" : ep.getProfile(); - byte[] payloadBytes = raw == null ? new byte[0] : raw; - MqttPayload payload = MqttProfileRegistry.operationalErrorPayload( - endpointName, - profile, - topic, - payloadBytes, - "message", - reason); - Map meta = new HashMap<>(); - meta.put("vin", "unknown"); - meta.put("identityResolved", "false"); - meta.put("identitySource", "UNKNOWN"); - meta.put("endpoint", endpointName == null ? "" : endpointName); - meta.put("topic", topic == null ? "" : topic); - meta.put("profile", profile == null ? "" : profile); - meta.put("operationalError", "true"); - meta.put("phase", "message"); - meta.put("reason", reason == null ? "" : reason); - dispatcher.dispatch(new RawFrame( - ProtocolId.MQTT_YUTONG, - 0, - 0, - payload, - payloadBytes, - meta, - Instant.now())); - } - - private static String escape(String value) { - if (value == null) { - return ""; - } - return value.replace("\\", "\\\\").replace("\"", "\\\""); - } - - private static MqttConnectOptions connectOptions(MqttInboundProperties.Endpoint ep) { - MqttConnectOptions options = new MqttConnectOptions(); - options.setAutomaticReconnect(true); - options.setCleanSession(ep.isCleanSession()); - if (ep.getKeepAliveSeconds() > 0) { - options.setKeepAliveInterval(ep.getKeepAliveSeconds()); - } - if (ep.getConnectionTimeoutSeconds() > 0) { - options.setConnectionTimeout(ep.getConnectionTimeoutSeconds()); - } - if (hasText(ep.getUsername())) { - options.setUserName(ep.getUsername()); - options.setPassword(ep.getPassword() == null ? new char[0] : ep.getPassword().toCharArray()); - } - if (isSslUri(ep.getUri()) && hasCustomTls(ep.getTls())) { - options.setSocketFactory(MqttTlsSupport.socketFactory(ep.getTls())); - if (ep.getTls() != null && !ep.getTls().isHostnameVerificationEnabled()) { - options.setSSLHostnameVerifier((hostname, session) -> true); - } - } - return options; - } - - private static String pahoServerUri(String value) { - URI uri = URI.create(value); - String scheme = uri.getScheme(); - String pahoScheme = switch (scheme == null ? "" : scheme.toLowerCase()) { - case "ssl", "tls", "mqtts" -> "ssl"; - default -> "tcp"; - }; - int port = uri.getPort() > 0 ? uri.getPort() : defaultPort(scheme); - return pahoScheme + "://" + uri.getHost() + ":" + port; - } - - private static int mapQos(int level) { - return switch (level) { - case 0 -> 0; - case 2 -> 2; - default -> 1; - }; - } - - private static long timeoutMs(MqttInboundProperties.Endpoint ep) { - int seconds = ep.getConnectionTimeoutSeconds() > 0 ? ep.getConnectionTimeoutSeconds() : 10; - return TimeUnit.SECONDS.toMillis(seconds); - } - - private static int defaultPort(String scheme) { - if (scheme == null) return 1883; - return switch (scheme.toLowerCase()) { - case "ssl", "tls", "mqtts" -> 8883; - default -> 1883; - }; - } - - private static boolean looksLikeVin(String value) { - return value != null && value.trim().length() == 17; - } - - private record IdentityResolution(VehicleIdentity identity, String errorMessage) {} - - private static boolean hasCustomTls(MqttInboundProperties.Tls tls) { - return tls != null && (hasText(tls.getCaPem()) - || (hasText(tls.getClientPem()) && hasText(tls.getClientKey()))); - } - - private static boolean isSslUri(String value) { - String scheme = URI.create(value).getScheme(); - return scheme != null && switch (scheme.toLowerCase()) { - case "ssl", "tls", "mqtts" -> true; - default -> false; - }; - } - - private static boolean hasText(String value) { - return value != null && !value.isBlank(); - } - - @Override - public void close() { - for (MqttAsyncClient c : clients) { - try { - if (c.isConnected()) { - c.disconnectForcibly(1000, 1000, false); - } - c.close(true); - } catch (Exception ignored) { - // best-effort - } - } - log.info("mqtt endpoint manager stopped, clients={}", clients.size()); - } -} diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/client/MqttTlsSupport.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/client/MqttTlsSupport.java deleted file mode 100644 index 2983c7a9..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/client/MqttTlsSupport.java +++ /dev/null @@ -1,260 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.client; - -import com.lingniu.ingest.inbound.mqtt.config.MqttInboundProperties; - -import javax.net.ssl.KeyManager; -import javax.net.ssl.KeyManagerFactory; -import javax.net.ssl.SSLEngine; -import javax.net.ssl.SSLContext; -import javax.net.ssl.SSLParameters; -import javax.net.ssl.SSLSocket; -import javax.net.ssl.SSLSocketFactory; -import javax.net.ssl.TrustManager; -import javax.net.ssl.TrustManagerFactory; -import javax.net.ssl.X509ExtendedTrustManager; -import javax.net.ssl.X509TrustManager; -import java.io.ByteArrayInputStream; -import java.io.IOException; -import java.net.InetAddress; -import java.net.Socket; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Path; -import java.security.KeyFactory; -import java.security.KeyStore; -import java.security.PrivateKey; -import java.security.cert.Certificate; -import java.security.cert.CertificateException; -import java.security.cert.CertificateFactory; -import java.security.cert.X509Certificate; -import java.security.spec.PKCS8EncodedKeySpec; -import java.util.ArrayList; -import java.util.Base64; -import java.util.Collection; -import java.util.List; - -public final class MqttTlsSupport { - - private MqttTlsSupport() {} - - public static SSLSocketFactory socketFactory(MqttInboundProperties.Tls tls) { - try { - KeyManager[] keyManagers = null; - TrustManager[] trustManagers = null; - if (tls != null && hasText(tls.getCaPem())) { - trustManagers = trustManagerFactory(Path.of(tls.getCaPem())).getTrustManagers(); - if (!tls.isHostnameVerificationEnabled()) { - trustManagers = withoutHostnameVerification(trustManagers); - } - } - if (tls != null && hasText(tls.getClientPem()) && hasText(tls.getClientKey())) { - keyManagers = keyManagerFactory(Path.of(tls.getClientPem()), Path.of(tls.getClientKey())).getKeyManagers(); - } - SSLContext context = SSLContext.getInstance("TLS"); - context.init(keyManagers, trustManagers, null); - SSLSocketFactory factory = context.getSocketFactory(); - if (tls != null && !tls.isHostnameVerificationEnabled()) { - return new HostnameVerificationDisablingSocketFactory(factory); - } - return factory; - } catch (Exception e) { - throw new IllegalStateException("mqtt tls config build failed", e); - } - } - - private static TrustManagerFactory trustManagerFactory(Path caPem) throws Exception { - KeyStore trustStore = KeyStore.getInstance(KeyStore.getDefaultType()); - trustStore.load(null, null); - int i = 0; - for (Certificate cert : readCertificates(caPem)) { - trustStore.setCertificateEntry("ca-" + i++, cert); - } - TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm()); - tmf.init(trustStore); - return tmf; - } - - private static KeyManagerFactory keyManagerFactory(Path clientPem, Path clientKey) throws Exception { - List chain = readCertificates(clientPem); - PrivateKey key = readPrivateKey(clientKey); - KeyStore keyStore = KeyStore.getInstance(KeyStore.getDefaultType()); - keyStore.load(null, null); - char[] password = new char[0]; - keyStore.setKeyEntry("mqtt-client", key, password, chain.toArray(Certificate[]::new)); - KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm()); - kmf.init(keyStore, password); - return kmf; - } - - private static List readCertificates(Path pem) throws Exception { - String text = Files.readString(pem, StandardCharsets.UTF_8); - List certificates = new ArrayList<>(); - CertificateFactory factory = CertificateFactory.getInstance("X.509"); - for (String block : pemBlocks(text, "CERTIFICATE")) { - try (ByteArrayInputStream in = new ByteArrayInputStream(Base64.getMimeDecoder().decode(block))) { - certificates.add(factory.generateCertificate(in)); - } - } - if (certificates.isEmpty()) { - throw new IllegalArgumentException("no certificate found in " + pem); - } - return certificates; - } - - private static PrivateKey readPrivateKey(Path pem) throws Exception { - String text = Files.readString(pem, StandardCharsets.UTF_8); - List blocks = pemBlocks(text, "PRIVATE KEY"); - if (blocks.isEmpty()) { - throw new IllegalArgumentException("no PKCS#8 private key found in " + pem); - } - byte[] der = Base64.getMimeDecoder().decode(blocks.getFirst()); - PKCS8EncodedKeySpec spec = new PKCS8EncodedKeySpec(der); - return KeyFactory.getInstance("RSA").generatePrivate(spec); - } - - private static List pemBlocks(String text, String type) { - String begin = "-----BEGIN " + type + "-----"; - String end = "-----END " + type + "-----"; - List blocks = new ArrayList<>(); - int pos = 0; - while (true) { - int start = text.indexOf(begin, pos); - if (start < 0) break; - int contentStart = start + begin.length(); - int stop = text.indexOf(end, contentStart); - if (stop < 0) break; - blocks.add(text.substring(contentStart, stop).replaceAll("\\s+", "")); - pos = stop + end.length(); - } - return blocks; - } - - private static boolean hasText(String value) { - return value != null && !value.isBlank(); - } - - private static TrustManager[] withoutHostnameVerification(TrustManager[] managers) { - TrustManager[] wrapped = new TrustManager[managers.length]; - for (int i = 0; i < managers.length; i++) { - TrustManager manager = managers[i]; - if (manager instanceof X509TrustManager x509) { - wrapped[i] = new ChainOnlyTrustManager(x509); - } else { - wrapped[i] = manager; - } - } - return wrapped; - } - - private static final class ChainOnlyTrustManager extends X509ExtendedTrustManager { - private final X509TrustManager delegate; - - private ChainOnlyTrustManager(X509TrustManager delegate) { - this.delegate = delegate; - } - - @Override - public void checkClientTrusted(X509Certificate[] chain, String authType, Socket socket) - throws CertificateException { - if (delegate instanceof X509ExtendedTrustManager extended) { - extended.checkClientTrusted(chain, authType, socket); - } else { - delegate.checkClientTrusted(chain, authType); - } - } - - @Override - public void checkServerTrusted(X509Certificate[] chain, String authType, Socket socket) - throws CertificateException { - delegate.checkServerTrusted(chain, authType); - } - - @Override - public void checkClientTrusted(X509Certificate[] chain, String authType, SSLEngine engine) - throws CertificateException { - if (delegate instanceof X509ExtendedTrustManager extended) { - extended.checkClientTrusted(chain, authType, engine); - } else { - delegate.checkClientTrusted(chain, authType); - } - } - - @Override - public void checkServerTrusted(X509Certificate[] chain, String authType, SSLEngine engine) - throws CertificateException { - delegate.checkServerTrusted(chain, authType); - } - - @Override - public void checkClientTrusted(X509Certificate[] chain, String authType) throws CertificateException { - delegate.checkClientTrusted(chain, authType); - } - - @Override - public void checkServerTrusted(X509Certificate[] chain, String authType) throws CertificateException { - delegate.checkServerTrusted(chain, authType); - } - - @Override - public X509Certificate[] getAcceptedIssuers() { - return delegate.getAcceptedIssuers(); - } - } - - private static final class HostnameVerificationDisablingSocketFactory extends SSLSocketFactory { - private final SSLSocketFactory delegate; - - private HostnameVerificationDisablingSocketFactory(SSLSocketFactory delegate) { - this.delegate = delegate; - } - - @Override - public String[] getDefaultCipherSuites() { - return delegate.getDefaultCipherSuites(); - } - - @Override - public String[] getSupportedCipherSuites() { - return delegate.getSupportedCipherSuites(); - } - - @Override - public Socket createSocket(Socket socket, String host, int port, boolean autoClose) throws IOException { - return prepare(delegate.createSocket(socket, host, port, autoClose)); - } - - @Override - public Socket createSocket() throws IOException { - return prepare(delegate.createSocket()); - } - - @Override - public Socket createSocket(String host, int port) throws IOException { - return prepare(delegate.createSocket(host, port)); - } - - @Override - public Socket createSocket(String host, int port, InetAddress localHost, int localPort) throws IOException { - return prepare(delegate.createSocket(host, port, localHost, localPort)); - } - - @Override - public Socket createSocket(InetAddress host, int port) throws IOException { - return prepare(delegate.createSocket(host, port)); - } - - @Override - public Socket createSocket(InetAddress address, int port, InetAddress localAddress, int localPort) throws IOException { - return prepare(delegate.createSocket(address, port, localAddress, localPort)); - } - - private static Socket prepare(Socket socket) { - if (socket instanceof SSLSocket sslSocket) { - SSLParameters parameters = sslSocket.getSSLParameters(); - parameters.setEndpointIdentificationAlgorithm(null); - sslSocket.setSSLParameters(parameters); - } - return socket; - } - } -} diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundAutoConfiguration.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundAutoConfiguration.java deleted file mode 100644 index ff1f5f87..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundAutoConfiguration.java +++ /dev/null @@ -1,83 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.config; - -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.inbound.mqtt.client.MqttEndpointManager; -import com.lingniu.ingest.inbound.mqtt.handler.MqttRealtimeHandler; -import com.lingniu.ingest.inbound.mqtt.mapper.YutongEventMapper; -import com.lingniu.ingest.inbound.mqtt.parser.MqttPayloadParser; -import com.lingniu.ingest.inbound.mqtt.profile.MqttProfile; -import com.lingniu.ingest.inbound.mqtt.profile.MqttProfileRegistry; -import com.lingniu.ingest.inbound.mqtt.profile.YutongMqttProfile; -import org.springframework.boot.ApplicationRunner; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Conditional; -import org.springframework.context.annotation.Lazy; - -import java.util.List; - -@AutoConfiguration -@EnableConfigurationProperties(MqttInboundProperties.class) -@Conditional(MqttInboundEnabledCondition.class) -public class MqttInboundAutoConfiguration { - - @Bean - @ConditionalOnMissingBean - public ObjectMapper mqttObjectMapper() { - return new ObjectMapper(); - } - - @Bean - @ConditionalOnMissingBean - public MqttPayloadParser mqttPayloadParser(ObjectMapper objectMapper) { - return new MqttPayloadParser(objectMapper); - } - - @Bean - @ConditionalOnMissingBean - public YutongEventMapper yutongEventMapper(VehicleIdentityResolver identityResolver) { - return new YutongEventMapper(identityResolver); - } - - @Bean - @ConditionalOnMissingBean - public YutongMqttProfile yutongMqttProfile(MqttPayloadParser parser, YutongEventMapper mapper) { - return new YutongMqttProfile(parser, mapper); - } - - @Bean - @ConditionalOnMissingBean - public MqttProfileRegistry mqttProfileRegistry(List profiles) { - return new MqttProfileRegistry(profiles); - } - - @Bean - @ConditionalOnMissingBean - public MqttRealtimeHandler mqttRealtimeHandler(MqttProfileRegistry profileRegistry) { - return new MqttRealtimeHandler(profileRegistry); - } - - @Bean(destroyMethod = "close") - @Lazy(false) - @ConditionalOnMissingBean - public MqttEndpointManager mqttEndpointManager(MqttInboundProperties props, - MqttProfileRegistry profileRegistry, - VehicleIdentityResolver identityResolver, - Dispatcher dispatcher) { - return new MqttEndpointManager(props, profileRegistry, identityResolver, dispatcher); - } - - @Bean - @ConditionalOnProperty(prefix = "lingniu.ingest.mqtt", - name = "auto-startup", - havingValue = "true", - matchIfMissing = true) - public ApplicationRunner mqttEndpointStartupRunner(MqttEndpointManager manager) { - return args -> manager.start(); - } -} diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundEnabledCondition.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundEnabledCondition.java deleted file mode 100644 index 12366dad..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundEnabledCondition.java +++ /dev/null @@ -1,40 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.config; - -import org.springframework.context.annotation.Condition; -import org.springframework.context.annotation.ConditionContext; -import org.springframework.core.env.Environment; -import org.springframework.core.type.AnnotatedTypeMetadata; - -final class MqttInboundEnabledCondition implements Condition { - - @Override - public boolean matches(ConditionContext context, AnnotatedTypeMetadata metadata) { - Environment env = context.getEnvironment(); - Boolean direct = parseBoolean(env.getProperty("lingniu.ingest.mqtt.enabled")); - if (direct != null) { - return direct; - } - return isTrue(env.getProperty("YUTONG_MQTT_ENABLED")); - } - - private static boolean isTrue(String value) { - Boolean parsed = parseBoolean(value); - return parsed != null && parsed; - } - - private static Boolean parseBoolean(String value) { - if (value == null || value.isBlank() || value.contains("${")) { - return null; - } - String normalized = value.trim().toLowerCase(); - if ("true".equals(normalized) || "1".equals(normalized) || "yes".equals(normalized) - || "on".equals(normalized)) { - return true; - } - if ("false".equals(normalized) || "0".equals(normalized) || "no".equals(normalized) - || "off".equals(normalized)) { - return false; - } - return null; - } -} diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundProperties.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundProperties.java deleted file mode 100644 index 5aeced23..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundProperties.java +++ /dev/null @@ -1,93 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.config; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -import java.util.ArrayList; -import java.util.List; - -/** - * MQTT 接入配置。支持多 endpoint（每个 endpoint 可对应一个车厂 / 一个 broker）。 - * - *

这是 JSON/MQTT 厂商入口，不参与 GB32960 TCP 32960 端口接入；如果同一辆车同时存在 - * MQTT 和 32960 数据，需要在下游按 protocol/source 做归因。 - */ -@ConfigurationProperties(prefix = "lingniu.ingest.mqtt") -public class MqttInboundProperties { - - /** 总开关。关闭时不会创建任何 MQTT 客户端连接。 */ - private boolean enabled = false; - /** 是否随 Spring Boot 应用启动立即连接 broker。测试装配时可关闭，生产默认开启。 */ - private boolean autoStartup = true; - /** 多 broker/多 topic 配置；每个 endpoint 独立重连、订阅并转成内部 VehicleEvent。 */ - private List endpoints = new ArrayList<>(); - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public boolean isAutoStartup() { return autoStartup; } - public void setAutoStartup(boolean autoStartup) { this.autoStartup = autoStartup; } - public List getEndpoints() { return endpoints; } - public void setEndpoints(List endpoints) { this.endpoints = endpoints; } - - public static class Endpoint { - /** 逻辑名称，用于指标与日志。 */ - private String name; - /** 对端 URI。形如 ssl://host:port 或 tcp://host:port。 */ - private String uri; - /** 订阅 topic，支持 + / # 通配符。 */ - private String topic; - /** QoS 0/1/2。 */ - private int qos = 1; - private String clientId; - private String username; - private String password; - /** false 表示保留 broker 会话，重连后可继续接收离线期间的 QoS1/2 消息。 */ - private boolean cleanSession = false; - private int keepAliveSeconds = 20; - private int connectionTimeoutSeconds = 10; - /** JSON 字段适配器 profile。默认 yutong。未来可支持 haipote / others。 */ - private String profile = "yutong"; - private Tls tls = new Tls(); - - public String getName() { return name; } - public void setName(String name) { this.name = name; } - public String getUri() { return uri; } - public void setUri(String uri) { this.uri = uri; } - public String getTopic() { return topic; } - public void setTopic(String topic) { this.topic = topic; } - public int getQos() { return qos; } - public void setQos(int qos) { this.qos = qos; } - public String getClientId() { return clientId; } - public void setClientId(String clientId) { this.clientId = clientId; } - public String getUsername() { return username; } - public void setUsername(String username) { this.username = username; } - public String getPassword() { return password; } - public void setPassword(String password) { this.password = password; } - public boolean isCleanSession() { return cleanSession; } - public void setCleanSession(boolean cleanSession) { this.cleanSession = cleanSession; } - public int getKeepAliveSeconds() { return keepAliveSeconds; } - public void setKeepAliveSeconds(int keepAliveSeconds) { this.keepAliveSeconds = keepAliveSeconds; } - public int getConnectionTimeoutSeconds() { return connectionTimeoutSeconds; } - public void setConnectionTimeoutSeconds(int connectionTimeoutSeconds) { this.connectionTimeoutSeconds = connectionTimeoutSeconds; } - public String getProfile() { return profile; } - public void setProfile(String profile) { this.profile = profile; } - public Tls getTls() { return tls; } - public void setTls(Tls tls) { this.tls = tls; } - } - - public static class Tls { - private String caPem; - private String clientPem; - private String clientKey; - private boolean hostnameVerificationEnabled = true; - public String getCaPem() { return caPem; } - public void setCaPem(String caPem) { this.caPem = caPem; } - public String getClientPem() { return clientPem; } - public void setClientPem(String clientPem) { this.clientPem = clientPem; } - public String getClientKey() { return clientKey; } - public void setClientKey(String clientKey) { this.clientKey = clientKey; } - public boolean isHostnameVerificationEnabled() { return hostnameVerificationEnabled; } - public void setHostnameVerificationEnabled(boolean hostnameVerificationEnabled) { - this.hostnameVerificationEnabled = hostnameVerificationEnabled; - } - } -} diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/handler/MqttRealtimeHandler.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/handler/MqttRealtimeHandler.java deleted file mode 100644 index 75b978f8..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/handler/MqttRealtimeHandler.java +++ /dev/null @@ -1,35 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.handler; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.annotation.EventEmit; -import com.lingniu.ingest.api.annotation.MessageMapping; -import com.lingniu.ingest.api.annotation.ProtocolHandler; -import com.lingniu.ingest.api.annotation.RateLimited; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.inbound.mqtt.model.MqttPayload; -import com.lingniu.ingest.inbound.mqtt.profile.MqttProfileRegistry; - -import java.util.List; - -/** - * MQTT 接入 Handler。 - * - *

MQTT 没有"命令字"概念，使用默认的 wildcard 匹配（{@code command} 为空）。 - * 同一 Dispatcher 仍能正确路由，因为协议 ID 做了隔离。 - */ -@ProtocolHandler(protocol = ProtocolId.MQTT_YUTONG) -public class MqttRealtimeHandler { - - private final MqttProfileRegistry profileRegistry; - - public MqttRealtimeHandler(MqttProfileRegistry profileRegistry) { - this.profileRegistry = profileRegistry; - } - - @MessageMapping(desc = "MQTT profile 实时数据") - @RateLimited(perVin = 20) - @EventEmit({VehicleEvent.Realtime.class, VehicleEvent.Location.class, VehicleEvent.Passthrough.class}) - public List onMqtt(MqttPayload payload) { - return profileRegistry.toEvents(payload); - } -} diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/mapper/YutongEventMapper.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/mapper/YutongEventMapper.java deleted file mode 100644 index 01b9e1ef..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/mapper/YutongEventMapper.java +++ /dev/null @@ -1,260 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.mapper; - -import com.fasterxml.jackson.databind.JsonNode; -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.LocationPayload; -import com.lingniu.ingest.api.event.RealtimePayload; -import com.lingniu.ingest.api.event.TelemetryMetadataValues; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.spi.EventMapper; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentityLookup; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.inbound.mqtt.model.MqttPayload; - -import java.time.Instant; -import java.util.ArrayList; -import java.util.Base64; -import java.util.List; -import java.util.Map; -import java.util.UUID; - -/** - * 宇通 MQTT payload → 领域事件。 - * - *

字段名取自旧项目的 {@code ReceptionDataVO}（大写 + 下划线），对字段映射一致性做出显式断言， - * 便于与旧服务双跑对账。 - */ -public final class YutongEventMapper implements EventMapper { - - private final VehicleIdentityResolver identityResolver; - - public YutongEventMapper(VehicleIdentityResolver identityResolver) { - if (identityResolver == null) { - throw new IllegalArgumentException("identityResolver must not be null"); - } - this.identityResolver = identityResolver; - } - - @Override - public List toEvents(MqttPayload payload) { - JsonNode d = payload.data(); - if (d == null || !d.isObject()) return List.of(); - - Instant ingestTime = Instant.now(); - String externalDeviceId = firstNonBlank(payload.deviceId(), payload.vin()); - IdentityResolution identity = resolveIdentity(payload); - String vin = identity.vin(); - Map meta = metadata(payload, externalDeviceId, vin, identity); - - if (d.path("_parseError").asBoolean(false)) { - return List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - vin == null || vin.isBlank() ? "unknown" : vin, - ProtocolId.MQTT_YUTONG, - payload.deviceTime(), - ingestTime, - null, - withMeta(meta, - "parseError", "true", - "parseErrorMessage", d.path("_parseErrorMessage").asText("")), - 0, - rawBytes(d.path("_rawBase64").asText("")))); - } - - Double speed = optDouble(d, "METER_SPEED"); - Double mileage = optDouble(d, "TOTAL_MILEAGE"); - Double soc = optDouble(d, "BATTERY_CAPACITY_SOC"); - Double fcVoltage = optDouble(d, "VOLTAGE_OF_FC"); - Double fcCurrent = optDouble(d, "CURRENT_OF_FC"); - Double fcTemp = optDouble(d, "TEMPT_OF_FC"); - Double leftH2 = optDouble(d, "LEFT_HYDROGEN"); - Double lowH2Pressure = optDouble(d, "HYDROGEN_LOW_PRESSURE"); - Double longitude = optCoord(d, "LONGITUDE"); - Double latitude = optCoord(d, "LATITUDE"); - Double direction = optDouble(d, "GPSDirection"); - Double acc = optDouble(d, "ACC_PEDAL_APT"); - Double brake = optDouble(d, "BRAKEPEDALAPT"); - Integer gear = optInt(d, "ON_GEAR"); - - RealtimePayload realtime = new RealtimePayload( - speed, mileage, soc, null, null, - fcVoltage, fcCurrent, fcTemp, leftH2, null, lowH2Pressure, - null, null, null, gear, acc, brake, - longitude, latitude, null, direction, - null, null); - - List out = new ArrayList<>(2); - out.add(new VehicleEvent.Realtime( - UUID.randomUUID().toString(), - vin, ProtocolId.MQTT_YUTONG, payload.deviceTime(), ingestTime, - null, meta, realtime)); - - if (longitude != null && latitude != null) { - LocationPayload loc = new LocationPayload( - longitude, latitude, 0.0, - speed == null ? 0.0 : speed, - direction == null ? 0.0 : direction, - 0, 0); - Map locationMeta = mileage == null - ? meta - : withMeta(meta, "total_mileage_km", TelemetryMetadataValues.doubleValue(mileage)); - out.add(new VehicleEvent.Location( - UUID.randomUUID().toString(), - vin, ProtocolId.MQTT_YUTONG, payload.deviceTime(), ingestTime, - null, locationMeta, loc)); - } - return out; - } - - private IdentityResolution resolveIdentity(MqttPayload payload) { - if (payload.hasIdentitySnapshot()) { - VehicleIdentitySource source = VehicleIdentitySource.UNKNOWN; - try { - source = VehicleIdentitySource.valueOf(payload.identitySource()); - } catch (IllegalArgumentException ignored) { - // 保守兜底，外部化的 payload 如果带了未知 source 不应影响接收主链路。 - } - return new IdentityResolution( - new VehicleIdentity(payload.resolvedVin(), payload.identityResolved(), source), - payload.identityErrorMessage().isBlank() ? null : payload.identityErrorMessage()); - } - String externalDeviceId = firstNonBlank(payload.deviceId(), payload.vin()); - String externalVin = payload.externalVin() == null ? "" : payload.externalVin(); - String phone = payload.phone() == null ? "" : payload.phone(); - String plateNo = payload.plateNo() == null ? "" : payload.plateNo(); - try { - if (externalDeviceId == null - || externalDeviceId.isBlank() - || "unknown".equalsIgnoreCase(externalDeviceId.trim())) { - if (!externalVin.isBlank() || !phone.isBlank() || !plateNo.isBlank()) { - return IdentityResolution.ok(identityResolver.resolve(new VehicleIdentityLookup( - ProtocolId.MQTT_YUTONG, - externalVin, - phone, - "", - plateNo))); - } - return IdentityResolution.ok(new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN)); - } - VehicleIdentity bound = identityResolver.resolve(new VehicleIdentityLookup( - ProtocolId.MQTT_YUTONG, - externalVin, - phone, - externalDeviceId, - plateNo)); - if (bound.resolved() || !looksLikeVin(externalDeviceId)) { - return IdentityResolution.ok(bound); - } - return IdentityResolution.ok(identityResolver.resolve(new VehicleIdentityLookup( - ProtocolId.MQTT_YUTONG, - externalDeviceId, - phone, - externalDeviceId, - plateNo))); - } catch (RuntimeException e) { - return new IdentityResolution( - new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN), - e.getMessage() == null ? "" : e.getMessage()); - } - } - - private static String firstNonBlank(String... values) { - for (String value : values) { - if (value != null && !value.isBlank()) { - return value; - } - } - return ""; - } - - private static boolean looksLikeVin(String value) { - if (value == null || value.isBlank()) { - return false; - } - String trimmed = value.trim(); - return trimmed.length() == 17; - } - - private static Double optDouble(JsonNode n, String field) { - JsonNode v = n.get(field); - if (v == null || v.isNull() || !v.isNumber()) return null; - return v.asDouble(); - } - - private static Integer optInt(JsonNode n, String field) { - JsonNode v = n.get(field); - if (v == null || v.isNull() || !v.isNumber()) return null; - return v.asInt(); - } - - /** 宇通坐标约定：1e6 整数 或 已经是小数。同时过滤无效值 999999 / 0。 */ - private static Double optCoord(JsonNode n, String field) { - Double v = optDouble(n, field); - if (v == null) return null; - if (v.intValue() == 999_999 || v == 0.0) return null; - if (Math.abs(v) > 1000) return v / 1_000_000.0; - return v; - } - - private static Map withMeta(Map meta, String key, String value) { - java.util.HashMap copy = new java.util.HashMap<>(meta); - copy.put(key, value == null ? "" : value); - return Map.copyOf(copy); - } - - private static Map metadata(MqttPayload payload, - String externalDeviceId, - String vin, - IdentityResolution identity) { - java.util.HashMap meta = new java.util.HashMap<>(); - meta.put("endpoint", payload.endpointName() == null ? "" : payload.endpointName()); - meta.put("topic", payload.topic() == null ? "" : payload.topic()); - meta.put("profile", payload.profile() == null ? "" : payload.profile()); - meta.put("externalVin", payload.externalVin() == null ? "" : payload.externalVin()); - meta.put("phone", payload.phone() == null ? "" : payload.phone()); - meta.put("mqttDeviceId", externalDeviceId == null ? "" : externalDeviceId); - meta.put("plateNo", payload.plateNo() == null ? "" : payload.plateNo()); - meta.put("vin", vin == null || vin.isBlank() ? "unknown" : vin); - meta.put("identityResolved", Boolean.toString(identity.identity().resolved())); - meta.put("identitySource", identity.identity().source().name()); - if (identity.errorMessage() != null) { - meta.put("identityError", "true"); - meta.put("identityErrorMessage", identity.errorMessage()); - } - return Map.copyOf(meta); - } - - private static Map withMeta(Map meta, - String firstKey, - String firstValue, - String secondKey, - String secondValue) { - java.util.HashMap copy = new java.util.HashMap<>(meta); - copy.put(firstKey, firstValue == null ? "" : firstValue); - copy.put(secondKey, secondValue == null ? "" : secondValue); - return Map.copyOf(copy); - } - - private static byte[] rawBytes(String base64) { - if (base64 == null || base64.isBlank()) { - return new byte[0]; - } - try { - return Base64.getDecoder().decode(base64); - } catch (IllegalArgumentException e) { - return new byte[0]; - } - } - - private record IdentityResolution(VehicleIdentity identity, String errorMessage) { - private static IdentityResolution ok(VehicleIdentity identity) { - return new IdentityResolution(identity, null); - } - - private String vin() { - return identity.vin(); - } - } -} diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/model/MqttPayload.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/model/MqttPayload.java deleted file mode 100644 index ba454c64..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/model/MqttPayload.java +++ /dev/null @@ -1,82 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.model; - -import com.fasterxml.jackson.annotation.JsonIgnore; -import com.fasterxml.jackson.databind.JsonNode; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentitySource; - -import java.time.Instant; - -/** - * MQTT 解析后的统一消息体：{@code Dispatcher} 收到的 RawFrame payload 就是这个。 - * - * @param endpointName 来源 endpoint 逻辑名 - * @param profile 字段映射 profile（如 yutong / haipote） - * @param topic 原始 topic - * @param vin VIN（解析自 payload 的 device / vin 字段，保留兼容旧调用） - * @param externalVin 外部 VIN 字段 - * @param phone 外部手机号 / SIM 字段 - * @param deviceId 外部设备号 / 终端 ID / IMEI 字段 - * @param plateNo 外部车牌字段 - * @param resolvedVin 已解析的内部 VIN，供 RAW 和业务事件复用同一次身份查询 - * @param deviceTime 设备上报时刻 - * @param data 原始 JSON 树，供 Mapper 按 profile 提取字段 - */ -public record MqttPayload( - String endpointName, - String profile, - String topic, - String vin, - String externalVin, - String phone, - String deviceId, - String plateNo, - @JsonIgnore String resolvedVin, - @JsonIgnore boolean identityResolved, - @JsonIgnore String identitySource, - @JsonIgnore String identityErrorMessage, - Instant deviceTime, - JsonNode data -) { - public MqttPayload { - resolvedVin = resolvedVin == null ? "" : resolvedVin.trim(); - identitySource = identitySource == null ? "" : identitySource.trim(); - identityErrorMessage = identityErrorMessage == null ? "" : identityErrorMessage.trim(); - } - - public MqttPayload(String endpointName, - String profile, - String topic, - String vin, - String externalVin, - String phone, - String deviceId, - String plateNo, - Instant deviceTime, - JsonNode data) { - this(endpointName, profile, topic, vin, externalVin, phone, deviceId, plateNo, - "", false, "", "", deviceTime, data); - } - - public MqttPayload(String endpointName, - String profile, - String topic, - String vin, - Instant deviceTime, - JsonNode data) { - this(endpointName, profile, topic, vin, "", "", vin, "", deviceTime, data); - } - - public MqttPayload withIdentity(VehicleIdentity identity, String errorMessage) { - VehicleIdentity safe = identity == null - ? new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN) - : identity; - return new MqttPayload(endpointName, profile, topic, vin, externalVin, phone, deviceId, plateNo, - safe.vin(), safe.resolved(), safe.source().name(), errorMessage, deviceTime, data); - } - - @JsonIgnore - public boolean hasIdentitySnapshot() { - return !identitySource.isBlank(); - } -} diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/parser/MqttPayloadParser.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/parser/MqttPayloadParser.java deleted file mode 100644 index bc9bae9e..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/parser/MqttPayloadParser.java +++ /dev/null @@ -1,73 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.parser; - -import com.fasterxml.jackson.databind.JsonNode; -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.api.spi.DecodeException; -import com.lingniu.ingest.inbound.mqtt.model.MqttPayload; - -import java.time.Instant; -import java.time.LocalDateTime; -import java.time.ZoneId; -import java.time.format.DateTimeFormatter; - -/** - * 宇通 MQTT JSON 消息基础解析器（对标旧 {@code VehicleDataReceptionVO} + {@code ReceptionDataVO}）。 - * - *

约定宇通 MQTT 的 payload JSON 形如： - * 

{@code
- * {
- *   "id": "xxx",
- *   "device": "LTEST000000000001",   // VIN
- *   "code": "C001",
- *   "time": "20260413100000",        // yyyyMMddHHmmss
- *   "data": { "METER_SPEED": 52, "LATITUDE": 39916527, ... }
- * }
- * }
- * - *

其它厂商只需新增 {@code MqttProfile} 实现，公共 MQTT endpoint 生命周期不需要修改。 - */ -public final class MqttPayloadParser { - - private static final DateTimeFormatter TIME_FMT = DateTimeFormatter.ofPattern("yyyyMMddHHmmss"); - private static final ZoneId ZONE = ZoneId.of("Asia/Shanghai"); - - private final ObjectMapper mapper; - - public MqttPayloadParser(ObjectMapper mapper) { - this.mapper = mapper; - } - - public MqttPayload parse(String endpointName, String profile, String topic, byte[] bytes) { - try { - JsonNode root = mapper.readTree(bytes); - String externalVin = text(root, "vin", "VIN"); - String deviceId = text(root, "device", "deviceId", "terminalId", "imei", "IMEI"); - String phone = text(root, "sim", "phone", "mobile"); - String plateNo = text(root, "plateNo", "carNo", "plate"); - String vin = !externalVin.isBlank() ? externalVin : deviceId; - Instant deviceTime = parseTime(text(root, "time")); - JsonNode data = root.has("data") ? root.get("data") : root; - return new MqttPayload(endpointName, profile, topic, - vin, externalVin, phone, deviceId, plateNo, deviceTime, data); - } catch (Exception e) { - throw new DecodeException("mqtt payload parse failed: " + e.getMessage(), e); - } - } - - private static String text(JsonNode root, String... keys) { - for (String k : keys) { - JsonNode n = root.get(k); - if (n != null && !n.isNull()) return n.asText(); - } - return ""; - } - - private static Instant parseTime(String raw) { - if (raw == null || raw.isEmpty()) return Instant.now(); - try { - return LocalDateTime.parse(raw, TIME_FMT).atZone(ZONE).toInstant(); - } catch (Exception e) { - return Instant.now(); - } - } -} diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/profile/MqttProfile.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/profile/MqttProfile.java deleted file mode 100644 index 45ddeb85..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/profile/MqttProfile.java +++ /dev/null @@ -1,20 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.profile; - -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.inbound.mqtt.model.MqttPayload; - -import java.util.List; - -/** - * MQTT 厂商 profile 边界。 - * - *

endpoint 只关心 profile 名称，具体 payload 字段结构、VIN/时间提取和事件映射都收敛在 profile 内。 - */ -public interface MqttProfile { - - String name(); - - MqttPayload parse(String endpointName, String topic, byte[] bytes); - - List toEvents(MqttPayload payload); -} diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/profile/MqttProfileRegistry.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/profile/MqttProfileRegistry.java deleted file mode 100644 index e1521b23..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/profile/MqttProfileRegistry.java +++ /dev/null @@ -1,164 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.profile; - -import com.fasterxml.jackson.databind.node.JsonNodeFactory; -import com.fasterxml.jackson.databind.node.ObjectNode; -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.spi.DecodeException; -import com.lingniu.ingest.inbound.mqtt.model.MqttPayload; - -import java.time.Instant; -import java.util.Base64; -import java.util.Collection; -import java.util.HashMap; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Locale; -import java.util.Map; -import java.util.UUID; - -public final class MqttProfileRegistry { - - private final Map profiles; - - public MqttProfileRegistry(Collection profiles) { - Map indexed = new LinkedHashMap<>(); - for (MqttProfile profile : profiles) { - String key = normalize(profile.name()); - if (key.isEmpty()) { - throw new IllegalArgumentException("mqtt profile name must not be blank"); - } - MqttProfile old = indexed.putIfAbsent(key, profile); - if (old != null) { - throw new IllegalArgumentException("duplicated mqtt profile: " + key); - } - } - this.profiles = Map.copyOf(indexed); - } - - public MqttPayload parse(String endpointName, String profileName, String topic, byte[] bytes) { - MqttProfile profile; - try { - profile = resolve(profileName); - } catch (DecodeException e) { - return parseErrorPayload(endpointName, profileName, topic, bytes, e.getMessage(), true); - } - try { - return profile.parse(endpointName, topic, bytes); - } catch (DecodeException e) { - return parseErrorPayload(endpointName, profile.name(), topic, bytes, e.getMessage(), false); - } - } - - public List toEvents(MqttPayload payload) { - if (payload != null && payload.data() != null - && payload.data().path("_operationalError").asBoolean(false)) { - return List.of(operationalErrorEvent(payload)); - } - if (payload != null && payload.data() != null - && payload.data().path("_unsupportedProfile").asBoolean(false)) { - return List.of(parseErrorEvent(payload)); - } - return resolve(payload.profile()).toEvents(payload); - } - - public MqttProfile resolve(String profileName) { - String key = normalize(profileName); - MqttProfile profile = profiles.get(key); - if (profile == null) { - throw new DecodeException("unsupported mqtt profile: " + profileName); - } - return profile; - } - - private static String normalize(String raw) { - return raw == null ? "" : raw.trim().toLowerCase(Locale.ROOT); - } - - private static MqttPayload parseErrorPayload(String endpointName, - String profileName, - String topic, - byte[] bytes, - String message, - boolean unsupportedProfile) { - ObjectNode data = JsonNodeFactory.instance.objectNode(); - data.put("_parseError", true); - data.put("_parseErrorMessage", message == null ? "" : message); - data.put("_unsupportedProfile", unsupportedProfile); - data.put("_rawBase64", Base64.getEncoder().encodeToString(bytes == null ? new byte[0] : bytes)); - return new MqttPayload(endpointName, normalize(profileName), topic, "unknown", Instant.now(), data); - } - - public static MqttPayload operationalErrorPayload(String endpointName, - String profileName, - String topic, - byte[] bytes, - String phase, - String reason) { - ObjectNode data = JsonNodeFactory.instance.objectNode(); - data.put("_operationalError", true); - data.put("_phase", phase == null ? "" : phase); - data.put("_reason", reason == null ? "" : reason); - data.put("_rawBase64", Base64.getEncoder().encodeToString(bytes == null ? new byte[0] : bytes)); - return new MqttPayload(endpointName, normalize(profileName), topic, "unknown", Instant.now(), data); - } - - private static VehicleEvent.Passthrough parseErrorEvent(MqttPayload payload) { - Map metadata = new HashMap<>(); - metadata.put("endpoint", payload.endpointName() == null ? "" : payload.endpointName()); - metadata.put("topic", payload.topic() == null ? "" : payload.topic()); - metadata.put("profile", payload.profile() == null ? "" : payload.profile()); - metadata.put("parseError", "true"); - metadata.put("unsupportedProfile", Boolean.toString(payload.data().path("_unsupportedProfile").asBoolean(false))); - metadata.put("parseErrorMessage", payload.data().path("_parseErrorMessage").asText("")); - addUnknownIdentity(metadata); - return new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - "unknown", - ProtocolId.MQTT_YUTONG, - payload.deviceTime(), - Instant.now(), - null, - Map.copyOf(metadata), - 0, - rawBytes(payload.data().path("_rawBase64").asText(""))); - } - - private static VehicleEvent.Passthrough operationalErrorEvent(MqttPayload payload) { - Map metadata = new HashMap<>(); - metadata.put("endpoint", payload.endpointName() == null ? "" : payload.endpointName()); - metadata.put("topic", payload.topic() == null ? "" : payload.topic()); - metadata.put("profile", payload.profile() == null ? "" : payload.profile()); - metadata.put("operationalError", "true"); - metadata.put("phase", payload.data().path("_phase").asText("")); - metadata.put("reason", payload.data().path("_reason").asText("")); - addUnknownIdentity(metadata); - return new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - "unknown", - ProtocolId.MQTT_YUTONG, - payload.deviceTime(), - Instant.now(), - null, - Map.copyOf(metadata), - 0, - rawBytes(payload.data().path("_rawBase64").asText(""))); - } - - private static void addUnknownIdentity(Map metadata) { - metadata.put("vin", "unknown"); - metadata.put("identityResolved", "false"); - metadata.put("identitySource", "UNKNOWN"); - } - - private static byte[] rawBytes(String base64) { - if (base64 == null || base64.isBlank()) { - return new byte[0]; - } - try { - return Base64.getDecoder().decode(base64); - } catch (IllegalArgumentException e) { - return new byte[0]; - } - } -} diff --git a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/profile/YutongMqttProfile.java b/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/profile/YutongMqttProfile.java deleted file mode 100644 index 426f2efb..00000000 --- a/modules/inbound/inbound-mqtt/src/main/java/com/lingniu/ingest/inbound/mqtt/profile/YutongMqttProfile.java +++ /dev/null @@ -1,36 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.profile; - -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.inbound.mqtt.mapper.YutongEventMapper; -import com.lingniu.ingest.inbound.mqtt.model.MqttPayload; -import com.lingniu.ingest.inbound.mqtt.parser.MqttPayloadParser; - -import java.util.List; - -public final class YutongMqttProfile implements MqttProfile { - - public static final String NAME = "yutong"; - - private final MqttPayloadParser parser; - private final YutongEventMapper mapper; - - public YutongMqttProfile(MqttPayloadParser parser, YutongEventMapper mapper) { - this.parser = parser; - this.mapper = mapper; - } - - @Override - public String name() { - return NAME; - } - - @Override - public MqttPayload parse(String endpointName, String topic, byte[] bytes) { - return parser.parse(endpointName, NAME, topic, bytes); - } - - @Override - public List toEvents(MqttPayload payload) { - return mapper.toEvents(payload); - } -} diff --git a/modules/inbound/inbound-mqtt/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/inbound/inbound-mqtt/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index c8aa81c4..00000000 --- a/modules/inbound/inbound-mqtt/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.inbound.mqtt.config.MqttInboundAutoConfiguration diff --git a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/MqttEndpointManagerTest.java b/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/MqttEndpointManagerTest.java deleted file mode 100644 index e4bfcb10..00000000 --- a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/MqttEndpointManagerTest.java +++ /dev/null @@ -1,473 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt; - -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.sink.EventSink; -import com.lingniu.ingest.core.concurrency.AsyncBatchExecutor; -import com.lingniu.ingest.core.concurrency.DisruptorEventBus; -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.core.dispatcher.HandlerDefinition; -import com.lingniu.ingest.core.dispatcher.HandlerInvoker; -import com.lingniu.ingest.core.dispatcher.HandlerRegistry; -import com.lingniu.ingest.core.pipeline.InterceptorChain; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentityBinding; -import com.lingniu.ingest.identity.VehicleIdentityLookup; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.inbound.mqtt.client.MqttEndpointManager; -import com.lingniu.ingest.inbound.mqtt.config.MqttInboundProperties; -import com.lingniu.ingest.inbound.mqtt.handler.MqttRealtimeHandler; -import com.lingniu.ingest.inbound.mqtt.mapper.YutongEventMapper; -import com.lingniu.ingest.inbound.mqtt.model.MqttPayload; -import com.lingniu.ingest.inbound.mqtt.parser.MqttPayloadParser; -import com.lingniu.ingest.inbound.mqtt.profile.MqttProfileRegistry; -import com.lingniu.ingest.inbound.mqtt.profile.YutongMqttProfile; -import org.junit.jupiter.api.Test; - -import java.lang.reflect.Method; -import java.nio.charset.StandardCharsets; -import java.util.List; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CopyOnWriteArrayList; -import java.util.concurrent.CountDownLatch; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicInteger; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class MqttEndpointManagerTest { - - @Test - void inboundMessageRawArchiveUsesResolvedVehicleIdentity() throws Exception { - MqttInboundProperties.Endpoint endpoint = new MqttInboundProperties.Endpoint(); - endpoint.setName("endpoint-a"); - endpoint.setTopic("/ytforward/#"); - endpoint.setProfile("yutong"); - byte[] payload = """ - {"device":"YT-DEVICE-001","time":"20260622150000","data":{"METER_SPEED":31,"TOTAL_MILEAGE":1024}} - """.trim().getBytes(StandardCharsets.UTF_8); - InMemoryVehicleIdentityService identities = new InMemoryVehicleIdentityService(); - identities.bind(new VehicleIdentityBinding( - ProtocolId.MQTT_YUTONG, - "LNVIN000000MQTT01", - "", - "YT-DEVICE-001", - "")); - RecordingSink sink = new RecordingSink(2); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - MqttProfileRegistry profileRegistry = new MqttProfileRegistry(List.of( - new YutongMqttProfile( - new MqttPayloadParser(new ObjectMapper()), - new YutongEventMapper(identities)))); - Dispatcher dispatcher = new Dispatcher( - registry(profileRegistry), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - MqttEndpointManager manager = new MqttEndpointManager( - new MqttInboundProperties(), profileRegistry, identities, dispatcher); - - invokeOnMessage(manager, endpoint, "/ytforward/vehicle/YT-DEVICE-001", payload); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.vin()).isEqualTo("LNVIN000000MQTT01"); - assertThat(archive.metadata()) - .containsEntry("vin", "LNVIN000000MQTT01") - .containsEntry("mqttDeviceId", "YT-DEVICE-001") - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_DEVICE_ID"); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void inboundMessageReusesResolvedIdentityForRawArchiveAndMappedEvents() throws Exception { - MqttInboundProperties.Endpoint endpoint = new MqttInboundProperties.Endpoint(); - endpoint.setName("endpoint-a"); - endpoint.setTopic("/ytforward/#"); - endpoint.setProfile("yutong"); - byte[] payload = """ - {"device":"YT-DEVICE-001","time":"20260622150000","data":{"METER_SPEED":31,"TOTAL_MILEAGE":1024}} - """.trim().getBytes(StandardCharsets.UTF_8); - CountingIdentityResolver identities = new CountingIdentityResolver(new VehicleIdentity( - "LNVIN000000MQTT01", true, VehicleIdentitySource.BOUND_DEVICE_ID)); - RecordingSink sink = new RecordingSink(2); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - MqttProfileRegistry profileRegistry = new MqttProfileRegistry(List.of( - new YutongMqttProfile( - new MqttPayloadParser(new ObjectMapper()), - new YutongEventMapper(identities)))); - Dispatcher dispatcher = new Dispatcher( - registry(profileRegistry), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - MqttEndpointManager manager = new MqttEndpointManager( - new MqttInboundProperties(), profileRegistry, identities, dispatcher); - - invokeOnMessage(manager, endpoint, "/ytforward/vehicle/YT-DEVICE-001", payload); - - assertThat(sink.await()).isTrue(); - assertThat(identities.calls()).isEqualTo(1); - assertThat(sink.events) - .extracting(VehicleEvent::vin) - .contains("LNVIN000000MQTT01"); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.parsedJson()) - .contains("\"deviceId\":\"YT-DEVICE-001\"") - .doesNotContain("resolvedVin") - .doesNotContain("identityResolved") - .doesNotContain("identitySource") - .doesNotContain("identityErrorMessage"); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void malformedInboundMessageArchivesParseFailureMetadataAndPassthrough() throws Exception { - MqttInboundProperties.Endpoint endpoint = new MqttInboundProperties.Endpoint(); - endpoint.setName("endpoint-a"); - endpoint.setTopic("/ytforward/#"); - endpoint.setProfile("yutong"); - byte[] badPayload = "{bad-json".getBytes(StandardCharsets.UTF_8); - RecordingSink sink = new RecordingSink(2); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - MqttProfileRegistry profileRegistry = new MqttProfileRegistry(List.of( - new YutongMqttProfile( - new MqttPayloadParser(new ObjectMapper()), - new YutongEventMapper(new InMemoryVehicleIdentityService())))); - Dispatcher dispatcher = new Dispatcher( - registry(profileRegistry), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - MqttEndpointManager manager = new MqttEndpointManager( - new MqttInboundProperties(), profileRegistry, new InMemoryVehicleIdentityService(), dispatcher); - - invokeOnMessage(manager, endpoint, "/ytforward/bad", badPayload); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.source()).isEqualTo(ProtocolId.MQTT_YUTONG); - assertThat(archive.rawBytes()).containsExactly(badPayload); - assertThat(archive.metadata()) - .containsEntry("endpoint", "endpoint-a") - .containsEntry("topic", "/ytforward/bad") - .containsEntry("profile", "yutong") - .containsEntry("parseError", "true") - .containsEntry("vin", "unknown"); - assertThat(archive.metadata().get("parseErrorMessage")) - .contains("mqtt payload parse failed"); - }); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(ProtocolId.MQTT_YUTONG); - assertThat(passthrough.data()).containsExactly(badPayload); - assertThat(passthrough.metadata()) - .containsEntry("parseError", "true") - .containsEntry("endpoint", "endpoint-a") - .containsEntry("profile", "yutong"); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void endpointInitializationFailureIsArchivedAndDispatchedAsPassthrough() throws Exception { - MqttInboundProperties props = new MqttInboundProperties(); - MqttInboundProperties.Endpoint endpoint = new MqttInboundProperties.Endpoint(); - endpoint.setName("bad-endpoint"); - endpoint.setUri("://bad-uri"); - endpoint.setTopic("/vehicles/#"); - endpoint.setProfile("yutong"); - props.setEndpoints(List.of(endpoint)); - RecordingSink sink = new RecordingSink(2); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - MqttProfileRegistry profileRegistry = new MqttProfileRegistry(List.of()); - Dispatcher dispatcher = new Dispatcher( - registry(profileRegistry), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - MqttEndpointManager manager = new MqttEndpointManager( - props, profileRegistry, new InMemoryVehicleIdentityService(), dispatcher); - - manager.start(); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.source()).isEqualTo(ProtocolId.MQTT_YUTONG); - assertThat(archive.metadata()) - .containsEntry("endpoint", "bad-endpoint") - .containsEntry("profile", "yutong") - .containsEntry("operationalError", "true") - .containsEntry("phase", "init") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - }); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(ProtocolId.MQTT_YUTONG); - assertThat(passthrough.metadata()) - .containsEntry("operationalError", "true") - .containsEntry("endpoint", "bad-endpoint") - .containsEntry("profile", "yutong") - .containsEntry("phase", "init"); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void profileRuntimeFailureIsArchivedAndDispatchedAsPassthrough() throws Exception { - MqttInboundProperties.Endpoint endpoint = new MqttInboundProperties.Endpoint(); - endpoint.setName("endpoint-a"); - endpoint.setTopic("/vehicles/#"); - endpoint.setProfile("broken"); - byte[] payload = "{\"device\":\"YT-DEVICE-001\"}".getBytes(StandardCharsets.UTF_8); - RecordingSink sink = new RecordingSink(2); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - MqttProfileRegistry profileRegistry = new MqttProfileRegistry(List.of(new BrokenProfile())); - Dispatcher dispatcher = new Dispatcher( - registry(profileRegistry), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - MqttEndpointManager manager = new MqttEndpointManager( - new MqttInboundProperties(), profileRegistry, new InMemoryVehicleIdentityService(), dispatcher); - - invokeOnMessage(manager, endpoint, "/vehicles/broken", payload); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.source()).isEqualTo(ProtocolId.MQTT_YUTONG); - assertThat(archive.rawBytes()).containsExactly(payload); - assertThat(archive.metadata()) - .containsEntry("endpoint", "endpoint-a") - .containsEntry("topic", "/vehicles/broken") - .containsEntry("profile", "broken") - .containsEntry("operationalError", "true") - .containsEntry("phase", "message") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - assertThat(archive.metadata().get("reason")).contains("profile blew up"); - }); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(ProtocolId.MQTT_YUTONG); - assertThat(passthrough.data()).containsExactly(payload); - assertThat(passthrough.metadata()) - .containsEntry("operationalError", "true") - .containsEntry("endpoint", "endpoint-a") - .containsEntry("profile", "broken") - .containsEntry("phase", "message"); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void rejectsMissingIdentityResolver() throws Exception { - MqttProfileRegistry profileRegistry = new MqttProfileRegistry(List.of()); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of()); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - registry(profileRegistry), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - - assertThatThrownBy(() -> new MqttEndpointManager( - new MqttInboundProperties(), profileRegistry, null, dispatcher)) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("identityResolver"); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void identityResolverFailureStillArchivesAndDispatchesRealtimeWithUnknownIdentity() throws Exception { - MqttInboundProperties.Endpoint endpoint = new MqttInboundProperties.Endpoint(); - endpoint.setName("endpoint-a"); - endpoint.setTopic("/ytforward/#"); - endpoint.setProfile("yutong"); - byte[] payload = """ - {"device":"YT-DEVICE-001","time":"20260622150000","data":{"METER_SPEED":31,"TOTAL_MILEAGE":1024}} - """.trim().getBytes(StandardCharsets.UTF_8); - var failingIdentity = (com.lingniu.ingest.identity.VehicleIdentityResolver) lookup -> { - throw new IllegalStateException("identity backend unavailable"); - }; - RecordingSink sink = new RecordingSink(2); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - MqttProfileRegistry profileRegistry = new MqttProfileRegistry(List.of( - new YutongMqttProfile( - new MqttPayloadParser(new ObjectMapper()), - new YutongEventMapper(failingIdentity)))); - Dispatcher dispatcher = new Dispatcher( - registry(profileRegistry), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - MqttEndpointManager manager = new MqttEndpointManager( - new MqttInboundProperties(), profileRegistry, failingIdentity, dispatcher); - - invokeOnMessage(manager, endpoint, "/ytforward/vehicle/YT-DEVICE-001", payload); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.source()).isEqualTo(ProtocolId.MQTT_YUTONG); - assertThat(archive.vin()).isEqualTo("unknown"); - assertThat(archive.rawBytes()).containsExactly(payload); - assertThat(archive.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("mqttDeviceId", "YT-DEVICE-001") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN") - .containsEntry("identityError", "true") - .containsEntry("identityErrorMessage", "identity backend unavailable"); - }); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Realtime.class); - assertThat(event.vin()).isEqualTo("unknown"); - assertThat(event.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("mqttDeviceId", "YT-DEVICE-001") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN") - .containsEntry("identityError", "true") - .containsEntry("identityErrorMessage", "identity backend unavailable"); - }); - - batchExecutor.close(); - eventBus.close(); - } - - private static HandlerRegistry registry(MqttProfileRegistry profileRegistry) throws NoSuchMethodException { - HandlerRegistry registry = new HandlerRegistry(); - MqttRealtimeHandler handler = new MqttRealtimeHandler(profileRegistry); - registry.register(new HandlerDefinition( - ProtocolId.MQTT_YUTONG, - 0, - 0, - "test mqtt", - handler, - MqttRealtimeHandler.class.getMethod("onMqtt", MqttPayload.class), - MqttPayload.class, - null, - null, - null)); - return registry; - } - - private static void invokeOnMessage(MqttEndpointManager manager, - MqttInboundProperties.Endpoint endpoint, - String topic, - byte[] payload) throws Exception { - Method method = MqttEndpointManager.class.getDeclaredMethod( - "onMessage", MqttInboundProperties.Endpoint.class, String.class, byte[].class); - method.setAccessible(true); - method.invoke(manager, endpoint, topic, payload); - } - - private static final class RecordingSink implements EventSink { - private final List events = new CopyOnWriteArrayList<>(); - private final CountDownLatch latch; - - private RecordingSink(int expectedEvents) { - this.latch = new CountDownLatch(expectedEvents); - } - - @Override - public String name() { - return "recording"; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - events.add(event); - latch.countDown(); - return CompletableFuture.completedFuture(null); - } - - private boolean await() throws InterruptedException { - return latch.await(3, TimeUnit.SECONDS); - } - } - - private static final class CountingIdentityResolver implements VehicleIdentityResolver { - private final VehicleIdentity identity; - private final AtomicInteger calls = new AtomicInteger(); - - private CountingIdentityResolver(VehicleIdentity identity) { - this.identity = identity; - } - - @Override - public VehicleIdentity resolve(VehicleIdentityLookup lookup) { - calls.incrementAndGet(); - return identity; - } - - private int calls() { - return calls.get(); - } - } - - private static final class BrokenProfile implements com.lingniu.ingest.inbound.mqtt.profile.MqttProfile { - @Override - public String name() { - return "broken"; - } - - @Override - public MqttPayload parse(String endpointName, String topic, byte[] bytes) { - throw new IllegalStateException("profile blew up"); - } - - @Override - public List toEvents(MqttPayload payload) { - return List.of(); - } - } -} diff --git a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/MqttProfileRegistryTest.java b/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/MqttProfileRegistryTest.java deleted file mode 100644 index c6d8add3..00000000 --- a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/MqttProfileRegistryTest.java +++ /dev/null @@ -1,139 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt; - -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.inbound.mqtt.mapper.YutongEventMapper; -import com.lingniu.ingest.inbound.mqtt.parser.MqttPayloadParser; -import com.lingniu.ingest.inbound.mqtt.profile.MqttProfileRegistry; -import com.lingniu.ingest.inbound.mqtt.profile.YutongMqttProfile; -import org.junit.jupiter.api.Test; - -import java.nio.charset.StandardCharsets; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class MqttProfileRegistryTest { - - private final MqttProfileRegistry registry = new MqttProfileRegistry(List.of( - new YutongMqttProfile(new MqttPayloadParser(new ObjectMapper()), - new YutongEventMapper(new InMemoryVehicleIdentityService())))); - - @Test - void resolvesProfileCaseInsensitiveAndRoutesParseAndMapping() { - byte[] payload = """ - { - "device": "LTEST000000000001", - "time": "20260413100000", - "data": { - "METER_SPEED": 52.3, - "TOTAL_MILEAGE": 123456.7, - "BATTERY_CAPACITY_SOC": 70, - "LONGITUDE": 116397128, - "LATITUDE": 39916527 - } - } - """.getBytes(StandardCharsets.UTF_8); - - var parsed = registry.parse("endpoint-a", "YuTong", "/ytforward/shln/dev1", payload); - - assertThat(parsed.profile()).isEqualTo("yutong"); - assertThat(parsed.vin()).isEqualTo("LTEST000000000001"); - assertThat(registry.toEvents(parsed)) - .extracting(VehicleEvent::vin) - .containsOnly("LTEST000000000001"); - } - - @Test - void parserKeepsCommonExternalIdentityFieldsForColdStorageAndMapping() { - byte[] payload = """ - { - "vin": "LVIN0000000000001", - "terminalId": "terminal-001", - "sim": "13800138000", - "plateNo": "粤B12345", - "time": "20260413100000", - "data": { "METER_SPEED": 12.3 } - } - """.getBytes(StandardCharsets.UTF_8); - - var parsed = registry.parse("endpoint-a", "yutong", "/ytforward/shln/terminal-001", payload); - - assertThat(parsed.externalVin()).isEqualTo("LVIN0000000000001"); - assertThat(parsed.deviceId()).isEqualTo("terminal-001"); - assertThat(parsed.phone()).isEqualTo("13800138000"); - assertThat(parsed.plateNo()).isEqualTo("粤B12345"); - } - - @Test - void unsupportedProfileBecomesPassthroughEventForOperationalTraceability() { - byte[] payload = "{\"device\":\"D1\"}".getBytes(StandardCharsets.UTF_8); - - var parsed = registry.parse("endpoint-a", "unknown", "/x", payload); - - assertThat(parsed.profile()).isEqualTo("unknown"); - assertThat(parsed.data().path("_parseError").asBoolean()).isTrue(); - assertThat(parsed.data().path("_unsupportedProfile").asBoolean()).isTrue(); - assertThat(registry.toEvents(parsed)).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.data()).containsExactly(payload); - assertThat(passthrough.metadata()) - .containsEntry("parseError", "true") - .containsEntry("unsupportedProfile", "true") - .containsEntry("profile", "unknown") - .containsEntry("vin", "unknown") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - }); - } - - @Test - void malformedPayloadBecomesPassthroughEventForOperationalTraceability() { - byte[] payload = "{bad-json".getBytes(StandardCharsets.UTF_8); - - var parsed = registry.parse("endpoint-a", "yutong", "/ytforward/bad", payload); - - assertThat(parsed.vin()).isEqualTo("unknown"); - assertThat(parsed.data().path("_parseError").asBoolean()).isTrue(); - assertThat(registry.toEvents(parsed)).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(com.lingniu.ingest.api.ProtocolId.MQTT_YUTONG); - assertThat(passthrough.data()).containsExactly(payload); - assertThat(passthrough.metadata()) - .containsEntry("parseError", "true") - .containsEntry("vin", "unknown") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - }); - } - - @Test - void operationalPayloadBecomesPassthroughEventWithFailureMetadata() { - byte[] payload = """ - {"operationalError":true,"endpoint":"endpoint-a","phase":"init","reason":"bad uri"} - """.getBytes(StandardCharsets.UTF_8); - - var parsed = MqttProfileRegistry.operationalErrorPayload( - "endpoint-a", "yutong", "/operational", payload, "init", "bad uri"); - - assertThat(registry.toEvents(parsed)).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(com.lingniu.ingest.api.ProtocolId.MQTT_YUTONG); - assertThat(passthrough.vin()).isEqualTo("unknown"); - assertThat(passthrough.data()).containsExactly(payload); - assertThat(passthrough.metadata()) - .containsEntry("operationalError", "true") - .containsEntry("endpoint", "endpoint-a") - .containsEntry("profile", "yutong") - .containsEntry("phase", "init") - .containsEntry("reason", "bad uri") - .containsEntry("vin", "unknown") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - }); - } -} diff --git a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/MqttTlsSupportTest.java b/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/MqttTlsSupportTest.java deleted file mode 100644 index 1e9ab16e..00000000 --- a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/MqttTlsSupportTest.java +++ /dev/null @@ -1,101 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt; - -import com.lingniu.ingest.inbound.mqtt.client.MqttTlsSupport; -import com.lingniu.ingest.inbound.mqtt.config.MqttInboundProperties; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import javax.net.ssl.SSLSocketFactory; -import java.nio.file.Files; -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; - -class MqttTlsSupportTest { - - @TempDir - Path tempDir; - - @Test - void buildsTrustManagerFromCaPem() throws Exception { - Path ca = tempDir.resolve("ca.pem"); - Files.writeString(ca, CERT); - MqttInboundProperties.Tls tls = new MqttInboundProperties.Tls(); - tls.setCaPem(ca.toString()); - - SSLSocketFactory factory = MqttTlsSupport.socketFactory(tls); - - assertThat(factory).isNotNull(); - } - - @Test - void buildsMutualTlsFromClientCertificateAndKey() throws Exception { - Path ca = tempDir.resolve("ca.pem"); - Path cert = tempDir.resolve("client.pem"); - Path key = tempDir.resolve("client.key"); - Files.writeString(ca, CERT); - Files.writeString(cert, CERT); - Files.writeString(key, KEY); - MqttInboundProperties.Tls tls = new MqttInboundProperties.Tls(); - tls.setCaPem(ca.toString()); - tls.setClientPem(cert.toString()); - tls.setClientKey(key.toString()); - - SSLSocketFactory factory = MqttTlsSupport.socketFactory(tls); - - assertThat(factory).isNotNull(); - } - - private static final String CERT = """ - -----BEGIN CERTIFICATE----- - MIIC/zCCAeegAwIBAgIUPg+j3PAaXVWYykrese0Ssb/blk4wDQYJKoZIhvcNAQEL - BQAwDzENMAsGA1UEAwwEdGVzdDAeFw0yNjA2MjIwNTA0MzNaFw0yNjA2MjMwNTA0 - MzNaMA8xDTALBgNVBAMMBHRlc3QwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEK - AoIBAQChuMSubmmNY13b4PUGBoUO+EIy074ZfbRMPHCGJhfSnZVWdETBflOe3kNy - ouejxN2eT3Xoqf7slqTPySJo8hZQjukORNtudMPWQkiBrvaWPCttlYtCxpxhxRw0 - VnIo/jYP2Fmwt2buHV//Kn039amDDr3auZWVXEIGLcafVpd/3v7c+1NLbMPR/uK8 - 62Yk6UG1rlvOYBIttNp2Y9fGZLkMLA5fE3xsPbg8VeWuN9/xjPtZGZ5qUYHaQn3Z - DzmZ8U9q5TGCbcpoVEIooRSb2lIILAqw2FRAR7ashCcJf6QzPVdhoZ8QIovJoViB - nzuFP0I4VYp9fT1pRmBYjcZtIsSfAgMBAAGjUzBRMB0GA1UdDgQWBBSzG5+aGFJr - qgnVlOmtdz8iGzU5RjAfBgNVHSMEGDAWgBSzG5+aGFJrqgnVlOmtdz8iGzU5RjAP - BgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQB7/6Q1e+acJobo/wnW - 9DiGzNaownFRs4yz8EgIYVcc7iROjBrVFrcORJAcFrg1uh0Q4rFcslMmEqFM8ryV - IJ1/L6PcQUp4QlKGbbc9LarhalEdFxhX8IAIp2QCNaG8h9/WczZ5/TczwODfy1tW - BFoLG19/dXd5HXLqxiLIpez2f2a/bbUkTu1Yj9QQUFUX2r6SO6LOWmZQsfboq040 - oeJhDzhL218ze1t7qhBk4rbju8NLFwtdhWZ8M7JBzlj5PfACBewd0eosFtb1Tz3N - eAZn4M/GkzBh4t1M55Jt+GwYBD2A7trDRUCUQxtJ1QVzr2gbIB/FfoGvP0qnE4gk - R3PW - -----END CERTIFICATE----- - """; - - private static final String KEY = """ - -----BEGIN PRIVATE KEY----- - MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQChuMSubmmNY13b - 4PUGBoUO+EIy074ZfbRMPHCGJhfSnZVWdETBflOe3kNyouejxN2eT3Xoqf7slqTP - ySJo8hZQjukORNtudMPWQkiBrvaWPCttlYtCxpxhxRw0VnIo/jYP2Fmwt2buHV// - Kn039amDDr3auZWVXEIGLcafVpd/3v7c+1NLbMPR/uK862Yk6UG1rlvOYBIttNp2 - Y9fGZLkMLA5fE3xsPbg8VeWuN9/xjPtZGZ5qUYHaQn3ZDzmZ8U9q5TGCbcpoVEIo - oRSb2lIILAqw2FRAR7ashCcJf6QzPVdhoZ8QIovJoViBnzuFP0I4VYp9fT1pRmBY - jcZtIsSfAgMBAAECggEAQu+cbIwjoRM3NnpqQAezzAniMHJmlNtsJD/B3SxoINL7 - jDCMgr/cMX3SUeDuWmDxz4QZC+dMrbT+W0hnNyO4K7iy6qaCYjnvEsAVjaOSyYT2 - /qDuZoGZGXiBn4IGN0RcsPs9yEBo2HaNFKqL8Hz8H9QarayxpoPsie0pcCrhgtlr - fsVN2AMhbEU15SpD0nkjWkqG51m1EY36seSzAwQod2UasD+eNdZ89a2o849cfuA3 - 1F1TyghVIm0RmAdC1yWuud8YY4xvtpfy1+3RjShW0jgA0nVHfP2UgYfBAJ8+TFGy - +hVCgnnt/eMQ1fCszwLkq5spRcqNr4oOVcm+nR718QKBgQDSvEXaaQgqhARfyN/c - /1qFgOBT34a7pEJXotEC6aDV3YxivAt2jDB4TC0NVS5IGP9NYm4oi0Fiw1v0V5eZ - n8nmq0gkfBjW1y37U4mP74QyKY9rwyjyjjsYqr/aTS8e2ZY4QWmWxm9y5a3Q0Ifu - UwzLD5eqiXNLNvvNZoggo+dVsQKBgQDEdWAntCdTZEYO3fp4/ygCJisPeFgy/0Gw - GsgHCkWR4ZyzmBmFSvf3Es3MmcH3QgkfIROCOMaZ0YcGCLzr/RMdYkgpa9DvhJnG - WZDtumxQAW3G3moAYu2r2wXfOjgsMBDlRBH0mBWTBjpovbWBs1NMFfxsetQYY7zG - aQGLDPJDTwKBgQCD6QQUrlBFRLP0PSocDN9d2AkTl0SgKja44prQputdU7vvhePr - Bd/FPXGp+drpmHQevXFVAa4hI0ZpEXc822+naynSZLerq7AFtQnTxkrKl4dGHjiA - dBV74E4NWOkY93x3pEJy9a2Hj0uY/R9JSEUmypDWWAmKWFWQAhFN1SsWUQKBgD7e - xTfPimpAg78MQLTqCvatGkioHamsUGw4Fd1S5zKpPcmnmjsy46nZBa09Y3pqUpr4 - rdKVstDU4d4He9YVtkFIC4nd7A5KpB962EuLxk/QNT5YPRoEjsTZocZvTjyt4SpN - n2VkKjtT2etdErIAHl8SBib9I9TuTiI8xnamXP03AoGATMdUyehOBqG3+YlS31c8 - z9vcIn2egPOjFCWK64FQIuwpmVU5sx6aYIRDb3/jxYqYCaMRcnMSSp/Mrn2yBnui - oRry31MaC/aTSumn/r6WMz7FEYrbsfr9//bYr+tiTrtimWAe4yTwrqFve1Bw2bOQ - P16PTZ2/SMJDmiLR1MIkCns= - -----END PRIVATE KEY----- - """; -} diff --git a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/YutongEventMapperTest.java b/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/YutongEventMapperTest.java deleted file mode 100644 index a2aa3702..00000000 --- a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/YutongEventMapperTest.java +++ /dev/null @@ -1,235 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt; - -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.identity.VehicleIdentityBinding; -import com.lingniu.ingest.inbound.mqtt.mapper.YutongEventMapper; -import com.lingniu.ingest.inbound.mqtt.model.MqttPayload; -import com.lingniu.ingest.inbound.mqtt.parser.MqttPayloadParser; -import org.junit.jupiter.api.Test; - -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class YutongEventMapperTest { - - private final ObjectMapper objectMapper = new ObjectMapper(); - private final MqttPayloadParser parser = new MqttPayloadParser(objectMapper); - private final YutongEventMapper mapper = new YutongEventMapper(new InMemoryVehicleIdentityService()); - - @Test - void rejectsMissingIdentityResolver() { - assertThatThrownBy(() -> new YutongEventMapper(null)) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("identityResolver"); - } - - @Test - void parsesYutongSamplePayload() { - String json = """ - { - "id": "abc", - "device": "LTEST000000000001", - "code": "C1", - "time": "20260413100000", - "data": { - "METER_SPEED": 52.3, - "TOTAL_MILEAGE": 123456.7, - "BATTERY_CAPACITY_SOC": 70, - "LONGITUDE": 116397128, - "LATITUDE": 39916527, - "GPSDirection": 90, - "LEFT_HYDROGEN": 5.5, - "VOLTAGE_OF_FC": 600.0, - "CURRENT_OF_FC": 120.0, - "TEMPT_OF_FC": 65.0, - "ON_GEAR": 3, - "ACC_PEDAL_APT": 25, - "BRAKEPEDALAPT": 0 - } - } - """; - - MqttPayload payload = parser.parse("yutong", "yutong", "/ytforward/shln/dev1", json.getBytes()); - assertThat(payload.vin()).isEqualTo("LTEST000000000001"); - - List events = mapper.toEvents(payload); - assertThat(events).hasSize(2); - - VehicleEvent.Realtime rt = (VehicleEvent.Realtime) events.stream() - .filter(e -> e instanceof VehicleEvent.Realtime).findFirst().orElseThrow(); - assertThat(rt.source()).isEqualTo(ProtocolId.MQTT_YUTONG); - assertThat(rt.vin()).isEqualTo("LTEST000000000001"); - assertThat(rt.payload().speedKmh()).isEqualTo(52.3); - assertThat(rt.payload().batterySoc()).isEqualTo(70.0); - assertThat(rt.payload().longitude()).isEqualTo(116.397128, org.assertj.core.data.Offset.offset(0.000001)); - assertThat(rt.payload().latitude()).isEqualTo(39.916527, org.assertj.core.data.Offset.offset(0.000001)); - assertThat(rt.payload().hydrogenRemainingKg()).isEqualTo(5.5); - - VehicleEvent.Location loc = (VehicleEvent.Location) events.stream() - .filter(e -> e instanceof VehicleEvent.Location).findFirst().orElseThrow(); - assertThat(loc.payload().longitude()).isEqualTo(116.397128, org.assertj.core.data.Offset.offset(0.000001)); - assertThat(loc.metadata()).containsEntry("total_mileage_km", "123456.7"); - } - - @Test - void resolvesVinThroughSharedIdentityServiceWhenDeviceIsAnExternalId() throws Exception { - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - identity.bind(new VehicleIdentityBinding( - ProtocolId.MQTT_YUTONG, - "VIN-INTERNAL-001", - "", - "mqtt-device-001", - "")); - YutongEventMapper mapperWithIdentity = new YutongEventMapper(identity); - - MqttPayload payload = new MqttPayload( - "yutong", - "yutong", - "/ytforward/shln/mqtt-device-001", - "mqtt-device-001", - java.time.Instant.parse("2026-06-22T08:00:00Z"), - objectMapper.readTree(""" - { - "METER_SPEED": 18.5, - "TOTAL_MILEAGE": 99.1 - } - """)); - - List events = mapperWithIdentity.toEvents(payload); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event.vin()).isEqualTo("VIN-INTERNAL-001"); - assertThat(event.metadata()) - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_DEVICE_ID") - .containsEntry("mqttDeviceId", "mqtt-device-001"); - }); - } - - @Test - void resolvesVinThroughSharedIdentityServiceWhenPayloadUsesTerminalId() { - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - identity.bind(new VehicleIdentityBinding( - ProtocolId.MQTT_YUTONG, - "VIN-INTERNAL-TID", - "", - "terminal-001", - "")); - YutongEventMapper mapperWithIdentity = new YutongEventMapper(identity); - String json = """ - { - "terminalId": "terminal-001", - "time": "20260413100000", - "data": { - "METER_SPEED": 18.5, - "TOTAL_MILEAGE": 99.1 - } - } - """; - - MqttPayload payload = parser.parse( - "yutong", "yutong", "/ytforward/shln/terminal-001", json.getBytes()); - List events = mapperWithIdentity.toEvents(payload); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event.vin()).isEqualTo("VIN-INTERNAL-TID"); - assertThat(event.metadata()) - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_DEVICE_ID") - .containsEntry("mqttDeviceId", "terminal-001"); - }); - } - - @Test - void prefersBoundDeviceIdentityOverVinLengthGuess() throws Exception { - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - identity.bind(new VehicleIdentityBinding( - ProtocolId.MQTT_YUTONG, - "VIN-INTERNAL-017", - "", - "DEVICEID000000000", - "")); - YutongEventMapper mapperWithIdentity = new YutongEventMapper(identity); - - MqttPayload payload = new MqttPayload( - "yutong", - "yutong", - "/ytforward/shln/DEVICEID000000000", - "DEVICEID000000000", - java.time.Instant.parse("2026-06-22T08:00:00Z"), - objectMapper.readTree("{\"METER_SPEED\":18.5}")); - - List events = mapperWithIdentity.toEvents(payload); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event.vin()).isEqualTo("VIN-INTERNAL-017"); - assertThat(event.metadata()).containsEntry("identitySource", "BOUND_DEVICE_ID"); - }); - } - - @Test - void identityResolverFailureStillProducesRealtimeEventWithUnknownVin() throws Exception { - YutongEventMapper mapperWithFailingIdentity = new YutongEventMapper(lookup -> { - throw new IllegalStateException("identity backend unavailable"); - }); - MqttPayload payload = new MqttPayload( - "yutong", - "yutong", - "/ytforward/shln/mqtt-device-001", - "mqtt-device-001", - java.time.Instant.parse("2026-06-22T08:00:00Z"), - objectMapper.readTree(""" - { - "METER_SPEED": 18.5, - "TOTAL_MILEAGE": 99.1 - } - """)); - - List events = mapperWithFailingIdentity.toEvents(payload); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Realtime.class); - assertThat(event.vin()).isEqualTo("unknown"); - assertThat(event.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("mqttDeviceId", "mqtt-device-001") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN") - .containsEntry("identityError", "true") - .containsEntry("identityErrorMessage", "identity backend unavailable"); - }); - } - - @Test - void parseErrorPayloadKeepsDiagnosticMessageOnPassthroughEvent() throws Exception { - MqttPayload payload = new MqttPayload( - "endpoint-a", - "yutong", - "/ytforward/shln/bad-device", - "unknown", - java.time.Instant.parse("2026-06-22T08:00:00Z"), - objectMapper.readTree(""" - { - "_parseError": true, - "_parseErrorMessage": "Unexpected character at byte 1", - "_rawBase64": "e2JhZC1qc29u" - } - """)); - - List events = mapper.toEvents(payload); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.data()).containsExactly("{bad-json".getBytes(java.nio.charset.StandardCharsets.UTF_8)); - assertThat(event.metadata()) - .containsEntry("parseError", "true") - .containsEntry("parseErrorMessage", "Unexpected character at byte 1"); - }); - } -} diff --git a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundAutoConfigurationTest.java b/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundAutoConfigurationTest.java deleted file mode 100644 index da622ffc..00000000 --- a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundAutoConfigurationTest.java +++ /dev/null @@ -1,36 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.config; - -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.inbound.mqtt.profile.MqttProfileRegistry; -import org.junit.jupiter.api.Test; -import org.springframework.core.env.MapPropertySource; -import org.springframework.core.env.StandardEnvironment; - -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class MqttInboundAutoConfigurationTest { - - @Test - void doesNotCreateEndpointFromLegacySingleEndpointEnvironmentKeys() { - MqttInboundProperties props = new MqttInboundProperties(); - StandardEnvironment environment = new StandardEnvironment(); - environment.getPropertySources().addFirst(new MapPropertySource("test", Map.of( - "MQTT_URI", "tcp://127.0.0.1:1883", - "MQTT_TOPIC", "/legacy/#", - "YUTONG_MQTT_ENDPOINT_NAME", "legacy-yutong"))); - - new MqttInboundAutoConfiguration().mqttEndpointManager( - props, - new MqttProfileRegistry(List.of()), - lookup -> new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN), - new Dispatcher(null, null, null, null, null)); - - assertThat(props.getEndpoints()).isEmpty(); - assertThat(environment.getProperty("MQTT_URI")).isEqualTo("tcp://127.0.0.1:1883"); - } -} diff --git a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundEnabledConditionTest.java b/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundEnabledConditionTest.java deleted file mode 100644 index 2379fe69..00000000 --- a/modules/inbound/inbound-mqtt/src/test/java/com/lingniu/ingest/inbound/mqtt/config/MqttInboundEnabledConditionTest.java +++ /dev/null @@ -1,68 +0,0 @@ -package com.lingniu.ingest.inbound.mqtt.config; - -import org.junit.jupiter.api.Test; -import org.springframework.beans.factory.config.ConfigurableListableBeanFactory; -import org.springframework.beans.factory.support.BeanDefinitionRegistry; -import org.springframework.context.annotation.ConditionContext; -import org.springframework.core.env.MapPropertySource; -import org.springframework.core.env.StandardEnvironment; -import org.springframework.core.io.ResourceLoader; - -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class MqttInboundEnabledConditionTest { - - private final MqttInboundEnabledCondition condition = new MqttInboundEnabledCondition(); - - @Test - void ignoresLegacyGenericMqttEnabledEnvironmentAlias() { - assertThat(condition.matches(context(Map.of("MQTT_ENABLED", "true")), null)).isFalse(); - } - - @Test - void acceptsYutongMqttEnabledEnvironmentAlias() { - assertThat(condition.matches(context(Map.of("YUTONG_MQTT_ENABLED", "true")), null)).isTrue(); - } - - @Test - void directPropertyOverridesEnvironmentAlias() { - assertThat(condition.matches(context(Map.of( - "lingniu.ingest.mqtt.enabled", "false", - "YUTONG_MQTT_ENABLED", "true")), null)).isFalse(); - } - - private static ConditionContext context(Map properties) { - StandardEnvironment environment = new StandardEnvironment(); - environment.getPropertySources().addFirst(new MapPropertySource("test", properties)); - return new TestConditionContext(environment); - } - - private record TestConditionContext(StandardEnvironment environment) implements ConditionContext { - @Override - public BeanDefinitionRegistry getRegistry() { - return null; - } - - @Override - public ConfigurableListableBeanFactory getBeanFactory() { - return null; - } - - @Override - public StandardEnvironment getEnvironment() { - return environment; - } - - @Override - public ResourceLoader getResourceLoader() { - return null; - } - - @Override - public ClassLoader getClassLoader() { - return null; - } - } -} diff --git a/modules/protocols/protocol-gb32960/pom.xml b/modules/protocols/protocol-gb32960/pom.xml deleted file mode 100644 index 9d4aa350..00000000 --- a/modules/protocols/protocol-gb32960/pom.xml +++ /dev/null @@ -1,64 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - protocol-gb32960 - protocol-gb32960 - GB/T 32960 新能源车国标协议接入（Netty Server，默认 TCP 9000）。 - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - ingest-core - - - com.lingniu.ingest - ingest-codec-common - - - org.springframework.boot - spring-boot-starter - - - org.springframework.boot - spring-boot-configuration-processor - true - - - io.netty - netty-all - - - com.fasterxml.jackson.core - jackson-databind - - - com.fasterxml.jackson.datatype - jackson-datatype-jsr310 - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.springframework.boot - spring-boot-test - test - - - diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/auth/Gb32960PlatformAuthorizer.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/auth/Gb32960PlatformAuthorizer.java deleted file mode 100644 index 43d880db..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/auth/Gb32960PlatformAuthorizer.java +++ /dev/null @@ -1,103 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.auth; - -import com.lingniu.ingest.protocol.gb32960.config.Gb32960Properties; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.util.Arrays; -import java.util.Collections; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.Objects; -import java.util.Set; - -/** - * 平台登入（0x05）账号白名单校验器。 - * - *

与 {@link Gb32960VinAuthorizer} 区别： - *

    - *
  • 车辆登入靠 VIN 白名单（无密码） - *
  • 平台登入有 12B 用户名 + 20B 密码 + 可选 IP 白名单 - *
- * - *

平台账号不仅用于 0x05 鉴权，也会绑定到 channel attribute，后续 0x02/0x03 上报会用它选择 - * vendor profile。账号配错时，可能表现为 0x30+ 厂商字段全部落 Raw。 - * - *

匹配规则： - *

    - *
  1. 配置列表为空 → 返回 {@link Result#ALLOW_NO_POLICY}（兼容老行为，放行但标记未鉴权） - *
  2. 用户名不存在 → {@link Result#DENY_UNKNOWN_USER} - *
  3. 密码不匹配 → {@link Result#DENY_BAD_PASSWORD} - *
  4. IP 白名单非空且 peer IP 不在其中 → {@link Result#DENY_IP_NOT_ALLOWED} - *
  5. 全部通过 → {@link Result#ALLOW} - *
- */ -public final class Gb32960PlatformAuthorizer { - - private static final Logger log = LoggerFactory.getLogger(Gb32960PlatformAuthorizer.class); - - private final Map byUsername; - - public Gb32960PlatformAuthorizer(List platforms) { - Map map = new HashMap<>(); - for (Gb32960Properties.Auth.Platform p : platforms) { - if (p.getUsername() == null || p.getUsername().isBlank()) continue; - map.put(p.getUsername().trim(), p); - } - this.byUsername = Collections.unmodifiableMap(map); - log.info("[gb32960] platform authorizer loaded {} platform(s)", this.byUsername.size()); - } - - /** - * 鉴权一次平台登入请求。 - * - * @param username 终端发来的用户名 - * @param password 终端发来的密码 - * @param peerIp 连接对端 IP（纯 IPv4/IPv6，不含端口） - */ - public Result authenticate(String username, String password, String peerIp) { - if (byUsername.isEmpty()) return Result.ALLOW_NO_POLICY; - Gb32960Properties.Auth.Platform entry = byUsername.get(username == null ? "" : username.trim()); - if (entry == null) return Result.DENY_UNKNOWN_USER; - if (!Objects.equals(entry.getPassword(), password)) return Result.DENY_BAD_PASSWORD; - Set allowed = entry.getAllowedIps() == null - ? Set.of() - : entry.getAllowedIps().stream() - .filter(Objects::nonNull) - .flatMap(value -> Arrays.stream(value.split(","))) - .map(String::trim) - .filter(ip -> !ip.isBlank()) - .collect(java.util.stream.Collectors.toSet()); - if (!allowed.isEmpty() && (peerIp == null || !allowed.contains(peerIp))) { - return Result.DENY_IP_NOT_ALLOWED; - } - return Result.ALLOW; - } - - public int size() { - return byUsername.size(); - } - - public boolean isEnforcing() { - return !byUsername.isEmpty(); - } - - /** 鉴权结果枚举，调用方根据结果选择应答标志和日志级别。 */ - public enum Result { - /** 匹配成功。 */ - ALLOW(true), - /** 配置列表为空，放行但标记未鉴权。 */ - ALLOW_NO_POLICY(true), - /** 用户名不存在。 */ - DENY_UNKNOWN_USER(false), - /** 密码错误。 */ - DENY_BAD_PASSWORD(false), - /** IP 白名单拒绝。 */ - DENY_IP_NOT_ALLOWED(false); - - private final boolean accepted; - Result(boolean accepted) { this.accepted = accepted; } - public boolean accepted() { return accepted; } - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/auth/Gb32960VinAuthorizer.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/auth/Gb32960VinAuthorizer.java deleted file mode 100644 index 5c26b817..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/auth/Gb32960VinAuthorizer.java +++ /dev/null @@ -1,67 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.auth; - -import com.lingniu.ingest.protocol.gb32960.config.Gb32960Properties; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.util.Collections; -import java.util.Set; -import java.util.concurrent.atomic.AtomicReference; - -/** - * GB/T 32960 VIN 白名单校验器。 - * - *

线程安全：内部持有 {@code AtomicReference}，支持运行时热替换白名单 - * （未来可扩展为从 Nacos / 数据库定时刷新）。 - * - *

该校验只决定是否允许设备继续进入接收链路，不负责车辆主数据注册。生产环境若打开白名单， - * 需要确保新增车辆先同步到这里，否则终端会收到 VIN_NOT_EXIST 应答并断开。 - */ -public final class Gb32960VinAuthorizer { - - private static final Logger log = LoggerFactory.getLogger(Gb32960VinAuthorizer.class); - - private final boolean enabled; - private final boolean caseSensitive; - private final AtomicReference> whitelist; - - public Gb32960VinAuthorizer(Gb32960Properties.Auth cfg) { - this.enabled = cfg.isEnabled(); - this.caseSensitive = cfg.isCaseSensitive(); - this.whitelist = new AtomicReference<>(cfg.normalizedWhitelist()); - log.info("[gb32960] VIN authorizer enabled={} caseSensitive={} whitelistSize={}", - enabled, caseSensitive, this.whitelist.get().size()); - } - - /** - * 鉴权。 - * - * @return true 通过；false 拒绝（调用方应发送 0x04 VIN_NOT_EXIST 应答并关闭连接） - */ - public boolean isAllowed(String vin) { - if (!enabled) return true; - if (vin == null || vin.isBlank()) return false; - // 默认大小写不敏感，以容忍部分平台把 VIN 小写转发；规范 VIN 本身仍应为大写。 - String key = caseSensitive ? vin.trim() : vin.trim().toUpperCase(); - return whitelist.get().contains(key); - } - - public boolean isEnforcing() { - return enabled; - } - - public int whitelistSize() { - return whitelist.get().size(); - } - - /** 运行时热更新。接入 Nacos/DB 监听后调用。 */ - public void replaceWhitelist(Set newVins) { - Set norm = Collections.newSetFromMap(new java.util.concurrent.ConcurrentHashMap<>(newVins.size())); - for (String v : newVins) { - if (v == null || v.isBlank()) continue; - norm.add(caseSensitive ? v.trim() : v.trim().toUpperCase()); - } - this.whitelist.set(norm); - log.info("[gb32960] VIN whitelist replaced size={}", norm.size()); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParser.java deleted file mode 100644 index b9809441..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParser.java +++ /dev/null @@ -1,239 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.api.spi.DecodeException; -import com.lingniu.ingest.protocol.gb32960.codec.profile.Gb32960ProfileRegistry; -import com.lingniu.ingest.protocol.gb32960.codec.profile.VendorExtensionSelector; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlockType; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 实时/补发上报数据单元循环解析器。 - * - *

数据单元布局（表 7）：{@code timestamp(6B) | (typeFlag(1B) + infoBody)* | [0xFF 签名信息]}。 - * 其中时间戳由 {@link Gb32960MessageDecoder} 负责剥离，本 parser 只处理信息体循环。 - * - *

支持的信息类型标志（表 9）： - * - * - * - * - * - * - * - * - * - * - * - * - * - * - * - *
20162025含义
0x010x01整车数据
0x020x02驱动电机数据
0x030x03燃料电池发动机及车载氢系统数据
0x040x04发动机数据
0x050x05车辆位置数据（2025 版增加坐标系字段）
0x06 极值0x06 报警 
0x07 报警0x07 动力蓄电池最小并联单元电压 
0x08 储能电压0x08 动力蓄电池温度 
0x09 储能温度— 
—0x30燃料电池电堆
—0x31超级电容器
—0x32超级电容器极值
—0xFF签名数据开始标识（2025 版新增）
- * - *

处理策略： - *

    - *
  • 遇到 {@code 0xFF}（2025 版签名起始标识）：停止循环，剩余字节作为签名原始字节返回 - *
  • 遇到未注册的类型：作为单个 {@link InfoBlock.Raw} 包装剩余字节并终止循环，避免丢失整帧 - *
- */ -public final class Gb32960BodyParser { - - private static final Logger log = LoggerFactory.getLogger(Gb32960BodyParser.class); - - /** 2025 版实时上报尾部签名标识（表 9）。 */ - private static final int SIGNATURE_MARKER = 0xFF; - - private final Gb32960ProfileRegistry profileRegistry; - private final VendorExtensionSelector selector; - - /** - * 单块解析异常时是否兜底为 {@link InfoBlock.Raw} 后继续。由 - * {@link com.lingniu.ingest.protocol.gb32960.config.Gb32960Properties.Parse#isLenientBlockFailure()} - * 注入；默认 true。实际行为在 - * {@link #parse(Gb32960ParserContext, ByteBuffer)} 主循环里实现。 - */ - private boolean lenientBlockFailure = true; - - public void setLenientBlockFailure(boolean lenientBlockFailure) { - this.lenientBlockFailure = lenientBlockFailure; - } - - /** - * 单 registry 构造（向后兼容旧测试）。等价于 default profile + 无 vendor 扩展。 - */ - public Gb32960BodyParser(InfoBlockParserRegistry registry) { - this.profileRegistry = new Gb32960ProfileRegistry( - new java.util.ArrayList<>(), // ignored, defaultRegistry overridden via wrapper below - new com.lingniu.ingest.protocol.gb32960.codec.profile.VendorExtensionCatalog(java.util.Map.of()), - java.util.Set.of()) { - @Override public InfoBlockParserRegistry forProfile(String name) { return registry; } - @Override public InfoBlockParserRegistry defaultRegistry() { return registry; } - }; - this.selector = VendorExtensionSelector.NONE; - } - - /** - * 标准构造：profile 注册表 + vendor 选择器。运行期由 - * {@link Gb32960MessageDecoder} 调用 {@link #parse(Gb32960ParserContext, ByteBuffer)} - * 携带上下文，按 selector 路由到具体 profile 的 registry。 - */ - public Gb32960BodyParser(Gb32960ProfileRegistry profileRegistry, VendorExtensionSelector selector) { - this.profileRegistry = profileRegistry; - this.selector = selector; - } - - /** 旧入口：无上下文 → 走默认 profile。保留用于不需要 vendor 路由的单测。 */ - public Gb32960MessageDecoder.BodyParseResult parse(ProtocolVersion version, ByteBuffer body) { - return parse(Gb32960ParserContext.minimal(version), body); - } - - /** 主入口：根据上下文按 selector 选 vendor profile，然后照旧解析循环。 */ - public Gb32960MessageDecoder.BodyParseResult parse(Gb32960ParserContext ctx, ByteBuffer body) { - ProtocolVersion version = ctx.version(); - // selector 只决定“标准 parser 集合上是否叠加某个 vendor 扩展集合”；标准 typeCode 始终可解析。 - InfoBlockParserRegistry registry = profileRegistry.forProfile(selector.select(ctx)); - List blocks = new ArrayList<>(8); - byte[] signature = null; - // 捕获 info-block 区间的起止位置，用于未知块出现时打 dump 帮助手动定位漂移点。 - // body 由 Gb32960MessageDecoder 通过 ByteBuffer.wrap(frame, bodyStart+6, dataLength-6) 构造， - // arrayOffset()=0，position() = bodyStart+6，limit() = bodyStart + dataLength。 - final int infoBlocksStart = body.position(); - final int infoBlocksEnd = body.limit(); - - while (body.hasRemaining()) { - int typeStartPos = body.position(); - int typeCode = body.get() & 0xFF; - - if (typeCode == SIGNATURE_MARKER && version == ProtocolVersion.V2025) { - // 剩余全部字节即为签名内容（表 8） - signature = new byte[body.remaining()]; - body.get(signature); - break; - } - - InfoBlockParser parser = registry.find(version, typeCode); - if (parser == null) { - // lenient：未知类型无法安全跳过，吞剩余字节作为 Raw 后终止。 - // 真实 typeCode 透传到 Raw 字段，便于在日志/JSON 里直接看到漂移落点。 - // 如果该未知块实际是 TLV，应优先新增专用 TLV parser，让主循环可以继续解析后续块。 - int remaining = body.remaining(); - byte[] rest = new byte[remaining]; - body.get(rest); - log.warn("[gb32960] unknown info block typeCode=0x{} version={} bodyPos={} parsedBlocks={} remaining={} preview={}", - Integer.toHexString(typeCode), version, typeStartPos, - blocks.size(), remaining, hexPreview(rest, 16)); - // 一次性把整段 info-block 区间的原始字节打出来，并在 typeStartPos 处插入标记， - // 帮助人工对比规范字段长度、定位是哪个早期块对齐错。 - if (body.hasArray()) { - log.warn("[gb32960] body dump (info-block region) infoBlocksStart={} infoBlocksEnd={} typeStartPos={} totalLen={} alreadyParsed={}\n{}", - infoBlocksStart, infoBlocksEnd, typeStartPos, - infoBlocksEnd - infoBlocksStart, blocks.size(), - hexDumpWithMarker(body.array(), infoBlocksStart, infoBlocksEnd, typeStartPos)); - } - blocks.add(new InfoBlock.Raw(typeCode, InfoBlockType.RAW, rest)); - break; - } - - int fixedLen = parser.fixedLength(); - int posBefore = body.position(); - - // 预检：固定长度块在剩余字节不足时，lenient 模式兜 Raw 后 break；严格模式保持旧行为抛异常。 - if (fixedLen >= 0 && body.remaining() < fixedLen) { - if (!lenientBlockFailure) { - throw new DecodeException( - "info block 0x" + Integer.toHexString(typeCode) - + " needs " + fixedLen + " bytes but got " + body.remaining()); - } - // 固定长度块尾部缺字节时不猜测字段；保留剩余 bytes，方便后续用 RAW 回放复盘。 - int remaining = body.remaining(); - byte[] tail = new byte[remaining]; - body.get(tail); - log.warn("[gb32960] truncated block typeCode=0x{} declaredFixedLen={} remaining={} — wrapping tail as Raw", - Integer.toHexString(typeCode), fixedLen, remaining); - blocks.add(new InfoBlock.Raw(typeCode, InfoBlockType.RAW, tail)); - break; - } - - InfoBlock block; - try { - block = parser.parse(body); - } catch (DecodeException | java.nio.BufferUnderflowException | IndexOutOfBoundsException e) { - if (!lenientBlockFailure) { - if (e instanceof DecodeException de) throw de; - throw new DecodeException( - "parser " + parser.getClass().getSimpleName() + " failed: " + e.getMessage(), e); - } - // 回滚到 parser.parse 入口，按块类型分支处理。 - body.position(posBefore); - if (fixedLen >= 0) { - // 固定长度块：按 fixedLen 截取 → 兜 Raw → continue 循环 - byte[] corrupt = new byte[fixedLen]; - body.get(corrupt); - log.warn("[gb32960] block parse failed typeCode=0x{} parser={} fixedLen={} — isolated as Raw, continuing", - Integer.toHexString(typeCode), parser.getClass().getSimpleName(), fixedLen, e); - blocks.add(new InfoBlock.Raw(typeCode, InfoBlockType.RAW, corrupt)); - continue; - } - // 变长块：无法安全找下一块边界 → 剩余整段兜 Raw + break - int remaining = body.remaining(); - byte[] tail = new byte[remaining]; - body.get(tail); - log.warn("[gb32960] variable-length block parse failed typeCode=0x{} parser={} remaining={} — wrapping remainder as Raw, stopping loop", - Integer.toHexString(typeCode), parser.getClass().getSimpleName(), remaining, e); - blocks.add(new InfoBlock.Raw(typeCode, InfoBlockType.RAW, tail)); - break; - } - - int consumed = body.position() - posBefore; - if (fixedLen >= 0 && consumed != fixedLen) { - // parser 契约违反（声明 fixedLen=X 但实际读了 Y），保持抛异常以暴露 parser bug， - // 不走 lenient 兜底，避免静默误解析长期误判。 - throw new DecodeException( - "parser " + parser.getClass().getSimpleName() - + " consumed " + consumed - + " bytes but declared " + fixedLen); - } - if (log.isDebugEnabled()) { - log.debug("[gb32960] block typeCode=0x{} parser={} pos {}→{} consumed={} declaredFixed={}", - Integer.toHexString(typeCode), parser.getClass().getSimpleName(), - typeStartPos, body.position(), consumed, fixedLen); - } - blocks.add(block); - } - - return new Gb32960MessageDecoder.BodyParseResult(blocks, signature); - } - - private static String hexPreview(byte[] bytes, int max) { - int n = Math.min(bytes.length, max); - StringBuilder sb = new StringBuilder(n * 2); - for (int i = 0; i < n; i++) sb.append(String.format("%02x", bytes[i])); - if (bytes.length > max) sb.append(".."); - return sb.toString(); - } - - /** - * 16 字节/行的 hex dump，遇到 typeStartPos 时在该字节后插入 {@code |<< - *
  • 0x01 车辆登入（表 6） - *
  • 0x04 车辆登出（表 28） - *
  • 0x05 平台登入（表 29） - *
  • 0x06 平台登出（表 30） - *
  • 0x07 心跳（空） - *
  • 0x08 终端校时（空） - *
  • 0x09 激活（表 B.3） - *
  • 0x0A 激活应答（表 B.4） - *
  • 0x0B 密钥交换（表 31） - * - * - *

    其它命令（查询/设置/控制/自定义）保留为 {@link CommandBody.RawCommand}。 - */ -public final class Gb32960CommandParser { - - private static final ZoneId ZONE_GMT8 = ZoneId.of("Asia/Shanghai"); - - public CommandBody parse(CommandType command, ByteBuffer buf) { - return switch (command) { - case VEHICLE_LOGIN -> parseVehicleLogin(buf); - case VEHICLE_LOGOUT -> parseVehicleLogout(buf); - case PLATFORM_LOGIN -> parsePlatformLogin(buf); - case PLATFORM_LOGOUT -> parsePlatformLogout(buf); - case HEARTBEAT -> new CommandBody.Heartbeat(); - case TIME_CALIBRATION -> new CommandBody.TimeCalibration(); - case ACTIVATION -> parseActivation(buf); - case ACTIVATION_RESPONSE -> parseActivationResponse(buf); - case KEY_EXCHANGE -> parseKeyExchange(buf); - case QUERY -> parseQueryParams(buf); - case SET_PARAMS -> parseSetParams(buf); - case TERMINAL_CONTROL -> parseTerminalControl(buf); - default -> captureRaw(command, buf); - }; - } - - /** 表 6：车辆登入。 */ - private CommandBody.VehicleLogin parseVehicleLogin(ByteBuffer buf) { - Instant time = readTime(buf); - int serialNo = buf.getShort() & 0xFFFF; - String iccid = readAscii(buf, 20); - // 2025 版新增：电池管理系统数 n + 各系统包数 + 编码列表 - int batterySystemCount = buf.hasRemaining() ? (buf.get() & 0xFF) : 0; - List packCounts = new ArrayList<>(); - int totalPacks = 0; - for (int i = 0; i < batterySystemCount && buf.hasRemaining(); i++) { - int c = buf.get() & 0xFF; - packCounts.add(c); - totalPacks += c; - } - List codes = new ArrayList<>(totalPacks); - for (int i = 0; i < totalPacks && buf.remaining() >= 24; i++) { - codes.add(readAscii(buf, 24)); - } - return new CommandBody.VehicleLogin(time, serialNo, iccid, batterySystemCount, packCounts, codes); - } - - private CommandBody.VehicleLogout parseVehicleLogout(ByteBuffer buf) { - return new CommandBody.VehicleLogout(readTime(buf), buf.getShort() & 0xFFFF); - } - - private CommandBody.PlatformLogin parsePlatformLogin(ByteBuffer buf) { - Instant time = readTime(buf); - int serial = buf.getShort() & 0xFFFF; - String user = readAscii(buf, 12); - // 标准密码字段为 20B，但现场存在扩展到 32B 的平台登入包；这里按剩余长度-1 读取， - // 最后一字节仍保留给加密规则，兼容两种形态。 - int passwordLength = Math.max(0, buf.remaining() - 1); - String pwd = readAscii(buf, passwordLength); - EncryptType enc = EncryptType.of(buf.get() & 0xFF); - return new CommandBody.PlatformLogin(time, serial, user, pwd, enc); - } - - private CommandBody.PlatformLogout parsePlatformLogout(ByteBuffer buf) { - return new CommandBody.PlatformLogout(readTime(buf), buf.getShort() & 0xFFFF); - } - - private CommandBody.Activation parseActivation(ByteBuffer buf) { - Instant time = readTime(buf); - String chipId = readAscii(buf, 16); - int keyLen = buf.getShort() & 0xFFFF; - byte[] pubKey = new byte[Math.min(keyLen, buf.remaining())]; - buf.get(pubKey); - String vin = readAscii(buf, 17); - byte[] signature = new byte[buf.remaining()]; - buf.get(signature); - return new CommandBody.Activation(time, chipId, pubKey, vin, signature); - } - - private CommandBody.ActivationResponse parseActivationResponse(ByteBuffer buf) { - return new CommandBody.ActivationResponse(buf.get() & 0xFF, buf.get() & 0xFF); - } - - private CommandBody.KeyExchange parseKeyExchange(ByteBuffer buf) { - EncryptType keyType = EncryptType.of(buf.get() & 0xFF); - int keyLen = buf.getShort() & 0xFFFF; - byte[] key = new byte[Math.min(keyLen, buf.remaining() - 12)]; - buf.get(key); - Instant activateAt = readTime(buf); - Instant expireAt = readTime(buf); - return new CommandBody.KeyExchange(keyType, key, activateAt, expireAt); - } - - private CommandBody.RawCommand captureRaw(CommandType command, ByteBuffer buf) { - byte[] bytes = new byte[buf.remaining()]; - buf.get(bytes); - return new CommandBody.RawCommand(command.code(), bytes); - } - - // ========================================================================= - // 表 B.5-B.13 下行命令 - // ========================================================================= - - /** - * 参数查询 0x80（表 B.5）：{@code time(6) + count(1) + paramId(count)}。 - * 如果 count > 0，则是查询命令；如果 count == 0 可能是终端应答（表 B.6）， - * 应答带参数项列表（表 B.7），paramValue 长度根据 paramId 查表 B.8 决定。 - */ - private CommandBody parseQueryParams(ByteBuffer buf) { - Instant time = readTime(buf); - int count = buf.get() & 0xFF; - - // 判定是查询命令还是应答：命令方向下 count 后紧跟 paramId 列表（每项 1 字节）； - // 应答方向下 count 后是完整的 paramId + paramValue 列表（表 B.7）。 - // 通过剩余字节数判断：命令形态 remaining == count，应答形态 remaining > count - if (buf.remaining() == count) { - List ids = new ArrayList<>(count); - for (int i = 0; i < count; i++) ids.add(buf.get() & 0xFF); - return new CommandBody.QueryParams(time, ids); - } - // 作为应答处理 - Map params = readParamItems(buf, count); - return new CommandBody.QueryParamsResponse(time, params); - } - - /** 参数设置 0x81（表 B.9）：{@code time(6) + count(1) + paramItems}。 */ - private CommandBody.SetParams parseSetParams(ByteBuffer buf) { - Instant time = readTime(buf); - int count = buf.get() & 0xFF; - Map params = readParamItems(buf, count); - return new CommandBody.SetParams(time, params); - } - - /** 读取 N 个参数项（表 B.7）：{@code paramId(1) + paramValue(按表 B.8 解释)}。 */ - private Map readParamItems(ByteBuffer buf, int count) { - Map out = new HashMap<>(count); - for (int i = 0; i < count && buf.hasRemaining(); i++) { - int paramId = buf.get() & 0xFF; - int len = paramValueLength(paramId, buf); - if (len < 0 || len > buf.remaining()) { - // 长度未知或越界：吞剩余字节作为该参数值后终止 - byte[] rest = new byte[buf.remaining()]; - buf.get(rest); - out.put(paramId, rest); - break; - } - byte[] value = new byte[len]; - buf.get(value); - out.put(paramId, value); - } - return out; - } - - /** - * 按表 B.8 返回参数值长度。 - * - *

    对于变长参数（0x05 域名 / 0x0E 公共平台域名），前一个参数（0x04/0x0D）给出长度； - * 这里简化处理：变长参数使用 {@code buf.get() & 0xFF + 1} 预读 + 取值。 - */ - private int paramValueLength(int paramId, ByteBuffer buf) { - return switch (paramId) { - case 0x01 -> 2; // 本地存储时间周期 WORD - case 0x02 -> 1; // 正常信息上报时间周期 - case 0x03 -> 1; // 报警信息上报时间周期 - case 0x04 -> 1; // 平台域名长度 M - case 0x05 -> { - // 平台域名 M 字节，M 由前一个参数的值给出。 - // 安全起见这里一次吞掉剩余字节（实际场景参数 0x04 和 0x05 通常一起下发） - yield buf.remaining(); - } - case 0x06 -> 2; // 平台端口 WORD - case 0x07, 0x08 -> 5; // 硬件/固件版本 5 字节 - case 0x09 -> 1; // 心跳发送周期 - case 0x0A -> 2; // 终端应答超时时间 - case 0x0B -> 2; // 平台应答超时时间 - case 0x0C -> 1; // 登入失败后下次登入间隔 - case 0x0D -> 1; // 公共平台域名长度 N - case 0x0E -> buf.remaining(); // 公共平台域名 - case 0x0F -> 2; // 公共平台端口 - case 0x10 -> 1; // 是否抽样监测中 - default -> -1; // 未知参数：无法决定长度 - }; - } - - /** 车载终端控制 0x82（表 B.10）。 */ - private CommandBody.TerminalControl parseTerminalControl(ByteBuffer buf) { - Instant time = readTime(buf); - int cmdId = buf.get() & 0xFF; - CommandBody.ControlCommand commandId = CommandBody.ControlCommand.of(cmdId); - CommandBody.ControlPayload payload = switch (commandId) { - case REMOTE_UPGRADE -> parseRemoteUpgrade(buf); - case ALARM_COMMAND -> parseAlarmCommand(buf); - case SHUTDOWN, RESET, FACTORY_RESET, DISCONNECT, SAMPLING_LINK_ON -> new CommandBody.ControlPayload.None(); - case UNKNOWN -> captureRawPayload(buf); - }; - return new CommandBody.TerminalControl(time, commandId, payload); - } - - /** - * 表 B.12 远程升级参数。字段之间用半角分号 {@code ;} 分隔。 - * - *

    字段顺序：URL;apn;dialUser;dialPwd;serverAddress;serverPort;manufacturerCode;hwVer;fwVer;timeoutMin - */ - private CommandBody.ControlPayload.RemoteUpgrade parseRemoteUpgrade(ByteBuffer buf) { - byte[] all = new byte[buf.remaining()]; - buf.get(all); - String s = new String(all, Charset.forName("GBK")); - String[] parts = s.split(";", -1); - String url = parts.length > 0 ? parts[0] : ""; - String apn = parts.length > 1 ? parts[1] : ""; - String user = parts.length > 2 ? parts[2] : ""; - String pwd = parts.length > 3 ? parts[3] : ""; - byte[] addr = parts.length > 4 ? parts[4].getBytes(StandardCharsets.US_ASCII) : new byte[0]; - int port = parts.length > 5 ? parseIntSafe(parts[5]) : 0; - String manu = parts.length > 6 ? parts[6] : ""; - String hw = parts.length > 7 ? parts[7] : ""; - String fw = parts.length > 8 ? parts[8] : ""; - int timeoutMin = parts.length > 9 ? parseIntSafe(parts[9]) : 0; - return new CommandBody.ControlPayload.RemoteUpgrade( - apn, user, pwd, addr, port, manu, hw, fw, url, timeoutMin); - } - - /** 表 B.13 报警/预警命令。 */ - private CommandBody.ControlPayload.AlarmCommand parseAlarmCommand(ByteBuffer buf) { - int warningLevel = buf.hasRemaining() ? (buf.get() & 0xFF) : 0; - byte[] info = new byte[buf.remaining()]; - buf.get(info); - return new CommandBody.ControlPayload.AlarmCommand(warningLevel, info); - } - - private CommandBody.ControlPayload.RawPayload captureRawPayload(ByteBuffer buf) { - byte[] bytes = new byte[buf.remaining()]; - buf.get(bytes); - return new CommandBody.ControlPayload.RawPayload(bytes); - } - - private static int parseIntSafe(String s) { - try { return Integer.parseInt(s.trim()); } catch (Exception e) { return 0; } - } - - /** 表 5：时间定义 YY MM DD HH MM SS，GMT+8。 */ - private Instant readTime(ByteBuffer buf) { - if (buf.remaining() < 6) { - throw new DecodeException("time field requires 6 bytes, got " + buf.remaining()); - } - int year = 2000 + (buf.get() & 0xFF); - int month = buf.get() & 0xFF; - int day = buf.get() & 0xFF; - int hour = buf.get() & 0xFF; - int minute = buf.get() & 0xFF; - int second = buf.get() & 0xFF; - try { - return LocalDateTime.of(year, month, day, hour, minute, second).atZone(ZONE_GMT8).toInstant(); - } catch (Exception e) { - throw new DecodeException("invalid time: " + year + "-" + month + "-" + day - + " " + hour + ":" + minute + ":" + second, e); - } - } - - private String readAscii(ByteBuffer buf, int len) { - int safeLen = Math.min(len, buf.remaining()); - byte[] b = new byte[safeLen]; - buf.get(b); - return new String(b, StandardCharsets.US_ASCII).trim(); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FrameDecoder.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FrameDecoder.java deleted file mode 100644 index 62fca57b..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FrameDecoder.java +++ /dev/null @@ -1,104 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import io.netty.buffer.ByteBuf; -import io.netty.buffer.ByteBufUtil; -import io.netty.channel.ChannelHandlerContext; -import io.netty.handler.codec.ByteToMessageDecoder; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import com.lingniu.ingest.codec.BccChecksum; - -import java.util.List; - -/** - * Netty 层的帧分帧器：从字节流里提取一条完整的 32960 帧。 - * - *

    支持两种起始符：{@code 0x23 0x23 (##)} 2016 版，{@code 0x24 0x24 ($$)} 2025 版。 - * 只负责拆包粘包，不做业务解析。具体字段解析交给 {@link Gb32960MessageDecoder}。 - * - *

    开启 DEBUG 日志可以看到每次读到的字节、每条产出的完整帧以及因长度/起始符不匹配被丢弃的字节数。 - */ -public class Gb32960FrameDecoder extends ByteToMessageDecoder { - - private static final Logger log = LoggerFactory.getLogger(Gb32960FrameDecoder.class); - - private static final int HEADER_LEN = 24; // 起始(2) + cmd(1) + rsp(1) + vin(17) + enc(1) + len(2) - private static final int MAX_FRAME = 8 * 1024; - - @Override - protected void decode(ChannelHandlerContext ctx, ByteBuf in, List out) { - if (log.isDebugEnabled() && in.readableBytes() > 0) { - int peek = Math.min(in.readableBytes(), 64); - log.debug("[gb32960] frame-decoder rx bytes={} head={}", - in.readableBytes(), - ByteBufUtil.hexDump(in, in.readerIndex(), peek)); - } - while (in.readableBytes() >= HEADER_LEN) { - int startIdx = findStart(in); - if (startIdx < 0 || startIdx > in.writerIndex() - HEADER_LEN) { - int drop = Math.max(0, in.readableBytes() - 1); - if (drop > 0) { - log.debug("[gb32960] frame-decoder skip {} bytes (no start symbol)", drop); - in.skipBytes(drop); - } - return; - } - if (startIdx != in.readerIndex()) { - int drop = startIdx - in.readerIndex(); - log.debug("[gb32960] frame-decoder skip {} leading bytes before start", drop); - in.skipBytes(drop); - } - int dataLen = in.getUnsignedShort(in.readerIndex() + 22); - int frameLen = frameLength(in, in.readerIndex(), dataLen); - if (frameLen > MAX_FRAME) { - log.warn("[gb32960] frame-decoder oversized frame len={} peer={}, skip start", - frameLen, ctx.channel().remoteAddress()); - in.skipBytes(2); - continue; - } - if (in.readableBytes() < frameLen) return; - - byte[] frame = new byte[frameLen]; - in.readBytes(frame); - if (log.isDebugEnabled()) { - log.debug("[gb32960] frame-decoder emit len={} start={} peer={}", - frameLen, - String.format("%02x%02x", frame[0], frame[1]), - ctx.channel().remoteAddress()); - } - out.add(frame); - } - } - - private static int frameLength(ByteBuf in, int readerIndex, int dataLen) { - int standardLen = HEADER_LEN + dataLen + 1; - int command = in.getUnsignedByte(readerIndex + 2); - if (command != 0x05 || dataLen != 41) { - return standardLen; - } - // 现场平台登入包存在"声明 41B、实际 53B"的兼容形态；只有扩展长度 BCC 成立时才按 53B 分帧， - // 否则回退标准长度，避免误吞下一条帧。 - int extendedLen = HEADER_LEN + 53 + 1; - if (in.readableBytes() < extendedLen) { - return standardLen; - } - byte[] extended = new byte[extendedLen]; - in.getBytes(readerIndex, extended); - if (BccChecksum.verify(extended, 2, extended.length - 3, extended[extended.length - 1])) { - return extendedLen; - } - return standardLen; - } - - /** 查找下一个合法起始位置：两字节连续 0x23 0x23 或 0x24 0x24。 */ - private static int findStart(ByteBuf in) { - int limit = in.writerIndex() - 1; - for (int i = in.readerIndex(); i < limit; i++) { - byte b = in.getByte(i); - if ((b == 0x23 || b == 0x24) && in.getByte(i + 1) == b) { - return i; - } - } - return -1; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FrameEncoder.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FrameEncoder.java deleted file mode 100644 index 64eaae75..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FrameEncoder.java +++ /dev/null @@ -1,114 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.protocol.gb32960.model.CommandType; -import com.lingniu.ingest.protocol.gb32960.model.EncryptType; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -import com.lingniu.ingest.protocol.gb32960.model.ResponseFlag; - -import java.io.ByteArrayOutputStream; -import java.nio.charset.StandardCharsets; -import java.time.Instant; -import java.time.LocalDateTime; -import java.time.ZoneId; - -/** - * GB/T 32960.3 下行帧编码器。主要用于应答场景：平台收到终端上行命令后发送应答。 - * - *

    应答规则（§6.3.2）：服务端发送应答时，变更应答标志、保留报文时间、删除其余报文内容、 - * 重新计算校验位。 - * - *

    该类只做协议编码，不写 Channel。是否立即关闭连接由 {@code Gb32960AckService} - * 根据鉴权结果决定。 - */ -public final class Gb32960FrameEncoder { - - private static final ZoneId ZONE_GMT8 = ZoneId.of("Asia/Shanghai"); - - private Gb32960FrameEncoder() {} - - /** - * 构造简易应答帧：复用原命令的 command/vin/encryptType，修改应答标志， - * 可选携带数据单元（通常只带 6 字节采集时间）。 - * - * @param version 协议版本（决定起始符） - * @param command 命令标识（与原命令相同） - * @param responseFlag 应答标志：{@link ResponseFlag#SUCCESS}/{@link ResponseFlag#VIN_NOT_EXIST} 等 - * @param vin 17 字节 VIN - * @param collectTime 数据采集时间（保留原帧时间，若无则传 {@code Instant.now()}） - * @param extraBody 附加数据（可为 null） - */ - public static byte[] buildResponse(ProtocolVersion version, - CommandType command, - ResponseFlag responseFlag, - String vin, - Instant collectTime, - byte[] extraBody) { - return buildResponse(version, command, responseFlag, vinPadded(vin), collectTime, extraBody); - } - - /** - * 同 {@link #buildResponse(ProtocolVersion, CommandType, ResponseFlag, String, Instant, byte[])}， - * 但直接接收 17 字节原始 VIN。用于回执时原样回显请求帧的 VIN 字节，避免 - * 平台登入这类 VIN 全 0 的场景被我们用空格重写后导致对端判定 VIN 不匹配。 - */ - public static byte[] buildResponse(ProtocolVersion version, - CommandType command, - ResponseFlag responseFlag, - byte[] vin17, - Instant collectTime, - byte[] extraBody) { - ByteArrayOutputStream body = new ByteArrayOutputStream(); - // 数据采集时间 6 字节（表 5） - if (collectTime != null) { - LocalDateTime ldt = LocalDateTime.ofInstant(collectTime, ZONE_GMT8); - body.write(ldt.getYear() - 2000); - body.write(ldt.getMonthValue()); - body.write(ldt.getDayOfMonth()); - body.write(ldt.getHour()); - body.write(ldt.getMinute()); - body.write(ldt.getSecond()); - } - if (extraBody != null && extraBody.length > 0) { - body.write(extraBody, 0, extraBody.length); - } - - byte[] bodyBytes = body.toByteArray(); - int dataLength = bodyBytes.length; - - ByteArrayOutputStream out = new ByteArrayOutputStream(); - // 起始符 - if (version == ProtocolVersion.V2025) { - out.write(0x24); out.write(0x24); - } else { - out.write(0x23); out.write(0x23); - } - out.write(command.code()); - out.write(responseFlag.code()); - // VIN（17 字节，由调用方给出；平台登入等场景直接回显请求的原始字节） - byte[] vinBytes = vin17 != null && vin17.length == 17 ? vin17 : vinPadded(null); - out.write(vinBytes, 0, 17); - // 加密方式：应答不加密 - out.write(EncryptType.UNENCRYPTED.code()); - // 数据单元长度 WORD 大端 - out.write((dataLength >> 8) & 0xFF); - out.write(dataLength & 0xFF); - out.write(bodyBytes, 0, bodyBytes.length); - // BCC：从 cmd(offset 2) 到数据单元结尾 - byte[] almost = out.toByteArray(); - byte bcc = BccChecksum.compute(almost, 2, almost.length - 2); - out.write(bcc & 0xFF); - return out.toByteArray(); - } - - private static byte[] vinPadded(String vin) { - byte[] out = new byte[17]; - // 填充空格 - java.util.Arrays.fill(out, (byte) ' '); - if (vin == null) return out; - byte[] src = vin.getBytes(StandardCharsets.US_ASCII); - int len = Math.min(src.length, 17); - System.arraycopy(src, 0, out, 0, len); - return out; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960MessageDecoder.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960MessageDecoder.java deleted file mode 100644 index ecc2884b..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960MessageDecoder.java +++ /dev/null @@ -1,180 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.api.spi.DecodeException; -import com.lingniu.ingest.api.spi.FrameDecoder; -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.protocol.gb32960.model.CommandBody; -import com.lingniu.ingest.protocol.gb32960.model.CommandType; -import com.lingniu.ingest.protocol.gb32960.model.EncryptType; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Header; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -import com.lingniu.ingest.protocol.gb32960.model.ResponseFlag; - -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.time.Instant; -import java.time.LocalDateTime; -import java.time.ZoneId; -import java.util.Collections; -import java.util.List; - -/** - * 完整帧 → {@link Gb32960Message} 的解码器。 - * - *

    支持 GB/T 32960.3 两个版本： - *

      - *
    • {@code 0x23 0x23 (##)} → 2016 版（{@link ProtocolVersion#V2016}） - *
    • {@code 0x24 0x24 ($$)} → 2025 版（{@link ProtocolVersion#V2025}） - *
    - * - *

    完整数据包结构（表 2 / 表 B.1）： - * 

    - *   offset 0:  起始符 (2B)
- *   offset 2:  命令标识 (1B)
- *   offset 3:  应答标志 (1B)
- *   offset 4:  唯一识别码 VIN (17B ASCII)
- *   offset 21: 数据单元加密方式 (1B)
- *   offset 22: 数据单元长度 (2B WORD 大端)
- *   offset 24: 数据单元 (dataLength 字节)
- *   last byte: 校验码 (1B BCC)
- *
    - * - *

    BCC 校验范围：从命令单元的第一个字节开始（offset 2），同后一字节异或， - * 直到校验码前一字节为止。注意：当数据单元存在加密时，应先加密后校验，解析时先校验后解密。 - * - *

    上游 Netty 的 frame handler 已经保证 {@code buffer} 内是一条完整帧。 - */ -public final class Gb32960MessageDecoder implements FrameDecoder { - - /** 最小帧长：2(起始符)+1(cmd)+1(rsp)+17(vin)+1(enc)+2(len)+0(body)+1(bcc)。 */ - private static final int MIN_FRAME = 25; - private static final ZoneId ZONE_GMT8 = ZoneId.of("Asia/Shanghai"); - - private final Gb32960BodyParser bodyParser; - private final Gb32960CommandParser commandParser; - - public Gb32960MessageDecoder(Gb32960BodyParser bodyParser) { - this(bodyParser, new Gb32960CommandParser()); - } - - public Gb32960MessageDecoder(Gb32960BodyParser bodyParser, Gb32960CommandParser commandParser) { - this.bodyParser = bodyParser; - this.commandParser = commandParser; - } - - @Override - public Gb32960Message decode(ByteBuffer buffer) { - return decode(buffer, null); - } - - /** - * 带 platformAccount 提示的解码。Handler 在已经完成 0x05 PLATFORM_LOGIN 鉴权后通过 - * channel attribute 取到账号名再传进来；以便 BodyParser 内部的 vendor profile selector - * 路由。其他命令传 {@code null} 即可。 - */ - public Gb32960Message decode(ByteBuffer buffer, String platformAccount) { - int frameLen = buffer.remaining(); - if (frameLen < MIN_FRAME) { - throw new DecodeException("frame too short: " + frameLen); - } - - byte[] frame = new byte[frameLen]; - buffer.get(frame); - - ProtocolVersion version = detectVersion(frame); - - // BCC 校验：从 cmd(offset 2) 开始到 BCC 前一字节 - byte expectedBcc = frame[frame.length - 1]; - if (!BccChecksum.verify(frame, 2, frame.length - 3, expectedBcc)) { - throw new DecodeException("BCC mismatch"); - } - - int commandCode = frame[2] & 0xFF; - int responseCode = frame[3] & 0xFF; - String vin = new String(frame, 4, 17, StandardCharsets.US_ASCII).trim(); - int encryptCode = frame[21] & 0xFF; - int dataLength = ((frame[22] & 0xFF) << 8) | (frame[23] & 0xFF); - int actualDataLength = frame.length - 25; - - int bodyStart = 24; - if (bodyStart + dataLength + 1 != frame.length - && !isExtendedPlatformLogin(commandCode, dataLength, actualDataLength)) { - throw new DecodeException("declared body length " + dataLength - + " does not match frame remainder " + (frame.length - bodyStart - 1)); - } - - CommandType cmdType = CommandType.of(commandCode); - ResponseFlag rspFlag = ResponseFlag.of(responseCode); - EncryptType encType = EncryptType.of(encryptCode); - - // 实时 / 补发上报：解析信息体（和 2025 版尾部签名） - if (cmdType == CommandType.REALTIME_REPORT || cmdType == CommandType.RESEND_REPORT) { - if (dataLength < 6) throw new DecodeException("report body missing timestamp"); - Instant eventTime = parseTimestamp(frame, bodyStart); - ByteBuffer body = ByteBuffer.wrap(frame, bodyStart + 6, dataLength - 6); - Gb32960ParserContext ctx = new Gb32960ParserContext(version, vin, platformAccount); - BodyParseResult result = bodyParser.parse(ctx, body); - Gb32960Header header = new Gb32960Header( - version, cmdType, rspFlag, vin, encType, dataLength, eventTime); - return new Gb32960Message(header, result.blocks(), null, result.signature()); - } - - // 其他命令：解析 command body - // 车辆/平台登入登出 body 首 6B 为采集时间（表 5），预解析并写入 header.eventTime， - // 以便 handler 构造应答帧时按 §6.3.2 保留原报文时间。 - Instant eventTime = null; - if (hasLeadingTimestamp(cmdType) && dataLength >= 6) { - eventTime = parseTimestamp(frame, bodyStart); - } - Gb32960Header headerStub = new Gb32960Header( - version, cmdType, rspFlag, vin, encType, dataLength, eventTime); - int commandBodyLength = isExtendedPlatformLogin(commandCode, dataLength, actualDataLength) - ? actualDataLength - : dataLength; - ByteBuffer body = ByteBuffer.wrap(frame, bodyStart, commandBodyLength); - CommandBody cmdBody = commandParser.parse(cmdType, body); - return new Gb32960Message(headerStub, Collections.emptyList(), cmdBody, null); - } - - private static boolean isExtendedPlatformLogin(int commandCode, int declaredDataLength, int actualDataLength) { - // 与 FrameDecoder 的兼容规则保持一致：部分平台登录帧声明长度 41B，但真实账号/密码体为 53B。 - return commandCode == 0x05 && declaredDataLength == 41 && actualDataLength == 53; - } - - private static ProtocolVersion detectVersion(byte[] frame) { - if (frame[0] == 0x23 && frame[1] == 0x23) return ProtocolVersion.V2016; - if (frame[0] == 0x24 && frame[1] == 0x24) return ProtocolVersion.V2025; - throw new DecodeException("unknown start symbol: " - + String.format("%02x %02x", frame[0] & 0xFF, frame[1] & 0xFF)); - } - - private static boolean hasLeadingTimestamp(CommandType cmd) { - return cmd == CommandType.VEHICLE_LOGIN - || cmd == CommandType.VEHICLE_LOGOUT - || cmd == CommandType.PLATFORM_LOGIN - || cmd == CommandType.PLATFORM_LOGOUT; - } - - /** 时间定义（表 5）：YY MM DD HH MM SS，各 1 字节，GMT+8。 */ - private static Instant parseTimestamp(byte[] frame, int offset) { - int year = 2000 + (frame[offset] & 0xFF); - int month = frame[offset + 1] & 0xFF; - int day = frame[offset + 2] & 0xFF; - int hour = frame[offset + 3] & 0xFF; - int minute = frame[offset + 4] & 0xFF; - int second = frame[offset + 5] & 0xFF; - try { - return LocalDateTime.of(year, month, day, hour, minute, second) - .atZone(ZONE_GMT8) - .toInstant(); - } catch (Exception e) { - throw new DecodeException("invalid timestamp " + year + "-" + month + "-" + day - + " " + hour + ":" + minute + ":" + second, e); - } - } - - /** 实时上报 body 解析结果：信息体列表 + 可选签名字节。 */ - public record BodyParseResult(List blocks, byte[] signature) {} -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960ParserContext.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960ParserContext.java deleted file mode 100644 index addcd7e2..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960ParserContext.java +++ /dev/null @@ -1,23 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -/** - * Body parser 调用上下文：携带选 profile 时需要的三元组。 - * - *

    由 {@link Gb32960MessageDecoder} 在解析帧时构造（vin / version 来自帧字节， - * platformAccount 由 handler 通过 channel attribute 传入）。 - * - * @param version 协议版本（来自帧起始符） - * @param vin 17 字节 VIN（trim 后），平台登入 0x05 等帧 VIN 全 0 时为空串 - * @param platformAccount 平台登入用户名；非平台对接场景为 null - * - *

    广东燃料电池扩展等厂商字段通常不是靠 VIN 就能可靠判断，上游平台账号是更稳定的路由线索。 - */ -public record Gb32960ParserContext(ProtocolVersion version, String vin, String platformAccount) { - - /** 占位上下文：版本不可缺，vin/account 允许为 null，便于内部构造和单测。 */ - public static Gb32960ParserContext minimal(ProtocolVersion version) { - return new Gb32960ParserContext(version, null, null); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/InfoBlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/InfoBlockParser.java deleted file mode 100644 index e8bc5790..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/InfoBlockParser.java +++ /dev/null @@ -1,34 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 单一信息体解析器 SPI。 - * - *

    替代旧代码里 2500 行 {@code informationReporting()} 巨方法。每个 - * {@code (ProtocolVersion, typeCode)} 组合对应一个独立实现。同一个协议 typeCode - * 在 2016 / 2025 可能语义不同（如 0x07），因此需要两个 Parser 分别声明 version。 - * - *

    实现约定： - *

      - *
    • {@link #parse(ByteBuffer)} 被调用时 buffer 的 position 已指向信息体第一个字节（不含 1 字节类型前缀） - *
    • 解析完成后 position 应恰好移动到信息体末尾 - *
    • 实现应无状态，可被单例复用 - *
    - */ -public interface InfoBlockParser { - - /** 本 Parser 适用的协议版本（{@link ProtocolVersion#V2016} 或 {@link ProtocolVersion#V2025}）。 */ - ProtocolVersion version(); - - /** 本 Parser 对应的协议信息类型标志字节（表 9）。 */ - int typeCode(); - - /** 信息体固定长度；变长类型返回 -1 并自行在 parse 中读取长度字段。 */ - int fixedLength(); - - InfoBlock parse(ByteBuffer buffer); -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/InfoBlockParserRegistry.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/InfoBlockParserRegistry.java deleted file mode 100644 index 563b96b6..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/InfoBlockParserRegistry.java +++ /dev/null @@ -1,35 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.util.HashMap; -import java.util.List; -import java.util.Map; - -/** - * 双键策略注册中心：按 {@code (ProtocolVersion, typeCode)} 索引解析器。 - * - *

    同一 typeCode 在 2016 / 2025 版本可能语义不同（如 0x07 在 2016 是报警，在 2025 是 - * 动力蓄电池最小并联单元电压），必须按版本分派。 - */ -public final class InfoBlockParserRegistry { - - private final Map parsers = new HashMap<>(); - - public InfoBlockParserRegistry(List list) { - for (InfoBlockParser p : list) { - // 后注册覆盖前注册：vendor profile 可以用同一 typeCode 替换默认解析器。 - parsers.put(new Key(p.version(), p.typeCode()), p); - } - } - - public InfoBlockParser find(ProtocolVersion version, int typeCode) { - return parsers.get(new Key(version, typeCode)); - } - - public int size() { - return parsers.size(); - } - - private record Key(ProtocolVersion version, int typeCode) {} -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/ValueDecoder.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/ValueDecoder.java deleted file mode 100644 index 4a7b39bf..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/ValueDecoder.java +++ /dev/null @@ -1,82 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import java.nio.ByteBuffer; - -/** - * 规范异常值处理辅助：GB/T 32960.3 约定 {@code 0xFE}/{@code 0xFFFE}/{@code 0xFFFFFFFE} 为异常， - * {@code 0xFF}/{@code 0xFFFF}/{@code 0xFFFFFFFF} 为无效。本类把字段读取 + 异常检查封装起来， - * 异常或无效时返回 {@code null}，让 record 字段保持 {@link Integer}/{@link Double} 装箱类型。 - */ -public final class ValueDecoder { - - private ValueDecoder() {} - - // ------------ 1 字节无符号 ------------ - - public static Integer u8(ByteBuffer buf) { - int v = buf.get() & 0xFF; - if (v == 0xFE || v == 0xFF) return null; - return v; - } - - /** 原始值，不做异常处理（状态位等不能以 null 表达时用此方法）。 */ - public static int u8raw(ByteBuffer buf) { - return buf.get() & 0xFF; - } - - // ------------ 2 字节无符号大端 ------------ - - public static Integer u16(ByteBuffer buf) { - int v = buf.getShort() & 0xFFFF; - if (v == 0xFFFE || v == 0xFFFF) return null; - return v; - } - - public static int u16raw(ByteBuffer buf) { - // 仅用于长度、数量、bitmask 等协议控制字段；业务测量值应优先走 u16/scaledU16。 - return buf.getShort() & 0xFFFF; - } - - // ------------ 4 字节无符号大端 ------------ - - public static Long u32(ByteBuffer buf) { - long v = buf.getInt() & 0xFFFFFFFFL; - if (v == 0xFFFFFFFEL || v == 0xFFFFFFFFL) return null; - return v; - } - - public static long u32raw(ByteBuffer buf) { - // 仅用于故障码、标志位等需要保留全部 bit 的字段。 - return buf.getInt() & 0xFFFFFFFFL; - } - - // ------------ 业务标度辅助 ------------ - - /** 读 u16，scale 后返回（例如 0.1 转换）。异常/无效返回 null。 */ - public static Double scaledU16(ByteBuffer buf, double scale) { - Integer raw = u16(buf); - return raw == null ? null : raw * scale; - } - - /** 读 u16 + 偏移 + scale。例：读总电流 -> scaledU16WithOffset(buf, 0.1, -1000) */ - public static Double scaledU16WithOffset(ByteBuffer buf, double scale, double offset) { - Integer raw = u16(buf); - return raw == null ? null : raw * scale + offset; - } - - public static Double scaledU32(ByteBuffer buf, double scale) { - Long raw = u32(buf); - return raw == null ? null : raw * scale; - } - - public static Double scaledU32WithOffset(ByteBuffer buf, double scale, double offset) { - Long raw = u32(buf); - return raw == null ? null : raw * scale + offset; - } - - /** 温度字段：原始值 - 40 ℃，范围 -40~+210。 */ - public static Integer tempOffset40(ByteBuffer buf) { - Integer raw = u8(buf); - return raw == null ? null : raw - 40; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/AlarmV2016BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/AlarmV2016BlockParser.java deleted file mode 100644 index 60f719db..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/AlarmV2016BlockParser.java +++ /dev/null @@ -1,46 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2016; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -// ProtocolVersion still used by version() override below - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 报警数据 (0x07) —— GB/T 32960.3-2016 附录 B 表 B.17。变长。 - * maxAlarmLevel(1) + generalAlarmFlag(4) + 4 组故障列表(count(1)+entries(4*N))。 - * - *

    2025 版报警数据 typeCode 改为 0x06 且尾部多 N5 个 (flagBit,level) —— 见 v2025 包。 - */ -public final class AlarmV2016BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x07; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - Integer maxLevel = ValueDecoder.u8(buf); - long generalFlag = ValueDecoder.u32raw(buf); - List battery = readFaultList(buf); - List motor = readFaultList(buf); - List engine = readFaultList(buf); - List other = readFaultList(buf); - return new InfoBlock.Gb32960V2016.Alarm( - maxLevel, generalFlag, battery, motor, engine, other); - } - - static List readFaultList(ByteBuffer buf) { - int count = buf.get() & 0xFF; - List out = new ArrayList<>(count); - for (int i = 0; i < count; i++) { - // 故障码按 4 字节无符号值处理，避免高位为 1 时变成负数。 - out.add(buf.getInt() & 0xFFFFFFFFL); - } - return out; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/DriveMotorV2016BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/DriveMotorV2016BlockParser.java deleted file mode 100644 index 9980b4a2..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/DriveMotorV2016BlockParser.java +++ /dev/null @@ -1,46 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2016; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 驱动电机数据 (0x02) —— GB/T 32960.3-2016 表 B.16。变长。 - * 1 字节电机个数 + N × 12 字节电机： - * serialNo(1) state(1) ctrlTempC(1) rpm(2,offset -20000) - * torque(2,offset -20000,0.1 N·m) motorTempC(1) - * ctrlVoltageV(2,0.1V) ctrlCurrentA(2,0.1A,offset -1000)。 - */ -public final class DriveMotorV2016BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x02; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int count = buf.get() & 0xFF; - List motors = new ArrayList<>(count); - for (int i = 0; i < count; i++) { - // rpm/torque 在标准里是带偏移的无符号数，先按原始无符号读，再统一减偏移。 - Integer serialNo = ValueDecoder.u8(buf); - Integer state = ValueDecoder.u8(buf); - Integer ctrlTempC = ValueDecoder.tempOffset40(buf); - Integer rawRpm = ValueDecoder.u16(buf); - Integer rpm = rawRpm == null ? null : rawRpm - 20_000; - Integer rawTq = ValueDecoder.u16(buf); - Double torque = rawTq == null ? null : (rawTq - 20_000) * 0.1; - Integer motorTempC = ValueDecoder.tempOffset40(buf); - Double ctrlVoltage = ValueDecoder.scaledU16(buf, 0.1); - Double ctrlCurrent = ValueDecoder.scaledU16WithOffset(buf, 0.1, -1000.0); - motors.add(new InfoBlock.Gb32960V2016.DriveMotor.Motor( - serialNo, state, ctrlTempC, rpm, torque, motorTempC, ctrlVoltage, ctrlCurrent)); - } - return new InfoBlock.Gb32960V2016.DriveMotor(motors); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/EngineV2016BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/EngineV2016BlockParser.java deleted file mode 100644 index 580857ae..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/EngineV2016BlockParser.java +++ /dev/null @@ -1,28 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2016; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 发动机数据 (0x04) —— GB/T 32960.3-2016 表 20。固定 5 字节： - * engineState(1) + crankshaftRpm(2) + fuelConsumptionRate(2,0.01 L/100km)。 - * 停车充电过程和增程式车辆纯电模式无需传输。 - */ -public final class EngineV2016BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x04; } - @Override public int fixedLength() { return 5; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - Integer engineState = ValueDecoder.u8(buf); - Integer crankshaftRpm = ValueDecoder.u16(buf); - Double fuelRate = ValueDecoder.scaledU16(buf, 0.01); - return new InfoBlock.Gb32960V2016.Engine(engineState, crankshaftRpm, fuelRate); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/ExtremeValueV2016BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/ExtremeValueV2016BlockParser.java deleted file mode 100644 index d3c2df23..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/ExtremeValueV2016BlockParser.java +++ /dev/null @@ -1,41 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2016; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 动力蓄电池极值数据 (0x06) —— GB/T 32960.3-2016 表 B.16。固定 14 字节。 - * 2025 版 0x06 语义已变更为报警数据。 - */ -public final class ExtremeValueV2016BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x06; } - @Override public int fixedLength() { return 14; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - // 极值块只给出子系统号/单体号/探针号及对应值，不包含完整单体数组。 - Integer maxVSub = ValueDecoder.u8(buf); - Integer maxVCell = ValueDecoder.u8(buf); - Double maxV = ValueDecoder.scaledU16(buf, 0.001); - Integer minVSub = ValueDecoder.u8(buf); - Integer minVCell = ValueDecoder.u8(buf); - Double minV = ValueDecoder.scaledU16(buf, 0.001); - Integer maxTSub = ValueDecoder.u8(buf); - Integer maxTProbe = ValueDecoder.u8(buf); - Integer maxT = ValueDecoder.tempOffset40(buf); - Integer minTSub = ValueDecoder.u8(buf); - Integer minTProbe = ValueDecoder.u8(buf); - Integer minT = ValueDecoder.tempOffset40(buf); - return new InfoBlock.Gb32960V2016.Extreme( - maxVSub, maxVCell, maxV, - minVSub, minVCell, minV, - maxTSub, maxTProbe, maxT, - minTSub, minTProbe, minT); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/FuelCellV2016BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/FuelCellV2016BlockParser.java deleted file mode 100644 index b47fa280..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/FuelCellV2016BlockParser.java +++ /dev/null @@ -1,55 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2016; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 燃料电池数据 (0x03) —— GB/T 32960.3-2016 表 B.13。变长。 - * fcVoltageV(2) + fcCurrentA(2) + hydrogenConsumption(2,0.01 kg/100km) - * + probeCount N(2) + probes[N](1B,-40 offset) + 后续氢系统 10 字节固定段。 - */ -public final class FuelCellV2016BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x03; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - Double fcVoltage = ValueDecoder.scaledU16(buf, 0.1); - Double fcCurrent = ValueDecoder.scaledU16(buf, 0.1); - Double consumption = ValueDecoder.scaledU16(buf, 0.01); - int probeCount = ValueDecoder.u16raw(buf); - - // 防御：probeCount 异常时回退为空列表避免吞 buffer - List probes = new ArrayList<>(); - int tailFixed = 2 + 1 + 2 + 1 + 2 + 1 + 1; // 尾部固定 10 字节 - // probeCount 来自车端报文，错误值会导致尾部固定段错位；这里先确保尾部还能完整解析。 - if (probeCount >= 0 && probeCount <= buf.remaining() - tailFixed) { - for (int i = 0; i < probeCount; i++) { - Integer t = ValueDecoder.tempOffset40(buf); - if (t != null) probes.add(t); - } - } - Double maxHydrogenTempC = ValueDecoder.scaledU16WithOffset(buf, 0.1, -40.0); - Integer maxHydrogenTempProbeId = ValueDecoder.u8(buf); - Double maxHydrogenConcentration = ValueDecoder.scaledU16(buf, 0.000001); - Integer maxHydrogenConcentrationProbeId = ValueDecoder.u8(buf); - Double maxHydrogenPressure = ValueDecoder.scaledU16(buf, 0.1); - Integer maxHydrogenPressureProbeId = ValueDecoder.u8(buf); - Integer hvDcDcStatus = ValueDecoder.u8(buf); - - return new InfoBlock.Gb32960V2016.FuelCell( - fcVoltage, fcCurrent, consumption, probes, - maxHydrogenTempC, maxHydrogenTempProbeId, - maxHydrogenConcentration, maxHydrogenConcentrationProbeId, - maxHydrogenPressure, maxHydrogenPressureProbeId, - hvDcDcStatus); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/PositionV2016BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/PositionV2016BlockParser.java deleted file mode 100644 index f69fbf46..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/PositionV2016BlockParser.java +++ /dev/null @@ -1,33 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2016; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 车辆位置数据 (0x05) —— GB/T 32960.3-2016 表 22。固定 9 字节： - * 状态位(1) + 经度(4,1e-6 度) + 纬度(4,1e-6 度)。 - * - *

    状态位 bit0 0=有效定位 / bit1 0=北纬 1=南纬 / bit2 0=东经 1=西经。 - */ -public final class PositionV2016BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x05; } - @Override public int fixedLength() { return 9; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int statusFlag = buf.get() & 0xFF; - long lonRaw = buf.getInt() & 0xFFFFFFFFL; - long latRaw = buf.getInt() & 0xFFFFFFFFL; - double longitude = lonRaw / 1_000_000.0; - double latitude = latRaw / 1_000_000.0; - // 标准用状态位表达南纬/西经；原始经纬度本身始终按正数传输。 - if ((statusFlag & 0b100) != 0) longitude = -longitude; - if ((statusFlag & 0b010) != 0) latitude = -latitude; - return new InfoBlock.Gb32960V2016.Position(statusFlag, longitude, latitude); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/TemperatureV2016BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/TemperatureV2016BlockParser.java deleted file mode 100644 index 939ef2a4..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/TemperatureV2016BlockParser.java +++ /dev/null @@ -1,40 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2016; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 可充电储能装置温度数据 (0x09) —— GB/T 32960.3-2016 表 B.20。变长。 - * subSystemCount(1) + 每子系统 [batteryIndex(1) + probeCount(2) + probes(probeCount*1,-40 offset)]。 - * - *

    2025 版已删除该 typeCode（动力蓄电池温度改用 0x08）。 - */ -public final class TemperatureV2016BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x09; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int subCount = buf.get() & 0xFF; - int maxT = Integer.MIN_VALUE; - int minT = Integer.MAX_VALUE; - for (int i = 0; i < subCount; i++) { - buf.get(); // batteryIndex - int probes = buf.getShort() & 0xFFFF; - // 与电压块类似，这里暂不保存每个探针，只输出可查询的最高/最低温摘要。 - for (int p = 0; p < probes; p++) { - int t = (buf.get() & 0xFF) - 40; - if (t > maxT) maxT = t; - if (t < minT) minT = t; - } - } - if (maxT == Integer.MIN_VALUE) maxT = 0; - if (minT == Integer.MAX_VALUE) minT = 0; - return new InfoBlock.Gb32960V2016.Temperature(subCount, maxT, minT); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/VehicleV2016BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/VehicleV2016BlockParser.java deleted file mode 100644 index a0df6f7c..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/VehicleV2016BlockParser.java +++ /dev/null @@ -1,44 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2016; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 整车数据 (0x01) —— GB/T 32960.3-2016 表 B.10。固定 20 字节： - * vehicleState(1) + chargingState(1) + runningMode(1) + speedKmh(2) - * + totalMileageKm(4) + totalVoltageV(2) + totalCurrentA(2) - * + socPercent(1) + dcDcStatus(1) + gearRaw(1) + insulationKohm(2) - * + accelPedal(1) + brakePedal(1)。 - * - *

    该块是实时事件和 snapshot 的核心公共字段来源，字段名变更会影响前端常用查询。 - */ -public final class VehicleV2016BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x01; } - @Override public int fixedLength() { return 20; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - Integer vehicleState = ValueDecoder.u8(buf); - Integer chargingState = ValueDecoder.u8(buf); - Integer runningMode = ValueDecoder.u8(buf); - Double speedKmh = ValueDecoder.scaledU16(buf, 0.1); - Double mileageKm = ValueDecoder.scaledU32(buf, 0.1); - Double totalVoltageV = ValueDecoder.scaledU16(buf, 0.1); - Double totalCurrentA = ValueDecoder.scaledU16WithOffset(buf, 0.1, -1000.0); - Integer soc = ValueDecoder.u8(buf); - Integer dcDcStatus = ValueDecoder.u8(buf); - Integer gearRaw = ValueDecoder.u8raw(buf); - Integer insulation = ValueDecoder.u16(buf); - Integer accelPedal = ValueDecoder.u8(buf); - Integer brakePedal = ValueDecoder.u8(buf); - return new InfoBlock.Gb32960V2016.Vehicle(vehicleState, chargingState, runningMode, - speedKmh, mileageKm, totalVoltageV, totalCurrentA, soc, dcDcStatus, gearRaw, - insulation, accelPedal, brakePedal); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/VoltageV2016BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/VoltageV2016BlockParser.java deleted file mode 100644 index 1aa5f9ba..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2016/VoltageV2016BlockParser.java +++ /dev/null @@ -1,51 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2016; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 可充电储能装置电压数据 (0x08) —— GB/T 32960.3-2016 表 B.18。变长。 - * subSystemCount(1) + 每子系统 [batteryIndex(1) + voltage(2,0.1V) + current(2,0.1A,offset -1000) - * + cellCount(2) + frameCellStart(2) + frameCellCount(1) + cellVoltages(frameCellCount*2,0.001V)]。 - * - *

    PoC 汇总：保留子系统数 + 总电压 + 单体最高/最低值。 - * 2025 版 0x08 语义已变更为动力蓄电池温度。 - */ -public final class VoltageV2016BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x08; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int subCount = buf.get() & 0xFF; - double totalV = 0; - double maxCell = Double.MIN_VALUE; - double minCell = Double.MAX_VALUE; - for (int i = 0; i < subCount; i++) { - buf.get(); // batteryIndex - double voltage = (buf.getShort() & 0xFFFF) * 0.1; - totalV += voltage; - buf.getShort(); // current (skip) - int cellCount = buf.getShort() & 0xFFFF; - buf.getShort(); // frameCellStart - int frameCellCount = buf.get() & 0xFF; - // 报文可能只上传本帧范围内的单体电压；聚合层只保存最高/最低/总压摘要。 - int actual = Math.min(frameCellCount, cellCount); - for (int c = 0; c < actual; c++) { - double cellV = (buf.getShort() & 0xFFFF) * 0.001; - if (cellV > maxCell) maxCell = cellV; - if (cellV < minCell) minCell = cellV; - } - // 如果 frameCellCount 大于 cellCount，剩余字节仍要消费掉，避免影响后续子系统解析。 - for (int c = actual; c < frameCellCount; c++) buf.getShort(); - } - if (maxCell == Double.MIN_VALUE) maxCell = 0; - if (minCell == Double.MAX_VALUE) minCell = 0; - return new InfoBlock.Gb32960V2016.Voltage(subCount, maxCell, minCell, totalV); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/AlarmV2025BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/AlarmV2025BlockParser.java deleted file mode 100644 index 0817f17b..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/AlarmV2025BlockParser.java +++ /dev/null @@ -1,53 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2025; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 报警数据 (0x06) —— GB/T 32960.3-2025 §7.2.4.9 表 23。变长。 - * 前半部分与 2016 版相同，尾部新增"通用报警故障总数(1) + N5×(flagBit(1)+level(1))"。 - */ -public final class AlarmV2025BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2025; } - @Override public int typeCode() { return 0x06; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - Integer maxLevel = ValueDecoder.u8(buf); - long generalFlag = ValueDecoder.u32raw(buf); - List battery = readFaultList(buf); - List motor = readFaultList(buf); - List engine = readFaultList(buf); - List other = readFaultList(buf); - - int generalAlarmCount = buf.hasRemaining() ? (buf.get() & 0xFF) : 0; - List levels = new ArrayList<>(generalAlarmCount); - // 2025 新增通用报警等级列表；尾部不完整时保留已读条目，避免整帧丢弃。 - for (int i = 0; i < generalAlarmCount && buf.remaining() >= 2; i++) { - int bit = buf.get() & 0xFF; - int level = buf.get() & 0xFF; - levels.add(new InfoBlock.Gb32960V2025.Alarm.GeneralAlarmEntry(bit, level)); - } - - return new InfoBlock.Gb32960V2025.Alarm( - maxLevel, generalFlag, battery, motor, engine, other, levels); - } - - private static List readFaultList(ByteBuffer buf) { - int count = buf.get() & 0xFF; - List out = new ArrayList<>(count); - for (int i = 0; i < count; i++) { - // 故障码按 4 字节无符号值处理，避免高位为 1 时变成负数。 - out.add(buf.getInt() & 0xFFFFFFFFL); - } - return out; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/BatteryTemperatureV2025BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/BatteryTemperatureV2025BlockParser.java deleted file mode 100644 index eda41c16..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/BatteryTemperatureV2025BlockParser.java +++ /dev/null @@ -1,38 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2025; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 动力蓄电池温度数据 (0x08) —— GB/T 32960.3-2025 §7.2.4.3。变长。 - * 2016 版 0x08 为可充电储能装置电压，由 v2016 包处理。 - */ -public final class BatteryTemperatureV2025BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2025; } - @Override public int typeCode() { return 0x08; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int subCount = buf.get() & 0xFF; - List packs = new ArrayList<>(subCount); - for (int i = 0; i < subCount; i++) { - int packNo = buf.get() & 0xFF; - int probeCount = buf.getShort() & 0xFFFF; - List temps = new ArrayList<>(probeCount); - // probeCount 是车端声明值；实际数组按剩余字节裁剪，防止坏包读穿。 - int safeCount = Math.min(probeCount, buf.remaining()); - for (int k = 0; k < safeCount; k++) { - temps.add((buf.get() & 0xFF) - 40); - } - packs.add(new InfoBlock.Gb32960V2025.BatteryTemperature.Pack(packNo, probeCount, temps)); - } - return new InfoBlock.Gb32960V2025.BatteryTemperature(subCount, packs); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/DriveMotorV2025BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/DriveMotorV2025BlockParser.java deleted file mode 100644 index 16c13ec8..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/DriveMotorV2025BlockParser.java +++ /dev/null @@ -1,48 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2025; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 驱动电机数据 (0x02) —— GB/T 32960.3-2025 表 16。变长。 - * 1 字节电机个数 + N × 14 字节电机： - * serialNo(1) state(1) ctrlTempC(1) rpm(2,offset -32000) - * torque(4,offset -20000,0.1 N·m) motorTempC(1) - * ctrlVoltageV(2,0.1V) ctrlCurrentA(2,0.1A,offset -1000)。 - * - *

    相比 2016 版：rpm 偏移变为 -32000；torque 由 2 字节扩展为 4 字节。 - */ -public final class DriveMotorV2025BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2025; } - @Override public int typeCode() { return 0x02; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int count = buf.get() & 0xFF; - List motors = new ArrayList<>(count); - for (int i = 0; i < count; i++) { - // 2025 版 torque 扩为 4 字节，不能沿用 2016 的 u16 解析。 - Integer serialNo = ValueDecoder.u8(buf); - Integer state = ValueDecoder.u8(buf); - Integer ctrlTempC = ValueDecoder.tempOffset40(buf); - Integer rawRpm = ValueDecoder.u16(buf); - Integer rpm = rawRpm == null ? null : rawRpm - 32_000; - Long rawTq = ValueDecoder.u32(buf); - Double torque = rawTq == null ? null : (rawTq - 20_000L) * 0.1; - Integer motorTempC = ValueDecoder.tempOffset40(buf); - Double ctrlVoltage = ValueDecoder.scaledU16(buf, 0.1); - Double ctrlCurrent = ValueDecoder.scaledU16WithOffset(buf, 0.1, -1000.0); - motors.add(new InfoBlock.Gb32960V2025.DriveMotor.Motor( - serialNo, state, ctrlTempC, rpm, torque, motorTempC, ctrlVoltage, ctrlCurrent)); - } - return new InfoBlock.Gb32960V2025.DriveMotor(motors); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/EngineV2025BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/EngineV2025BlockParser.java deleted file mode 100644 index 48661477..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/EngineV2025BlockParser.java +++ /dev/null @@ -1,27 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2025; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 发动机数据 (0x04) —— GB/T 32960.3-2025。固定 5 字节。 - * 字段与 2016 版完全一致；保留独立类只为遵守"双版本不共用"原则。 - */ -public final class EngineV2025BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2025; } - @Override public int typeCode() { return 0x04; } - @Override public int fixedLength() { return 5; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - Integer engineState = ValueDecoder.u8(buf); - Integer crankshaftRpm = ValueDecoder.u16(buf); - Double fuelRate = ValueDecoder.scaledU16(buf, 0.01); - return new InfoBlock.Gb32960V2025.Engine(engineState, crankshaftRpm, fuelRate); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/FuelCellStackV2025BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/FuelCellStackV2025BlockParser.java deleted file mode 100644 index b7bba34f..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/FuelCellStackV2025BlockParser.java +++ /dev/null @@ -1,46 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2025; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 燃料电池电堆数据 (0x30) —— GB/T 32960.3-2025 §7.2.4.6 表 18/19。变长。 - * - *

    V2016 的同 typeCode 厂商扩展由 v2016 包独立处理。 - */ -public final class FuelCellStackV2025BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2025; } - @Override public int typeCode() { return 0x30; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int stackCount = buf.get() & 0xFF; - List stacks = new ArrayList<>(stackCount); - for (int i = 0; i < stackCount; i++) { - int stackNo = buf.get() & 0xFF; - Double voltage = ValueDecoder.scaledU16(buf, 0.1); - Double current = ValueDecoder.scaledU16(buf, 0.1); - Double h2InletPressure = ValueDecoder.scaledU16WithOffset(buf, 0.1, -100.0); - Double airInletPressure = ValueDecoder.scaledU16WithOffset(buf, 0.1, -100.0); - Integer airInletTemp = ValueDecoder.tempOffset40(buf); - int probeCount = buf.getShort() & 0xFFFF; - List temps = new ArrayList<>(probeCount); - // 坏包 probeCount 过大时只消费剩余可用字节；声明数量仍保存在 stack 对象外层上下文中。 - int safeCount = Math.min(probeCount, buf.remaining()); - for (int k = 0; k < safeCount; k++) { - temps.add((buf.get() & 0xFF) - 40); - } - stacks.add(new InfoBlock.Gb32960V2025.FuelCellStack.Stack( - stackNo, voltage, current, h2InletPressure, airInletPressure, airInletTemp, temps)); - } - return new InfoBlock.Gb32960V2025.FuelCellStack(stackCount, stacks); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/FuelCellV2025BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/FuelCellV2025BlockParser.java deleted file mode 100644 index 72edbd76..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/FuelCellV2025BlockParser.java +++ /dev/null @@ -1,36 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2025; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 燃料电池发动机及车载氢系统数据 (0x03) —— GB/T 32960.3-2025 §7.2.4.5 表 17。固定 13 字节。 - */ -public final class FuelCellV2025BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2025; } - @Override public int typeCode() { return 0x03; } - @Override public int fixedLength() { return 13; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - Double maxTempC = ValueDecoder.scaledU16WithOffset(buf, 0.1, -40.0); - Integer maxTempProbeId = ValueDecoder.u8(buf); - Double maxConcentration = ValueDecoder.scaledU16(buf, 0.000001); - Integer maxConcentrationProbeId = ValueDecoder.u8(buf); - Double maxPressure = ValueDecoder.scaledU16(buf, 0.1); - Integer maxPressureProbeId = ValueDecoder.u8(buf); - Integer hvDcDcStatus = ValueDecoder.u8(buf); - Integer remainingH2Percent = ValueDecoder.u8(buf); - Integer hvDcDcControllerTempC = ValueDecoder.tempOffset40(buf); - return new InfoBlock.Gb32960V2025.FuelCell( - maxTempC, maxTempProbeId, - maxConcentration, maxConcentrationProbeId, - maxPressure, maxPressureProbeId, - hvDcDcStatus, remainingH2Percent, hvDcDcControllerTempC); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/MinParallelVoltageV2025BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/MinParallelVoltageV2025BlockParser.java deleted file mode 100644 index 54f9a638..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/MinParallelVoltageV2025BlockParser.java +++ /dev/null @@ -1,42 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2025; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 动力蓄电池最小并联单元电压数据 (0x07) —— GB/T 32960.3-2025 §7.2.4.2。变长。 - * 2016 版 0x07 为报警，由 v2016 包处理。 - */ -public final class MinParallelVoltageV2025BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2025; } - @Override public int typeCode() { return 0x07; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int subCount = buf.get() & 0xFF; - List packs = new ArrayList<>(subCount); - for (int i = 0; i < subCount; i++) { - int packNo = buf.get() & 0xFF; - Double voltage = ValueDecoder.scaledU16(buf, 0.1); - Double current = ValueDecoder.scaledU16WithOffset(buf, 0.1, -3000.0); - int totalUnits = buf.getShort() & 0xFFFF; - List frameVoltages = new ArrayList<>(totalUnits); - // 每个最小并联单元电压占 2 字节；不足时只读完整单元，避免半个值污染后续解析。 - int safeCount = Math.min(totalUnits, buf.remaining() / 2); - for (int k = 0; k < safeCount; k++) { - frameVoltages.add((buf.getShort() & 0xFFFF) * 0.001); - } - packs.add(new InfoBlock.Gb32960V2025.MinParallelVoltage.Pack( - packNo, voltage, current, totalUnits, frameVoltages)); - } - return new InfoBlock.Gb32960V2025.MinParallelVoltage(subCount, packs); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/PositionV2025BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/PositionV2025BlockParser.java deleted file mode 100644 index d6a65c84..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/PositionV2025BlockParser.java +++ /dev/null @@ -1,34 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2025; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 车辆位置数据 (0x05) —— GB/T 32960.3-2025。固定 10 字节， - * 比 2016 版多一个坐标系字节： - * 状态位(1) + 坐标系(1) + 经度(4) + 纬度(4)。 - * - *

    坐标系：0x01=WGS84 / 0x02=GCJ-02 / 0x03=其他。 - */ -public final class PositionV2025BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2025; } - @Override public int typeCode() { return 0x05; } - @Override public int fixedLength() { return 10; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int statusFlag = buf.get() & 0xFF; - int coordSystem = buf.get() & 0xFF; - long lonRaw = buf.getInt() & 0xFFFFFFFFL; - long latRaw = buf.getInt() & 0xFFFFFFFFL; - double longitude = lonRaw / 1_000_000.0; - double latitude = latRaw / 1_000_000.0; - if ((statusFlag & 0b100) != 0) longitude = -longitude; - if ((statusFlag & 0b010) != 0) latitude = -latitude; - return new InfoBlock.Gb32960V2025.Position(statusFlag, coordSystem, longitude, latitude); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/SuperCapacitorExtremeV2025BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/SuperCapacitorExtremeV2025BlockParser.java deleted file mode 100644 index 919e6c56..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/SuperCapacitorExtremeV2025BlockParser.java +++ /dev/null @@ -1,40 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2025; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 超级电容器极值数据 (0x32) —— GB/T 32960.3-2025 §7.2.4.11 表 26。固定 18 字节。 - */ -public final class SuperCapacitorExtremeV2025BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2025; } - @Override public int typeCode() { return 0x32; } - @Override public int fixedLength() { return 18; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - // 2025 超级电容极值的单体/探针编号为 2 字节，区别于 2016 蓄电池极值的 1 字节编号。 - Integer maxVSys = ValueDecoder.u8(buf); - Integer maxVCell = ValueDecoder.u16(buf); - Double maxV = ValueDecoder.scaledU16(buf, 0.001); - Integer minVSys = ValueDecoder.u8(buf); - Integer minVCell = ValueDecoder.u16(buf); - Double minV = ValueDecoder.scaledU16(buf, 0.001); - Integer maxTSys = ValueDecoder.u8(buf); - Integer maxTProbe = ValueDecoder.u16(buf); - Integer maxT = ValueDecoder.tempOffset40(buf); - Integer minTSys = ValueDecoder.u8(buf); - Integer minTProbe = ValueDecoder.u16(buf); - Integer minT = ValueDecoder.tempOffset40(buf); - return new InfoBlock.Gb32960V2025.SuperCapacitorExtreme( - maxVSys, maxVCell, maxV, - minVSys, minVCell, minV, - maxTSys, maxTProbe, maxT, - minTSys, minTProbe, minT); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/SuperCapacitorV2025BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/SuperCapacitorV2025BlockParser.java deleted file mode 100644 index 96655812..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/SuperCapacitorV2025BlockParser.java +++ /dev/null @@ -1,42 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2025; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 超级电容器数据 (0x31) —— GB/T 32960.3-2025 §7.2.4.10 表 25。变长。 - */ -public final class SuperCapacitorV2025BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2025; } - @Override public int typeCode() { return 0x31; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int systemNo = buf.get() & 0xFF; - Double totalVoltage = ValueDecoder.scaledU16(buf, 0.1); - Double totalCurrent = ValueDecoder.scaledU16WithOffset(buf, 0.1, -3000.0); - int cellCount = buf.getShort() & 0xFFFF; - List cells = new ArrayList<>(cellCount); - // 电压数组按 2 字节一个值裁剪，避免声明数量异常时读穿温度段。 - int safeCellCount = Math.min(cellCount, buf.remaining() / 2); - for (int i = 0; i < safeCellCount; i++) { - cells.add((buf.getShort() & 0xFFFF) * 0.001); - } - int probeCount = buf.hasRemaining() ? (buf.getShort() & 0xFFFF) : 0; - List temps = new ArrayList<>(probeCount); - // 温度探针每个 1 字节，按剩余字节裁剪。 - int safeProbeCount = Math.min(probeCount, buf.remaining()); - for (int i = 0; i < safeProbeCount; i++) { - temps.add((buf.get() & 0xFF) - 40); - } - return new InfoBlock.Gb32960V2025.SuperCapacitor(systemNo, totalVoltage, totalCurrent, cells, temps); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/VehicleV2025BlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/VehicleV2025BlockParser.java deleted file mode 100644 index e6ee6a40..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/v2025/VehicleV2025BlockParser.java +++ /dev/null @@ -1,39 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.v2025; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 整车数据 (0x01) —— GB/T 32960.3-2025。固定 18 字节。 - * 相比 2016 版删除加速踏板和制动踏板。 - * - *

    为保持查询接口稳定，后续转换层会把 2016/2025 两版整车字段映射到同一组逻辑字段。 - */ -public final class VehicleV2025BlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2025; } - @Override public int typeCode() { return 0x01; } - @Override public int fixedLength() { return 18; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - Integer vehicleState = ValueDecoder.u8(buf); - Integer chargingState = ValueDecoder.u8(buf); - Integer runningMode = ValueDecoder.u8(buf); - Double speedKmh = ValueDecoder.scaledU16(buf, 0.1); - Double mileageKm = ValueDecoder.scaledU32(buf, 0.1); - Double totalVoltageV = ValueDecoder.scaledU16(buf, 0.1); - Double totalCurrentA = ValueDecoder.scaledU16WithOffset(buf, 0.1, -1000.0); - Integer soc = ValueDecoder.u8(buf); - Integer dcDcStatus = ValueDecoder.u8(buf); - Integer gearRaw = ValueDecoder.u8raw(buf); - Integer insulation = ValueDecoder.u16(buf); - return new InfoBlock.Gb32960V2025.Vehicle(vehicleState, chargingState, runningMode, - speedKmh, mileageKm, totalVoltageV, totalCurrentA, soc, dcDcStatus, gearRaw, - insulation); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcAirConditionerBlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcAirConditionerBlockParser.java deleted file mode 100644 index 67a822d9..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcAirConditionerBlockParser.java +++ /dev/null @@ -1,34 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 广东燃料电池规范 §7.2.3.7 表 18：空调数据。typeCode {@code 0x33}。 - * - *

    固定 5 字节： - * 

    - *   status(1)                        0 关闭 / 1 启动
- *   powerKw(2, offset -5, 1 kw, range -5~+30)
- *   compressorInputVoltageV(2, 0.1V, 0~1000V)
- *
    - */ -public final class GdFcAirConditionerBlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x33; } - @Override public int fixedLength() { return 5; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - Integer status = ValueDecoder.u8raw(buf); - int rawPower = buf.getShort() & 0xFFFF; - Integer power = (rawPower == 0xFFFE || rawPower == 0xFFFF) ? null : rawPower - 5; - Double inputV = ValueDecoder.scaledU16(buf, 0.1); - return new InfoBlock.GuangdongFc.AirConditioner(status, power, inputV); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcAuxiliaryBlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcAuxiliaryBlockParser.java deleted file mode 100644 index 919a6ee9..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcAuxiliaryBlockParser.java +++ /dev/null @@ -1,61 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 广东燃料电池规范 §7.2.3.5 表 15/16：辅助系统数据。typeCode {@code 0x31}。变长。 - * - * 
    - *   subSystemCount(1)
- *   For each:
- *     airCompressorMotorVoltageV(2, 0.1V, 0~1000V)
- *     airCompressorPowerKw(2, offset -5, 1 kw, range -5~+30)
- *     hydrogenPumpMotorVoltageV(2, 0.2V, 0~50V)
- *     hydrogenPumpPowerKw(2, offset -5, 1 kw)
- *     waterPumpVoltageV(2, 0.1V)
- *     ptcVoltageV(2, 0.1V)
- *     ptcPowerKw(2, offset -5, 1 kw)
- *     lowVoltageBatteryVoltageV(2, 0.1V, 0~32V)
- *
    - * - * 每子系统 16 字节。 - */ -public final class GdFcAuxiliaryBlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x31; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int count = buf.get() & 0xFF; - List subs = new ArrayList<>(count); - for (int i = 0; i < count; i++) { - Double airCompV = ValueDecoder.scaledU16(buf, 0.1); - Integer airCompP = readPowerOffsetMinus5(buf); - Double h2PumpV = ValueDecoder.scaledU16(buf, 0.2); - Integer h2PumpP = readPowerOffsetMinus5(buf); - Double waterPumpV = ValueDecoder.scaledU16(buf, 0.1); - Double ptcV = ValueDecoder.scaledU16(buf, 0.1); - Integer ptcP = readPowerOffsetMinus5(buf); - Double lvBattV = ValueDecoder.scaledU16(buf, 0.1); - subs.add(new InfoBlock.GuangdongFc.Auxiliary.Subsystem( - airCompV, airCompP, h2PumpV, h2PumpP, waterPumpV, ptcV, ptcP, lvBattV)); - } - return new InfoBlock.GuangdongFc.Auxiliary(count, subs); - } - - /** 功率字段：raw(2B) - 5，0xFFFE/0xFFFF 异常/无效。 */ - private static Integer readPowerOffsetMinus5(ByteBuffer buf) { - int raw = buf.getShort() & 0xFFFF; - if (raw == 0xFFFE || raw == 0xFFFF) return null; - return raw - 5; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcDcDcBlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcDcDcBlockParser.java deleted file mode 100644 index 7d088197..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcDcDcBlockParser.java +++ /dev/null @@ -1,39 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 广东燃料电池规范 §7.2.3.6 表 17：DC/DC（燃料电池系统）数据。typeCode {@code 0x32}。 - * - *

    固定 9 字节： - * 

    - *   inputVoltageV(2, 0.1V, 0~400V)
- *   inputCurrentA(2, 0.1A, 0~600A)
- *   outputVoltageV(2, 0.1V, 0~700V)
- *   outputCurrentA(2, 0.1A, 0~600A)
- *   controllerTempC(1, -40 offset)
- *
    - * - *

    只有 vendor profile 命中 {@code guangdong-fc} 时才会解析到该结构；否则同一 typeCode 会落 Raw。 - */ -public final class GdFcDcDcBlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x32; } - @Override public int fixedLength() { return 9; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - Double inV = ValueDecoder.scaledU16(buf, 0.1); - Double inC = ValueDecoder.scaledU16(buf, 0.1); - Double outV = ValueDecoder.scaledU16(buf, 0.1); - Double outC = ValueDecoder.scaledU16(buf, 0.1); - Integer ctrlTemp = ValueDecoder.tempOffset40(buf); - return new InfoBlock.GuangdongFc.DcDc(inV, inC, outV, outC, ctrlTemp); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcDemoExtensionBlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcDemoExtensionBlockParser.java deleted file mode 100644 index 8457d615..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcDemoExtensionBlockParser.java +++ /dev/null @@ -1,47 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 广东燃料电池规范 §7.2.3.14 表 27：燃料电池汽车示范应用数据扩展。typeCode {@code 0x80}。 - * - *

    结构（typeCode 由外层消费）： - * 

    - *   dataLength (2B WORD) = 9     表示后续扩展数据长度
- *   stackTempC (1B, -40 offset, -40~+210°C)
- *   airCompressorVoltageV (2B, 0.1V, 0~2000V)
- *   airCompressorCurrentA (2B, 0.1A, -1000 offset, -1000~+1000A)
- *   hydrogenPumpVoltageV (2B, 0.1V, 0~2000V)
- *   hydrogenPumpCurrentA (2B, 0.1A, -1000 offset, -1000~+1000A)
- *
    - * - * 含 length 字段共 11 字节 body（不含外层 1B typeCode）。 - */ -public final class GdFcDemoExtensionBlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x80; } - @Override public int fixedLength() { return 11; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int declaredLen = buf.getShort() & 0xFFFF; - // declaredLen 通常 = 9，但即便不是也按 9 字节固定布局解，避免无谓的失败。 - Integer stackTemp = ValueDecoder.tempOffset40(buf); - Double acV = ValueDecoder.scaledU16(buf, 0.1); - Double acC = ValueDecoder.scaledU16WithOffset(buf, 0.1, -1000.0); - Double h2V = ValueDecoder.scaledU16(buf, 0.1); - Double h2C = ValueDecoder.scaledU16WithOffset(buf, 0.1, -1000.0); - // 防御：若 declaredLen 与实际不一致，仅在 DEBUG 留痕。 - if (declaredLen != 9 && org.slf4j.LoggerFactory.getLogger(GdFcDemoExtensionBlockParser.class).isDebugEnabled()) { - org.slf4j.LoggerFactory.getLogger(GdFcDemoExtensionBlockParser.class) - .debug("[gb32960/gd-fc/0x80] declaredLen={} (expected 9), parsed by fixed layout", declaredLen); - } - return new InfoBlock.GuangdongFc.DemoExtension(stackTemp, acV, acC, h2V, h2C); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcStackBlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcStackBlockParser.java deleted file mode 100644 index 1e6d4c43..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcStackBlockParser.java +++ /dev/null @@ -1,77 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.List; - -/** - * 广东燃料电池汽车示范应用城市群综合监管平台 §7.2.3.4 表 13/14：燃料电池电堆数据。 - * - *

    typeCode {@code 0x30}，工程上观察到对端在 GB/T 32960.3-2016 帧（{@code 2323} - * 起始符）里下发该块。本规范在 GB/T 32960.3-2016 标准里属于 {@code 0x30~0x7F} 预留区段， - * 因此只有当对端被 vendor profile 选中后才启用此 parser。 - * - *

    结构： - * 

    - *   stackCount (1B)
- *   For each stack:
- *     engineWorkState (1B)                         00 关闭 / 01 打开 / FE 异常 / FF 无效
- *     stackWaterOutletTempC (1B, -40 offset)
- *     hydrogenInletPressureKpa (2B WORD, -100 offset, 0.1 kPa)
- *     airInletPressureKpa (2B WORD, -100 offset, 0.1 kPa)
- *     airInletTempC (1B, -40 offset)
- *     maxCellVoltageId (2B WORD)
- *     minCellVoltageId (2B WORD)
- *     maxCellVoltageV (2B WORD, 0.001 V)
- *     minCellVoltageV (2B WORD, 0.001 V)
- *     avgCellVoltageV (2B WORD, 0.001 V)
- *     cellCount (2B WORD)
- *     frameCellStart (2B WORD)
- *     frameCellCount (1B)
- *     frameCellVoltagesV (2 * frameCellCount, 0.001 V)
- *
    - */ -public final class GdFcStackBlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x30; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int stackCount = buf.get() & 0xFF; - List stacks = new ArrayList<>(stackCount); - for (int i = 0; i < stackCount; i++) { - Integer workState = ValueDecoder.u8raw(buf); - Integer waterTemp = ValueDecoder.tempOffset40(buf); - Double h2InletP = ValueDecoder.scaledU16WithOffset(buf, 0.1, -100.0); - Double airInletP = ValueDecoder.scaledU16WithOffset(buf, 0.1, -100.0); - Integer airInletTemp = ValueDecoder.tempOffset40(buf); - Integer maxId = ValueDecoder.u16(buf); - Integer minId = ValueDecoder.u16(buf); - Double maxV = ValueDecoder.scaledU16(buf, 0.001); - Double minV = ValueDecoder.scaledU16(buf, 0.001); - Double avgV = ValueDecoder.scaledU16(buf, 0.001); - Integer cellCount = ValueDecoder.u16(buf); - Integer frameStart = ValueDecoder.u16(buf); - int frameCells = buf.get() & 0xFF; - List frameVoltages = new ArrayList<>(frameCells); - // 对端会把单体电压按 frameCellStart/frameCellCount 分帧上送；这里只解析当前帧片段， - // 完整电压数组由查询层按同一 vin+eventTime 合并，避免入站侧持有跨帧状态。 - int safe = Math.min(frameCells, buf.remaining() / 2); - for (int k = 0; k < safe; k++) { - frameVoltages.add((buf.getShort() & 0xFFFF) * 0.001); - } - stacks.add(new InfoBlock.GuangdongFc.Stack.Item( - workState, waterTemp, h2InletP, airInletP, airInletTemp, - maxId, minId, maxV, minV, avgV, - cellCount, frameStart, frameCells, frameVoltages)); - } - return new InfoBlock.GuangdongFc.Stack(stackCount, stacks); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcVehicleInfoBlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcVehicleInfoBlockParser.java deleted file mode 100644 index f2df20b3..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcVehicleInfoBlockParser.java +++ /dev/null @@ -1,37 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.ValueDecoder; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 广东燃料电池规范 §7.2.3.8 表 19：车辆信息（补充）。typeCode {@code 0x34}。 - * - *

    固定 7 字节： - * 

    - *   collisionAlarm(1)                0 无 / 1 有
- *   ambientTempC(2, -40 offset, 1°C)
- *   ambientPressureKpa(2, -100 offset, 0.1 kPa)
- *   hydrogenMassKg(2, 0.1 kg)
- *
    - */ -public final class GdFcVehicleInfoBlockParser implements InfoBlockParser { - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x34; } - @Override public int fixedLength() { return 7; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - Integer collision = ValueDecoder.u8raw(buf); - int rawTemp = buf.getShort() & 0xFFFF; - // 规范保留值按缺失处理，避免 snapshot 中出现 65494/65495 这种不可读温度。 - Integer tempC = (rawTemp == 0xFFFE || rawTemp == 0xFFFF) ? null : rawTemp - 40; - Double pressure = ValueDecoder.scaledU16WithOffset(buf, 0.1, -100.0); - Double h2Mass = ValueDecoder.scaledU16(buf, 0.1); - return new InfoBlock.GuangdongFc.VehicleInfo(collision, tempC, pressure, h2Mass); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcVendorTlvBlockParser.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcVendorTlvBlockParser.java deleted file mode 100644 index 16e22ced..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcVendorTlvBlockParser.java +++ /dev/null @@ -1,57 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; - -import java.nio.ByteBuffer; - -/** - * 广东燃料电池 peer 的厂商私有 TLV 块兜底 parser。 - * - *

    同一 peer 在广东规范 v1.0 表 8 列出的 0x30~0x34 / 0x80 之外，**还会下发**一些 - * 在该规范里完全未定义的扩展 typeCode（线上首先观察到 {@code 0x83}，未来不排除有 - * 0x84 等）。这些块经验上是 **TLV 格式**： - * - * 

    - *   typeCode (1B, 由 BodyParser 外层消费)
- *   length   (2B WORD, 大端)
- *   payload  (length 字节)
- *
    - * - *

    本 parser 严格按 length 消费字节，不贪心读到 buffer 末尾，保证 BodyParser - * 主循环在该块之后仍能继续解析后面的标准/扩展块。 - * - *

    typeCode 通过构造器注入，未来若发现新的 vendor typeCode，只需在 catalog 里多注册 - * 一个 {@code new GdFcVendorTlvBlockParser(0x84)} 即可，无需新写 parser 类。 - * - *

    length 字段宽度为 2 字节是从 {@code 0x83} 实际字节序列推断而来（0x0024=36 与 - * payload 实际长度匹配）；如未来对端的某个 typeCode 用 1B length 而非 2B，将出现 - * 字节漂移并被 BodyParser 的 hex dump 立刻暴露，可针对性新增独立 parser。 - */ -public final class GdFcVendorTlvBlockParser implements InfoBlockParser { - - private final int typeCode; - - public GdFcVendorTlvBlockParser(int typeCode) { - if (typeCode < 0 || typeCode > 0xFF) { - throw new IllegalArgumentException("typeCode out of range: " + typeCode); - } - this.typeCode = typeCode; - } - - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return typeCode; } - @Override public int fixedLength() { return -1; } - - @Override - public InfoBlock parse(ByteBuffer buf) { - int len = buf.getShort() & 0xFFFF; - // 防御：若 length 大于剩余可读字节，截短到剩余长度（避免 BufferUnderflowException） - // declaredLength 仍保留原值，后续可以在 API/日志里看出对端长度声明是否异常。 - int safe = Math.min(len, buf.remaining()); - byte[] payload = new byte[safe]; - buf.get(payload); - return new InfoBlock.GuangdongFc.VendorTlv(typeCode, len, payload); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/Gb32960ProfileRegistry.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/Gb32960ProfileRegistry.java deleted file mode 100644 index 881029ef..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/Gb32960ProfileRegistry.java +++ /dev/null @@ -1,65 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.profile; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParserRegistry; - -import java.util.ArrayList; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.Set; - -/** - * 命名 profile → {@link InfoBlockParserRegistry} 映射。 - * - *

    构造时接收： - *

      - *
    • {@code defaultParsers} —— 标准 GB/T 32960 parser 集合（v2016 + v2025） - *
    • {@code catalog} —— vendor 扩展套件目录 - *
    • {@code enabledExtensionNames} —— 在配置文件里被引用过的 vendor 套件名集合 - *
    - * - * 构造内部为 default profile 和每个 enabled 扩展套件分别构建一个 registry： - * 套件 registry = default parsers + catalog 里该套件的 parsers。运行期通过 - * {@link #forProfile(String)} 按名字查找；查不到或传 null 时返回 default。 - */ -public class Gb32960ProfileRegistry { - - /** 约定的默认 profile 名（不会出现在配置里，仅作为 forProfile(null) 的 fallback 标识）。 */ - public static final String DEFAULT_PROFILE = "__default__"; - - private final InfoBlockParserRegistry defaultRegistry; - private final Map registriesByProfile; - - public Gb32960ProfileRegistry(List defaultParsers, - VendorExtensionCatalog catalog, - Set enabledExtensionNames) { - this.defaultRegistry = new InfoBlockParserRegistry(defaultParsers); - Map map = new HashMap<>(); - for (String name : enabledExtensionNames) { - List vendorParsers = catalog.parsersFor(name); - if (vendorParsers.isEmpty()) { - throw new IllegalStateException( - "vendor extension '" + name + "' referenced in config but not present in catalog. " - + "Available: " + catalog.names()); - } - List combined = new ArrayList<>(defaultParsers.size() + vendorParsers.size()); - combined.addAll(defaultParsers); - // vendor parser 只新增扩展 typeCode，不应覆盖标准 parser；若未来需覆盖，必须显式设计优先级。 - combined.addAll(vendorParsers); - map.put(name, new InfoBlockParserRegistry(combined)); - } - this.registriesByProfile = Map.copyOf(map); - } - - /** 按 profile 名取 registry；null 或未注册的名字返回默认 registry。 */ - public InfoBlockParserRegistry forProfile(String profileName) { - if (profileName == null) return defaultRegistry; - InfoBlockParserRegistry r = registriesByProfile.get(profileName); - return r != null ? r : defaultRegistry; - } - - public InfoBlockParserRegistry defaultRegistry() { - return defaultRegistry; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/RuleBasedVendorExtensionSelector.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/RuleBasedVendorExtensionSelector.java deleted file mode 100644 index 83561ed9..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/RuleBasedVendorExtensionSelector.java +++ /dev/null @@ -1,118 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.profile; - -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960ParserContext; -import com.lingniu.ingest.protocol.gb32960.config.Gb32960Properties; - -import java.util.ArrayList; -import java.util.List; -import java.util.Locale; -import java.util.Map; -import java.util.Optional; -import java.util.Set; -import java.util.concurrent.ConcurrentHashMap; - -/** - * 配置驱动的 vendor 扩展选择器。按 {@link Gb32960Properties#getVendorExtensions()} 配置的 - * 顺序自上而下首匹（first-match-wins），命中的 entry 的 {@code name} 即返回值。 - * - *

    命中字段：{@code platformAccount} / {@code vin} / {@code vinPrefix}（任一命中即整条 - * entry 命中）。比较为 case-insensitive。 - * - *

    结果按 {@code (account, vin)} 缓存以避免每帧重复扫描；约定 cache key 不含 version - * 因为版本不参与匹配。Cache 用 {@link Optional} 包裹以同时缓存 "no match"。 - */ -public final class RuleBasedVendorExtensionSelector implements VendorExtensionSelector { - - private final List rules; - private final ConcurrentHashMap> cache = new ConcurrentHashMap<>(); - - public RuleBasedVendorExtensionSelector(List entries, - Set validNames) { - List compiled = new ArrayList<>(entries.size()); - for (Gb32960Properties.VendorExtension e : entries) { - if (e.getName() == null || e.getName().isBlank()) { - throw new IllegalStateException("vendor extension entry missing 'name'"); - } - if (!validNames.contains(e.getName())) { - throw new IllegalStateException( - "vendor extension '" + e.getName() + "' not registered in catalog. " - + "Available: " + validNames); - } - compiled.add(CompiledRule.from(e)); - } - this.rules = List.copyOf(compiled); - } - - @Override - public String select(Gb32960ParserContext ctx) { - if (rules.isEmpty()) return null; - CacheKey key = new CacheKey(normalize(ctx.platformAccount()), normalize(ctx.vin())); - Optional cached = cache.get(key); - if (cached != null) return cached.orElse(null); - // 32960 高频上报通常同一连接/同一 VIN 连续上报，缓存可避免每帧扫描所有规则。 - String hit = scan(key); - cache.put(key, Optional.ofNullable(hit)); - return hit; - } - - private String scan(CacheKey key) { - for (CompiledRule r : rules) { - if (r.matches(key.account, key.vin)) return r.name; - } - return null; - } - - private static String normalize(String s) { - return s == null || s.isEmpty() ? null : s.toUpperCase(Locale.ROOT); - } - - /** 暴露给测试和 actuator 端点（如果以后加）。 */ - public int cacheSize() { return cache.size(); } - - private record CacheKey(String account, String vin) {} - - /** 预编译后的单条规则：所有列表已 normalize 成大写 set。 */ - private static final class CompiledRule { - final String name; - final Set accounts; - final Set vins; - final List vinPrefixes; - - CompiledRule(String name, Set accounts, Set vins, List vinPrefixes) { - this.name = name; - this.accounts = accounts; - this.vins = vins; - this.vinPrefixes = vinPrefixes; - } - - static CompiledRule from(Gb32960Properties.VendorExtension e) { - Gb32960Properties.VendorExtension.Match m = e.getMatch(); - return new CompiledRule( - e.getName(), - Set.copyOf(toUpper(m.getPlatformAccounts())), - Set.copyOf(toUpper(m.getVins())), - List.copyOf(toUpper(m.getVinPrefixes()))); - } - - boolean matches(String account, String vin) { - if (account != null && accounts.contains(account)) return true; - if (vin != null && vins.contains(vin)) return true; - if (vin != null) { - for (String prefix : vinPrefixes) { - if (vin.startsWith(prefix)) return true; - } - } - return false; - } - - private static List toUpper(List list) { - if (list == null || list.isEmpty()) return List.of(); - List out = new ArrayList<>(list.size()); - for (String s : list) { - if (s == null || s.isBlank()) continue; - out.add(s.trim().toUpperCase(Locale.ROOT)); - } - return out; - } - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/VendorExtensionCatalog.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/VendorExtensionCatalog.java deleted file mode 100644 index 3d89a9da..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/VendorExtensionCatalog.java +++ /dev/null @@ -1,41 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.profile; - -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; - -import java.util.Collections; -import java.util.List; -import java.util.Map; - -/** - * Vendor 扩展套件目录：从套件名映射到该套件包含的 {@link InfoBlockParser} 列表。 - * - *

    Catalog 在启动时由 {@code Gb32960AutoConfiguration} 一次性构建并注入到 - * {@link Gb32960ProfileRegistry}，运行期不可变。每个 entry 代表一个**已实现的**厂商扩展 - * 协议（如广东燃料电池汽车示范规范），其 parser 集合会在选中时**追加**到默认标准 parser - * 集合之上，组成完整的 profile registry。 - * - *

    当前内置 key： - *

      - *
    • {@code guangdong-fc} —— 广东燃料电池汽车示范规范 0x30~0x34、0x80 - *
    - */ -public final class VendorExtensionCatalog { - - private final Map> extensions; - - public VendorExtensionCatalog(Map> extensions) { - // 只冻结顶层 map；parser 实例本身按无状态单例使用，不应在运行期修改。 - this.extensions = Map.copyOf(extensions); - } - - /** 已注册的全部 vendor 扩展名集合。 */ - public java.util.Set names() { - return extensions.keySet(); - } - - /** 取指定 vendor 扩展套件的 parser 列表；不存在返回空列表。 */ - public List parsersFor(String name) { - List list = extensions.get(name); - return list == null ? Collections.emptyList() : list; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/VendorExtensionSelector.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/VendorExtensionSelector.java deleted file mode 100644 index cbfd408d..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/profile/VendorExtensionSelector.java +++ /dev/null @@ -1,21 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.profile; - -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960ParserContext; - -/** - * 给定上下文返回应用的 vendor 扩展套件名（{@code null} 表示走默认 profile）。 - * - *

    实现需要线程安全且 O(1) 期望复杂度（建议内部缓存）。 - * 命中结果会直接影响 0x30+ 等厂商字段是否解析成结构化字段，还是退化为 Raw。 - */ -public interface VendorExtensionSelector { - - /** - * @return 命中的扩展套件名（与 {@code Gb32960ProfileRegistry} 注册名一致）， - * 未命中返回 {@code null}。 - */ - String select(Gb32960ParserContext ctx); - - /** 永远返回 null 的实现，用于不启用任何 vendor 扩展的默认场景。 */ - VendorExtensionSelector NONE = ctx -> null; -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfiguration.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfiguration.java deleted file mode 100644 index 513b3f51..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfiguration.java +++ /dev/null @@ -1,253 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.config; - -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.protocol.gb32960.auth.Gb32960PlatformAuthorizer; -import com.lingniu.ingest.protocol.gb32960.auth.Gb32960VinAuthorizer; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960BodyParser; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960CommandParser; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParserRegistry; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.AlarmV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.DriveMotorV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.EngineV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.ExtremeValueV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.FuelCellV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.PositionV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.TemperatureV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VoltageV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.AlarmV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.BatteryTemperatureV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.DriveMotorV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.EngineV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.FuelCellStackV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.FuelCellV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.MinParallelVoltageV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.PositionV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.SuperCapacitorExtremeV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.SuperCapacitorV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.VehicleV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcAirConditionerBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcAuxiliaryBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcDcDcBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcDemoExtensionBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcStackBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcVehicleInfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcVendorTlvBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.profile.Gb32960ProfileRegistry; -import com.lingniu.ingest.protocol.gb32960.codec.profile.RuleBasedVendorExtensionSelector; -import com.lingniu.ingest.protocol.gb32960.codec.profile.VendorExtensionCatalog; -import com.lingniu.ingest.protocol.gb32960.codec.profile.VendorExtensionSelector; -import com.lingniu.ingest.protocol.gb32960.handler.Gb32960RealtimeHandler; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960AccessService; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960AckService; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960ConnectionDiagnostics; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960FrameDiagnostics; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960NettyServer; -import com.lingniu.ingest.protocol.gb32960.mapper.Gb32960EventMapper; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; - -import java.util.List; - -/** - * GB/T 32960.3 协议接入模块的自动装配。 - * - *

    每个 Parser Bean 都会被 {@link InfoBlockParserRegistry} 按 - * {@code (ProtocolVersion, typeCode)} 注册。2016/2025 可同时加载。 - * - *

    只有 {@code lingniu.ingest.gb32960.enabled=true} 时才会装配 32960 协议解析组件。 - * TCP 监听由 {@code lingniu.ingest.gb32960.server.enabled} 单独控制。 - */ -@AutoConfiguration -@EnableConfigurationProperties(Gb32960Properties.class) -@ConditionalOnProperty(prefix = "lingniu.ingest.gb32960", name = "enabled", havingValue = "true") -public class Gb32960AutoConfiguration { - - // ================= 2016 版信息体 Parser（com...parser.v2016 包） ================= - - @Bean public InfoBlockParser gb32960V2016Vehicle() { return new VehicleV2016BlockParser(); } - @Bean public InfoBlockParser gb32960V2016DriveMotor() { return new DriveMotorV2016BlockParser(); } - @Bean public InfoBlockParser gb32960V2016FuelCell() { return new FuelCellV2016BlockParser(); } - @Bean public InfoBlockParser gb32960V2016Engine() { return new EngineV2016BlockParser(); } - @Bean public InfoBlockParser gb32960V2016Position() { return new PositionV2016BlockParser(); } - @Bean public InfoBlockParser gb32960V2016Extreme() { return new ExtremeValueV2016BlockParser(); } - @Bean public InfoBlockParser gb32960V2016Alarm() { return new AlarmV2016BlockParser(); } - @Bean public InfoBlockParser gb32960V2016Voltage() { return new VoltageV2016BlockParser(); } - @Bean public InfoBlockParser gb32960V2016Temperature() { return new TemperatureV2016BlockParser(); } - // 注意：0x30~0x7F 在 GB/T 32960.3-2016 附录 B 表 B.3 是"预留"区段，没有任何字段定义。 - // peer 在 V2016 帧里发 0x30/0x31/0x32 是越界使用，强行用 2025 字段布局解读会得到非物理值。 - // 故 V2016 不注册任何 0x30+ parser；遇到时由 Gb32960BodyParser 走 Raw 兜底并打 WARN。 - - // ================= 2025 版信息体 Parser（com...parser.v2025 包） ================= - - @Bean public InfoBlockParser gb32960V2025Vehicle() { return new VehicleV2025BlockParser(); } - @Bean public InfoBlockParser gb32960V2025DriveMotor() { return new DriveMotorV2025BlockParser(); } - @Bean public InfoBlockParser gb32960V2025FuelCell() { return new FuelCellV2025BlockParser(); } - @Bean public InfoBlockParser gb32960V2025Engine() { return new EngineV2025BlockParser(); } - @Bean public InfoBlockParser gb32960V2025Position() { return new PositionV2025BlockParser(); } - @Bean public InfoBlockParser gb32960V2025Alarm() { return new AlarmV2025BlockParser(); } - @Bean public InfoBlockParser gb32960V2025MinParallelVoltage() { return new MinParallelVoltageV2025BlockParser(); } - @Bean public InfoBlockParser gb32960V2025BatteryTemp() { return new BatteryTemperatureV2025BlockParser(); } - @Bean public InfoBlockParser gb32960V2025FuelCellStack() { return new FuelCellStackV2025BlockParser(); } - @Bean public InfoBlockParser gb32960V2025SuperCap() { return new SuperCapacitorV2025BlockParser(); } - @Bean public InfoBlockParser gb32960V2025SuperCapExtreme() { return new SuperCapacitorExtremeV2025BlockParser(); } - - // ================= Vendor 扩展套件目录 ================= - - /** - * 内置 vendor 扩展套件目录。当前注册： - *

      - *
    • {@code guangdong-fc} —— 广东燃料电池汽车示范规范 0x30~0x34、0x80 - *
    - * 未来新增其他厂商扩展时直接在这里加 entry。 - */ - @Bean - @ConditionalOnMissingBean - public VendorExtensionCatalog gb32960VendorExtensionCatalog() { - return new VendorExtensionCatalog(java.util.Map.of( - "guangdong-fc", List.of( - new GdFcStackBlockParser(), - new GdFcAuxiliaryBlockParser(), - new GdFcDcDcBlockParser(), - new GdFcAirConditionerBlockParser(), - new GdFcVehicleInfoBlockParser(), - new GdFcDemoExtensionBlockParser(), - // 0x83 厂商私有 TLV 兜底（同 peer 在广东规范之外又下发的未公开扩展） - // 这里只注册已经在线上见过的 typeCode；新增 typeCode 应先确认长度字段格式。 - new GdFcVendorTlvBlockParser(0x83)))); - } - - // ================= Core components ================= - - /** - * Profile 注册表：把全部已注入的标准 parser 作为 default profile， - * 同时按配置 {@code lingniu.ingest.gb32960.vendor-extensions} 里被引用的 vendor 套件名 - * 分别构建一份 "default + vendor" 的 registry。 - */ - @Bean - @ConditionalOnMissingBean - public Gb32960ProfileRegistry gb32960ProfileRegistry(List parsers, - VendorExtensionCatalog catalog, - Gb32960Properties props) { - java.util.Set enabled = new java.util.LinkedHashSet<>(); - for (Gb32960Properties.VendorExtension e : props.getVendorExtensions()) { - if (e.getName() != null && !e.getName().isBlank()) enabled.add(e.getName()); - } - // enabled 只来自配置里实际引用过的扩展，避免无关 vendor parser 进入默认解析路径。 - return new Gb32960ProfileRegistry(parsers, catalog, enabled); - } - - /** Vendor 扩展选择器：按配置 list 顺序首匹。 */ - @Bean - @ConditionalOnMissingBean - public VendorExtensionSelector gb32960VendorExtensionSelector(Gb32960Properties props, - VendorExtensionCatalog catalog) { - if (props.getVendorExtensions() == null || props.getVendorExtensions().isEmpty()) { - // 没有 vendor 配置时强制 NONE，0x30+ 在 2016 帧里会 Raw 兜底，防止误套扩展解析。 - return VendorExtensionSelector.NONE; - } - return new RuleBasedVendorExtensionSelector(props.getVendorExtensions(), catalog.names()); - } - - @Bean - @ConditionalOnMissingBean - public Gb32960BodyParser gb32960BodyParser(Gb32960ProfileRegistry profileRegistry, - VendorExtensionSelector selector, - Gb32960Properties props) { - Gb32960BodyParser parser = new Gb32960BodyParser(profileRegistry, selector); - parser.setLenientBlockFailure(props.getParse().isLenientBlockFailure()); - return parser; - } - - @Bean - @ConditionalOnMissingBean - public Gb32960CommandParser gb32960CommandParser() { - return new Gb32960CommandParser(); - } - - @Bean - @ConditionalOnMissingBean - public Gb32960MessageDecoder gb32960MessageDecoder(Gb32960BodyParser bodyParser, - Gb32960CommandParser commandParser) { - return new Gb32960MessageDecoder(bodyParser, commandParser); - } - - @Bean - @ConditionalOnMissingBean - public Gb32960EventMapper gb32960EventMapper() { - return new Gb32960EventMapper(); - } - - @Bean - @ConditionalOnMissingBean - public Gb32960RealtimeHandler gb32960RealtimeHandler(Gb32960EventMapper mapper) { - return new Gb32960RealtimeHandler(mapper); - } - - @Bean - @ConditionalOnMissingBean - public Gb32960VinAuthorizer gb32960VinAuthorizer(Gb32960Properties props) { - return new Gb32960VinAuthorizer(props.getAuth()); - } - - @Bean - @ConditionalOnMissingBean - public Gb32960PlatformAuthorizer gb32960PlatformAuthorizer(Gb32960Properties props) { - return new Gb32960PlatformAuthorizer(props.getAuth().getPlatforms()); - } - - @Bean - @ConditionalOnMissingBean - public Gb32960AccessService gb32960AccessService(Gb32960VinAuthorizer authorizer, - Gb32960PlatformAuthorizer platformAuthorizer) { - return new Gb32960AccessService(authorizer, platformAuthorizer); - } - - @Bean - @ConditionalOnMissingBean - public Gb32960AckService gb32960AckService() { - return new Gb32960AckService(); - } - - @Bean - @ConditionalOnMissingBean - public Gb32960FrameDiagnostics gb32960FrameDiagnostics(Gb32960Properties props) { - return new Gb32960FrameDiagnostics( - props.getDiagnostics().getMaxLoggedFrameKeysPerChannel()); - } - - @Bean - @ConditionalOnMissingBean - public Gb32960ConnectionDiagnostics gb32960ConnectionDiagnostics() { - return new Gb32960ConnectionDiagnostics(); - } - - @Bean - @ConditionalOnMissingBean - @ConditionalOnProperty(prefix = "lingniu.ingest.gb32960.server", name = "enabled", - havingValue = "true", matchIfMissing = true) - public Gb32960NettyServer gb32960NettyServer(Gb32960Properties props, - Gb32960MessageDecoder decoder, - Dispatcher dispatcher, - Gb32960AccessService accessService, - Gb32960AckService ackService, - Gb32960FrameDiagnostics diagnostics, - Gb32960ConnectionDiagnostics connectionDiagnostics) { - return new Gb32960NettyServer( - props.getPort(), - props.getWorkerThreads(), - props.getIdleReadSeconds(), - decoder, - dispatcher, - accessService, - ackService, - diagnostics, - connectionDiagnostics, - props.getTls()); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960Properties.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960Properties.java deleted file mode 100644 index 4ea74189..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960Properties.java +++ /dev/null @@ -1,277 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.config; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -import java.util.ArrayList; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Set; - -@ConfigurationProperties(prefix = "lingniu.ingest.gb32960") -public class Gb32960Properties { - - /** 是否装配 32960 协议解析组件。TCP 监听由 {@link #server} 单独控制。 */ - private boolean enabled = false; - /** 32960 TCP 监听开关。默认跟随协议启用后启动，查询类应用可单独关闭。 */ - private Server server = new Server(); - /** 32960 TCP 监听端口；当前生产验证线使用 32960。 */ - private int port = 9000; - /** Netty worker 线程数；0 表示按 Netty/CPU 默认策略分配。 */ - private int workerThreads = 0; - - /** - * 读空闲告警阈值（秒）。连接在该时长内没有任何上行帧则打一条 WARN 日志，方便排查 - * "对端登入成功但不推数据"之类的静默场景。仅告警不断链；0 或负数禁用。 - */ - private int idleReadSeconds = 60; - - /** VIN 白名单认证。关闭时任何 VIN 都能登入。 */ - private Auth auth = new Auth(); - - /** TLS 双向认证。 */ - private Tls tls = new Tls(); - - /** - * 报文解析行为配置。 - * - *

    默认 {@code lenientBlockFailure=true}：单个信息块解析异常时兜成 - * {@link com.lingniu.ingest.protocol.gb32960.model.InfoBlock.Raw} 继续解析， - * 不再整帧放弃。关闭后恢复旧行为（任意异常直接抛 {@code DecodeException} - * 放弃整帧），仅在灰度回滚时使用。 - */ - private Parse parse = new Parse(); - - /** - * 高频入站诊断配置。用于限制每个连接保存的"首次见到帧形态"去重 key 数量， - * 避免异常设备制造大量不同 Raw 组合时造成 channel attribute 无界增长。 - */ - private Diagnostics diagnostics = new Diagnostics(); - - /** - * 厂商扩展协议（vendor extensions）路由配置。 - * - *

    每个 entry 声明一个**已被实现**的 vendor 扩展套件名（{@code name}）以及该套件 - * 的命中规则（{@link VendorExtension.Match}）。命中规则由 - * {@code RuleBasedVendorExtensionSelector} 在收到帧时按上下文（platformAccount / - * vin / vin-prefix）逐条按 list 顺序首匹，命中后用对应套件的 parser 集合替换默认的 - * 标准解析器集合（仍包含 0x01..0x09 标准块）。 - * - *

    套件名必须与 {@code VendorExtensionCatalog} 里注册的 key 一致； - * 当前内置：{@code guangdong-fc}（广东燃料电池汽车示范规范 0x30~0x34 + 0x80）。 - * - *

    未匹配任何规则时回退到默认 profile，仅保留 GB/T 32960 标准块。 - */ - private List vendorExtensions = new ArrayList<>(); - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public Server getServer() { return server; } - public void setServer(Server server) { this.server = server; } - public int getPort() { return port; } - public void setPort(int port) { this.port = port; } - public int getWorkerThreads() { return workerThreads; } - public void setWorkerThreads(int workerThreads) { this.workerThreads = workerThreads; } - public int getIdleReadSeconds() { return idleReadSeconds; } - public void setIdleReadSeconds(int idleReadSeconds) { this.idleReadSeconds = idleReadSeconds; } - public Auth getAuth() { return auth; } - public void setAuth(Auth auth) { this.auth = auth; } - public Tls getTls() { return tls; } - public void setTls(Tls tls) { this.tls = tls; } - public Parse getParse() { return parse; } - public void setParse(Parse parse) { this.parse = parse; } - public Diagnostics getDiagnostics() { return diagnostics; } - public void setDiagnostics(Diagnostics diagnostics) { this.diagnostics = diagnostics; } - public List getVendorExtensions() { return vendorExtensions; } - public void setVendorExtensions(List vendorExtensions) { - this.vendorExtensions = vendorExtensions; - } - - public static class Server { - private boolean enabled = true; - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - } - - /** - * VIN 白名单认证。 - * - *

    GB/T 32960 协议没有设备账号密码，身份由报文头的 17 字节 VIN + 数据单元里的 ICCID - * 决定。此处通过内存白名单实现最小可用的接入控制： - *

      - *
    • {@code enabled=false}（默认）：接受任意 VIN 接入，兼容未注册的测试终端 - *
    • {@code enabled=true}： - *
        - *
      • 车辆登入 0x01：VIN 不在白名单时返回 {@code 0x04 VIN_NOT_EXIST} 应答，关闭连接 - *
      • 实时上报 0x02 / 补发 0x03 / 登出 0x04 / 心跳 0x07：VIN 不在白名单时丢弃，关闭连接 - *
      - *
    - */ - public static class Auth { - private boolean enabled = false; - /** 是否大小写敏感比较 VIN。GB 16735 要求 VIN 大写，默认大小写不敏感以容错。 */ - private boolean caseSensitive = false; - private List whitelist = new ArrayList<>(); - - /** - * 平台登入 (0x05) 账号列表。GB/T 32960.3 表 29 规定平台登入携带 - * 12B 用户名 + 20B 密码 + 1B 加密规则。此处维护允许登入的账号集合， - * 收到 0x05 时比对 username / password： - *
      - *
    • 匹配且（可选）IP 在 {@code allowedIps} 内 → 返回 {@code SUCCESS} 并产出登入事件 - *
    • 不匹配 → 返回 {@code 0x02 OTHER_ERROR} 并关闭连接 - *
    - * 列表为空时默认放行所有平台登入（兼容旧行为）。 - */ - private List platforms = new ArrayList<>(); - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public boolean isCaseSensitive() { return caseSensitive; } - public void setCaseSensitive(boolean caseSensitive) { this.caseSensitive = caseSensitive; } - public List getWhitelist() { return whitelist; } - public void setWhitelist(List whitelist) { this.whitelist = whitelist; } - public List getPlatforms() { return platforms; } - public void setPlatforms(List platforms) { this.platforms = platforms; } - - /** 转换为规范化 Set 供 O(1) 查找。 */ - public Set normalizedWhitelist() { - Set out = new LinkedHashSet<>(whitelist.size()); - for (String v : whitelist) { - if (v == null || v.isBlank()) continue; - out.add(caseSensitive ? v.trim() : v.trim().toUpperCase()); - } - return out; - } - - /** - * 平台账号定义。对应 GB/T 32960.3 表 29 的字段。 - * - * @param username 12 字节以内的用户名 - * @param password 20 字节以内的密码（明文配置；推荐通过 env var 注入避免入库代码） - * @param allowedIps 可选：允许的客户端 IP 列表，为空不做 IP 限制 - * @param description 可选：平台描述（仅日志用） - */ - public static class Platform { - private String username; - private String password; - private List allowedIps = new ArrayList<>(); - private String description; - - public String getUsername() { return username; } - public void setUsername(String username) { this.username = username; } - public String getPassword() { return password; } - public void setPassword(String password) { this.password = password; } - public List getAllowedIps() { return allowedIps; } - public void setAllowedIps(List allowedIps) { this.allowedIps = allowedIps; } - public String getDescription() { return description; } - public void setDescription(String description) { this.description = description; } - } - } - - /** - * TLS 双向认证。 - * - *

    启用后 Netty pipeline 前置 {@code SslHandler}，服务端加载自身证书+私钥， - * 强制要求客户端提供证书（{@code NEED_CLIENT_AUTH}）并使用 CA 证书验证。 - */ - public static class Tls { - private boolean enabled = false; - /** 服务端证书 PEM 路径。文件路径或 classpath 资源。 */ - private String certPath; - /** 服务端私钥 PEM 路径（PKCS#8 未加密）。 */ - private String keyPath; - /** 受信任的 CA 证书 PEM 路径。用于验证客户端证书。 */ - private String trustCertPath; - /** 是否要求客户端证书。默认 true（双向）；false 则变成单向 TLS。 */ - private boolean requireClientAuth = true; - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public String getCertPath() { return certPath; } - public void setCertPath(String certPath) { this.certPath = certPath; } - public String getKeyPath() { return keyPath; } - public void setKeyPath(String keyPath) { this.keyPath = keyPath; } - public String getTrustCertPath() { return trustCertPath; } - public void setTrustCertPath(String trustCertPath) { this.trustCertPath = trustCertPath; } - public boolean isRequireClientAuth() { return requireClientAuth; } - public void setRequireClientAuth(boolean requireClientAuth) { this.requireClientAuth = requireClientAuth; } - } - - /** - * 报文解析容错策略。 - */ - public static class Parse { - /** - * 单块解析异常时是否兜底为 {@link com.lingniu.ingest.protocol.gb32960.model.InfoBlock.Raw} - * 继续解析。默认开启。 - * - *

      - *
    • {@code true}（默认）：parser 抛 DecodeException / BufferUnderflowException / - * IndexOutOfBoundsException 时，固定长度块按 fixedLen 截取 Raw 后 continue； - * 变长块或剩余字节不足时，剩余全部兜成 Raw 后 break 循环。 - *
    • {@code false}：保留旧行为，任意异常直接抛出 DecodeException 放弃整帧。 - *
    - */ - private boolean lenientBlockFailure = true; - - public boolean isLenientBlockFailure() { return lenientBlockFailure; } - public void setLenientBlockFailure(boolean lenientBlockFailure) { - this.lenientBlockFailure = lenientBlockFailure; - } - } - - /** - * 入站诊断限流/限内存策略。 - */ - public static class Diagnostics { - /** 每个 TCP channel 最多保存的首次帧诊断 key 数量。 */ - private int maxLoggedFrameKeysPerChannel = 128; - - public int getMaxLoggedFrameKeysPerChannel() { - return maxLoggedFrameKeysPerChannel; - } - - public void setMaxLoggedFrameKeysPerChannel(int maxLoggedFrameKeysPerChannel) { - this.maxLoggedFrameKeysPerChannel = maxLoggedFrameKeysPerChannel; - } - } - - /** - * 单条 vendor 扩展路由配置。{@code name} 必须与 - * {@code VendorExtensionCatalog} 注册的 key 一致。 - */ - public static class VendorExtension { - /** 扩展套件名，必须在 catalog 中已注册（如 {@code guangdong-fc}）。 */ - private String name; - /** 命中规则。任一字段命中即视为整条 entry 命中。 */ - private Match match = new Match(); - - public String getName() { return name; } - public void setName(String name) { this.name = name; } - public Match getMatch() { return match; } - public void setMatch(Match match) { this.match = match; } - - /** - * 命中规则。所有字段为 OR 关系：任意一个命中即匹配。 - * 字段内部为 case-insensitive 比较；字段为空 list 视为不参与匹配（不会主动命中）。 - */ - public static class Match { - /** 平台登入用户名精确匹配列表（来自 0x05 PlatformLogin 的 username 字段）。 */ - private List platformAccounts = new ArrayList<>(); - /** VIN 精确匹配列表。 */ - private List vins = new ArrayList<>(); - /** VIN 前缀匹配列表（startsWith）。 */ - private List vinPrefixes = new ArrayList<>(); - - public List getPlatformAccounts() { return platformAccounts; } - public void setPlatformAccounts(List platformAccounts) { - this.platformAccounts = platformAccounts; - } - public List getVins() { return vins; } - public void setVins(List vins) { this.vins = vins; } - public List getVinPrefixes() { return vinPrefixes; } - public void setVinPrefixes(List vinPrefixes) { this.vinPrefixes = vinPrefixes; } - } - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/handler/Gb32960RealtimeHandler.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/handler/Gb32960RealtimeHandler.java deleted file mode 100644 index b0480d15..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/handler/Gb32960RealtimeHandler.java +++ /dev/null @@ -1,62 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.handler; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.annotation.EventEmit; -import com.lingniu.ingest.api.annotation.MessageMapping; -import com.lingniu.ingest.api.annotation.ProtocolHandler; -import com.lingniu.ingest.api.annotation.RateLimited; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.protocol.gb32960.mapper.Gb32960EventMapper; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; - -import java.util.List; - -/** - * GB/T 32960 协议 Handler：Dispatcher 路由到此的所有 0x01~0x07 命令都委托给 - * {@link Gb32960EventMapper} 转成 {@link VehicleEvent}。 - * - *

    这是 GB32960 协议在通用 Dispatcher 管线里的**唯一入口** Bean——移除将导致所有 - * GB32960 帧产出的事件丢失。构造器注入 {@link Gb32960EventMapper}，全部业务逻辑限制 - * 在纯函数里，不触碰数据库 / 不做 IO。产出的事件由 Dispatcher 统一投递到 Disruptor； - * 下游 sink 再决定写 raw archive、TDengine 历史库或转发 Kafka。 - * - *

    当前 32960 专用 snapshot 查询不依赖 Realtime/Location 事件表，而是通过 raw archive URI - * 回读原始 .bin 并即时解码合并，避免不同车辆或不同子包之间做大范围 join。 - */ -@ProtocolHandler(protocol = ProtocolId.GB32960) -public class Gb32960RealtimeHandler { - - private final Gb32960EventMapper mapper; - - public Gb32960RealtimeHandler(Gb32960EventMapper mapper) { - this.mapper = mapper; - } - - /** 实时 + 补发上报。 */ - @MessageMapping(command = {0x02, 0x03}, desc = "实时/补发上报") - @RateLimited(perVin = 50) - @EventEmit({VehicleEvent.Realtime.class, VehicleEvent.Location.class}) - public List onReport(Gb32960Message msg) { - // 这里保留 Realtime/Location 事件供实时消费者使用；历史全字段查询走 raw archive 解码。 - return mapper.toEvents(msg); - } - - /** 车辆登入 / 登出 / 心跳：统一映射为 session 事件。 */ - @MessageMapping(command = {0x01, 0x04, 0x07}, desc = "登入/登出/心跳") - @EventEmit({VehicleEvent.Login.class, VehicleEvent.Logout.class, VehicleEvent.Heartbeat.class}) - public List onSession(Gb32960Message msg) { - return mapper.toEvents(msg); - } - - /** - * 平台登入 / 登出（0x05 / 0x06）：外部下级平台握手，产出 - * {@link VehicleEvent.Login} / {@link VehicleEvent.Logout}， - * vin 字段形如 {@code platform:}，metadata 带 {@code kind=platform}， - * 下游消费者可据此区分车辆会话与平台会话。 - */ - @MessageMapping(command = {0x05, 0x06}, desc = "平台登入/登出") - @EventEmit({VehicleEvent.Login.class, VehicleEvent.Logout.class}) - public List onPlatform(Gb32960Message msg) { - return mapper.toEvents(msg); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AccessService.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AccessService.java deleted file mode 100644 index ed9097ea..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AccessService.java +++ /dev/null @@ -1,77 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.inbound; - -import com.lingniu.ingest.protocol.gb32960.auth.Gb32960PlatformAuthorizer; -import com.lingniu.ingest.protocol.gb32960.auth.Gb32960VinAuthorizer; -import io.netty.channel.Channel; -import io.netty.util.AttributeKey; - -import java.net.InetSocketAddress; - -/** - * Access boundary for GB/T 32960 inbound channels. - * - *

    Keeps authentication and channel-scoped platform identity out of the Netty - * handler so the hot path can stay small and testable. - * - *

    平台登入成功后，username 会绑定到 Netty channel attribute。后续实时上报帧没有账号字段， - * 只能通过同一连接上的 attribute 找回平台账号，用于 Hyundai/广东扩展等 vendor profile 选择。 - */ -public final class Gb32960AccessService { - - public static final AttributeKey PLATFORM_ACCOUNT_ATTR = - AttributeKey.valueOf("gb32960.platformAccount"); - - private final Gb32960VinAuthorizer vinAuthorizer; - private final Gb32960PlatformAuthorizer platformAuthorizer; - - public Gb32960AccessService(Gb32960VinAuthorizer vinAuthorizer, - Gb32960PlatformAuthorizer platformAuthorizer) { - this.vinAuthorizer = vinAuthorizer; - this.platformAuthorizer = platformAuthorizer; - } - - public boolean isVinAllowed(String vin) { - return vinAuthorizer.isAllowed(vin); - } - - public boolean isVinWhitelistEnforcing() { - return vinAuthorizer.isEnforcing(); - } - - public int vinWhitelistSize() { - return vinAuthorizer.whitelistSize(); - } - - public boolean isPlatformAuthEnforcing() { - return platformAuthorizer.isEnforcing(); - } - - public int platformPolicySize() { - return platformAuthorizer.size(); - } - - public Gb32960PlatformAuthorizer.Result authenticatePlatformLogin(String username, - String password, - Channel channel) { - // 鉴权策略可以按来源 IP 做限制，因此这里把远端地址从 Netty channel 中抽出。 - return platformAuthorizer.authenticate(username, password, peerIp(channel)); - } - - public void bindPlatformAccount(Channel channel, String username) { - // 只绑定到当前连接，断线重连后必须重新平台登入，避免跨连接复用旧账号。 - channel.attr(PLATFORM_ACCOUNT_ATTR).set(username); - } - - public String platformAccount(Channel channel) { - return channel.hasAttr(PLATFORM_ACCOUNT_ATTR) - ? channel.attr(PLATFORM_ACCOUNT_ATTR).get() - : null; - } - - private static String peerIp(Channel channel) { - if (channel.remoteAddress() instanceof InetSocketAddress i) { - return i.getAddress().getHostAddress(); - } - return null; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AckService.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AckService.java deleted file mode 100644 index 8ac4918d..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AckService.java +++ /dev/null @@ -1,121 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.inbound; - -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960FrameEncoder; -import com.lingniu.ingest.protocol.gb32960.model.CommandType; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -import com.lingniu.ingest.protocol.gb32960.model.ResponseFlag; -import io.netty.buffer.Unpooled; -import io.netty.channel.Channel; -import io.netty.channel.ChannelHandlerContext; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.net.InetSocketAddress; -import java.time.Instant; - -/** - * Builds and writes GB/T 32960 response frames. - * - *

    ACK 需要尽量复用请求帧里的原始 17 字节 VIN。平台登入这类命令可能没有真实 VIN， - * 如果把 trim 后的字符串重新 padding，部分上级平台会认为应答 VIN 与请求不一致。 - */ -public final class Gb32960AckService { - - private static final Logger log = LoggerFactory.getLogger(Gb32960AckService.class); - - public void writeResponse(ChannelHandlerContext ctx, - ProtocolVersion version, - CommandType command, - ResponseFlag responseFlag, - byte[] rawVin, - Instant eventTime, - byte[] data, - String tag) { - writeResponse(ctx.channel(), version, command, responseFlag, rawVin, eventTime, data, tag); - } - - public void writeResponse(Channel channel, - ProtocolVersion version, - CommandType command, - ResponseFlag responseFlag, - byte[] rawVin, - Instant eventTime, - byte[] data, - String tag) { - // rawVin 版本用于严格回显请求帧 VIN 字节，优先给入站 handler 使用。 - byte[] ack = Gb32960FrameEncoder.buildResponse( - version, command, responseFlag, rawVin, eventTime, data); - write(channel, ack, tag); - } - - public void writeAndClose(ChannelHandlerContext ctx, - ProtocolVersion version, - CommandType command, - ResponseFlag responseFlag, - String vin, - Instant eventTime, - byte[] data, - String tag) { - byte[] ack = Gb32960FrameEncoder.buildResponse( - version, command, responseFlag, vin, eventTime, data); - ctx.writeAndFlush(Unpooled.wrappedBuffer(ack)).addListener(f -> ctx.close()); - } - - public void writeAndClose(ChannelHandlerContext ctx, - ProtocolVersion version, - CommandType command, - ResponseFlag responseFlag, - byte[] rawVin, - Instant eventTime, - byte[] data, - String tag) { - byte[] ack = Gb32960FrameEncoder.buildResponse( - version, command, responseFlag, rawVin, eventTime, data); - String hex = hex(ack); - ctx.writeAndFlush(Unpooled.wrappedBuffer(ack)).addListener(f -> { - if (f.isSuccess()) { - log.info("[gb32960] {} flushed peer={} len={} hex={}", - tag, addr(ctx.channel()), ack.length, hex); - } else { - log.warn("[gb32960] {} flush FAILED peer={} len={} hex={}", - tag, addr(ctx.channel()), ack.length, hex, f.cause()); - } - ctx.close(); - }); - } - - private static void write(Channel channel, byte[] ack, String tag) { - String hex = hex(ack); - boolean highFrequency = tag.startsWith("report-ack-") || tag.startsWith("heartbeat-ack"); - channel.writeAndFlush(Unpooled.wrappedBuffer(ack)).addListener(f -> { - if (f.isSuccess()) { - if (highFrequency) { - // 实时上报 ACK 频率高，默认降到 DEBUG，避免生产日志被正常心跳/上报刷满。 - if (log.isDebugEnabled()) { - log.debug("[gb32960] {} flushed peer={} len={} hex={}", - tag, addr(channel), ack.length, hex); - } - } else { - log.info("[gb32960] {} flushed peer={} len={} hex={}", - tag, addr(channel), ack.length, hex); - } - } else { - log.warn("[gb32960] {} flush FAILED peer={} len={} hex={}", - tag, addr(channel), ack.length, hex, f.cause()); - } - }); - } - - private static String hex(byte[] bytes) { - StringBuilder sb = new StringBuilder(bytes.length * 2); - for (byte b : bytes) sb.append(String.format("%02x", b)); - return sb.toString(); - } - - private static String addr(Channel channel) { - if (channel.remoteAddress() instanceof InetSocketAddress i) { - return i.getAddress().getHostAddress() + ":" + i.getPort(); - } - return "unknown"; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandler.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandler.java deleted file mode 100644 index 3f28adb9..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandler.java +++ /dev/null @@ -1,510 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.inbound; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.protocol.gb32960.auth.Gb32960PlatformAuthorizer; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.model.CommandBody; -import com.lingniu.ingest.protocol.gb32960.model.CommandType; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; -import com.lingniu.ingest.protocol.gb32960.model.ResponseFlag; -import io.netty.channel.ChannelHandlerContext; -import io.netty.channel.SimpleChannelInboundHandler; -import io.netty.handler.timeout.IdleState; -import io.netty.handler.timeout.IdleStateEvent; -import io.netty.util.AttributeKey; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.net.InetSocketAddress; -import java.nio.ByteBuffer; -import java.time.Instant; -import java.util.concurrent.CompletableFuture; -import java.util.HashMap; -import java.util.List; -import java.util.Map; - -/** - * Netty 入站处理器：字节 → Gb32960Message → 认证 → RawFrame → Dispatcher。 - * - *

    这是 32960 生产链路的入口边界。它只做协议层职责：解码、鉴权、应答、组装 - * {@link RawFrame} 并投递 Dispatcher；原始包归档、TDengine 历史索引、Kafka 转发都在 - * 下游 sink 中完成，避免 Netty EventLoop 被存储 IO 阻塞。 - * - *

    认证逻辑： - *

      - *
    • {@link Gb32960VinAuthorizer#isEnforcing()} = false：放行所有 VIN - *
    • {@link Gb32960VinAuthorizer#isEnforcing()} = true： - *
        - *
      • 车辆登入 {@code 0x01}：VIN 在白名单则返回成功应答（保留原帧时间）并派发； - * 不在则返回 {@code 0x04 VIN_NOT_EXIST} 应答并关闭连接 - *
      • 其他命令：VIN 在白名单则派发；不在则 DEBUG 日志后关闭连接 - *
      - *
    - * 注意应答帧由 {@link Gb32960AckService} 构造并写回 Channel。 - */ -public class Gb32960ChannelHandler extends SimpleChannelInboundHandler { - - private static final Logger log = LoggerFactory.getLogger(Gb32960ChannelHandler.class); - - /** - * 0x05 PLATFORM_LOGIN 鉴权通过后写入；后续 0x02/0x03 实时上报帧从 channel attribute 取出 - * 作为 {@code Gb32960ParserContext.platformAccount}，供 vendor profile selector 路由。 - * 普通车端直连场景（0x01 VEHICLE_LOGIN）此 attribute 始终为 null。 - */ - public static final AttributeKey PLATFORM_ACCOUNT_ATTR = - Gb32960AccessService.PLATFORM_ACCOUNT_ATTR; - - private static final AttributeKey> DURABLE_ACK_CHAIN_ATTR = - AttributeKey.valueOf("gb32960.durableAckChain"); - - private final Gb32960MessageDecoder decoder; - private final Dispatcher dispatcher; - private final Gb32960AccessService accessService; - private final Gb32960AckService ackService; - private final Gb32960FrameDiagnostics diagnostics; - private final Gb32960ConnectionDiagnostics connectionDiagnostics; - - public Gb32960ChannelHandler(Gb32960MessageDecoder decoder, - Dispatcher dispatcher, - Gb32960AccessService accessService, - Gb32960AckService ackService, - Gb32960FrameDiagnostics diagnostics, - Gb32960ConnectionDiagnostics connectionDiagnostics) { - this.decoder = decoder; - this.dispatcher = dispatcher; - this.accessService = accessService; - this.ackService = ackService; - this.diagnostics = diagnostics; - this.connectionDiagnostics = connectionDiagnostics; - } - - @Override - public void channelActive(ChannelHandlerContext ctx) { - connectionDiagnostics.opened(sessionId(ctx), addr(ctx)); - log.info("[gb32960] connection opened peer={}", addr(ctx)); - } - - @Override - public void channelInactive(ChannelHandlerContext ctx) { - connectionDiagnostics.closed(sessionId(ctx)); - log.info("[gb32960] connection closed peer={}", addr(ctx)); - } - - /** - * 读空闲告警：仅记录日志，不断链。用于排查"对端登入成功后长时间不推数据"场景。 - * 由 pipeline 中的 {@link io.netty.handler.timeout.IdleStateHandler}（可选）触发。 - */ - @Override - public void userEventTriggered(ChannelHandlerContext ctx, Object evt) throws Exception { - if (evt instanceof IdleStateEvent idle && idle.state() == IdleState.READER_IDLE) { - connectionDiagnostics.readIdle(sessionId(ctx)); - log.warn("[gb32960] read idle peer={} noInboundFrameFor>=thresholdSeconds (see idleReadSeconds)", - addr(ctx)); - } - super.userEventTriggered(ctx, evt); - } - - @Override - protected void channelRead0(ChannelHandlerContext ctx, byte[] frame) { - Gb32960Message msg; - try { - String platformAccount = accessService.platformAccount(ctx.channel()); - msg = decoder.decode(ByteBuffer.wrap(frame), platformAccount); - } catch (Exception e) { - // 解码失败只丢当前帧，不主动断开连接。真实设备偶发脏字节时，FrameDecoder 会继续寻找下一帧头。 - log.warn("[gb32960] decode failed peer={} len={}", addr(ctx), frame.length, e); - dispatchMalformed(ctx, frame, e.getMessage()); - return; - } - - String vin = msg.header().vin(); - CommandType cmd = msg.header().command(); - connectionDiagnostics.received(sessionId(ctx), cmd, vin); - // 原样回显请求帧的 17 字节 VIN（frame[4..20]），避免平台登入 VIN 全 0 - // 被 vinPadded 重写为空格导致对端判定 VIN 不匹配。 - byte[] rawVin = new byte[17]; - System.arraycopy(frame, 4, rawVin, 0, 17); - - // 平台登入/登出（0x05/0x06）没有 VIN，不进 VIN 白名单校验 - boolean isPlatformCommand = cmd == CommandType.PLATFORM_LOGIN || cmd == CommandType.PLATFORM_LOGOUT; - if (!isPlatformCommand && !accessService.isVinAllowed(vin)) { - handleUnauthorized(ctx, msg); - return; - } - - // 鉴权通过：先完成命令侧状态变更/日志，再统一 dispatch 到通用管线。 - // 需要成功 ACK 的已接收帧，等待配置的 Kafka dispatch 边界完成后再回 ACK。 - // 平台登入鉴权失败会短路 return，不进 dispatch。 - switch (cmd) { - case VEHICLE_LOGIN -> logVehicleLogin(ctx, msg); - case VEHICLE_LOGOUT -> logVehicleLogout(ctx, msg); - case PLATFORM_LOGIN -> { - if (!handlePlatformLogin(ctx, msg, rawVin)) return; - } - case PLATFORM_LOGOUT -> logPlatformLogout(ctx); - case REALTIME_REPORT, RESEND_REPORT, HEARTBEAT -> logReportOrHeartbeat(ctx, msg); - case TIME_CALIBRATION -> { } - default -> logOtherFrame(ctx, msg); - } - - RawFrame rf = rawFrame(ctx, msg, frame); - if (!requiresDurableAck(cmd)) { - dispatcher.dispatch(rf); - return; - } - - enqueueDurableAck(ctx, msg, rawVin, vin, cmd, dispatcher.dispatchAndAwait(rf)); - } - - private void enqueueDurableAck(ChannelHandlerContext ctx, - Gb32960Message msg, - byte[] rawVin, - String vin, - CommandType cmd, - CompletableFuture dispatchFuture) { - CompletableFuture previous = ctx.channel().attr(DURABLE_ACK_CHAIN_ATTR).get(); - if (previous == null) { - previous = CompletableFuture.completedFuture(null); - } - - CompletableFuture next = previous - .handle((ignored, previousFailure) -> null) - .thenCompose(ignored -> dispatchFuture.handle((ok, failure) -> failure)) - .thenCompose(failure -> runOnEventLoop(ctx, - () -> handleDurableAckResult(ctx, msg, rawVin, vin, cmd, failure))); - ctx.channel().attr(DURABLE_ACK_CHAIN_ATTR).set(next); - } - - private CompletableFuture runOnEventLoop(ChannelHandlerContext ctx, Runnable task) { - CompletableFuture done = new CompletableFuture<>(); - Runnable wrapped = () -> { - try { - task.run(); - done.complete(null); - } catch (Throwable t) { - done.completeExceptionally(t); - } - }; - if (ctx.executor().inEventLoop()) { - wrapped.run(); - } else { - ctx.executor().execute(wrapped); - } - return done; - } - - private void handleDurableAckResult(ChannelHandlerContext ctx, - Gb32960Message msg, - byte[] rawVin, - String vin, - CommandType cmd, - Throwable failure) { - if (!ctx.channel().isActive()) { - return; - } - if (failure != null) { - log.warn("[gb32960] dispatch failed before ack peer={} vin={} cmd=0x{}", - addr(ctx), vin, Integer.toHexString(cmd.code()), failure); - ctx.close(); - return; - } - writeAckForCommand(ctx, msg, rawVin); - } - - private RawFrame rawFrame(ChannelHandlerContext ctx, Gb32960Message msg, byte[] frame) { - Map sourceMeta = new HashMap<>(4); - sourceMeta.put("vin", msg.header().vin()); - sourceMeta.put("peer", addr(ctx)); - String platformAccount = accessService.platformAccount(ctx.channel()); - if (platformAccount != null && !platformAccount.isBlank()) { - sourceMeta.put("platformAccount", platformAccount); - } - // RawFrame.rawBytes 是后续 raw archive 和按 rawArchiveUri 回查的源头；不要在这里裁剪或重编码。 - return new RawFrame( - ProtocolId.GB32960, - msg.header().command().code(), - 0, - msg, - frame, - sourceMeta, - Instant.now()); - } - - private void dispatchMalformed(ChannelHandlerContext ctx, byte[] frame, String errorMessage) { - Map sourceMeta = new HashMap<>(4); - sourceMeta.put("vin", "unknown"); - sourceMeta.put("peer", addr(ctx)); - sourceMeta.put("parseError", "true"); - sourceMeta.put("parseErrorMessage", errorMessage == null ? "" : errorMessage); - dispatcher.dispatch(new RawFrame( - ProtocolId.GB32960, - 0, - 0, - frame, - frame == null ? new byte[0] : frame, - sourceMeta, - Instant.now())); - } - - private static boolean requiresDurableAck(CommandType cmd) { - return switch (cmd) { - case VEHICLE_LOGIN, - VEHICLE_LOGOUT, - PLATFORM_LOGIN, - PLATFORM_LOGOUT, - REALTIME_REPORT, - RESEND_REPORT, - HEARTBEAT, - TIME_CALIBRATION -> true; - default -> false; - }; - } - - private void writeAckForCommand(ChannelHandlerContext ctx, Gb32960Message msg, byte[] rawVin) { - switch (msg.header().command()) { - case VEHICLE_LOGIN -> writeVehicleLoginAck(ctx, msg, rawVin); - case VEHICLE_LOGOUT -> writeVehicleLogoutAck(ctx, msg, rawVin); - case PLATFORM_LOGIN -> writePlatformLoginAck(ctx, msg, rawVin); - case PLATFORM_LOGOUT -> writePlatformLogoutAck(ctx, msg, rawVin); - case REALTIME_REPORT, RESEND_REPORT, HEARTBEAT -> writeReportOrHeartbeatAck(ctx, msg, rawVin); - case TIME_CALIBRATION -> writeTimeCalibrationAck(ctx, msg, rawVin); - default -> { } - } - } - - /** 0x01 车辆登入：按 §6.3.2 带原采集时间回成功 ACK + INFO 日志。 */ - private void logVehicleLogin(ChannelHandlerContext ctx, Gb32960Message msg) { - log.info("[gb32960] vehicle login peer={} vin={} protocolVersion={}", - addr(ctx), msg.header().vin(), msg.header().protocolVersion()); - } - - private void writeVehicleLoginAck(ChannelHandlerContext ctx, Gb32960Message msg, byte[] rawVin) { - ackService.writeResponse( - ctx, - msg.header().protocolVersion(), - CommandType.VEHICLE_LOGIN, - ResponseFlag.SUCCESS, - rawVin, - eventOrNow(msg), - null, - "vehicle-login-ack"); - } - - /** 0x04 车辆登出：INFO 日志 + 带原采集时间回成功 ACK。 */ - private void logVehicleLogout(ChannelHandlerContext ctx, Gb32960Message msg) { - log.info("[gb32960] vehicle logout peer={} vin={}", addr(ctx), msg.header().vin()); - } - - private void writeVehicleLogoutAck(ChannelHandlerContext ctx, Gb32960Message msg, byte[] rawVin) { - ackService.writeResponse(ctx, - msg.header().protocolVersion(), - CommandType.VEHICLE_LOGOUT, - ResponseFlag.SUCCESS, - rawVin, - eventOrNow(msg), - null, - "vehicle-logout-ack"); - } - - /** - * 0x05 平台登入（表 29）：含 12B username + 20B password + 加密规则。 - * - *
      - *
    • 消息体解析缺失：关闭连接，返回 false 短路 - *
    • 鉴权失败：写 {@code 0x02 OTHER_ERROR} NACK + 关闭连接，返回 false 短路 - *
    • 鉴权通过：把 username 钉到 channel attribute（供后续 vendor profile 路由），INFO 日志， - * 返回 true 继续 dispatch；成功 ACK 等 dispatch 持久化边界完成后再写 - *
    - * - * @return true 表示鉴权通过应继续 dispatch；false 表示调用方应 short-circuit return - */ - private boolean handlePlatformLogin(ChannelHandlerContext ctx, Gb32960Message msg, byte[] rawVin) { - if (!(msg.commandBody() instanceof CommandBody.PlatformLogin pl)) { - log.warn("[gb32960] platform login peer={} body unparsed, closing", addr(ctx)); - ctx.close(); - return false; - } - Gb32960PlatformAuthorizer.Result result = accessService.authenticatePlatformLogin( - pl.username(), pl.password(), ctx.channel()); - if (!result.accepted()) { - log.warn("[gb32960] platform login REJECTED peer={} username={} reason={}", - addr(ctx), pl.username(), result); - ackService.writeAndClose( - ctx, - msg.header().protocolVersion(), - CommandType.PLATFORM_LOGIN, - ResponseFlag.OTHER_ERROR, - rawVin, - eventOrNow(msg), - null, - "platform-login-nack"); - return false; - } - log.info("[gb32960] platform login peer={} username={} encryptRule={} serial={} time={} policy={}", - addr(ctx), pl.username(), pl.encryptRule(), pl.serialNo(), pl.loginTime(), result); - // 把 username 钉到 channel attribute，供后续 0x02/0x03 帧的 vendor profile 路由 - accessService.bindPlatformAccount(ctx.channel(), pl.username()); - connectionDiagnostics.platformLoginAccepted(sessionId(ctx), pl.username()); - return true; - } - - private void writePlatformLoginAck(ChannelHandlerContext ctx, Gb32960Message msg, byte[] rawVin) { - ackService.writeResponse( - ctx, - msg.header().protocolVersion(), - CommandType.PLATFORM_LOGIN, - ResponseFlag.SUCCESS, - rawVin, - eventOrNow(msg), - null, - "platform-login-ack"); - } - - /** 0x06 平台登出：INFO 日志 + 带原采集时间回成功 ACK。 */ - private void logPlatformLogout(ChannelHandlerContext ctx) { - log.info("[gb32960] platform logout peer={}", addr(ctx)); - } - - private void writePlatformLogoutAck(ChannelHandlerContext ctx, Gb32960Message msg, byte[] rawVin) { - ackService.writeResponse(ctx, - msg.header().protocolVersion(), - CommandType.PLATFORM_LOGOUT, - ResponseFlag.SUCCESS, - rawVin, - eventOrNow(msg), - null, - "platform-logout-ack"); - } - - /** - * 0x02/0x03/0x07 实时 / 补发 / 心跳：记录帧诊断；成功 ACK 在配置的 Kafka dispatch 边界后写回。 - * 部分车端实现严格按规范，没收到应答会停止后续上报或重连。ack-tag 按 cmd.code 区分便于日志筛选。 - * - *

    注意：应答成功表示当前配置的 Kafka sink 已完成该帧映射事件的 dispatch 边界。 - * 生产排查历史数据问题时，应继续检查 raw archive sink、Kafka topic 和 history app 的 - * TDengine raw_frames 写入日志。 - * - *

    日志策略（按 (channel, vin, rawTypeSignature) 去重，避免高频帧刷屏）： - *

      - *
    • 0x02 / 0x03 首次见到某 (vin, normal) → INFO 一条，含 peer/vin/platformAccount/version/parsedBlocks。 - *
    • 0x02 / 0x03 首次见到某 (vin, rawTypes=…) → WARN 一条，列出 raw typeCode。提示对端可能没走预期 - * vendor profile（例：peer 是广东燃料电池但 platformAccount 没匹配上 - * {@code vendor-extensions.platform-accounts} → 0x30+ 落 Raw）。 - *
    • 同 (vin, signature) 后续帧 → DEBUG（默认看不到；troubleshoot 时打开 Gb32960ChannelHandler 的 DEBUG）。 - *
    • 0x07 心跳：DEBUG（量大且无信息体）。 - *
    - * 帧内字段全量 JSON 仍保留 DEBUG。 - */ - private void logReportOrHeartbeat(ChannelHandlerContext ctx, Gb32960Message msg) { - CommandType cmd = msg.header().command(); - String platformAccount = accessService.platformAccount(ctx.channel()); - if (cmd == CommandType.HEARTBEAT) { - if (log.isDebugEnabled()) { - log.debug("[gb32960] HEARTBEAT peer={} vin={} platformAccount={}", - addr(ctx), msg.header().vin(), platformAccount); - } - return; - } - List rawTypes = Gb32960FrameDiagnostics.collectRawTypeHex(msg.infoBlocks()); - int parsedBlocks = msg.infoBlocks().size() - rawTypes.size(); - String vin = msg.header().vin(); - boolean firstSeen = diagnostics.markFirstSeen(ctx.channel(), vin, rawTypes); - if (firstSeen) { - if (rawTypes.isEmpty()) { - log.info("[gb32960] {} (first frame for vin) peer={} vin={} platformAccount={} version={} parsedBlocks={}", - cmd.name(), addr(ctx), vin, platformAccount, - msg.header().protocolVersion(), parsedBlocks); - } else { - log.warn("[gb32960] {} peer={} vin={} platformAccount={} version={} parsedBlocks={} rawTypes={} — vendor profile likely not selected for this peer", - cmd.name(), addr(ctx), vin, platformAccount, - msg.header().protocolVersion(), parsedBlocks, rawTypes); - } - } - if (log.isDebugEnabled()) { - log.debug("[gb32960] frame peer={} vin={} cmd=0x{} parsedBlocks={} rawTypes={} json={}", - addr(ctx), vin, Integer.toHexString(cmd.code()), - parsedBlocks, rawTypes, Gb32960FrameDiagnostics.toJson(msg)); - } - } - - private void writeReportOrHeartbeatAck(ChannelHandlerContext ctx, Gb32960Message msg, byte[] rawVin) { - CommandType cmd = msg.header().command(); - ackService.writeResponse(ctx, - msg.header().protocolVersion(), - cmd, - ResponseFlag.SUCCESS, - rawVin, - eventOrNow(msg), - null, - "report-ack-0x" + Integer.toHexString(cmd.code())); - } - - /** 0x08 终端校时：平台应答 data 段为平台当前时间 6B（用 Instant.now()）。 */ - private void writeTimeCalibrationAck(ChannelHandlerContext ctx, Gb32960Message msg, byte[] rawVin) { - ackService.writeResponse(ctx, - msg.header().protocolVersion(), - CommandType.TIME_CALIBRATION, - ResponseFlag.SUCCESS, - rawVin, - Instant.now(), - null, - "time-calibration-ack"); - } - - /** 其它未单独处理的命令（如激活 0x09、密钥交换 0x0A）：仅 DEBUG 日志，仍会进 dispatcher。 */ - private void logOtherFrame(ChannelHandlerContext ctx, Gb32960Message msg) { - if (log.isDebugEnabled()) { - log.debug("[gb32960] frame peer={} vin={} cmd=0x{} infoBlocks={} json={}", - addr(ctx), msg.header().vin(), Integer.toHexString(msg.header().command().code()), - msg.infoBlocks().size(), Gb32960FrameDiagnostics.toJson(msg)); - } - } - - /** 优先使用帧内采集时间；为 null 时回落到 {@link Instant#now()}。 */ - private static Instant eventOrNow(Gb32960Message msg) { - return msg.header().eventTime() != null ? msg.header().eventTime() : Instant.now(); - } - - /** VIN 未授权：对登入命令下发 VIN_NOT_EXIST 应答；其他命令静默关闭。 */ - private void handleUnauthorized(ChannelHandlerContext ctx, Gb32960Message msg) { - String vin = msg.header().vin(); - CommandType cmd = msg.header().command(); - if (cmd == CommandType.VEHICLE_LOGIN) { - // 规范要求登入失败要回 ACK，让终端能明确知道 VIN 不存在，而不是当作网络超时重试。 - ackService.writeAndClose( - ctx, - msg.header().protocolVersion(), - CommandType.VEHICLE_LOGIN, - ResponseFlag.VIN_NOT_EXIST, - vin, - msg.header().eventTime() != null ? msg.header().eventTime() : Instant.now(), - null, - "vehicle-login-nack"); - log.info("[gb32960] reject login peer={} vin={} (not in whitelist)", addr(ctx), vin); - } else { - log.debug("[gb32960] drop frame peer={} vin={} cmd=0x{} (not in whitelist)", - addr(ctx), vin, Integer.toHexString(cmd.code())); - ctx.close(); - } - } - - @Override - public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) { - log.warn("[gb32960] channel error peer={}", addr(ctx), cause); - ctx.close(); - } - - private static String addr(ChannelHandlerContext ctx) { - if (ctx.channel().remoteAddress() instanceof InetSocketAddress i) { - return i.getAddress().getHostAddress() + ":" + i.getPort(); - } - return "unknown"; - } - - private static String sessionId(ChannelHandlerContext ctx) { - return ctx.channel().id().asLongText(); - } - -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ConnectionDiagnostics.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ConnectionDiagnostics.java deleted file mode 100644 index a7be8bd5..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ConnectionDiagnostics.java +++ /dev/null @@ -1,113 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.inbound; - -import com.lingniu.ingest.protocol.gb32960.model.CommandType; - -import java.time.Clock; -import java.time.Duration; -import java.time.Instant; -import java.util.Comparator; -import java.util.List; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ConcurrentMap; - -public class Gb32960ConnectionDiagnostics { - - private final Clock clock; - private final ConcurrentMap activeSessions = new ConcurrentHashMap<>(); - - public Gb32960ConnectionDiagnostics() { - this(Clock.systemUTC()); - } - - Gb32960ConnectionDiagnostics(Clock clock) { - this.clock = clock; - } - - public void opened(String sessionId, String peer) { - Instant now = clock.instant(); - activeSessions.put(sessionId, new MutableSession(sessionId, peer, now)); - } - - public void received(String sessionId, CommandType command, String vin) { - activeSessions.computeIfPresent(sessionId, (ignored, session) -> { - session.lastInboundAt = clock.instant(); - session.lastCommandCode = command.code(); - session.lastCommand = command.name(); - session.vin = vin == null ? "" : vin; - return session; - }); - } - - public void platformLoginAccepted(String sessionId, String platformAccount) { - activeSessions.computeIfPresent(sessionId, (ignored, session) -> { - session.platformAccount = platformAccount == null ? "" : platformAccount; - return session; - }); - } - - public void readIdle(String sessionId) { - activeSessions.computeIfPresent(sessionId, (ignored, session) -> { - session.readIdleCount++; - return session; - }); - } - - public void closed(String sessionId) { - activeSessions.remove(sessionId); - } - - public List activeSessions() { - Instant now = clock.instant(); - return activeSessions.values().stream() - .map(session -> session.snapshot(now)) - .sorted(Comparator.comparing(SessionSnapshot::connectedAt)) - .toList(); - } - - public record SessionSnapshot( - String sessionId, - String peer, - Instant connectedAt, - Instant lastInboundAt, - long idleSeconds, - int lastCommandCode, - String lastCommand, - String vin, - String platformAccount, - long readIdleCount) { - } - - private static final class MutableSession { - private final String sessionId; - private final String peer; - private final Instant connectedAt; - private Instant lastInboundAt; - private int lastCommandCode; - private String lastCommand = ""; - private String vin = ""; - private String platformAccount = ""; - private long readIdleCount; - - private MutableSession(String sessionId, String peer, Instant connectedAt) { - this.sessionId = sessionId; - this.peer = peer; - this.connectedAt = connectedAt; - this.lastInboundAt = connectedAt; - } - - private SessionSnapshot snapshot(Instant now) { - long idleSeconds = Math.max(0, Duration.between(lastInboundAt, now).toSeconds()); - return new SessionSnapshot( - sessionId, - peer, - connectedAt, - lastInboundAt, - idleSeconds, - lastCommandCode, - lastCommand, - vin, - platformAccount, - readIdleCount); - } - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960FrameDiagnostics.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960FrameDiagnostics.java deleted file mode 100644 index d4d67363..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960FrameDiagnostics.java +++ /dev/null @@ -1,87 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.inbound; - -import com.fasterxml.jackson.databind.ObjectMapper; -import com.fasterxml.jackson.databind.SerializationFeature; -import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import io.netty.channel.Channel; -import io.netty.util.AttributeKey; - -import java.util.ArrayList; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Set; - -/** - * Channel-scoped diagnostics for high-frequency GB/T 32960 reports. - * - *

    每个 TCP channel 只记录有限个首次出现的 (vin, rawTypeSignature)，用于发现厂商 profile - * 没命中、字段解析退化为 Raw 的问题。它不参与业务判断，也不会影响 raw archive 和历史查询。 - */ -public final class Gb32960FrameDiagnostics { - - private static final AttributeKey> LOGGED_FRAME_KEYS_ATTR = - AttributeKey.valueOf("gb32960.loggedFrameKeys"); - - private static final ObjectMapper DEBUG_JSON = new ObjectMapper() - .registerModule(new JavaTimeModule()) - .disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS); - - private final int maxKeysPerChannel; - - public Gb32960FrameDiagnostics() { - this(128); - } - - public Gb32960FrameDiagnostics(int maxKeysPerChannel) { - this.maxKeysPerChannel = Math.max(1, maxKeysPerChannel); - } - - public boolean markFirstSeen(Channel channel, String vin, List rawTypes) { - String signature = rawTypes == null || rawTypes.isEmpty() ? "NORMAL" : "RAW:" + rawTypes; - return markFirstSeenKey(channel, vin + "|" + signature); - } - - public int seenKeyCount(Channel channel) { - Set seen = channel.attr(LOGGED_FRAME_KEYS_ATTR).get(); - return seen == null ? 0 : seen.size(); - } - - public static List collectRawTypeHex(List blocks) { - List raws = new ArrayList<>(0); - for (InfoBlock b : blocks) { - if (b instanceof InfoBlock.Raw r) { - raws.add(String.format("0x%02x", r.typeCode())); - } - } - return raws; - } - - public static String toJson(Gb32960Message msg) { - try { - return DEBUG_JSON.writeValueAsString(msg); - } catch (Exception e) { - return ""; - } - } - - private boolean markFirstSeenKey(Channel channel, String key) { - Set seen = channel.attr(LOGGED_FRAME_KEYS_ATTR).get(); - if (seen == null) { - seen = new LinkedHashSet<>(); - channel.attr(LOGGED_FRAME_KEYS_ATTR).set(seen); - } - boolean added = seen.add(key); - if (added) trimToLimit(seen); - return added; - } - - private void trimToLimit(Set seen) { - while (seen.size() > maxKeysPerChannel) { - // LinkedHashSet 保持插入顺序，超限时淘汰最早的签名，避免异常设备导致 attribute 无界增长。 - String first = seen.iterator().next(); - seen.remove(first); - } - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960NettyServer.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960NettyServer.java deleted file mode 100644 index 49e67501..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960NettyServer.java +++ /dev/null @@ -1,181 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.inbound; - -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960FrameDecoder; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.config.Gb32960Properties; -import io.netty.bootstrap.ServerBootstrap; -import io.netty.channel.Channel; -import io.netty.channel.ChannelInitializer; -import io.netty.channel.ChannelOption; -import io.netty.channel.EventLoopGroup; -import io.netty.channel.nio.NioEventLoopGroup; -import io.netty.channel.socket.SocketChannel; -import io.netty.channel.socket.nio.NioServerSocketChannel; -import io.netty.handler.ssl.ClientAuth; -import io.netty.handler.ssl.SslContext; -import io.netty.handler.ssl.SslContextBuilder; -import io.netty.handler.timeout.IdleStateHandler; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.beans.factory.DisposableBean; -import org.springframework.beans.factory.InitializingBean; - -import java.io.File; -import java.io.InputStream; -import java.net.InetSocketAddress; -import java.nio.file.Files; -import java.nio.file.Paths; -import java.util.concurrent.ThreadFactory; - -/** - * GB/T 32960 Netty Server 启动器。 - * - *

    EventLoop 只做解码与投递 {@link Dispatcher}，业务处理由 Disruptor 侧的虚拟线程承担。 - * 端口是否启动由 {@code lingniu.ingest.gb32960.enabled} 控制，生产 32960 线应确认该开关、 - * 端口、防火墙和上游平台账号同时匹配。 - * - *

    支持可选 TLS：{@link Gb32960Properties.Tls#isEnabled()} = true 时，pipeline 前置 - * {@code SslHandler}，加载服务端证书 + 私钥 + 受信 CA，默认要求客户端证书（双向 TLS）。 - */ -public class Gb32960NettyServer implements InitializingBean, DisposableBean { - - private static final Logger log = LoggerFactory.getLogger(Gb32960NettyServer.class); - - private final int port; - private final int workerThreads; - private final int idleReadSeconds; - private final Gb32960MessageDecoder messageDecoder; - private final Dispatcher dispatcher; - private final Gb32960AccessService accessService; - private final Gb32960AckService ackService; - private final Gb32960FrameDiagnostics diagnostics; - private final Gb32960ConnectionDiagnostics connectionDiagnostics; - private final Gb32960Properties.Tls tlsConfig; - - private EventLoopGroup boss; - private EventLoopGroup workers; - private Channel serverChannel; - private SslContext sslContext; - - public Gb32960NettyServer(int port, int workerThreads, int idleReadSeconds, - Gb32960MessageDecoder messageDecoder, - Dispatcher dispatcher, - Gb32960AccessService accessService, - Gb32960AckService ackService, - Gb32960FrameDiagnostics diagnostics, - Gb32960ConnectionDiagnostics connectionDiagnostics, - Gb32960Properties.Tls tlsConfig) { - this.port = port; - this.workerThreads = workerThreads; - this.idleReadSeconds = idleReadSeconds; - this.messageDecoder = messageDecoder; - this.dispatcher = dispatcher; - this.accessService = accessService; - this.ackService = ackService; - this.diagnostics = diagnostics; - this.connectionDiagnostics = connectionDiagnostics; - this.tlsConfig = tlsConfig; - } - - @Override - public void afterPropertiesSet() throws Exception { - if (tlsConfig != null && tlsConfig.isEnabled()) { - this.sslContext = buildSslContext(tlsConfig); - log.info("[gb32960] TLS enabled cert={} clientAuth={}", - tlsConfig.getCertPath(), - tlsConfig.isRequireClientAuth() ? "REQUIRED" : "OPTIONAL"); - } - - ThreadFactory bossTf = runnable -> new Thread(runnable, "gb32960-boss"); - ThreadFactory wkTf = runnable -> new Thread(runnable, "gb32960-worker"); - this.boss = new NioEventLoopGroup(1, bossTf); - this.workers = new NioEventLoopGroup(workerThreads == 0 - ? Runtime.getRuntime().availableProcessors() * 2 : workerThreads, wkTf); - - ServerBootstrap b = new ServerBootstrap(); - b.group(boss, workers) - .channel(NioServerSocketChannel.class) - .option(ChannelOption.SO_BACKLOG, 1024) - .childOption(ChannelOption.TCP_NODELAY, true) - .childOption(ChannelOption.SO_KEEPALIVE, true) - .childHandler(new ChannelInitializer() { - @Override - protected void initChannel(SocketChannel ch) { - // Pipeline 顺序不能调换：TLS 先解密，FrameDecoder 再做拆包，ChannelHandler 最后做业务应答和派发。 - if (sslContext != null) { - ch.pipeline().addLast("ssl", sslContext.newHandler(ch.alloc())); - } - if (idleReadSeconds > 0) { - // 只监听读空闲：readerIdle=N，writerIdle=0，allIdle=0。 - // 触发后由 Gb32960ChannelHandler#userEventTriggered 仅打告警日志，不断链。 - ch.pipeline().addLast("idle", - new IdleStateHandler(idleReadSeconds, 0, 0)); - } - ch.pipeline() - .addLast("frame", new Gb32960FrameDecoder()) - .addLast("dispatch", - new Gb32960ChannelHandler(messageDecoder, dispatcher, - accessService, ackService, diagnostics, - connectionDiagnostics)); - } - }); - this.serverChannel = b.bind(port).sync().channel(); - log.info("[gb32960] Netty server listening on :{} (tls={}, vinWhitelist={}, platformAuth={}, idleReadSeconds={})", - getBoundPort(), - sslContext != null, - accessService.isVinWhitelistEnforcing() ? accessService.vinWhitelistSize() : "DISABLED", - accessService.isPlatformAuthEnforcing() ? accessService.platformPolicySize() : "DISABLED", - idleReadSeconds > 0 ? idleReadSeconds : "DISABLED"); - } - - /** 实际绑定端口（支持 port=0 随机分配后查询）。 */ - public int getBoundPort() { - if (serverChannel != null && serverChannel.localAddress() instanceof InetSocketAddress a) { - return a.getPort(); - } - return port; - } - - private static SslContext buildSslContext(Gb32960Properties.Tls cfg) throws Exception { - if (cfg.getCertPath() == null || cfg.getKeyPath() == null) { - throw new IllegalStateException("TLS enabled but cert-path/key-path not configured"); - } - SslContextBuilder builder = SslContextBuilder.forServer( - openPem(cfg.getCertPath()), - openPem(cfg.getKeyPath())); - if (cfg.getTrustCertPath() != null && !cfg.getTrustCertPath().isBlank()) { - try (InputStream trust = openPem(cfg.getTrustCertPath())) { - builder.trustManager(trust); - } - } - builder.clientAuth(cfg.isRequireClientAuth() ? ClientAuth.REQUIRE : ClientAuth.OPTIONAL); - return builder.build(); - } - - /** 读取 PEM 文件，支持 file:// 绝对路径或相对路径。 */ - private static InputStream openPem(String path) throws Exception { - String stripped = path.startsWith("file://") ? path.substring("file://".length()) : path; - File f = new File(stripped); - if (!f.isAbsolute()) { - f = Paths.get(System.getProperty("user.dir"), stripped).toFile(); - } - if (!f.exists()) { - throw new IllegalStateException("TLS PEM not found: " + f.getAbsolutePath()); - } - return Files.newInputStream(f.toPath()); - } - - @Override - public void destroy() { - try { - if (serverChannel != null) serverChannel.close().sync(); - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); - } finally { - if (boss != null) boss.shutdownGracefully(); - if (workers != null) workers.shutdownGracefully(); - log.info("[gb32960] Netty server stopped"); - } - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/mapper/Gb32960EventMapper.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/mapper/Gb32960EventMapper.java deleted file mode 100644 index f0ea087f..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/mapper/Gb32960EventMapper.java +++ /dev/null @@ -1,344 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.mapper; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.AlarmPayload; -import com.lingniu.ingest.api.event.LocationPayload; -import com.lingniu.ingest.api.event.RealtimePayload; -import com.lingniu.ingest.api.event.TelemetryMetadataValues; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.spi.EventMapper; -import com.lingniu.ingest.protocol.gb32960.model.CommandBody; -import com.lingniu.ingest.protocol.gb32960.model.CommandType; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; -import com.lingniu.ingest.protocol.gb32960.model.GeneralAlarmFlagBit; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; - -import java.time.Instant; -import java.util.ArrayList; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Map; -import java.util.Set; -import java.util.UUID; -import java.util.stream.Collectors; - -/** - * 32960 → 领域事件 Adapter。 - * - *

    实时/补发上报产出 {@code Realtime + Location}（可选）+ {@code Alarm}（可选）； - * 车辆登入/登出/心跳产出对应的会话事件。 - * - *

    该 Mapper 只生成统一领域事件，字段是面向实时总线的精简视图。32960 的完整包字段、 - * 厂商扩展字段和 snapshot 合并字段由历史查询服务从 raw archive 重新解码，不在这里展开。 - */ -public final class Gb32960EventMapper implements EventMapper { - - @Override - public List toEvents(Gb32960Message message) { - var header = message.header(); - Instant eventTime = header.eventTime() != null ? header.eventTime() : Instant.now(); - Instant ingestTime = Instant.now(); - - List out = new ArrayList<>(3); - Map baseMeta = Map.of( - "command", header.command().name(), - "protocolVersion", header.protocolVersion().name()); - - CommandType cmd = header.command(); - if (cmd == CommandType.REALTIME_REPORT || cmd == CommandType.RESEND_REPORT) { - // 补发包与实时包进入同一套实时事件模型，差异保留在 metadata.command 里供下游判断。 - buildRealtime(message, header, baseMeta, eventTime, ingestTime, out); - return out; - } - if (cmd == CommandType.VEHICLE_LOGIN) { - String iccid = ""; - if (message.commandBody() instanceof CommandBody.VehicleLogin vl) iccid = vl.iccid(); - out.add(new VehicleEvent.Login( - UUID.randomUUID().toString(), - header.vin(), ProtocolId.GB32960, eventTime, ingestTime, - null, baseMeta, iccid, header.protocolVersion().name())); - } else if (cmd == CommandType.VEHICLE_LOGOUT) { - out.add(new VehicleEvent.Logout( - UUID.randomUUID().toString(), - header.vin(), ProtocolId.GB32960, eventTime, ingestTime, - null, baseMeta)); - } else if (cmd == CommandType.HEARTBEAT) { - out.add(new VehicleEvent.Heartbeat( - UUID.randomUUID().toString(), - header.vin(), ProtocolId.GB32960, eventTime, ingestTime, - null, baseMeta)); - } else if (cmd == CommandType.PLATFORM_LOGIN) { - // 平台登入 0x05：映射为 VehicleEvent.Login + metadata.kind=platform。 - // 使用 "platform:" + username 作为 vin 字段，既能送到 vehicle.session topic - // 下游消费者又能靠 metadata.kind 区分车辆/平台登入。 - if (message.commandBody() instanceof CommandBody.PlatformLogin pl) { - Map meta = new java.util.HashMap<>(baseMeta); - meta.put("kind", "platform"); - meta.put("username", pl.username() == null ? "" : pl.username()); - meta.put("encryptRule", pl.encryptRule() == null ? "" : pl.encryptRule().name()); - meta.put("serial", String.valueOf(pl.serialNo())); - out.add(new VehicleEvent.Login( - UUID.randomUUID().toString(), - "platform:" + (pl.username() == null ? "" : pl.username()), - ProtocolId.GB32960, - pl.loginTime() != null ? pl.loginTime() : eventTime, - ingestTime, - null, meta, "", header.protocolVersion().name())); - } - } else if (cmd == CommandType.PLATFORM_LOGOUT) { - if (message.commandBody() instanceof CommandBody.PlatformLogout plo) { - Map meta = new java.util.HashMap<>(baseMeta); - meta.put("kind", "platform"); - meta.put("serial", String.valueOf(plo.serialNo())); - out.add(new VehicleEvent.Logout( - UUID.randomUUID().toString(), - "platform:unknown", - ProtocolId.GB32960, - plo.logoutTime() != null ? plo.logoutTime() : eventTime, - ingestTime, - null, meta)); - } - } - return out; - } - - private void buildRealtime(Gb32960Message message, - com.lingniu.ingest.protocol.gb32960.model.Gb32960Header header, - Map meta, - Instant eventTime, Instant ingestTime, - List out) { - // 整车数据：先查 V2016，再 V2025；每帧只会命中其一，extracted 出来变成统一 - // 视图（VehicleView）供后面 payload 构造使用 - VehicleView v = extractVehicleView(message); - // 位置数据：同样两版本都查 - PositionView p = extractPositionView(message); - // 燃料电池：V2016 字段更全（带电压/电流），V2025 字段更少 - InfoBlock.Gb32960V2016.FuelCell fc16 = - message.findBlock(InfoBlock.Gb32960V2016.FuelCell.class).orElse(null); - InfoBlock.Gb32960V2025.FuelCell fc25 = - message.findBlock(InfoBlock.Gb32960V2025.FuelCell.class).orElse(null); - // 报警 - AlarmView alarm = extractAlarmView(message); - - if (v != null) { - Double fcVoltageV = fc16 != null ? fc16.fcVoltageV() : null; - Double fcCurrentA = fc16 != null ? fc16.fcCurrentA() : null; - Double fcTempC = fc16 != null ? fc16.maxHydrogenSystemTempC() - : (fc25 != null ? fc25.maxHydrogenSystemTempC() : null); - Double hydrogenHighPressure = fc16 != null ? fc16.maxHydrogenPressureMpa() - : (fc25 != null ? fc25.maxHydrogenPressureMpa() : null); - Double hydrogenRemaining = fc25 != null && fc25.remainingHydrogenPercent() != null - ? fc25.remainingHydrogenPercent().doubleValue() : null; - - RealtimePayload payload = new RealtimePayload( - v.speedKmh, - v.totalMileageKm, - v.socPercent != null ? v.socPercent.doubleValue() : null, - v.totalVoltageV, - v.totalCurrentA, - fcVoltageV, - fcCurrentA, - fcTempC, - hydrogenRemaining, - hydrogenHighPressure, - null, - mapVehicleState(v.vehicleState), - mapChargingState(v.chargingState), - mapRunningMode(v.runningMode), - extractGearLevel(v.gearRaw), - v.acceleratorPedal != null ? v.acceleratorPedal.doubleValue() : null, - v.brakePedal != null ? v.brakePedal.doubleValue() : null, - p != null ? p.longitude : null, - p != null ? p.latitude : null, - null, - null, - null, - null); - out.add(new VehicleEvent.Realtime( - UUID.randomUUID().toString(), - header.vin(), ProtocolId.GB32960, eventTime, ingestTime, - null, meta, payload)); - } - if (p != null) { - // 位置事件只从 POSITION 子包生成。没有 POSITION 时不从整车数据推断经纬度，避免制造假定位。 - LocationPayload loc = new LocationPayload( - p.longitude, p.latitude, 0.0, - v != null && v.speedKmh != null ? v.speedKmh : 0.0, - 0.0, 0, p.statusFlag); - Map locationMeta = v != null && v.totalMileageKm != null - ? withMeta(meta, "total_mileage_km", TelemetryMetadataValues.doubleValue(v.totalMileageKm)) - : meta; - out.add(new VehicleEvent.Location( - UUID.randomUUID().toString(), - header.vin(), ProtocolId.GB32960, eventTime, ingestTime, - null, locationMeta, loc)); - } - if (alarm != null && shouldEmitAlarm(alarm)) { - // 只在存在告警等级或氢安全关键位时发 Alarm，减少正常高频帧对告警消费者的噪声。 - List faultCodes = new ArrayList<>(); - addFaultCodes(faultCodes, "BAT", alarm.batteryFaults); - addFaultCodes(faultCodes, "MOT", alarm.motorFaults); - addFaultCodes(faultCodes, "ENG", alarm.engineFaults); - addFaultCodes(faultCodes, "OTH", alarm.otherFaults); - Set activeBitEnums = GeneralAlarmFlagBit.parse(alarm.generalAlarmFlag); - Set activeBits = activeBitEnums.stream() - .map(Enum::name) - .collect(Collectors.toCollection(java.util.LinkedHashSet::new)); - boolean hydrogenLeakDetected = activeBitEnums.contains(GeneralAlarmFlagBit.HYDROGEN_LEAK); - AlarmPayload ap = new AlarmPayload( - mapAlarmLevel(alarm.maxLevel, hydrogenLeakDetected), - (int) (alarm.generalAlarmFlag & 0xFFFF), - "GB32960_ALARM", - faultCodes, - activeBits, - p != null ? p.longitude : null, - p != null ? p.latitude : null, - safetyCategory(activeBitEnums), - hydrogenLeakDetected, - hydrogenLeakDetected - ? AlarmPayload.HydrogenLeakLevel.CRITICAL - : AlarmPayload.HydrogenLeakLevel.NONE, - hydrogenLeakDetected); - out.add(new VehicleEvent.Alarm( - UUID.randomUUID().toString(), - header.vin(), ProtocolId.GB32960, eventTime, ingestTime, - null, meta, ap)); - } - } - - /** 跨版本的整车数据视图，把 V2016 的踏板字段（V2025 没有）放在一起。 */ - private record VehicleView( - Integer vehicleState, Integer chargingState, Integer runningMode, - Double speedKmh, Double totalMileageKm, Double totalVoltageV, Double totalCurrentA, - Integer socPercent, Integer gearRaw, - Integer acceleratorPedal, Integer brakePedal) {} - - private static VehicleView extractVehicleView(Gb32960Message m) { - var v16 = m.findBlock(InfoBlock.Gb32960V2016.Vehicle.class).orElse(null); - if (v16 != null) { - return new VehicleView(v16.vehicleState(), v16.chargingState(), v16.runningMode(), - v16.speedKmh(), v16.totalMileageKm(), v16.totalVoltageV(), v16.totalCurrentA(), - v16.socPercent(), v16.gearRaw(), - v16.acceleratorPedal(), v16.brakePedal()); - } - var v25 = m.findBlock(InfoBlock.Gb32960V2025.Vehicle.class).orElse(null); - if (v25 != null) { - return new VehicleView(v25.vehicleState(), v25.chargingState(), v25.runningMode(), - v25.speedKmh(), v25.totalMileageKm(), v25.totalVoltageV(), v25.totalCurrentA(), - v25.socPercent(), v25.gearRaw(), - null, null); - } - return null; - } - - /** 跨版本的位置数据视图。 */ - private record PositionView(int statusFlag, double longitude, double latitude) {} - - private static PositionView extractPositionView(Gb32960Message m) { - var p16 = m.findBlock(InfoBlock.Gb32960V2016.Position.class).orElse(null); - if (p16 != null) return new PositionView(p16.statusFlag(), p16.longitude(), p16.latitude()); - var p25 = m.findBlock(InfoBlock.Gb32960V2025.Position.class).orElse(null); - if (p25 != null) return new PositionView(p25.statusFlag(), p25.longitude(), p25.latitude()); - return null; - } - - /** 跨版本的报警视图（仅暴露公共字段，2025 的 generalAlarmLevels 暂未上送下游）。 */ - private record AlarmView(Integer maxLevel, long generalAlarmFlag, - List batteryFaults, List motorFaults, - List engineFaults, List otherFaults) {} - - private static AlarmView extractAlarmView(Gb32960Message m) { - var a16 = m.findBlock(InfoBlock.Gb32960V2016.Alarm.class).orElse(null); - if (a16 != null) { - return new AlarmView(a16.maxLevel(), a16.generalAlarmFlag(), - a16.batteryFaults(), a16.motorFaults(), a16.engineFaults(), a16.otherFaults()); - } - var a25 = m.findBlock(InfoBlock.Gb32960V2025.Alarm.class).orElse(null); - if (a25 != null) { - return new AlarmView(a25.maxLevel(), a25.generalAlarmFlag(), - a25.batteryFaults(), a25.motorFaults(), a25.engineFaults(), a25.otherFaults()); - } - return null; - } - - /** 挡位字节按附录 A.1 解析：低 4 位为挡位值 0000-1111。 */ - private static Integer extractGearLevel(Integer gearRaw) { - if (gearRaw == null) return null; - if ((gearRaw & 0x80) != 0) return null; // bit7: 挡位无效 - return gearRaw & 0x0F; - } - - private static RealtimePayload.VehicleState mapVehicleState(Integer code) { - if (code == null) return null; - return switch (code) { - case 0x01 -> RealtimePayload.VehicleState.STARTED; - case 0x02 -> RealtimePayload.VehicleState.SHUTDOWN; - case 0x03 -> RealtimePayload.VehicleState.OTHER; - default -> RealtimePayload.VehicleState.INVALID; - }; - } - - private static RealtimePayload.ChargingState mapChargingState(Integer code) { - if (code == null) return null; - return switch (code) { - case 0x01 -> RealtimePayload.ChargingState.PARKED_CHARGING; - case 0x02 -> RealtimePayload.ChargingState.DRIVING_CHARGING; - case 0x03 -> RealtimePayload.ChargingState.UNCHARGED; - case 0x04 -> RealtimePayload.ChargingState.CHARGED; - default -> RealtimePayload.ChargingState.INVALID; - }; - } - - private static RealtimePayload.RunningMode mapRunningMode(Integer code) { - if (code == null) return null; - return switch (code) { - case 0x01 -> RealtimePayload.RunningMode.ELECTRIC; - case 0x02 -> RealtimePayload.RunningMode.HYBRID; - case 0x03 -> RealtimePayload.RunningMode.FUEL; - default -> RealtimePayload.RunningMode.INVALID; - }; - } - - private static boolean shouldEmitAlarm(AlarmView alarm) { - if (alarm.maxLevel != null && alarm.maxLevel > 0) return true; - return GeneralAlarmFlagBit.HYDROGEN_LEAK.isSet(alarm.generalAlarmFlag) - || GeneralAlarmFlagBit.HYDROGEN_PRESSURE_ABNORMAL.isSet(alarm.generalAlarmFlag) - || GeneralAlarmFlagBit.HYDROGEN_TEMP_ABNORMAL.isSet(alarm.generalAlarmFlag); - } - - private static AlarmPayload.AlarmLevel mapAlarmLevel(Integer code, boolean hydrogenLeakDetected) { - if (hydrogenLeakDetected) return AlarmPayload.AlarmLevel.CRITICAL; - if (code == null) return AlarmPayload.AlarmLevel.INFO; - return switch (code) { - case 1 -> AlarmPayload.AlarmLevel.MINOR; - case 2 -> AlarmPayload.AlarmLevel.MAJOR; - case 3, 4 -> AlarmPayload.AlarmLevel.CRITICAL; - default -> AlarmPayload.AlarmLevel.INFO; - }; - } - - private static AlarmPayload.SafetyCategory safetyCategory(Set activeBits) { - if (activeBits.contains(GeneralAlarmFlagBit.HYDROGEN_LEAK)) { - return AlarmPayload.SafetyCategory.HYDROGEN_LEAK; - } - if (activeBits.contains(GeneralAlarmFlagBit.HYDROGEN_PRESSURE_ABNORMAL)) { - return AlarmPayload.SafetyCategory.TANK_PRESSURE; - } - if (activeBits.contains(GeneralAlarmFlagBit.HYDROGEN_TEMP_ABNORMAL)) { - return AlarmPayload.SafetyCategory.TANK_TEMPERATURE; - } - return AlarmPayload.SafetyCategory.GENERAL; - } - - private static void addFaultCodes(List out, String prefix, List codes) { - if (codes == null || codes.isEmpty()) return; - for (Long c : codes) out.add(prefix + "-" + Long.toHexString(c)); - } - - private static Map withMeta(Map meta, String key, String value) { - java.util.HashMap copy = new java.util.HashMap<>(meta); - copy.put(key, value == null ? "" : value); - return Map.copyOf(copy); - } - -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/CommandBody.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/CommandBody.java deleted file mode 100644 index ca80804f..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/CommandBody.java +++ /dev/null @@ -1,225 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.model; - -import java.time.Instant; -import java.util.List; -import java.util.Map; - -/** - * 命令数据单元（非实时上报）。sealed 集合；每个子类型对应一条 {@link CommandType}。 - * - *

    实时/补发上报的数据单元不走这里，而是产出 {@link InfoBlock} 列表。 - * - *

    上行类型（由车载终端发起）：VehicleLogin/VehicleLogout/PlatformLogin/PlatformLogout/ - * Heartbeat/TimeCalibration/Activation/KeyExchange。 - * - *

    下行类型（由平台发起）：QueryParams/SetParams/TerminalControl/ActivationResponse。 - * 参见 GB/T 32960.3-2025 附录 B 表 B.5~B.13。 - */ -public sealed interface CommandBody - permits CommandBody.VehicleLogin, - CommandBody.VehicleLogout, - CommandBody.PlatformLogin, - CommandBody.PlatformLogout, - CommandBody.Heartbeat, - CommandBody.TimeCalibration, - CommandBody.Activation, - CommandBody.ActivationResponse, - CommandBody.KeyExchange, - CommandBody.QueryParams, - CommandBody.QueryParamsResponse, - CommandBody.SetParams, - CommandBody.TerminalControl, - CommandBody.RawCommand { - - /** - * 车辆登入 0x01（表 6）。 - * - * @param loginTime 数据采集时间（表 5） - * @param serialNo 登入流水号 1~65531，按天循环累加 - * @param iccid 20 字节 SIM 卡 ICCID，终端从 SIM 卡获取 - * @param batterySystemCount 电池管理系统数 n (0~20，2025 版新增) - * @param batteryPackCounts 每个电池管理系统对应的动力蓄电池包个数 - * @param batteryCodes 动力蓄电池包编码列表（GB/T 34014，24 字节/个） - */ - record VehicleLogin( - Instant loginTime, - int serialNo, - String iccid, - int batterySystemCount, - List batteryPackCounts, - List batteryCodes - ) implements CommandBody {} - - /** 车辆登出 0x04（表 28）。 */ - record VehicleLogout(Instant logoutTime, int serialNo) implements CommandBody {} - - /** 平台登入 0x05（表 29）。 */ - record PlatformLogin( - Instant loginTime, - int serialNo, - String username, - String password, - EncryptType encryptRule - ) implements CommandBody {} - - /** 平台登出 0x06（表 30）。 */ - record PlatformLogout(Instant logoutTime, int serialNo) implements CommandBody {} - - /** 心跳 0x07：数据单元为空（B.3.5.10）。 */ - record Heartbeat() implements CommandBody {} - - /** 终端校时 0x08：信息标志及信息体为空（B.3.5.11）。 */ - record TimeCalibration() implements CommandBody {} - - /** 激活 0x09（表 B.3）。 */ - record Activation( - Instant collectTime, - String chipId, - byte[] publicKey, - String vin, - byte[] signature - ) implements CommandBody {} - - /** 激活结果应答 0x0A（表 B.4）。 */ - record ActivationResponse(int status, int info) implements CommandBody {} - - /** - * 数据单元加密密钥交换 0x0B（表 31）。 - * - * @param keyType 密钥类型（{@link EncryptType}） - * @param key 密钥字节 - * @param activateAt 启用时间 - * @param expireAt 失效时间 - */ - record KeyExchange( - EncryptType keyType, - byte[] key, - Instant activateAt, - Instant expireAt - ) implements CommandBody {} - - // ======================================================================== - // 下行命令 0x80 / 0x81 / 0x82 —— 附录 B 表 B.5 ~ B.13 - // ======================================================================== - - /** - * 查询命令 0x80（表 B.5）。平台向车载终端发送，请求查询指定参数。 - * - * @param queryTime 参数查询时间 - * @param paramIds 待查询的参数 ID 列表（表 B.8：0x01-0x10，2025 版新增 0x0D-0x10） - */ - record QueryParams(Instant queryTime, List paramIds) implements CommandBody {} - - /** - * 参数查询返回 0x80 应答（表 B.6/B.7/B.8）。车载终端响应平台。 - * - * @param respondTime 返回查询参数时间 - * @param params 参数项键值对（key = paramId，value = bytes，需按表 B.8 解释） - */ - record QueryParamsResponse(Instant respondTime, Map params) implements CommandBody {} - - /** - * 设置命令 0x81（表 B.9）。平台向车载终端发送，修改车载终端参数。 - * - *

    参数 ID 范围见表 B.8，参数值定义见表 B.8 各字段说明。 - * 其中 0x07 硬件版本、0x08 固件版本为只读，不在 set 命令中出现。 - * - * @param setTime 参数设置时间 - * @param params 参数项键值对 - */ - record SetParams(Instant setTime, Map params) implements CommandBody {} - - /** - * 车载终端控制命令 0x82（表 B.10）。 - * - * @param time 时间 - * @param commandId 控制命令（{@link ControlCommand}） - * @param payload 根据不同控制命令的负载 - */ - record TerminalControl( - Instant time, - ControlCommand commandId, - ControlPayload payload - ) implements CommandBody {} - - /** 控制命令枚举（表 B.11）。 */ - enum ControlCommand { - /** 0x01：远程升级，参数见表 B.12。 */ - REMOTE_UPGRADE(0x01), - /** 0x02：车载终端关机。 */ - SHUTDOWN(0x02), - /** 0x03：车载终端复位。 */ - RESET(0x03), - /** 0x04：车载终端恢复出厂设置。 */ - FACTORY_RESET(0x04), - /** 0x05：断开数据通信链路。 */ - DISCONNECT(0x05), - /** 0x06：车载终端报警/预警，参数见表 B.13。 */ - ALARM_COMMAND(0x06), - /** 0x07：开启抽样监测链路。 */ - SAMPLING_LINK_ON(0x07), - /** 未知命令 / 用户自定义 0x80-0xFE。 */ - UNKNOWN(-1); - - private final int code; - ControlCommand(int code) { this.code = code; } - public int code() { return code; } - - public static ControlCommand of(int code) { - for (ControlCommand c : values()) if (c.code == code) return c; - return UNKNOWN; - } - } - - /** 控制命令的负载。sealed：每种控制命令对应一种 payload 类型。 */ - sealed interface ControlPayload - permits ControlPayload.None, ControlPayload.RemoteUpgrade, ControlPayload.AlarmCommand, ControlPayload.RawPayload { - - /** 无参数（SHUTDOWN / RESET / FACTORY_RESET / DISCONNECT / SAMPLING_LINK_ON）。 */ - record None() implements ControlPayload {} - - /** - * 远程升级参数（表 B.12）。 - * - * @param apn 拨号点名称 - * @param dialUser 拨号用户名 - * @param dialPassword 拨号密码 - * @param serverAddress 升级服务器地址（IP 或域名，IPv4 时前 2 字节为 0） - * @param serverPort 升级服务器端口 - * @param manufacturerCode 车载终端制造商 ID（4 字节英文大写字母或数字） - * @param hardwareVersion 硬件版本（5 字节） - * @param firmwareVersion 固件版本（5 字节） - * @param upgradeUrl 升级 URL 地址（建议 FTP） - * @param connectTimeoutMinutes 连接到升级服务器时限（0~60000 分钟） - */ - record RemoteUpgrade( - String apn, - String dialUser, - String dialPassword, - byte[] serverAddress, - int serverPort, - String manufacturerCode, - String hardwareVersion, - String firmwareVersion, - String upgradeUrl, - int connectTimeoutMinutes - ) implements ControlPayload {} - - /** - * 报警/预警命令（表 B.13）。 - * - * @param warningLevel 警告等级：0x00 无报警 / 0x01-0x04 1-4 级报警 / 0xFF 无效 - * @param info 预留字节，可变长 - */ - record AlarmCommand(int warningLevel, byte[] info) implements ControlPayload {} - - /** 未解析的自定义 payload（0x80-0xFE 用户自定义等）。 */ - record RawPayload(byte[] bytes) implements ControlPayload {} - } - - /** - * 未解析或厂商自定义的命令体（0xC0~0xFE 等）。 - * 保留原始字节以便下游冷存分析。 - */ - record RawCommand(int commandCode, byte[] bytes) implements CommandBody {} -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/CommandType.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/CommandType.java deleted file mode 100644 index 0bcc3fb5..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/CommandType.java +++ /dev/null @@ -1,60 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.model; - -/** - * 命令标识。GB/T 32960.3-2025 表 3 和附录 B 表 B.2 "命令标识定义"。 - * - *

    命令标识是发起方的唯一标识，区分数据传输的方向与种类。 - * - *

    方向约定： - *

      - *
    • 上行：车载终端 → 远程服务与管理平台 - *
    • 下行：远程服务与管理平台 → 车载终端 - *
    • 上行/下行：双向使用 - *
    - */ -public enum CommandType { - /** 0x01：车辆登入（上行）—— 见表 6。 */ - VEHICLE_LOGIN(0x01), - /** 0x02：实时信息上报（上行）—— 见表 7。 */ - REALTIME_REPORT(0x02), - /** 0x03：补发信息上报（上行）—— 数据格式与实时上报一致，用于通信链路恢复后补发 7d 内存储的数据。 */ - RESEND_REPORT(0x03), - /** 0x04：车辆登出（上行）—— 见表 28。 */ - VEHICLE_LOGOUT(0x04), - /** 0x05：平台登入（上行）—— 见表 29；2025 版附录 B.2 中 0x05/0x06 改为"平台传输数据占用，自定义"。 */ - PLATFORM_LOGIN(0x05), - /** 0x06：平台登出（上行）—— 见表 30。 */ - PLATFORM_LOGOUT(0x06), - /** 0x07：心跳（上行）—— 数据单元为空，见 B.3.5.10。 */ - HEARTBEAT(0x07), - /** 0x08：终端校时（上行）—— 信息标志及信息体为空，见 B.3.5.11。 */ - TIME_CALIBRATION(0x08), - /** 0x09：激活（上行）—— 见表 B.3。 */ - ACTIVATION(0x09), - /** 0x0A：激活应答（下行）—— 见表 B.4。 */ - ACTIVATION_RESPONSE(0x0A), - /** 0x0B：数据单元加密密钥交换（上行/下行）—— 见表 31。 */ - KEY_EXCHANGE(0x0B), - /** 0x80：查询命令（下行）—— 见表 B.5 ~ B.8。 */ - QUERY(0x80), - /** 0x81：设置命令（下行）—— 见表 B.9。 */ - SET_PARAMS(0x81), - /** 0x82：车载终端控制命令（下行）—— 见表 B.10 ~ B.13。 */ - TERMINAL_CONTROL(0x82); - - private final int code; - - CommandType(int code) { - this.code = code; - } - - public int code() { - return code; - } - - /** 按协议编码查找命令类型；未识别时抛异常以便 DLQ 路由。 */ - public static CommandType of(int code) { - for (CommandType c : values()) if (c.code == code) return c; - throw new IllegalArgumentException("unknown gb32960 command: 0x" + Integer.toHexString(code)); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/EncryptType.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/EncryptType.java deleted file mode 100644 index 769a0b5f..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/EncryptType.java +++ /dev/null @@ -1,38 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.model; - -/** - * 数据单元加密方式。GB/T 32960.3-2025 表 2 "数据单元加密方式"。 - * - *

    当数据单元存在加密时，应先加密后校验，解析端先校验后解密。 - */ -public enum EncryptType { - /** 0x01：数据不加密。 */ - UNENCRYPTED(0x01), - /** 0x02：RSA 算法加密。 */ - RSA(0x02), - /** 0x03：AES（高级加密标准）算法加密。 */ - AES(0x03), - /** 0x04：SM2 算法加密（2025 版新增）。 */ - SM2(0x04), - /** 0x05：SM4 算法加密（2025 版新增）。 */ - SM4(0x05), - /** 0xFE：异常。 */ - ABNORMAL(0xFE), - /** 0xFF：无效。 */ - INVALID(0xFF); - - private final int code; - - EncryptType(int code) { - this.code = code; - } - - public int code() { - return code; - } - - public static EncryptType of(int code) { - for (EncryptType e : values()) if (e.code == code) return e; - return INVALID; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/Gb32960Header.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/Gb32960Header.java deleted file mode 100644 index 765e2880..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/Gb32960Header.java +++ /dev/null @@ -1,36 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.model; - -import java.time.Instant; - -/** - * GB/T 32960.3 数据包报文头（含起始符/命令单元/VIN/加密方式/数据长度），见表 2 / 表 B.1。 - * - *

    完整数据包结构： - * 

    - *   起始符 (2B)            ## (0x23 0x23) 或 $$ (0x24 0x24)
- *   命令标识 (1B)          CommandType
- *   应答标志 (1B)          ResponseFlag
- *   唯一识别码 (17B STRING) VIN 或自定义编码
- *   数据单元加密方式 (1B)   EncryptType
- *   数据单元长度 (2B WORD)  0 ~ 65531
- *   数据单元 (NB)          见 §7
- *   校验码 (1B BCC)        对命令单元到数据单元结尾异或
- *
    - * - * @param protocolVersion 协议版本，由起始符推断 - * @param command 命令标识（表 3 / B.2） - * @param responseFlag 应答标志（表 4） - * @param vin 唯一识别码（17 字节 ASCII） - * @param encryptType 数据单元加密方式（表 2） - * @param dataLength 数据单元长度（字节） - * @param eventTime 数据采集时间。仅实时/补发上报携带，其它命令为 null - */ -public record Gb32960Header( - ProtocolVersion protocolVersion, - CommandType command, - ResponseFlag responseFlag, - String vin, - EncryptType encryptType, - int dataLength, - Instant eventTime -) {} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/Gb32960Message.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/Gb32960Message.java deleted file mode 100644 index 39ace8d5..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/Gb32960Message.java +++ /dev/null @@ -1,43 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.model; - -import java.util.Collections; -import java.util.List; -import java.util.Optional; - -/** - * 完整解析后的 32960 消息。一条消息要么是实时/补发上报（携带 {@link InfoBlock} 列表）， - * 要么是登入/登出/平台/心跳/校时/激活/密钥交换等命令（携带 {@link CommandBody}）， - * 二者互斥。 - * - * @param header 报文头 - * @param infoBlocks 实时/补发上报的信息体列表；其他命令为空列表 - * @param commandBody 非实时上报的命令数据单元；实时/补发上报为 null - * @param signature 实时上报尾部的签名原始字节（2025 版新增，2016 版为 null） - */ -public record Gb32960Message( - Gb32960Header header, - List infoBlocks, - CommandBody commandBody, - byte[] signature -) { - - /** 便捷构造：只含信息体（实时/补发上报，2016 版常用）。 */ - public Gb32960Message(Gb32960Header header, List infoBlocks) { - this(header, infoBlocks == null ? Collections.emptyList() : infoBlocks, null, null); - } - - /** 便捷构造：非实时命令。 */ - public Gb32960Message(Gb32960Header header, CommandBody commandBody) { - this(header, Collections.emptyList(), commandBody, null); - } - - public boolean isReport() { - return header.command() == CommandType.REALTIME_REPORT - || header.command() == CommandType.RESEND_REPORT; - } - - @SuppressWarnings("unchecked") - public Optional findBlock(Class type) { - return (Optional) infoBlocks.stream().filter(type::isInstance).findFirst(); - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/GeneralAlarmFlagBit.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/GeneralAlarmFlagBit.java deleted file mode 100644 index 0843becd..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/GeneralAlarmFlagBit.java +++ /dev/null @@ -1,104 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.model; - -import java.util.ArrayList; -import java.util.EnumSet; -import java.util.List; -import java.util.Set; - -/** - * 通用报警标志位定义（GB/T 32960.3-2025 表 24）。4 字节 32 位位图中的每个位对应一种报警， - * 2025 版将 2016 版 bit0-15 扩展到 bit0-27，新增了热事件、氢系统、超级电容等位定义。 - * - *

    "1" 表示报警，"0" 表示正常；标志维持到报警条件解除。 - */ -public enum GeneralAlarmFlagBit { - /** bit0：温度差异报警 */ - TEMP_DIFFERENCE(0), - /** bit1：电池高温报警 */ - BATTERY_HIGH_TEMP(1), - /** bit2：车载储能装置类型过压报警 */ - STORAGE_OVER_VOLTAGE(2), - /** bit3：车载储能装置类型欠压报警 */ - STORAGE_UNDER_VOLTAGE(3), - /** bit4：SOC 低报警 */ - SOC_LOW(4), - /** bit5：最小并联单元过压报警 */ - MIN_PARALLEL_UNIT_OVER_VOLTAGE(5), - /** bit6：最小并联单元欠压报警 */ - MIN_PARALLEL_UNIT_UNDER_VOLTAGE(6), - /** bit7：SOC 过高报警 */ - SOC_HIGH(7), - /** bit8：SOC 跳变报警 */ - SOC_JUMP(8), - /** bit9：可充电储能系统不匹配报警 */ - STORAGE_MISMATCH(9), - /** bit10：最小并联单元一致性差报警 */ - MIN_PARALLEL_UNIT_INCONSISTENT(10), - /** bit11：绝缘电阻失效报警 */ - INSULATION_FAILURE(11), - /** bit12：DC-DC 温度报警 */ - DCDC_TEMP(12), - /** bit13：制动系统报警 */ - BRAKE_SYSTEM(13), - /** bit14：DC-DC 状态报警 */ - DCDC_STATUS(14), - /** bit15：驱动电机控制器温度报警 */ - MOTOR_CONTROLLER_TEMP(15), - /** bit16：高压互锁状态报警 */ - HV_INTERLOCK(16), - /** bit17：驱动电机温度报警 */ - MOTOR_TEMP(17), - /** bit18：车载储能装置类型过充报警 */ - STORAGE_OVER_CHARGE(18), - /** bit19：驱动电机超速报警（2025 版新增） */ - MOTOR_OVER_SPEED(19), - /** bit20：驱动电机过流报警（2025 版新增） */ - MOTOR_OVER_CURRENT(20), - /** bit21：超级电容过温报警（2025 版新增） */ - SUPER_CAP_OVER_TEMP(21), - /** bit22：超级电容过压报警（2025 版新增） */ - SUPER_CAP_OVER_VOLTAGE(22), - /** bit23：可充电储能装置热事件报警（2025 版新增，出现即为 4 级故障） */ - STORAGE_THERMAL_EVENT(23), - /** bit24：氢气泄漏异常报警（2025 版新增） */ - HYDROGEN_LEAK(24), - /** bit25：车载氢系统压力异常报警（2025 版新增） */ - HYDROGEN_PRESSURE_ABNORMAL(25), - /** bit26：车载氢系统温度异常报警（2025 版新增） */ - HYDROGEN_TEMP_ABNORMAL(26), - /** bit27：燃料电池电堆超温报警（2025 版新增） */ - FUEL_CELL_STACK_OVER_TEMP(27); - - private final int bitIndex; - - GeneralAlarmFlagBit(int bitIndex) { - this.bitIndex = bitIndex; - } - - public int bitIndex() { - return bitIndex; - } - - /** 当前位是否在 flag 中置位。 */ - public boolean isSet(long flag) { - return (flag & (1L << bitIndex)) != 0; - } - - /** 从 32 位标志解出所有置位的报警。 */ - public static Set parse(long flag) { - EnumSet out = EnumSet.noneOf(GeneralAlarmFlagBit.class); - for (GeneralAlarmFlagBit bit : values()) { - if (bit.isSet(flag)) out.add(bit); - } - return out; - } - - /** 作为可读 fault code 列表（形如 {@code SOC_LOW, MOTOR_TEMP}）。 */ - public static List asCodeList(long flag) { - List out = new ArrayList<>(); - for (GeneralAlarmFlagBit bit : values()) { - if (bit.isSet(flag)) out.add(bit.name()); - } - return out; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/InfoBlock.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/InfoBlock.java deleted file mode 100644 index e775eb26..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/InfoBlock.java +++ /dev/null @@ -1,463 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.model; - -import java.util.List; - -/** - * 已解析的信息体基接口。按 {@code <协议+版本>.<数据类型>} 严格分组： - * - *

      - *
    • {@link Gb32960V2016} —— GB/T 32960.3-2016 附录 B 表 B.10~B.20 - *
    • {@link Gb32960V2025} —— GB/T 32960.3-2025 §7.2.4 表 10~27 - *
    • {@link GuangdongFc} —— 广东燃料电池汽车示范应用城市群综合监管平台规范 v1.0 - *
    • {@link Raw} —— 未实现 parser 的原始字节兜底 - *
    - * - *

    哪怕同一个数据类型在两个版本字段一致（如 Engine 5 字节固定布局），也按规则 - * 复制成两份 record，绝不跨版本共用 record 类型，以便消费方在 instanceof / switch - * 时能严格区分数据来源。 - * - *

    异常值约定： - *

      - *
    • 单字节：{@code 0xFE} 异常 / {@code 0xFF} 无效 - *
    • 双字节：{@code 0xFFFE} 异常 / {@code 0xFFFF} 无效 - *
    • 四字节：{@code 0xFFFFFFFE} 异常 / {@code 0xFFFFFFFF} 无效 - *
    - * 对外 record 使用 {@link Double} / {@link Integer} 等装箱类型，异常或无效时设为 - * {@code null}。 - */ -public sealed interface InfoBlock - permits InfoBlock.Gb32960V2016, - InfoBlock.Gb32960V2025, - InfoBlock.GuangdongFc, - InfoBlock.Raw { - - InfoBlockType type(); - - // ======================================================================== - // GB/T 32960.3-2016 - // ======================================================================== - - /** GB/T 32960.3-2016 附录 B 定义的全部信息体类型。 */ - sealed interface Gb32960V2016 extends InfoBlock - permits Gb32960V2016.Vehicle, - Gb32960V2016.DriveMotor, - Gb32960V2016.FuelCell, - Gb32960V2016.Engine, - Gb32960V2016.Position, - Gb32960V2016.Extreme, - Gb32960V2016.Alarm, - Gb32960V2016.Voltage, - Gb32960V2016.Temperature { - - /** 0x01 整车数据。固定 20 字节（含加速/制动踏板）。 */ - record Vehicle( - Integer vehicleState, - Integer chargingState, - Integer runningMode, - Double speedKmh, - Double totalMileageKm, - Double totalVoltageV, - Double totalCurrentA, - Integer socPercent, - Integer dcDcStatus, - Integer gearRaw, - Integer insulationResistanceKohm, - Integer acceleratorPedal, - Integer brakePedal - ) implements Gb32960V2016 { - @Override public InfoBlockType type() { return InfoBlockType.VEHICLE; } - } - - /** 0x02 驱动电机数据。变长。每电机 12 字节（rpm 偏移 20000，torque 2 字节偏移 20000）。 */ - record DriveMotor(List motors) implements Gb32960V2016 { - @Override public InfoBlockType type() { return InfoBlockType.DRIVE_MOTOR; } - - public record Motor( - Integer serialNo, - Integer state, - Integer controllerTempC, - Integer rpm, - Double torqueNm, - Integer motorTempC, - Double controllerInputVoltageV, - Double controllerDcCurrentA - ) {} - } - - /** 0x03 燃料电池数据（2016 版字段顺序，含 N×探针温度 1B）。变长。 */ - record FuelCell( - Double fcVoltageV, - Double fcCurrentA, - Double hydrogenConsumptionKgPer100km, - List probeTempC, - Double maxHydrogenSystemTempC, - Integer maxHydrogenSystemTempProbeId, - Double maxHydrogenConcentrationPercent, - Integer maxHydrogenConcentrationProbeId, - Double maxHydrogenPressureMpa, - Integer maxHydrogenPressureProbeId, - Integer hvDcDcStatus - ) implements Gb32960V2016 { - @Override public InfoBlockType type() { return InfoBlockType.FUEL_CELL_V2016; } - } - - /** 0x04 发动机数据。固定 5 字节。 */ - record Engine( - Integer engineState, - Integer crankshaftRpm, - Double fuelConsumptionRate - ) implements Gb32960V2016 { - @Override public InfoBlockType type() { return InfoBlockType.ENGINE; } - } - - /** 0x05 车辆位置数据。固定 9 字节。 */ - record Position( - int statusFlag, - double longitude, - double latitude - ) implements Gb32960V2016 { - @Override public InfoBlockType type() { return InfoBlockType.POSITION_V2016; } - } - - /** 0x06 动力蓄电池极值数据。固定 14 字节。 */ - record Extreme( - Integer maxCellVoltageSubSysNo, - Integer maxCellVoltageCellId, - Double maxCellVoltageV, - Integer minCellVoltageSubSysNo, - Integer minCellVoltageCellId, - Double minCellVoltageV, - Integer maxTempSubSysNo, - Integer maxTempProbeId, - Integer maxTempC, - Integer minTempSubSysNo, - Integer minTempProbeId, - Integer minTempC - ) implements Gb32960V2016 { - @Override public InfoBlockType type() { return InfoBlockType.EXTREME_V2016; } - } - - /** 0x07 报警数据。变长（4 个故障代码列表）。 */ - record Alarm( - Integer maxLevel, - long generalAlarmFlag, - List batteryFaults, - List motorFaults, - List engineFaults, - List otherFaults - ) implements Gb32960V2016 { - @Override public InfoBlockType type() { return InfoBlockType.ALARM_V2016; } - } - - /** 0x08 可充电储能装置电压数据。变长。PoC 仅保留汇总。 */ - record Voltage( - int subSystemCount, - double maxCellVoltageV, - double minCellVoltageV, - double totalVoltageV - ) implements Gb32960V2016 { - @Override public InfoBlockType type() { return InfoBlockType.VOLTAGE_V2016; } - } - - /** 0x09 可充电储能装置温度数据。变长。 */ - record Temperature( - int subSystemCount, - int maxTempC, - int minTempC - ) implements Gb32960V2016 { - @Override public InfoBlockType type() { return InfoBlockType.TEMPERATURE_V2016; } - } - } - - // ======================================================================== - // GB/T 32960.3-2025 - // ======================================================================== - - /** GB/T 32960.3-2025 §7.2.4 定义的全部信息体类型。 */ - sealed interface Gb32960V2025 extends InfoBlock - permits Gb32960V2025.Vehicle, - Gb32960V2025.DriveMotor, - Gb32960V2025.FuelCell, - Gb32960V2025.Engine, - Gb32960V2025.Position, - Gb32960V2025.Alarm, - Gb32960V2025.MinParallelVoltage, - Gb32960V2025.BatteryTemperature, - Gb32960V2025.FuelCellStack, - Gb32960V2025.SuperCapacitor, - Gb32960V2025.SuperCapacitorExtreme { - - /** 0x01 整车数据。固定 18 字节（删除加速/制动踏板）。 */ - record Vehicle( - Integer vehicleState, - Integer chargingState, - Integer runningMode, - Double speedKmh, - Double totalMileageKm, - Double totalVoltageV, - Double totalCurrentA, - Integer socPercent, - Integer dcDcStatus, - Integer gearRaw, - Integer insulationResistanceKohm - ) implements Gb32960V2025 { - @Override public InfoBlockType type() { return InfoBlockType.VEHICLE; } - } - - /** 0x02 驱动电机数据。变长。每电机 14 字节（rpm 偏移 32000，torque 4 字节偏移 20000）。 */ - record DriveMotor(List motors) implements Gb32960V2025 { - @Override public InfoBlockType type() { return InfoBlockType.DRIVE_MOTOR; } - - public record Motor( - Integer serialNo, - Integer state, - Integer controllerTempC, - Integer rpm, - Double torqueNm, - Integer motorTempC, - Double controllerInputVoltageV, - Double controllerDcCurrentA - ) {} - } - - /** 0x03 燃料电池发动机及车载氢系统数据。固定 13 字节。 */ - record FuelCell( - Double maxHydrogenSystemTempC, - Integer maxHydrogenSystemTempProbeId, - Double maxHydrogenConcentrationPercent, - Integer maxHydrogenConcentrationProbeId, - Double maxHydrogenPressureMpa, - Integer maxHydrogenPressureProbeId, - Integer hvDcDcStatus, - Integer remainingHydrogenPercent, - Integer hvDcDcControllerTempC - ) implements Gb32960V2025 { - @Override public InfoBlockType type() { return InfoBlockType.FUEL_CELL_V2025; } - } - - /** 0x04 发动机数据。固定 5 字节。字段同 2016 版。 */ - record Engine( - Integer engineState, - Integer crankshaftRpm, - Double fuelConsumptionRate - ) implements Gb32960V2025 { - @Override public InfoBlockType type() { return InfoBlockType.ENGINE; } - } - - /** 0x05 车辆位置数据。固定 10 字节（多坐标系字节）。 */ - record Position( - int statusFlag, - int coordSystem, - double longitude, - double latitude - ) implements Gb32960V2025 { - @Override public InfoBlockType type() { return InfoBlockType.POSITION_V2025; } - } - - /** 0x06 报警数据。变长（4 个故障代码列表 + 通用报警故障等级列表）。 */ - record Alarm( - Integer maxLevel, - long generalAlarmFlag, - List batteryFaults, - List motorFaults, - List engineFaults, - List otherFaults, - List generalAlarmLevels - ) implements Gb32960V2025 { - @Override public InfoBlockType type() { return InfoBlockType.ALARM_V2025; } - - /** 通用报警故障等级列表项（2 字节）：bit 对应表 24 位序号，level 为故障等级。 */ - public record GeneralAlarmEntry(int bit, int level) {} - } - - /** 0x07 动力蓄电池最小并联单元电压数据。变长。 */ - record MinParallelVoltage(int subSystemCount, List packs) implements Gb32960V2025 { - @Override public InfoBlockType type() { return InfoBlockType.MIN_PARALLEL_VOLTAGE_V2025; } - - /** totalUnits 是车端声明数量；frameVoltages 是本帧实际解析出的完整电压值。 */ - public record Pack( - int packNo, - Double packVoltageV, - Double packCurrentA, - int totalUnits, - List frameVoltages - ) {} - } - - /** 0x08 动力蓄电池温度数据。变长。 */ - record BatteryTemperature(int subSystemCount, List packs) implements Gb32960V2025 { - @Override public InfoBlockType type() { return InfoBlockType.BATTERY_TEMPERATURE_V2025; } - - /** probeCount 是车端声明数量；probeTempsC 是本帧实际解析出的温度值。 */ - public record Pack(int packNo, int probeCount, List probeTempsC) {} - } - - /** 0x30 燃料电池电堆数据。变长（2025 标准）。 */ - record FuelCellStack(int stackCount, List stacks) implements Gb32960V2025 { - @Override public InfoBlockType type() { return InfoBlockType.FUEL_CELL_STACK; } - - public record Stack( - int stackNo, - Double voltageV, - Double currentA, - // 标准字段名写 pressure，协议比例按 kPa 命名，避免与 2016 Mpa 字段混淆。 - Double hydrogenInletPressureKpa, - Double airInletPressureKpa, - Integer airInletTempC, - List coolantOutletProbeTempsC - ) {} - } - - /** 0x31 超级电容器数据。变长。 */ - record SuperCapacitor( - int systemNo, - Double totalVoltageV, - Double totalCurrentA, - List cellVoltages, - List probeTempsC - ) implements Gb32960V2025 { - @Override public InfoBlockType type() { return InfoBlockType.SUPER_CAPACITOR; } - } - - /** 0x32 超级电容器极值数据。固定 18 字节。 */ - record SuperCapacitorExtreme( - Integer maxVoltageSystemNo, - Integer maxVoltageCellId, - Double maxCellVoltageV, - Integer minVoltageSystemNo, - Integer minVoltageCellId, - Double minCellVoltageV, - Integer maxTempSystemNo, - Integer maxTempProbeId, - Integer maxTempC, - Integer minTempSystemNo, - Integer minTempProbeId, - Integer minTempC - ) implements Gb32960V2025 { - @Override public InfoBlockType type() { return InfoBlockType.SUPER_CAPACITOR_EXTREME; } - } - } - - // ======================================================================== - // 广东燃料电池汽车示范应用城市群综合监管平台规范 v1.0（基于 GB/T 32960.3-2016 帧头） - // ======================================================================== - - /** 广东燃料电池汽车示范应用规范定义的扩展信息体（typeCode 0x30~0x34、0x80）+ 同 peer 的厂商私有 TLV。 */ - sealed interface GuangdongFc extends InfoBlock - permits GuangdongFc.Stack, - GuangdongFc.Auxiliary, - GuangdongFc.DcDc, - GuangdongFc.AirConditioner, - GuangdongFc.VehicleInfo, - GuangdongFc.DemoExtension, - GuangdongFc.VendorTlv { - - /** 0x30 燃料电池电堆数据（§7.2.3.4 表 13/14）。变长。 */ - record Stack(int stackCount, List stacks) implements GuangdongFc { - @Override public InfoBlockType type() { return InfoBlockType.GD_FC_STACK; } - - public record Item( - Integer engineWorkState, - Integer stackWaterOutletTempC, - Double hydrogenInletPressureKpa, - Double airInletPressureKpa, - Integer airInletTempC, - Integer maxCellVoltageId, - Integer minCellVoltageId, - Double maxCellVoltageV, - Double minCellVoltageV, - Double avgCellVoltageV, - Integer cellCount, - Integer frameCellStart, - Integer frameCellCount, - List frameCellVoltagesV - ) {} - } - - /** 0x31 辅助系统数据（§7.2.3.5 表 15/16）。变长。 */ - record Auxiliary(int subSystemCount, List subsystems) implements GuangdongFc { - @Override public InfoBlockType type() { return InfoBlockType.GD_FC_AUXILIARY; } - - public record Subsystem( - Double airCompressorMotorVoltageV, - Integer airCompressorPowerKw, - Double hydrogenPumpMotorVoltageV, - Integer hydrogenPumpPowerKw, - Double waterPumpVoltageV, - Double ptcVoltageV, - Integer ptcPowerKw, - Double lowVoltageBatteryVoltageV - ) {} - } - - /** 0x32 DC/DC（燃料电池系统）数据（§7.2.3.6 表 17）。固定 9 字节。 */ - record DcDc( - Double inputVoltageV, - Double inputCurrentA, - Double outputVoltageV, - Double outputCurrentA, - Integer controllerTempC - ) implements GuangdongFc { - @Override public InfoBlockType type() { return InfoBlockType.GD_FC_DCDC; } - } - - /** 0x33 空调数据（§7.2.3.7 表 18）。固定 5 字节。 */ - record AirConditioner( - Integer status, - Integer powerKw, - Double compressorInputVoltageV - ) implements GuangdongFc { - @Override public InfoBlockType type() { return InfoBlockType.GD_FC_AIR_CONDITIONER; } - } - - /** 0x34 车辆信息数据（§7.2.3.8 表 19）。固定 7 字节。 */ - record VehicleInfo( - Integer collisionAlarm, - Integer ambientTempC, - Double ambientPressureKpa, - Double hydrogenMassKg - ) implements GuangdongFc { - @Override public InfoBlockType type() { return InfoBlockType.GD_FC_VEHICLE_INFO; } - } - - /** 0x80 燃料电池汽车示范应用数据扩展（§7.2.3.14 表 27）。 */ - record DemoExtension( - Integer stackTempC, - Double airCompressorVoltageV, - Double airCompressorCurrentA, - Double hydrogenPumpVoltageV, - Double hydrogenPumpCurrentA - ) implements GuangdongFc { - @Override public InfoBlockType type() { return InfoBlockType.GD_FC_DEMO_EXTENSION; } - } - - /** - * 厂商私有 TLV 块兜底容器。当对端在广东规范之外又下发了一个未公开的 typeCode - * （目前线上观察到 {@code 0x83}），但该块明显是 TLV 格式（type(1) + length(2) + - * value(N)）时，本 record 保留原始字节直到对端提供正式协议文档为止。 - * - *

    关键在于对应的 {@code GdFcVendorTlvBlockParser} 严格按 length 字段消费字节， - * 因此 BodyParser 主循环在该块之后**仍能继续解析其它信息体**，不会被吞光。 - * - * @param typeCode 触发的 typeCode（保留以便诊断/分类） - * @param declaredLength TLV 的 length 字段值（payload 字节数） - * @param payload 原始 payload 字节，长度 = declaredLength - */ - record VendorTlv(int typeCode, int declaredLength, byte[] payload) implements GuangdongFc { - @Override public InfoBlockType type() { return InfoBlockType.GD_FC_VENDOR_TLV; } - } - } - - // ======================================================================== - // Raw / Unknown - // ======================================================================== - - /** - * 尚未实现 Parser 的原始信息体（含自定义 0x80~0xFE），保留字节透传到冷存。 - * - * @param typeCode 触发兜底时的真实信息体类型字节，便于诊断未知段或确认 parser 漂移落点 - * @param type 枚举形态，目前固定 {@link InfoBlockType#RAW}（保留字段为后续真正分类预留） - * @param bytes 自该未知 type 字节之后到 body 结尾的剩余原始字节 - */ - record Raw(int typeCode, InfoBlockType type, byte[] bytes) implements InfoBlock {} -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/InfoBlockType.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/InfoBlockType.java deleted file mode 100644 index 1fef0255..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/InfoBlockType.java +++ /dev/null @@ -1,63 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.model; - -/** - * 信息体语义类型（逻辑分类）。不与协议 typeCode 一一对应。 - * - *

    因为 2016 和 2025 的 typeCode 0x06/0x07/0x08/0x09 语义不同，本枚举按逻辑含义 - * 分类，而不是按 typeCode。真正的协议 typeCode 路由由 {@link com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser#typeCode()} - * 和 {@link com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParserRegistry} 负责。 - */ -public enum InfoBlockType { - /** 整车数据（表 10）。 */ - VEHICLE, - /** 驱动电机数据（表 15/16）。 */ - DRIVE_MOTOR, - /** 燃料电池数据（2016 版，带电压/电流/温度探针列表）。 */ - FUEL_CELL_V2016, - /** 燃料电池发动机及车载氢系统数据（2025 版，表 17）。 */ - FUEL_CELL_V2025, - /** 发动机数据（表 20）。 */ - ENGINE, - /** 车辆位置数据（2016 版，表 B.15）。 */ - POSITION_V2016, - /** 车辆位置数据（2025 版，表 21，带坐标系字段）。 */ - POSITION_V2025, - /** 动力蓄电池极值数据（2016 版，仅 2016 存在）。 */ - EXTREME_V2016, - /** 报警数据（2016 版）。 */ - ALARM_V2016, - /** 报警数据（2025 版，表 23，新增通用报警故障等级列表）。 */ - ALARM_V2025, - /** 可充电储能装置电压数据（2016 版）。 */ - VOLTAGE_V2016, - /** 可充电储能装置温度数据（2016 版）。 */ - TEMPERATURE_V2016, - /** 动力蓄电池最小并联单元电压数据（2025 版，表 11/12）。 */ - MIN_PARALLEL_VOLTAGE_V2025, - /** 动力蓄电池温度数据（2025 版，表 13/14）。 */ - BATTERY_TEMPERATURE_V2025, - /** 燃料电池电堆数据（2025 版，表 18/19）。 */ - FUEL_CELL_STACK, - /** 超级电容器数据（2025 版，表 25）。 */ - SUPER_CAPACITOR, - /** 超级电容器极值数据（2025 版，表 26）。 */ - SUPER_CAPACITOR_EXTREME, - /** 自定义数据 0x80~0xFE（表 27）。 */ - USER_DEFINED, - /** 广东燃料电池规范 0x30 燃料电池电堆数据（表 14）。 */ - GD_FC_STACK, - /** 广东燃料电池规范 0x31 辅助系统数据（表 16）。 */ - GD_FC_AUXILIARY, - /** 广东燃料电池规范 0x32 DC/DC 数据（表 17）。 */ - GD_FC_DCDC, - /** 广东燃料电池规范 0x33 空调数据（表 18）。 */ - GD_FC_AIR_CONDITIONER, - /** 广东燃料电池规范 0x34 车辆信息数据（表 19）。 */ - GD_FC_VEHICLE_INFO, - /** 广东燃料电池规范 0x80 燃料电池汽车示范应用数据扩展（表 27）。 */ - GD_FC_DEMO_EXTENSION, - /** 同 peer 的厂商私有 TLV 兜底（如 0x83），等待对端文档后升级为正式 parser。 */ - GD_FC_VENDOR_TLV, - /** 原始/未解析。 */ - RAW -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/ProtocolVersion.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/ProtocolVersion.java deleted file mode 100644 index 12a304b8..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/ProtocolVersion.java +++ /dev/null @@ -1,37 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.model; - -/** - * GB/T 32960.3 协议版本。 - * - *

    通过数据包起始符区分： - *

      - *
    • {@code ## (0x23 0x23)} → {@link #V2016} 对应 GB/T 32960.3-2016 - *
    • {@code $$ (0x24 0x24)} → {@link #V2025} 对应 GB/T 32960.3-2025 - *
    - * - *

    两版核心差异（GB/T 32960.3-2025 前言）： - *

      - *
    • 信息类型 0x06/0x07/0x08 语义重构： - * - * - * - * - * - * - *
      类型20162025
      0x06动力蓄电池极值报警数据
      0x07报警数据动力蓄电池最小并联单元电压
      0x08可充电储能装置电压动力蓄电池温度
      0x09可充电储能装置温度(已删除，合入 0x08)
      - *
    • 新增 0x30 燃料电池电堆 / 0x31 超级电容 / 0x32 超级电容极值 - *
    • 0x05 位置数据新增 1 字节坐标系（表 21） - *
    • 0x03 燃料电池数据字段重构：拆分为"燃料电池发动机及车载氢系统" - * + 独立 0x30"燃料电池电堆"（表 17/18/19） - *
    • 驱动电机 0x02 转速偏移由 {@code 20000} 改为 {@code 32000}，转矩由 2 字节扩展为 4 字节 - *
    • 实时上报数据单元尾部增加签名信息（以类型 {@code 0xFF} 标识，表 8） - *
    • 加密方式新增 {@code 0x04 SM2} / {@code 0x05 SM4} - *
    • 命令标识新增 {@code 0x07 心跳 / 0x08 校时 / 0x09 激活 / 0x0A 激活应答 / 0x0B 密钥交换} - *
    - */ -public enum ProtocolVersion { - /** GB/T 32960.3-2016。起始符 {@code 0x23 0x23 (##)}。 */ - V2016, - /** GB/T 32960.3-2025。起始符 {@code 0x24 0x24 ($$)}。 */ - V2025 -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/ResponseFlag.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/ResponseFlag.java deleted file mode 100644 index e58845a1..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/ResponseFlag.java +++ /dev/null @@ -1,46 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.model; - -/** - * 应答标志。GB/T 32960.3-2025 表 4 "应答标志定义"。 - * - *

    命令的主动发起方应答标志为 {@code 0xFE}（{@link #COMMAND}），表示此包为命令包； - * 当应答标志不是 {@code 0xFE} 时，此包表示为应答包。服务端发送应答时，应变更应答标志， - * 保留报文时间，删除其余报文内容，并重新计算校验位。 - */ -public enum ResponseFlag { - /** 0x01：成功 —— 接收到的信息正确。 */ - SUCCESS(0x01), - /** 0x02：其他错误 —— 其他收到的信息存在格式及内容错误。 */ - OTHER_ERROR(0x02), - /** 0x03：VIN 重复。 */ - VIN_DUPLICATED(0x03), - /** 0x04：VIN 不存在（2025 版新增）。 */ - VIN_NOT_EXIST(0x04), - /** 0x05：验签错误（2025 版新增）。 */ - SIGNATURE_ERROR(0x05), - /** 0x06：数据结构错误（2025 版新增）。 */ - STRUCTURE_ERROR(0x06), - /** 0x07：解密错误（2025 版新增）。 */ - DECRYPT_ERROR(0x07), - /** 0xFE：命令 —— 表示数据包为命令包，而非应答包。 */ - COMMAND(0xFE); - - private final int code; - - ResponseFlag(int code) { - this.code = code; - } - - public int code() { - return code; - } - - public boolean isCommand() { - return this == COMMAND; - } - - public static ResponseFlag of(int code) { - for (ResponseFlag r : values()) if (r.code == code) return r; - return OTHER_ERROR; - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/SignatureInfo.java b/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/SignatureInfo.java deleted file mode 100644 index 2b961ed0..00000000 --- a/modules/protocols/protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/model/SignatureInfo.java +++ /dev/null @@ -1,66 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.model; - -import java.nio.ByteBuffer; - -/** - * 实时信息的车端数字签名（GB/T 32960.3-2025 表 8）。 - * - *

    签名信息出现在实时上报数据单元的末尾，紧跟在信息体循环后，由 {@code 0xFF} - * 作为"签名数据开始标识"引出（见表 9）。签名范围：从数据采集时间的第一字节到签名 - * 信息前一个字节的数字签名。 - * - *

    表 8 字段： - * 

    - *   signType(1)    签名类型：0x01 SM2 / 0x02 RSA / 0x03 ECC，其他预留
- *   rLen(2 WORD)   签名 R 值长度
- *   r(rLen)        签名 R 值
- *   sLen(2 WORD)   签名 S 值长度
- *   s(sLen)        签名 S 值
- *
    - */ -public record SignatureInfo(SignType signType, byte[] r, byte[] s) { - - public enum SignType { - /** 0x01：SM2 国密签名。 */ - SM2(0x01), - /** 0x02：RSA 签名。 */ - RSA(0x02), - /** 0x03：ECC 签名。 */ - ECC(0x03), - /** 其他预留。 */ - UNKNOWN(0xFF); - - private final int code; - SignType(int code) { this.code = code; } - public int code() { return code; } - - public static SignType of(int code) { - for (SignType t : values()) if (t.code == code) return t; - return UNKNOWN; - } - } - - /** - * 尝试解析签名字节。非法数据时返回 null 并保留原始字节由冷存分析。 - * - * @param bytes 签名原始字节（不含前缀的 0xFF 签名开始标识） - */ - public static SignatureInfo tryParse(byte[] bytes) { - if (bytes == null || bytes.length < 5) return null; - ByteBuffer buf = ByteBuffer.wrap(bytes); - try { - SignType type = SignType.of(buf.get() & 0xFF); - int rLen = buf.getShort() & 0xFFFF; - if (rLen > buf.remaining() - 2) return null; - byte[] r = new byte[rLen]; - buf.get(r); - int sLen = buf.getShort() & 0xFFFF; - if (sLen > buf.remaining()) return null; - byte[] s = new byte[sLen]; - buf.get(s); - return new SignatureInfo(type, r, s); - } catch (Exception e) { - return null; - } - } -} diff --git a/modules/protocols/protocol-gb32960/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/protocols/protocol-gb32960/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index 2a10e523..00000000 --- a/modules/protocols/protocol-gb32960/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.protocol.gb32960.config.Gb32960AutoConfiguration diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParserIsolationTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParserIsolationTest.java deleted file mode 100644 index 4e4ec2d8..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParserIsolationTest.java +++ /dev/null @@ -1,172 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.api.spi.DecodeException; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.PositionV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlockType; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayOutputStream; -import java.nio.ByteBuffer; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -/** - * 验证 {@link Gb32960BodyParser} 单块异常隔离行为。 - * - *

    三个隔离场景： - *

      - *
    1. 固定长度块 parser 抛异常 → 失败块兜 Raw（按 fixedLen 截取）+ 后续块继续解析 - *
    2. 固定长度块剩余字节不足（截断帧） → 兜 Raw(剩余) + break 循环 - *
    3. 变长块 parser 抛异常 → 兜 Raw(从失败块起剩余全部) + break 循环 - *
    - * - *

    外加一个严格模式回退测试：{@code lenientBlockFailure=false} 时异常应抛 DecodeException。 - */ -class Gb32960BodyParserIsolationTest { - - /** 模拟一个固定长度的 Position parser，声明 fixedLength=9，parse 时主动抛异常。 */ - private static final InfoBlockParser EXPLODING_FIXED_LEN_POSITION = new InfoBlockParser() { - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x05; } - @Override public int fixedLength() { return 9; } - @Override public InfoBlock parse(ByteBuffer buffer) { - buffer.get(); - buffer.get(); - throw new DecodeException("simulated parser failure in Position"); - } - }; - - /** 模拟一个变长 Alarm parser（fixedLength=-1），消费若干字节后抛。 */ - private static final InfoBlockParser EXPLODING_VAR_LEN_ALARM = new InfoBlockParser() { - @Override public ProtocolVersion version() { return ProtocolVersion.V2016; } - @Override public int typeCode() { return 0x07; } - @Override public int fixedLength() { return -1; } - @Override public InfoBlock parse(ByteBuffer buffer) { - buffer.get(); - throw new DecodeException("simulated parser failure in Alarm list-length read"); - } - }; - - @Test - void fixedLengthBlockFailure_isIsolated_subsequentBlocksStillParsed() { - InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of( - new VehicleV2016BlockParser(), - EXPLODING_FIXED_LEN_POSITION)); - - Gb32960BodyParser parser = new Gb32960BodyParser(registry); - - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeValidVehicle(os); // 1 + 20 = 21B - writePositionTypeAnd9ByteBody(os); // 1 + 9 = 10B（parser 会爆） - writeValidVehicle(os); // 1 + 20 = 21B - ByteBuffer body = ByteBuffer.wrap(os.toByteArray()); - - var result = parser.parse(ProtocolVersion.V2016, body); - - assertThat(result.blocks()).hasSize(3); - assertThat(result.blocks().get(0)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class); - assertThat(result.blocks().get(1)).isInstanceOfSatisfying(InfoBlock.Raw.class, raw -> { - assertThat(raw.typeCode()).isEqualTo(0x05); - assertThat(raw.type()).isEqualTo(InfoBlockType.RAW); - assertThat(raw.bytes()).hasSize(9); - }); - assertThat(result.blocks().get(2)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class); - assertThat(body.hasRemaining()).isFalse(); - } - - @Test - void truncatedFixedLengthBlock_isWrappedAsRaw_loopTerminates() { - InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of( - new VehicleV2016BlockParser(), - new PositionV2016BlockParser())); - - Gb32960BodyParser parser = new Gb32960BodyParser(registry); - - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeValidVehicle(os); - os.write(0x05); - os.write(0); - os.write(0); - os.write(0); - ByteBuffer body = ByteBuffer.wrap(os.toByteArray()); - - var result = parser.parse(ProtocolVersion.V2016, body); - - assertThat(result.blocks()).hasSize(2); - assertThat(result.blocks().get(0)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class); - assertThat(result.blocks().get(1)).isInstanceOfSatisfying(InfoBlock.Raw.class, raw -> { - assertThat(raw.typeCode()).isEqualTo(0x05); - assertThat(raw.bytes()).hasSize(3); - }); - } - - @Test - void variableLengthBlockFailure_swallowsRemainderAsRaw_loopBreaks() { - InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of( - new VehicleV2016BlockParser(), - EXPLODING_VAR_LEN_ALARM)); - - Gb32960BodyParser parser = new Gb32960BodyParser(registry); - - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeValidVehicle(os); - os.write(0x07); - for (int i = 0; i < 10; i++) os.write(0xAA); - writeValidVehicle(os); - ByteBuffer body = ByteBuffer.wrap(os.toByteArray()); - - var result = parser.parse(ProtocolVersion.V2016, body); - - assertThat(result.blocks()).hasSize(2); - assertThat(result.blocks().get(0)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class); - assertThat(result.blocks().get(1)).isInstanceOfSatisfying(InfoBlock.Raw.class, raw -> { - assertThat(raw.typeCode()).isEqualTo(0x07); - assertThat(raw.bytes()).hasSize(31); - }); - } - - @Test - void strictMode_throwsOnAnyBlockFailure() { - InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of( - new VehicleV2016BlockParser(), - EXPLODING_FIXED_LEN_POSITION)); - - Gb32960BodyParser parser = new Gb32960BodyParser(registry); - parser.setLenientBlockFailure(false); - - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeValidVehicle(os); - writePositionTypeAnd9ByteBody(os); - ByteBuffer body = ByteBuffer.wrap(os.toByteArray()); - - assertThatThrownBy(() -> parser.parse(ProtocolVersion.V2016, body)) - .isInstanceOf(DecodeException.class); - } - - private static void writeValidVehicle(ByteArrayOutputStream os) { - os.write(0x01); - os.write(0x01); - os.write(0x01); - os.write(0x01); - os.write(0); os.write(0); - os.write(0); os.write(0); os.write(0); os.write(0); - os.write(0); os.write(0); - os.write(0); os.write(0); - os.write(50); - os.write(0x01); - os.write(0); - os.write(0); os.write(0); - os.write(0); - os.write(0); - } - - private static void writePositionTypeAnd9ByteBody(ByteArrayOutputStream os) { - os.write(0x05); - for (int i = 0; i < 9; i++) os.write(0); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960DecoderGoldenTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960DecoderGoldenTest.java deleted file mode 100644 index 91b92f66..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960DecoderGoldenTest.java +++ /dev/null @@ -1,106 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.AlarmV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.DriveMotorV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.EngineV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.ExtremeValueV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.FuelCellV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.PositionV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.TemperatureV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VoltageV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import org.junit.jupiter.api.DynamicTest; -import org.junit.jupiter.api.TestFactory; - -import java.io.IOException; -import java.net.URISyntaxException; -import java.nio.ByteBuffer; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.Paths; -import java.util.Collection; -import java.util.List; -import java.util.stream.Collectors; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -/** - * 黄金样本集回放测试：遍历 {@code src/test/resources/samples/gb32960/*.hex}， - * 逐帧解码并断言命令类型来自 {@link com.lingniu.ingest.protocol.gb32960.model.CommandType} 有效值。 - * - *

    样本文件目前为空，该测试会优雅地跳过。加入样本后会自动变为 N 条动态用例。 - */ -class Gb32960DecoderGoldenTest { - - private final Gb32960MessageDecoder decoder = new Gb32960MessageDecoder( - new Gb32960BodyParser(new InfoBlockParserRegistry(List.of( - new VehicleV2016BlockParser(), - new PositionV2016BlockParser(), - new DriveMotorV2016BlockParser(), - new FuelCellV2016BlockParser(), - new EngineV2016BlockParser(), - new ExtremeValueV2016BlockParser(), - new AlarmV2016BlockParser(), - new VoltageV2016BlockParser(), - new TemperatureV2016BlockParser())))); - - @TestFactory - Collection replaySamples() throws URISyntaxException, IOException { - var url = getClass().getClassLoader().getResource("samples/gb32960"); - if (url == null) return List.of(); - Path dir = Paths.get(url.toURI()); - try (Stream s = Files.list(dir)) { - return s.filter(p -> p.toString().endsWith(".hex")) - .sorted() - .map(this::toTest) - .collect(Collectors.toList()); - } - } - - private DynamicTest toTest(Path sample) { - return DynamicTest.dynamicTest(sample.getFileName().toString(), () -> { - byte[] frame = readHex(sample); - Gb32960Message msg = decoder.decode(ByteBuffer.wrap(frame)); - assertThat(msg.header().vin()).hasSize(17); - assertThat(msg.header().command()).isNotNull(); - // 749 字节真实生产帧（realtime_002/003/010/200）：peer 在 V2016 帧里下发了 - // 0x30/0x31/0x32 等 typeCode，但这些在 GB/T 32960.3-2016 附录 B 表 B.3 是"预留"区， - // 没有任何字段定义。因此**期望产生 Raw 兜底块**——这是符合规范的正确行为。 - // 若未来对端切换到 V2025 (2424 起始) 或者迁移到 0x80~0xFE 用户自定义区，可再调整断言。 - String name = sample.getFileName().toString(); - boolean isRealProductionFrame = frame.length == 774; - if (isRealProductionFrame) { - assertThat(msg.findBlock(InfoBlock.Raw.class)) - .as("[%s] 749字节真实生产帧应有 Raw 兜底块（peer 越界使用 0x30+ 预留 typeCode）", name) - .isPresent(); - } - // 总电流必须落在协议规定的 -1000~+1000 A 区间，防回归到 -3000 偏移 - msg.findBlock(InfoBlock.Gb32960V2016.Vehicle.class).ifPresent(v -> { - if (v.totalCurrentA() != null) { - assertThat(v.totalCurrentA()) - .as("[%s] vehicle.totalCurrentA 越界，疑似偏移常量回归", name) - .isBetween(-1000.0, 1000.0); - } - }); - }); - } - - private static byte[] readHex(Path path) throws IOException { - StringBuilder sb = new StringBuilder(); - for (String line : Files.readAllLines(path)) { - String trimmed = line.trim(); - if (trimmed.isEmpty() || trimmed.startsWith("#")) continue; - sb.append(trimmed); - } - String hex = sb.toString(); - int len = hex.length() / 2; - byte[] out = new byte[len]; - for (int i = 0; i < len; i++) { - out[i] = (byte) Integer.parseInt(hex.substring(i * 2, i * 2 + 2), 16); - } - return out; - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960DecoderTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960DecoderTest.java deleted file mode 100644 index 37609f4a..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960DecoderTest.java +++ /dev/null @@ -1,158 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.PositionV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.model.CommandBody; -import com.lingniu.ingest.protocol.gb32960.model.CommandType; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayOutputStream; -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -/** - * 构造一条合成 32960 实时上报帧，跑通 Frame 解码 → Body 解析 → InfoBlock 的全链路。 - * - *

    样本无需外部文件：本测试既验证解码正确性，也作为 {@code samples/} 黄金样本的期望值参考实现。 - */ -public class Gb32960DecoderTest { - - @Test - void decodesSyntheticRealtimeReport() { - byte[] frame = buildRealtimeFrame("LTEST000000000001"); - - InfoBlockParserRegistry registry = new InfoBlockParserRegistry( - List.of(new VehicleV2016BlockParser(), new PositionV2016BlockParser())); - Gb32960BodyParser bodyParser = new Gb32960BodyParser(registry); - Gb32960MessageDecoder decoder = new Gb32960MessageDecoder(bodyParser); - - Gb32960Message msg = decoder.decode(ByteBuffer.wrap(frame)); - - assertThat(msg.header().command()).isEqualTo(CommandType.REALTIME_REPORT); - assertThat(msg.header().vin()).isEqualTo("LTEST000000000001"); - assertThat(msg.header().eventTime()).isNotNull(); - - InfoBlock.Gb32960V2016.Vehicle v = msg.findBlock(InfoBlock.Gb32960V2016.Vehicle.class).orElseThrow(); - assertThat(v.socPercent()).isEqualTo(70); - assertThat(v.speedKmh()).isEqualTo(52.3, org.assertj.core.data.Offset.offset(0.01)); - assertThat(v.gearRaw()).isEqualTo(0x0F); - - InfoBlock.Gb32960V2016.Position p = msg.findBlock(InfoBlock.Gb32960V2016.Position.class).orElseThrow(); - assertThat(p.longitude()).isEqualTo(116.397128, org.assertj.core.data.Offset.offset(0.000001)); - assertThat(p.latitude()).isEqualTo(39.916527, org.assertj.core.data.Offset.offset(0.000001)); - } - - @Test - void decodesPlatformLoginWithMd5HexPasswordLongerThanDeclaredGbField() { - byte[] frame = buildPlatformLoginFrameWithExtendedPassword( - "Hyundai", - "f2e3445d7cda409fb4f278f6fb890734"); - - Gb32960MessageDecoder decoder = new Gb32960MessageDecoder(new Gb32960BodyParser( - new InfoBlockParserRegistry(List.of()))); - - Gb32960Message msg = decoder.decode(ByteBuffer.wrap(frame)); - - assertThat(msg.header().command()).isEqualTo(CommandType.PLATFORM_LOGIN); - assertThat(msg.commandBody()).isInstanceOf(CommandBody.PlatformLogin.class); - CommandBody.PlatformLogin login = (CommandBody.PlatformLogin) msg.commandBody(); - assertThat(login.username()).isEqualTo("Hyundai"); - assertThat(login.password()).isEqualTo("f2e3445d7cda409fb4f278f6fb890734"); - } - - /** - * 构造一条合法的 32960 实时上报帧： - * header + 6B 时间戳(2024-01-02 03:04:05) + 0x01 整车 + 0x05 位置 + BCC - */ - public static byte[] buildRealtimeFrame(String vin) { - ByteArrayOutputStream body = new ByteArrayOutputStream(); - // 时间戳 - body.write(24); body.write(1); body.write(2); body.write(3); body.write(4); body.write(5); - - // 0x01 整车 20 字节 - body.write(0x01); - body.write(0x01); // vehicle state = 启动 - body.write(0x03); // charging = 未充电 - body.write(0x01); // 纯电 - writeU16(body, 523); // 车速 52.3 km/h - writeU32(body, 1234567); // 里程 123456.7 km - writeU16(body, 6000); // 600.0 V - writeU16(body, 1100); // 110.0 A (110 - (1000-1000) = 10) 实际计算：1100*0.1 - 1000 = -890；此处只为结构测试 - body.write(70); // SOC - body.write(0x01); - body.write(0x0F); - writeU16(body, 500); - body.write(30); - body.write(0); - - // 0x05 位置 9 字节 - body.write(0x05); - body.write(0x00); // 有效 + 北纬 + 东经 - writeU32(body, 116_397_128L); - writeU32(body, 39_916_527L); - - byte[] bodyBytes = body.toByteArray(); - - ByteArrayOutputStream out = new ByteArrayOutputStream(); - out.write(0x23); out.write(0x23); - out.write(0x02); // 实时上报 - out.write(0xFE); // 应答 / 命令 - byte[] vinBytes = vin.getBytes(StandardCharsets.US_ASCII); - out.write(vinBytes, 0, 17); - out.write(0x01); // 不加密 - writeU16(out, bodyBytes.length); - out.write(bodyBytes, 0, bodyBytes.length); - - byte[] almost = out.toByteArray(); - byte bcc = BccChecksum.compute(almost, 2, almost.length - 2); - out.write(bcc & 0xFF); - return out.toByteArray(); - } - - private static void writeU16(ByteArrayOutputStream os, int v) { - os.write((v >> 8) & 0xFF); - os.write(v & 0xFF); - } - - private static void writeU32(ByteArrayOutputStream os, long v) { - os.write((int) ((v >> 24) & 0xFF)); - os.write((int) ((v >> 16) & 0xFF)); - os.write((int) ((v >> 8) & 0xFF)); - os.write((int) (v & 0xFF)); - } - - private static byte[] buildPlatformLoginFrameWithExtendedPassword(String username, String password) { - ByteArrayOutputStream body = new ByteArrayOutputStream(); - body.write(26); body.write(6); body.write(22); body.write(20); body.write(40); body.write(45); - writeU16(body, 1); - writePaddedAscii(body, username, 12); - writePaddedAscii(body, password, password.length()); - body.write(0x01); - - byte[] bodyBytes = body.toByteArray(); - ByteArrayOutputStream out = new ByteArrayOutputStream(); - out.write(0x23); out.write(0x23); - out.write(0x05); - out.write(0xFE); - for (int i = 0; i < 17; i++) out.write(0); - out.write(0x01); - writeU16(out, 41); - out.write(bodyBytes, 0, bodyBytes.length); - byte[] almost = out.toByteArray(); - out.write(BccChecksum.compute(almost, 2, almost.length - 2) & 0xFF); - return out.toByteArray(); - } - - private static void writePaddedAscii(ByteArrayOutputStream os, String value, int len) { - byte[] bytes = value.getBytes(StandardCharsets.US_ASCII); - int copy = Math.min(bytes.length, len); - os.write(bytes, 0, copy); - for (int i = copy; i < len; i++) os.write(0); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FrameDecoderTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FrameDecoderTest.java deleted file mode 100644 index 70a3c0b0..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FrameDecoderTest.java +++ /dev/null @@ -1,59 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.codec.BccChecksum; -import io.netty.channel.embedded.EmbeddedChannel; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayOutputStream; -import java.nio.charset.StandardCharsets; - -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960FrameDecoderTest { - - @Test - void keepsPlatformLoginWithMd5HexPasswordAsOneFrame() { - byte[] frame = buildPlatformLoginFrame("Hyundai", "f2e3445d7cda409fb4f278f6fb890734"); - EmbeddedChannel channel = new EmbeddedChannel(new Gb32960FrameDecoder()); - - assertThat(channel.writeInbound(frame)).isTrue(); - - byte[] decoded = channel.readInbound(); - assertThat(decoded).isEqualTo(frame); - assertThat((Object) channel.readInbound()).isNull(); - } - - private static byte[] buildPlatformLoginFrame(String username, String password) { - ByteArrayOutputStream body = new ByteArrayOutputStream(); - body.write(26); body.write(6); body.write(22); body.write(20); body.write(40); body.write(45); - writeU16(body, 1); - writePaddedAscii(body, username, 12); - writePaddedAscii(body, password, password.length()); - body.write(0x01); - - ByteArrayOutputStream out = new ByteArrayOutputStream(); - out.write(0x23); out.write(0x23); - out.write(0x05); - out.write(0xFE); - for (int i = 0; i < 17; i++) out.write(0); - out.write(0x01); - writeU16(out, 41); - byte[] bodyBytes = body.toByteArray(); - out.write(bodyBytes, 0, bodyBytes.length); - byte[] almost = out.toByteArray(); - out.write(BccChecksum.compute(almost, 2, almost.length - 2) & 0xFF); - return out.toByteArray(); - } - - private static void writeU16(ByteArrayOutputStream os, int v) { - os.write((v >> 8) & 0xFF); - os.write(v & 0xFF); - } - - private static void writePaddedAscii(ByteArrayOutputStream os, String value, int len) { - byte[] bytes = value.getBytes(StandardCharsets.US_ASCII); - int copy = Math.min(bytes.length, len); - os.write(bytes, 0, copy); - for (int i = copy; i < len; i++) os.write(0); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FullBlocksTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FullBlocksTest.java deleted file mode 100644 index 1620957b..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960FullBlocksTest.java +++ /dev/null @@ -1,152 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.AlarmV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.DriveMotorV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.PositionV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.TemperatureV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VoltageV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayOutputStream; -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -/** - * 验证新增的变长信息体 Parser（0x02 驱动电机 / 0x07 报警 / 0x08 电压 / 0x09 温度） - * 与主解码器协同工作。 - */ -class Gb32960FullBlocksTest { - - private final Gb32960MessageDecoder decoder = new Gb32960MessageDecoder( - new Gb32960BodyParser(new InfoBlockParserRegistry(List.of( - new VehicleV2016BlockParser(), - new PositionV2016BlockParser(), - new DriveMotorV2016BlockParser(), - new AlarmV2016BlockParser(), - new VoltageV2016BlockParser(), - new TemperatureV2016BlockParser())))); - - @Test - void parsesDriveMotorBlock() { - byte[] frame = buildFrame(os -> { - os.write(0x02); // drive motor - os.write(1); // 1 motor - os.write(1); // serial - os.write(0x01); // state - os.write(100); // controllerTemp = 60 - writeU16(os, 23_000); // rpm = 3000 - writeU16(os, 21_000); // torque raw, actual = 100 Nm - os.write(90); // motorTemp = 50 - writeU16(os, 5400); // voltage 540.0V - writeU16(os, 11000); // current raw → 100 A - }); - Gb32960Message msg = decoder.decode(ByteBuffer.wrap(frame)); - InfoBlock.Gb32960V2016.DriveMotor dm = msg.findBlock(InfoBlock.Gb32960V2016.DriveMotor.class).orElseThrow(); - assertThat(dm.motors()).hasSize(1); - var m = dm.motors().get(0); - assertThat(m.rpm()).isEqualTo(3000); - assertThat(m.torqueNm()).isEqualTo(100.0); - assertThat(m.controllerInputVoltageV()).isEqualTo(540.0); - } - - @Test - void parsesAlarmBlock() { - byte[] frame = buildFrame(os -> { - os.write(0x07); // alarm - os.write(2); // max level - writeU32(os, 0x0000_0003L);// general flag - os.write(1); writeU32(os, 0xDEAD_BEEFL); // battery faults 1 - os.write(0); // motor faults 0 - os.write(0); // engine faults 0 - os.write(1); writeU32(os, 0xCAFE_BABEL); // other faults 1 - }); - Gb32960Message msg = decoder.decode(ByteBuffer.wrap(frame)); - InfoBlock.Gb32960V2016.Alarm a = msg.findBlock(InfoBlock.Gb32960V2016.Alarm.class).orElseThrow(); - assertThat(a.maxLevel()).isEqualTo(2); - assertThat(a.batteryFaults()).hasSize(1); - assertThat(a.otherFaults()).hasSize(1); - } - - @Test - void parsesVoltageBlock() { - byte[] frame = buildFrame(os -> { - os.write(0x08); // voltage - os.write(1); // 1 subsystem - os.write(1); // battery index - writeU16(os, 3800); // 380.0 V - writeU16(os, 11_000); // current raw - writeU16(os, 96); // total cells - writeU16(os, 1); // start - os.write(3); // frame cells - writeU16(os, 3500); // 3.5V - writeU16(os, 3600); // 3.6V - writeU16(os, 3300); // 3.3V - }); - Gb32960Message msg = decoder.decode(ByteBuffer.wrap(frame)); - InfoBlock.Gb32960V2016.Voltage v = msg.findBlock(InfoBlock.Gb32960V2016.Voltage.class).orElseThrow(); - assertThat(v.subSystemCount()).isEqualTo(1); - assertThat(v.maxCellVoltageV()).isEqualTo(3.6, org.assertj.core.data.Offset.offset(0.001)); - assertThat(v.minCellVoltageV()).isEqualTo(3.3, org.assertj.core.data.Offset.offset(0.001)); - } - - @Test - void parsesTemperatureBlock() { - byte[] frame = buildFrame(os -> { - os.write(0x09); // temperature - os.write(1); // subsystems - os.write(1); // battery index - writeU16(os, 4); // probes - os.write(60); // actual 20 - os.write(65); // actual 25 - os.write(70); // actual 30 - os.write(55); // actual 15 - }); - Gb32960Message msg = decoder.decode(ByteBuffer.wrap(frame)); - InfoBlock.Gb32960V2016.Temperature t = msg.findBlock(InfoBlock.Gb32960V2016.Temperature.class).orElseThrow(); - assertThat(t.maxTempC()).isEqualTo(30); - assertThat(t.minTempC()).isEqualTo(15); - } - - // ===== frame builder helpers ===== - - private static byte[] buildFrame(java.util.function.Consumer bodyWriter) { - ByteArrayOutputStream body = new ByteArrayOutputStream(); - // 时间戳 - body.write(24); body.write(1); body.write(2); body.write(3); body.write(4); body.write(5); - bodyWriter.accept(body); - - byte[] bodyBytes = body.toByteArray(); - ByteArrayOutputStream out = new ByteArrayOutputStream(); - out.write(0x23); out.write(0x23); - out.write(0x02); // cmd: realtime - out.write(0xFE); // response flag - byte[] vin = "LTEST000000000010".getBytes(StandardCharsets.US_ASCII); - out.write(vin, 0, 17); - out.write(0x01); // not encrypted - writeU16(out, bodyBytes.length); - out.write(bodyBytes, 0, bodyBytes.length); - byte[] almost = out.toByteArray(); - byte bcc = BccChecksum.compute(almost, 2, almost.length - 2); - out.write(bcc & 0xFF); - return out.toByteArray(); - } - - private static void writeU16(ByteArrayOutputStream os, int v) { - os.write((v >> 8) & 0xFF); - os.write(v & 0xFF); - } - - private static void writeU32(ByteArrayOutputStream os, long v) { - os.write((int) ((v >> 24) & 0xFF)); - os.write((int) ((v >> 16) & 0xFF)); - os.write((int) ((v >> 8) & 0xFF)); - os.write((int) (v & 0xFF)); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/GuangdongFcEndToEndTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/GuangdongFcEndToEndTest.java deleted file mode 100644 index 2a3a959e..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/GuangdongFcEndToEndTest.java +++ /dev/null @@ -1,296 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec; - -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.AlarmV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.DriveMotorV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.PositionV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.TemperatureV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VoltageV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcAirConditionerBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcAuxiliaryBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcDcDcBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcDemoExtensionBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcStackBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcVehicleInfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.profile.Gb32960ProfileRegistry; -import com.lingniu.ingest.protocol.gb32960.codec.profile.RuleBasedVendorExtensionSelector; -import com.lingniu.ingest.protocol.gb32960.codec.profile.VendorExtensionCatalog; -import com.lingniu.ingest.protocol.gb32960.config.Gb32960Properties; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960ChannelHandler; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayOutputStream; -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.util.List; -import java.util.Set; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.within; - -/** - * 端到端集成测试：构造一帧带广东燃料电池规范扩展块的 V2016 帧，验证 - * vendor profile selector 路由 + GuangdongFc 系列 parser 全链路解码。 - * - *

    覆盖： - *

      - *
    • 9 个 GB/T 32960 标准块（0x01~0x09）原生解析 - *
    • 0x30 GuangdongFc.Stack（含 1 个电堆、4 个单体电压样本） - *
    • 0x32 GuangdongFc.DcDc - *
    • 0x33 GuangdongFc.AirConditioner - *
    • 0x34 GuangdongFc.VehicleInfo - *
    • 0x80 GuangdongFc.DemoExtension - *
    • VendorExtensionSelector 按 platformAccount=lingniu 命中 - *
    - * - *

    0x31 Auxiliary 没有放进帧里（变长且字段相对琐碎），单独由 unit test 覆盖。 - */ -class GuangdongFcEndToEndTest { - - private final Gb32960MessageDecoder decoder = buildDecoder(); - - @Test - void parsesRealtimeFrameWithGuangdongFcExtensions() { - byte[] frame = buildFrame(); - // 用 platformAccount=lingniu 触发 vendor profile - Gb32960Message msg = decoder.decode(ByteBuffer.wrap(frame), "lingniu"); - - // ---- 标准块 ---- - var v = msg.findBlock(InfoBlock.Gb32960V2016.Vehicle.class).orElseThrow(); - assertThat(v.socPercent()).isEqualTo(85); - - var pos = msg.findBlock(InfoBlock.Gb32960V2016.Position.class).orElseThrow(); - assertThat(pos.longitude()).isCloseTo(120.848158, within(1e-6)); - - // ---- vendor 块：0x30 Stack ---- - var stack = msg.findBlock(InfoBlock.GuangdongFc.Stack.class).orElseThrow(); - assertThat(stack.stackCount()).isEqualTo(1); - var s0 = stack.stacks().get(0); - assertThat(s0.engineWorkState()).isEqualTo(1); - assertThat(s0.stackWaterOutletTempC()).isEqualTo(50); // 90-40 - assertThat(s0.maxCellVoltageV()).isCloseTo(0.745, within(1e-6)); // 745 mV - assertThat(s0.minCellVoltageV()).isCloseTo(0.730, within(1e-6)); - assertThat(s0.avgCellVoltageV()).isCloseTo(0.738, within(1e-6)); - assertThat(s0.cellCount()).isEqualTo(400); - assertThat(s0.frameCellCount()).isEqualTo(4); - assertThat(s0.frameCellVoltagesV()).hasSize(4) - .allSatisfy(c -> assertThat(c).isBetween(0.7, 0.8)); - - // ---- vendor 块：0x32 DcDc ---- - var dcdc = msg.findBlock(InfoBlock.GuangdongFc.DcDc.class).orElseThrow(); - assertThat(dcdc.inputVoltageV()).isCloseTo(300.0, within(1e-6)); - assertThat(dcdc.outputCurrentA()).isCloseTo(110.0, within(1e-6)); - - // ---- vendor 块：0x33 AirConditioner ---- - var ac = msg.findBlock(InfoBlock.GuangdongFc.AirConditioner.class).orElseThrow(); - assertThat(ac.status()).isEqualTo(1); - assertThat(ac.powerKw()).isEqualTo(15); - - // ---- vendor 块：0x34 VehicleInfo ---- - var info = msg.findBlock(InfoBlock.GuangdongFc.VehicleInfo.class).orElseThrow(); - assertThat(info.collisionAlarm()).isEqualTo(0); - assertThat(info.ambientTempC()).isEqualTo(30); - assertThat(info.hydrogenMassKg()).isCloseTo(8.0, within(1e-6)); - - // ---- vendor 块：0x80 DemoExtension ---- - var demo = msg.findBlock(InfoBlock.GuangdongFc.DemoExtension.class).orElseThrow(); - assertThat(demo.stackTempC()).isEqualTo(50); - assertThat(demo.airCompressorVoltageV()).isCloseTo(372.0, within(1e-6)); - - // 不应有 Raw 兜底：所有块都被结构化解析 - assertThat(msg.findBlock(InfoBlock.Raw.class)).isEmpty(); - } - - @Test - void unmatchedAccountFallsBackToDefaultAndProducesRawForVendorBlocks() { - byte[] frame = buildFrame(); - // 不传 account → selector 返回 null → default profile（无 vendor parser） - Gb32960Message msg = decoder.decode(ByteBuffer.wrap(frame), null); - - // 标准块仍能解析 - assertThat(msg.findBlock(InfoBlock.Gb32960V2016.Vehicle.class)).isPresent(); - // 但 0x30 应触发 unknown → Raw 兜底 - assertThat(msg.findBlock(InfoBlock.Raw.class)).isPresent(); - assertThat(msg.findBlock(InfoBlock.GuangdongFc.Stack.class)).isEmpty(); - } - - // ==================================================================== - // 帧构造：标准块 0x01/0x05/0x07/0x08/0x09 + vendor 块 0x30/0x32/0x33/0x34/0x80 - // 不带 0x02/0x04/0x06/0x31，纯粹为了控制帧长和测试焦点。 - // ==================================================================== - private static byte[] buildFrame() { - ByteArrayOutputStream body = new ByteArrayOutputStream(); - // 时间戳 6B - body.write(24); body.write(4); body.write(15); body.write(11); body.write(0); body.write(0); - - // 0x01 整车数据 20B - body.write(0x01); - body.write(0x01); // vehicleState 启动 - body.write(0x03); // chargingState 未充电 - body.write(0x02); // runningMode 混动 - writeU16(body, 0); // 速度 - writeU32(body, 100_000L); // 累计里程 = 10000 km - writeU16(body, 5605); // totalVoltage 560.5 V - writeU16(body, 10000); // totalCurrent 0 A (after -1000 offset) - body.write(85); // SOC - body.write(0x01); // DC-DC 工作 - body.write(0x00); // 挡位 - writeU16(body, 10000); // 绝缘电阻 - body.write(0x00); // accelPedal - body.write(0x00); // brakePedal - - // 0x05 位置数据 9B - body.write(0x05); - body.write(0x00); // 状态：有效定位 - writeU32(body, 120_848_158L); // 经度 120.848158 - writeU32(body, 31_497_236L); // 纬度 31.497236 - - // 0x07 报警数据 10B：level=0 + flag=0 + 4 个空 fault list - body.write(0x07); - body.write(0x00); // maxLevel - writeU32(body, 0); // generalFlag - body.write(0x00); // batteryFaultCount - body.write(0x00); // motorFaultCount - body.write(0x00); // engineFaultCount - body.write(0x00); // otherFaultCount - - // 0x08 储能电压：1 sub × 0 cell（保持简洁） - body.write(0x08); - body.write(0x01); // subCount - body.write(0x01); // batteryIndex - writeU16(body, 5605); // voltage - writeU16(body, 10000); // current - writeU16(body, 0); // cellCount - writeU16(body, 1); // frameCellStart - body.write(0x00); // frameCellCount = 0 - - // 0x09 储能温度：1 sub × 0 probe - body.write(0x09); - body.write(0x01); // subCount - body.write(0x01); // batteryIndex - writeU16(body, 0); // probeCount = 0 - - // 0x30 GuangdongFc.Stack：1 stack + 4 单体电压 - body.write(0x30); - body.write(0x01); // stackCount - body.write(0x01); // engineWorkState 打开 - body.write(90); // 水温 90-40 = 50°C - writeU16(body, 1850); // h2 入口压力 raw=1850 → 85 kPa - writeU16(body, 1500); // air 入口压力 raw=1500 → 50 kPa - body.write(70); // air 入口温度 70-40 = 30°C - writeU16(body, 12); // maxCellId - writeU16(body, 88); // minCellId - writeU16(body, 745); // maxCellV 745 mV - writeU16(body, 730); // minCellV 730 mV - writeU16(body, 738); // avgCellV 738 mV - writeU16(body, 400); // cellCount - writeU16(body, 1); // frameCellStart - body.write(0x04); // frameCellCount = 4 - writeU16(body, 745); - writeU16(body, 740); - writeU16(body, 735); - writeU16(body, 730); - - // 0x32 GuangdongFc.DcDc 9B - body.write(0x32); - writeU16(body, 3000); // inputV 300.0 - writeU16(body, 1200); // inputC 120.0 - writeU16(body, 5600); // outputV 560.0 - writeU16(body, 1100); // outputC 110.0 - body.write(105); // ctrlTemp 105-40 = 65°C - - // 0x33 GuangdongFc.AirConditioner 5B - body.write(0x33); - body.write(0x01); // status 启动 - writeU16(body, 20); // power 20-5 = 15 kw - writeU16(body, 5400); // compressorInputV 540.0 - - // 0x34 GuangdongFc.VehicleInfo 7B - body.write(0x34); - body.write(0x00); // collision 无 - writeU16(body, 70); // ambientTemp 70-40 = 30°C - writeU16(body, 1100); // ambientPressure raw=1100 → 10.0 kPa - writeU16(body, 80); // h2Mass 80*0.1 = 8.0 kg - - // 0x80 GuangdongFc.DemoExtension 12B（含 2B 内层 length=9） - body.write(0x80); - writeU16(body, 9); // declared length - body.write(90); // stackTemp 50°C - writeU16(body, 3720); // ac voltage 372.0 - writeU16(body, 10000); // ac current 0 A - writeU16(body, 13); // h2 pump voltage 1.3 - writeU16(body, 10000); // h2 pump current 0 A - - byte[] bodyBytes = body.toByteArray(); - - // 帧封装 - ByteArrayOutputStream out = new ByteArrayOutputStream(); - out.write(0x23); out.write(0x23); - out.write(0x02); // cmd realtime - out.write(0xFE); // 命令包 - byte[] vin = "LNVFC000000000001".getBytes(StandardCharsets.US_ASCII); - out.write(vin, 0, 17); - out.write(0x01); // 不加密 - writeU16(out, bodyBytes.length); - out.write(bodyBytes, 0, bodyBytes.length); - byte[] almost = out.toByteArray(); - byte bcc = BccChecksum.compute(almost, 2, almost.length - 2); - out.write(bcc & 0xFF); - return out.toByteArray(); - } - - private static void writeU16(ByteArrayOutputStream os, int v) { - os.write((v >> 8) & 0xFF); - os.write(v & 0xFF); - } - - private static void writeU32(ByteArrayOutputStream os, long v) { - os.write((int) ((v >> 24) & 0xFF)); - os.write((int) ((v >> 16) & 0xFF)); - os.write((int) ((v >> 8) & 0xFF)); - os.write((int) (v & 0xFF)); - } - - // ==================================================================== - // Decoder 装配：手动构造一个挂着 guangdong-fc 路由的 BodyParser - // ==================================================================== - private static Gb32960MessageDecoder buildDecoder() { - var standardParsers = List.of( - (com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser) new VehicleV2016BlockParser(), - new DriveMotorV2016BlockParser(), - new PositionV2016BlockParser(), - new AlarmV2016BlockParser(), - new VoltageV2016BlockParser(), - new TemperatureV2016BlockParser()); - var catalog = new VendorExtensionCatalog(java.util.Map.of( - "guangdong-fc", List.of( - new GdFcStackBlockParser(), - new GdFcAuxiliaryBlockParser(), - new GdFcDcDcBlockParser(), - new GdFcAirConditionerBlockParser(), - new GdFcVehicleInfoBlockParser(), - new GdFcDemoExtensionBlockParser()))); - var profileRegistry = new Gb32960ProfileRegistry( - standardParsers, catalog, Set.of("guangdong-fc")); - - // selector：lingniu 账号 → guangdong-fc - var entry = new Gb32960Properties.VendorExtension(); - entry.setName("guangdong-fc"); - var match = new Gb32960Properties.VendorExtension.Match(); - match.setPlatformAccounts(List.of("lingniu")); - entry.setMatch(match); - var selector = new RuleBasedVendorExtensionSelector( - List.of(entry), Set.of("guangdong-fc")); - - var bodyParser = new Gb32960BodyParser(profileRegistry, selector); - // PLATFORM_ACCOUNT_ATTR 仅供 handler 使用，这里不需要 - return new Gb32960MessageDecoder(bodyParser); - } - - // suppress unused-import warning for the channel handler attr key - @SuppressWarnings("unused") - private static final Object KEEP_ATTR_REF = Gb32960ChannelHandler.PLATFORM_ACCOUNT_ATTR; -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcParserTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcParserTest.java deleted file mode 100644 index 0aedda7f..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcParserTest.java +++ /dev/null @@ -1,87 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong; - -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import org.junit.jupiter.api.Test; - -import java.nio.ByteBuffer; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.within; - -/** - * 单元测试：广东燃料电池规范 fixed-length parser（0x32 / 0x33 / 0x34 / 0x80）逐字段还原。 - * 0x30/0x31 是变长，集成层会在 BodyParser 全链路测试里覆盖。 - */ -class GdFcParserTest { - - @Test - void dcDcBlockParsesAllFields() { - // inV=300.0 inC=120.0 outV=560.0 outC=110.5 ctrlTemp=65°C - ByteBuffer buf = ByteBuffer.wrap(new byte[]{ - 0x0B, (byte) 0xB8, // 0x0BB8 = 3000 → 300.0 V - 0x04, (byte) 0xB0, // 0x04B0 = 1200 → 120.0 A - 0x15, (byte) 0xE0, // 0x15E0 = 5600 → 560.0 V - 0x04, 0x52, // 0x0452 = 1106 → 110.6 A - 0x69 // 0x69 = 105 - 40 = 65 °C - }); - InfoBlock.GuangdongFc.DcDc b = - (InfoBlock.GuangdongFc.DcDc) new GdFcDcDcBlockParser().parse(buf); - assertThat(b.inputVoltageV()).isCloseTo(300.0, within(1e-6)); - assertThat(b.inputCurrentA()).isCloseTo(120.0, within(1e-6)); - assertThat(b.outputVoltageV()).isCloseTo(560.0, within(1e-6)); - assertThat(b.outputCurrentA()).isCloseTo(110.6, within(1e-6)); - assertThat(b.controllerTempC()).isEqualTo(65); - } - - @Test - void airConditionerParsesPowerOffsetAndStatus() { - // status=01 power raw=20 (= 15 kw after -5 offset) inputV=540.0 - ByteBuffer buf = ByteBuffer.wrap(new byte[]{ - 0x01, - 0x00, 0x14, // 20 - 5 = 15 kw - 0x15, (byte) 0x18 // 0x1518 = 5400 → 540.0 V - }); - InfoBlock.GuangdongFc.AirConditioner b = - (InfoBlock.GuangdongFc.AirConditioner) new GdFcAirConditionerBlockParser().parse(buf); - assertThat(b.status()).isEqualTo(1); - assertThat(b.powerKw()).isEqualTo(15); - assertThat(b.compressorInputVoltageV()).isCloseTo(540.0, within(1e-6)); - } - - @Test - void vehicleInfoParsesAmbientFields() { - // collision=00 ambientTemp raw=70 (=30°C) ambientPressure raw=1100 (10kPa) h2Mass raw=80 (8.0 kg) - ByteBuffer buf = ByteBuffer.wrap(new byte[]{ - 0x00, - 0x00, 0x46, // 70 - 40 = 30°C - 0x04, 0x4C, // 0x044C = 1100 → 110*0.1 - 100 = 10.0 kPa - 0x00, 0x50 // 0x0050 = 80 → 8.0 kg - }); - InfoBlock.GuangdongFc.VehicleInfo b = - (InfoBlock.GuangdongFc.VehicleInfo) new GdFcVehicleInfoBlockParser().parse(buf); - assertThat(b.collisionAlarm()).isEqualTo(0); - assertThat(b.ambientTempC()).isEqualTo(30); - assertThat(b.ambientPressureKpa()).isCloseTo(10.0, within(1e-6)); - assertThat(b.hydrogenMassKg()).isCloseTo(8.0, within(1e-6)); - } - - @Test - void demoExtensionParsesInnerLengthAndFields() { - // length=9, stackTemp=50, acV=372.0, acC=0, h2V=1.3V, h2C=0 - ByteBuffer buf = ByteBuffer.wrap(new byte[]{ - 0x00, 0x09, // declared length - 0x5A, // 90 - 40 = 50°C - 0x0E, (byte) 0x88, // 0x0E88 = 3720 → 372.0V - 0x27, 0x10, // 0x2710 = 10000 → 0 A (after -1000) - 0x00, 0x0D, // 0x000D = 13 → 1.3V - 0x27, 0x10 // 0 A - }); - InfoBlock.GuangdongFc.DemoExtension b = - (InfoBlock.GuangdongFc.DemoExtension) new GdFcDemoExtensionBlockParser().parse(buf); - assertThat(b.stackTempC()).isEqualTo(50); - assertThat(b.airCompressorVoltageV()).isCloseTo(372.0, within(1e-6)); - assertThat(b.airCompressorCurrentA()).isCloseTo(0.0, within(1e-6)); - assertThat(b.hydrogenPumpVoltageV()).isCloseTo(1.3, within(1e-6)); - assertThat(b.hydrogenPumpCurrentA()).isCloseTo(0.0, within(1e-6)); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcVendorTlvBlockParserTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcVendorTlvBlockParserTest.java deleted file mode 100644 index e8d93891..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/parser/vendor/guangdong/GdFcVendorTlvBlockParserTest.java +++ /dev/null @@ -1,118 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong; - -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960BodyParser; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParserRegistry; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayOutputStream; -import java.nio.ByteBuffer; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -/** - * 验证 {@link GdFcVendorTlvBlockParser} 严格按 length 消费字节，不会贪心吞光后续信息体。 - * - *

    这是关键回归 case：用户问"加 0x83 兜底会不会把后面其它块也吃掉？"——通过构造 - * 一个 {@code 0x83 TLV + 0x01 Vehicle} 串联的 body，断言 BodyParser 解出**两个块**而非一个。 - */ -class GdFcVendorTlvBlockParserTest { - - @Test - void parsesPayloadOfDeclaredLengthExactly() { - // 0x83 typeCode 已被外层消费，buffer 从 length 字段开始 - byte[] data = bytes( - 0x00, 0x05, // length = 5 - 0xAA, 0xBB, 0xCC, 0xDD, 0xEE); // 5 bytes payload - ByteBuffer buf = ByteBuffer.wrap(data); - - InfoBlock.GuangdongFc.VendorTlv tlv = - (InfoBlock.GuangdongFc.VendorTlv) new GdFcVendorTlvBlockParser(0x83).parse(buf); - - assertThat(tlv.typeCode()).isEqualTo(0x83); - assertThat(tlv.declaredLength()).isEqualTo(5); - assertThat(tlv.payload()).containsExactly(0xAA, 0xBB, 0xCC, 0xDD, 0xEE); - assertThat(buf.remaining()).as("应正好消费 2+5=7 字节").isZero(); - } - - @Test - void doesNotSwallowSubsequentBlocksWhenChainedInBodyParser() { - // 构造 body：0x83 TLV(length=4) + 0x01 Vehicle(20B 标准布局) - ByteArrayOutputStream body = new ByteArrayOutputStream(); - // 0x83 TLV：3+4 = 7 字节 - body.write(0x83); - body.write(0x00); body.write(0x04); // length = 4 - body.write(0x11); body.write(0x22); body.write(0x33); body.write(0x44); - // 0x01 Vehicle: typeCode + 20 字节 body - body.write(0x01); - body.write(0x01); // vehicleState - body.write(0x03); // chargingState - body.write(0x02); // runningMode - write16(body, 0); // speed - write32(body, 100_000L); // mileage - write16(body, 5605); // totalVoltage - write16(body, 10000); // totalCurrent - body.write(85); // SOC - body.write(0x01); // dcDcStatus - body.write(0x00); // gear - write16(body, 10000); // insulation - body.write(0x00); // accelPedal - body.write(0x00); // brakePedal - - // 装配 body parser：注册 0x83 vendor TLV + 0x01 标准 Vehicle - InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of( - new GdFcVendorTlvBlockParser(0x83), - new VehicleV2016BlockParser())); - Gb32960BodyParser parser = new Gb32960BodyParser(registry); - - ByteBuffer buf = ByteBuffer.wrap(body.toByteArray()); - Gb32960MessageDecoder.BodyParseResult result = parser.parse(ProtocolVersion.V2016, buf); - - // 关键断言：两个块都解析出来了，0x01 没被 0x83 吞掉 - assertThat(result.blocks()).hasSize(2); - assertThat(result.blocks().get(0)).isInstanceOf(InfoBlock.GuangdongFc.VendorTlv.class); - assertThat(result.blocks().get(1)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class); - - InfoBlock.GuangdongFc.VendorTlv tlv = (InfoBlock.GuangdongFc.VendorTlv) result.blocks().get(0); - assertThat(tlv.declaredLength()).isEqualTo(4); - assertThat(tlv.payload()).containsExactly(0x11, 0x22, 0x33, 0x44); - - InfoBlock.Gb32960V2016.Vehicle v = (InfoBlock.Gb32960V2016.Vehicle) result.blocks().get(1); - assertThat(v.socPercent()).isEqualTo(85); - assertThat(v.totalVoltageV()).isEqualTo(560.5); - } - - @Test - void truncatesPayloadIfDeclaredLengthExceedsRemaining() { - // 防御场景：length 字段声称 100 字节但 buffer 只剩 3 字节 - byte[] data = bytes(0x00, 0x64, 0x01, 0x02, 0x03); - ByteBuffer buf = ByteBuffer.wrap(data); - InfoBlock.GuangdongFc.VendorTlv tlv = - (InfoBlock.GuangdongFc.VendorTlv) new GdFcVendorTlvBlockParser(0x83).parse(buf); - assertThat(tlv.declaredLength()).isEqualTo(100); - assertThat(tlv.payload()).hasSize(3).containsExactly(0x01, 0x02, 0x03); - } - - private static byte[] bytes(int... values) { - byte[] out = new byte[values.length]; - for (int i = 0; i < values.length; i++) out[i] = (byte) values[i]; - return out; - } - - private static void write16(ByteArrayOutputStream os, int v) { - os.write((v >> 8) & 0xFF); - os.write(v & 0xFF); - } - - private static void write32(ByteArrayOutputStream os, long v) { - os.write((int) ((v >> 24) & 0xFF)); - os.write((int) ((v >> 16) & 0xFF)); - os.write((int) ((v >> 8) & 0xFF)); - os.write((int) (v & 0xFF)); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/profile/RuleBasedVendorExtensionSelectorTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/profile/RuleBasedVendorExtensionSelectorTest.java deleted file mode 100644 index c4e4ddff..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/profile/RuleBasedVendorExtensionSelectorTest.java +++ /dev/null @@ -1,106 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.codec.profile; - -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960ParserContext; -import com.lingniu.ingest.protocol.gb32960.config.Gb32960Properties; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -import org.junit.jupiter.api.Test; - -import java.util.List; -import java.util.Set; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class RuleBasedVendorExtensionSelectorTest { - - @Test - void platformAccountMatchHits() { - var ext = entry("guangdong-fc", List.of("lingniu"), List.of(), List.of()); - var selector = new RuleBasedVendorExtensionSelector(List.of(ext), Set.of("guangdong-fc")); - var ctx = new Gb32960ParserContext(ProtocolVersion.V2016, "LNV1234567890ABCD", "lingniu"); - assertThat(selector.select(ctx)).isEqualTo("guangdong-fc"); - } - - @Test - void vinExactMatchHits() { - var ext = entry("guangdong-fc", List.of(), List.of("LNV1234567890ABCD"), List.of()); - var selector = new RuleBasedVendorExtensionSelector(List.of(ext), Set.of("guangdong-fc")); - var ctx = new Gb32960ParserContext(ProtocolVersion.V2016, "lnv1234567890abcd", null); - assertThat(selector.select(ctx)) - .as("VIN match should be case-insensitive") - .isEqualTo("guangdong-fc"); - } - - @Test - void vinPrefixMatchHits() { - var ext = entry("guangdong-fc", List.of(), List.of(), List.of("LNVFC", "LZG")); - var selector = new RuleBasedVendorExtensionSelector(List.of(ext), Set.of("guangdong-fc")); - var ctx = new Gb32960ParserContext(ProtocolVersion.V2016, "LNVFC0001", null); - assertThat(selector.select(ctx)).isEqualTo("guangdong-fc"); - } - - @Test - void noMatchReturnsNull() { - var ext = entry("guangdong-fc", List.of("other"), List.of(), List.of()); - var selector = new RuleBasedVendorExtensionSelector(List.of(ext), Set.of("guangdong-fc")); - var ctx = new Gb32960ParserContext(ProtocolVersion.V2016, "ZZZ", "lingniu"); - assertThat(selector.select(ctx)).isNull(); - } - - @Test - void firstMatchWinsTopDown() { - var first = entry("guangdong-fc", List.of("lingniu"), List.of(), List.of()); - var second = entry("guangdong-fc", List.of(), List.of(), List.of("LNV")); - var selector = new RuleBasedVendorExtensionSelector( - List.of(first, second), Set.of("guangdong-fc")); - // 即便两条都能命中，应返回第一条匹配 entry 的 name（这里凑巧一样） - var ctx = new Gb32960ParserContext(ProtocolVersion.V2016, "LNV1", "lingniu"); - assertThat(selector.select(ctx)).isEqualTo("guangdong-fc"); - } - - @Test - void emptyConfigReturnsNullWithoutScanning() { - var selector = new RuleBasedVendorExtensionSelector(List.of(), Set.of("guangdong-fc")); - var ctx = new Gb32960ParserContext(ProtocolVersion.V2016, "ANY", "ANY"); - assertThat(selector.select(ctx)).isNull(); - } - - @Test - void unknownExtensionNameRejectedAtConstruction() { - var ext = entry("not-in-catalog", List.of("x"), List.of(), List.of()); - assertThatThrownBy(() -> - new RuleBasedVendorExtensionSelector(List.of(ext), Set.of("guangdong-fc"))) - .isInstanceOf(IllegalStateException.class) - .hasMessageContaining("not-in-catalog"); - } - - @Test - void resultIsCachedPerAccountVinTuple() { - var ext = entry("guangdong-fc", List.of("lingniu"), List.of(), List.of()); - var selector = new RuleBasedVendorExtensionSelector(List.of(ext), Set.of("guangdong-fc")); - var ctx1 = new Gb32960ParserContext(ProtocolVersion.V2016, "VIN1", "lingniu"); - var ctx2 = new Gb32960ParserContext(ProtocolVersion.V2016, "VIN1", "lingniu"); - selector.select(ctx1); - selector.select(ctx2); - // 同一个 (account, vin) 应只产生一个 cache 条目 - assertThat(selector.cacheSize()).isEqualTo(1); - - var ctx3 = new Gb32960ParserContext(ProtocolVersion.V2016, "VIN2", "lingniu"); - selector.select(ctx3); - assertThat(selector.cacheSize()).isEqualTo(2); - } - - private static Gb32960Properties.VendorExtension entry(String name, - List accounts, - List vins, - List vinPrefixes) { - var e = new Gb32960Properties.VendorExtension(); - e.setName(name); - var m = new Gb32960Properties.VendorExtension.Match(); - m.setPlatformAccounts(accounts); - m.setVins(vins); - m.setVinPrefixes(vinPrefixes); - e.setMatch(m); - return e; - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfigurationTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfigurationTest.java deleted file mode 100644 index 69bf3899..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfigurationTest.java +++ /dev/null @@ -1,37 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.config; - -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960NettyServer; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; - -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960AutoConfigurationTest { - - private final ApplicationContextRunner runner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(Gb32960AutoConfiguration.class)) - .withBean(Dispatcher.class, () -> new Dispatcher(null, null, null, null, null)) - .withPropertyValues( - "lingniu.ingest.gb32960.enabled=true", - "lingniu.ingest.gb32960.port=0"); - - @Test - void createsDecoderWithoutTcpServerWhenServerDisabled() { - runner.withPropertyValues("lingniu.ingest.gb32960.server.enabled=false") - .run(context -> { - assertThat(context).hasSingleBean(Gb32960MessageDecoder.class); - assertThat(context).doesNotHaveBean(Gb32960NettyServer.class); - }); - } - - @Test - void createsTcpServerByDefaultWhenProtocolEnabled() { - runner.run(context -> { - assertThat(context).hasSingleBean(Gb32960MessageDecoder.class); - assertThat(context).hasSingleBean(Gb32960NettyServer.class); - }); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/handler/Gb32960RealtimeHandlerTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/handler/Gb32960RealtimeHandlerTest.java deleted file mode 100644 index b46862d3..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/handler/Gb32960RealtimeHandlerTest.java +++ /dev/null @@ -1,79 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.handler; - -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.protocol.gb32960.mapper.Gb32960EventMapper; -import com.lingniu.ingest.protocol.gb32960.model.CommandBody; -import com.lingniu.ingest.protocol.gb32960.model.CommandType; -import com.lingniu.ingest.protocol.gb32960.model.EncryptType; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Header; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -import com.lingniu.ingest.protocol.gb32960.model.ResponseFlag; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -/** - * 保障 {@link Gb32960RealtimeHandler} 正确委托给 {@link Gb32960EventMapper}， - * 不吞掉任何需要上送下游的事件。 - */ -class Gb32960RealtimeHandlerTest { - - private final Gb32960RealtimeHandler handler = new Gb32960RealtimeHandler(new Gb32960EventMapper()); - - @Test - void onPlatform_login_emitsLoginEventWithKindPlatform() { - Gb32960Message msg = buildPlatformLoginMessage("lingniu"); - - List events = handler.onPlatform(msg); - - assertThat(events).hasSize(1); - VehicleEvent first = events.get(0); - assertThat(first).isInstanceOf(VehicleEvent.Login.class); - assertThat(first.vin()).isEqualTo("platform:lingniu"); - assertThat(first.metadata()) - .containsEntry("kind", "platform") - .containsEntry("username", "lingniu"); - } - - @Test - void onPlatform_logout_emitsLogoutEventWithKindPlatform() { - Gb32960Message msg = new Gb32960Message( - new Gb32960Header( - ProtocolVersion.V2016, - CommandType.PLATFORM_LOGOUT, - ResponseFlag.COMMAND, - "00000000000000000", - EncryptType.UNENCRYPTED, - 0, - null), - new CommandBody.PlatformLogout(Instant.parse("2026-04-20T10:00:00Z"), 1)); - - List events = handler.onPlatform(msg); - - assertThat(events).hasSize(1); - assertThat(events.get(0)).isInstanceOf(VehicleEvent.Logout.class); - assertThat(events.get(0).metadata()).containsEntry("kind", "platform"); - } - - private static Gb32960Message buildPlatformLoginMessage(String username) { - return new Gb32960Message( - new Gb32960Header( - ProtocolVersion.V2016, - CommandType.PLATFORM_LOGIN, - ResponseFlag.COMMAND, - "00000000000000000", - EncryptType.UNENCRYPTED, - 0, - null), - new CommandBody.PlatformLogin( - Instant.parse("2026-04-20T10:00:00Z"), - 1, - username, - "secret", - EncryptType.UNENCRYPTED)); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AccessServiceTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AccessServiceTest.java deleted file mode 100644 index d6453115..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AccessServiceTest.java +++ /dev/null @@ -1,86 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.inbound; - -import com.lingniu.ingest.protocol.gb32960.auth.Gb32960PlatformAuthorizer; -import com.lingniu.ingest.protocol.gb32960.auth.Gb32960VinAuthorizer; -import com.lingniu.ingest.protocol.gb32960.config.Gb32960Properties; -import io.netty.channel.embedded.EmbeddedChannel; -import org.junit.jupiter.api.Test; - -import java.net.InetSocketAddress; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960AccessServiceTest { - - @Test - void bindPlatformAccount_makesAccountAvailableForLaterFrames() { - Gb32960AccessService service = newService(false); - EmbeddedChannel channel = new EmbeddedChannel(); - - service.bindPlatformAccount(channel, "lingniu"); - - assertThat(service.platformAccount(channel)).isEqualTo("lingniu"); - } - - @Test - void authenticatePlatformLogin_rejectsUnknownPeerIpWhenPolicyRestrictsIt() { - Gb32960Properties.Auth auth = new Gb32960Properties.Auth(); - Gb32960Properties.Auth.Platform platform = new Gb32960Properties.Auth.Platform(); - platform.setUsername("lingniu"); - platform.setPassword("secret"); - platform.setAllowedIps(List.of("10.0.0.1")); - auth.setPlatforms(List.of(platform)); - - Gb32960AccessService service = new Gb32960AccessService( - new Gb32960VinAuthorizer(auth), - new Gb32960PlatformAuthorizer(auth.getPlatforms())); - EmbeddedChannel channel = new EmbeddedChannel(); - channel.connect(new InetSocketAddress("10.0.0.2", 9001)); - - assertThat(service.authenticatePlatformLogin("lingniu", "secret", channel) - .accepted()).isFalse(); - } - - @Test - void authenticatePlatformLogin_ignoresBlankAllowedIps() { - Gb32960Properties.Auth auth = new Gb32960Properties.Auth(); - Gb32960Properties.Auth.Platform platform = new Gb32960Properties.Auth.Platform(); - platform.setUsername("lingniu"); - platform.setPassword("secret"); - platform.setAllowedIps(List.of("", " ", "\t")); - auth.setPlatforms(List.of(platform)); - - Gb32960AccessService service = new Gb32960AccessService( - new Gb32960VinAuthorizer(auth), - new Gb32960PlatformAuthorizer(auth.getPlatforms())); - EmbeddedChannel channel = new EmbeddedChannel(); - channel.connect(new InetSocketAddress("115.29.187.205", 9001)); - - assertThat(service.authenticatePlatformLogin("lingniu", "secret", channel) - .accepted()).isTrue(); - } - - @Test - void authenticatePlatformLogin_acceptsCommaSeparatedAllowedIps() { - Gb32960Properties.Auth auth = new Gb32960Properties.Auth(); - Gb32960Properties.Auth.Platform platform = new Gb32960Properties.Auth.Platform(); - platform.setUsername("lingniu"); - platform.setPassword("secret"); - platform.setAllowedIps(List.of("115.29.187.205,127.0.0.1")); - auth.setPlatforms(List.of(platform)); - - Gb32960PlatformAuthorizer authorizer = new Gb32960PlatformAuthorizer(auth.getPlatforms()); - - assertThat(authorizer.authenticate("lingniu", "secret", "127.0.0.1") - .accepted()).isTrue(); - } - - private static Gb32960AccessService newService(boolean vinAuthEnabled) { - Gb32960Properties.Auth auth = new Gb32960Properties.Auth(); - auth.setEnabled(vinAuthEnabled); - return new Gb32960AccessService( - new Gb32960VinAuthorizer(auth), - new Gb32960PlatformAuthorizer(auth.getPlatforms())); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AckServiceTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AckServiceTest.java deleted file mode 100644 index 4f914119..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960AckServiceTest.java +++ /dev/null @@ -1,42 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.inbound; - -import com.lingniu.ingest.protocol.gb32960.model.CommandType; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -import com.lingniu.ingest.protocol.gb32960.model.ResponseFlag; -import io.netty.buffer.ByteBuf; -import io.netty.channel.embedded.EmbeddedChannel; -import org.junit.jupiter.api.Test; - -import java.nio.charset.StandardCharsets; -import java.time.Instant; - -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960AckServiceTest { - - @Test - void writeResponse_writesGb32960AckFrameToChannel() { - Gb32960AckService service = new Gb32960AckService(); - EmbeddedChannel channel = new EmbeddedChannel(); - byte[] rawVin = "LTEST000000000001".getBytes(StandardCharsets.US_ASCII); - - service.writeResponse( - channel, - ProtocolVersion.V2016, - CommandType.VEHICLE_LOGIN, - ResponseFlag.SUCCESS, - rawVin, - Instant.parse("2026-04-20T10:00:00Z"), - null, - "vehicle-login-ack"); - - ByteBuf out = channel.readOutbound(); - assertThat(out).isNotNull(); - byte[] bytes = new byte[out.readableBytes()]; - out.readBytes(bytes); - assertThat(bytes[0]).isEqualTo((byte) 0x23); - assertThat(bytes[1]).isEqualTo((byte) 0x23); - assertThat(bytes[2]).isEqualTo((byte) CommandType.VEHICLE_LOGIN.code()); - assertThat(bytes[3]).isEqualTo((byte) ResponseFlag.SUCCESS.code()); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandlerAckBoundaryTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandlerAckBoundaryTest.java deleted file mode 100644 index 4e9cb5d9..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ChannelHandlerAckBoundaryTest.java +++ /dev/null @@ -1,235 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.inbound; - -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.sink.EventSink; -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.core.concurrency.AsyncBatchExecutor; -import com.lingniu.ingest.core.concurrency.DisruptorEventBus; -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.core.dispatcher.HandlerInvoker; -import com.lingniu.ingest.core.dispatcher.HandlerRegistry; -import com.lingniu.ingest.core.pipeline.InterceptorChain; -import com.lingniu.ingest.protocol.gb32960.auth.Gb32960PlatformAuthorizer; -import com.lingniu.ingest.protocol.gb32960.auth.Gb32960VinAuthorizer; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960BodyParser; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960DecoderTest; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParserRegistry; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.PositionV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.config.Gb32960Properties; -import io.netty.buffer.ByteBuf; -import io.netty.channel.embedded.EmbeddedChannel; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayOutputStream; -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.List; -import java.util.concurrent.CompletableFuture; - -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960ChannelHandlerAckBoundaryTest { - - @Test - void realtimeReportAckWaitsForDispatchFutureCompletion() { - try (DispatchHarness dispatch = new DispatchHarness()) { - EmbeddedChannel channel = new EmbeddedChannel(handlerWith(dispatch.dispatcher())); - - assertThat(channel.writeInbound(Gb32960DecoderTest.buildRealtimeFrame("LTEST000000000001"))).isFalse(); - assertThat((Object) channel.readOutbound()).isNull(); - - dispatch.awaitPublishCount(1); - dispatch.completeDurability(); - - ByteBuf ack = awaitOutbound(channel); - assertThat(ack).isNotNull(); - assertThat(ack.getUnsignedByte(2)).isEqualTo((short) 0x02); - assertThat(ack.getUnsignedByte(3)).isEqualTo((short) 0x01); - ack.release(); - channel.finishAndReleaseAll(); - } - } - - @Test - void realtimeReportAckIsNotWrittenWhenDispatchFutureFails() { - try (DispatchHarness dispatch = new DispatchHarness()) { - EmbeddedChannel channel = new EmbeddedChannel(handlerWith(dispatch.dispatcher())); - - assertThat(channel.writeInbound(Gb32960DecoderTest.buildRealtimeFrame("LTEST000000000001"))).isFalse(); - - dispatch.awaitPublishCount(1); - dispatch.failDurability(); - - assertThat((Object) channel.readOutbound()).isNull(); - assertThat(awaitInactive(channel)).isTrue(); - channel.finishAndReleaseAll(); - } - } - - @Test - void durableAcksAreWrittenInInboundOrderWhenLaterDispatchCompletesFirst() { - try (DispatchHarness dispatch = new DispatchHarness()) { - EmbeddedChannel channel = new EmbeddedChannel(handlerWith(dispatch.dispatcher())); - - assertThat(channel.writeInbound(Gb32960DecoderTest.buildRealtimeFrame("LTEST000000000001"))).isFalse(); - assertThat(channel.writeInbound(buildHeartbeatFrame("LTEST000000000001"))).isFalse(); - assertThat((Object) channel.readOutbound()).isNull(); - - dispatch.awaitPublishCount(2); - dispatch.completeDurability(1); - channel.runPendingTasks(); - assertThat((Object) channel.readOutbound()).isNull(); - - dispatch.completeDurability(0); - - ByteBuf firstAck = awaitOutbound(channel); - ByteBuf secondAck = awaitOutbound(channel); - assertThat(firstAck).isNotNull(); - assertThat(secondAck).isNotNull(); - assertThat(firstAck.getUnsignedByte(2)).isEqualTo((short) 0x02); - assertThat(secondAck.getUnsignedByte(2)).isEqualTo((short) 0x07); - firstAck.release(); - secondAck.release(); - channel.finishAndReleaseAll(); - } - } - - private static Gb32960ChannelHandler handlerWith(Dispatcher dispatcher) { - return new Gb32960ChannelHandler( - decoder(), - dispatcher, - accessService(), - new Gb32960AckService(), - new Gb32960FrameDiagnostics(16), - new Gb32960ConnectionDiagnostics()); - } - - private static Gb32960MessageDecoder decoder() { - return new Gb32960MessageDecoder(new Gb32960BodyParser(new InfoBlockParserRegistry(List.of( - new VehicleV2016BlockParser(), - new PositionV2016BlockParser())))); - } - - private static Gb32960AccessService accessService() { - Gb32960Properties.Auth auth = new Gb32960Properties.Auth(); - auth.setEnabled(false); - return new Gb32960AccessService( - new Gb32960VinAuthorizer(auth), - new Gb32960PlatformAuthorizer(auth.getPlatforms())); - } - - private static byte[] buildHeartbeatFrame(String vin) { - ByteArrayOutputStream out = new ByteArrayOutputStream(); - out.write(0x23); - out.write(0x23); - out.write(0x07); - out.write(0xFE); - byte[] vinBytes = vin.getBytes(StandardCharsets.US_ASCII); - out.write(vinBytes, 0, 17); - out.write(0x01); - out.write(0x00); - out.write(0x00); - - byte[] almost = out.toByteArray(); - out.write(BccChecksum.compute(almost, 2, almost.length - 2) & 0xFF); - return out.toByteArray(); - } - - private static ByteBuf awaitOutbound(EmbeddedChannel channel) { - long deadline = System.currentTimeMillis() + 3000; - while (System.currentTimeMillis() < deadline) { - channel.runPendingTasks(); - ByteBuf outbound = channel.readOutbound(); - if (outbound != null) { - return outbound; - } - Thread.onSpinWait(); - } - return null; - } - - private static boolean awaitInactive(EmbeddedChannel channel) { - long deadline = System.currentTimeMillis() + 3000; - while (System.currentTimeMillis() < deadline) { - channel.runPendingTasks(); - if (!channel.isActive()) { - return true; - } - Thread.onSpinWait(); - } - return false; - } - - private static final class DispatchHarness implements AutoCloseable { - private final ControlledSink sink = new ControlledSink(); - private final DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - private final AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - private final Dispatcher dispatcher = new Dispatcher( - new HandlerRegistry(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - - Dispatcher dispatcher() { - return dispatcher; - } - - void completeDurability() { - completeDurability(0); - } - - void completeDurability(int index) { - sink.future(index).complete(null); - } - - void failDurability() { - sink.future(0).completeExceptionally(new IllegalStateException("durability failed")); - } - - void awaitPublishCount(int count) { - long deadline = System.currentTimeMillis() + 3000; - while (System.currentTimeMillis() < deadline) { - synchronized (sink.futures) { - if (sink.futures.size() >= count) { - return; - } - } - Thread.onSpinWait(); - } - throw new AssertionError("expected " + count + " sink publishes, actual=" + sink.futures.size()); - } - - @Override - public void close() { - batchExecutor.close(); - eventBus.close(); - } - } - - private static final class ControlledSink implements EventSink { - private final List> futures = new ArrayList<>(); - - @Override - public String name() { - return "kafka"; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - CompletableFuture future = new CompletableFuture<>(); - synchronized (futures) { - futures.add(future); - } - return future; - } - - private CompletableFuture future(int index) { - synchronized (futures) { - return futures.get(index); - } - } - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ConnectionDiagnosticsTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ConnectionDiagnosticsTest.java deleted file mode 100644 index cc68c65c..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960ConnectionDiagnosticsTest.java +++ /dev/null @@ -1,78 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.inbound; - -import com.lingniu.ingest.protocol.gb32960.model.CommandType; -import org.junit.jupiter.api.Test; - -import java.time.Clock; -import java.time.Instant; -import java.time.ZoneOffset; - -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960ConnectionDiagnosticsTest { - - private static final Instant NOW = Instant.parse("2026-06-29T10:00:00Z"); - - @Test - void tracksActiveSessionCommandPlatformAccountAndReadIdleCount() { - MutableClock clock = new MutableClock(NOW); - Gb32960ConnectionDiagnostics diagnostics = new Gb32960ConnectionDiagnostics(clock); - - diagnostics.opened("session-1", "127.0.0.1:65469"); - clock.advanceSeconds(5); - diagnostics.received("session-1", CommandType.PLATFORM_LOGIN, ""); - diagnostics.platformLoginAccepted("session-1", "Hyundai"); - clock.advanceSeconds(30); - diagnostics.readIdle("session-1"); - - assertThat(diagnostics.activeSessions()).singleElement().satisfies(session -> { - assertThat(session.sessionId()).isEqualTo("session-1"); - assertThat(session.peer()).isEqualTo("127.0.0.1:65469"); - assertThat(session.connectedAt()).isEqualTo(NOW); - assertThat(session.lastInboundAt()).isEqualTo(NOW.plusSeconds(5)); - assertThat(session.idleSeconds()).isEqualTo(30); - assertThat(session.lastCommandCode()).isEqualTo(0x05); - assertThat(session.lastCommand()).isEqualTo("PLATFORM_LOGIN"); - assertThat(session.vin()).isEmpty(); - assertThat(session.platformAccount()).isEqualTo("Hyundai"); - assertThat(session.readIdleCount()).isEqualTo(1); - }); - } - - @Test - void removesSessionWhenChannelCloses() { - Gb32960ConnectionDiagnostics diagnostics = new Gb32960ConnectionDiagnostics(Clock.fixed(NOW, ZoneOffset.UTC)); - - diagnostics.opened("session-1", "127.0.0.1:65469"); - diagnostics.closed("session-1"); - - assertThat(diagnostics.activeSessions()).isEmpty(); - } - - private static final class MutableClock extends Clock { - private Instant instant; - - private MutableClock(Instant instant) { - this.instant = instant; - } - - void advanceSeconds(long seconds) { - instant = instant.plusSeconds(seconds); - } - - @Override - public ZoneOffset getZone() { - return ZoneOffset.UTC; - } - - @Override - public Clock withZone(java.time.ZoneId zone) { - return this; - } - - @Override - public Instant instant() { - return instant; - } - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960FrameDiagnosticsTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960FrameDiagnosticsTest.java deleted file mode 100644 index 254adf4d..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/inbound/Gb32960FrameDiagnosticsTest.java +++ /dev/null @@ -1,45 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.inbound; - -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlockType; -import io.netty.channel.embedded.EmbeddedChannel; -import org.junit.jupiter.api.Test; - -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960FrameDiagnosticsTest { - - @Test - void shouldLogFirstSeenRawSignatureOnlyOncePerChannel() { - Gb32960FrameDiagnostics diagnostics = new Gb32960FrameDiagnostics(8); - EmbeddedChannel channel = new EmbeddedChannel(); - - List rawTypes = List.of("0x30", "0x83"); - - assertThat(diagnostics.markFirstSeen(channel, "VIN001", rawTypes)).isTrue(); - assertThat(diagnostics.markFirstSeen(channel, "VIN001", rawTypes)).isFalse(); - } - - @Test - void markFirstSeen_keepsBoundedKeysPerChannel() { - Gb32960FrameDiagnostics diagnostics = new Gb32960FrameDiagnostics(2); - EmbeddedChannel channel = new EmbeddedChannel(); - - assertThat(diagnostics.markFirstSeen(channel, "VIN001", List.of("0x30"))).isTrue(); - assertThat(diagnostics.markFirstSeen(channel, "VIN001", List.of("0x31"))).isTrue(); - assertThat(diagnostics.markFirstSeen(channel, "VIN001", List.of("0x32"))).isTrue(); - - assertThat(diagnostics.seenKeyCount(channel)).isLessThanOrEqualTo(2); - } - - @Test - void collectRawTypeHex_returnsOnlyRawBlocks() { - List rawTypes = Gb32960FrameDiagnostics.collectRawTypeHex(List.of( - new InfoBlock.Raw(0x30, InfoBlockType.RAW, new byte[] {1}), - new InfoBlock.Raw(0x83, InfoBlockType.RAW, new byte[] {2}))); - - assertThat(rawTypes).containsExactly("0x30", "0x83"); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/mapper/Gb32960EventMapperTest.java b/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/mapper/Gb32960EventMapperTest.java deleted file mode 100644 index 8f4e1d15..00000000 --- a/modules/protocols/protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/mapper/Gb32960EventMapperTest.java +++ /dev/null @@ -1,88 +0,0 @@ -package com.lingniu.ingest.protocol.gb32960.mapper; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.AlarmPayload; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960BodyParser; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960DecoderTest; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParserRegistry; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.PositionV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.model.CommandType; -import com.lingniu.ingest.protocol.gb32960.model.EncryptType; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Header; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; -import com.lingniu.ingest.protocol.gb32960.model.GeneralAlarmFlagBit; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion; -import com.lingniu.ingest.protocol.gb32960.model.ResponseFlag; -import org.junit.jupiter.api.Test; - -import java.nio.ByteBuffer; -import java.time.Instant; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960EventMapperTest { - - @Test - void realtimeReportProducesRealtimeAndLocationEvents() { - byte[] frame = Gb32960DecoderTest.buildRealtimeFrame("LTEST000000000002"); - - Gb32960BodyParser body = new Gb32960BodyParser(new InfoBlockParserRegistry( - List.of(new VehicleV2016BlockParser(), new PositionV2016BlockParser()))); - Gb32960Message msg = new Gb32960MessageDecoder(body).decode(ByteBuffer.wrap(frame)); - - List events = new Gb32960EventMapper().toEvents(msg); - - assertThat(events).hasSize(2); - assertThat(events).anyMatch(e -> e instanceof VehicleEvent.Realtime); - assertThat(events).anyMatch(e -> e instanceof VehicleEvent.Location); - assertThat(events).allMatch(e -> e.source() == ProtocolId.GB32960); - assertThat(events).allMatch(e -> e.vin().equals("LTEST000000000002")); - VehicleEvent.Location location = events.stream() - .filter(VehicleEvent.Location.class::isInstance) - .map(VehicleEvent.Location.class::cast) - .findFirst() - .orElseThrow(); - assertThat(location.metadata()).containsEntry("total_mileage_km", "123456.7"); - } - - @Test - void hydrogenLeakAlarmBitProducesCriticalSafetyAlarmEvent() { - long hydrogenLeakFlag = 1L << GeneralAlarmFlagBit.HYDROGEN_LEAK.bitIndex(); - Gb32960Message msg = new Gb32960Message( - new Gb32960Header( - ProtocolVersion.V2025, - CommandType.REALTIME_REPORT, - ResponseFlag.COMMAND, - "LTEST000000000003", - EncryptType.UNENCRYPTED, - 0, - Instant.parse("2026-06-22T01:00:00Z")), - List.of(new InfoBlock.Gb32960V2025.Alarm( - 0, - hydrogenLeakFlag, - List.of(), - List.of(), - List.of(), - List.of(), - List.of()))); - - List events = new Gb32960EventMapper().toEvents(msg); - - VehicleEvent.Alarm alarm = events.stream() - .filter(VehicleEvent.Alarm.class::isInstance) - .map(VehicleEvent.Alarm.class::cast) - .findFirst() - .orElseThrow(); - assertThat(alarm.payload().level()).isEqualTo(AlarmPayload.AlarmLevel.CRITICAL); - assertThat(alarm.payload().safetyCategory()).isEqualTo(AlarmPayload.SafetyCategory.HYDROGEN_LEAK); - assertThat(alarm.payload().hydrogenLeakDetected()).isTrue(); - assertThat(alarm.payload().hydrogenLeakLevel()).isEqualTo(AlarmPayload.HydrogenLeakLevel.CRITICAL); - assertThat(alarm.payload().hydrogenLeakActionRequired()).isTrue(); - assertThat(alarm.payload().activeBits()).contains("HYDROGEN_LEAK"); - } -} diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/README.md b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/README.md deleted file mode 100644 index 1b3628c4..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/README.md +++ /dev/null @@ -1,41 +0,0 @@ -# GB/T 32960 黄金样本集 - -> 本目录用于回放测试与双跑对账。所有样本从旧服务 `lingniu-vehicle-data-reception` 线上抓包后脱敏（VIN 替换为 `LTEST<序号>XXXXXXXX`）。 - -## 文件命名 - -``` -<命令类型>_<场景>_<序号>.hex -``` - -命令类型取自 `CommandType` 枚举： -- `vehicle_login` (0x01) -- `realtime_report` (0x02) -- `resend_report` (0x03) -- `vehicle_logout` (0x04) -- `heartbeat` (0x07) - -## 文件格式 - -每个 `.hex` 文件是一行紧凑十六进制，字节之间**无分隔符**。 -行首允许以 `#` 开头写注释，测试代码忽略注释行与空行。 - -示例 `realtime_report_basic_001.hex`: -``` -# 2017-05-21 10:00:00 宇通 YT01，SOC 70%，车速 52.3 km/h -2323020000010203040506070809101112131415160100352016051510000001010012350000037102710000A064010505050A0005010A00102003E8... -``` - -## 新增样本步骤 - -1. 从生产抓包工具（tcpdump / rg-samples）导出一帧完整字节（含 `0x23 0x23` 起始和 BCC 尾） -2. VIN 替换为测试段：`LTEST0000<9位序号>` -3. 放入本目录，文件名按规范 -4. 在 `Gb32960DecoderGoldenTest` 中添加断言（期望 VIN / 命令 / 关键字段） -5. 运行 `mvn -pl protocol-gb32960 test` - -## 验证目标 - -- 100% 解析一致性（新服务解析结果 === 旧服务解析结果） -- BCC 校验通过 -- 事件映射字段单位一致（车速 km/h、里程 km、经纬度十进制度） diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/heartbeat_001.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/heartbeat_001.hex deleted file mode 100644 index f827bd45..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/heartbeat_001.hex +++ /dev/null @@ -1 +0,0 @@ -232307fe4c54455354303030303030303030303131010000a2 diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/heartbeat_002.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/heartbeat_002.hex deleted file mode 100644 index d9711570..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/heartbeat_002.hex +++ /dev/null @@ -1 +0,0 @@ -232307fe4c54455354303030303030303030303231010000a1 diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_001.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_001.hex deleted file mode 100644 index 5d175a1f..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_001.hex +++ /dev/null @@ -1 +0,0 @@ -232302fe4c544553543030303030303030303030310102101a040d0c213901010302037a000531c2157825fb44012e27103400020101025575304eca6b157a288b030c82050a00aa0002686c02940200000100630101050006c4f414015eedea06018c0f0701310ebf01014a01054907000000000000000000080101157825fb00900001900eea0eea0eea0ee60ee60ee60ef00eef0ef10eeb0eec0eeb0eed0eee0eee0eeb0eed0eea0ed30ed20ed20ed30ed50ed40ecc0ecd0ecc0ece0ece0ece0ed40ed40ed40ed30ed10ed00ed20ed10ed00ece0ece0ecf0ed40ed40ed40eda0eda0edb0ebf0ed40ed30ed60ed60ed50ed40ed50ede0edf0ede0ede0eee0eed0eec0ee80ee80ee50ef00eef0eef0ee80ee90eea0ee40ee70ee80eeb0eeb0eea0eee0eef0eef0ef60ef60ef50ef80ef70ef50ef60ef50ef60eef0ef00ef10ef60ef80ef90efa0ef70ef90efa0ef80ef80efc0efc0efc0ef50ef50ef50ef20ef30ef40ef50ef50ef40ef50ef50ef70ef40ef40ef50ef80ef80ef70ef70ef50ef40ef70ef70ef70efa0efd0efc0ef70ef80ef70ef80ef60ef50f050f070f070ef00eed0ef009010100084a4a4a4a494a4a493001026c08fcffffff0003003a02ee02e402e5000000000031010c800006ffffffff0c80ffffffff0088320c8004c2159002c35333ffffffffff34ffffffffff001c8000096c0c802742ffffffff8300250019000e0b002300020000000000014f00ffffffffffff00000282ffffffff1ffe1fedff1fa0 diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_002.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_002.hex deleted file mode 100644 index 8eb90545..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_002.hex +++ /dev/null @@ -1 +0,0 @@ -232302fe4c544553543030303030303030303030320102ed1a040d0c213a01010302013600036758162826284c012e2710010002010101485bf84de44b165526ec030da9017400d200026a6b023a0200000100b101010402ffff0015050107373d6c01d2e6880601020f8701520efe010141010540070000000000000000000801011628262800900001900f860f870f860f6d0f6c0f6c0f870f860f870f6d0f700f6e0f7e0f7d0f710f640f630f650f6e0f800f7e0f260f250f250f810f810f820f620f660f670f810f820f830f670f680f670f780f770f770f670f660f670f640f630f620f1a0f1c0f1c0f500f500f4d0f690f6b0f690f540f530f5a0f420f440f440f6e0f6e0f6d0f460f450f450f720f6c0f6b0f650f640f670f500f4f0f4c0f150f140f150f780f760f780efe0eff0eff0f7b0f7c0f7d0f640f640f640f6e0f6d0f6e0f360f390f390f5c0f5e0f5c0f650f660f660f7e0f800f800f640f620f640f810f800f800f300f330f2f0f800f820f830f670f660f680f820f840f820f650f670f660f820f840f830f680f660f660f820f810f810f680f660f660f720f720f710f680f670f67090101000841414141404040403001026b092eff00ff000100730ca80c800ca4006c00016c0ca40ca80ca30ca30ca30ca30ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca40ca431010dc00005ffffffff0dc0ffffffff008a320dc000ff15e7009e4833ffffffffff34ffffffffff002f8000096b0dc02715000d000e83002402040100002700080000000000c900000000ffffffffffffffffffffffff1ffe15c8631f18 diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_003.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_003.hex deleted file mode 100644 index b038497c..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_003.hex +++ /dev/null @@ -1 +0,0 @@ -232302fe4c544553543030303030303030303030330102ed1a040d0c2200010203010000000418d914ed27103a020007d0000002010104454e204e204814be2710030006000000b400024a4902bc02000001006701000400ffff0012050006be89500161217106013d0e91012c0e860101470105460700000000000000000008010114ed271000900001900e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e8a0e870e870e880e880e880e880e880e880e880e880e880e870e860e860e880e880e8a0e880e880e880e880e880e880e880e880e880e880e880e880e910e900e900e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e880e88090101000847474747464746463001004907c6ff00ff00000039000000000004006c00016c00000004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400000004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040005000500050005000500050005000531010eb00005ffffffff0eb0ffffffff0081320eb01b3b14e613234733ffffffffff34ffffffffff001d800009490eb02710000d000e83002402040100002700080000000000d500000000ffffffffffffffffffffffff1ffe167c021f7b diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_010.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_010.hex deleted file mode 100644 index 43eab7b3..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_010.hex +++ /dev/null @@ -1 +0,0 @@ -232302fe4c544553543030303030303030303031300102ed1a040d0c220501020301006400049992163027df4f012e27101e00020101015952da51365215f4279f030005000000000002625f02a80200000100a801000400ffff0000050106c3d5cf016240a406010b0f90015b0f4001014a01054907000000000000000000080101163027df00900001900f8c0f8d0f8d0f8d0f8e0f8e0f8f0f8f0f8e0f8f0f900f900f8f0f900f7a0f7c0f800f800f7f0f7c0f7c0f7c0f7e0f7e0f810f7f0f7f0f7e0f820f830f500f500f500f840f850f840f840f820f850f850f820f820f730f750f730f690f6a0f680f620f5f0f600f500f500f500f630f5a0f6d0f6c0f6c0f6d0f500f500f500f6b0f6b0f6c0f6a0f5d0f5f0f5c0f5f0f610f5c0f5e0f5a0f5c0f5c0f610f630f630f640f510f520f540f500f500f500f610f550f500f400f430f410f500f510f500f4e0f4f0f530f500f500f4f0f550f550f560f550f570f560f540f570f560f510f540f530f540f560f5a0f5a0f590f570f7e0f7e0f810f580f4e0f530f570f550f510f500f480f4d0f870f7e0f7e0f7c0f7e0f7f0f7c0f7b0f810f7d0f7e0f7f09010100084a4a4a4a494949493001005f07daff00ff00000038000000000004006c00016c0000000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400000004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000400040004000500050005000500050005000500053101003c0005ffffffff003cffffffff008a32003c00000000ffff4f33ffffffffff34ffffffffff002c8000095f003c2710000d000e830024020401000027000800000000000000000000ffffffffffffffffffffffff1ffe17ae631ff9 diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_020.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_020.hex deleted file mode 100644 index d07198d2..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_020.hex +++ /dev/null @@ -1 +0,0 @@ -232302fe4c544553543030303030303030303032330101d61a040d0c22070101030203010004c1ee17fc261337012e271032000201010157705a4fe26e17f2260c0309d50bba021a00027f6e022b030000010106010105000733a7d401d832920601840eda011c0ebf01044b0101480700000000000000000008010117fc261300a20001a20ecc0eca0ec30ecb0ec90ec70eca0ec90ec20ec20ec40ec20ec70eca0eca0ec90ece0ecd0ec80ecd0eca0ecc0ec90ec80ece0ec70ec50ebf0ed40ed20ed40ed30ed40ed30ed50ed40ed30ed50ed20ed50ed50ed30ecc0ecb0ec80ecf0ec90ed00ec70ecd0ec70ec50ec70ece0eca0eca0ec90eca0ed10ec90ed00ecf0eca0ed10ed30ed10ecf0ed20ed00ed20ed40ed50ed20ed20ed60ed20ed30ed20ed50ed50ed70ed40ed10ed40ed30ed20ed30ed20ecc0ed00ed10ecc0ed30ed10ed00ecb0ec90ecd0ec80ed10ec50ec80ed20ec10ec90ec50ec90ec30ec50ed40ed40ed40ed40ed40ed40ed60ed50ed30ed50ed50ed50ed50ed40ed50ed40ed30ed20ed20ed30ed60ed50eda0ed00ed70ed80ed60ed70ed50ed60ece0ed50ed40ed60ed40ed10ed30ed20ed40ed40ed40ecf0ed80ed60ed50ed60ed70ed60ed60ed40ed50ed70eda0901010020484a4a4b4b4b4b4a484b4b4b4b4b4b4a484a4b4b4b4b4b4a484a4b4b4b4b4b4acc diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_100.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_100.hex deleted file mode 100644 index 25de83e9..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_100.hex +++ /dev/null @@ -1 +0,0 @@ -232302fe4c544553543030303030303030303130370101ae1a040d0c39263001016908fc09c4410003007403200320032001b000c9c803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e8cb diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_200.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_200.hex deleted file mode 100644 index 0069f5b0..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_200.hex +++ /dev/null @@ -1 +0,0 @@ -232302fe4c544553543030303030303030303231310102ed1a040d0c3a120101030200000002533c150727203e01002710000002010104464e204e204814fc2710030004000200be00025353023a01000001010201010402ffff00130500072eb49401de012e06013f0ea7012a0e9301014001033f070000000000000000000801011507272000900001900ea60ea60ea60ea60ea60ea60ea50ea60ea60ea60ea60ea60ea60ea50e960e990e960e980e980e990e970e990e970e960e980e980e990e980e950e970e950e940e950e960e970e940e990e960e960e950e950e930e960e9b0e9c0e9c0e990e9b0e940e950e930e980e9a0e9a0e940e950ea50ea50ea50ea50ea50ea60ea70ea60ea50ea50ea50e9d0e9e0e9e0e9e0e990e970e970e990e980e9a0e9d0e9a0e9c0e9a0e990e9a0e9a0e9b0e9c0e9c0e980e950e980e990e990e990e980e970e970e980e970e970e990e990e9a0e990e990e990e9a0e980e960e960e990e970e970e970e970e980e950e930e9b0e980e9a0e990e9a0e990e940e940e950e940e950e950e990e990e9b0e9a0e9b0e9b0e960e950e960e950e950e950e950e940e99090101000840403f403f3f3f3f300102530708ff00ff00000037000000000000006c00016c00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000031010e880005ffffffff0e88ffffffff008a320e880002150e00014733ffffffffff34ffffffffff0044800009530e882710000d000e830024020401000027000800000000005401000000ffffffffffffffffffffffff1ffe1d7b011fdd diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/resend_001.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/resend_001.hex deleted file mode 100644 index bf30e283..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/resend_001.hex +++ /dev/null @@ -1 +0,0 @@ -232303fe4c544553543030303030303030303032320102101a040d0d0b0a010203010000000532ad156d271c45011007d0006502010103514e204e205c003f27100300070000000000025e5c03b601000001014d0100050106c3aa530160540f0601010ee301120ede01014801054707000000000000000000080101156d271c00900001900ee30ee30ee30ee00ee20ee20ee20ee10ee20ee30ee20ee20ee20ee10ee30ee10ee10ede0ee10ee10ee10ee00ee10ee10ee10ee00ee00ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee10ee1090101000848484848474848473001005c0816ffffff0000003a000a0000000100000000003101003c0005ffffffff003cffffffff008b32003c00000000ffff5233ffffffffff34ffffffffff004c8000095c003c2710ffffffff8300250019000e0b002300020000000000000000ffffffffffff0000000affffffff1ffe1fedff0089 diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/resend_005.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/resend_005.hex deleted file mode 100644 index 55197f67..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/resend_005.hex +++ /dev/null @@ -1 +0,0 @@ -232303fe4c544553543030303030303030303230390101ae1a040d0c391b3001005307ee09c4410037000100000000000001b000c9c803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e803e8be diff --git a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/vehicle_login_001.hex b/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/vehicle_login_001.hex deleted file mode 100644 index 0583697c..00000000 --- a/modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/vehicle_login_001.hex +++ /dev/null @@ -1 +0,0 @@ -232301fe4c5445535430303030303030303030313901001e1a040d0d0b1a000c38393836303332313435323039303735363530390100bd diff --git a/modules/protocols/protocol-jsatl12/pom.xml b/modules/protocols/protocol-jsatl12/pom.xml deleted file mode 100644 index b922c0e2..00000000 --- a/modules/protocols/protocol-jsatl12/pom.xml +++ /dev/null @@ -1,66 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - protocol-jsatl12 - protocol-jsatl12 - - 苏标（JSATL12）主动安全报警附件上传协议。独立 TCP 端口（默认 7612）， - 与 JT808 使用相同的转义规则，但数据帧带 01cd 魔数前缀，用于大文件分块上传。 - - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - ingest-core - - - com.lingniu.ingest - protocol-jt808 - - - com.lingniu.ingest - vehicle-identity - - - com.lingniu.ingest - vehicle-identity-test-support - test - - - com.lingniu.ingest - sink-archive - - - org.springframework.boot - spring-boot-starter - - - io.netty - netty-all - - - com.github.ben-manes.caffeine - caffeine - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12FrameDecoder.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12FrameDecoder.java deleted file mode 100644 index ddf7cd5a..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12FrameDecoder.java +++ /dev/null @@ -1,102 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.codec; - -import io.netty.buffer.ByteBuf; -import io.netty.channel.ChannelHandlerContext; -import io.netty.handler.codec.ByteToMessageDecoder; - -import java.util.List; - -/** - * 苏标报警附件协议分帧。 - * - *

    两种帧并存： - *

      - *
    1. 标准 JT/T 808 帧：以 {@code 0x7e} 为分界，承载 0x1210 / 0x1211 / 0x9212 等信令 - *
    2. 数据帧：以 {@code 0x30 0x31 0x63 0x64}（即 ASCII "01cd"）为魔数，后续 4 字节大端长度 + 数据 - *
    - * - *

    本 decoder 合并处理：根据第一个字节决定分帧策略，输出带类型标记的 {@link Jsatl12RawFrame}。 - */ -public class Jsatl12FrameDecoder extends ByteToMessageDecoder { - - private static final byte[] DATA_MAGIC = {0x30, 0x31, 0x63, 0x64}; - private static final int DEFAULT_MAX_FRAME = 64 * 1024; - private final int maxFrameLength; - - public Jsatl12FrameDecoder() { - this(DEFAULT_MAX_FRAME); - } - - public Jsatl12FrameDecoder(int maxFrameLength) { - this.maxFrameLength = maxFrameLength <= 0 ? DEFAULT_MAX_FRAME : maxFrameLength; - } - - @Override - protected void decode(ChannelHandlerContext ctx, ByteBuf in, List out) { - while (in.readableBytes() >= 1) { - byte first = in.getByte(in.readerIndex()); - if (first == 0x7e) { - if (!decodeJtFrame(in, out)) return; - } else if (first == DATA_MAGIC[0]) { - if (!decodeDataFrame(in, out)) return; - } else { - in.skipBytes(1); - } - } - } - - private boolean decodeJtFrame(ByteBuf in, List out) { - if (in.readableBytes() < 2) return false; - int endIdx = in.indexOf(in.readerIndex() + 1, in.writerIndex(), (byte) 0x7e); - if (endIdx < 0) { - if (in.readableBytes() > maxFrameLength) { - byte[] malformed = new byte[in.readableBytes()]; - in.readBytes(malformed); - out.add(new Jsatl12RawFrame(Jsatl12RawFrame.Type.MALFORMED, malformed)); - return true; - } - return false; - } - int len = endIdx - (in.readerIndex() + 1); - if (len <= 0) { - in.skipBytes(1); - return true; - } - byte[] raw = new byte[len]; - in.skipBytes(1); - in.readBytes(raw); - in.skipBytes(1); - out.add(new Jsatl12RawFrame(Jsatl12RawFrame.Type.JT_MESSAGE, raw)); - return true; - } - - private boolean decodeDataFrame(ByteBuf in, List out) { - if (in.readableBytes() < DATA_MAGIC.length + 4) return false; - for (int i = 0; i < DATA_MAGIC.length; i++) { - if (in.getByte(in.readerIndex() + i) != DATA_MAGIC[i]) { - in.skipBytes(1); - return true; - } - } - if (in.readableBytes() < 62) return false; - // DataPacket 布局：4B magic + 50B 文件名 + 4B offset + 4B length + data - long dataLen = in.getUnsignedInt(in.readerIndex() + 58); - if (dataLen > maxFrameLength) { - int malformedLen = in.readableBytes(); - long declaredTotal = 62L + dataLen; - if (declaredTotal <= Integer.MAX_VALUE && in.readableBytes() >= declaredTotal) { - malformedLen = (int) declaredTotal; - } - byte[] malformed = new byte[malformedLen]; - in.readBytes(malformed); - out.add(new Jsatl12RawFrame(Jsatl12RawFrame.Type.MALFORMED, malformed)); - return true; - } - int totalFrameLen = 62 + (int) dataLen; - if (in.readableBytes() < totalFrameLen) return false; - byte[] frame = new byte[totalFrameLen]; - in.readBytes(frame); - out.add(new Jsatl12RawFrame(Jsatl12RawFrame.Type.DATA_CHUNK, frame)); - return true; - } -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12RawFrame.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12RawFrame.java deleted file mode 100644 index 48444827..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12RawFrame.java +++ /dev/null @@ -1,8 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.codec; - -/** - * JSATL12 原始帧：区分信令帧、数据帧与无法继续解析但需要审计的畸形帧。 - */ -public record Jsatl12RawFrame(Type type, byte[] bytes) { - public enum Type { JT_MESSAGE, DATA_CHUNK, MALFORMED } -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/config/Jsatl12AutoConfiguration.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/config/Jsatl12AutoConfiguration.java deleted file mode 100644 index b1792e78..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/config/Jsatl12AutoConfiguration.java +++ /dev/null @@ -1,58 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.config; - -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.protocol.jsatl12.handler.Jsatl12ArchiveEventHandler; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12AttachmentTracker; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12JtMessageBridge; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12NettyServer; -import com.lingniu.ingest.protocol.jt808.codec.Jt808MessageDecoder; -import com.lingniu.ingest.protocol.jt808.config.Jt808AutoConfiguration; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import com.lingniu.ingest.sink.archive.config.SinkArchiveAutoConfiguration; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; - -@AutoConfiguration(after = {SinkArchiveAutoConfiguration.class, Jt808AutoConfiguration.class}) -@EnableConfigurationProperties(Jsatl12Properties.class) -@ConditionalOnProperty(prefix = "lingniu.ingest.jsatl12", name = "enabled", havingValue = "true") -@ConditionalOnBean({ArchiveStore.class, Jt808MessageDecoder.class}) -public class Jsatl12AutoConfiguration { - - @Bean - @ConditionalOnMissingBean - public Jsatl12AttachmentTracker jsatl12AttachmentTracker() { - return new Jsatl12AttachmentTracker(); - } - - @Bean - @ConditionalOnMissingBean - public Jsatl12JtMessageBridge jsatl12JtMessageBridge(Jt808MessageDecoder decoder, - VehicleIdentityResolver identityResolver, - Jsatl12AttachmentTracker attachmentTracker, - Dispatcher dispatcher) { - return new Jsatl12JtMessageBridge(decoder, identityResolver, attachmentTracker, dispatcher::dispatch); - } - - @Bean - @ConditionalOnMissingBean - public Jsatl12ArchiveEventHandler jsatl12ArchiveEventHandler() { - return new Jsatl12ArchiveEventHandler(); - } - - @Bean - @ConditionalOnMissingBean - public Jsatl12NettyServer jsatl12NettyServer(Jsatl12Properties props, - ArchiveStore archive, - Jsatl12JtMessageBridge jtMessageBridge, - Jsatl12AttachmentTracker attachmentTracker, - VehicleIdentityResolver identityResolver, - Dispatcher dispatcher) { - return new Jsatl12NettyServer(props.getPort(), props.getWorkerThreads(), - props.getMaxFrameLength(), archive, jtMessageBridge, attachmentTracker, identityResolver, dispatcher::dispatch); - } -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/config/Jsatl12Properties.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/config/Jsatl12Properties.java deleted file mode 100644 index b31a65de..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/config/Jsatl12Properties.java +++ /dev/null @@ -1,25 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.config; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -/** JSATL12 主动安全附件接入配置。 */ -@ConfigurationProperties(prefix = "lingniu.ingest.jsatl12") -public class Jsatl12Properties { - private boolean enabled = false; - private int port = 7612; - private int workerThreads = 0; - private int maxFrameLength = 64 * 1024; - /** 附件落盘根目录，支持 file:// / s3:// / oss:// 形式，由 sink-archive 解析。 */ - private String fileStore = "file:///var/lingniu/alarm-files"; - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public int getPort() { return port; } - public void setPort(int port) { this.port = port; } - public int getWorkerThreads() { return workerThreads; } - public void setWorkerThreads(int workerThreads) { this.workerThreads = workerThreads; } - public int getMaxFrameLength() { return maxFrameLength; } - public void setMaxFrameLength(int maxFrameLength) { this.maxFrameLength = maxFrameLength; } - public String getFileStore() { return fileStore; } - public void setFileStore(String fileStore) { this.fileStore = fileStore; } -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/handler/Jsatl12ArchiveEventHandler.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/handler/Jsatl12ArchiveEventHandler.java deleted file mode 100644 index 7f3d79e4..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/handler/Jsatl12ArchiveEventHandler.java +++ /dev/null @@ -1,137 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.handler; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.annotation.EventEmit; -import com.lingniu.ingest.api.annotation.MessageMapping; -import com.lingniu.ingest.api.annotation.ProtocolHandler; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12ArchiveFailure; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12ArchivedChunk; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12MalformedFrame; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12MalformedJtMessage; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12MessageId; - -import java.time.Instant; -import java.util.List; -import java.util.Map; -import java.util.UUID; - -@ProtocolHandler(protocol = ProtocolId.JSATL12) -public final class Jsatl12ArchiveEventHandler { - - @MessageMapping(command = Jsatl12MessageId.DATA_CHUNK_ARCHIVED, desc = "主动安全附件数据块已归档") - @EventEmit(VehicleEvent.MediaMeta.class) - public List onDataChunkArchived(Jsatl12ArchivedChunk chunk) { - Instant eventTime = chunk.archivedAt() == null ? Instant.now() : chunk.archivedAt(); - String peer = chunk.peer() == null || chunk.peer().isBlank() ? "unknown" : chunk.peer(); - return List.of(new VehicleEvent.MediaMeta( - UUID.randomUUID().toString(), - nullToUnknown(chunk.vin()), - ProtocolId.JSATL12, - eventTime, - Instant.now(), - null, - baseMeta( - "vin", nullToUnknown(chunk.vin()), - "phone", nullToEmpty(chunk.phone()), - "identityResolved", Boolean.toString(chunk.identityResolved()), - "identitySource", nullToEmpty(chunk.identitySource()), - "peer", peer, - "archiveKey", nullToEmpty(chunk.archiveKey()), - "archiveRef", nullToEmpty(chunk.archiveRef()), - "fileName", nullToEmpty(chunk.fileName()), - "fileOffset", Long.toString(Math.max(0, chunk.fileOffset())), - "declaredChunkSizeBytes", Long.toString(Math.max(0, chunk.declaredSizeBytes())), - "chunkSizeBytes", Long.toString(Math.max(0, chunk.sizeBytes()))), - "jsatl12-" + peer + "-" + eventTime.toEpochMilli(), - "jsatl12-data-chunk", - Math.max(0, chunk.sizeBytes()), - nullToEmpty(chunk.archiveRef()))); - } - - @MessageMapping(command = Jsatl12MessageId.MALFORMED_JT_MESSAGE, desc = "主动安全 JT 消息解析失败兜底") - @EventEmit(VehicleEvent.Passthrough.class) - public List onMalformedJtMessage(Jsatl12MalformedJtMessage malformed) { - Instant eventTime = malformed.receivedAt() == null ? Instant.now() : malformed.receivedAt(); - String peer = malformed.peer() == null || malformed.peer().isBlank() ? "unknown" : malformed.peer(); - return List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - "unknown", - ProtocolId.JSATL12, - eventTime, - Instant.now(), - null, - baseMeta( - "peer", peer, - "source", "jsatl12-jt-message", - "parseError", "true", - "parseErrorMessage", nullToEmpty(malformed.errorMessage())), - Jsatl12MessageId.MALFORMED_JT_MESSAGE, - malformed.rawBytes() == null ? new byte[0] : malformed.rawBytes())); - } - - @MessageMapping(command = Jsatl12MessageId.DATA_CHUNK_ARCHIVE_FAILED, desc = "主动安全附件数据块归档失败兜底") - @EventEmit(VehicleEvent.Passthrough.class) - public List onArchiveFailure(Jsatl12ArchiveFailure failure) { - Instant eventTime = failure.failedAt() == null ? Instant.now() : failure.failedAt(); - String peer = failure.peer() == null || failure.peer().isBlank() ? "unknown" : failure.peer(); - return List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - "unknown", - ProtocolId.JSATL12, - eventTime, - Instant.now(), - null, - baseMeta( - "peer", peer, - "source", "jsatl12-data-chunk", - "archiveError", "true", - "archiveErrorMessage", nullToEmpty(failure.errorMessage()), - "archiveKey", nullToEmpty(failure.archiveKey()), - "chunkSizeBytes", Long.toString(Math.max(0, failure.sizeBytes()))), - Jsatl12MessageId.DATA_CHUNK_ARCHIVE_FAILED, - failure.rawBytes() == null ? new byte[0] : failure.rawBytes())); - } - - @MessageMapping(command = Jsatl12MessageId.MALFORMED_FRAME, desc = "主动安全附件帧边界异常兜底") - @EventEmit(VehicleEvent.Passthrough.class) - public List onMalformedFrame(Jsatl12MalformedFrame malformed) { - Instant eventTime = malformed.receivedAt() == null ? Instant.now() : malformed.receivedAt(); - String peer = malformed.peer() == null || malformed.peer().isBlank() ? "unknown" : malformed.peer(); - return List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - "unknown", - ProtocolId.JSATL12, - eventTime, - Instant.now(), - null, - baseMeta( - "peer", peer, - "source", "jsatl12-frame", - "frameError", "true", - "frameErrorMessage", nullToEmpty(malformed.errorMessage()), - "parseError", "true", - "parseErrorMessage", nullToEmpty(malformed.errorMessage())), - Jsatl12MessageId.MALFORMED_FRAME, - malformed.rawBytes() == null ? new byte[0] : malformed.rawBytes())); - } - - private static Map baseMeta(String... kv) { - java.util.HashMap out = new java.util.HashMap<>(); - out.put("vin", "unknown"); - out.put("identityResolved", "false"); - out.put("identitySource", "UNKNOWN"); - for (int i = 0; i + 1 < kv.length; i += 2) { - out.put(kv[i], kv[i + 1] == null ? "" : kv[i + 1]); - } - return Map.copyOf(out); - } - - private static String nullToEmpty(String value) { - return value == null ? "" : value; - } - - private static String nullToUnknown(String value) { - return value == null || value.isBlank() ? "unknown" : value; - } -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ArchiveFailure.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ArchiveFailure.java deleted file mode 100644 index d2271739..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ArchiveFailure.java +++ /dev/null @@ -1,12 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import java.time.Instant; - -public record Jsatl12ArchiveFailure( - byte[] rawBytes, - String archiveKey, - long sizeBytes, - String peer, - String errorMessage, - Instant failedAt -) {} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ArchivedChunk.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ArchivedChunk.java deleted file mode 100644 index c3db11f2..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ArchivedChunk.java +++ /dev/null @@ -1,45 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import java.time.Instant; - -public record Jsatl12ArchivedChunk( - String archiveRef, - String archiveKey, - String fileName, - long fileOffset, - long declaredSizeBytes, - long sizeBytes, - String peer, - Instant archivedAt, - String phone, - String vin, - boolean identityResolved, - String identitySource -) { - public Jsatl12ArchivedChunk { - phone = phone == null ? "" : phone.trim(); - vin = vin == null || vin.isBlank() ? "unknown" : vin.trim(); - identitySource = identitySource == null || identitySource.isBlank() ? "UNKNOWN" : identitySource.trim(); - } - - public Jsatl12ArchivedChunk(String archiveRef, - String archiveKey, - String fileName, - long fileOffset, - long declaredSizeBytes, - long sizeBytes, - String peer, - Instant archivedAt) { - this(archiveRef, archiveKey, fileName, fileOffset, declaredSizeBytes, sizeBytes, peer, archivedAt, - "", "unknown", false, "UNKNOWN"); - } - - public Jsatl12ArchivedChunk(String archiveRef, - String archiveKey, - long sizeBytes, - String peer, - Instant archivedAt) { - this(archiveRef, archiveKey, "", 0, sizeBytes, sizeBytes, peer, archivedAt, - "", "unknown", false, "UNKNOWN"); - } -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12AttachmentTracker.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12AttachmentTracker.java deleted file mode 100644 index 27565499..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12AttachmentTracker.java +++ /dev/null @@ -1,165 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.HashSet; -import java.util.List; -import java.util.Map; -import java.util.Set; - -public final class Jsatl12AttachmentTracker { - - private final Map> filesByPhone = new HashMap<>(); - - public synchronized void registerAttachmentList(String phone, byte[] body) { - String key = normalizePhone(phone); - byte[] raw = body == null ? new byte[0] : body; - if (raw.length < 57) { - throw new IllegalArgumentException("jsatl12 T1210 body too short: " + raw.length); - } - int total = raw[56] & 0xFF; - int pos = 57; - Map files = filesByPhone.computeIfAbsent(key, ignored -> new HashMap<>()); - for (int i = 0; i < total && pos < raw.length; i++) { - int nameLen = raw[pos++] & 0xFF; - if (pos + nameLen + 4 > raw.length) { - throw new IllegalArgumentException("jsatl12 T1210 item truncated at index " + i); - } - String name = new String(raw, pos, nameLen, StandardCharsets.UTF_8).trim(); - pos += nameLen; - long size = Jsatl12DataPacket.readDword(raw, pos); - pos += 4; - files.put(name, new FileState(name, 0, size)); - } - } - - public synchronized Jsatl12DataPacket recordDataPacket(String phone, byte[] rawPacket) { - Jsatl12DataPacket packet = Jsatl12DataPacket.parse(rawPacket); - String key = normalizePhone(phone); - Map files = filesByPhone.computeIfAbsent(key, ignored -> new HashMap<>()); - FileState state = files.computeIfAbsent(packet.name(), - ignored -> new FileState(packet.name(), 0, packet.offset() + packet.length())); - state.ranges.add(new Range(packet.offset(), packet.offset() + packet.length())); - return packet; - } - - public synchronized Jsatl12UploadCompletionAck completeUpload(String phone, byte[] body) { - T1212 completion = parseT1212(body); - String key = normalizePhone(phone); - Map files = filesByPhone.computeIfAbsent(key, ignored -> new HashMap<>()); - FileState state = files.computeIfAbsent(completion.name(), - ignored -> new FileState(completion.name(), completion.type(), completion.size())); - mergeUnknownFileInto(key, state); - state.type = completion.type(); - state.size = completion.size(); - - List missing = missingRanges(state); - if (missing.isEmpty()) { - files.remove(completion.name()); - if (files.isEmpty()) { - filesByPhone.remove(key); - } - return new Jsatl12UploadCompletionAck(completion.name(), completion.type(), 0, List.of()); - } - return new Jsatl12UploadCompletionAck(completion.name(), completion.type(), 1, missing); - } - - private void mergeUnknownFileInto(String ownerKey, FileState target) { - if ("unknown".equals(ownerKey)) { - return; - } - Map unknownFiles = filesByPhone.get("unknown"); - if (unknownFiles == null) { - return; - } - FileState unknown = unknownFiles.remove(target.name); - if (unknown == null) { - return; - } - target.ranges.addAll(unknown.ranges); - if (unknownFiles.isEmpty()) { - filesByPhone.remove("unknown"); - } - } - - public synchronized Set knownFiles(String phone) { - Map files = filesByPhone.get(normalizePhone(phone)); - if (files == null) { - return Set.of(); - } - return Set.copyOf(new HashSet<>(files.keySet())); - } - - public synchronized String ownerPhone(String fileName) { - String name = fileName == null ? "" : fileName.trim(); - if (name.isBlank()) { - return ""; - } - for (Map.Entry> entry : filesByPhone.entrySet()) { - if (!"unknown".equals(entry.getKey()) && entry.getValue().containsKey(name)) { - return entry.getKey(); - } - } - return ""; - } - - private static List missingRanges(FileState state) { - List ranges = new ArrayList<>(state.ranges); - ranges.sort((a, b) -> Long.compare(a.start, b.start)); - List missing = new ArrayList<>(); - long cursor = 0; - for (Range range : ranges) { - if (range.end <= cursor) { - continue; - } - if (range.start > cursor) { - missing.add(new Jsatl12UploadCompletionAck.Range(cursor, range.start - cursor)); - } - cursor = Math.max(cursor, range.end); - if (cursor >= state.size) { - break; - } - } - if (cursor < state.size) { - missing.add(new Jsatl12UploadCompletionAck.Range(cursor, state.size - cursor)); - } - return List.copyOf(missing); - } - - private static T1212 parseT1212(byte[] body) { - byte[] raw = body == null ? new byte[0] : body; - if (raw.length < 6) { - throw new IllegalArgumentException("jsatl12 T1212 body too short: " + raw.length); - } - int nameLen = raw[0] & 0xFF; - if (raw.length < 1 + nameLen + 1 + 4) { - throw new IllegalArgumentException("jsatl12 T1212 body truncated"); - } - String name = new String(raw, 1, nameLen, StandardCharsets.UTF_8).trim(); - int type = raw[1 + nameLen] & 0xFF; - long size = Jsatl12DataPacket.readDword(raw, 2 + nameLen); - return new T1212(name, type, size); - } - - private static String normalizePhone(String phone) { - return phone == null || phone.isBlank() ? "unknown" : phone; - } - - private static final class FileState { - private final String name; - private final List ranges = new ArrayList<>(); - private int type; - private long size; - - private FileState(String name, int type, long size) { - this.name = name; - this.type = type; - this.size = Math.max(0, size); - } - } - - private record Range(long start, long end) {} - - private record T1212(String name, int type, long size) {} -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ChannelHandler.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ChannelHandler.java deleted file mode 100644 index 1768ce1b..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ChannelHandler.java +++ /dev/null @@ -1,249 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import com.github.benmanes.caffeine.cache.Cache; -import com.github.benmanes.caffeine.cache.Caffeine; -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentityLookup; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.protocol.jsatl12.codec.Jsatl12RawFrame; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import io.netty.channel.ChannelHandlerContext; -import io.netty.channel.SimpleChannelInboundHandler; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.IOException; -import java.net.InetSocketAddress; -import java.time.Duration; -import java.time.Instant; -import java.time.ZoneId; -import java.time.format.DateTimeFormatter; -import java.util.HashMap; -import java.util.Map; -import java.util.function.Consumer; - -/** - * JSATL12 入站处理器。 - * - *

    当前实现：把 DATA_CHUNK 帧写入 {@link ArchiveStore} 后派发统一事件引用，文件名按日期 + 对端 IP + - * 时间戳生成；JT_MESSAGE 帧复用 JT808 decoder 后进入统一 Dispatcher。 - * - *

    路径模板：{@code yyyy/MM/dd//.bin}。 - */ -public class Jsatl12ChannelHandler extends SimpleChannelInboundHandler { - - private static final Logger log = LoggerFactory.getLogger(Jsatl12ChannelHandler.class); - private static final DateTimeFormatter DAY_FMT = DateTimeFormatter.ofPattern("yyyy/MM/dd").withZone(ZoneId.systemDefault()); - - /** 为每个 Channel 持久化一个文件 key，保证同一次传输追加到同一个文件。 */ - private static final Cache KEYS = Caffeine.newBuilder() - .expireAfterAccess(Duration.ofMinutes(30)) - .maximumSize(10_000) - .build(); - - private final ArchiveStore archive; - private final Jsatl12JtMessageBridge jtMessageBridge; - private final Jsatl12AttachmentTracker attachmentTracker; - private final VehicleIdentityResolver identityResolver; - private final Consumer dataChunkDispatcher; - - public Jsatl12ChannelHandler(ArchiveStore archive, Jsatl12JtMessageBridge jtMessageBridge) { - this(archive, jtMessageBridge, new Jsatl12AttachmentTracker(), ignored -> {}); - } - - public Jsatl12ChannelHandler(ArchiveStore archive, - Jsatl12JtMessageBridge jtMessageBridge, - Consumer dataChunkDispatcher) { - this(archive, jtMessageBridge, new Jsatl12AttachmentTracker(), null, dataChunkDispatcher); - } - - public Jsatl12ChannelHandler(ArchiveStore archive, - Jsatl12JtMessageBridge jtMessageBridge, - Jsatl12AttachmentTracker attachmentTracker, - Consumer dataChunkDispatcher) { - this(archive, jtMessageBridge, attachmentTracker, null, dataChunkDispatcher); - } - - public Jsatl12ChannelHandler(ArchiveStore archive, - Jsatl12JtMessageBridge jtMessageBridge, - Jsatl12AttachmentTracker attachmentTracker, - VehicleIdentityResolver identityResolver, - Consumer dataChunkDispatcher) { - this.archive = archive; - this.jtMessageBridge = jtMessageBridge; - this.attachmentTracker = attachmentTracker == null ? new Jsatl12AttachmentTracker() : attachmentTracker; - this.identityResolver = identityResolver; - this.dataChunkDispatcher = dataChunkDispatcher; - } - - @Override - protected void channelRead0(ChannelHandlerContext ctx, Jsatl12RawFrame frame) { - switch (frame.type()) { - case JT_MESSAGE -> { - jtMessageBridge.dispatch(frame.bytes(), addr(ctx)) - .ifPresent(response -> ctx.writeAndFlush(io.netty.buffer.Unpooled.wrappedBuffer(response))); - } - case MALFORMED -> { - byte[] raw = frame.bytes() == null ? new byte[0] : frame.bytes(); - Instant now = Instant.now(); - String peer = addr(ctx); - String errorMessage = "malformed jsatl12 frame"; - Jsatl12MalformedFrame malformed = new Jsatl12MalformedFrame(raw, peer, errorMessage, now); - dataChunkDispatcher.accept(new RawFrame( - ProtocolId.JSATL12, - Jsatl12MessageId.MALFORMED_FRAME, - 0, - malformed, - raw, - unresolvedMeta( - "source", "jsatl12-frame", - "peer", peer, - "frameError", "true", - "frameErrorMessage", errorMessage, - "frameSizeBytes", Integer.toString(raw.length)), - now)); - } - case DATA_CHUNK -> { - String key = KEYS.get(ctx, k -> { - String day = DAY_FMT.format(Instant.now()); - return "jsatl12/" + day + "/" + addr(ctx).replace(':', '_') - + "/" + Instant.now().toEpochMilli() + ".bin"; - }); - try { - byte[] all = frame.bytes(); - Jsatl12DataPacket packet = attachmentTracker.recordDataPacket(null, all); - byte[] payload = packet.data(); - int payloadLen = payload.length; - if (payloadLen > 0) { - String phone = attachmentTracker.ownerPhone(packet.name()); - IdentityResolution identity = resolveIdentity(phone); - String archiveRef = archive.append(key, payload); - Instant now = Instant.now(); - Jsatl12ArchivedChunk archived = new Jsatl12ArchivedChunk( - archiveRef, - key, - packet.name(), - packet.offset(), - packet.length(), - payloadLen, - addr(ctx), - now, - phone, - identity.identity().vin(), - identity.identity().resolved(), - identity.identity().source().name()); - dataChunkDispatcher.accept(new RawFrame( - ProtocolId.JSATL12, - Jsatl12MessageId.DATA_CHUNK_ARCHIVED, - 0, - archived, - null, - identityMeta(identity, - "phone", phone, - "source", "jsatl12-data-chunk", - "peer", addr(ctx), - "fileName", packet.name(), - "fileOffset", Long.toString(packet.offset()), - "declaredChunkSizeBytes", Long.toString(packet.length()), - "archiveRef", archiveRef, - "archiveKey", key, - "chunkSizeBytes", Integer.toString(payloadLen)), - now)); - } - } catch (IOException | RuntimeException e) { - log.warn("[jsatl12] write chunk failed key={}", key, e); - byte[] all = frame.bytes(); - int payloadLen = Math.max(0, all.length - 62); - Instant now = Instant.now(); - Jsatl12ArchiveFailure failure = new Jsatl12ArchiveFailure( - all, - key, - payloadLen, - addr(ctx), - e.getMessage() == null ? "" : e.getMessage(), - now); - dataChunkDispatcher.accept(new RawFrame( - ProtocolId.JSATL12, - Jsatl12MessageId.DATA_CHUNK_ARCHIVE_FAILED, - 0, - failure, - all, - unresolvedMeta( - "source", "jsatl12-data-chunk", - "peer", addr(ctx), - "archiveKey", key, - "archiveError", "true", - "archiveErrorMessage", failure.errorMessage(), - "chunkSizeBytes", Long.toString(payloadLen)), - now)); - } - } - } - } - - @Override - public void channelInactive(ChannelHandlerContext ctx) { - String key = KEYS.getIfPresent(ctx); - if (key != null) { - log.info("[jsatl12] channel closed, archived key={}", key); - KEYS.invalidate(ctx); - } - } - - private static Map unresolvedMeta(String... kv) { - HashMap out = new HashMap<>(); - out.put("vin", "unknown"); - out.put("identityResolved", "false"); - out.put("identitySource", "UNKNOWN"); - for (int i = 0; i + 1 < kv.length; i += 2) { - out.put(kv[i], kv[i + 1] == null ? "" : kv[i + 1]); - } - return Map.copyOf(out); - } - - private IdentityResolution resolveIdentity(String phone) { - if (identityResolver != null) { - try { - return new IdentityResolution( - identityResolver.resolve(new VehicleIdentityLookup(ProtocolId.JT808, "", phone, "", "")), - null); - } catch (RuntimeException e) { - return new IdentityResolution( - new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN), - e.getMessage() == null ? "" : e.getMessage()); - } - } - if (phone != null && !phone.isBlank()) { - return new IdentityResolution(new VehicleIdentity(phone, false, VehicleIdentitySource.FALLBACK_PHONE), null); - } - return new IdentityResolution(new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN), null); - } - - private static Map identityMeta(IdentityResolution resolution, String... kv) { - VehicleIdentity identity = resolution.identity(); - HashMap out = new HashMap<>(); - out.put("vin", identity.vin()); - out.put("identityResolved", Boolean.toString(identity.resolved())); - out.put("identitySource", identity.source().name()); - if (resolution.errorMessage() != null) { - out.put("identityError", "true"); - out.put("identityErrorMessage", resolution.errorMessage()); - } - for (int i = 0; i + 1 < kv.length; i += 2) { - out.put(kv[i], kv[i + 1] == null ? "" : kv[i + 1]); - } - return Map.copyOf(out); - } - - private static String addr(ChannelHandlerContext ctx) { - if (ctx.channel().remoteAddress() instanceof InetSocketAddress i) { - return i.getAddress().getHostAddress() + ":" + i.getPort(); - } - return "unknown"; - } - - private record IdentityResolution(VehicleIdentity identity, String errorMessage) {} -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12DataPacket.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12DataPacket.java deleted file mode 100644 index ba76c156..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12DataPacket.java +++ /dev/null @@ -1,66 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import java.nio.charset.StandardCharsets; -import java.util.Arrays; - -public record Jsatl12DataPacket(String name, long offset, long length, byte[] data) { - - private static final int HEADER_LENGTH = 62; - - public Jsatl12DataPacket { - name = name == null ? "" : name.trim(); - data = data == null ? new byte[0] : Arrays.copyOf(data, data.length); - if (offset < 0) { - throw new IllegalArgumentException("jsatl12 data packet offset must not be negative"); - } - if (length < 0) { - throw new IllegalArgumentException("jsatl12 data packet length must not be negative"); - } - if (length != data.length) { - throw new IllegalArgumentException("jsatl12 data packet length " + length - + " does not match data length " + data.length); - } - } - - @Override - public byte[] data() { - return Arrays.copyOf(data, data.length); - } - - public static Jsatl12DataPacket parse(byte[] bytes) { - byte[] raw = bytes == null ? new byte[0] : bytes; - if (raw.length < HEADER_LENGTH) { - throw new IllegalArgumentException("jsatl12 data packet too short: " + raw.length); - } - if (raw[0] != 0x30 || raw[1] != 0x31 || raw[2] != 0x63 || raw[3] != 0x64) { - throw new IllegalArgumentException("jsatl12 data packet magic mismatch"); - } - String name = readFixedString(raw, 4, 50); - long offset = readDword(raw, 54); - long length = readDword(raw, 58); - if (length > Integer.MAX_VALUE) { - throw new IllegalArgumentException("jsatl12 data packet length too large: " + length); - } - if (raw.length < HEADER_LENGTH + (int) length) { - throw new IllegalArgumentException("jsatl12 data packet body truncated: " + raw.length); - } - byte[] data = Arrays.copyOfRange(raw, HEADER_LENGTH, HEADER_LENGTH + (int) length); - return new Jsatl12DataPacket(name, offset, length, data); - } - - private static String readFixedString(byte[] raw, int offset, int length) { - int end = offset; - int max = Math.min(raw.length, offset + length); - while (end < max && raw[end] != 0) { - end++; - } - return new String(raw, offset, end - offset, StandardCharsets.UTF_8).trim(); - } - - static long readDword(byte[] raw, int offset) { - return ((raw[offset] & 0xFFL) << 24) - | ((raw[offset + 1] & 0xFFL) << 16) - | ((raw[offset + 2] & 0xFFL) << 8) - | (raw[offset + 3] & 0xFFL); - } -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12JtMessageBridge.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12JtMessageBridge.java deleted file mode 100644 index 60bb4797..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12JtMessageBridge.java +++ /dev/null @@ -1,184 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentityLookup; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.protocol.jt808.codec.Jt808Escape; -import com.lingniu.ingest.protocol.jt808.codec.Jt808FrameEncoder; -import com.lingniu.ingest.protocol.jt808.codec.Jt808MessageDecoder; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; - -import java.io.ByteArrayOutputStream; -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.time.Instant; -import java.util.HashMap; -import java.util.Map; -import java.util.Optional; -import java.util.function.Consumer; - -public final class Jsatl12JtMessageBridge { - - private final Jt808MessageDecoder decoder; - private final VehicleIdentityResolver identityResolver; - private final Jsatl12AttachmentTracker attachmentTracker; - private final Consumer dispatcher; - - public Jsatl12JtMessageBridge(Jt808MessageDecoder decoder, Consumer dispatcher) { - this(decoder, (VehicleIdentityResolver) null, dispatcher); - } - - public Jsatl12JtMessageBridge(Jt808MessageDecoder decoder, - VehicleIdentityResolver identityResolver, - Consumer dispatcher) { - this(decoder, identityResolver, new Jsatl12AttachmentTracker(), dispatcher); - } - - public Jsatl12JtMessageBridge(Jt808MessageDecoder decoder, - Jsatl12AttachmentTracker attachmentTracker, - Consumer dispatcher) { - this(decoder, null, attachmentTracker, dispatcher); - } - - public Jsatl12JtMessageBridge(Jt808MessageDecoder decoder, - VehicleIdentityResolver identityResolver, - Jsatl12AttachmentTracker attachmentTracker, - Consumer dispatcher) { - this.decoder = decoder; - this.identityResolver = identityResolver; - this.attachmentTracker = attachmentTracker == null ? new Jsatl12AttachmentTracker() : attachmentTracker; - this.dispatcher = dispatcher; - } - - public Optional dispatch(byte[] jtMessageBytes, String peer) { - Instant receivedAt = Instant.now(); - String peerValue = peer == null ? "" : peer; - byte[] raw = jtMessageBytes == null ? new byte[0] : jtMessageBytes; - byte[] unescaped; - Jt808Message message; - try { - unescaped = Jt808Escape.unescape(raw, 0, raw.length); - message = decoder.decode(ByteBuffer.wrap(unescaped)); - } catch (RuntimeException e) { - dispatchMalformed(raw, peerValue, e, receivedAt); - return Optional.empty(); - } - try { - IdentityResolution identity = resolveIdentity(message.header().phone()); - Map meta = new HashMap<>(); - meta.put("vin", identity.identity().vin()); - meta.put("phone", message.header().phone()); - meta.put("identityResolved", Boolean.toString(identity.identity().resolved())); - meta.put("identitySource", identity.identity().source().name()); - if (identity.errorMessage() != null) { - meta.put("identityError", "true"); - meta.put("identityErrorMessage", identity.errorMessage()); - } - meta.put("peer", peerValue); - meta.put("source", "jsatl12-signaling"); - dispatcher.accept(new RawFrame( - ProtocolId.JT808, - message.header().messageId(), - 0, - message, - unescaped, - meta, - receivedAt)); - return handleJsatl12Signaling(message); - } catch (RuntimeException e) { - dispatchMalformed(raw, peerValue, e, receivedAt); - return Optional.empty(); - } - } - - private Optional handleJsatl12Signaling(Jt808Message message) { - if (!(message.body() instanceof Jt808Body.Raw rawBody)) { - return Optional.empty(); - } - if (message.header().messageId() == 0x1210) { - attachmentTracker.registerAttachmentList(message.header().phone(), rawBody.bytes()); - return Optional.empty(); - } - if (message.header().messageId() == 0x1212) { - Jsatl12UploadCompletionAck ack = - attachmentTracker.completeUpload(message.header().phone(), rawBody.bytes()); - return Optional.of(Jt808FrameEncoder.encode( - 0x9212, - message.header().phone(), - message.header().serialNo(), - encodeAckBody(ack), - message.header().version())); - } - return Optional.empty(); - } - - private static byte[] encodeAckBody(Jsatl12UploadCompletionAck ack) { - byte[] name = ack.name().getBytes(StandardCharsets.UTF_8); - if (name.length > 255) { - throw new IllegalArgumentException("jsatl12 completion ack filename too long: " + ack.name()); - } - ByteArrayOutputStream out = new ByteArrayOutputStream(4 + name.length + ack.missingRanges().size() * 8); - out.write(name.length); - out.write(name, 0, name.length); - out.write(ack.type() & 0xFF); - out.write(ack.result() & 0xFF); - out.write(ack.missingRanges().size() & 0xFF); - for (Jsatl12UploadCompletionAck.Range range : ack.missingRanges()) { - writeDword(out, range.offset()); - writeDword(out, range.length()); - } - return out.toByteArray(); - } - - private static void writeDword(ByteArrayOutputStream out, long value) { - if (value < 0 || value > 0xFFFF_FFFFL) { - throw new IllegalArgumentException("jsatl12 dword out of range: " + value); - } - out.write((int) ((value >>> 24) & 0xFF)); - out.write((int) ((value >>> 16) & 0xFF)); - out.write((int) ((value >>> 8) & 0xFF)); - out.write((int) (value & 0xFF)); - } - - private IdentityResolution resolveIdentity(String phone) { - if (identityResolver == null) { - return new IdentityResolution( - new VehicleIdentity(phone, false, VehicleIdentitySource.FALLBACK_PHONE), - null); - } - try { - return new IdentityResolution( - identityResolver.resolve(new VehicleIdentityLookup(ProtocolId.JT808, "", phone, "", "")), - null); - } catch (RuntimeException e) { - return new IdentityResolution( - new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN), - e.getMessage() == null ? "" : e.getMessage()); - } - } - - private void dispatchMalformed(byte[] raw, String peer, RuntimeException error, Instant receivedAt) { - Map meta = new HashMap<>(); - meta.put("vin", "unknown"); - meta.put("identityResolved", "false"); - meta.put("identitySource", VehicleIdentitySource.UNKNOWN.name()); - meta.put("peer", peer); - meta.put("source", "jsatl12-jt-message"); - meta.put("parseError", "true"); - meta.put("parseErrorMessage", error.getMessage() == null ? "" : error.getMessage()); - dispatcher.accept(new RawFrame( - ProtocolId.JSATL12, - Jsatl12MessageId.MALFORMED_JT_MESSAGE, - 0, - new Jsatl12MalformedJtMessage(raw, peer, error.getMessage(), receivedAt), - raw, - meta, - receivedAt)); - } - - private record IdentityResolution(VehicleIdentity identity, String errorMessage) {} -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12MalformedFrame.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12MalformedFrame.java deleted file mode 100644 index fff586e3..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12MalformedFrame.java +++ /dev/null @@ -1,10 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import java.time.Instant; - -public record Jsatl12MalformedFrame( - byte[] rawBytes, - String peer, - String errorMessage, - Instant receivedAt -) {} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12MalformedJtMessage.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12MalformedJtMessage.java deleted file mode 100644 index 3814155f..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12MalformedJtMessage.java +++ /dev/null @@ -1,10 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import java.time.Instant; - -public record Jsatl12MalformedJtMessage( - byte[] rawBytes, - String peer, - String errorMessage, - Instant receivedAt -) {} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12MessageId.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12MessageId.java deleted file mode 100644 index e170150d..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12MessageId.java +++ /dev/null @@ -1,11 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -public final class Jsatl12MessageId { - - public static final int DATA_CHUNK_ARCHIVED = 0x30316364; - public static final int MALFORMED_JT_MESSAGE = 0x30316A65; - public static final int DATA_CHUNK_ARCHIVE_FAILED = 0x30316665; - public static final int MALFORMED_FRAME = 0x30316672; - - private Jsatl12MessageId() {} -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12NettyServer.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12NettyServer.java deleted file mode 100644 index d765de4b..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12NettyServer.java +++ /dev/null @@ -1,111 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.protocol.jsatl12.codec.Jsatl12FrameDecoder; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import io.netty.bootstrap.ServerBootstrap; -import io.netty.channel.Channel; -import io.netty.channel.ChannelInitializer; -import io.netty.channel.ChannelOption; -import io.netty.channel.EventLoopGroup; -import io.netty.channel.nio.NioEventLoopGroup; -import io.netty.channel.socket.SocketChannel; -import io.netty.channel.socket.nio.NioServerSocketChannel; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.beans.factory.DisposableBean; -import org.springframework.beans.factory.InitializingBean; - -import java.util.concurrent.ThreadFactory; -import java.util.function.Consumer; - -/** - * 苏标报警附件 Netty 服务端。独立端口（默认 7612）。 - */ -public final class Jsatl12NettyServer implements InitializingBean, DisposableBean { - - private static final Logger log = LoggerFactory.getLogger(Jsatl12NettyServer.class); - - private final int port; - private final int workerThreads; - private final int maxFrameLength; - private final ArchiveStore archive; - private final Jsatl12JtMessageBridge jtMessageBridge; - private final Jsatl12AttachmentTracker attachmentTracker; - private final VehicleIdentityResolver identityResolver; - private final Consumer dataChunkDispatcher; - - private EventLoopGroup boss; - private EventLoopGroup workers; - private Channel serverChannel; - - public Jsatl12NettyServer(int port, - int workerThreads, - int maxFrameLength, - ArchiveStore archive, - Jsatl12JtMessageBridge jtMessageBridge, - Jsatl12AttachmentTracker attachmentTracker, - Consumer dataChunkDispatcher) { - this(port, workerThreads, maxFrameLength, archive, jtMessageBridge, attachmentTracker, null, dataChunkDispatcher); - } - - public Jsatl12NettyServer(int port, - int workerThreads, - int maxFrameLength, - ArchiveStore archive, - Jsatl12JtMessageBridge jtMessageBridge, - Jsatl12AttachmentTracker attachmentTracker, - VehicleIdentityResolver identityResolver, - Consumer dataChunkDispatcher) { - this.port = port; - this.workerThreads = workerThreads; - this.maxFrameLength = maxFrameLength; - this.archive = archive; - this.jtMessageBridge = jtMessageBridge; - this.attachmentTracker = attachmentTracker == null ? new Jsatl12AttachmentTracker() : attachmentTracker; - this.identityResolver = identityResolver; - this.dataChunkDispatcher = dataChunkDispatcher; - } - - @Override - public void afterPropertiesSet() throws Exception { - ThreadFactory bossTf = r -> new Thread(r, "jsatl12-boss"); - ThreadFactory wkTf = r -> new Thread(r, "jsatl12-worker"); - this.boss = new NioEventLoopGroup(1, bossTf); - int threads = workerThreads > 0 ? workerThreads : Runtime.getRuntime().availableProcessors(); - this.workers = new NioEventLoopGroup(threads, wkTf); - - ServerBootstrap b = new ServerBootstrap(); - b.group(boss, workers) - .channel(NioServerSocketChannel.class) - .option(ChannelOption.SO_BACKLOG, 256) - .childOption(ChannelOption.TCP_NODELAY, true) - .childOption(ChannelOption.SO_KEEPALIVE, true) - .childHandler(new ChannelInitializer() { - @Override - protected void initChannel(SocketChannel ch) { - ch.pipeline() - .addLast("frame", new Jsatl12FrameDecoder(maxFrameLength)) - .addLast("dispatch", new Jsatl12ChannelHandler( - archive, jtMessageBridge, attachmentTracker, identityResolver, dataChunkDispatcher)); - } - }); - this.serverChannel = b.bind(port).sync().channel(); - log.info("jsatl12 Netty server listening on :{} workers={} maxFrameLength={}", - port, threads, maxFrameLength); - } - - @Override - public void destroy() { - try { - if (serverChannel != null) serverChannel.close().sync(); - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); - } finally { - if (boss != null) boss.shutdownGracefully(); - if (workers != null) workers.shutdownGracefully(); - log.info("jsatl12 Netty server stopped"); - } - } -} diff --git a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12UploadCompletionAck.java b/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12UploadCompletionAck.java deleted file mode 100644 index b09de61a..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12UploadCompletionAck.java +++ /dev/null @@ -1,17 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import java.util.List; - -public record Jsatl12UploadCompletionAck( - String name, - int type, - int result, - List missingRanges -) { - public Jsatl12UploadCompletionAck { - name = name == null ? "" : name; - missingRanges = missingRanges == null ? List.of() : List.copyOf(missingRanges); - } - - public record Range(long offset, long length) {} -} diff --git a/modules/protocols/protocol-jsatl12/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/protocols/protocol-jsatl12/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index 1080dfb0..00000000 --- a/modules/protocols/protocol-jsatl12/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.protocol.jsatl12.config.Jsatl12AutoConfiguration diff --git a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12DataPacketDecoderTest.java b/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12DataPacketDecoderTest.java deleted file mode 100644 index c46ff7e5..00000000 --- a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12DataPacketDecoderTest.java +++ /dev/null @@ -1,47 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.codec; - -import io.netty.channel.embedded.EmbeddedChannel; -import io.netty.buffer.Unpooled; -import org.junit.jupiter.api.Test; - -import java.nio.charset.StandardCharsets; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jsatl12DataPacketDecoderTest { - - @Test - void dataPacketUsesFilenameOffsetLengthLayoutInsteadOfMagicLengthLayout() { - EmbeddedChannel channel = new EmbeddedChannel(new Jsatl12FrameDecoder(128)); - byte[] packet = dataPacket("ADAS_001.jpg", 3, new byte[]{0x04, 0x05, 0x06}); - - channel.writeInbound(Unpooled.wrappedBuffer(packet)); - - Jsatl12RawFrame frame = channel.readInbound(); - assertThat(frame.type()).isEqualTo(Jsatl12RawFrame.Type.DATA_CHUNK); - assertThat(frame.bytes()).containsExactly(packet); - Object next = channel.readInbound(); - assertThat(next).isNull(); - } - - private static byte[] dataPacket(String name, int offset, byte[] data) { - byte[] out = new byte[62 + data.length]; - out[0] = 0x30; - out[1] = 0x31; - out[2] = 0x63; - out[3] = 0x64; - byte[] nameBytes = name.getBytes(StandardCharsets.US_ASCII); - System.arraycopy(nameBytes, 0, out, 4, nameBytes.length); - writeDword(out, 54, offset); - writeDword(out, 58, data.length); - System.arraycopy(data, 0, out, 62, data.length); - return out; - } - - private static void writeDword(byte[] out, int offset, long value) { - out[offset] = (byte) ((value >>> 24) & 0xFF); - out[offset + 1] = (byte) ((value >>> 16) & 0xFF); - out[offset + 2] = (byte) ((value >>> 8) & 0xFF); - out[offset + 3] = (byte) (value & 0xFF); - } -} diff --git a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12FrameDecoderTest.java b/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12FrameDecoderTest.java deleted file mode 100644 index 2899e1ec..00000000 --- a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/codec/Jsatl12FrameDecoderTest.java +++ /dev/null @@ -1,72 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.codec; - -import io.netty.buffer.Unpooled; -import io.netty.channel.embedded.EmbeddedChannel; -import org.junit.jupiter.api.Test; - -import java.nio.charset.StandardCharsets; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jsatl12FrameDecoderTest { - - @Test - void decodesDataChunkFrame() { - EmbeddedChannel channel = new EmbeddedChannel(new Jsatl12FrameDecoder(16)); - byte[] packet = dataPacket("ADAS_001.jpg", 0, new byte[]{0x01, 0x02, 0x03}); - - channel.writeInbound(Unpooled.wrappedBuffer(packet)); - - Jsatl12RawFrame frame = channel.readInbound(); - assertThat(frame.type()).isEqualTo(Jsatl12RawFrame.Type.DATA_CHUNK); - assertThat(frame.bytes()).containsExactly(packet); - } - - @Test - void oversizedDataFrameEmitsMalformedCandidate() { - EmbeddedChannel channel = new EmbeddedChannel(new Jsatl12FrameDecoder(2)); - byte[] packet = dataPacket("ADAS_001.jpg", 0, new byte[]{0x01, 0x02, 0x03}); - - channel.writeInbound(Unpooled.wrappedBuffer(packet)); - - Jsatl12RawFrame decoded = channel.readInbound(); - assertThat(decoded.type()).isEqualTo(Jsatl12RawFrame.Type.MALFORMED); - assertThat(decoded.bytes()).containsExactly(packet); - } - - @Test - void oversizedUnclosedJtFrameEmitsMalformedCandidate() { - EmbeddedChannel channel = new EmbeddedChannel(new Jsatl12FrameDecoder(4)); - - channel.writeInbound(Unpooled.wrappedBuffer(new byte[]{ - 0x7e, 0x01, 0x02, 0x03, 0x04, 0x05 - })); - - Jsatl12RawFrame decoded = channel.readInbound(); - assertThat(decoded.type()).isEqualTo(Jsatl12RawFrame.Type.MALFORMED); - assertThat(decoded.bytes()).containsExactly( - 0x7e, 0x01, 0x02, 0x03, 0x04, 0x05); - assertThat((Object) channel.readInbound()).isNull(); - } - - private static byte[] dataPacket(String name, int offset, byte[] data) { - byte[] out = new byte[62 + data.length]; - out[0] = 0x30; - out[1] = 0x31; - out[2] = 0x63; - out[3] = 0x64; - byte[] nameBytes = name.getBytes(StandardCharsets.US_ASCII); - System.arraycopy(nameBytes, 0, out, 4, nameBytes.length); - writeDword(out, 54, offset); - writeDword(out, 58, data.length); - System.arraycopy(data, 0, out, 62, data.length); - return out; - } - - private static void writeDword(byte[] out, int offset, long value) { - out[offset] = (byte) ((value >>> 24) & 0xFF); - out[offset + 1] = (byte) ((value >>> 16) & 0xFF); - out[offset + 2] = (byte) ((value >>> 8) & 0xFF); - out[offset + 3] = (byte) (value & 0xFF); - } -} diff --git a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/handler/Jsatl12ArchiveEventHandlerTest.java b/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/handler/Jsatl12ArchiveEventHandlerTest.java deleted file mode 100644 index 634b9d41..00000000 --- a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/handler/Jsatl12ArchiveEventHandlerTest.java +++ /dev/null @@ -1,175 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.handler; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12ArchiveFailure; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12ArchivedChunk; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12MalformedFrame; -import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12MalformedJtMessage; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jsatl12ArchiveEventHandlerTest { - - @Test - void archivedDataChunkProducesMediaMetaEvent() { - Jsatl12ArchiveEventHandler handler = new Jsatl12ArchiveEventHandler(); - Jsatl12ArchivedChunk chunk = new Jsatl12ArchivedChunk( - "file:///archive/jsatl12/2026/06/22/peer/file.bin", - "jsatl12/2026/06/22/peer/file.bin", - "ADAS_001.jpg", - 64, - 128, - 128, - "10.0.0.8:7612", - Instant.parse("2026-06-22T01:02:03Z")); - - List events = handler.onDataChunkArchived(chunk); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.MediaMeta.class); - VehicleEvent.MediaMeta media = (VehicleEvent.MediaMeta) event; - assertThat(media.source()).isEqualTo(ProtocolId.JSATL12); - assertThat(media.vin()).isEqualTo("unknown"); - assertThat(media.eventTime()).isEqualTo(Instant.parse("2026-06-22T01:02:03Z")); - assertThat(media.archiveRef()).isEqualTo("file:///archive/jsatl12/2026/06/22/peer/file.bin"); - assertThat(media.sizeBytes()).isEqualTo(128); - assertThat(media.mediaType()).isEqualTo("jsatl12-data-chunk"); - assertThat(media.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("archiveKey", "jsatl12/2026/06/22/peer/file.bin") - .containsEntry("fileName", "ADAS_001.jpg") - .containsEntry("fileOffset", "64") - .containsEntry("declaredChunkSizeBytes", "128") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - }); - } - - @Test - void archivedDataChunkKeepsResolvedVehicleIdentity() { - Jsatl12ArchiveEventHandler handler = new Jsatl12ArchiveEventHandler(); - Jsatl12ArchivedChunk chunk = new Jsatl12ArchivedChunk( - "file:///archive/jsatl12/2026/06/22/peer/file.bin", - "jsatl12/2026/06/22/peer/file.bin", - "ADAS_001.jpg", - 0, - 128, - 128, - "10.0.0.8:7612", - Instant.parse("2026-06-22T01:02:03Z"), - "123456789012", - "LNVIN00000001212", - true, - "BOUND_PHONE"); - - List events = handler.onDataChunkArchived(chunk); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.MediaMeta.class); - VehicleEvent.MediaMeta media = (VehicleEvent.MediaMeta) event; - assertThat(media.vin()).isEqualTo("LNVIN00000001212"); - assertThat(media.metadata()) - .containsEntry("vin", "LNVIN00000001212") - .containsEntry("phone", "123456789012") - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_PHONE"); - }); - } - - @Test - void malformedJtMessageProducesPassthroughEvent() { - Jsatl12ArchiveEventHandler handler = new Jsatl12ArchiveEventHandler(); - byte[] raw = new byte[]{0x01, 0x02, 0x03}; - Jsatl12MalformedJtMessage malformed = new Jsatl12MalformedJtMessage( - raw, - "10.0.0.8:7612", - "jt808 frame too short", - Instant.parse("2026-06-22T01:02:03Z")); - - List events = handler.onMalformedJtMessage(malformed); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(ProtocolId.JSATL12); - assertThat(passthrough.vin()).isEqualTo("unknown"); - assertThat(passthrough.eventTime()).isEqualTo(Instant.parse("2026-06-22T01:02:03Z")); - assertThat(passthrough.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("peer", "10.0.0.8:7612") - .containsEntry("parseError", "true") - .containsEntry("source", "jsatl12-jt-message") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - assertThat(passthrough.data()).containsExactly(raw); - }); - } - - @Test - void archiveFailureProducesPassthroughEvent() { - Jsatl12ArchiveEventHandler handler = new Jsatl12ArchiveEventHandler(); - byte[] raw = new byte[]{0x30, 0x31, 0x63, 0x64, 0, 0, 0, 1, 0x7f}; - Jsatl12ArchiveFailure failure = new Jsatl12ArchiveFailure( - raw, - "jsatl12/2026/06/22/peer/file.bin", - 1, - "10.0.0.8:7612", - "archive backend unavailable", - Instant.parse("2026-06-22T01:02:03Z")); - - List events = handler.onArchiveFailure(failure); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(ProtocolId.JSATL12); - assertThat(passthrough.vin()).isEqualTo("unknown"); - assertThat(passthrough.eventTime()).isEqualTo(Instant.parse("2026-06-22T01:02:03Z")); - assertThat(passthrough.data()).containsExactly(raw); - assertThat(passthrough.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("archiveError", "true") - .containsEntry("archiveKey", "jsatl12/2026/06/22/peer/file.bin") - .containsEntry("chunkSizeBytes", "1") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - }); - } - - @Test - void malformedFrameProducesPassthroughEvent() { - Jsatl12ArchiveEventHandler handler = new Jsatl12ArchiveEventHandler(); - byte[] raw = new byte[]{0x30, 0x31, 0x63, 0x64, 0, 0, 0, 3, 0x01, 0x02, 0x03}; - Jsatl12MalformedFrame malformed = new Jsatl12MalformedFrame( - raw, - "10.0.0.8:7612", - "malformed jsatl12 frame", - Instant.parse("2026-06-22T01:02:03Z")); - - List events = handler.onMalformedFrame(malformed); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(ProtocolId.JSATL12); - assertThat(passthrough.vin()).isEqualTo("unknown"); - assertThat(passthrough.eventTime()).isEqualTo(Instant.parse("2026-06-22T01:02:03Z")); - assertThat(passthrough.data()).containsExactly(raw); - assertThat(passthrough.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("peer", "10.0.0.8:7612") - .containsEntry("source", "jsatl12-frame") - .containsEntry("frameError", "true") - .containsEntry("frameErrorMessage", "malformed jsatl12 frame") - .containsEntry("parseError", "true") - .containsEntry("parseErrorMessage", "malformed jsatl12 frame") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - }); - } -} diff --git a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12AttachmentTrackerTest.java b/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12AttachmentTrackerTest.java deleted file mode 100644 index 931c6e25..00000000 --- a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12AttachmentTrackerTest.java +++ /dev/null @@ -1,101 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import org.junit.jupiter.api.Test; - -import java.nio.charset.StandardCharsets; -import static org.assertj.core.api.Assertions.assertThat; - -class Jsatl12AttachmentTrackerTest { - - @Test - void completionAckRequestsOnlyMissingRangesUntilFileIsComplete() { - Jsatl12AttachmentTracker tracker = new Jsatl12AttachmentTracker(); - String phone = "13800138000"; - String fileName = "ADAS_001.jpg"; - - tracker.registerAttachmentList(phone, t1210Body(fileName, 6)); - tracker.recordDataPacket(phone, dataPacket(fileName, 0, new byte[]{0x01, 0x02, 0x03})); - - Jsatl12UploadCompletionAck first = tracker.completeUpload(phone, t1212Body(fileName, 2, 6)); - - assertThat(first.name()).isEqualTo(fileName); - assertThat(first.type()).isEqualTo(2); - assertThat(first.result()).isEqualTo(1); - assertThat(first.missingRanges()).containsExactly(new Jsatl12UploadCompletionAck.Range(3, 3)); - - tracker.recordDataPacket(phone, dataPacket(fileName, 3, new byte[]{0x04, 0x05, 0x06})); - - Jsatl12UploadCompletionAck second = tracker.completeUpload(phone, t1212Body(fileName, 2, 6)); - - assertThat(second.result()).isZero(); - assertThat(second.missingRanges()).isEmpty(); - assertThat(tracker.knownFiles(phone)).doesNotContain(fileName); - } - - @Test - void dataPacketCanBeParsedFromProductionLayout() { - Jsatl12DataPacket packet = Jsatl12DataPacket.parse(dataPacket("BSD_002.bin", 12, new byte[]{0x01, 0x02})); - - assertThat(packet.name()).isEqualTo("BSD_002.bin"); - assertThat(packet.offset()).isEqualTo(12); - assertThat(packet.length()).isEqualTo(2); - assertThat(packet.data()).containsExactly(0x01, 0x02); - } - - private static byte[] t1210Body(String fileName, long size) { - byte[] name = fileName.getBytes(StandardCharsets.US_ASCII); - byte[] out = new byte[57 + 1 + name.length + 4]; - putAscii(out, 0, 7, "DEV0001"); - putAscii(out, 7, 7, "DEV0001"); - out[13] = 0x26; - out[14] = 0x06; - out[15] = 0x22; - out[16] = 0x10; - out[17] = 0x30; - out[18] = 0x45; - out[19] = 0x01; - out[20] = 0x01; - putAscii(out, 23, 32, "ALARM-NO-001"); - out[55] = 0; - out[56] = 1; - out[57] = (byte) name.length; - System.arraycopy(name, 0, out, 58, name.length); - writeDword(out, 58 + name.length, size); - return out; - } - - private static byte[] t1212Body(String fileName, int type, long size) { - byte[] name = fileName.getBytes(StandardCharsets.US_ASCII); - byte[] out = new byte[1 + name.length + 1 + 4]; - out[0] = (byte) name.length; - System.arraycopy(name, 0, out, 1, name.length); - out[1 + name.length] = (byte) type; - writeDword(out, 2 + name.length, size); - return out; - } - - private static byte[] dataPacket(String name, int offset, byte[] data) { - byte[] out = new byte[62 + data.length]; - out[0] = 0x30; - out[1] = 0x31; - out[2] = 0x63; - out[3] = 0x64; - putAscii(out, 4, 50, name); - writeDword(out, 54, offset); - writeDword(out, 58, data.length); - System.arraycopy(data, 0, out, 62, data.length); - return out; - } - - private static void putAscii(byte[] out, int offset, int len, String value) { - byte[] bytes = value.getBytes(StandardCharsets.US_ASCII); - System.arraycopy(bytes, 0, out, offset, Math.min(bytes.length, len)); - } - - private static void writeDword(byte[] out, int offset, long value) { - out[offset] = (byte) ((value >>> 24) & 0xFF); - out[offset + 1] = (byte) ((value >>> 16) & 0xFF); - out[offset + 2] = (byte) ((value >>> 8) & 0xFF); - out[offset + 3] = (byte) (value & 0xFF); - } -} diff --git a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ChannelHandlerTest.java b/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ChannelHandlerTest.java deleted file mode 100644 index 27486689..00000000 --- a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12ChannelHandlerTest.java +++ /dev/null @@ -1,297 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.identity.VehicleIdentityBinding; -import com.lingniu.ingest.protocol.jsatl12.codec.Jsatl12RawFrame; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import io.netty.channel.embedded.EmbeddedChannel; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayInputStream; -import java.io.IOException; -import java.io.InputStream; -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jsatl12ChannelHandlerTest { - - @Test - void dataChunkArchivesPayloadAndDispatchesQueryableArchiveReference() { - RecordingArchiveStore archive = new RecordingArchiveStore(); - List dispatched = new ArrayList<>(); - Jsatl12ChannelHandler handler = new Jsatl12ChannelHandler(archive, null, dispatched::add); - EmbeddedChannel channel = new EmbeddedChannel(handler); - - channel.writeInbound(new Jsatl12RawFrame( - Jsatl12RawFrame.Type.DATA_CHUNK, - dataPacket("ADAS_001.jpg", 0, new byte[]{0x01, 0x02, 0x03}))); - - assertThat(archive.lastChunk).containsExactly(0x01, 0x02, 0x03); - assertThat(dispatched).hasSize(1); - RawFrame frame = dispatched.getFirst(); - assertThat(frame.protocolId()).isEqualTo(ProtocolId.JSATL12); - assertThat(frame.command()).isEqualTo(Jsatl12MessageId.DATA_CHUNK_ARCHIVED); - assertThat(frame.payload()).isInstanceOf(Jsatl12ArchivedChunk.class); - Jsatl12ArchivedChunk chunk = (Jsatl12ArchivedChunk) frame.payload(); - assertThat(chunk.archiveRef()).isEqualTo("memory://" + archive.lastKey); - assertThat(chunk.sizeBytes()).isEqualTo(3); - assertThat(chunk.peer()).isEqualTo("unknown"); - assertThat(frame.sourceMeta()) - .containsEntry("source", "jsatl12-data-chunk") - .containsEntry("fileName", "ADAS_001.jpg") - .containsEntry("fileOffset", "0") - .containsEntry("vin", "unknown") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - } - - @Test - void dataChunkUsesAttachmentListPhoneForVehicleIdentity() { - RecordingArchiveStore archive = new RecordingArchiveStore(); - Jsatl12AttachmentTracker tracker = new Jsatl12AttachmentTracker(); - String phone = "123456789012"; - tracker.registerAttachmentList(phone, t1210Body("ADAS_001.jpg", 3)); - InMemoryVehicleIdentityService identities = new InMemoryVehicleIdentityService(); - identities.bind(new VehicleIdentityBinding( - ProtocolId.JT808, - "LNVIN00000001212", - phone, - "", - "")); - List dispatched = new ArrayList<>(); - Jsatl12ChannelHandler handler = new Jsatl12ChannelHandler( - archive, - null, - tracker, - identities, - dispatched::add); - EmbeddedChannel channel = new EmbeddedChannel(handler); - - channel.writeInbound(new Jsatl12RawFrame( - Jsatl12RawFrame.Type.DATA_CHUNK, - dataPacket("ADAS_001.jpg", 0, new byte[]{0x01, 0x02, 0x03}))); - - assertThat(dispatched).singleElement().satisfies(frame -> { - assertThat(frame.sourceMeta()) - .containsEntry("phone", phone) - .containsEntry("vin", "LNVIN00000001212") - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_PHONE"); - assertThat(frame.payload()).isInstanceOf(Jsatl12ArchivedChunk.class); - Jsatl12ArchivedChunk chunk = (Jsatl12ArchivedChunk) frame.payload(); - assertThat(chunk.phone()).isEqualTo(phone); - assertThat(chunk.vin()).isEqualTo("LNVIN00000001212"); - assertThat(chunk.identityResolved()).isTrue(); - assertThat(chunk.identitySource()).isEqualTo("BOUND_PHONE"); - }); - } - - @Test - void dataChunkStillArchivesWhenIdentityResolverFails() { - RecordingArchiveStore archive = new RecordingArchiveStore(); - Jsatl12AttachmentTracker tracker = new Jsatl12AttachmentTracker(); - String phone = "123456789012"; - tracker.registerAttachmentList(phone, t1210Body("ADAS_001.jpg", 3)); - List dispatched = new ArrayList<>(); - Jsatl12ChannelHandler handler = new Jsatl12ChannelHandler( - archive, - null, - tracker, - lookup -> { - throw new IllegalStateException("identity backend unavailable"); - }, - dispatched::add); - EmbeddedChannel channel = new EmbeddedChannel(handler); - byte[] frameBytes = dataPacket("ADAS_001.jpg", 0, new byte[]{0x01, 0x02, 0x03}); - - channel.writeInbound(new Jsatl12RawFrame(Jsatl12RawFrame.Type.DATA_CHUNK, frameBytes)); - - assertThat(archive.lastChunk).containsExactly(0x01, 0x02, 0x03); - assertThat(dispatched).singleElement().satisfies(frame -> { - assertThat(frame.protocolId()).isEqualTo(ProtocolId.JSATL12); - assertThat(frame.command()).isEqualTo(Jsatl12MessageId.DATA_CHUNK_ARCHIVED); - assertThat(frame.rawBytes()).isNull(); - assertThat(frame.payload()).isInstanceOf(Jsatl12ArchivedChunk.class); - Jsatl12ArchivedChunk chunk = (Jsatl12ArchivedChunk) frame.payload(); - assertThat(chunk.phone()).isEqualTo(phone); - assertThat(chunk.vin()).isEqualTo("unknown"); - assertThat(chunk.identityResolved()).isFalse(); - assertThat(chunk.identitySource()).isEqualTo("UNKNOWN"); - assertThat(frame.sourceMeta()) - .containsEntry("phone", phone) - .containsEntry("vin", "unknown") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN") - .containsEntry("identityError", "true") - .containsEntry("identityErrorMessage", "identity backend unavailable") - .containsEntry("archiveRef", "memory://" + archive.lastKey); - }); - } - - @Test - void dataChunkArchiveFailureDispatchesPassthroughCandidateWithRawPayload() { - FailingArchiveStore archive = new FailingArchiveStore(); - List dispatched = new ArrayList<>(); - Jsatl12ChannelHandler handler = new Jsatl12ChannelHandler(archive, null, dispatched::add); - EmbeddedChannel channel = new EmbeddedChannel(handler); - byte[] frameBytes = dataPacket("ADAS_001.jpg", 0, new byte[]{0x01, 0x02, 0x03}); - - channel.writeInbound(new Jsatl12RawFrame(Jsatl12RawFrame.Type.DATA_CHUNK, frameBytes)); - - assertThat(dispatched).singleElement().satisfies(frame -> { - assertThat(frame.protocolId()).isEqualTo(ProtocolId.JSATL12); - assertThat(frame.command()).isEqualTo(Jsatl12MessageId.DATA_CHUNK_ARCHIVE_FAILED); - assertThat(frame.rawBytes()).containsExactly(frameBytes); - assertThat(frame.payload()).isInstanceOf(Jsatl12ArchiveFailure.class); - Jsatl12ArchiveFailure failure = (Jsatl12ArchiveFailure) frame.payload(); - assertThat(failure.rawBytes()).containsExactly(frameBytes); - assertThat(failure.sizeBytes()).isEqualTo(3); - assertThat(failure.peer()).isEqualTo("unknown"); - assertThat(failure.errorMessage()).contains("archive backend unavailable"); - assertThat(frame.sourceMeta()) - .containsEntry("source", "jsatl12-data-chunk") - .containsEntry("archiveError", "true") - .containsEntry("chunkSizeBytes", "3") - .containsEntry("vin", "unknown") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - }); - } - - @Test - void malformedFrameDispatchesPassthroughCandidateWithRawPayload() { - RecordingArchiveStore archive = new RecordingArchiveStore(); - List dispatched = new ArrayList<>(); - Jsatl12ChannelHandler handler = new Jsatl12ChannelHandler(archive, null, dispatched::add); - EmbeddedChannel channel = new EmbeddedChannel(handler); - byte[] raw = new byte[]{ - 0x30, 0x31, 0x63, 0x64, - 0x00, 0x00, 0x00, 0x03, - 0x01, 0x02, 0x03 - }; - - channel.writeInbound(new Jsatl12RawFrame(Jsatl12RawFrame.Type.MALFORMED, raw)); - - assertThat(dispatched).singleElement().satisfies(frame -> { - assertThat(frame.protocolId()).isEqualTo(ProtocolId.JSATL12); - assertThat(frame.command()).isEqualTo(Jsatl12MessageId.MALFORMED_FRAME); - assertThat(frame.rawBytes()).containsExactly(raw); - assertThat(frame.payload()).isInstanceOf(Jsatl12MalformedFrame.class); - Jsatl12MalformedFrame malformed = (Jsatl12MalformedFrame) frame.payload(); - assertThat(malformed.rawBytes()).containsExactly(raw); - assertThat(malformed.peer()).isEqualTo("unknown"); - assertThat(malformed.errorMessage()).contains("malformed jsatl12 frame"); - assertThat(frame.sourceMeta()) - .containsEntry("source", "jsatl12-frame") - .containsEntry("frameError", "true") - .containsEntry("frameSizeBytes", Integer.toString(raw.length)) - .containsEntry("vin", "unknown") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - }); - } - - private static byte[] dataPacket(String name, int offset, byte[] data) { - byte[] out = new byte[62 + data.length]; - out[0] = 0x30; - out[1] = 0x31; - out[2] = 0x63; - out[3] = 0x64; - byte[] nameBytes = name.getBytes(StandardCharsets.US_ASCII); - System.arraycopy(nameBytes, 0, out, 4, nameBytes.length); - writeDword(out, 54, offset); - writeDword(out, 58, data.length); - System.arraycopy(data, 0, out, 62, data.length); - return out; - } - - private static byte[] t1210Body(String fileName, long size) { - byte[] name = fileName.getBytes(StandardCharsets.US_ASCII); - byte[] out = new byte[57 + 1 + name.length + 4]; - putAscii(out, 0, 7, "DEV0001"); - putAscii(out, 7, 7, "DEV0001"); - putAscii(out, 23, 32, "ALARM-NO-001"); - out[55] = 0; - out[56] = 1; - out[57] = (byte) name.length; - System.arraycopy(name, 0, out, 58, name.length); - writeDword(out, 58 + name.length, size); - return out; - } - - private static void putAscii(byte[] out, int offset, int len, String value) { - byte[] bytes = value.getBytes(StandardCharsets.US_ASCII); - System.arraycopy(bytes, 0, out, offset, Math.min(bytes.length, len)); - } - - private static void writeDword(byte[] out, int offset, long value) { - out[offset] = (byte) ((value >>> 24) & 0xFF); - out[offset + 1] = (byte) ((value >>> 16) & 0xFF); - out[offset + 2] = (byte) ((value >>> 8) & 0xFF); - out[offset + 3] = (byte) (value & 0xFF); - } - - private static final class RecordingArchiveStore implements ArchiveStore { - private String lastKey; - private byte[] lastChunk; - - @Override - public String put(String key, InputStream data, long length) { - throw new UnsupportedOperationException(); - } - - @Override - public String append(String key, byte[] chunk) { - this.lastKey = key; - this.lastChunk = chunk; - return "memory://" + key; - } - - @Override - public InputStream get(String key) { - return new ByteArrayInputStream(new byte[0]); - } - - @Override - public boolean exists(String key) { - return false; - } - - @Override - public long size(String key) throws IOException { - return lastChunk == null ? 0 : lastChunk.length; - } - } - - private static final class FailingArchiveStore implements ArchiveStore { - @Override - public String put(String key, InputStream data, long length) throws IOException { - throw new IOException("archive backend unavailable"); - } - - @Override - public String append(String key, byte[] chunk) throws IOException { - throw new IOException("archive backend unavailable"); - } - - @Override - public InputStream get(String key) { - return new ByteArrayInputStream(new byte[0]); - } - - @Override - public boolean exists(String key) { - return false; - } - - @Override - public long size(String key) throws IOException { - throw new IOException("archive backend unavailable"); - } - } -} diff --git a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12JtMessageBridgeTest.java b/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12JtMessageBridgeTest.java deleted file mode 100644 index 44045475..00000000 --- a/modules/protocols/protocol-jsatl12/src/test/java/com/lingniu/ingest/protocol/jsatl12/inbound/Jsatl12JtMessageBridgeTest.java +++ /dev/null @@ -1,237 +0,0 @@ -package com.lingniu.ingest.protocol.jsatl12.inbound; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.codec.BcdCodec; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.identity.VehicleIdentityBinding; -import com.lingniu.ingest.protocol.jt808.codec.BodyParserRegistry; -import com.lingniu.ingest.protocol.jt808.codec.Jt808Escape; -import com.lingniu.ingest.protocol.jt808.codec.Jt808FrameEncoder; -import com.lingniu.ingest.protocol.jt808.codec.Jt808MessageDecoder; -import com.lingniu.ingest.protocol.jt808.codec.parser.HeartbeatBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.PassthroughBodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayOutputStream; -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.util.ArrayList; -import java.util.List; -import java.util.Optional; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jsatl12JtMessageBridgeTest { - - @Test - void decodesJtMessageAndDispatchesAsJt808RawFrame() { - List dispatched = new ArrayList<>(); - InMemoryVehicleIdentityService identities = new InMemoryVehicleIdentityService(); - identities.bind(new VehicleIdentityBinding( - ProtocolId.JT808, - "LNVIN00000001212", - "123456789012", - "", - "")); - Jsatl12JtMessageBridge bridge = new Jsatl12JtMessageBridge( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new HeartbeatBodyParser()))), - identities, - dispatched::add); - - bridge.dispatch(buildFrame(Jt808MessageId.TERMINAL_HEARTBEAT, "123456789012", 3), "127.0.0.1:7612"); - - assertThat(dispatched).hasSize(1); - RawFrame frame = dispatched.get(0); - assertThat(frame.protocolId()).isEqualTo(ProtocolId.JT808); - assertThat(frame.command()).isEqualTo(Jt808MessageId.TERMINAL_HEARTBEAT); - assertThat(frame.sourceMeta()) - .containsEntry("vin", "LNVIN00000001212") - .containsEntry("phone", "123456789012") - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_PHONE"); - assertThat(frame.payload()).isInstanceOf(com.lingniu.ingest.protocol.jt808.model.Jt808Message.class); - assertThat(((com.lingniu.ingest.protocol.jt808.model.Jt808Message) frame.payload()).body()) - .isInstanceOf(Jt808Body.Heartbeat.class); - } - - @Test - void bridgeUsesExtendedJt808Parsers() { - List dispatched = new ArrayList<>(); - Jsatl12JtMessageBridge bridge = new Jsatl12JtMessageBridge( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new PassthroughBodyParser()))), - dispatched::add); - byte[] body = new byte[]{0x41, 0x01, 0x02}; - - bridge.dispatch(buildFrame(Jt808MessageId.TERMINAL_PASSTHROUGH, "123456789012", 4, body), - "127.0.0.1:7612"); - - assertThat(dispatched).hasSize(1); - assertThat(dispatched.get(0).command()).isEqualTo(Jt808MessageId.TERMINAL_PASSTHROUGH); - assertThat(((com.lingniu.ingest.protocol.jt808.model.Jt808Message) dispatched.get(0).payload()).body()) - .isInstanceOf(Jt808Body.Passthrough.class); - } - - @Test - void malformedJtMessageIsDispatchedAsJsatl12PassthroughCandidate() { - List dispatched = new ArrayList<>(); - Jsatl12JtMessageBridge bridge = new Jsatl12JtMessageBridge( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new HeartbeatBodyParser()))), - dispatched::add); - byte[] malformed = new byte[]{0x01, 0x02, 0x03}; - - bridge.dispatch(malformed, "127.0.0.1:7612"); - - assertThat(dispatched).hasSize(1); - RawFrame frame = dispatched.getFirst(); - assertThat(frame.protocolId()).isEqualTo(ProtocolId.JSATL12); - assertThat(frame.command()).isEqualTo(Jsatl12MessageId.MALFORMED_JT_MESSAGE); - assertThat(frame.rawBytes()).containsExactly(malformed); - assertThat(frame.sourceMeta()) - .containsEntry("peer", "127.0.0.1:7612") - .containsEntry("parseError", "true") - .containsEntry("source", "jsatl12-jt-message") - .containsEntry("vin", "unknown") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - assertThat(frame.payload()).isInstanceOf(Jsatl12MalformedJtMessage.class); - } - - @Test - void identityResolverFailureStillDispatchesDecodedJt808MessageWithUnknownIdentity() { - List dispatched = new ArrayList<>(); - Jsatl12JtMessageBridge bridge = new Jsatl12JtMessageBridge( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new HeartbeatBodyParser()))), - lookup -> { - throw new IllegalStateException("identity backend unavailable"); - }, - dispatched::add); - byte[] raw = buildFrame(Jt808MessageId.TERMINAL_HEARTBEAT, "123456789012", 3); - - bridge.dispatch(raw, "127.0.0.1:7612"); - - assertThat(dispatched).singleElement().satisfies(frame -> { - assertThat(frame.protocolId()).isEqualTo(ProtocolId.JT808); - assertThat(frame.command()).isEqualTo(Jt808MessageId.TERMINAL_HEARTBEAT); - assertThat(frame.rawBytes()).isNotNull(); - assertThat(frame.payload()).isInstanceOf(com.lingniu.ingest.protocol.jt808.model.Jt808Message.class); - assertThat(frame.sourceMeta()) - .containsEntry("source", "jsatl12-signaling") - .containsEntry("peer", "127.0.0.1:7612") - .containsEntry("vin", "unknown") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN") - .containsEntry("identityError", "true") - .containsEntry("identityErrorMessage", "identity backend unavailable"); - }); - } - - @Test - void uploadCompletionReturnsT9212RetransmissionAck() { - List dispatched = new ArrayList<>(); - Jsatl12AttachmentTracker tracker = new Jsatl12AttachmentTracker(); - Jsatl12JtMessageBridge bridge = new Jsatl12JtMessageBridge( - new Jt808MessageDecoder(new BodyParserRegistry(List.of())), - tracker, - dispatched::add); - String phone = "123456789012"; - String fileName = "ADAS_001.jpg"; - - Optional t1210Ack = bridge.dispatch(buildFrame(0x1210, phone, 10, t1210Body(fileName, 6)), - "127.0.0.1:7612"); - tracker.recordDataPacket(phone, dataPacket(fileName, 0, new byte[]{0x01, 0x02, 0x03})); - Optional t9212 = bridge.dispatch(buildFrame(0x1212, phone, 11, t1212Body(fileName, 2, 6)), - "127.0.0.1:7612"); - - assertThat(t1210Ack).isEmpty(); - assertThat(t9212).isPresent(); - byte[] unescaped = Jt808Escape.unescape(t9212.get(), 1, t9212.get().length - 2); - var ack = new Jt808MessageDecoder(new BodyParserRegistry(List.of())) - .decode(ByteBuffer.wrap(unescaped)); - assertThat(ack.header().messageId()).isEqualTo(0x9212); - assertThat(ack.header().phone()).isEqualTo(phone); - assertThat(ack.header().serialNo()).isEqualTo(11); - Jt808Body.Raw raw = (Jt808Body.Raw) ack.body(); - byte[] body = raw.bytes(); - int nameLen = body[0] & 0xFF; - assertThat(new String(body, 1, nameLen, StandardCharsets.UTF_8)).isEqualTo(fileName); - assertThat(body[1 + nameLen] & 0xFF).isEqualTo(2); - assertThat(body[2 + nameLen] & 0xFF).isEqualTo(1); - assertThat(body[3 + nameLen] & 0xFF).isEqualTo(1); - assertThat(readDword(body, 4 + nameLen)).isEqualTo(3); - assertThat(readDword(body, 8 + nameLen)).isEqualTo(3); - } - - private static byte[] buildFrame(int msgId, String phone, int serial) { - return buildFrame(msgId, phone, serial, new byte[0]); - } - - private static byte[] buildFrame(int msgId, String phone, int serial, byte[] body) { - byte[] framed = Jt808FrameEncoder.encode(msgId, phone, serial, body); - return Jt808Escape.unescape(framed, 1, framed.length - 2); - } - - private static void writeU16(ByteArrayOutputStream os, int v) { - os.write((v >> 8) & 0xFF); - os.write(v & 0xFF); - } - - private static byte[] t1210Body(String fileName, long size) { - byte[] name = fileName.getBytes(StandardCharsets.US_ASCII); - byte[] out = new byte[57 + 1 + name.length + 4]; - putAscii(out, 0, 7, "DEV0001"); - putAscii(out, 7, 7, "DEV0001"); - putAscii(out, 23, 32, "ALARM-NO-001"); - out[55] = 0; - out[56] = 1; - out[57] = (byte) name.length; - System.arraycopy(name, 0, out, 58, name.length); - writeDword(out, 58 + name.length, size); - return out; - } - - private static byte[] t1212Body(String fileName, int type, long size) { - byte[] name = fileName.getBytes(StandardCharsets.US_ASCII); - byte[] out = new byte[1 + name.length + 1 + 4]; - out[0] = (byte) name.length; - System.arraycopy(name, 0, out, 1, name.length); - out[1 + name.length] = (byte) type; - writeDword(out, 2 + name.length, size); - return out; - } - - private static byte[] dataPacket(String name, int offset, byte[] data) { - byte[] out = new byte[62 + data.length]; - out[0] = 0x30; - out[1] = 0x31; - out[2] = 0x63; - out[3] = 0x64; - putAscii(out, 4, 50, name); - writeDword(out, 54, offset); - writeDword(out, 58, data.length); - System.arraycopy(data, 0, out, 62, data.length); - return out; - } - - private static void putAscii(byte[] out, int offset, int len, String value) { - byte[] bytes = value.getBytes(StandardCharsets.US_ASCII); - System.arraycopy(bytes, 0, out, offset, Math.min(bytes.length, len)); - } - - private static void writeDword(byte[] out, int offset, long value) { - out[offset] = (byte) ((value >>> 24) & 0xFF); - out[offset + 1] = (byte) ((value >>> 16) & 0xFF); - out[offset + 2] = (byte) ((value >>> 8) & 0xFF); - out[offset + 3] = (byte) (value & 0xFF); - } - - private static long readDword(byte[] raw, int offset) { - return ((raw[offset] & 0xFFL) << 24) - | ((raw[offset + 1] & 0xFFL) << 16) - | ((raw[offset + 2] & 0xFFL) << 8) - | (raw[offset + 3] & 0xFFL); - } -} diff --git a/modules/protocols/protocol-jt1078/pom.xml b/modules/protocols/protocol-jt1078/pom.xml deleted file mode 100644 index 4110d5ec..00000000 --- a/modules/protocols/protocol-jt1078/pom.xml +++ /dev/null @@ -1,71 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - protocol-jt1078 - protocol-jt1078 - - JT/T 1078 音视频信令接入。复用 protocol-jt808 的 Netty server 与会话， - 只通过注册新的 BodyParser 热插拔扩展消息集（0x1003 / 0x1005 / 0x1205 / 0x1206）。 - - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - ingest-core - - - com.lingniu.ingest - protocol-jt808 - - - com.lingniu.ingest - vehicle-identity - - - com.lingniu.ingest - vehicle-identity-test-support - test - - - com.lingniu.ingest - sink-archive - - - org.springframework.boot - spring-boot-starter - - - io.netty - netty-all - - - com.github.ben-manes.caffeine - caffeine - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.springframework.boot - spring-boot-test - test - - - diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/FileListBodyParser.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/FileListBodyParser.java deleted file mode 100644 index 884a045a..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/FileListBodyParser.java +++ /dev/null @@ -1,23 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.codec.parser; - -import com.lingniu.ingest.protocol.jt1078.model.Jt1078MessageId; -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; - -import java.nio.ByteBuffer; - -/** 0x1205 终端上传文件列表，摘要字段结构化，明细原始 body 保留。 */ -public final class FileListBodyParser implements BodyParser { - @Override public int messageId() { return Jt1078MessageId.TERMINAL_FILE_LIST; } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer body) { - byte[] raw = new byte[body.remaining()]; - body.get(raw); - ByteBuffer b = ByteBuffer.wrap(raw); - int responseSerialNo = b.remaining() >= 2 ? b.getShort() & 0xFFFF : 0; - long fileCount = b.remaining() >= 4 ? b.getInt() & 0xFFFFFFFFL : 0L; - return new Jt808Body.FileList(responseSerialNo, fileCount, raw); - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/FileUploadCompleteBodyParser.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/FileUploadCompleteBodyParser.java deleted file mode 100644 index 9787cd65..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/FileUploadCompleteBodyParser.java +++ /dev/null @@ -1,23 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.codec.parser; - -import com.lingniu.ingest.protocol.jt1078.model.Jt1078MessageId; -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; - -import java.nio.ByteBuffer; - -/** 0x1206 文件上传完成通知。 */ -public final class FileUploadCompleteBodyParser implements BodyParser { - @Override public int messageId() { return Jt1078MessageId.TERMINAL_FILE_UPLOAD_DONE; } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer body) { - byte[] raw = new byte[body.remaining()]; - body.get(raw); - ByteBuffer b = ByteBuffer.wrap(raw); - int responseSerialNo = b.remaining() >= 2 ? b.getShort() & 0xFFFF : 0; - int result = b.hasRemaining() ? b.get() & 0xFF : 0; - return new Jt808Body.FileUploadComplete(responseSerialNo, result, raw); - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/MediaAttributesBodyParser.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/MediaAttributesBodyParser.java deleted file mode 100644 index 580fb11c..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/MediaAttributesBodyParser.java +++ /dev/null @@ -1,39 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.codec.parser; - -import com.lingniu.ingest.protocol.jt1078.model.Jt1078MessageId; -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; - -import java.nio.ByteBuffer; - -/** 0x1003 终端上传音视频属性。 */ -public final class MediaAttributesBodyParser implements BodyParser { - @Override public int messageId() { return Jt1078MessageId.TERMINAL_MEDIA_ATTRS_REPORT; } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer body) { - byte[] raw = new byte[body.remaining()]; - body.get(raw); - ByteBuffer b = ByteBuffer.wrap(raw); - return new Jt808Body.MediaAttributes( - readU8(b), - readU8(b), - readU8(b), - readU8(b), - readU16(b), - readU8(b) == 1, - readU8(b), - readU8(b), - readU8(b), - raw); - } - - private static int readU8(ByteBuffer b) { - return b.hasRemaining() ? b.get() & 0xFF : 0; - } - - private static int readU16(ByteBuffer b) { - return b.remaining() >= 2 ? b.getShort() & 0xFFFF : 0; - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/PassengerVolumeBodyParser.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/PassengerVolumeBodyParser.java deleted file mode 100644 index 7feedd24..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/codec/parser/PassengerVolumeBodyParser.java +++ /dev/null @@ -1,56 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.codec.parser; - -import com.lingniu.ingest.codec.BcdCodec; -import com.lingniu.ingest.protocol.jt1078.model.Jt1078MessageId; -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; - -import java.nio.ByteBuffer; - -/** 0x1005 终端上传乘客流量，字段结构化后仍按可查询透传事件入库。 */ -public final class PassengerVolumeBodyParser implements BodyParser { - @Override public int messageId() { return Jt1078MessageId.TERMINAL_PASSENGER_VOLUME; } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer body) { - byte[] raw = new byte[body.remaining()]; - body.get(raw); - ByteBuffer b = ByteBuffer.wrap(raw); - return new Jt808Body.PassengerVolume( - readU8(b), - readBcdTime(b), - readBcdTime(b), - readU16(b), - readU16(b), - raw); - } - - private static int readU8(ByteBuffer b) { - return b.hasRemaining() ? b.get() & 0xFF : 0; - } - - private static int readU16(ByteBuffer b) { - return b.remaining() >= 2 ? b.getShort() & 0xFFFF : 0; - } - - private static String readBcdTime(ByteBuffer b) { - if (b.remaining() < 6) { - byte[] tail = new byte[b.remaining()]; - b.get(tail); - return ""; - } - byte[] time = new byte[6]; - b.get(time); - String ts = BcdCodec.decode(time); - if (ts.length() != 12) { - return ""; - } - return "20" + ts.substring(0, 2) - + "-" + ts.substring(2, 4) - + "-" + ts.substring(4, 6) - + "T" + ts.substring(6, 8) - + ":" + ts.substring(8, 10) - + ":" + ts.substring(10, 12); - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078AutoConfiguration.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078AutoConfiguration.java deleted file mode 100644 index 8cbaae3a..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078AutoConfiguration.java +++ /dev/null @@ -1,105 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.config; - -import com.lingniu.ingest.core.concurrency.DisruptorEventBus; -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.protocol.jt1078.codec.parser.FileUploadCompleteBodyParser; -import com.lingniu.ingest.protocol.jt1078.codec.parser.FileListBodyParser; -import com.lingniu.ingest.protocol.jt1078.codec.parser.MediaAttributesBodyParser; -import com.lingniu.ingest.protocol.jt1078.codec.parser.PassengerVolumeBodyParser; -import com.lingniu.ingest.protocol.jt1078.handler.Jt1078MalformedRtpHandler; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MalformedRtpEventPublisher; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MediaArchiveService; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MediaStreamServer; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078UdpMediaStreamServer; -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.config.Jt808AutoConfiguration; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import com.lingniu.ingest.sink.archive.config.SinkArchiveAutoConfiguration; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; - -/** - * jt1078 在 jt808 之前装配它自己的 Parser Bean，Spring 会把它们注入到 jt808 的 - * {@code BodyParserRegistry} 里，实现"同一个 Netty 端口复用、消息集热插拔"。 - */ -@AutoConfiguration(before = Jt808AutoConfiguration.class, after = SinkArchiveAutoConfiguration.class) -@EnableConfigurationProperties(Jt1078Properties.class) -@ConditionalOnProperty(prefix = "lingniu.ingest.jt1078", name = "enabled", havingValue = "true") -public class Jt1078AutoConfiguration { - - @Bean - public BodyParser jt1078MediaAttributesBodyParser() { - return new MediaAttributesBodyParser(); - } - - @Bean - public BodyParser jt1078PassengerVolumeBodyParser() { - return new PassengerVolumeBodyParser(); - } - - @Bean - public BodyParser jt1078FileListBodyParser() { - return new FileListBodyParser(); - } - - @Bean - public BodyParser jt1078FileUploadCompleteBodyParser() { - return new FileUploadCompleteBodyParser(); - } - - @Bean - @ConditionalOnMissingBean - @ConditionalOnBean(ArchiveStore.class) - public Jt1078MediaArchiveService jt1078MediaArchiveService(ArchiveStore archive, - VehicleIdentityResolver identityResolver, - DisruptorEventBus eventBus, - Jt1078Properties props) { - return new Jt1078MediaArchiveService( - archive, identityResolver, eventBus::publish, props.getMediaStream().getSegmentSeconds()); - } - - @Bean - @ConditionalOnMissingBean - public Jt1078MalformedRtpEventPublisher jt1078MalformedRtpEventPublisher(VehicleIdentityResolver identityResolver, - Dispatcher dispatcher) { - return new Jt1078MalformedRtpEventPublisher(identityResolver, dispatcher::dispatch); - } - - @Bean - @ConditionalOnMissingBean - public Jt1078MalformedRtpHandler jt1078MalformedRtpHandler(VehicleIdentityResolver identityResolver) { - return new Jt1078MalformedRtpHandler(identityResolver); - } - - @Bean - @ConditionalOnMissingBean - @ConditionalOnBean(Jt1078MediaArchiveService.class) - @ConditionalOnExpression("${lingniu.ingest.jt1078.media-stream.enabled:true} " - + "&& ${lingniu.ingest.jt1078.media-stream.tcp-enabled:true}") - public Jt1078MediaStreamServer jt1078MediaStreamServer(Jt1078Properties props, - Jt1078MediaArchiveService archiveService, - Jt1078MalformedRtpEventPublisher malformedPublisher) { - Jt1078Properties.MediaStream media = props.getMediaStream(); - return new Jt1078MediaStreamServer( - media.getPort(), media.getWorkerThreads(), media.getMaxPacketLength(), archiveService, malformedPublisher); - } - - @Bean - @ConditionalOnMissingBean - @ConditionalOnBean(Jt1078MediaArchiveService.class) - @ConditionalOnExpression("${lingniu.ingest.jt1078.media-stream.enabled:true} " - + "&& ${lingniu.ingest.jt1078.media-stream.udp-enabled:false}") - public Jt1078UdpMediaStreamServer jt1078UdpMediaStreamServer(Jt1078Properties props, - Jt1078MediaArchiveService archiveService, - Jt1078MalformedRtpEventPublisher malformedPublisher) { - Jt1078Properties.MediaStream media = props.getMediaStream(); - return new Jt1078UdpMediaStreamServer( - media.getUdpPort(), media.getWorkerThreads(), media.getMaxPacketLength(), archiveService, malformedPublisher); - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078Properties.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078Properties.java deleted file mode 100644 index 770c5a55..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078Properties.java +++ /dev/null @@ -1,49 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.config; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -@ConfigurationProperties(prefix = "lingniu.ingest.jt1078") -public class Jt1078Properties { - - /** 是否启用 JT/T 1078 音视频能力；它不参与 32960 快照/历史字段查询。 */ - private boolean enabled = false; - private MediaStream mediaStream = new MediaStream(); - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public MediaStream getMediaStream() { return mediaStream; } - public void setMediaStream(MediaStream mediaStream) { this.mediaStream = mediaStream; } - - public static class MediaStream { - /** 媒体流子模块开关，便于只启用信令、不落媒体文件。 */ - private boolean enabled = true; - /** TCP 媒体流监听，适合生产默认路径。 */ - private boolean tcpEnabled = true; - /** UDP 媒体流监听默认关闭，开启前需要确认网络和包乱序处理能力。 */ - private boolean udpEnabled = false; - private int port = 11078; - private int udpPort = 11078; - private int workerThreads = 0; - /** 单包最大长度保护，防止异常媒体流耗尽内存。 */ - private int maxPacketLength = 1024 * 1024; - /** 媒体文件分段秒数，影响归档文件粒度。 */ - private int segmentSeconds = 300; - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public boolean isTcpEnabled() { return tcpEnabled; } - public void setTcpEnabled(boolean tcpEnabled) { this.tcpEnabled = tcpEnabled; } - public boolean isUdpEnabled() { return udpEnabled; } - public void setUdpEnabled(boolean udpEnabled) { this.udpEnabled = udpEnabled; } - public int getPort() { return port; } - public void setPort(int port) { this.port = port; } - public int getUdpPort() { return udpPort; } - public void setUdpPort(int udpPort) { this.udpPort = udpPort; } - public int getWorkerThreads() { return workerThreads; } - public void setWorkerThreads(int workerThreads) { this.workerThreads = workerThreads; } - public int getMaxPacketLength() { return maxPacketLength; } - public void setMaxPacketLength(int maxPacketLength) { this.maxPacketLength = maxPacketLength; } - public int getSegmentSeconds() { return segmentSeconds; } - public void setSegmentSeconds(int segmentSeconds) { this.segmentSeconds = segmentSeconds; } - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078SignalAutoConfiguration.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078SignalAutoConfiguration.java deleted file mode 100644 index 60585cfc..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078SignalAutoConfiguration.java +++ /dev/null @@ -1,25 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.config; - -import com.lingniu.ingest.protocol.jt1078.handler.Jt1078MediaSignalHandler; -import com.lingniu.ingest.protocol.jt808.config.Jt808AutoConfiguration; -import com.lingniu.ingest.protocol.jt808.mapper.Jt808EventMapper; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.context.annotation.Bean; - -/** - * 1078 信令复用 JT808 解码和 mapper，必须在 JT808 AutoConfiguration 之后判断。 - */ -@AutoConfiguration(after = Jt808AutoConfiguration.class) -@ConditionalOnProperty(prefix = "lingniu.ingest.jt1078", name = "enabled", havingValue = "true") -@ConditionalOnBean(Jt808EventMapper.class) -public class Jt1078SignalAutoConfiguration { - - @Bean - @ConditionalOnMissingBean - public Jt1078MediaSignalHandler jt1078MediaSignalHandler(Jt808EventMapper mapper) { - return new Jt1078MediaSignalHandler(mapper); - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/downlink/Jt1078Commands.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/downlink/Jt1078Commands.java deleted file mode 100644 index 34027988..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/downlink/Jt1078Commands.java +++ /dev/null @@ -1,198 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.downlink; - -import com.lingniu.ingest.protocol.jt1078.model.Jt1078MessageId; -import com.lingniu.ingest.protocol.jt808.downlink.Jt808Commands; - -import java.io.ByteArrayOutputStream; -import java.nio.charset.Charset; - -/** JT/T 1078 下行信令 body 构造器，外层帧仍由 JT808 通道发送。 */ -public final class Jt1078Commands { - - private static final Charset GBK = Charset.forName("GBK"); - - private Jt1078Commands() {} - - /** 0x1003 查询终端音视频属性。 */ - public static Jt808Commands.DownlinkCommand queryMediaAttributes() { - return new Jt808Commands.DownlinkCommand(Jt1078MessageId.PLATFORM_QUERY_MEDIA_ATTRS, new byte[0]); - } - - /** 0x9101 实时音视频传输请求。 */ - public static Jt808Commands.DownlinkCommand realtimePlay(String ip, - int tcpPort, - int udpPort, - int channelNo, - int mediaType, - int streamType) { - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeStringWithU8Length(os, ip); - writeU16(os, tcpPort); - writeU16(os, udpPort); - writeU8(os, channelNo); - writeU8(os, mediaType); - writeU8(os, streamType); - return new Jt808Commands.DownlinkCommand(Jt1078MessageId.PLATFORM_REALTIME_PLAY, os.toByteArray()); - } - - /** 0x9102 音视频实时传输控制。 */ - public static Jt808Commands.DownlinkCommand realtimeControl(int channelNo, - int command, - int closeType, - int streamType) { - ByteArrayOutputStream os = new ByteArrayOutputStream(4); - writeU8(os, channelNo); - writeU8(os, command); - writeU8(os, closeType); - writeU8(os, streamType); - return new Jt808Commands.DownlinkCommand(Jt1078MessageId.PLATFORM_REALTIME_CONTROL, os.toByteArray()); - } - - /** 0x9201 平台下发远程录像回放请求。 */ - public static Jt808Commands.DownlinkCommand historyPlay(String ip, - int tcpPort, - int udpPort, - int channelNo, - int mediaType, - int streamType, - int storageType, - int playbackMode, - int playbackSpeed, - String startTime, - String endTime) { - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeStringWithU8Length(os, ip); - writeU16(os, tcpPort); - writeU16(os, udpPort); - writeU8(os, channelNo); - writeU8(os, mediaType); - writeU8(os, streamType); - writeU8(os, storageType); - writeU8(os, playbackMode); - writeU8(os, playbackSpeed); - writeBcdTime(os, startTime); - writeBcdTime(os, endTime); - return new Jt808Commands.DownlinkCommand(Jt1078MessageId.PLATFORM_HISTORY_PLAY, os.toByteArray()); - } - - /** 0x9202 平台下发远程录像回放控制。 */ - public static Jt808Commands.DownlinkCommand historyControl(int channelNo, - int playbackMode, - int playbackSpeed, - String playbackTime) { - ByteArrayOutputStream os = new ByteArrayOutputStream(9); - writeU8(os, channelNo); - writeU8(os, playbackMode); - writeU8(os, playbackSpeed); - writeBcdTime(os, playbackTime); - return new Jt808Commands.DownlinkCommand(Jt1078MessageId.PLATFORM_HISTORY_CONTROL, os.toByteArray()); - } - - /** 0x9205 查询资源列表。 */ - public static Jt808Commands.DownlinkCommand resourceSearch(int channelNo, - String startTime, - String endTime, - long warnBit1, - long warnBit2, - int mediaType, - int streamType, - int storageType) { - ByteArrayOutputStream os = new ByteArrayOutputStream(24); - writeU8(os, channelNo); - writeBcdTime(os, startTime); - writeBcdTime(os, endTime); - writeU32(os, warnBit1); - writeU32(os, warnBit2); - writeU8(os, mediaType); - writeU8(os, streamType); - writeU8(os, storageType); - return new Jt808Commands.DownlinkCommand(Jt1078MessageId.PLATFORM_RESOURCE_SEARCH, os.toByteArray()); - } - - /** 0x9206 文件上传指令。 */ - public static Jt808Commands.DownlinkCommand fileUpload(String ip, - int port, - String username, - String password, - String path, - int channelNo, - String startTime, - String endTime, - long warnBit1, - long warnBit2, - int mediaType, - int streamType, - int storageType, - int condition) { - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeStringWithU8Length(os, ip); - writeU16(os, port); - writeStringWithU8Length(os, username); - writeStringWithU8Length(os, password); - writeStringWithU8Length(os, path); - writeU8(os, channelNo); - writeBcdTime(os, startTime); - writeBcdTime(os, endTime); - writeU32(os, warnBit1); - writeU32(os, warnBit2); - writeU8(os, mediaType); - writeU8(os, streamType); - writeU8(os, storageType); - writeU8(os, condition); - return new Jt808Commands.DownlinkCommand(Jt1078MessageId.PLATFORM_FILE_UPLOAD, os.toByteArray()); - } - - /** 0x9207 文件上传控制。 */ - public static Jt808Commands.DownlinkCommand fileUploadControl(int responseSerialNo, int command) { - ByteArrayOutputStream os = new ByteArrayOutputStream(3); - writeU16(os, responseSerialNo); - writeU8(os, command); - return new Jt808Commands.DownlinkCommand(Jt1078MessageId.PLATFORM_FILE_UPLOAD_CONTROL, os.toByteArray()); - } - - private static void writeStringWithU8Length(ByteArrayOutputStream os, String value) { - byte[] bytes = value == null ? new byte[0] : value.getBytes(GBK); - if (bytes.length > 255) { - throw new IllegalArgumentException("jt1078 string field too long: " + bytes.length); - } - writeU8(os, bytes.length); - os.write(bytes, 0, bytes.length); - } - - private static void writeBcdTime(ByteArrayOutputStream os, String value) { - String digits = value == null ? "" : value.replaceAll("\\D", ""); - if (digits.length() == 14 && digits.startsWith("20")) { - digits = digits.substring(2); - } - if (digits.isBlank()) { - digits = "000000000000"; - } - if (digits.length() != 12) { - throw new IllegalArgumentException("jt1078 time must be YYMMDDHHMMSS or yyyy-MM-dd HH:mm:ss: " + value); - } - for (int i = 0; i < digits.length(); i += 2) { - int high = Character.digit(digits.charAt(i), 10); - int low = Character.digit(digits.charAt(i + 1), 10); - if (high < 0 || low < 0) { - throw new IllegalArgumentException("jt1078 time contains non-decimal digit: " + value); - } - os.write((high << 4) | low); - } - } - - private static void writeU8(ByteArrayOutputStream os, int v) { - os.write(v & 0xFF); - } - - private static void writeU16(ByteArrayOutputStream os, int v) { - os.write((v >> 8) & 0xFF); - os.write(v & 0xFF); - } - - private static void writeU32(ByteArrayOutputStream os, long v) { - os.write((int) ((v >> 24) & 0xFF)); - os.write((int) ((v >> 16) & 0xFF)); - os.write((int) ((v >> 8) & 0xFF)); - os.write((int) (v & 0xFF)); - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/handler/Jt1078MalformedRtpHandler.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/handler/Jt1078MalformedRtpHandler.java deleted file mode 100644 index 5939384f..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/handler/Jt1078MalformedRtpHandler.java +++ /dev/null @@ -1,84 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.handler; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.annotation.EventEmit; -import com.lingniu.ingest.api.annotation.MessageMapping; -import com.lingniu.ingest.api.annotation.ProtocolHandler; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentityLookup; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MalformedRtpPacket; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MediaMessageId; - -import java.time.Instant; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.Objects; -import java.util.UUID; - -@ProtocolHandler(protocol = ProtocolId.JT1078) -public final class Jt1078MalformedRtpHandler { - - private final VehicleIdentityResolver identityResolver; - - public Jt1078MalformedRtpHandler(VehicleIdentityResolver identityResolver) { - this.identityResolver = Objects.requireNonNull(identityResolver, "identityResolver must not be null"); - } - - @MessageMapping(command = Jt1078MediaMessageId.MALFORMED_RTP, desc = "JT1078 RTP 入口坏包兜底") - @EventEmit(VehicleEvent.Passthrough.class) - public List onMalformedRtp(Jt1078MalformedRtpPacket malformed) { - if (malformed == null) { - return List.of(); - } - Instant eventTime = malformed.receivedAt() == null ? Instant.now() : malformed.receivedAt(); - IdentityResolution identity = resolveIdentity(malformed); - Map metadata = new HashMap<>(); - metadata.put("transport", safe(malformed.transport())); - metadata.put("peer", safe(malformed.peer())); - metadata.put("sim", safe(malformed.sim())); - metadata.put("vin", identity.identity().vin()); - metadata.put("identityResolved", Boolean.toString(identity.identity().resolved())); - metadata.put("identitySource", identity.identity().source().name()); - if (identity.errorMessage() != null) { - metadata.put("identityError", "true"); - metadata.put("identityErrorMessage", identity.errorMessage()); - } - metadata.put("malformedRtp", "true"); - metadata.put("parseError", "true"); - metadata.put("parseErrorMessage", safe(malformed.reason())); - metadata.put("packetLength", Integer.toString(malformed.packetLength())); - metadata.put("reason", safe(malformed.reason())); - return List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - identity.identity().vin(), - ProtocolId.JT1078, - eventTime, - Instant.now(), - null, - Map.copyOf(metadata), - Jt1078MediaMessageId.MALFORMED_RTP, - malformed.rawBytes() == null ? new byte[0] : malformed.rawBytes())); - } - - private IdentityResolution resolveIdentity(Jt1078MalformedRtpPacket malformed) { - try { - return new IdentityResolution(identityResolver.resolve(new VehicleIdentityLookup( - ProtocolId.JT1078, "", safe(malformed.sim()), "", "")), null); - } catch (RuntimeException e) { - return new IdentityResolution( - new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN), - e.getMessage() == null ? "" : e.getMessage()); - } - } - - private static String safe(String value) { - return value == null ? "" : value; - } - - private record IdentityResolution(VehicleIdentity identity, String errorMessage) { - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/handler/Jt1078MediaSignalHandler.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/handler/Jt1078MediaSignalHandler.java deleted file mode 100644 index 9d7b56bd..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/handler/Jt1078MediaSignalHandler.java +++ /dev/null @@ -1,36 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.handler; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.annotation.EventEmit; -import com.lingniu.ingest.api.annotation.MessageMapping; -import com.lingniu.ingest.api.annotation.ProtocolHandler; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.protocol.jt1078.model.Jt1078MessageId; -import com.lingniu.ingest.protocol.jt808.mapper.Jt808EventMapper; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; - -import java.util.List; - -/** - * JT/T 1078 信令复用 JT808 包头、连接与 dispatcher，但路由独立放在 1078 模块内。 - */ -@ProtocolHandler(protocol = ProtocolId.JT808) -public final class Jt1078MediaSignalHandler { - - private final Jt808EventMapper mapper; - - public Jt1078MediaSignalHandler(Jt808EventMapper mapper) { - this.mapper = mapper; - } - - @MessageMapping(command = { - Jt1078MessageId.TERMINAL_MEDIA_ATTRS_REPORT, - Jt1078MessageId.TERMINAL_PASSENGER_VOLUME, - Jt1078MessageId.TERMINAL_FILE_LIST, - Jt1078MessageId.TERMINAL_FILE_UPLOAD_DONE - }, desc = "JT/T 1078 音视频信令") - @EventEmit({VehicleEvent.MediaMeta.class, VehicleEvent.Passthrough.class}) - public List onMediaSignal(Jt808Message msg) { - return mapper.toEvents(msg); - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MalformedRtpEventPublisher.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MalformedRtpEventPublisher.java deleted file mode 100644 index 5ebff586..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MalformedRtpEventPublisher.java +++ /dev/null @@ -1,76 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentityLookup; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.identity.VehicleIdentitySource; - -import java.time.Instant; -import java.util.HashMap; -import java.util.Map; -import java.util.Objects; -import java.util.function.Consumer; - -public final class Jt1078MalformedRtpEventPublisher { - - private final VehicleIdentityResolver identityResolver; - private final Consumer dispatcher; - - public Jt1078MalformedRtpEventPublisher(VehicleIdentityResolver identityResolver, - Consumer dispatcher) { - this.identityResolver = Objects.requireNonNull(identityResolver, "identityResolver must not be null"); - this.dispatcher = Objects.requireNonNull(dispatcher, "dispatcher must not be null"); - } - - public void publish(Jt1078MalformedRtpPacket malformed) { - if (malformed == null) { - return; - } - Instant now = malformed.receivedAt() == null ? Instant.now() : malformed.receivedAt(); - IdentityResolution identity = resolveIdentity(malformed); - Map metadata = new HashMap<>(); - metadata.put("transport", safe(malformed.transport())); - metadata.put("peer", safe(malformed.peer())); - metadata.put("sim", safe(malformed.sim())); - metadata.put("identityResolved", Boolean.toString(identity.identity().resolved())); - metadata.put("identitySource", identity.identity().source().name()); - if (identity.errorMessage() != null) { - metadata.put("identityError", "true"); - metadata.put("identityErrorMessage", identity.errorMessage()); - } - metadata.put("malformedRtp", "true"); - metadata.put("parseError", "true"); - metadata.put("parseErrorMessage", safe(malformed.reason())); - metadata.put("packetLength", Integer.toString(malformed.packetLength())); - metadata.put("reason", safe(malformed.reason())); - metadata.put("vin", identity.identity().vin()); - dispatcher.accept(new RawFrame( - ProtocolId.JT1078, - Jt1078MediaMessageId.MALFORMED_RTP, - 0, - malformed, - malformed.rawBytes() == null ? new byte[0] : malformed.rawBytes(), - Map.copyOf(metadata), - now - )); - } - - private IdentityResolution resolveIdentity(Jt1078MalformedRtpPacket malformed) { - try { - return new IdentityResolution(identityResolver.resolve(new VehicleIdentityLookup( - ProtocolId.JT1078, "", malformed.sim(), "", "")), null); - } catch (RuntimeException e) { - return new IdentityResolution( - new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN), - e.getMessage() == null ? "" : e.getMessage()); - } - } - - private static String safe(String value) { - return value == null ? "" : value; - } - - private record IdentityResolution(VehicleIdentity identity, String errorMessage) {} -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MalformedRtpPacket.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MalformedRtpPacket.java deleted file mode 100644 index c31e0a72..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MalformedRtpPacket.java +++ /dev/null @@ -1,22 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -import java.time.Instant; - -public record Jt1078MalformedRtpPacket( - String transport, - String peer, - String sim, - byte[] rawBytes, - int packetLength, - String reason, - Instant receivedAt -) { - public Jt1078MalformedRtpPacket(String transport, - String peer, - byte[] rawBytes, - int packetLength, - String reason, - Instant receivedAt) { - this(transport, peer, "", rawBytes, packetLength, reason, receivedAt); - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaArchiveService.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaArchiveService.java deleted file mode 100644 index 352b72c0..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaArchiveService.java +++ /dev/null @@ -1,197 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -import com.github.benmanes.caffeine.cache.Cache; -import com.github.benmanes.caffeine.cache.Caffeine; -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentityLookup; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.time.Duration; -import java.time.Instant; -import java.time.ZoneId; -import java.time.format.DateTimeFormatter; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.UUID; -import java.util.function.Consumer; - -public final class Jt1078MediaArchiveService { - - private static final Logger log = LoggerFactory.getLogger(Jt1078MediaArchiveService.class); - private static final DateTimeFormatter DAY_FMT = DateTimeFormatter.ofPattern("yyyy/MM/dd").withZone(ZoneId.systemDefault()); - - private final ArchiveStore archive; - private final VehicleIdentityResolver identityResolver; - private final Consumer eventPublisher; - private final int segmentSeconds; - private final Cache publishedSegmentRefs = Caffeine.newBuilder() - .expireAfterAccess(Duration.ofHours(2)) - .maximumSize(200_000) - .build(); - - public Jt1078MediaArchiveService(ArchiveStore archive, - VehicleIdentityResolver identityResolver, - Consumer eventPublisher, - int segmentSeconds) { - this.archive = archive; - this.identityResolver = identityResolver; - this.eventPublisher = eventPublisher; - this.segmentSeconds = Math.max(60, segmentSeconds); - } - - public void archive(Jt1078RtpPacket packet, String peer) { - if (packet == null || packet.payload() == null || packet.payload().length == 0) { - return; - } - IdentityResolution identity = resolveIdentity(packet); - String vin = identity.identity().vin(); - long segment = segment(packet.timestamp()); - String key = key(vin, packet, segment); - try { - // JT1078 是 RTP 分片流，同一分段追加写入 .media；只发布一次 MediaMeta 作为索引事件。 - String uri = archive.append(key, packet.payload()); - long segmentSizeBytes = archiveSize(key, packet.payload().length); - publishOnce(packet, peer, identity, vin, segment, key, uri, segmentSizeBytes); - } catch (Exception e) { - log.warn("[jt1078] media archive failed sim={} channel={} sequence={} peer={}", - packet.sim(), packet.channelId(), packet.sequence(), peer, e); - publishArchiveFailure(packet, peer, identity, vin, segment, key, e); - } - } - - private void publishArchiveFailure(Jt1078RtpPacket packet, - String peer, - IdentityResolution identity, - String vin, - long segment, - String archiveKey, - Exception error) { - Instant now = Instant.now(); - Map metadata = baseMetadata(packet, peer, identity); - metadata.put("sequence", Integer.toString(packet.sequence())); - metadata.put("segment", Long.toString(segment)); - metadata.put("archiveKey", archiveKey == null ? "" : archiveKey); - metadata.put("archiveError", "true"); - metadata.put("archiveErrorMessage", error.getMessage() == null ? "" : error.getMessage()); - eventPublisher.accept(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - vin, - ProtocolId.JT1078, - now, - now, - null, - Map.copyOf(metadata), - packet.dataType(), - packet.payload())); - } - - private void publishOnce(Jt1078RtpPacket packet, - String peer, - IdentityResolution identity, - String vin, - long segment, - String archiveKey, - String uri, - long segmentSizeBytes) { - String segmentKey = archiveKey == null || archiveKey.isBlank() - ? packet.sim() + ':' + packet.channelId() + ':' + packet.dataType() + ':' + segment - : archiveKey; - String old = publishedSegmentRefs.getIfPresent(segmentKey); - if (old != null) { - // 同一个分段只发一次索引事件，避免每个 RTP 包都在历史库产生一条 media meta。 - return; - } - publishedSegmentRefs.put(segmentKey, uri); - Instant now = Instant.now(); - Map metadata = baseMetadata(packet, peer, identity); - metadata.put("segment", Long.toString(segment)); - metadata.put("sequence", Integer.toString(packet.sequence())); - metadata.put("segmentSizeBytes", Long.toString(segmentSizeBytes)); - metadata.put("archiveKey", archiveKey == null ? "" : archiveKey); - eventPublisher.accept(new VehicleEvent.MediaMeta( - UUID.randomUUID().toString(), - vin, - ProtocolId.JT1078, - now, - now, - null, - Map.copyOf(metadata), - packet.sim() + "-" + packet.channelId() + "-" + segment, - mediaType(packet), - segmentSizeBytes, - uri)); - } - - private long archiveSize(String key, int fallbackBytes) { - try { - return archive.size(key); - } catch (Exception e) { - return Math.max(0, fallbackBytes); - } - } - - private IdentityResolution resolveIdentity(Jt1078RtpPacket packet) { - try { - return new IdentityResolution(identityResolver.resolve(new VehicleIdentityLookup( - ProtocolId.JT1078, "", packet.sim(), "", "")), null); - } catch (RuntimeException e) { - return new IdentityResolution( - new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN), - e.getMessage() == null ? "" : e.getMessage()); - } - } - - private static Map baseMetadata(Jt1078RtpPacket packet, - String peer, - IdentityResolution identity) { - Map metadata = new LinkedHashMap<>(); - metadata.put("sim", packet.sim()); - metadata.put("channelId", Integer.toString(packet.channelId())); - metadata.put("dataType", Integer.toString(packet.dataType())); - metadata.put("packetType", Integer.toString(packet.packetType())); - metadata.put("peer", peer == null ? "" : peer); - metadata.put("vin", identity.identity().vin()); - metadata.put("identityResolved", Boolean.toString(identity.identity().resolved())); - metadata.put("identitySource", identity.identity().source().name()); - if (identity.errorMessage() != null) { - metadata.put("identityError", "true"); - metadata.put("identityErrorMessage", identity.errorMessage()); - } - return metadata; - } - - private long segment(long timestamp) { - long seconds = timestamp > 10_000_000_000L ? timestamp / 1000 : timestamp; - // 按配置秒数对齐分段，保证同一路通道同一时间窗落到同一个 archive key。 - return seconds - Math.floorMod(seconds, segmentSeconds); - } - - private static String key(String vin, Jt1078RtpPacket packet, long segment) { - String day = DAY_FMT.format(Instant.ofEpochSecond(Math.max(0, segment))); - String safeVin = safe(vin); - return "jt1078/" + day + "/" + safeVin + "/ch-" + packet.channelId() - + "/dt-" + packet.dataType() + "/" + segment + ".media"; - } - - private static String mediaType(Jt1078RtpPacket packet) { - return "rtp,dataType=" + packet.dataType() - + ",packetType=" + packet.packetType() - + ",channel=" + packet.channelId(); - } - - private static String safe(String raw) { - if (raw == null || raw.isBlank()) { - return "unknown"; - } - return raw.replaceAll("[^A-Za-z0-9_.-]", "_"); - } - - private record IdentityResolution(VehicleIdentity identity, String errorMessage) { - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaMessageId.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaMessageId.java deleted file mode 100644 index 5ee98aeb..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaMessageId.java +++ /dev/null @@ -1,9 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -public final class Jt1078MediaMessageId { - - /** 内部消息：JT1078 RTP 入口坏包，走 Dispatcher 统一 RawArchive + Passthrough 链路。 */ - public static final int MALFORMED_RTP = 0x72747065; // "rtpe" - - private Jt1078MediaMessageId() {} -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaStreamHandler.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaStreamHandler.java deleted file mode 100644 index d4bd4e66..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaStreamHandler.java +++ /dev/null @@ -1,36 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -import io.netty.channel.ChannelHandlerContext; -import io.netty.channel.SimpleChannelInboundHandler; - -import java.net.InetSocketAddress; - -public final class Jt1078MediaStreamHandler extends SimpleChannelInboundHandler { - - private final Jt1078MediaArchiveService archiveService; - private final Jt1078MalformedRtpEventPublisher malformedPublisher; - - public Jt1078MediaStreamHandler(Jt1078MediaArchiveService archiveService, - Jt1078MalformedRtpEventPublisher malformedPublisher) { - this.archiveService = archiveService; - this.malformedPublisher = malformedPublisher; - } - - @Override - protected void channelRead0(ChannelHandlerContext ctx, Object msg) { - if (msg instanceof Jt1078RtpPacket packet) { - archiveService.archive(packet, addr(ctx)); - } else if (msg instanceof Jt1078MalformedRtpPacket malformed) { - malformedPublisher.publish(malformed); - } else { - ctx.fireChannelRead(msg); - } - } - - private static String addr(ChannelHandlerContext ctx) { - if (ctx.channel().remoteAddress() instanceof InetSocketAddress i) { - return i.getAddress().getHostAddress() + ":" + i.getPort(); - } - return "unknown"; - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaStreamServer.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaStreamServer.java deleted file mode 100644 index 476f0863..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078MediaStreamServer.java +++ /dev/null @@ -1,83 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -import io.netty.bootstrap.ServerBootstrap; -import io.netty.channel.Channel; -import io.netty.channel.ChannelInitializer; -import io.netty.channel.ChannelOption; -import io.netty.channel.EventLoopGroup; -import io.netty.channel.nio.NioEventLoopGroup; -import io.netty.channel.socket.SocketChannel; -import io.netty.channel.socket.nio.NioServerSocketChannel; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.beans.factory.DisposableBean; -import org.springframework.beans.factory.InitializingBean; - -import java.util.concurrent.ThreadFactory; - -public final class Jt1078MediaStreamServer implements InitializingBean, DisposableBean { - - private static final Logger log = LoggerFactory.getLogger(Jt1078MediaStreamServer.class); - - private final int port; - private final int workerThreads; - private final int maxPacketLength; - private final Jt1078MediaArchiveService archiveService; - private final Jt1078MalformedRtpEventPublisher malformedPublisher; - - private EventLoopGroup boss; - private EventLoopGroup workers; - private Channel serverChannel; - - public Jt1078MediaStreamServer(int port, - int workerThreads, - int maxPacketLength, - Jt1078MediaArchiveService archiveService, - Jt1078MalformedRtpEventPublisher malformedPublisher) { - this.port = port; - this.workerThreads = workerThreads; - this.maxPacketLength = maxPacketLength; - this.archiveService = archiveService; - this.malformedPublisher = malformedPublisher; - } - - @Override - public void afterPropertiesSet() throws Exception { - ThreadFactory bossTf = r -> new Thread(r, "jt1078-media-boss"); - ThreadFactory wkTf = r -> new Thread(r, "jt1078-media-worker"); - this.boss = new NioEventLoopGroup(1, bossTf); - this.workers = new NioEventLoopGroup( - workerThreads == 0 ? Runtime.getRuntime().availableProcessors() : workerThreads, - wkTf); - - ServerBootstrap b = new ServerBootstrap(); - b.group(boss, workers) - .channel(NioServerSocketChannel.class) - .option(ChannelOption.SO_BACKLOG, 1024) - .childOption(ChannelOption.TCP_NODELAY, true) - .childOption(ChannelOption.SO_KEEPALIVE, true) - .childHandler(new ChannelInitializer() { - @Override - protected void initChannel(SocketChannel ch) { - ch.pipeline() - .addLast("rtp", new Jt1078RtpPacketDecoder(maxPacketLength)) - .addLast("archive", new Jt1078MediaStreamHandler(archiveService, malformedPublisher)); - } - }); - this.serverChannel = b.bind(port).sync().channel(); - log.info("[jt1078] media stream server listening on :{}", port); - } - - @Override - public void destroy() { - try { - if (serverChannel != null) serverChannel.close().sync(); - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); - } finally { - if (boss != null) boss.shutdownGracefully(); - if (workers != null) workers.shutdownGracefully(); - log.info("[jt1078] media stream server stopped"); - } - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078ReceivedMediaStreamHandler.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078ReceivedMediaStreamHandler.java deleted file mode 100644 index 3ebf6ab9..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078ReceivedMediaStreamHandler.java +++ /dev/null @@ -1,27 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -import io.netty.channel.SimpleChannelInboundHandler; -import io.netty.channel.ChannelHandlerContext; - -public final class Jt1078ReceivedMediaStreamHandler extends SimpleChannelInboundHandler { - - private final Jt1078MediaArchiveService archiveService; - private final Jt1078MalformedRtpEventPublisher malformedPublisher; - - public Jt1078ReceivedMediaStreamHandler(Jt1078MediaArchiveService archiveService, - Jt1078MalformedRtpEventPublisher malformedPublisher) { - this.archiveService = archiveService; - this.malformedPublisher = malformedPublisher; - } - - @Override - protected void channelRead0(ChannelHandlerContext ctx, Object msg) { - if (msg instanceof Jt1078ReceivedRtpPacket received) { - archiveService.archive(received.packet(), received.peer()); - } else if (msg instanceof Jt1078MalformedRtpPacket malformed) { - malformedPublisher.publish(malformed); - } else { - ctx.fireChannelRead(msg); - } - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078ReceivedRtpPacket.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078ReceivedRtpPacket.java deleted file mode 100644 index 7405bc5e..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078ReceivedRtpPacket.java +++ /dev/null @@ -1,10 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -public record Jt1078ReceivedRtpPacket( - Jt1078RtpPacket packet, - String peer -) { - public Jt1078ReceivedRtpPacket { - peer = peer == null ? "" : peer; - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078RtpPacket.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078RtpPacket.java deleted file mode 100644 index 92d53e46..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078RtpPacket.java +++ /dev/null @@ -1,11 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -public record Jt1078RtpPacket( - int sequence, - String sim, - int channelId, - int dataType, - int packetType, - long timestamp, - byte[] payload -) {} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078RtpPacketDecoder.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078RtpPacketDecoder.java deleted file mode 100644 index 953743ae..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078RtpPacketDecoder.java +++ /dev/null @@ -1,83 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -import io.netty.buffer.ByteBuf; -import io.netty.channel.ChannelHandlerContext; -import io.netty.handler.codec.ByteToMessageDecoder; - -import java.time.Instant; -import java.util.List; - -/** - * JT/T 1078 TCP RTP 包解码器。 - * - *

    包头固定魔数 0x30316364，header 固定 28 字节，随后是 bodyLength 指定的媒体负载。 - */ -public final class Jt1078RtpPacketDecoder extends ByteToMessageDecoder { - - private final int maxPacketLength; - - public Jt1078RtpPacketDecoder(int maxPacketLength) { - this.maxPacketLength = maxPacketLength; - } - - @Override - protected void decode(ChannelHandlerContext ctx, ByteBuf in, List out) throws Exception { - while (true) { - if (in.readableBytes() < Jt1078RtpPacketParser.HEADER_LEN) { - return; - } - int frameStart = findMagic(in); - if (frameStart < 0) { - in.skipBytes(Math.max(0, in.readableBytes() - 3)); - return; - } - if (frameStart > in.readerIndex()) { - in.skipBytes(frameStart - in.readerIndex()); - } - if (in.readableBytes() < Jt1078RtpPacketParser.HEADER_LEN) { - return; - } - int length = in.getUnsignedShort(in.readerIndex() + 26); - int packetLength = Jt1078RtpPacketParser.HEADER_LEN + length; - if (packetLength > maxPacketLength) { - String sim = Jt1078RtpPacketParser.peekSim(in); - int available = in.readableBytes(); - int rawLength = Math.min(available, packetLength); - byte[] raw = new byte[rawLength]; - in.readBytes(raw); - out.add(new Jt1078MalformedRtpPacket( - "tcp", - peer(ctx), - sim, - raw, - packetLength, - "jt1078 rtp packet too large: " + packetLength, - Instant.now())); - return; - } - if (in.readableBytes() < packetLength) { - return; - } - ByteBuf packet = in.readSlice(packetLength); - out.add(Jt1078RtpPacketParser.parse(packet)); - } - } - - private static String peer(ChannelHandlerContext ctx) { - if (ctx == null || ctx.channel() == null || ctx.channel().remoteAddress() == null) { - return ""; - } - return ctx.channel().remoteAddress().toString(); - } - - private static int findMagic(ByteBuf in) { - int start = in.readerIndex(); - int end = in.writerIndex() - 3; - for (int i = start; i < end; i++) { - if (in.getInt(i) == Jt1078RtpPacketParser.MAGIC) { - return i; - } - } - return -1; - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078RtpPacketParser.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078RtpPacketParser.java deleted file mode 100644 index 639f1691..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078RtpPacketParser.java +++ /dev/null @@ -1,61 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -import io.netty.buffer.ByteBuf; -import io.netty.handler.codec.CorruptedFrameException; - -final class Jt1078RtpPacketParser { - - static final int MAGIC = 0x30316364; - static final int HEADER_LEN = 28; - - private Jt1078RtpPacketParser() {} - - static Jt1078RtpPacket parse(ByteBuf in) { - int magic = in.readInt(); - if (magic != MAGIC) { - throw new CorruptedFrameException("jt1078 invalid magic: 0x" + Integer.toHexString(magic)); - } - int sequence = in.readUnsignedShort(); - String sim = readBcd(in, 6); - int channelId = in.readUnsignedByte(); - int dataAndPacketType = in.readUnsignedByte(); - // 高 4 位是数据类型，低 4 位是分包类型；归档 key 需要二者共同区分。 - int dataType = (dataAndPacketType >>> 4) & 0x0F; - int packetType = dataAndPacketType & 0x0F; - long timestamp = in.readLong(); - in.skipBytes(4); - int bodyLength = in.readUnsignedShort(); - if (bodyLength > in.readableBytes()) { - throw new CorruptedFrameException("jt1078 body length exceeds readable bytes: " - + bodyLength + " > " + in.readableBytes()); - } - byte[] payload = new byte[bodyLength]; - in.readBytes(payload); - return new Jt1078RtpPacket(sequence, sim, channelId, dataType, packetType, timestamp, payload); - } - - static String peekSim(ByteBuf in) { - if (in == null || in.readableBytes() < HEADER_LEN || in.getInt(in.readerIndex()) != MAGIC) { - return ""; - } - // UDP/TCP 解码器在完整 parse 前先取 SIM，用于日志和初步身份定位；不移动 readerIndex。 - StringBuilder sb = new StringBuilder(12); - int simOffset = in.readerIndex() + 6; - for (int i = 0; i < 6; i++) { - int b = in.getUnsignedByte(simOffset + i); - sb.append((b >>> 4) & 0x0F); - sb.append(b & 0x0F); - } - return sb.toString(); - } - - private static String readBcd(ByteBuf in, int len) { - StringBuilder sb = new StringBuilder(len * 2); - for (int i = 0; i < len; i++) { - int b = in.readUnsignedByte(); - sb.append((b >>> 4) & 0x0F); - sb.append(b & 0x0F); - } - return sb.toString(); - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078UdpMediaStreamServer.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078UdpMediaStreamServer.java deleted file mode 100644 index ffdd886e..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078UdpMediaStreamServer.java +++ /dev/null @@ -1,78 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -import io.netty.bootstrap.Bootstrap; -import io.netty.channel.Channel; -import io.netty.channel.ChannelInitializer; -import io.netty.channel.ChannelOption; -import io.netty.channel.EventLoopGroup; -import io.netty.channel.nio.NioEventLoopGroup; -import io.netty.channel.socket.DatagramChannel; -import io.netty.channel.socket.nio.NioDatagramChannel; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.beans.factory.DisposableBean; -import org.springframework.beans.factory.InitializingBean; - -import java.util.concurrent.ThreadFactory; - -public final class Jt1078UdpMediaStreamServer implements InitializingBean, DisposableBean { - - private static final Logger log = LoggerFactory.getLogger(Jt1078UdpMediaStreamServer.class); - - private final int port; - private final int workerThreads; - private final int maxPacketLength; - private final Jt1078MediaArchiveService archiveService; - private final Jt1078MalformedRtpEventPublisher malformedPublisher; - - private EventLoopGroup workers; - private Channel serverChannel; - - public Jt1078UdpMediaStreamServer(int port, - int workerThreads, - int maxPacketLength, - Jt1078MediaArchiveService archiveService, - Jt1078MalformedRtpEventPublisher malformedPublisher) { - this.port = port; - this.workerThreads = workerThreads; - this.maxPacketLength = maxPacketLength; - this.archiveService = archiveService; - this.malformedPublisher = malformedPublisher; - } - - @Override - public void afterPropertiesSet() throws Exception { - ThreadFactory wkTf = r -> new Thread(r, "jt1078-media-udp-worker"); - int threads = workerThreads == 0 ? Runtime.getRuntime().availableProcessors() : workerThreads; - this.workers = new NioEventLoopGroup(threads, wkTf); - - Bootstrap b = new Bootstrap(); - b.group(workers) - .channel(NioDatagramChannel.class) - .option(ChannelOption.SO_BROADCAST, false) - .option(ChannelOption.SO_RCVBUF, Math.max(1024 * 1024, maxPacketLength * 2)) - .handler(new ChannelInitializer() { - @Override - protected void initChannel(DatagramChannel ch) { - ch.pipeline() - .addLast("rtp", new Jt1078UdpRtpPacketDecoder(maxPacketLength)) - .addLast("archive", new Jt1078ReceivedMediaStreamHandler(archiveService, malformedPublisher)); - } - }); - this.serverChannel = b.bind(port).sync().channel(); - log.info("[jt1078] udp media stream server listening on :{} workers={} maxPacketLength={}", - port, threads, maxPacketLength); - } - - @Override - public void destroy() { - try { - if (serverChannel != null) serverChannel.close().sync(); - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); - } finally { - if (workers != null) workers.shutdownGracefully(); - log.info("[jt1078] udp media stream server stopped"); - } - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078UdpRtpPacketDecoder.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078UdpRtpPacketDecoder.java deleted file mode 100644 index f9bd4b6c..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/media/Jt1078UdpRtpPacketDecoder.java +++ /dev/null @@ -1,59 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.media; - -import io.netty.channel.ChannelHandlerContext; -import io.netty.channel.socket.DatagramPacket; -import io.netty.handler.codec.CorruptedFrameException; -import io.netty.handler.codec.MessageToMessageDecoder; - -import java.net.InetSocketAddress; -import java.time.Instant; -import java.util.List; - -public final class Jt1078UdpRtpPacketDecoder extends MessageToMessageDecoder { - - private final int maxPacketLength; - - public Jt1078UdpRtpPacketDecoder(int maxPacketLength) { - this.maxPacketLength = maxPacketLength; - } - - @Override - protected void decode(ChannelHandlerContext ctx, DatagramPacket msg, List out) throws Exception { - int packetLength = msg.content().readableBytes(); - if (packetLength > maxPacketLength) { - out.add(malformed(msg, packetLength, "jt1078 udp rtp packet too large: " + packetLength)); - return; - } - if (packetLength < Jt1078RtpPacketParser.HEADER_LEN) { - out.add(malformed(msg, packetLength, "jt1078 udp rtp packet too short: " + packetLength)); - return; - } - try { - out.add(new Jt1078ReceivedRtpPacket( - Jt1078RtpPacketParser.parse(msg.content().slice()), - peer(msg.sender()))); - } catch (CorruptedFrameException | IndexOutOfBoundsException e) { - out.add(malformed(msg, packetLength, e.getMessage())); - } - } - - private static Jt1078MalformedRtpPacket malformed(DatagramPacket msg, int packetLength, String reason) { - byte[] raw = new byte[Math.max(0, msg.content().readableBytes())]; - msg.content().getBytes(msg.content().readerIndex(), raw); - return new Jt1078MalformedRtpPacket( - "udp", - peer(msg.sender()), - Jt1078RtpPacketParser.peekSim(msg.content().slice()), - raw, - packetLength, - reason == null ? "" : reason, - Instant.now()); - } - - private static String peer(InetSocketAddress sender) { - if (sender == null) { - return ""; - } - return sender.getAddress().getHostAddress() + ":" + sender.getPort(); - } -} diff --git a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/model/Jt1078MessageId.java b/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/model/Jt1078MessageId.java deleted file mode 100644 index de31a886..00000000 --- a/modules/protocols/protocol-jt1078/src/main/java/com/lingniu/ingest/protocol/jt1078/model/Jt1078MessageId.java +++ /dev/null @@ -1,21 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.model; - -/** JT/T 1078 消息 ID 常量（核心信令子集）。 */ -public final class Jt1078MessageId { - - private Jt1078MessageId() {} - - public static final int TERMINAL_MEDIA_ATTRS_REPORT = 0x1003; - public static final int TERMINAL_PASSENGER_VOLUME = 0x1005; - public static final int TERMINAL_FILE_LIST = 0x1205; - public static final int TERMINAL_FILE_UPLOAD_DONE = 0x1206; - - public static final int PLATFORM_QUERY_MEDIA_ATTRS = 0x1003; - public static final int PLATFORM_REALTIME_PLAY = 0x9101; - public static final int PLATFORM_REALTIME_CONTROL = 0x9102; - public static final int PLATFORM_HISTORY_PLAY = 0x9201; - public static final int PLATFORM_HISTORY_CONTROL = 0x9202; - public static final int PLATFORM_RESOURCE_SEARCH = 0x9205; - public static final int PLATFORM_FILE_UPLOAD = 0x9206; - public static final int PLATFORM_FILE_UPLOAD_CONTROL = 0x9207; -} diff --git a/modules/protocols/protocol-jt1078/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/protocols/protocol-jt1078/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index 8539eae3..00000000 --- a/modules/protocols/protocol-jt1078/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1,2 +0,0 @@ -com.lingniu.ingest.protocol.jt1078.config.Jt1078AutoConfiguration -com.lingniu.ingest.protocol.jt1078.config.Jt1078SignalAutoConfiguration diff --git a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078EventMappingTest.java b/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078EventMappingTest.java deleted file mode 100644 index 522616ea..00000000 --- a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078EventMappingTest.java +++ /dev/null @@ -1,93 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078; - -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.protocol.jt1078.model.Jt1078MessageId; -import com.lingniu.ingest.protocol.jt808.mapper.Jt808EventMapper; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import org.junit.jupiter.api.Test; - -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt1078EventMappingTest { - - private final Jt808EventMapper mapper = new Jt808EventMapper(new InMemoryVehicleIdentityService()); - private final Jt808Header header = new Jt808Header( - 0x1003, 0, 0, false, Jt808Header.ProtocolVersion.V2013, "13800138000", 1, 0, 0); - - @Test - void mediaAttributesProduceQueryableMediaMetaEvent() { - Jt808Body.MediaAttributes attrs = new Jt808Body.MediaAttributes(1, 2, 3, 4, 320, true, 98, 4, 8, new byte[]{1, 2}); - - List events = mapper.toEvents(new Jt808Message(header, attrs)); - - assertThat(events).hasSize(1); - assertThat(events.getFirst()).isInstanceOf(VehicleEvent.MediaMeta.class); - VehicleEvent.MediaMeta event = (VehicleEvent.MediaMeta) events.getFirst(); - assertThat(event.mediaType()).contains("videoEncoding=98"); - assertThat(event.metadata()).containsEntry("jt1078Message", "mediaAttributes"); - } - - @Test - void fileUploadCompleteProducesPassthroughEventWithResultMetadata() { - Jt808Body.FileUploadComplete done = new Jt808Body.FileUploadComplete(0x1234, 0, new byte[]{0x12, 0x34, 0}); - - List events = mapper.toEvents(new Jt808Message(header, done)); - - assertThat(events).hasSize(1); - assertThat(events.getFirst()).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough event = (VehicleEvent.Passthrough) events.getFirst(); - assertThat(event.metadata()).containsEntry("jt1078Message", "fileUploadComplete"); - assertThat(event.metadata()).containsEntry("uploadResult", "0"); - } - - @Test - void fileListProducesPassthroughEventWithQueryableSummaryMetadata() { - Jt808Header fileListHeader = new Jt808Header( - Jt1078MessageId.TERMINAL_FILE_LIST, 0, 0, false, - Jt808Header.ProtocolVersion.V2013, "13800138000", 1, 0, 0); - byte[] raw = new byte[]{0x12, 0x34, 0, 0, 0, 2}; - Jt808Body.FileList fileList = new Jt808Body.FileList(0x1234, 2, raw); - - List events = mapper.toEvents(new Jt808Message(fileListHeader, fileList)); - - assertThat(events).hasSize(1); - assertThat(events.getFirst()).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough event = (VehicleEvent.Passthrough) events.getFirst(); - assertThat(event.passthroughType()).isEqualTo(Jt1078MessageId.TERMINAL_FILE_LIST); - assertThat(event.data()).containsExactly(raw); - assertThat(event.metadata()) - .containsEntry("jt1078Message", "fileList") - .containsEntry("responseSerialNo", "4660") - .containsEntry("fileCount", "2"); - } - - @Test - void passengerVolumeProducesPassthroughEventWithQueryableMetadata() { - Jt808Header passengerHeader = new Jt808Header( - Jt1078MessageId.TERMINAL_PASSENGER_VOLUME, 0, 0, false, - Jt808Header.ProtocolVersion.V2013, "13800138000", 1, 0, 0); - byte[] raw = new byte[]{1, 2, 3}; - Jt808Body.PassengerVolume passengerVolume = new Jt808Body.PassengerVolume( - 2, "2026-06-22T15:30:00", "2026-06-22T16:00:00", 21, 3, raw); - - List events = mapper.toEvents(new Jt808Message(passengerHeader, passengerVolume)); - - assertThat(events).hasSize(1); - assertThat(events.getFirst()).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough event = (VehicleEvent.Passthrough) events.getFirst(); - assertThat(event.passthroughType()).isEqualTo(Jt1078MessageId.TERMINAL_PASSENGER_VOLUME); - assertThat(event.data()).containsExactly(raw); - assertThat(event.metadata()) - .containsEntry("jt1078Message", "passengerVolume") - .containsEntry("channelId", "2") - .containsEntry("startTime", "2026-06-22T15:30:00") - .containsEntry("endTime", "2026-06-22T16:00:00") - .containsEntry("passengerGetOn", "21") - .containsEntry("passengerGetOff", "3"); - } -} diff --git a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078MalformedRtpEventPublisherTest.java b/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078MalformedRtpEventPublisherTest.java deleted file mode 100644 index 5e77dc33..00000000 --- a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078MalformedRtpEventPublisherTest.java +++ /dev/null @@ -1,109 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.identity.VehicleIdentityBinding; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MalformedRtpEventPublisher; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MalformedRtpPacket; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MediaMessageId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.ArrayList; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt1078MalformedRtpEventPublisherTest { - - @Test - void malformedRtpPacketProducesRawFrameForUnifiedArchiveAndPassthrough() { - ArrayList frames = new ArrayList<>(); - Jt1078MalformedRtpEventPublisher publisher = new Jt1078MalformedRtpEventPublisher( - new InMemoryVehicleIdentityService(), frames::add); - Instant receivedAt = Instant.parse("2026-06-22T08:30:00Z"); - byte[] raw = new byte[]{0x30, 0x31, 0x63, 0x64}; - - publisher.publish(new Jt1078MalformedRtpPacket( - "udp", "10.0.0.8:1078", raw, raw.length, "jt1078 udp rtp packet too short: 4", receivedAt)); - - assertThat(frames).hasSize(1); - RawFrame frame = frames.getFirst(); - assertThat(frame.protocolId()).isEqualTo(ProtocolId.JT1078); - assertThat(frame.command()).isEqualTo(Jt1078MediaMessageId.MALFORMED_RTP); - assertThat(frame.payload()).isInstanceOf(Jt1078MalformedRtpPacket.class); - assertThat(frame.rawBytes()).containsExactly(raw); - assertThat(frame.receivedAt()).isEqualTo(receivedAt); - assertThat(frame.sourceMeta()).containsEntry("transport", "udp") - .containsEntry("peer", "10.0.0.8:1078") - .containsEntry("malformedRtp", "true") - .containsEntry("parseError", "true") - .containsEntry("parseErrorMessage", "jt1078 udp rtp packet too short: 4") - .containsEntry("packetLength", "4") - .containsEntry("reason", "jt1078 udp rtp packet too short: 4"); - } - - @Test - void malformedRtpPacketUsesSharedIdentityWhenSimIsAvailable() { - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - identity.bind(new VehicleIdentityBinding( - ProtocolId.JT1078, "LNVIN107800000001", "013812345678", "", "")); - ArrayList frames = new ArrayList<>(); - Jt1078MalformedRtpEventPublisher publisher = new Jt1078MalformedRtpEventPublisher(identity, frames::add); - Instant receivedAt = Instant.parse("2026-06-22T08:30:00Z"); - byte[] raw = new byte[]{0x30, 0x31, 0x63, 0x64, 0, 1}; - - publisher.publish(new Jt1078MalformedRtpPacket( - "tcp", - "10.0.0.8:1078", - "013812345678", - raw, - 128, - "jt1078 rtp packet too large: 128", - receivedAt)); - - assertThat(frames).hasSize(1); - RawFrame frame = frames.getFirst(); - assertThat(frame.sourceMeta()) - .containsEntry("vin", "LNVIN107800000001") - .containsEntry("sim", "013812345678") - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_PHONE"); - } - - @Test - void identityResolverFailureStillPublishesMalformedRtpWithOriginalPayload() { - ArrayList frames = new ArrayList<>(); - Jt1078MalformedRtpEventPublisher publisher = new Jt1078MalformedRtpEventPublisher( - lookup -> { - throw new IllegalStateException("identity backend unavailable"); - }, - frames::add); - Instant receivedAt = Instant.parse("2026-06-22T08:30:00Z"); - byte[] raw = new byte[]{0x30, 0x31, 0x63, 0x64, 0, 1}; - - publisher.publish(new Jt1078MalformedRtpPacket( - "tcp", - "10.0.0.8:1078", - "013812345678", - raw, - raw.length, - "jt1078 rtp packet too short: 6", - receivedAt)); - - assertThat(frames).singleElement().satisfies(frame -> { - assertThat(frame.protocolId()).isEqualTo(ProtocolId.JT1078); - assertThat(frame.command()).isEqualTo(Jt1078MediaMessageId.MALFORMED_RTP); - assertThat(frame.rawBytes()).containsExactly(raw); - assertThat(frame.sourceMeta()) - .containsEntry("vin", "unknown") - .containsEntry("sim", "013812345678") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN") - .containsEntry("identityError", "true") - .containsEntry("identityErrorMessage", "identity backend unavailable") - .containsEntry("parseError", "true") - .containsEntry("parseErrorMessage", "jt1078 rtp packet too short: 6"); - }); - } -} diff --git a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078MalformedRtpHandlerTest.java b/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078MalformedRtpHandlerTest.java deleted file mode 100644 index 35f2bea0..00000000 --- a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078MalformedRtpHandlerTest.java +++ /dev/null @@ -1,134 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.RawArchiveKeys; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.identity.VehicleIdentityBinding; -import com.lingniu.ingest.protocol.jt1078.handler.Jt1078MalformedRtpHandler; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MalformedRtpPacket; -import org.junit.jupiter.api.Test; - -import java.lang.reflect.Constructor; -import java.time.Instant; -import java.util.Arrays; -import java.util.List; -import java.util.function.Consumer; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt1078MalformedRtpHandlerTest { - - @Test - void malformedRtpComponentsRequireInjectedIdentityResolver() { - assertThat(Arrays.stream(Jt1078MalformedRtpHandler.class.getConstructors()) - .map(Constructor::getParameterCount)) - .doesNotContain(0); - assertThat(Arrays.stream(com.lingniu.ingest.protocol.jt1078.media.Jt1078MalformedRtpEventPublisher.class - .getConstructors())) - .noneSatisfy(constructor -> assertThat(constructor.getParameterTypes()) - .containsExactly(Consumer.class)); - } - - @Test - void malformedRtpRawFramePayloadMapsToPassthroughWithDiagnostics() { - byte[] raw = new byte[]{0x30, 0x31, 0x63, 0x64}; - Jt1078MalformedRtpHandler handler = new Jt1078MalformedRtpHandler(new InMemoryVehicleIdentityService()); - - List events = handler.onMalformedRtp(new Jt1078MalformedRtpPacket( - "udp", - "10.0.0.8:1078", - raw, - raw.length, - "jt1078 udp rtp packet too short: 4", - Instant.parse("2026-06-22T08:30:00Z"))); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(ProtocolId.JT1078); - assertThat(passthrough.vin()).isEqualTo("unknown"); - assertThat(passthrough.eventTime()).isEqualTo(Instant.parse("2026-06-22T08:30:00Z")); - assertThat(passthrough.data()).containsExactly(raw); - assertThat(passthrough.metadata()) - .containsEntry("transport", "udp") - .containsEntry("peer", "10.0.0.8:1078") - .containsEntry("vin", "unknown") - .containsEntry("malformedRtp", "true") - .containsEntry("parseError", "true") - .containsEntry("parseErrorMessage", "jt1078 udp rtp packet too short: 4") - .containsEntry("packetLength", "4"); - }); - } - - @Test - void malformedRtpHandlerResolvesVinBySim() { - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - identity.bind(new VehicleIdentityBinding( - ProtocolId.JT1078, "LNVIN107800000001", "013812345678", "", "")); - Jt1078MalformedRtpHandler handler = new Jt1078MalformedRtpHandler(identity); - - List events = handler.onMalformedRtp(new Jt1078MalformedRtpPacket( - "tcp", - "10.0.0.8:1078", - "013812345678", - new byte[]{0x30, 0x31, 0x63, 0x64}, - 128, - "jt1078 rtp packet too large: 128", - Instant.parse("2026-06-22T08:30:00Z"))); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event.vin()).isEqualTo("LNVIN107800000001"); - assertThat(event.metadata()) - .containsEntry("vin", "LNVIN107800000001") - .containsEntry("sim", "013812345678") - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_PHONE"); - }); - } - - @Test - void identityResolverFailureStillProducesPassthroughWithRawPayload() { - Jt1078MalformedRtpHandler handler = new Jt1078MalformedRtpHandler(lookup -> { - throw new IllegalStateException("identity backend unavailable"); - }); - byte[] raw = new byte[]{0x30, 0x31, 0x63, 0x64}; - - List events = handler.onMalformedRtp(new Jt1078MalformedRtpPacket( - "udp", - "10.0.0.8:1078", - "013812345678", - raw, - raw.length, - "jt1078 udp rtp packet too short: 4", - Instant.parse("2026-06-22T08:30:00Z"))); - - assertThat(events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.vin()).isEqualTo("unknown"); - assertThat(passthrough.data()).containsExactly(raw); - assertThat(passthrough.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("sim", "013812345678") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN") - .containsEntry("identityError", "true") - .containsEntry("identityErrorMessage", "identity backend unavailable") - .containsEntry("parseError", "true"); - }); - } - - @Test - void dispatcherEnrichmentCanAttachRawArchiveReferenceToMalformedRtpPassthrough() { - byte[] raw = new byte[]{0x30, 0x31, 0x63, 0x64}; - Jt1078MalformedRtpHandler handler = new Jt1078MalformedRtpHandler(new InMemoryVehicleIdentityService()); - - VehicleEvent.Passthrough event = (VehicleEvent.Passthrough) handler.onMalformedRtp(new Jt1078MalformedRtpPacket( - "udp", "10.0.0.8:1078", raw, raw.length, "bad", Instant.parse("2026-06-22T08:30:00Z"))) - .getFirst(); - - assertThat(event.metadata()).doesNotContainKey(RawArchiveKeys.META_URI); - assertThat(event.data()).containsExactly(raw); - } -} diff --git a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078MediaArchiveServiceTest.java b/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078MediaArchiveServiceTest.java deleted file mode 100644 index f3b7916a..00000000 --- a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078MediaArchiveServiceTest.java +++ /dev/null @@ -1,198 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.identity.VehicleIdentityBinding; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MediaArchiveService; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078RtpPacket; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import com.lingniu.ingest.sink.archive.LocalArchiveStore; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.IOException; -import java.io.InputStream; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.ArrayList; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt1078MediaArchiveServiceTest { - - @TempDir - Path tempDir; - - @Test - void archivesPayloadBySegmentAndPublishesMediaMetaOncePerSegment() throws Exception { - LocalArchiveStore store = new LocalArchiveStore(tempDir.toString()); - List published = new ArrayList<>(); - Jt1078MediaArchiveService service = new Jt1078MediaArchiveService( - store, new InMemoryVehicleIdentityService(), published::add, 300); - - Jt1078RtpPacket p1 = new Jt1078RtpPacket(1, "013800138000", 3, 0, 0, 1_000L, new byte[]{1, 2}); - Jt1078RtpPacket p2 = new Jt1078RtpPacket(2, "013800138000", 3, 0, 1, 1_100L, new byte[]{3, 4, 5}); - service.archive(p1, "127.0.0.1:12345"); - service.archive(p2, "127.0.0.1:12345"); - - assertThat(published).hasSize(1); - VehicleEvent.MediaMeta meta = (VehicleEvent.MediaMeta) published.getFirst(); - assertThat(meta.source()).isEqualTo(ProtocolId.JT1078); - assertThat(meta.vin()).isEqualTo("013800138000"); - assertThat(meta.sizeBytes()).isEqualTo(2); - assertThat(meta.archiveRef()).startsWith("file://"); - assertThat(meta.metadata()) - .containsEntry("vin", "013800138000") - .containsEntry("channelId", "3") - .containsEntry("segment", "900") - .containsEntry("segmentSizeBytes", "2") - .containsEntry("archiveKey", "jt1078/1970/01/01/013800138000/ch-3/dt-0/900.media"); - - Path archived = Path.of(meta.archiveRef().substring("file://".length())); - assertThat(Files.readAllBytes(archived)).containsExactly(1, 2, 3, 4, 5); - } - - @Test - void archiveMediaMetaUsesResolvedInternalVinInMetadataAndArchiveKey() throws Exception { - LocalArchiveStore store = new LocalArchiveStore(tempDir.toString()); - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - identity.bind(new VehicleIdentityBinding( - ProtocolId.JT1078, "LNVIN107800000009", "013800138000", "", "")); - List published = new ArrayList<>(); - Jt1078MediaArchiveService service = new Jt1078MediaArchiveService( - store, identity, published::add, 300); - - service.archive(new Jt1078RtpPacket( - 1, "013800138000", 3, 0, 0, 1_000L, new byte[]{1, 2}), "127.0.0.1:12345"); - - assertThat(published).singleElement() - .isInstanceOf(VehicleEvent.MediaMeta.class) - .satisfies(event -> { - assertThat(event.vin()).isEqualTo("LNVIN107800000009"); - assertThat(event.metadata()) - .containsEntry("vin", "LNVIN107800000009") - .containsEntry("sim", "013800138000") - .containsEntry("identityResolved", "true") - .containsEntry("archiveKey", "jt1078/1970/01/01/LNVIN107800000009/ch-3/dt-0/900.media"); - }); - } - - @Test - void identityChangeWithinSameSegmentPublishesMediaMetaForNewArchiveKey() throws Exception { - LocalArchiveStore store = new LocalArchiveStore(tempDir.toString()); - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - List published = new ArrayList<>(); - Jt1078MediaArchiveService service = new Jt1078MediaArchiveService( - store, identity, published::add, 300); - - service.archive(new Jt1078RtpPacket( - 1, "013800138000", 3, 0, 0, 1_000L, new byte[]{1, 2}), "127.0.0.1:12345"); - identity.bind(new VehicleIdentityBinding( - ProtocolId.JT1078, "LNVIN107800000009", "013800138000", "", "")); - service.archive(new Jt1078RtpPacket( - 2, "013800138000", 3, 0, 1, 1_100L, new byte[]{3, 4}), "127.0.0.1:12345"); - - assertThat(published) - .extracting(event -> event.metadata().get("archiveKey")) - .containsExactly( - "jt1078/1970/01/01/013800138000/ch-3/dt-0/900.media", - "jt1078/1970/01/01/LNVIN107800000009/ch-3/dt-0/900.media"); - assertThat(published) - .extracting(VehicleEvent::vin) - .containsExactly("013800138000", "LNVIN107800000009"); - } - - @Test - void identityResolverFailureStillArchivesPayloadAndPublishesMediaMeta() throws Exception { - LocalArchiveStore store = new LocalArchiveStore(tempDir.toString()); - List published = new ArrayList<>(); - Jt1078MediaArchiveService service = new Jt1078MediaArchiveService( - store, - lookup -> { - throw new IllegalStateException("identity backend unavailable"); - }, - published::add, - 300); - - service.archive(new Jt1078RtpPacket( - 1, "013800138000", 3, 0, 0, 1_000L, new byte[]{1, 2}), "127.0.0.1:12345"); - - assertThat(published).singleElement() - .isInstanceOf(VehicleEvent.MediaMeta.class) - .satisfies(event -> { - VehicleEvent.MediaMeta media = (VehicleEvent.MediaMeta) event; - assertThat(media.vin()).isEqualTo("unknown"); - assertThat(media.archiveRef()).startsWith("file://"); - assertThat(media.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("sim", "013800138000") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN") - .containsEntry("identityError", "true") - .containsEntry("identityErrorMessage", "identity backend unavailable") - .containsEntry("archiveKey", "jt1078/1970/01/01/unknown/ch-3/dt-0/900.media"); - Path archived = Path.of(media.archiveRef().substring("file://".length())); - assertThat(Files.readAllBytes(archived)).containsExactly(1, 2); - }); - } - - @Test - void archiveFailurePublishesPassthroughErrorEventWithOriginalPayload() { - List published = new ArrayList<>(); - Jt1078MediaArchiveService service = new Jt1078MediaArchiveService( - new FailingArchiveStore(), new InMemoryVehicleIdentityService(), published::add, 300); - - Jt1078RtpPacket packet = new Jt1078RtpPacket( - 9, "013800138000", 4, 2, 1, 1_000L, new byte[]{9, 8, 7}); - - service.archive(packet, "127.0.0.1:9000"); - - assertThat(published).singleElement() - .satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(ProtocolId.JT1078); - assertThat(passthrough.vin()).isEqualTo("013800138000"); - assertThat(passthrough.passthroughType()).isEqualTo(2); - assertThat(passthrough.data()).containsExactly(9, 8, 7); - assertThat(passthrough.metadata()) - .containsEntry("archiveError", "true") - .containsEntry("vin", "013800138000") - .containsEntry("sim", "013800138000") - .containsEntry("channelId", "4") - .containsEntry("sequence", "9") - .containsEntry("peer", "127.0.0.1:9000") - .containsEntry("segment", "900") - .containsEntry("archiveKey", "jt1078/1970/01/01/013800138000/ch-4/dt-2/900.media"); - }); - } - - private static final class FailingArchiveStore implements ArchiveStore { - @Override - public String put(String key, InputStream data, long length) throws IOException { - throw new IOException("archive backend unavailable"); - } - - @Override - public String append(String key, byte[] chunk) throws IOException { - throw new IOException("archive backend unavailable"); - } - - @Override - public InputStream get(String key) throws IOException { - throw new IOException("archive backend unavailable"); - } - - @Override - public boolean exists(String key) { - return false; - } - - @Override - public long size(String key) throws IOException { - throw new IOException("archive backend unavailable"); - } - } -} diff --git a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078ParserTest.java b/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078ParserTest.java deleted file mode 100644 index 5770b162..00000000 --- a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078ParserTest.java +++ /dev/null @@ -1,87 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078; - -import com.lingniu.ingest.protocol.jt1078.codec.parser.FileUploadCompleteBodyParser; -import com.lingniu.ingest.protocol.jt1078.codec.parser.FileListBodyParser; -import com.lingniu.ingest.protocol.jt1078.codec.parser.MediaAttributesBodyParser; -import com.lingniu.ingest.protocol.jt1078.codec.parser.PassengerVolumeBodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import org.junit.jupiter.api.Test; - -import java.nio.ByteBuffer; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt1078ParserTest { - - private final Jt808Header header = new Jt808Header( - 0, 0, 0, false, Jt808Header.ProtocolVersion.V2013, "13800138000", 1, 0, 0); - - @Test - void parsesMediaAttributesAsSemanticBody() { - byte[] raw = new byte[]{1, 2, 3, 4, 0x01, 0x40, 1, 98, 4, 8}; - - Jt808Body body = new MediaAttributesBodyParser().parse(header, ByteBuffer.wrap(raw)); - - assertThat(body).isInstanceOf(Jt808Body.MediaAttributes.class); - Jt808Body.MediaAttributes attrs = (Jt808Body.MediaAttributes) body; - assertThat(attrs.inputAudioEncoding()).isEqualTo(1); - assertThat(attrs.inputAudioChannels()).isEqualTo(2); - assertThat(attrs.inputAudioSampleRate()).isEqualTo(3); - assertThat(attrs.inputAudioBitDepth()).isEqualTo(4); - assertThat(attrs.audioFrameLength()).isEqualTo(320); - assertThat(attrs.audioOutputSupported()).isTrue(); - assertThat(attrs.videoEncoding()).isEqualTo(98); - assertThat(attrs.maxAudioChannels()).isEqualTo(4); - assertThat(attrs.maxVideoChannels()).isEqualTo(8); - assertThat(attrs.raw()).containsExactly(raw); - } - - @Test - void parsesFileUploadCompleteAsSemanticBody() { - byte[] raw = new byte[]{0x12, 0x34, 1, 2, 3, 4}; - - Jt808Body body = new FileUploadCompleteBodyParser().parse(header, ByteBuffer.wrap(raw)); - - assertThat(body).isInstanceOf(Jt808Body.FileUploadComplete.class); - Jt808Body.FileUploadComplete done = (Jt808Body.FileUploadComplete) body; - assertThat(done.responseSerialNo()).isEqualTo(0x1234); - assertThat(done.result()).isEqualTo(1); - assertThat(done.raw()).containsExactly(raw); - } - - @Test - void parsesFileListSummaryAsSemanticBody() { - byte[] raw = new byte[]{0x12, 0x34, 0x00, 0x00, 0x00, 0x02, 9, 8, 7}; - - Jt808Body body = new FileListBodyParser().parse(header, ByteBuffer.wrap(raw)); - - assertThat(body).isInstanceOf(Jt808Body.FileList.class); - Jt808Body.FileList list = (Jt808Body.FileList) body; - assertThat(list.responseSerialNo()).isEqualTo(0x1234); - assertThat(list.fileCount()).isEqualTo(2); - assertThat(list.raw()).containsExactly(raw); - } - - @Test - void parsesPassengerVolumeAsSemanticBody() { - byte[] raw = new byte[]{ - 0x02, - 0x26, 0x06, 0x22, 0x15, 0x30, 0x00, - 0x26, 0x06, 0x22, 0x16, 0x00, 0x00, - 0x00, 0x15, - 0x00, 0x03 - }; - - Jt808Body body = new PassengerVolumeBodyParser().parse(header, ByteBuffer.wrap(raw)); - - assertThat(body).isInstanceOf(Jt808Body.PassengerVolume.class); - Jt808Body.PassengerVolume passengerVolume = (Jt808Body.PassengerVolume) body; - assertThat(passengerVolume.channelId()).isEqualTo(2); - assertThat(passengerVolume.startTime()).isEqualTo("2026-06-22T15:30:00"); - assertThat(passengerVolume.endTime()).isEqualTo("2026-06-22T16:00:00"); - assertThat(passengerVolume.passengerGetOn()).isEqualTo(21); - assertThat(passengerVolume.passengerGetOff()).isEqualTo(3); - assertThat(passengerVolume.raw()).containsExactly(raw); - } -} diff --git a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078RtpPacketDecoderTest.java b/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078RtpPacketDecoderTest.java deleted file mode 100644 index 6743e8cd..00000000 --- a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078RtpPacketDecoderTest.java +++ /dev/null @@ -1,91 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078; - -import com.lingniu.ingest.protocol.jt1078.media.Jt1078RtpPacket; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078RtpPacketDecoder; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MalformedRtpPacket; -import io.netty.buffer.Unpooled; -import io.netty.channel.embedded.EmbeddedChannel; -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt1078RtpPacketDecoderTest { - - @Test - void decodesCompleteTcpRtpPacketAndKeepsPayloadOnly() { - EmbeddedChannel channel = new EmbeddedChannel(new Jt1078RtpPacketDecoder(1024)); - - channel.writeInbound(Unpooled.wrappedBuffer(packet(7, "013800138000", 2, 3, 1, 123456789L, - new byte[]{1, 2, 3, 4}))); - - Jt1078RtpPacket packet = channel.readInbound(); - assertThat(packet.sequence()).isEqualTo(7); - assertThat(packet.sim()).isEqualTo("013800138000"); - assertThat(packet.channelId()).isEqualTo(2); - assertThat(packet.dataType()).isEqualTo(3); - assertThat(packet.packetType()).isEqualTo(1); - assertThat(packet.timestamp()).isEqualTo(123456789L); - assertThat(packet.payload()).containsExactly(1, 2, 3, 4); - } - - @Test - void waitsForFragmentedPayloadBeforeEmittingPacket() { - byte[] packet = packet(8, "013800138000", 1, 0, 0, 1L, new byte[]{9, 8, 7}); - EmbeddedChannel channel = new EmbeddedChannel(new Jt1078RtpPacketDecoder(1024)); - - channel.writeInbound(Unpooled.wrappedBuffer(packet, 0, packet.length - 1)); - assertThat((Object) channel.readInbound()).isNull(); - channel.writeInbound(Unpooled.wrappedBuffer(packet, packet.length - 1, 1)); - - Jt1078RtpPacket decoded = channel.readInbound(); - assertThat(decoded.payload()).containsExactly(9, 8, 7); - } - - @Test - void oversizedTcpRtpPacketEmitsMalformedCandidateInsteadOfClosingPipeline() { - byte[] packet = packet(9, "013800138000", 1, 0, 0, 1L, new byte[]{1, 2, 3, 4}); - EmbeddedChannel channel = new EmbeddedChannel(new Jt1078RtpPacketDecoder(30)); - - channel.writeInbound(Unpooled.wrappedBuffer(packet)); - - Jt1078MalformedRtpPacket malformed = channel.readInbound(); - assertThat(malformed.transport()).isEqualTo("tcp"); - assertThat(malformed.sim()).isEqualTo("013800138000"); - assertThat(malformed.reason()).contains("too large"); - assertThat(malformed.rawBytes()).containsExactly(packet); - assertThat(malformed.packetLength()).isEqualTo(packet.length); - assertThat((Object) channel.readInbound()).isNull(); - } - - static byte[] packet(int sequence, String sim, int channelId, int dataType, int packetType, - long timestamp, byte[] payload) { - byte[] out = new byte[28 + payload.length]; - out[0] = 0x30; - out[1] = 0x31; - out[2] = 0x63; - out[3] = 0x64; - out[4] = (byte) (sequence >>> 8); - out[5] = (byte) sequence; - byte[] bcd = bcd(sim); - System.arraycopy(bcd, 0, out, 6, bcd.length); - out[12] = (byte) channelId; - out[13] = (byte) ((dataType << 4) | (packetType & 0x0F)); - for (int i = 0; i < 8; i++) { - out[14 + i] = (byte) (timestamp >>> (56 - i * 8)); - } - out[26] = (byte) (payload.length >>> 8); - out[27] = (byte) payload.length; - System.arraycopy(payload, 0, out, 28, payload.length); - return out; - } - - private static byte[] bcd(String digits) { - byte[] out = new byte[6]; - for (int i = 0; i < out.length; i++) { - int hi = Character.digit(digits.charAt(i * 2), 10); - int lo = Character.digit(digits.charAt(i * 2 + 1), 10); - out[i] = (byte) ((hi << 4) | lo); - } - return out; - } -} diff --git a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078UdpRtpPacketDecoderTest.java b/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078UdpRtpPacketDecoderTest.java deleted file mode 100644 index 0e22d695..00000000 --- a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/Jt1078UdpRtpPacketDecoderTest.java +++ /dev/null @@ -1,89 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078; - -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MalformedRtpPacket; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078RtpPacket; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078ReceivedRtpPacket; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078UdpRtpPacketDecoder; -import io.netty.buffer.Unpooled; -import io.netty.channel.embedded.EmbeddedChannel; -import io.netty.channel.socket.DatagramPacket; -import org.junit.jupiter.api.Test; - -import java.net.InetSocketAddress; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt1078UdpRtpPacketDecoderTest { - - @Test - void decodesSingleUdpDatagramRtpPacket() { - EmbeddedChannel channel = new EmbeddedChannel(new Jt1078UdpRtpPacketDecoder(1024)); - InetSocketAddress sender = new InetSocketAddress("127.0.0.1", 19000); - InetSocketAddress recipient = new InetSocketAddress("127.0.0.1", 10780); - - channel.writeInbound(new DatagramPacket(Unpooled.wrappedBuffer(Jt1078RtpPacketDecoderTest.packet( - 9, "013800138000", 4, 0, 2, 123456789L, new byte[]{5, 6, 7})), recipient, sender)); - - Jt1078ReceivedRtpPacket received = channel.readInbound(); - Jt1078RtpPacket decoded = received.packet(); - assertThat(received.peer()).isEqualTo("127.0.0.1:19000"); - assertThat(decoded.sequence()).isEqualTo(9); - assertThat(decoded.sim()).isEqualTo("013800138000"); - assertThat(decoded.channelId()).isEqualTo(4); - assertThat(decoded.packetType()).isEqualTo(2); - assertThat(decoded.payload()).containsExactly(5, 6, 7); - } - - @Test - void shortUdpDatagramEmitsMalformedCandidateWithPeerAndRawBytes() { - EmbeddedChannel channel = new EmbeddedChannel(new Jt1078UdpRtpPacketDecoder(1024)); - InetSocketAddress sender = new InetSocketAddress("127.0.0.1", 19000); - InetSocketAddress recipient = new InetSocketAddress("127.0.0.1", 10780); - - channel.writeInbound(new DatagramPacket(Unpooled.wrappedBuffer(new byte[]{1, 2, 3}), recipient, sender)); - - Jt1078MalformedRtpPacket malformed = channel.readInbound(); - assertThat(malformed.transport()).isEqualTo("udp"); - assertThat(malformed.peer()).isEqualTo("127.0.0.1:19000"); - assertThat(malformed.reason()).contains("too short"); - assertThat(malformed.rawBytes()).containsExactly(1, 2, 3); - assertThat(malformed.packetLength()).isEqualTo(3); - } - - @Test - void invalidUdpDatagramEmitsMalformedCandidateInsteadOfThrowing() { - EmbeddedChannel channel = new EmbeddedChannel(new Jt1078UdpRtpPacketDecoder(1024)); - InetSocketAddress sender = new InetSocketAddress("127.0.0.1", 19000); - InetSocketAddress recipient = new InetSocketAddress("127.0.0.1", 10780); - byte[] invalid = Jt1078RtpPacketDecoderTest.packet( - 9, "013800138000", 4, 0, 2, 123456789L, new byte[]{5, 6, 7}); - invalid[0] = 0x12; - - channel.writeInbound(new DatagramPacket(Unpooled.wrappedBuffer(invalid), recipient, sender)); - - Jt1078MalformedRtpPacket malformed = channel.readInbound(); - assertThat(malformed.transport()).isEqualTo("udp"); - assertThat(malformed.peer()).isEqualTo("127.0.0.1:19000"); - assertThat(malformed.reason()).contains("invalid magic"); - assertThat(malformed.rawBytes()).containsExactly(invalid); - assertThat(malformed.packetLength()).isEqualTo(invalid.length); - } - - @Test - void oversizedUdpDatagramKeepsSimForIdentityResolution() { - EmbeddedChannel channel = new EmbeddedChannel(new Jt1078UdpRtpPacketDecoder(30)); - InetSocketAddress sender = new InetSocketAddress("127.0.0.1", 19000); - InetSocketAddress recipient = new InetSocketAddress("127.0.0.1", 10780); - byte[] oversized = Jt1078RtpPacketDecoderTest.packet( - 10, "013800138000", 4, 0, 2, 123456789L, new byte[]{5, 6, 7, 8}); - - channel.writeInbound(new DatagramPacket(Unpooled.wrappedBuffer(oversized), recipient, sender)); - - Jt1078MalformedRtpPacket malformed = channel.readInbound(); - assertThat(malformed.transport()).isEqualTo("udp"); - assertThat(malformed.sim()).isEqualTo("013800138000"); - assertThat(malformed.reason()).contains("too large"); - assertThat(malformed.rawBytes()).containsExactly(oversized); - assertThat(malformed.packetLength()).isEqualTo(oversized.length); - } -} diff --git a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078AutoConfigurationTest.java b/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078AutoConfigurationTest.java deleted file mode 100644 index d93a981f..00000000 --- a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/config/Jt1078AutoConfigurationTest.java +++ /dev/null @@ -1,270 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.config; - -import com.lingniu.ingest.core.concurrency.DisruptorEventBus; -import com.lingniu.ingest.core.config.IngestCoreAutoConfiguration; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.sink.EventSink; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentityRegistry; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.identity.config.VehicleIdentityAutoConfiguration; -import com.lingniu.ingest.protocol.jt1078.handler.Jt1078MediaSignalHandler; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MalformedRtpEventPublisher; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MalformedRtpPacket; -import com.lingniu.ingest.protocol.jt1078.media.Jt1078MediaArchiveService; -import com.lingniu.ingest.protocol.jt808.config.Jt808AutoConfiguration; -import com.lingniu.ingest.protocol.jt808.mapper.Jt808EventMapper; -import com.lingniu.ingest.session.DeviceSession; -import com.lingniu.ingest.session.SessionStore; -import com.lingniu.ingest.session.config.SessionCoreAutoConfiguration; -import com.lingniu.ingest.sink.archive.ArchiveStore; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.context.annotation.Bean; -import org.springframework.context.annotation.Configuration; - -import java.io.ByteArrayInputStream; -import java.io.IOException; -import java.io.InputStream; -import java.io.OutputStream; -import java.util.List; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CopyOnWriteArrayList; -import java.util.function.UnaryOperator; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt1078AutoConfigurationTest { - - private final ApplicationContextRunner runner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of( - IngestCoreAutoConfiguration.class, - Jt1078AutoConfiguration.class, - Jt1078SignalAutoConfiguration.class)) - .withUserConfiguration(SupportBeans.class) - .withPropertyValues( - "lingniu.ingest.jt1078.enabled=true", - "lingniu.ingest.jt1078.media-stream.enabled=false"); - - @Test - void mediaStreamDefaultsToLegacyProductionPort() { - Jt1078Properties props = new Jt1078Properties(); - - assertThat(props.getMediaStream().getPort()).isEqualTo(11078); - assertThat(props.getMediaStream().getUdpPort()).isEqualTo(11078); - } - - @Test - void mediaArchiveStartsWithoutJt808Mapper() { - runner.run(context -> { - assertThat(context).hasSingleBean(Jt1078MediaArchiveService.class); - assertThat(context).doesNotHaveBean(Jt808EventMapper.class); - assertThat(context).doesNotHaveBean(Jt1078MediaSignalHandler.class); - }); - } - - @Test - void mediaSignalHandlerStartsWhenJt808IsEnabled() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of( - Jt1078AutoConfiguration.class, - Jt1078SignalAutoConfiguration.class, - Jt808AutoConfiguration.class, - IngestCoreAutoConfiguration.class, - SessionCoreAutoConfiguration.class, - VehicleIdentityAutoConfiguration.class)) - .withUserConfiguration(SupportBeans.class) - .withPropertyValues( - "lingniu.ingest.jt1078.enabled=true", - "lingniu.ingest.jt1078.media-stream.enabled=false", - "lingniu.ingest.jt808.enabled=true", - "lingniu.ingest.jt808.port=0") - .run(context -> { - assertThat(context).hasSingleBean(Jt808EventMapper.class); - assertThat(context).hasSingleBean(Jt1078MediaSignalHandler.class); - }); - } - - @Test - void malformedRtpPublisherUsesConfiguredIdentityResolver() { - runner.run(context -> { - Jt1078MalformedRtpEventPublisher publisher = context.getBean(Jt1078MalformedRtpEventPublisher.class); - CapturingSink sink = context.getBean(CapturingSink.class); - - publisher.publish(new Jt1078MalformedRtpPacket( - "tcp", - "10.0.0.8:1078", - "013812345678", - new byte[]{0x30, 0x31, 0x63, 0x64}, - 128, - "jt1078 rtp packet too large: 128", - java.time.Instant.parse("2026-06-22T08:30:00Z"))); - - List events = sink.awaitCount(2); - assertThat(events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - assertThat(event.vin()).isEqualTo("LNVIN107800000001"); - assertThat(event.metadata()) - .containsEntry("sim", "013812345678") - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_PHONE") - .containsEntry("parseError", "true"); - }); - assertThat(events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - assertThat(event.vin()).isEqualTo("LNVIN107800000001"); - assertThat(event.metadata()) - .containsEntry("sim", "013812345678") - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_PHONE") - .containsEntry("parseError", "true") - .containsKey("rawArchiveUri"); - }); - }); - } - - @Configuration(proxyBeanMethods = false) - static class SupportBeans { - - @Bean - ArchiveStore archiveStore() { - return new InMemoryArchiveStore(); - } - - @Bean - VehicleIdentityResolver vehicleIdentityResolver() { - return lookup -> { - if (!lookup.vin().isBlank()) { - return new VehicleIdentity(lookup.vin(), true, VehicleIdentitySource.EXPLICIT_VIN); - } - if ("013812345678".equals(lookup.phone())) { - return new VehicleIdentity("LNVIN107800000001", true, VehicleIdentitySource.BOUND_PHONE); - } - return new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN); - }; - } - - @Bean - VehicleIdentityRegistry vehicleIdentityRegistry() { - return binding -> { - }; - } - - @Bean - SessionStore sessionStore() { - return new EmptySessionStore(); - } - - @Bean(destroyMethod = "close") - DisruptorEventBus disruptorEventBus(CapturingSink sink) { - return new DisruptorEventBus(1024, "blocking", List.of(sink)); - } - - @Bean - CapturingSink capturingSink() { - return new CapturingSink(); - } - } - - private static final class EmptySessionStore implements SessionStore { - @Override - public void put(DeviceSession session) { - } - - @Override - public Optional findBySessionId(String sessionId) { - return Optional.empty(); - } - - @Override - public Optional findByVin(String vin) { - return Optional.empty(); - } - - @Override - public Optional findByPhone(String phone) { - return Optional.empty(); - } - - @Override - public Optional update(String sessionId, UnaryOperator updater) { - return Optional.empty(); - } - - @Override - public void remove(String sessionId) { - } - - @Override - public int size() { - return 0; - } - } - - private static final class CapturingSink implements EventSink { - private final CopyOnWriteArrayList events = new CopyOnWriteArrayList<>(); - - @Override - public String name() { - return "capturing"; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - events.add(event); - return CompletableFuture.completedFuture(null); - } - - VehicleEvent awaitOne() throws InterruptedException { - for (int i = 0; i < 50; i++) { - if (!events.isEmpty()) { - return events.getFirst(); - } - Thread.sleep(20); - } - throw new AssertionError("expected one event"); - } - - List awaitCount(int count) throws InterruptedException { - for (int i = 0; i < 50; i++) { - if (events.size() >= count) { - return List.copyOf(events); - } - Thread.sleep(20); - } - throw new AssertionError("expected " + count + " events, got " + events.size()); - } - } - - private static final class InMemoryArchiveStore implements ArchiveStore { - - @Override - public String put(String key, InputStream data, long length) throws IOException { - data.transferTo(OutputStream.nullOutputStream()); - return "memory://" + key; - } - - @Override - public String append(String key, byte[] chunk) { - return "memory://" + key; - } - - @Override - public InputStream get(String key) { - return new ByteArrayInputStream(new byte[0]); - } - - @Override - public boolean exists(String key) { - return true; - } - - @Override - public long size(String key) { - return 0; - } - } -} diff --git a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/downlink/Jt1078CommandsTest.java b/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/downlink/Jt1078CommandsTest.java deleted file mode 100644 index 98289f0e..00000000 --- a/modules/protocols/protocol-jt1078/src/test/java/com/lingniu/ingest/protocol/jt1078/downlink/Jt1078CommandsTest.java +++ /dev/null @@ -1,127 +0,0 @@ -package com.lingniu.ingest.protocol.jt1078.downlink; - -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt1078CommandsTest { - - @Test - void realtimePlayEncodesServerPortsChannelAndStreamOptions() { - var command = Jt1078Commands.realtimePlay("10.1.2.3", 11078, 11079, 2, 1, 0); - - assertThat(command.messageId()).isEqualTo(0x9101); - assertThat(command.body()).containsExactly( - 0x08, '1', '0', '.', '1', '.', '2', '.', '3', - 0x2B, 0x46, - 0x2B, 0x47, - 0x02, - 0x01, - 0x00); - } - - @Test - void resourceSearchEncodesBcdTimeAndQueryOptions() { - var command = Jt1078Commands.resourceSearch( - 3, - "2026-06-22 08:09:10", - "260622180000", - 0x01020304L, - 0x05060708L, - 3, - 2, - 1); - - assertThat(command.messageId()).isEqualTo(0x9205); - assertThat(command.body()).containsExactly( - 0x03, - 0x26, 0x06, 0x22, 0x08, 0x09, 0x10, - 0x26, 0x06, 0x22, 0x18, 0x00, 0x00, - 0x01, 0x02, 0x03, 0x04, - 0x05, 0x06, 0x07, 0x08, - 0x03, - 0x02, - 0x01); - } - - @Test - void queryMediaAttributesUses1003RequestMessageIdWithEmptyBody() { - var command = Jt1078Commands.queryMediaAttributes(); - - assertThat(command.messageId()).isEqualTo(0x1003); - assertThat(command.body()).isEmpty(); - } - - @Test - void realtimeControlEncodesChannelCommandCloseTypeAndStreamType() { - var command = Jt1078Commands.realtimeControl(2, 1, 0, 1); - - assertThat(command.messageId()).isEqualTo(0x9102); - assertThat(command.body()).containsExactly(0x02, 0x01, 0x00, 0x01); - } - - @Test - void historyPlayEncodesPlaybackOptionsAndBcdTimes() { - var command = Jt1078Commands.historyPlay( - "media.example.cn", 11078, 0, 4, - 3, 2, 1, 0, 0, - "260622080910", "260622180000"); - - assertThat(command.messageId()).isEqualTo(0x9201); - assertThat(command.body()).containsExactly( - 0x10, 'm', 'e', 'd', 'i', 'a', '.', 'e', 'x', 'a', 'm', 'p', 'l', 'e', '.', 'c', 'n', - 0x2B, 0x46, - 0x00, 0x00, - 0x04, - 0x03, - 0x02, - 0x01, - 0x00, - 0x00, - 0x26, 0x06, 0x22, 0x08, 0x09, 0x10, - 0x26, 0x06, 0x22, 0x18, 0x00, 0x00); - } - - @Test - void historyControlEncodesPlaybackTime() { - var command = Jt1078Commands.historyControl(4, 5, 0, "260622090000"); - - assertThat(command.messageId()).isEqualTo(0x9202); - assertThat(command.body()).containsExactly( - 0x04, 0x05, 0x00, - 0x26, 0x06, 0x22, 0x09, 0x00, 0x00); - } - - @Test - void fileUploadEncodesServerAccountPathAndQueryOptions() { - var command = Jt1078Commands.fileUpload( - "10.1.2.3", 11078, "user", "pass", "/data/upload", - 2, "260622080910", "260622180000", - 1, 2, 3, 2, 1, 0x07); - - assertThat(command.messageId()).isEqualTo(0x9206); - assertThat(command.body()).containsExactly( - 0x08, '1', '0', '.', '1', '.', '2', '.', '3', - 0x2B, 0x46, - 0x04, 'u', 's', 'e', 'r', - 0x04, 'p', 'a', 's', 's', - 0x0C, '/', 'd', 'a', 't', 'a', '/', 'u', 'p', 'l', 'o', 'a', 'd', - 0x02, - 0x26, 0x06, 0x22, 0x08, 0x09, 0x10, - 0x26, 0x06, 0x22, 0x18, 0x00, 0x00, - 0x00, 0x00, 0x00, 0x01, - 0x00, 0x00, 0x00, 0x02, - 0x03, - 0x02, - 0x01, - 0x07); - } - - @Test - void fileUploadControlEncodesResponseSerialAndCommand() { - var command = Jt1078Commands.fileUploadControl(0x1234, 2); - - assertThat(command.messageId()).isEqualTo(0x9207); - assertThat(command.body()).containsExactly(0x12, 0x34, 0x02); - } -} diff --git a/modules/protocols/protocol-jt808/pom.xml b/modules/protocols/protocol-jt808/pom.xml deleted file mode 100644 index efcaadfd..00000000 --- a/modules/protocols/protocol-jt808/pom.xml +++ /dev/null @@ -1,64 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - protocol-jt808 - protocol-jt808 - JT/T 808 协议接入（Netty Server，默认 TCP 808）。 - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - ingest-core - - - com.lingniu.ingest - ingest-codec-common - - - com.lingniu.ingest - session-core - - - com.lingniu.ingest - vehicle-identity - - - com.lingniu.ingest - vehicle-identity-test-support - test - - - org.springframework.boot - spring-boot-starter - - - org.springframework.boot - spring-boot-configuration-processor - true - - - io.netty - netty-all - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/BodyParser.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/BodyParser.java deleted file mode 100644 index c8ed4678..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/BodyParser.java +++ /dev/null @@ -1,19 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; - -import java.nio.ByteBuffer; - -/** - * 单一消息体解析器 SPI。每个消息 ID 一个实现，替代旧代码里的巨型 switch。 - * - *

    约定：{@code parse} 调用时 buffer.position 指向 body 第 0 字节， - * 解析完成后 position 停在 body 末尾（不消费校验码）。 - */ -public interface BodyParser { - - int messageId(); - - Jt808Body parse(Jt808Header header, ByteBuffer body); -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/BodyParserRegistry.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/BodyParserRegistry.java deleted file mode 100644 index ea4751c2..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/BodyParserRegistry.java +++ /dev/null @@ -1,25 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -import java.util.HashMap; -import java.util.List; -import java.util.Map; - -/** - * JT808 消息体解析器注册中心。 - */ -public final class BodyParserRegistry { - - private final Map parsers = new HashMap<>(); - - public BodyParserRegistry(List list) { - for (BodyParser p : list) parsers.put(p.messageId(), p); - } - - public BodyParser find(int messageId) { - return parsers.get(messageId); - } - - public int size() { - return parsers.size(); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808Escape.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808Escape.java deleted file mode 100644 index 75f00215..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808Escape.java +++ /dev/null @@ -1,59 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -/** - * JT/T 808 转义规则： - * 

    - *   0x7e → 0x7d 0x02
- *   0x7d → 0x7d 0x01
- *
    - * 转义仅作用于 消息头 + 消息体 + 校验码，不包含首尾的 0x7e 标识位。 - */ -public final class Jt808Escape { - - private Jt808Escape() {} - - /** 反转义：从 framed 字节数组中剥除 0x7e 首尾后，恢复原始字节。 */ - public static byte[] unescape(byte[] framed, int offset, int length) { - byte[] out = new byte[length]; - int outLen = 0; - for (int i = offset; i < offset + length; i++) { - byte b = framed[i]; - if (b == 0x7d && i + 1 < offset + length) { - byte next = framed[i + 1]; - if (next == 0x02) { - out[outLen++] = 0x7e; - i++; - continue; - } else if (next == 0x01) { - out[outLen++] = 0x7d; - i++; - continue; - } - } - out[outLen++] = b; - } - byte[] trimmed = new byte[outLen]; - System.arraycopy(out, 0, trimmed, 0, outLen); - return trimmed; - } - - /** 转义：在原始字节上应用 7d/7e 替换，结果不含首尾 0x7e。 */ - public static byte[] escape(byte[] raw) { - byte[] out = new byte[raw.length * 2]; - int outLen = 0; - for (byte b : raw) { - if (b == 0x7e) { - out[outLen++] = 0x7d; - out[outLen++] = 0x02; - } else if (b == 0x7d) { - out[outLen++] = 0x7d; - out[outLen++] = 0x01; - } else { - out[outLen++] = b; - } - } - byte[] trimmed = new byte[outLen]; - System.arraycopy(out, 0, trimmed, 0, outLen); - return trimmed; - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameDecoder.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameDecoder.java deleted file mode 100644 index 6bcd5fc9..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameDecoder.java +++ /dev/null @@ -1,67 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -import io.netty.buffer.ByteBuf; -import io.netty.channel.ChannelHandlerContext; -import io.netty.handler.codec.ByteToMessageDecoder; - -import java.util.List; - -/** - * Netty 层 JT808 分帧 + 反转义。 - * - *

    寻找配对的 {@code 0x7e ... 0x7e}，提取中间字节并调用 {@link Jt808Escape#unescape} 还原， - * 输出到 pipeline 的是 {@code byte[]}（反转义后的原始数据，不含首尾 0x7e）。 - */ -public class Jt808FrameDecoder extends ByteToMessageDecoder { - - private static final int MAX_FRAME = 16 * 1024; - - @Override - protected void decode(ChannelHandlerContext ctx, ByteBuf in, List out) { - while (in.readableBytes() >= 2) { - int startIdx = in.indexOf(in.readerIndex(), in.writerIndex(), (byte) 0x7e); - if (startIdx < 0) { - int len = in.readableBytes(); - byte[] raw = new byte[len]; - in.readBytes(raw); - out.add(new Jt808MalformedFrame(raw, peer(ctx), "jt808 missing frame boundary")); - return; - } - if (startIdx != in.readerIndex()) { - int len = startIdx - in.readerIndex(); - byte[] raw = new byte[len]; - in.readBytes(raw); - out.add(new Jt808MalformedFrame(raw, peer(ctx), "jt808 leading bytes before frame boundary")); - } - int endIdx = in.indexOf(in.readerIndex() + 1, in.writerIndex(), (byte) 0x7e); - if (endIdx < 0) { - if (in.readableBytes() > MAX_FRAME) { - int len = in.readableBytes(); - byte[] raw = new byte[len]; - in.readBytes(raw); - out.add(new Jt808MalformedFrame(raw, peer(ctx), "jt808 unclosed frame too large: " + len)); - } - return; - } - int rawLen = endIdx - (in.readerIndex() + 1); - if (rawLen == 0) { - // 两个相邻 0x7e：视作空帧，跳过一个 - in.skipBytes(1); - continue; - } - byte[] raw = new byte[rawLen]; - in.skipBytes(1); - in.readBytes(raw); - in.skipBytes(1); // 结束 0x7e - byte[] unescaped = Jt808Escape.unescape(raw, 0, raw.length); - out.add(unescaped); - } - } - - private static String peer(ChannelHandlerContext ctx) { - if (ctx == null || ctx.channel() == null || ctx.channel().remoteAddress() == null) { - return ""; - } - return ctx.channel().remoteAddress().toString(); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameEncoder.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameEncoder.java deleted file mode 100644 index 3b052690..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameEncoder.java +++ /dev/null @@ -1,137 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.codec.BcdCodec; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; - -import java.io.ByteArrayOutputStream; -import java.util.ArrayList; -import java.util.List; - -/** - * JT/T 808 下行帧编码器：把 header + body 字节拼装 → 加 XOR 校验 → 转义 → 外包 0x7e。 - * - *

    支持 2013 与 2019 版基础头；短命令可直接编码，长命令通过 {@link #encodeSubpackages} - * 拆分为 JT/T 808 分包帧。 - */ -public final class Jt808FrameEncoder { - - private static final int MAX_BODY_LENGTH = 0x03FF; - - private Jt808FrameEncoder() {} - - public static byte[] encode(int messageId, String phone, int serialNo, byte[] body) { - return encode(messageId, phone, serialNo, body, Jt808Header.ProtocolVersion.V2013); - } - - public static byte[] encode(int messageId, String phone, int serialNo, byte[] body, - Jt808Header.ProtocolVersion version) { - byte[] raw = buildRaw(messageId, phone, serialNo, body, version, 0, 0); - return wrap(raw); - } - - public static List encodeSubpackages(int messageId, String phone, int serialNo, byte[] body) { - return encodeSubpackages(messageId, phone, serialNo, body, Jt808Header.ProtocolVersion.V2013); - } - - public static List encodeSubpackages(int messageId, - String phone, - int serialNo, - byte[] body, - Jt808Header.ProtocolVersion version) { - if (body == null) { - body = new byte[0]; - } - if (body.length <= MAX_BODY_LENGTH) { - return List.of(encode(messageId, phone, serialNo, body, version)); - } - int totalPackets = (body.length + MAX_BODY_LENGTH - 1) / MAX_BODY_LENGTH; - if (totalPackets > 0xFFFF) { - throw new IllegalArgumentException("jt808 subpackage count too large: " + totalPackets); - } - List frames = new ArrayList<>(totalPackets); - for (int i = 0; i < totalPackets; i++) { - int offset = i * MAX_BODY_LENGTH; - int len = Math.min(MAX_BODY_LENGTH, body.length - offset); - byte[] chunk = new byte[len]; - System.arraycopy(body, offset, chunk, 0, len); - frames.add(wrap(buildRaw(messageId, phone, serialNo, chunk, version, totalPackets, i + 1))); - } - return List.copyOf(frames); - } - - private static byte[] wrap(byte[] raw) { - byte bcc = BccChecksum.compute(raw, 0, raw.length); - - ByteArrayOutputStream almost = new ByteArrayOutputStream(raw.length + 1); - almost.write(raw, 0, raw.length); - almost.write(bcc & 0xFF); - - byte[] escaped = Jt808Escape.escape(almost.toByteArray()); - byte[] out = new byte[escaped.length + 2]; - out[0] = 0x7e; - System.arraycopy(escaped, 0, out, 1, escaped.length); - out[out.length - 1] = 0x7e; - return out; - } - - private static byte[] buildRaw(int messageId, String phone, int serialNo, byte[] body, - Jt808Header.ProtocolVersion version, int totalPackets, int packetSeq) { - if (body == null) { - body = new byte[0]; - } - if (body.length > MAX_BODY_LENGTH) { - throw new IllegalArgumentException("jt808 body too large without subpackage: " - + body.length + " > " + MAX_BODY_LENGTH); - } - boolean subpacket = totalPackets > 0; - boolean v2019 = version == Jt808Header.ProtocolVersion.V2019; - int phoneLen = v2019 ? 10 : 6; - int len = 2 /* msgId */ + 2 /* attrs */ + (v2019 ? 1 : 0) - + phoneLen + 2 /* serial */ + (subpacket ? 4 : 0) + body.length; - ByteArrayOutputStream os = new ByteArrayOutputStream(len); - - // msgId - os.write((messageId >> 8) & 0xFF); - os.write(messageId & 0xFF); - - // attrs: 2019 版 bit14 置 1 - int attrs = body.length & 0x03FF; - if (subpacket) attrs |= 0x2000; - if (v2019) attrs |= 0x4000; - os.write((attrs >> 8) & 0xFF); - os.write(attrs & 0xFF); - - // 2019 版多一个 protocolVersion 字节（值 1） - if (v2019) os.write(0x01); - - // phone BCD - String padded = padLeft(phone, phoneLen * 2); - byte[] bcd = BcdCodec.encode(padded); - os.write(bcd, 0, phoneLen); - - // serial - os.write((serialNo >> 8) & 0xFF); - os.write(serialNo & 0xFF); - - if (subpacket) { - os.write((totalPackets >> 8) & 0xFF); - os.write(totalPackets & 0xFF); - os.write((packetSeq >> 8) & 0xFF); - os.write(packetSeq & 0xFF); - } - - // body - os.write(body, 0, body.length); - return os.toByteArray(); - } - - private static String padLeft(String s, int len) { - if (s == null) s = ""; - if (s.length() >= len) return s.substring(s.length() - len); - StringBuilder sb = new StringBuilder(len); - for (int i = s.length(); i < len; i++) sb.append('0'); - sb.append(s); - return sb.toString(); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808MalformedFrame.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808MalformedFrame.java deleted file mode 100644 index dfe62dd2..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808MalformedFrame.java +++ /dev/null @@ -1,7 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -public record Jt808MalformedFrame( - byte[] rawBytes, - String peer, - String reason -) {} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808MessageDecoder.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808MessageDecoder.java deleted file mode 100644 index ee57bedd..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/Jt808MessageDecoder.java +++ /dev/null @@ -1,96 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -import com.lingniu.ingest.api.spi.DecodeException; -import com.lingniu.ingest.api.spi.FrameDecoder; -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.codec.BcdCodec; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; - -import java.nio.ByteBuffer; -import java.util.Arrays; - -/** - * JT808 完整帧（已反转义、含校验码，不含首尾 0x7e） → {@link Jt808Message}。 - * - *

    入参 {@link ByteBuffer} 的生命周期由调用方管理，本类只读取。 - */ -public final class Jt808MessageDecoder implements FrameDecoder { - - private static final int HEADER_LEN_2013 = 12; - private static final int HEADER_LEN_2019 = 16; // + phone 延长 4 字节 - - private final BodyParserRegistry registry; - - public Jt808MessageDecoder(BodyParserRegistry registry) { - this.registry = registry; - } - - @Override - public Jt808Message decode(ByteBuffer buffer) { - byte[] bytes = new byte[buffer.remaining()]; - buffer.get(bytes); - - if (bytes.length < HEADER_LEN_2013 + 1) { - throw new DecodeException("jt808 frame too short: " + bytes.length); - } - - // 校验码：从 msgId 开始到校验码前一字节 - byte expected = bytes[bytes.length - 1]; - if (!BccChecksum.verify(bytes, 0, bytes.length - 1, expected)) { - throw new DecodeException("jt808 XOR checksum mismatch"); - } - - int messageId = ((bytes[0] & 0xFF) << 8) | (bytes[1] & 0xFF); - int attrs = ((bytes[2] & 0xFF) << 8) | (bytes[3] & 0xFF); - int bodyLength = attrs & 0x03FF; - int encryptType = (attrs >> 10) & 0x07; - boolean subpacket = (attrs & 0x2000) != 0; - // JT/T 808-2019 通过属性 bit14 标识，终端手机号从 6B BCD 扩展到 10B BCD。 - boolean v2019 = (attrs & 0x4000) != 0; - - Jt808Header.ProtocolVersion version = v2019 - ? Jt808Header.ProtocolVersion.V2019 - : Jt808Header.ProtocolVersion.V2013; - int phoneLen = v2019 ? 10 : 6; - int headerLen = 4 /* msgId+attrs */ + (v2019 ? 1 : 0) /* protocol version */ + phoneLen + 2 /* serial */; - int phoneOffset = 4 + (v2019 ? 1 : 0); - - String phone = BcdCodec.decode(Arrays.copyOfRange(bytes, phoneOffset, phoneOffset + phoneLen)) - .replaceAll("^0+", ""); - int serial = ((bytes[phoneOffset + phoneLen] & 0xFF) << 8) - | (bytes[phoneOffset + phoneLen + 1] & 0xFF); - - int totalPkgs = 0, pkgSeq = 0; - int bodyStart = headerLen; - if (subpacket) { - // 分包头紧跟消息流水号之后；当前只暴露包总数/序号，重组由上游或后续模块处理。 - totalPkgs = ((bytes[bodyStart] & 0xFF) << 8) | (bytes[bodyStart + 1] & 0xFF); - pkgSeq = ((bytes[bodyStart + 2] & 0xFF) << 8) | (bytes[bodyStart + 3] & 0xFF); - bodyStart += 4; - } - - int bodyEnd = bodyStart + bodyLength; - if (bodyEnd > bytes.length - 1) { - throw new DecodeException("jt808 body length " + bodyLength + " exceeds frame"); - } - - Jt808Header header = new Jt808Header( - messageId, bodyLength, encryptType, subpacket, version, - phone, serial, totalPkgs, pkgSeq); - - ByteBuffer bodyBuf = ByteBuffer.wrap(bytes, bodyStart, bodyLength); - BodyParser parser = registry.find(messageId); - Jt808Body body; - if (parser == null) { - // 未实现的消息体保持 Raw，仍可进入 archive/event-history，避免协议扩展导致数据丢失。 - byte[] raw = new byte[bodyLength]; - bodyBuf.get(raw); - body = new Jt808Body.Raw(messageId, raw); - } else { - body = parser.parse(header, bodyBuf); - } - return new Jt808Message(header, body); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/AuthBodyParser.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/AuthBodyParser.java deleted file mode 100644 index 4ca31857..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/AuthBodyParser.java +++ /dev/null @@ -1,49 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec.parser; - -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; - -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.util.regex.Matcher; -import java.util.regex.Pattern; - -public final class AuthBodyParser implements BodyParser { - - private static final Pattern IMEI = Pattern.compile("(?= 2 ? body.getShort() & 0xFFFF : 0; - int batchType = body.remaining() >= 1 ? body.get() & 0xFF : 0; - List locations = new ArrayList<>(count); - for (int i = 0; i < count && body.remaining() >= 2; i++) { - int len = body.getShort() & 0xFFFF; - if (len > body.remaining()) { - len = body.remaining(); - } - ByteBuffer slice = body.slice(body.position(), len); - Jt808Body parsed = locationParser.parse(header, slice); - if (parsed instanceof Jt808Body.Location location) { - locations.add(location); - } - body.position(body.position() + len); - } - return new Jt808Body.LocationBatch(batchType, List.copyOf(locations)); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/LocationBodyParser.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/LocationBodyParser.java deleted file mode 100644 index de5da2b3..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/LocationBodyParser.java +++ /dev/null @@ -1,95 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec.parser; - -import com.lingniu.ingest.codec.BcdCodec; -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; - -import java.nio.ByteBuffer; -import java.time.LocalDateTime; -import java.time.ZoneId; -import java.util.LinkedHashMap; -import java.util.Map; - -/** - * 0x0200 位置信息汇报。 - * - *

    固定 28 字节 + 可选附加项。附加项按 JT/T 808 的 ID/长度/值 TLV 保留原始字节， - * 由上层统计或导出服务按配置解释。 - * 

    - *   alarmFlag(4)
- *   statusFlag(4)
- *   latitude(4)   // 1/1000000 度
- *   longitude(4)  // 1/1000000 度
- *   altitudeM(2)
- *   speed(2)      // 1/10 km/h
- *   direction(2)
- *   time(6 BCD, YYMMDDHHMMSS)
- *
    - */ -public final class LocationBodyParser implements BodyParser { - - @Override - public int messageId() { - return Jt808MessageId.TERMINAL_LOCATION; - } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer buf) { - long alarmFlag = buf.getInt() & 0xFFFFFFFFL; - long statusFlag = buf.getInt() & 0xFFFFFFFFL; - long latRaw = buf.getInt() & 0xFFFFFFFFL; - long lonRaw = buf.getInt() & 0xFFFFFFFFL; - int altitudeM = buf.getShort(); - double speed = (buf.getShort() & 0xFFFF) * 0.1; - int direction = buf.getShort() & 0xFFFF; - - byte[] timeBcd = new byte[6]; - buf.get(timeBcd); - String ts = BcdCodec.decode(timeBcd); - - LocalDateTime ldt = LocalDateTime.of( - 2000 + Integer.parseInt(ts.substring(0, 2)), - Integer.parseInt(ts.substring(2, 4)), - Integer.parseInt(ts.substring(4, 6)), - Integer.parseInt(ts.substring(6, 8)), - Integer.parseInt(ts.substring(8, 10)), - Integer.parseInt(ts.substring(10, 12))); - - Map extensions = parseExtensions(buf); - - return new Jt808Body.Location( - alarmFlag, statusFlag, - lonRaw / 1_000_000.0, latRaw / 1_000_000.0, - altitudeM, speed, direction, - ldt.atZone(ZoneId.of("Asia/Shanghai")).toInstant(), - extensions); - } - - private static Map parseExtensions(ByteBuffer buf) { - if (!buf.hasRemaining()) { - return Map.of(); - } - Map out = new LinkedHashMap<>(); - while (buf.remaining() >= 2) { - int id = buf.get() & 0xFF; - int len = buf.get() & 0xFF; - if (len > buf.remaining()) { - byte[] truncated = new byte[buf.remaining()]; - buf.get(truncated); - out.put(id, truncated); - break; - } - byte[] value = new byte[len]; - buf.get(value); - out.put(id, value); - } - if (buf.hasRemaining()) { - byte[] tail = new byte[buf.remaining()]; - buf.get(tail); - out.put(0xFFFF, tail); - } - return Map.copyOf(out); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/MediaEventBodyParser.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/MediaEventBodyParser.java deleted file mode 100644 index 182d2fe8..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/MediaEventBodyParser.java +++ /dev/null @@ -1,26 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec.parser; - -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; - -import java.nio.ByteBuffer; - -public final class MediaEventBodyParser implements BodyParser { - - @Override - public int messageId() { - return Jt808MessageId.TERMINAL_MEDIA_EVENT; - } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer body) { - long mediaId = body.remaining() >= 4 ? body.getInt() & 0xFFFFFFFFL : 0; - int mediaType = body.hasRemaining() ? body.get() & 0xFF : 0; - int mediaFormat = body.hasRemaining() ? body.get() & 0xFF : 0; - int eventCode = body.hasRemaining() ? body.get() & 0xFF : 0; - int channelId = body.hasRemaining() ? body.get() & 0xFF : 0; - return new Jt808Body.MediaEvent(mediaId, mediaType, mediaFormat, eventCode, channelId); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/MediaUploadBodyParser.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/MediaUploadBodyParser.java deleted file mode 100644 index 8a0c3869..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/MediaUploadBodyParser.java +++ /dev/null @@ -1,38 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec.parser; - -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; - -import java.nio.ByteBuffer; - -public final class MediaUploadBodyParser implements BodyParser { - - private final LocationBodyParser locationParser = new LocationBodyParser(); - - @Override - public int messageId() { - return Jt808MessageId.TERMINAL_MEDIA_UPLOAD; - } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer body) { - long mediaId = body.remaining() >= 4 ? body.getInt() & 0xFFFFFFFFL : 0; - int mediaType = body.hasRemaining() ? body.get() & 0xFF : 0; - int mediaFormat = body.hasRemaining() ? body.get() & 0xFF : 0; - int eventCode = body.hasRemaining() ? body.get() & 0xFF : 0; - int channelId = body.hasRemaining() ? body.get() & 0xFF : 0; - Jt808Body.Location location = null; - if (body.remaining() >= 28) { - ByteBuffer loc = body.slice(body.position(), 28); - if (locationParser.parse(header, loc) instanceof Jt808Body.Location parsed) { - location = parsed; - } - body.position(body.position() + 28); - } - byte[] data = new byte[body.remaining()]; - body.get(data); - return new Jt808Body.MediaUpload(mediaId, mediaType, mediaFormat, eventCode, channelId, location, data); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/PassthroughBodyParser.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/PassthroughBodyParser.java deleted file mode 100644 index 83c47848..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/PassthroughBodyParser.java +++ /dev/null @@ -1,24 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec.parser; - -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; - -import java.nio.ByteBuffer; - -public final class PassthroughBodyParser implements BodyParser { - - @Override - public int messageId() { - return Jt808MessageId.TERMINAL_PASSTHROUGH; - } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer body) { - int type = body.hasRemaining() ? body.get() & 0xFF : 0; - byte[] data = new byte[body.remaining()]; - body.get(data); - return new Jt808Body.Passthrough(type, data); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/RegisterBodyParser.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/RegisterBodyParser.java deleted file mode 100644 index bdd76ca4..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/RegisterBodyParser.java +++ /dev/null @@ -1,54 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec.parser; - -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; - -import java.nio.ByteBuffer; -import java.nio.charset.Charset; -import java.nio.charset.StandardCharsets; - -/** - * 0x0100 终端注册。 - * - *

    2013 body 格式： - * 

    - *   provinceId(2)
- *   cityId(2)
- *   makerId(5 ASCII)
- *   deviceType(20 ASCII) -- 2011 版为 8
- *   deviceId(7 ASCII)
- *   plateColor(1)
- *   plate(GBK, 剩余全部字节)
- *
    - */ -public final class RegisterBodyParser implements BodyParser { - - private static final Charset GBK = Charset.forName("GBK"); - - @Override - public int messageId() { - return Jt808MessageId.TERMINAL_REGISTER; - } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer buf) { - int province = buf.getShort() & 0xFFFF; - int city = buf.getShort() & 0xFFFF; - String maker = readAscii(buf, 5); - String deviceType = readAscii(buf, 20); - String deviceId = readAscii(buf, 7); - int plateColor = buf.get() & 0xFF; - byte[] plateBytes = new byte[buf.remaining()]; - buf.get(plateBytes); - String plate = new String(plateBytes, GBK).trim(); - return new Jt808Body.Register(province, city, maker, deviceType, deviceId, plateColor, plate); - } - - private static String readAscii(ByteBuffer buf, int len) { - byte[] b = new byte[len]; - buf.get(b); - return new String(b, StandardCharsets.US_ASCII).trim(); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/TerminalAttrsBodyParser.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/TerminalAttrsBodyParser.java deleted file mode 100644 index b30c24ff..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/TerminalAttrsBodyParser.java +++ /dev/null @@ -1,58 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec.parser; - -import com.lingniu.ingest.codec.BcdCodec; -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; - -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; - -public final class TerminalAttrsBodyParser implements BodyParser { - - @Override - public int messageId() { - return Jt808MessageId.TERMINAL_ATTRS_REPORT; - } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer body) { - int terminalType = readU16(body); - String maker = readAscii(body, 5); - String model = readAscii(body, 20); - String terminalId = readAscii(body, 7); - String iccid = readBcd(body, 10); - String hardwareVersion = readAscii(body, 10); - String firmwareVersion = readAscii(body, 10); - int gnssProperty = body.hasRemaining() ? body.get() & 0xFF : 0; - int commProperty = body.hasRemaining() ? body.get() & 0xFF : 0; - if (body.hasRemaining()) { - body.position(body.limit()); - } - return new Jt808Body.TerminalAttrs( - terminalType, maker, model, terminalId, iccid, hardwareVersion, firmwareVersion, - gnssProperty, commProperty); - } - - private static int readU16(ByteBuffer body) { - return body.remaining() >= 2 ? body.getShort() & 0xFFFF : 0; - } - - private static String readAscii(ByteBuffer body, int len) { - int n = Math.min(len, body.remaining()); - byte[] bytes = new byte[n]; - body.get(bytes); - if (n < len && body.hasRemaining()) { - body.position(Math.min(body.limit(), body.position() + (len - n))); - } - return new String(bytes, StandardCharsets.US_ASCII).trim(); - } - - private static String readBcd(ByteBuffer body, int len) { - int n = Math.min(len, body.remaining()); - byte[] bytes = new byte[n]; - body.get(bytes); - return BcdCodec.decode(bytes).trim(); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/TerminalParamsReportBodyParser.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/TerminalParamsReportBodyParser.java deleted file mode 100644 index a88e50e5..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/TerminalParamsReportBodyParser.java +++ /dev/null @@ -1,36 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec.parser; - -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; - -import java.nio.ByteBuffer; -import java.util.LinkedHashMap; -import java.util.Map; - -public final class TerminalParamsReportBodyParser implements BodyParser { - - @Override - public int messageId() { - return Jt808MessageId.TERMINAL_PARAMS_REPORT; - } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer body) { - int responseSerialNo = body.remaining() >= 2 ? body.getShort() & 0xFFFF : 0; - int count = body.remaining() >= 1 ? body.get() & 0xFF : 0; - Map parameters = new LinkedHashMap<>(); - for (int i = 0; i < count && body.remaining() >= 5; i++) { - long paramId = body.getInt() & 0xFFFFFFFFL; - int len = body.get() & 0xFF; - if (len > body.remaining()) { - len = body.remaining(); - } - byte[] value = new byte[len]; - body.get(value); - parameters.put(paramId, value); - } - return new Jt808Body.ParamsReport(responseSerialNo, Map.copyOf(parameters)); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/UnregisterBodyParser.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/UnregisterBodyParser.java deleted file mode 100644 index 7b2d3a5f..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/codec/parser/UnregisterBodyParser.java +++ /dev/null @@ -1,22 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec.parser; - -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; - -import java.nio.ByteBuffer; - -public final class UnregisterBodyParser implements BodyParser { - - @Override - public int messageId() { - return Jt808MessageId.TERMINAL_UNREGISTER; - } - - @Override - public Jt808Body parse(Jt808Header header, ByteBuffer body) { - body.position(body.limit()); - return new Jt808Body.Unregister(); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/config/Jt808AutoConfiguration.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/config/Jt808AutoConfiguration.java deleted file mode 100644 index d19fef39..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/config/Jt808AutoConfiguration.java +++ /dev/null @@ -1,117 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.config; - -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.identity.VehicleIdentityRegistry; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.protocol.jt808.codec.BodyParser; -import com.lingniu.ingest.protocol.jt808.codec.BodyParserRegistry; -import com.lingniu.ingest.protocol.jt808.codec.Jt808MessageDecoder; -import com.lingniu.ingest.protocol.jt808.codec.parser.AuthBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.HeartbeatBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.LocationBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.LocationBatchBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.MediaEventBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.MediaUploadBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.PassthroughBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.RegisterBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.TerminalAttrsBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.TerminalParamsReportBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.UnregisterBodyParser; -import com.lingniu.ingest.protocol.jt808.downlink.Jt808CommandDispatcher; -import com.lingniu.ingest.protocol.jt808.handler.Jt808LocationHandler; -import com.lingniu.ingest.protocol.jt808.inbound.Jt808NettyServer; -import com.lingniu.ingest.protocol.jt808.mapper.Jt808EventMapper; -import com.lingniu.ingest.protocol.jt808.session.Jt808ChannelRegistry; -import com.lingniu.ingest.protocol.jt808.session.Jt808PendingRequests; -import com.lingniu.ingest.session.CommandDispatcher; -import com.lingniu.ingest.session.SessionStore; -import com.lingniu.ingest.session.config.SessionCoreAutoConfiguration; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; - -import java.util.List; - -@AutoConfiguration(before = SessionCoreAutoConfiguration.class) -@EnableConfigurationProperties(Jt808Properties.class) -@ConditionalOnProperty(prefix = "lingniu.ingest.jt808", name = "enabled", havingValue = "true") -public class Jt808AutoConfiguration { - - @Bean public BodyParser jt808RegisterBodyParser() { return new RegisterBodyParser(); } - @Bean public BodyParser jt808AuthBodyParser() { return new AuthBodyParser(); } - @Bean public BodyParser jt808LocationBodyParser() { return new LocationBodyParser(); } - @Bean public BodyParser jt808HeartbeatBodyParser() { return new HeartbeatBodyParser(); } - @Bean public BodyParser jt808UnregisterBodyParser() { return new UnregisterBodyParser(); } - @Bean public BodyParser jt808TerminalParamsReportBodyParser() { return new TerminalParamsReportBodyParser(); } - @Bean public BodyParser jt808TerminalAttrsBodyParser() { return new TerminalAttrsBodyParser(); } - @Bean public BodyParser jt808LocationBatchBodyParser() { return new LocationBatchBodyParser(); } - @Bean public BodyParser jt808MediaEventBodyParser() { return new MediaEventBodyParser(); } - @Bean public BodyParser jt808MediaUploadBodyParser() { return new MediaUploadBodyParser(); } - @Bean public BodyParser jt808PassthroughBodyParser() { return new PassthroughBodyParser(); } - - @Bean - @ConditionalOnMissingBean - public BodyParserRegistry jt808BodyParserRegistry(List parsers) { - return new BodyParserRegistry(parsers); - } - - @Bean - @ConditionalOnMissingBean - public Jt808MessageDecoder jt808MessageDecoder(BodyParserRegistry registry) { - return new Jt808MessageDecoder(registry); - } - - @Bean - @ConditionalOnMissingBean - public Jt808EventMapper jt808EventMapper(VehicleIdentityResolver identityResolver) { - return new Jt808EventMapper(identityResolver); - } - - @Bean - @ConditionalOnMissingBean - public Jt808LocationHandler jt808LocationHandler(Jt808EventMapper mapper) { - return new Jt808LocationHandler(mapper); - } - - @Bean - @ConditionalOnMissingBean - public Jt808ChannelRegistry jt808ChannelRegistry() { - return new Jt808ChannelRegistry(); - } - - @Bean - @ConditionalOnMissingBean - public Jt808PendingRequests jt808PendingRequests() { - return new Jt808PendingRequests(); - } - - /** - * JT808 模块启用时抢占 {@link CommandDispatcher} Bean。 - * 通过 {@code @AutoConfiguration(before = SessionCoreAutoConfiguration.class)} 确保本 Bean - * 先注册，session-core 的 {@code @ConditionalOnMissingBean} 会因此跳过 Noop 默认实现。 - */ - @Bean - @ConditionalOnMissingBean - public CommandDispatcher jt808CommandDispatcher(SessionStore sessions, - Jt808ChannelRegistry channelRegistry, - Jt808PendingRequests pendingRequests) { - return new Jt808CommandDispatcher(sessions, channelRegistry, pendingRequests); - } - - @Bean - @ConditionalOnMissingBean - public Jt808NettyServer jt808NettyServer(Jt808Properties props, - Jt808MessageDecoder decoder, - Dispatcher dispatcher, - SessionStore sessions, - VehicleIdentityRegistry identityRegistry, - VehicleIdentityResolver identityResolver, - Jt808ChannelRegistry channelRegistry, - Jt808PendingRequests pendingRequests) { - return new Jt808NettyServer(props.getPort(), props.getWorkerThreads(), - decoder, dispatcher, sessions, identityRegistry, identityResolver, - channelRegistry, pendingRequests); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/config/Jt808Properties.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/config/Jt808Properties.java deleted file mode 100644 index da28853d..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/config/Jt808Properties.java +++ /dev/null @@ -1,20 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.config; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -@ConfigurationProperties(prefix = "lingniu.ingest.jt808") -public class Jt808Properties { - /** 是否启动 JT/T 808 TCP 监听。与 GB32960 独立，通常用于部标终端。 */ - private boolean enabled = false; - /** JT808 默认监听端口；不要与 32960 端口共用。 */ - private int port = 808; - /** Netty worker 线程数；0 表示使用框架默认值。 */ - private int workerThreads = 0; - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public int getPort() { return port; } - public void setPort(int port) { this.port = port; } - public int getWorkerThreads() { return workerThreads; } - public void setWorkerThreads(int workerThreads) { this.workerThreads = workerThreads; } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808CommandDispatcher.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808CommandDispatcher.java deleted file mode 100644 index 7ba34622..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808CommandDispatcher.java +++ /dev/null @@ -1,134 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.downlink; - -import com.lingniu.ingest.protocol.jt808.codec.Jt808FrameEncoder; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import com.lingniu.ingest.protocol.jt808.session.Jt808ChannelRegistry; -import com.lingniu.ingest.protocol.jt808.session.Jt808PendingRequests; -import com.lingniu.ingest.session.CommandDispatcher; -import com.lingniu.ingest.session.DeviceSession; -import com.lingniu.ingest.session.SessionStore; -import io.netty.buffer.Unpooled; -import io.netty.channel.Channel; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.time.Duration; -import java.util.Optional; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CompletionException; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicInteger; - -/** - * JT808 侧的 {@link CommandDispatcher} 实现：把 session-core 的调用桥接到具体 Netty Channel。 - * - *

    {@link #notify(String, Object)} 异步下发后立即返回； - * {@link #request(String, Object, Class, Duration)} 通过 {@link Jt808PendingRequests} 等待设备应答。 - * - *

    入参 {@code command} 必须是 {@link Jt808Commands.DownlinkCommand}，其他类型返回失败 future。 - */ -public final class Jt808CommandDispatcher implements CommandDispatcher { - - private static final Logger log = LoggerFactory.getLogger(Jt808CommandDispatcher.class); - - private final SessionStore sessions; - private final Jt808ChannelRegistry channelRegistry; - private final Jt808PendingRequests pendingRequests; - - public Jt808CommandDispatcher(SessionStore sessions, - Jt808ChannelRegistry channelRegistry, - Jt808PendingRequests pendingRequests) { - this.sessions = sessions; - this.channelRegistry = channelRegistry; - this.pendingRequests = pendingRequests; - } - - @Override - public CompletableFuture notify(String sessionId, Object command) { - CommandContext ctx; - try { - ctx = resolve(sessionId, command); - } catch (IllegalArgumentException e) { - return CompletableFuture.failedFuture(e); - } - return writeFrames(ctx, false); - } - - @Override - public CompletableFuture request(String sessionId, Object command, - Class responseType, Duration timeout) { - CommandContext ctx; - try { - ctx = resolve(sessionId, command); - } catch (IllegalArgumentException e) { - return CompletableFuture.failedFuture(e); - } - - CompletableFuture pending = pendingRequests.await(ctx.phone, ctx.serial); - writeFrames(ctx, true).whenComplete((unused, ex) -> { - if (ex != null) { - // 写 channel 失败时必须移除 pending，否则后续同 phone/serial 重用会匹配到旧 future。 - pendingRequests.cancel(ctx.phone, ctx.serial); - pending.completeExceptionally(ex); - } - }); - - return pending.orTimeout(timeout.toMillis(), TimeUnit.MILLISECONDS) - .whenComplete((r, ex) -> { - // 超时/取消同样清理 pending，避免长期运行后内存泄漏。 - if (ex != null) pendingRequests.cancel(ctx.phone, ctx.serial); - }) - .thenApply(msg -> { - if (responseType.isInstance(msg)) return responseType.cast(msg); - throw new CompletionException(new ClassCastException( - "jt808 response type mismatch, expect=" + responseType.getSimpleName() - + " actual=" + msg.getClass().getSimpleName())); - }); - } - - private CompletableFuture writeFrames(CommandContext ctx, boolean request) { - var frames = Jt808FrameEncoder.encodeSubpackages( - ctx.cmd.messageId(), ctx.phone, ctx.serial, ctx.cmd.body()); - CompletableFuture cf = new CompletableFuture<>(); - AtomicInteger remaining = new AtomicInteger(frames.size()); - for (byte[] frame : frames) { - ctx.channel.writeAndFlush(Unpooled.wrappedBuffer(frame)).addListener(f -> { - if (!f.isSuccess()) { - if (request) { - pendingRequests.cancel(ctx.phone, ctx.serial); - } - cf.completeExceptionally(f.cause()); - return; - } - if (remaining.decrementAndGet() == 0) { - cf.complete(null); - } - }); - } - return cf; - } - - private CommandContext resolve(String sessionId, Object command) { - if (!(command instanceof Jt808Commands.DownlinkCommand cmd)) { - throw new IllegalArgumentException( - "expected Jt808Commands.DownlinkCommand, got " + command.getClass().getName()); - } - String phone = resolvePhone(sessionId); - Channel channel = channelRegistry.find(phone) - .orElseThrow(() -> new IllegalStateException("no active channel for phone=" + phone)); - int serial = channelRegistry.nextSerial(phone); - return new CommandContext(phone, serial, channel, cmd); - } - - private String resolvePhone(String sessionId) { - Optional s = sessions.findBySessionId(sessionId) - .or(() -> sessions.findByPhone(sessionId)) - .or(() -> sessions.findByVin(sessionId)); - // HTTP 调用方可以传 sessionId/VIN/phone；最终写 Netty Channel 必须落到 JT808 phone。 - return s.map(DeviceSession::phone) - .filter(p -> p != null && !p.isBlank()) - .orElse(sessionId); // 回落：直接把 sessionId 当 phone - } - - private record CommandContext(String phone, int serial, Channel channel, Jt808Commands.DownlinkCommand cmd) {} -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808Commands.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808Commands.java deleted file mode 100644 index d62b7a62..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808Commands.java +++ /dev/null @@ -1,118 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.downlink; - -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; - -import java.io.ByteArrayOutputStream; -import java.nio.charset.Charset; -import java.nio.charset.StandardCharsets; -import java.util.List; - -/** - * 常用下行命令 body 构造器。返回的是 body 字节，外层由 {@link com.lingniu.ingest.protocol.jt808.codec.Jt808FrameEncoder} 包装。 - */ -public final class Jt808Commands { - - private static final Charset GBK = Charset.forName("GBK"); - - private Jt808Commands() {} - - /** 0x8001 平台通用应答（ack 一条终端上行）。 */ - public static DownlinkCommand platformAck(int ackSerial, int ackMsgId, int result) { - ByteArrayOutputStream os = new ByteArrayOutputStream(5); - writeU16(os, ackSerial); - writeU16(os, ackMsgId); - os.write(result & 0xFF); - return new DownlinkCommand(Jt808MessageId.PLATFORM_GENERAL_RESPONSE, os.toByteArray()); - } - - /** 0x8100 注册应答（result=0 成功）。 */ - public static DownlinkCommand registerAck(int ackSerial, int result, String token) { - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeU16(os, ackSerial); - os.write(result & 0xFF); - if (result == 0 && token != null) { - byte[] tokenBytes = token.getBytes(StandardCharsets.US_ASCII); - os.write(tokenBytes, 0, tokenBytes.length); - } - return new DownlinkCommand(Jt808MessageId.PLATFORM_REGISTER_ACK, os.toByteArray()); - } - - /** 0x8201 位置信息查询。 */ - public static DownlinkCommand queryLocation() { - return new DownlinkCommand(Jt808MessageId.PLATFORM_QUERY_LOCATION, new byte[0]); - } - - /** 0x8104 查询终端参数（全量）。 */ - public static DownlinkCommand queryParameters() { - return new DownlinkCommand(Jt808MessageId.PLATFORM_QUERY_PARAMS, new byte[0]); - } - - /** 0x8107 查询终端属性。 */ - public static DownlinkCommand queryAttributes() { - return new DownlinkCommand(Jt808MessageId.PLATFORM_QUERY_ATTRS, new byte[0]); - } - - /** - * 0x8103 设置终端参数：参数项列表格式。 - * - *

    body 形如 {@code count(1) + N*paramItem}，每个 paramItem 为 {@code paramId(4) + length(1) + value}。 - */ - public static DownlinkCommand setParameters(java.util.List items) { - ByteArrayOutputStream os = new ByteArrayOutputStream(); - os.write(items.size() & 0xFF); - for (ParamItem item : items) { - writeU32(os, item.paramId()); - os.write(item.value().length & 0xFF); - os.write(item.value(), 0, item.value().length); - } - return new DownlinkCommand(Jt808MessageId.PLATFORM_SET_PARAMS, os.toByteArray()); - } - - /** 0x8105 终端控制：命令字(1) + 命令参数(GBK)。 */ - public static DownlinkCommand terminalControl(int commandWord, String commandParams) { - ByteArrayOutputStream os = new ByteArrayOutputStream(); - os.write(commandWord & 0xFF); - if (commandParams != null) { - byte[] bytes = commandParams.getBytes(GBK); - os.write(bytes, 0, bytes.length); - } - return new DownlinkCommand(Jt808MessageId.PLATFORM_TERMINAL_CONTROL, os.toByteArray()); - } - - /** 0x8203 人工确认报警：报警消息流水号(WORD) + 报警类型(DWORD)。 */ - public static DownlinkCommand alarmAck(int responseSerialNo, long alarmType) { - ByteArrayOutputStream os = new ByteArrayOutputStream(6); - writeU16(os, responseSerialNo); - writeU32(os, alarmType); - return new DownlinkCommand(Jt808MessageId.PLATFORM_ALARM_ACK, os.toByteArray()); - } - - /** 0x8601 删除圆形区域：区域总数(BYTE) + N * 区域ID(DWORD)。 */ - public static DownlinkCommand deleteArea(List areaIds) { - if (areaIds.size() > 255) { - throw new IllegalArgumentException("jt808 delete-area supports at most 255 area ids"); - } - ByteArrayOutputStream os = new ByteArrayOutputStream(1 + areaIds.size() * 4); - os.write(areaIds.size() & 0xFF); - for (Long areaId : areaIds) { - writeU32(os, areaId == null ? 0 : areaId); - } - return new DownlinkCommand(Jt808MessageId.PLATFORM_DELETE_CIRCLE_AREA, os.toByteArray()); - } - - public record DownlinkCommand(int messageId, byte[] body) {} - - public record ParamItem(long paramId, byte[] value) {} - - private static void writeU16(ByteArrayOutputStream os, int v) { - os.write((v >> 8) & 0xFF); - os.write(v & 0xFF); - } - - private static void writeU32(ByteArrayOutputStream os, long v) { - os.write((int) ((v >> 24) & 0xFF)); - os.write((int) ((v >> 16) & 0xFF)); - os.write((int) ((v >> 8) & 0xFF)); - os.write((int) (v & 0xFF)); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/handler/Jt808LocationHandler.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/handler/Jt808LocationHandler.java deleted file mode 100644 index 83750860..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/handler/Jt808LocationHandler.java +++ /dev/null @@ -1,81 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.handler; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.annotation.AsyncBatch; -import com.lingniu.ingest.api.annotation.EventEmit; -import com.lingniu.ingest.api.annotation.MessageMapping; -import com.lingniu.ingest.api.annotation.ProtocolHandler; -import com.lingniu.ingest.api.annotation.RateLimited; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.protocol.jt808.mapper.Jt808EventMapper; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; - -import java.util.ArrayList; -import java.util.List; - -/** - * JT808 示例 Handler：演示注解驱动的路由与事件产出。 - * - *

    位置上报使用 {@code @AsyncBatch} 批量聚合（对标旧代码里的 4000/1s 批量落库）。 - * 带 {@code @AsyncBatch} 的方法签名约定为 {@code List}。 - */ -@ProtocolHandler(protocol = ProtocolId.JT808) -public class Jt808LocationHandler { - - private final Jt808EventMapper mapper; - - public Jt808LocationHandler(Jt808EventMapper mapper) { - this.mapper = mapper; - } - - @MessageMapping(command = { - Jt808MessageId.TERMINAL_LOCATION, - Jt808MessageId.TERMINAL_LOCATION_BATCH - }, desc = "位置信息汇报（批量）") - @RateLimited(perVin = 20) - @AsyncBatch(size = 4000, waitMs = 1000, poolSize = 2) - @EventEmit(VehicleEvent.Location.class) - public List onLocationBatch(List batch) { - List all = new ArrayList<>(batch.size()); - for (Jt808Message msg : batch) { - all.addAll(mapper.toEvents(msg)); - } - return all; - } - - @MessageMapping(command = {Jt808MessageId.TERMINAL_REGISTER, Jt808MessageId.TERMINAL_AUTH}, desc = "注册/鉴权") - @EventEmit(VehicleEvent.Login.class) - public List onLogin(Jt808Message msg) { - return mapper.toEvents(msg); - } - - @MessageMapping(command = { - Jt808MessageId.TERMINAL_UNREGISTER, - Jt808MessageId.TERMINAL_PARAMS_REPORT, - Jt808MessageId.TERMINAL_ATTRS_REPORT, - Jt808MessageId.TERMINAL_MEDIA_EVENT, - Jt808MessageId.TERMINAL_MEDIA_UPLOAD, - Jt808MessageId.TERMINAL_PASSTHROUGH - }, desc = "终端状态/属性/多媒体/透传") - @EventEmit({VehicleEvent.Logout.class, VehicleEvent.Login.class, VehicleEvent.MediaMeta.class, VehicleEvent.Passthrough.class}) - public List onExtended(Jt808Message msg) { - return mapper.toEvents(msg); - } - - @MessageMapping(command = Jt808MessageId.TERMINAL_HEARTBEAT, desc = "心跳") - @EventEmit(VehicleEvent.Heartbeat.class) - public List onHeartbeat(Jt808Message msg) { - return mapper.toEvents(msg); - } - - @MessageMapping(desc = "未解析 JT808 上行兜底透传") - @EventEmit(VehicleEvent.Passthrough.class) - public List onRaw(Jt808Message msg) { - if (!(msg.body() instanceof Jt808Body.Raw) && !(msg.body() instanceof Jt808Body.Malformed)) { - return List.of(); - } - return mapper.toEvents(msg); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/inbound/Jt808ChannelHandler.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/inbound/Jt808ChannelHandler.java deleted file mode 100644 index f849f288..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/inbound/Jt808ChannelHandler.java +++ /dev/null @@ -1,547 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.inbound; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentityBinding; -import com.lingniu.ingest.identity.VehicleIdentityLookup; -import com.lingniu.ingest.identity.VehicleIdentityRegistry; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.protocol.jt808.codec.Jt808FrameEncoder; -import com.lingniu.ingest.protocol.jt808.codec.Jt808MalformedFrame; -import com.lingniu.ingest.protocol.jt808.codec.Jt808MessageDecoder; -import com.lingniu.ingest.protocol.jt808.downlink.Jt808Commands; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808LocationAdditionalInfo; -import com.lingniu.ingest.protocol.jt808.model.Jt808LocationFlagInfo; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; -import com.lingniu.ingest.protocol.jt808.session.Jt808ChannelRegistry; -import com.lingniu.ingest.protocol.jt808.session.Jt808PendingRequests; -import com.lingniu.ingest.session.DeviceSession; -import com.lingniu.ingest.session.SessionStore; -import io.netty.buffer.Unpooled; -import io.netty.channel.ChannelHandlerContext; -import io.netty.channel.SimpleChannelInboundHandler; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.net.InetSocketAddress; -import java.nio.ByteBuffer; -import java.nio.charset.StandardCharsets; -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.time.Instant; -import java.util.HashMap; -import java.util.Map; -import java.util.UUID; - -/** - * JT808 Netty 入站处理器。职责： - *

      - *
    1. 解码 → 构造 RawFrame → 交给 {@link Dispatcher} - *
    2. 对 0x0100 注册 / 0x0102 鉴权 → 创建 / 更新 {@link DeviceSession} 并绑定 Channel - *
    3. 对 0x0001 终端通用应答 → 唤醒 {@link Jt808PendingRequests} 里等待的 future - *
    4. 对 0x0002 心跳 → 下发 0x8001 通用应答（若平台要求） - *
    5. 连接断开 → 清理 Channel 绑定 - *
    - */ -public class Jt808ChannelHandler extends SimpleChannelInboundHandler { - - private static final Logger log = LoggerFactory.getLogger(Jt808ChannelHandler.class); - private static final String PROCESSING_ERROR_PREFIX = "processing failed: "; - - private final Jt808MessageDecoder decoder; - private final Dispatcher dispatcher; - private final SessionStore sessions; - private final VehicleIdentityRegistry identityRegistry; - private final VehicleIdentityResolver identityResolver; - private final Jt808ChannelRegistry channelRegistry; - private final Jt808PendingRequests pendingRequests; - - public Jt808ChannelHandler(Jt808MessageDecoder decoder, Dispatcher dispatcher, - SessionStore sessions, VehicleIdentityRegistry identityRegistry, - Jt808ChannelRegistry channelRegistry, - Jt808PendingRequests pendingRequests) { - this(decoder, dispatcher, sessions, identityRegistry, - identityRegistry instanceof VehicleIdentityResolver resolver ? resolver : null, - channelRegistry, pendingRequests); - } - - public Jt808ChannelHandler(Jt808MessageDecoder decoder, Dispatcher dispatcher, - SessionStore sessions, VehicleIdentityRegistry identityRegistry, - VehicleIdentityResolver identityResolver, - Jt808ChannelRegistry channelRegistry, - Jt808PendingRequests pendingRequests) { - this.decoder = decoder; - this.dispatcher = dispatcher; - this.sessions = sessions; - this.identityRegistry = identityRegistry; - this.identityResolver = identityResolver; - this.channelRegistry = channelRegistry; - this.pendingRequests = pendingRequests; - } - - @Override - protected void channelRead0(ChannelHandlerContext ctx, Object msg) { - if (msg instanceof byte[] frame) { - handleDecodedFrame(ctx, frame); - } else if (msg instanceof Jt808MalformedFrame malformed) { - dispatchFrameError(ctx, malformed); - } else { - ctx.fireChannelRead(msg); - } - } - - private void handleDecodedFrame(ChannelHandlerContext ctx, byte[] frame) { - Jt808Message msg; - try { - msg = decoder.decode(ByteBuffer.wrap(frame)); - } catch (Exception e) { - log.warn("[jt808] decode failed peer={} len={} error={}", addr(ctx), frame.length, e.getMessage()); - log.debug("[jt808] decode failed stack peer={}", addr(ctx), e); - dispatchMalformed(ctx, frame, e.getMessage(), false); - return; - } - - try { - processDecodedFrame(ctx, frame, msg); - } catch (RuntimeException e) { - // 解码成功但业务处理失败时仍派发一条 malformed RawFrame， - // 保证原始帧能被 archive、Kafka 和历史索引留痕，方便事后排查。 - log.warn("[jt808] processing failed peer={} phone={} msgId=0x{} error={}", - addr(ctx), msg.header().phone(), Integer.toHexString(msg.header().messageId()), e.getMessage()); - log.debug("[jt808] processing failed stack peer={}", addr(ctx), e); - dispatchProcessingError(ctx, frame, msg, e.getMessage()); - } - } - - private void processDecodedFrame(ChannelHandlerContext ctx, byte[] frame, Jt808Message msg) { - // ====== 会话维护 & 同步应答匹配 ====== - String phone = msg.header().phone(); - IdentityResolution identity = resolveIdentity(msg); - if (msg.body() instanceof Jt808Body.Register reg) { - identity = registerIdentity(msg, reg, identity); - } - byte[] registerAck = null; - switch (msg.header().messageId()) { - case Jt808MessageId.TERMINAL_REGISTER -> { - DeviceSession session = upsertSession(ctx, msg, identity.identity()); - channelRegistry.bind(phone, ctx.channel()); - var command = Jt808Commands.registerAck(msg.header().serialNo(), 0, session.token()); - registerAck = Jt808FrameEncoder.encode( - command.messageId(), - phone, - channelRegistry.nextSerial(phone), - command.body()); - } - case Jt808MessageId.TERMINAL_AUTH -> { - upsertSession(ctx, msg, identity.identity()); - channelRegistry.bind(phone, ctx.channel()); - ackTerminal(ctx, phone, msg.header().serialNo(), msg.header().messageId(), 0); - } - case Jt808MessageId.TERMINAL_HEARTBEAT -> { - channelRegistry.bind(phone, ctx.channel()); - ackTerminal(ctx, phone, msg.header().serialNo(), msg.header().messageId(), 0); - } - case Jt808MessageId.TERMINAL_GENERAL_RESPONSE -> { - // body: ackSerial(2) + ackMsgId(2) + result(1) - if (msg.body() instanceof Jt808Body.Raw raw && raw.bytes().length >= 5) { - int ackSerial = ((raw.bytes()[0] & 0xFF) << 8) | (raw.bytes()[1] & 0xFF); - pendingRequests.complete(phone, ackSerial, msg); - } - } - default -> { - // 定位/其他上行：也可能是对同步命令的应答 - pendingRequests.complete(phone, msg.header().serialNo(), msg); - } - } - - // ====== 派发到 Dispatcher ====== - Map meta = new HashMap<>(32); - meta.put("vin", identity.identity().vin()); - meta.put("phone", phone); - meta.put("identityResolved", Boolean.toString(identity.identity().resolved())); - meta.put("identitySource", identity.identity().source().name()); - if (identity.errorMessage() != null) { - meta.put("identityError", "true"); - meta.put("identityErrorMessage", identity.errorMessage()); - } - meta.put("seq", Integer.toString(msg.header().serialNo())); - meta.put("messageId", "0x" + Jt808MessageId.hex(msg.header().messageId())); - meta.put("messageName", Jt808MessageId.nameOf(msg.header().messageId())); - meta.put("reportType", Jt808MessageId.reportTypeOf(msg.header().messageId())); - putBodyMetadata(meta, msg.body()); - meta.put("peer", addr(ctx)); - RawFrame rf = new RawFrame( - ProtocolId.JT808, - msg.header().messageId(), - 0, - msg, - frame, - meta, - Instant.now()); - if (registerAck != null) { - byte[] ack = registerAck; - dispatcher.dispatchAndAwait(rf) - .thenRun(() -> ctx.writeAndFlush(Unpooled.wrappedBuffer(ack))) - .exceptionally(ex -> { - log.warn("[jt808] register ack withheld because dispatch durability failed peer={} phone={} seq={} error={}", - addr(ctx), phone, msg.header().serialNo(), - ex.getMessage() == null ? ex.getClass().getSimpleName() : ex.getMessage()); - return null; - }); - return; - } - dispatcher.dispatch(rf); - } - - private IdentityResolution resolveIdentity(Jt808Message msg) { - String phone = msg.header().phone(); - if (identityResolver == null) { - // 没有绑定表时不伪造 VIN；phone 会保留在 metadata/session 中用于下行定位。 - return new IdentityResolution( - new VehicleIdentity("unknown", false, VehicleIdentitySource.FALLBACK_PHONE), - null); - } - try { - // 优先把 JT808 phone/device/plate 映射到内部 VIN，后续所有事件都用内部 VIN 做主键。 - return new IdentityResolution( - normalizeUnresolvedIdentity(identityResolver.resolve(new VehicleIdentityLookup( - ProtocolId.JT808, - "", - phone, - deviceId(msg.body()), - plate(msg.body())))), - null); - } catch (RuntimeException e) { - return new IdentityResolution( - new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN), - e.getMessage() == null ? "" : e.getMessage()); - } - } - - private static VehicleIdentity normalizeUnresolvedIdentity(VehicleIdentity identity) { - if (identity == null) { - return new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN); - } - if (identity.resolved()) { - return identity; - } - return new VehicleIdentity("unknown", false, identity.source()); - } - - private IdentityResolution registerIdentity(Jt808Message msg, - Jt808Body.Register reg, - IdentityResolution current) { - VehicleIdentity currentIdentity = current.identity(); - boolean alreadyResolved = currentIdentity != null && currentIdentity.resolved(); - String vin = alreadyResolved ? currentIdentity.vin() : generatedInternalVin(msg.header().phone(), reg); - identityRegistry.bind(new VehicleIdentityBinding( - ProtocolId.JT808, - vin, - msg.header().phone(), - reg.deviceId(), - reg.plate(), - Map.of( - "province", Integer.toString(reg.province()), - "city", Integer.toString(reg.city()), - "maker", reg.maker(), - "deviceType", reg.deviceType(), - "plateColor", Integer.toString(reg.plateColor())))); - VehicleIdentitySource source = alreadyResolved ? currentIdentity.source() : VehicleIdentitySource.REGISTERED; - return new IdentityResolution(new VehicleIdentity(vin, true, source), current.errorMessage()); - } - - private void dispatchFrameError(ChannelHandlerContext ctx, Jt808MalformedFrame malformed) { - String peer = malformed.peer() == null || malformed.peer().isBlank() ? addr(ctx) : malformed.peer(); - dispatchMalformed(peer, malformed.rawBytes(), malformed.reason(), true); - } - - private void dispatchMalformed(ChannelHandlerContext ctx, byte[] frame, String errorMessage, boolean frameError) { - dispatchMalformed(addr(ctx), frame, errorMessage, frameError); - } - - private void dispatchMalformed(String peer, byte[] frame, String errorMessage, boolean frameError) { - String message = errorMessage == null ? "" : errorMessage; - // 无法解出 header 的坏帧也进入 Dispatcher，后续 archive sink 可以保留 raw bytes。 - Jt808Message malformed = new Jt808Message( - new Jt808Header(0, frame == null ? 0 : frame.length, 0, false, - Jt808Header.ProtocolVersion.V2013, "unknown", 0, 0, 0), - new Jt808Body.Malformed(frame == null ? new byte[0] : frame, peer, message)); - Map meta = new HashMap<>(8); - meta.put("vin", "unknown"); - meta.put("peer", peer); - meta.put("identityResolved", "false"); - meta.put("identitySource", VehicleIdentitySource.UNKNOWN.name()); - if (frameError) { - meta.put("frameError", "true"); - meta.put("frameErrorMessage", message); - } else { - meta.put("parseError", "true"); - meta.put("parseErrorMessage", message); - } - putBodyMetadata(meta, malformed.body()); - dispatcher.dispatch(new RawFrame( - ProtocolId.JT808, - 0, - 0, - malformed, - frame == null ? new byte[0] : frame, - meta, - Instant.now())); - } - - private void dispatchProcessingError(ChannelHandlerContext ctx, byte[] frame, Jt808Message msg, String errorMessage) { - String message = errorMessage == null ? "" : errorMessage; - Jt808Message failed = new Jt808Message( - new Jt808Header(0, frame == null ? 0 : frame.length, 0, false, - msg.header().version(), msg.header().phone(), msg.header().serialNo(), 0, 0), - new Jt808Body.Malformed(frame == null ? new byte[0] : frame, addr(ctx), - PROCESSING_ERROR_PREFIX + message)); - Map meta = new HashMap<>(16); - meta.put("vin", "unknown"); - meta.put("phone", msg.header().phone()); - meta.put("peer", addr(ctx)); - meta.put("seq", Integer.toString(msg.header().serialNo())); - meta.put("messageId", "0x" + Jt808MessageId.hex(msg.header().messageId())); - meta.put("messageName", Jt808MessageId.nameOf(msg.header().messageId())); - meta.put("reportType", Jt808MessageId.reportTypeOf(msg.header().messageId())); - meta.put("identityResolved", "false"); - meta.put("identitySource", VehicleIdentitySource.UNKNOWN.name()); - meta.put("processingError", "true"); - meta.put("processingErrorMessage", message); - putBodyMetadata(meta, failed.body()); - dispatcher.dispatch(new RawFrame( - ProtocolId.JT808, - 0, - 0, - failed, - frame == null ? new byte[0] : frame, - meta, - Instant.now())); - } - - private DeviceSession upsertSession(ChannelHandlerContext ctx, Jt808Message msg, VehicleIdentity identity) { - String phone = msg.header().phone(); - String sessionId = "jt808-" + phone; - DeviceSession session = sessions.findBySessionId(sessionId) - .map(s -> s.touch(Instant.now())) - .orElseGet(() -> new DeviceSession( - sessionId, ProtocolId.JT808, identity.vin(), phone, - generateToken(phone), - addr(ctx), - Instant.now(), Instant.now(), - Map.of("protocolVersion", msg.header().version().name()))); - session = session.withVin(identity.vin()); - sessions.put(session); - return session; - } - - private static String deviceId(Jt808Body body) { - if (body instanceof Jt808Body.Register reg) { - return reg.deviceId(); - } - if (body instanceof Jt808Body.Auth auth) { - return auth.imei(); - } - return ""; - } - - private static String plate(Jt808Body body) { - return body instanceof Jt808Body.Register reg ? reg.plate() : ""; - } - - private static void putBodyMetadata(Map meta, Jt808Body body) { - switch (body) { - case Jt808Body.Register reg -> { - put(meta, "body.province", reg.province()); - put(meta, "body.city", reg.city()); - put(meta, "body.maker", reg.maker()); - put(meta, "body.deviceType", reg.deviceType()); - put(meta, "body.deviceId", reg.deviceId()); - put(meta, "body.plateColor", reg.plateColor()); - put(meta, "body.plate", reg.plate()); - } - case Jt808Body.Auth auth -> { - put(meta, "body.token", auth.token()); - put(meta, "body.imei", auth.imei()); - put(meta, "body.softwareVersion", auth.softwareVersion()); - } - case Jt808Body.Location location -> putLocationMetadata(meta, "body.", location); - case Jt808Body.LocationBatch batch -> { - put(meta, "body.batchType", batch.batchType()); - put(meta, "body.locationCount", batch.locations() == null ? 0 : batch.locations().size()); - } - case Jt808Body.ParamsReport report -> { - put(meta, "body.responseSerialNo", report.responseSerialNo()); - put(meta, "body.parameterCount", report.parameters() == null ? 0 : report.parameters().size()); - if (report.parameters() != null) { - report.parameters().forEach((id, bytes) -> - put(meta, "body.param.0x" + String.format("%08X", id), toHex(bytes))); - } - } - case Jt808Body.TerminalAttrs attrs -> { - put(meta, "body.terminalType", attrs.terminalType()); - put(meta, "body.maker", attrs.maker()); - put(meta, "body.terminalModel", attrs.terminalModel()); - put(meta, "body.terminalId", attrs.terminalId()); - put(meta, "body.iccid", attrs.iccid()); - put(meta, "body.hardwareVersion", attrs.hardwareVersion()); - put(meta, "body.firmwareVersion", attrs.firmwareVersion()); - put(meta, "body.gnssProperty", attrs.gnssProperty()); - put(meta, "body.commProperty", attrs.commProperty()); - } - case Jt808Body.MediaEvent event -> { - put(meta, "body.mediaId", event.mediaId()); - put(meta, "body.mediaType", event.mediaType()); - put(meta, "body.mediaFormat", event.mediaFormat()); - put(meta, "body.eventCode", event.eventCode()); - put(meta, "body.channelId", event.channelId()); - } - case Jt808Body.MediaUpload upload -> { - put(meta, "body.mediaId", upload.mediaId()); - put(meta, "body.mediaType", upload.mediaType()); - put(meta, "body.mediaFormat", upload.mediaFormat()); - put(meta, "body.eventCode", upload.eventCode()); - put(meta, "body.channelId", upload.channelId()); - put(meta, "body.dataSizeBytes", upload.data() == null ? 0 : upload.data().length); - if (upload.location() != null) { - putLocationMetadata(meta, "body.location.", upload.location()); - } - } - case Jt808Body.MediaAttributes attrs -> { - put(meta, "body.inputAudioEncoding", attrs.inputAudioEncoding()); - put(meta, "body.inputAudioChannels", attrs.inputAudioChannels()); - put(meta, "body.inputAudioSampleRate", attrs.inputAudioSampleRate()); - put(meta, "body.inputAudioBitDepth", attrs.inputAudioBitDepth()); - put(meta, "body.audioFrameLength", attrs.audioFrameLength()); - put(meta, "body.audioOutputSupported", attrs.audioOutputSupported()); - put(meta, "body.videoEncoding", attrs.videoEncoding()); - put(meta, "body.maxAudioChannels", attrs.maxAudioChannels()); - put(meta, "body.maxVideoChannels", attrs.maxVideoChannels()); - put(meta, "body.rawSizeBytes", attrs.raw() == null ? 0 : attrs.raw().length); - } - case Jt808Body.FileUploadComplete complete -> { - put(meta, "body.responseSerialNo", complete.responseSerialNo()); - put(meta, "body.result", complete.result()); - put(meta, "body.rawSizeBytes", complete.raw() == null ? 0 : complete.raw().length); - } - case Jt808Body.FileList files -> { - put(meta, "body.responseSerialNo", files.responseSerialNo()); - put(meta, "body.fileCount", files.fileCount()); - put(meta, "body.rawSizeBytes", files.raw() == null ? 0 : files.raw().length); - } - case Jt808Body.PassengerVolume volume -> { - put(meta, "body.channelId", volume.channelId()); - put(meta, "body.startTime", volume.startTime()); - put(meta, "body.endTime", volume.endTime()); - put(meta, "body.passengerGetOn", volume.passengerGetOn()); - put(meta, "body.passengerGetOff", volume.passengerGetOff()); - put(meta, "body.rawSizeBytes", volume.raw() == null ? 0 : volume.raw().length); - } - case Jt808Body.Passthrough passthrough -> { - put(meta, "body.passthroughType", "0x" + String.format("%02X", passthrough.passthroughType())); - put(meta, "body.dataSizeBytes", passthrough.data() == null ? 0 : passthrough.data().length); - put(meta, "body.dataHex", toHex(passthrough.data())); - } - case Jt808Body.Raw raw -> { - put(meta, "body.rawMessageId", "0x" + Jt808MessageId.hex(raw.messageId())); - put(meta, "body.rawSizeBytes", raw.bytes() == null ? 0 : raw.bytes().length); - put(meta, "body.rawHex", toHex(raw.bytes())); - } - case Jt808Body.Malformed malformed -> { - put(meta, "body.peer", malformed.peer()); - put(meta, "body.errorMessage", malformed.errorMessage()); - put(meta, "body.rawSizeBytes", malformed.bytes() == null ? 0 : malformed.bytes().length); - } - case Jt808Body.Heartbeat ignored -> { - } - case Jt808Body.Unregister ignored -> { - } - } - } - - private static void putLocationMetadata(Map meta, String prefix, Jt808Body.Location location) { - put(meta, prefix + "alarmFlag", location.alarmFlag()); - put(meta, prefix + "statusFlag", location.statusFlag()); - meta.putAll(Jt808LocationFlagInfo.decodeStatus(location.statusFlag(), prefix + "status.")); - meta.putAll(Jt808LocationFlagInfo.decodeAlarm(location.alarmFlag(), prefix + "alarm.")); - put(meta, prefix + "longitude", location.longitude()); - put(meta, prefix + "latitude", location.latitude()); - put(meta, prefix + "altitudeM", location.altitudeM()); - put(meta, prefix + "speedKmh", location.speedKmh()); - put(meta, prefix + "directionDeg", location.directionDeg()); - put(meta, prefix + "deviceTime", location.deviceTime()); - if (location.extensionItems() != null) { - location.extensionItems().forEach((id, bytes) -> - put(meta, prefix + "extra.0x" + String.format("%02X", id), toHex(bytes))); - meta.putAll(Jt808LocationAdditionalInfo.decode(location.extensionItems(), prefix + "extra.")); - } - } - - private static void put(Map meta, String key, Object value) { - if (value != null) { - meta.put(key, String.valueOf(value)); - } - } - - private static String toHex(byte[] bytes) { - if (bytes == null || bytes.length == 0) { - return ""; - } - StringBuilder out = new StringBuilder(bytes.length * 2); - for (byte b : bytes) { - out.append(String.format("%02X", b & 0xFF)); - } - return out.toString(); - } - - private void ackTerminal(ChannelHandlerContext ctx, String phone, int ackSerial, int ackMsgId, int result) { - var cmd = Jt808Commands.platformAck(ackSerial, ackMsgId, result); - byte[] frame = Jt808FrameEncoder.encode(cmd.messageId(), phone, channelRegistry.nextSerial(phone), cmd.body()); - ctx.writeAndFlush(Unpooled.wrappedBuffer(frame)); - } - - private static String generateToken(String phone) { - return UUID.nameUUIDFromBytes(("jt808:" + phone).getBytes()).toString().replace("-", "").substring(0, 16); - } - - private static String generatedInternalVin(String phone, Jt808Body.Register reg) { - String material = String.join("|", phone, reg.deviceId(), reg.plate(), reg.maker(), reg.deviceType()); - try { - byte[] digest = MessageDigest.getInstance("SHA-256").digest(material.getBytes(StandardCharsets.UTF_8)); - StringBuilder suffix = new StringBuilder(12); - for (int i = 0; i < digest.length && suffix.length() < 12; i++) { - suffix.append(String.format("%02X", digest[i] & 0xFF)); - } - return "JT808" + suffix.substring(0, 12); - } catch (NoSuchAlgorithmException e) { - throw new IllegalStateException("SHA-256 not available", e); - } - } - - @Override - public void channelInactive(ChannelHandlerContext ctx) { - channelRegistry.unbind(ctx.channel()) - .ifPresent(phone -> sessions.remove("jt808-" + phone)); - } - - @Override - public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) { - log.warn("[jt808] channel error peer={}", addr(ctx), cause); - ctx.close(); - } - - private static String addr(ChannelHandlerContext ctx) { - if (ctx.channel().remoteAddress() instanceof InetSocketAddress i) { - return i.getAddress().getHostAddress() + ":" + i.getPort(); - } - return "unknown"; - } - - private record IdentityResolution(VehicleIdentity identity, String errorMessage) {} -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/inbound/Jt808NettyServer.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/inbound/Jt808NettyServer.java deleted file mode 100644 index f7d63a1a..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/inbound/Jt808NettyServer.java +++ /dev/null @@ -1,104 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.inbound; - -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.identity.VehicleIdentityRegistry; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.protocol.jt808.codec.Jt808FrameDecoder; -import com.lingniu.ingest.protocol.jt808.codec.Jt808MessageDecoder; -import com.lingniu.ingest.protocol.jt808.session.Jt808ChannelRegistry; -import com.lingniu.ingest.protocol.jt808.session.Jt808PendingRequests; -import com.lingniu.ingest.session.SessionStore; -import io.netty.bootstrap.ServerBootstrap; -import io.netty.channel.Channel; -import io.netty.channel.ChannelInitializer; -import io.netty.channel.ChannelOption; -import io.netty.channel.EventLoopGroup; -import io.netty.channel.nio.NioEventLoopGroup; -import io.netty.channel.socket.SocketChannel; -import io.netty.channel.socket.nio.NioServerSocketChannel; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.beans.factory.DisposableBean; -import org.springframework.beans.factory.InitializingBean; - -import java.util.concurrent.ThreadFactory; - -public class Jt808NettyServer implements InitializingBean, DisposableBean { - - private static final Logger log = LoggerFactory.getLogger(Jt808NettyServer.class); - - private final int port; - private final int workerThreads; - private final Jt808MessageDecoder messageDecoder; - private final Dispatcher dispatcher; - private final SessionStore sessions; - private final VehicleIdentityRegistry identityRegistry; - private final VehicleIdentityResolver identityResolver; - private final Jt808ChannelRegistry channelRegistry; - private final Jt808PendingRequests pendingRequests; - - private EventLoopGroup boss; - private EventLoopGroup workers; - private Channel serverChannel; - - public Jt808NettyServer(int port, int workerThreads, - Jt808MessageDecoder messageDecoder, Dispatcher dispatcher, - SessionStore sessions, - VehicleIdentityRegistry identityRegistry, - VehicleIdentityResolver identityResolver, - Jt808ChannelRegistry channelRegistry, - Jt808PendingRequests pendingRequests) { - this.port = port; - this.workerThreads = workerThreads; - this.messageDecoder = messageDecoder; - this.dispatcher = dispatcher; - this.sessions = sessions; - this.identityRegistry = identityRegistry; - this.identityResolver = identityResolver; - this.channelRegistry = channelRegistry; - this.pendingRequests = pendingRequests; - } - - @Override - public void afterPropertiesSet() throws Exception { - ThreadFactory bossTf = r -> new Thread(r, "jt808-boss"); - ThreadFactory wkTf = r -> new Thread(r, "jt808-worker"); - this.boss = new NioEventLoopGroup(1, bossTf); - this.workers = new NioEventLoopGroup( - workerThreads == 0 ? Runtime.getRuntime().availableProcessors() * 2 : workerThreads, - wkTf); - - ServerBootstrap b = new ServerBootstrap(); - b.group(boss, workers) - .channel(NioServerSocketChannel.class) - .option(ChannelOption.SO_BACKLOG, 1024) - .childOption(ChannelOption.TCP_NODELAY, true) - .childOption(ChannelOption.SO_KEEPALIVE, true) - .childHandler(new ChannelInitializer() { - @Override - protected void initChannel(SocketChannel ch) { - ch.pipeline() - .addLast("frame", new Jt808FrameDecoder()) - .addLast("dispatch", new Jt808ChannelHandler( - messageDecoder, dispatcher, sessions, - identityRegistry, identityResolver, - channelRegistry, pendingRequests)); - } - }); - this.serverChannel = b.bind(port).sync().channel(); - log.info("[jt808] Netty server listening on :{}", port); - } - - @Override - public void destroy() { - try { - if (serverChannel != null) serverChannel.close().sync(); - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); - } finally { - if (boss != null) boss.shutdownGracefully(); - if (workers != null) workers.shutdownGracefully(); - log.info("[jt808] Netty server stopped"); - } - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/mapper/Jt808EventMapper.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/mapper/Jt808EventMapper.java deleted file mode 100644 index d336762b..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/mapper/Jt808EventMapper.java +++ /dev/null @@ -1,379 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.mapper; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.LocationPayload; -import com.lingniu.ingest.api.event.TelemetryMetadataValues; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.spi.EventMapper; -import com.lingniu.ingest.identity.VehicleIdentity; -import com.lingniu.ingest.identity.VehicleIdentityLookup; -import com.lingniu.ingest.identity.VehicleIdentityResolver; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808LocationAdditionalInfo; -import com.lingniu.ingest.protocol.jt808.model.Jt808LocationFlagInfo; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; - -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; -import java.util.UUID; - -/** - * JT808 消息 → 领域事件映射。sealed body 允许 switch 穷尽处理。 - * - *

    备注：JT808 里没有强制的 VIN 字段，事件 VIN 统一通过 {@link VehicleIdentityResolver} - * 解析；未绑定时才降级为终端手机号，metadata 同步保留内部 vin 与外部 phone。 - */ -public final class Jt808EventMapper implements EventMapper { - - private static final String PROCESSING_ERROR_PREFIX = "processing failed: "; - - private final VehicleIdentityResolver identityResolver; - - public Jt808EventMapper(VehicleIdentityResolver identityResolver) { - this.identityResolver = identityResolver; - } - - @Override - public List toEvents(Jt808Message message) { - var header = message.header(); - Instant ingestTime = Instant.now(); - IdentityResolution identity = resolveIdentity(message); - Map meta = baseMeta(header, identity); - String vin = identity.vin(); - - return switch (message.body()) { - case Jt808Body.Location loc -> List.of(new VehicleEvent.Location( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, loc.deviceTime(), ingestTime, null, locationMeta(meta, loc), - new LocationPayload(loc.longitude(), loc.latitude(), - loc.altitudeM(), loc.speedKmh(), loc.directionDeg(), - loc.alarmFlag(), loc.statusFlag(), totalMileageKm(loc)))); - - case Jt808Body.Register reg -> List.of(new VehicleEvent.Login( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, registerMeta(meta, reg), - reg.deviceId(), header.version().name())); - - case Jt808Body.Auth auth -> List.of(new VehicleEvent.Login( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, meta, - auth.imei() == null ? "" : auth.imei(), header.version().name())); - - case Jt808Body.Heartbeat _ -> List.of(new VehicleEvent.Heartbeat( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, meta)); - - case Jt808Body.Unregister _ -> List.of(new VehicleEvent.Logout( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, meta)); - - case Jt808Body.LocationBatch batch -> locationBatchEvents(vin, meta, batch, ingestTime); - - case Jt808Body.ParamsReport params -> List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, - withMeta(meta, "paramsCount", Integer.toString(params.parameters().size())), - header.messageId(), - encodeParamsSummary(params))); - - case Jt808Body.TerminalAttrs attrs -> List.of(new VehicleEvent.Login( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, - withMeta(meta, "terminalId", attrs.terminalId()), - attrs.iccid(), header.version().name())); - - case Jt808Body.MediaEvent media -> List.of(new VehicleEvent.MediaMeta( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, - withMeta(meta, "mediaEventCode", Integer.toString(media.eventCode())), - Long.toString(media.mediaId()), - mediaType(media.mediaType(), media.mediaFormat(), media.channelId()), - 0, - "")); - - case Jt808Body.MediaUpload media -> mediaUploadEvents(vin, meta, media, ingestTime); - - case Jt808Body.MediaAttributes attrs -> List.of(new VehicleEvent.MediaMeta( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, - withMeta(withMeta(meta, "jt1078Message", "mediaAttributes"), - "maxVideoChannels", Integer.toString(attrs.maxVideoChannels())), - "jt1078-attrs-" + header.phone(), - mediaAttributesType(attrs), - attrs.raw() == null ? 0 : attrs.raw().length, - "")); - - case Jt808Body.FileUploadComplete done -> List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, - withMeta(withMeta(withMeta(meta, "jt1078Message", "fileUploadComplete"), - "responseSerialNo", Integer.toString(done.responseSerialNo())), - "uploadResult", Integer.toString(done.result())), - header.messageId(), - done.raw() == null ? new byte[0] : done.raw())); - - case Jt808Body.FileList list -> List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, - withMeta(withMeta(withMeta(meta, "jt1078Message", "fileList"), - "responseSerialNo", Integer.toString(list.responseSerialNo())), - "fileCount", Long.toString(list.fileCount())), - header.messageId(), - list.raw() == null ? new byte[0] : list.raw())); - - case Jt808Body.PassengerVolume volume -> List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, - passengerVolumeMeta(meta, volume), - header.messageId(), - volume.raw() == null ? new byte[0] : volume.raw())); - - case Jt808Body.Passthrough passthrough -> List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, - withMeta(meta, "passthroughType", "0x" + Integer.toHexString(passthrough.passthroughType())), - header.messageId(), - passthrough.data())); - - case Jt808Body.Raw raw -> List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, - withMeta(meta, "rawBody", "true"), - raw.messageId(), - raw.bytes())); - - case Jt808Body.Malformed malformed -> { - String errorMessage = malformed.errorMessage() == null ? "" : malformed.errorMessage(); - boolean processingError = isProcessingError(errorMessage); - boolean frameError = isFrameError(errorMessage); - String publicErrorMessage = processingError - ? errorMessage.substring(PROCESSING_ERROR_PREFIX.length()) - : errorMessage; - Map errorMeta = withMeta(meta, - processingError ? "processingError" : frameError ? "frameError" : "parseError", "true"); - errorMeta = withMeta(errorMeta, "peer", malformed.peer() == null ? "" : malformed.peer()); - errorMeta = withMeta(errorMeta, - processingError ? "processingErrorMessage" : frameError ? "frameErrorMessage" : "parseErrorMessage", - publicErrorMessage); - yield List.of(new VehicleEvent.Passthrough( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, - errorMeta, - header.messageId(), - malformed.bytes() == null ? new byte[0] : malformed.bytes())); - } - }; - } - - private IdentityResolution resolveIdentity(Jt808Message message) { - if (message.body() instanceof Jt808Body.Malformed) { - // 坏帧没有可信 header/body，统一 unknown，原始字节仍由 passthrough 事件保留。 - return IdentityResolution.ok(new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN)); - } - var header = message.header(); - try { - return IdentityResolution.ok(normalizeUnresolvedIdentity(identityResolver.resolve(new VehicleIdentityLookup( - ProtocolId.JT808, "", header.phone(), deviceId(message.body()), plate(message.body()))))); - } catch (RuntimeException e) { - return new IdentityResolution( - new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN), - e.getMessage() == null ? "" : e.getMessage()); - } - } - - private static VehicleIdentity normalizeUnresolvedIdentity(VehicleIdentity identity) { - if (identity == null) { - return new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN); - } - if (identity.resolved()) { - return identity; - } - return new VehicleIdentity("unknown", false, identity.source()); - } - - private static Map baseMeta(Jt808Header header, IdentityResolution identity) { - java.util.HashMap meta = new java.util.HashMap<>(); - meta.put("messageId", "0x" + Integer.toHexString(header.messageId())); - meta.put("protocolVersion", header.version().name()); - meta.put("vin", identity.vin()); - meta.put("phone", header.phone()); - meta.put("identityResolved", Boolean.toString(identity.identity().resolved())); - meta.put("identitySource", identity.identity().source().name()); - if (identity.errorMessage() != null) { - meta.put("identityError", "true"); - meta.put("identityErrorMessage", identity.errorMessage()); - } - return Map.copyOf(meta); - } - - private static boolean isFrameError(String errorMessage) { - return errorMessage != null && (errorMessage.startsWith("jt808 missing frame boundary") - || errorMessage.startsWith("jt808 leading bytes before frame boundary") - || errorMessage.startsWith("jt808 unclosed frame too large")); - } - - private static boolean isProcessingError(String errorMessage) { - return errorMessage != null && errorMessage.startsWith(PROCESSING_ERROR_PREFIX); - } - - private static List locationBatchEvents(String vin, - Map meta, - Jt808Body.LocationBatch batch, - Instant ingestTime) { - List out = new ArrayList<>(batch.locations().size()); - Map batchMeta = withMeta(meta, "batchType", Integer.toString(batch.batchType())); - // 批量位置拆成多条 Location 事件，方便历史库按单点时间查询和导出。 - for (Jt808Body.Location loc : batch.locations()) { - out.add(locationEvent(vin, batchMeta, loc, ingestTime)); - } - return out; - } - - private static List mediaUploadEvents(String vin, - Map meta, - Jt808Body.MediaUpload media, - Instant ingestTime) { - List out = new ArrayList<>(2); - if (media.location() != null) { - // 多媒体上传消息内嵌位置时同时产出 Location，避免只看到媒体元数据看不到拍摄位置。 - out.add(locationEvent(vin, withMeta(meta, "mediaId", Long.toString(media.mediaId())), - media.location(), ingestTime)); - } - out.add(new VehicleEvent.MediaMeta( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, ingestTime, ingestTime, null, - withMeta(meta, "mediaEventCode", Integer.toString(media.eventCode())), - Long.toString(media.mediaId()), - mediaType(media.mediaType(), media.mediaFormat(), media.channelId()), - media.data() == null ? 0 : media.data().length, - "")); - return out; - } - - private static Map passengerVolumeMeta(Map meta, - Jt808Body.PassengerVolume volume) { - java.util.HashMap copy = new java.util.HashMap<>(meta); - copy.put("jt1078Message", "passengerVolume"); - copy.put("channelId", Integer.toString(volume.channelId())); - copy.put("startTime", volume.startTime() == null ? "" : volume.startTime()); - copy.put("endTime", volume.endTime() == null ? "" : volume.endTime()); - copy.put("passengerGetOn", Integer.toString(volume.passengerGetOn())); - copy.put("passengerGetOff", Integer.toString(volume.passengerGetOff())); - return Map.copyOf(copy); - } - - private static VehicleEvent.Location locationEvent(String vin, - Map meta, - Jt808Body.Location loc, - Instant ingestTime) { - return new VehicleEvent.Location( - UUID.randomUUID().toString(), - vin, ProtocolId.JT808, loc.deviceTime(), ingestTime, null, locationMeta(meta, loc), - new LocationPayload(loc.longitude(), loc.latitude(), - loc.altitudeM(), loc.speedKmh(), loc.directionDeg(), - loc.alarmFlag(), loc.statusFlag(), totalMileageKm(loc))); - } - - private static Map locationMeta(Map meta, Jt808Body.Location loc) { - java.util.HashMap copy = new java.util.HashMap<>(meta); - copy.putAll(Jt808LocationFlagInfo.decodeStatus(loc.statusFlag(), "jt808.status.")); - copy.putAll(Jt808LocationFlagInfo.decodeAlarm(loc.alarmFlag(), "jt808.alarm.")); - if (!loc.extensionItems().isEmpty()) { - loc.extensionItems().forEach((id, bytes) -> - copy.put("jt808.extra.0x" + String.format("%02X", id), hex(bytes))); - copy.putAll(Jt808LocationAdditionalInfo.decode(loc.extensionItems())); - Double totalMileageKm = totalMileageKm(loc); - if (totalMileageKm != null) { - copy.put("total_mileage_km", TelemetryMetadataValues.doubleValue(totalMileageKm)); - } - } - return Map.copyOf(copy); - } - - private static Map registerMeta(Map meta, Jt808Body.Register reg) { - java.util.HashMap copy = new java.util.HashMap<>(meta); - copy.put("jt808.register.province", Integer.toString(reg.province())); - copy.put("jt808.register.city", Integer.toString(reg.city())); - copy.put("jt808.register.maker", nullToEmpty(reg.maker())); - copy.put("jt808.register.deviceType", nullToEmpty(reg.deviceType())); - copy.put("jt808.register.deviceId", nullToEmpty(reg.deviceId())); - copy.put("jt808.register.plateColor", Integer.toString(reg.plateColor())); - copy.put("jt808.register.plate", nullToEmpty(reg.plate())); - return Map.copyOf(copy); - } - - private static Map withMeta(Map meta, String key, String value) { - java.util.HashMap copy = new java.util.HashMap<>(meta); - copy.put(key, value == null ? "" : value); - return Map.copyOf(copy); - } - - private static Double totalMileageKm(Jt808Body.Location loc) { - return Jt808LocationAdditionalInfo.mileageKm(loc.extensionItems()); - } - - private static String hex(byte[] bytes) { - if (bytes == null || bytes.length == 0) { - return ""; - } - StringBuilder sb = new StringBuilder(bytes.length * 2); - for (byte b : bytes) { - sb.append(Character.forDigit((b >> 4) & 0x0F, 16)); - sb.append(Character.forDigit(b & 0x0F, 16)); - } - return sb.toString().toUpperCase(java.util.Locale.ROOT); - } - - private static byte[] encodeParamsSummary(Jt808Body.ParamsReport params) { - return ("responseSerialNo=" + params.responseSerialNo() - + ",parameters=" + params.parameters().keySet()).getBytes(java.nio.charset.StandardCharsets.UTF_8); - } - - private static String mediaType(int mediaType, int mediaFormat, int channelId) { - return "type=" + mediaType + ",format=" + mediaFormat + ",channel=" + channelId; - } - - private static String mediaAttributesType(Jt808Body.MediaAttributes attrs) { - return "audioEncoding=" + attrs.inputAudioEncoding() - + ",audioChannels=" + attrs.inputAudioChannels() - + ",audioSampleRate=" + attrs.inputAudioSampleRate() - + ",audioBitDepth=" + attrs.inputAudioBitDepth() - + ",audioFrameLength=" + attrs.audioFrameLength() - + ",audioOutput=" + attrs.audioOutputSupported() - + ",videoEncoding=" + attrs.videoEncoding() - + ",maxAudioChannels=" + attrs.maxAudioChannels() - + ",maxVideoChannels=" + attrs.maxVideoChannels(); - } - - private static String deviceId(Jt808Body body) { - if (body instanceof Jt808Body.Register reg) { - return reg.deviceId(); - } - if (body instanceof Jt808Body.Auth auth) { - return auth.imei(); - } - return ""; - } - - private static String plate(Jt808Body body) { - return body instanceof Jt808Body.Register reg ? reg.plate() : ""; - } - - private static String nullToEmpty(String value) { - return value == null ? "" : value; - } - - private record IdentityResolution(VehicleIdentity identity, String errorMessage) { - private static IdentityResolution ok(VehicleIdentity identity) { - return new IdentityResolution(identity, null); - } - - private String vin() { - return identity.vin(); - } - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808Body.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808Body.java deleted file mode 100644 index e651c375..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808Body.java +++ /dev/null @@ -1,163 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.model; - -import java.time.Instant; -import java.util.List; -import java.util.Map; - -/** - * 已解析的消息体。sealed，新增消息体时必须同步新增子类型与 Parser。 - */ -public sealed interface Jt808Body - permits Jt808Body.Register, Jt808Body.Auth, Jt808Body.Location, - Jt808Body.LocationBatch, Jt808Body.Unregister, - Jt808Body.ParamsReport, Jt808Body.TerminalAttrs, - Jt808Body.MediaEvent, Jt808Body.MediaUpload, - Jt808Body.MediaAttributes, Jt808Body.FileUploadComplete, - Jt808Body.FileList, Jt808Body.PassengerVolume, - Jt808Body.Passthrough, Jt808Body.Heartbeat, Jt808Body.Raw, - Jt808Body.Malformed { - - /** 终端注册 0x0100。 */ - record Register( - int province, - int city, - String maker, - String deviceType, - String deviceId, - int plateColor, - String plate - ) implements Jt808Body {} - - /** 终端鉴权 0x0102。 */ - record Auth(String token, String imei, String softwareVersion) implements Jt808Body {} - - /** 位置信息汇报 0x0200。 */ - record Location( - long alarmFlag, - long statusFlag, - double longitude, - double latitude, - int altitudeM, - double speedKmh, - int directionDeg, - Instant deviceTime, - Map extensionItems - ) implements Jt808Body { - public Location(long alarmFlag, - long statusFlag, - double longitude, - double latitude, - int altitudeM, - double speedKmh, - int directionDeg, - Instant deviceTime) { - this(alarmFlag, statusFlag, longitude, latitude, altitudeM, speedKmh, - directionDeg, deviceTime, Map.of()); - } - - public Location { - extensionItems = extensionItems == null ? Map.of() : Map.copyOf(extensionItems); - } - } - - /** 位置信息批量上传 0x0704。 */ - record LocationBatch( - int batchType, - List locations - ) implements Jt808Body {} - - /** 终端注销 0x0003。 */ - record Unregister() implements Jt808Body {} - - /** 查询终端参数应答 0x0104。 */ - record ParamsReport( - int responseSerialNo, - Map parameters - ) implements Jt808Body {} - - /** 查询终端属性应答 0x0107。 */ - record TerminalAttrs( - int terminalType, - String maker, - String terminalModel, - String terminalId, - String iccid, - String hardwareVersion, - String firmwareVersion, - int gnssProperty, - int commProperty - ) implements Jt808Body {} - - /** 多媒体事件信息上传 0x0800。 */ - record MediaEvent( - long mediaId, - int mediaType, - int mediaFormat, - int eventCode, - int channelId - ) implements Jt808Body {} - - /** 多媒体数据上传 0x0801。 */ - record MediaUpload( - long mediaId, - int mediaType, - int mediaFormat, - int eventCode, - int channelId, - Location location, - byte[] data - ) implements Jt808Body {} - - /** JT/T 1078 终端上传音视频属性 0x1003。 */ - record MediaAttributes( - int inputAudioEncoding, - int inputAudioChannels, - int inputAudioSampleRate, - int inputAudioBitDepth, - int audioFrameLength, - boolean audioOutputSupported, - int videoEncoding, - int maxAudioChannels, - int maxVideoChannels, - byte[] raw - ) implements Jt808Body {} - - /** JT/T 1078 文件上传完成通知 0x1206。 */ - record FileUploadComplete( - int responseSerialNo, - int result, - byte[] raw - ) implements Jt808Body {} - - /** JT/T 1078 终端上传文件列表 0x1205。 */ - record FileList( - int responseSerialNo, - long fileCount, - byte[] raw - ) implements Jt808Body {} - - /** JT/T 1078 终端上传乘客流量 0x1005。 */ - record PassengerVolume( - int channelId, - String startTime, - String endTime, - int passengerGetOn, - int passengerGetOff, - byte[] raw - ) implements Jt808Body {} - - /** 数据上行透传 0x0900。 */ - record Passthrough( - int passthroughType, - byte[] data - ) implements Jt808Body {} - - /** 心跳 0x0002（无 body）。 */ - record Heartbeat() implements Jt808Body {} - - /** 未实现 Parser 的消息体，透传原始字节。 */ - record Raw(int messageId, byte[] bytes) implements Jt808Body {} - - /** 入站帧整体解析失败，保留原始字节便于冷存、查询和排障。 */ - record Malformed(byte[] bytes, String peer, String errorMessage) implements Jt808Body {} -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808Header.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808Header.java deleted file mode 100644 index e1ce252b..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808Header.java +++ /dev/null @@ -1,28 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.model; - -/** - * JT/T 808 消息头。 - * - * @param messageId 消息 ID - * @param bodyLength 消息体长度（从 attrs 低 10 位解出） - * @param encryptType 加密类型（0=不加密 / 1=RSA） - * @param subpacket 是否分包 - * @param version 协议版本（2013 或 2019） - * @param phone 终端手机号 / 设备识别号（2013:12 位，2019:20 位） - * @param serialNo 消息流水号 - * @param totalPackets 分包总数（不分包时为 0） - * @param packetSeq 分包序号（从 1 开始） - */ -public record Jt808Header( - int messageId, - int bodyLength, - int encryptType, - boolean subpacket, - ProtocolVersion version, - String phone, - int serialNo, - int totalPackets, - int packetSeq -) { - public enum ProtocolVersion { V2013, V2019 } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808LocationAdditionalInfo.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808LocationAdditionalInfo.java deleted file mode 100644 index 3e5edcb7..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808LocationAdditionalInfo.java +++ /dev/null @@ -1,247 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.model; - -import java.math.BigDecimal; -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; - -public final class Jt808LocationAdditionalInfo { - - public static final String DEFAULT_PREFIX = "jt808.extra."; - - private static final String[] VEHICLE_SIGNAL_BITS = { - "low_beam", - "high_beam", - "right_turn", - "left_turn", - "brake", - "reverse", - "fog_lamp", - "position_lamp", - "horn", - "air_conditioner", - "neutral", - "retarder", - "abs", - "heater", - "clutch" - }; - - private static final String[] IO_STATUS_BITS = { - "deep_sleep", - "sleep" - }; - - private Jt808LocationAdditionalInfo() { - } - - public static Map decode(Map extensionItems) { - return decode(extensionItems, DEFAULT_PREFIX); - } - - public static Map decode(Map extensionItems, String prefix) { - if (extensionItems == null || extensionItems.isEmpty()) { - return Map.of(); - } - String p = prefix == null ? "" : prefix; - Map out = new LinkedHashMap<>(); - extensionItems.forEach((id, bytes) -> decodeItem(out, p, id == null ? -1 : id, bytes)); - return Map.copyOf(out); - } - - public static Double mileageKm(Map extensionItems) { - if (extensionItems == null) { - return null; - } - byte[] bytes = extensionItems.get(0x01); - if (bytes == null || bytes.length != 4) { - return null; - } - return u32(bytes, 0) * 0.1; - } - - private static void decodeItem(Map out, String prefix, int id, byte[] bytes) { - if (bytes == null) { - return; - } - switch (id) { - case 0x01 -> put(out, prefix + "mileage_km", scaled(bytes, 4, 1)); - case 0x02 -> put(out, prefix + "fuel_l", scaled(bytes, 2, 1)); - case 0x03 -> put(out, prefix + "recorder_speed_kmh", scaled(bytes, 2, 1)); - case 0x04 -> put(out, prefix + "manual_alarm_event_id", unsigned(bytes, 2)); - case 0x05 -> decodeTirePressure(out, prefix, bytes); - case 0x06 -> put(out, prefix + "compartment_temp_c", signedHighBitWord(bytes)); - case 0x11 -> decodeOverspeed(out, prefix, bytes); - case 0x12 -> decodeAreaRouteAlarm(out, prefix, bytes); - case 0x13 -> decodeRouteTime(out, prefix, bytes); - case 0x25 -> decodeVehicleSignal(out, prefix, bytes); - case 0x2A -> decodeIoStatus(out, prefix, bytes); - case 0x2B -> decodeAnalog(out, prefix, bytes); - case 0x30 -> put(out, prefix + "network_signal_strength", unsigned(bytes, 1)); - case 0x31 -> put(out, prefix + "gnss_satellite_count", unsigned(bytes, 1)); - default -> { - } - } - } - - private static void decodeTirePressure(Map out, String prefix, byte[] bytes) { - if (bytes.length != 30) { - return; - } - List values = new ArrayList<>(); - List validIndexes = new ArrayList<>(); - List invalidIndexes = new ArrayList<>(); - for (int i = 0; i < bytes.length; i++) { - int value = u8(bytes, i); - if (value == 0xFF) { - invalidIndexes.add(i); - } else { - validIndexes.add(i); - values.add(Integer.toString(value)); - } - } - put(out, prefix + "tire_pressure_pa_values", String.join(",", values)); - put(out, prefix + "tire_pressure_valid_indexes", ranges(validIndexes)); - put(out, prefix + "tire_pressure_invalid_indexes", ranges(invalidIndexes)); - } - - private static void decodeOverspeed(Map out, String prefix, byte[] bytes) { - if (bytes.length != 1 && bytes.length != 5) { - return; - } - put(out, prefix + "overspeed.position_type", Integer.toString(u8(bytes, 0))); - if (bytes.length == 5) { - put(out, prefix + "overspeed.area_route_id", Long.toString(u32(bytes, 1))); - } - } - - private static void decodeAreaRouteAlarm(Map out, String prefix, byte[] bytes) { - if (bytes.length != 6) { - return; - } - put(out, prefix + "area_route_alarm.position_type", Integer.toString(u8(bytes, 0))); - put(out, prefix + "area_route_alarm.area_route_id", Long.toString(u32(bytes, 1))); - put(out, prefix + "area_route_alarm.direction", Integer.toString(u8(bytes, 5))); - } - - private static void decodeRouteTime(Map out, String prefix, byte[] bytes) { - if (bytes.length != 7) { - return; - } - put(out, prefix + "route_time.segment_id", Long.toString(u32(bytes, 0))); - put(out, prefix + "route_time.duration_seconds", Integer.toString(u16(bytes, 4))); - put(out, prefix + "route_time.result", Integer.toString(u8(bytes, 6))); - } - - private static void decodeVehicleSignal(Map out, String prefix, byte[] bytes) { - if (bytes.length != 4) { - return; - } - long raw = u32(bytes, 0); - put(out, prefix + "vehicle_signal_raw", Long.toString(raw)); - for (int i = 0; i < VEHICLE_SIGNAL_BITS.length; i++) { - put(out, prefix + "vehicle_signal." + VEHICLE_SIGNAL_BITS[i], Boolean.toString(bit(raw, i))); - } - } - - private static void decodeIoStatus(Map out, String prefix, byte[] bytes) { - if (bytes.length != 2) { - return; - } - int raw = u16(bytes, 0); - put(out, prefix + "io_status_raw", Integer.toString(raw)); - for (int i = 0; i < IO_STATUS_BITS.length; i++) { - put(out, prefix + "io_status." + IO_STATUS_BITS[i], Boolean.toString(bit(raw, i))); - } - } - - private static void decodeAnalog(Map out, String prefix, byte[] bytes) { - if (bytes.length != 4) { - return; - } - long raw = u32(bytes, 0); - put(out, prefix + "analog_raw", Long.toString(raw)); - put(out, prefix + "analog_ad0", Integer.toString((int) (raw & 0xFFFF))); - put(out, prefix + "analog_ad1", Integer.toString((int) ((raw >>> 16) & 0xFFFF))); - } - - private static String scaled(byte[] bytes, int expectedLength, int scale) { - if (bytes.length != expectedLength) { - return null; - } - long raw = expectedLength == 4 ? u32(bytes, 0) : u16(bytes, 0); - return BigDecimal.valueOf(raw, scale).stripTrailingZeros().toPlainString(); - } - - private static String unsigned(byte[] bytes, int expectedLength) { - if (bytes.length != expectedLength) { - return null; - } - return switch (expectedLength) { - case 1 -> Integer.toString(u8(bytes, 0)); - case 2 -> Integer.toString(u16(bytes, 0)); - case 4 -> Long.toString(u32(bytes, 0)); - default -> null; - }; - } - - private static String signedHighBitWord(byte[] bytes) { - if (bytes.length != 2) { - return null; - } - int raw = u16(bytes, 0); - int value = (raw & 0x8000) == 0 ? raw : -(raw & 0x7FFF); - return Integer.toString(value); - } - - private static int u8(byte[] bytes, int offset) { - return bytes[offset] & 0xFF; - } - - private static int u16(byte[] bytes, int offset) { - return (u8(bytes, offset) << 8) | u8(bytes, offset + 1); - } - - private static long u32(byte[] bytes, int offset) { - return ((long) u8(bytes, offset) << 24) - | ((long) u8(bytes, offset + 1) << 16) - | ((long) u8(bytes, offset + 2) << 8) - | u8(bytes, offset + 3); - } - - private static boolean bit(long raw, int bit) { - return ((raw >>> bit) & 1L) == 1L; - } - - private static void put(Map out, String key, String value) { - if (value != null) { - out.put(key, value); - } - } - - private static String ranges(List indexes) { - if (indexes.isEmpty()) { - return ""; - } - List ranges = new ArrayList<>(); - int start = indexes.getFirst(); - int previous = start; - for (int i = 1; i < indexes.size(); i++) { - int current = indexes.get(i); - if (current == previous + 1) { - previous = current; - continue; - } - ranges.add(formatRange(start, previous)); - start = current; - previous = current; - } - ranges.add(formatRange(start, previous)); - return String.join(",", ranges); - } - - private static String formatRange(int start, int end) { - return start == end ? Integer.toString(start) : start + "-" + end; - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808LocationFlagInfo.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808LocationFlagInfo.java deleted file mode 100644 index ea964d78..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808LocationFlagInfo.java +++ /dev/null @@ -1,99 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.model; - -import java.util.LinkedHashMap; -import java.util.Map; - -public final class Jt808LocationFlagInfo { - - private static final String[] ALARM_BITS = { - "emergency", - "overspeed", - "fatigue_driving", - "warning", - "gnss_module_fault", - "gnss_antenna_disconnected", - "gnss_antenna_short_circuit", - "terminal_main_power_undervoltage", - "terminal_main_power_lost", - "display_fault", - "tts_fault", - "camera_fault", - "transport_ic_card_fault", - "overspeed_warning", - "fatigue_driving_warning", - "prohibited_driving", - null, - null, - "daily_driving_timeout", - "overtime_parking", - "area_in_out", - "route_in_out", - "route_time_abnormal", - "route_deviation", - "vss_fault", - "fuel_abnormal", - "stolen", - "illegal_ignition", - "illegal_displacement", - "collision_rollover", - "rollover_warning" - }; - - private Jt808LocationFlagInfo() { - } - - public static Map decodeAlarm(long alarmFlag, String prefix) { - Map out = new LinkedHashMap<>(); - String p = prefix == null ? "" : prefix; - for (int i = 0; i < ALARM_BITS.length; i++) { - String name = ALARM_BITS[i]; - if (name != null) { - out.put(p + name, Boolean.toString(bit(alarmFlag, i))); - } - } - return Map.copyOf(out); - } - - public static Map decodeStatus(long statusFlag, String prefix) { - Map out = new LinkedHashMap<>(); - String p = prefix == null ? "" : prefix; - out.put(p + "acc_on", Boolean.toString(bit(statusFlag, 0))); - out.put(p + "positioned", Boolean.toString(bit(statusFlag, 1))); - out.put(p + "south_latitude", Boolean.toString(bit(statusFlag, 2))); - out.put(p + "west_longitude", Boolean.toString(bit(statusFlag, 3))); - out.put(p + "out_of_service", Boolean.toString(bit(statusFlag, 4))); - out.put(p + "encrypted", Boolean.toString(bit(statusFlag, 5))); - out.put(p + "forward_collision_warning", Boolean.toString(bit(statusFlag, 6))); - out.put(p + "lane_departure_warning", Boolean.toString(bit(statusFlag, 7))); - out.put(p + "load_status", loadStatus(statusFlag)); - out.put(p + "fuel_circuit_cut", Boolean.toString(bit(statusFlag, 10))); - out.put(p + "electric_circuit_cut", Boolean.toString(bit(statusFlag, 11))); - out.put(p + "door_locked", Boolean.toString(bit(statusFlag, 12))); - out.put(p + "door1_open", Boolean.toString(bit(statusFlag, 13))); - out.put(p + "door2_open", Boolean.toString(bit(statusFlag, 14))); - out.put(p + "door3_open", Boolean.toString(bit(statusFlag, 15))); - out.put(p + "door4_open", Boolean.toString(bit(statusFlag, 16))); - out.put(p + "door5_open", Boolean.toString(bit(statusFlag, 17))); - out.put(p + "gps_used", Boolean.toString(bit(statusFlag, 18))); - out.put(p + "beidou_used", Boolean.toString(bit(statusFlag, 19))); - out.put(p + "glonass_used", Boolean.toString(bit(statusFlag, 20))); - out.put(p + "galileo_used", Boolean.toString(bit(statusFlag, 21))); - out.put(p + "moving", Boolean.toString(bit(statusFlag, 22))); - return Map.copyOf(out); - } - - private static String loadStatus(long statusFlag) { - int code = (int) ((statusFlag >>> 8) & 0x03); - return switch (code) { - case 0 -> "EMPTY"; - case 1 -> "HALF_LOADED"; - case 2 -> "RESERVED"; - case 3 -> "FULL"; - default -> "UNKNOWN"; - }; - } - - private static boolean bit(long raw, int bit) { - return ((raw >>> bit) & 1L) == 1L; - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808Message.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808Message.java deleted file mode 100644 index 61a60bb2..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808Message.java +++ /dev/null @@ -1,6 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.model; - -/** - * 完整解析后的 JT808 消息。 - */ -public record Jt808Message(Jt808Header header, Jt808Body body) {} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808MessageId.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808MessageId.java deleted file mode 100644 index 358b020e..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/model/Jt808MessageId.java +++ /dev/null @@ -1,91 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.model; - -/** - * JT/T 808 消息 ID 常量集（节选核心消息，按需继续扩展）。 - * - *

    使用常量类而非 enum：808 消息种类多（超过 80 个），新增高频，常量类比 enum 维护成本更低。 - */ -public final class Jt808MessageId { - - private Jt808MessageId() {} - - // ===== 终端上行 ===== - public static final int TERMINAL_GENERAL_RESPONSE = 0x0001; - public static final int TERMINAL_HEARTBEAT = 0x0002; - public static final int TERMINAL_UNREGISTER = 0x0003; - public static final int TERMINAL_REGISTER = 0x0100; - public static final int TERMINAL_AUTH = 0x0102; - public static final int TERMINAL_PARAMS_REPORT = 0x0104; - public static final int TERMINAL_ATTRS_REPORT = 0x0107; - public static final int TERMINAL_LOCATION = 0x0200; - public static final int TERMINAL_LOCATION_BATCH = 0x0704; - public static final int TERMINAL_EVENT_REPORT = 0x0301; - public static final int TERMINAL_CAN_DATA_REPORT = 0x0705; - public static final int TERMINAL_MEDIA_EVENT = 0x0800; - public static final int TERMINAL_MEDIA_UPLOAD = 0x0801; - public static final int TERMINAL_PASSTHROUGH = 0x0900; - - // ===== 平台下行 ===== - public static final int PLATFORM_GENERAL_RESPONSE = 0x8001; - public static final int PLATFORM_REGISTER_ACK = 0x8100; - public static final int PLATFORM_SET_PARAMS = 0x8103; - public static final int PLATFORM_QUERY_PARAMS = 0x8104; - public static final int PLATFORM_TERMINAL_CONTROL = 0x8105; - public static final int PLATFORM_QUERY_ATTRS = 0x8107; - public static final int PLATFORM_QUERY_LOCATION = 0x8201; - public static final int PLATFORM_TRACK_CONTROL = 0x8202; - public static final int PLATFORM_ALARM_ACK = 0x8203; - public static final int PLATFORM_DELETE_CIRCLE_AREA = 0x8601; - - public static String nameOf(int messageId) { - return switch (messageId) { - case TERMINAL_GENERAL_RESPONSE -> "TERMINAL_GENERAL_RESPONSE"; - case TERMINAL_HEARTBEAT -> "TERMINAL_HEARTBEAT"; - case TERMINAL_UNREGISTER -> "TERMINAL_UNREGISTER"; - case TERMINAL_REGISTER -> "TERMINAL_REGISTER"; - case TERMINAL_AUTH -> "TERMINAL_AUTH"; - case TERMINAL_PARAMS_REPORT -> "TERMINAL_PARAMS_REPORT"; - case TERMINAL_ATTRS_REPORT -> "TERMINAL_ATTRS_REPORT"; - case TERMINAL_LOCATION -> "TERMINAL_LOCATION"; - case TERMINAL_LOCATION_BATCH -> "TERMINAL_LOCATION_BATCH"; - case TERMINAL_EVENT_REPORT -> "TERMINAL_EVENT_REPORT"; - case TERMINAL_CAN_DATA_REPORT -> "TERMINAL_CAN_DATA_REPORT"; - case TERMINAL_MEDIA_EVENT -> "TERMINAL_MEDIA_EVENT"; - case TERMINAL_MEDIA_UPLOAD -> "TERMINAL_MEDIA_UPLOAD"; - case TERMINAL_PASSTHROUGH -> "TERMINAL_PASSTHROUGH"; - case PLATFORM_GENERAL_RESPONSE -> "PLATFORM_GENERAL_RESPONSE"; - case PLATFORM_REGISTER_ACK -> "PLATFORM_REGISTER_ACK"; - case PLATFORM_SET_PARAMS -> "PLATFORM_SET_PARAMS"; - case PLATFORM_QUERY_PARAMS -> "PLATFORM_QUERY_PARAMS"; - case PLATFORM_TERMINAL_CONTROL -> "PLATFORM_TERMINAL_CONTROL"; - case PLATFORM_QUERY_ATTRS -> "PLATFORM_QUERY_ATTRS"; - case PLATFORM_QUERY_LOCATION -> "PLATFORM_QUERY_LOCATION"; - case PLATFORM_TRACK_CONTROL -> "PLATFORM_TRACK_CONTROL"; - case PLATFORM_ALARM_ACK -> "PLATFORM_ALARM_ACK"; - case PLATFORM_DELETE_CIRCLE_AREA -> "PLATFORM_DELETE_CIRCLE_AREA"; - default -> "UNKNOWN_0x" + hex(messageId); - }; - } - - public static String reportTypeOf(int messageId) { - return switch (messageId) { - case TERMINAL_LOCATION -> "LOCATION"; - case TERMINAL_LOCATION_BATCH -> "LOCATION_BATCH"; - case TERMINAL_HEARTBEAT -> "HEARTBEAT"; - case TERMINAL_AUTH -> "AUTH"; - case TERMINAL_REGISTER -> "REGISTER"; - case TERMINAL_UNREGISTER -> "UNREGISTER"; - case TERMINAL_GENERAL_RESPONSE -> "TERMINAL_RESPONSE"; - case TERMINAL_PARAMS_REPORT -> "PARAMS_REPORT"; - case TERMINAL_ATTRS_REPORT -> "ATTRS_REPORT"; - case TERMINAL_MEDIA_EVENT -> "MEDIA_EVENT"; - case TERMINAL_MEDIA_UPLOAD -> "MEDIA_UPLOAD"; - case TERMINAL_PASSTHROUGH -> "PASSTHROUGH"; - default -> "RAW"; - }; - } - - public static String hex(int messageId) { - return String.format("%04X", messageId & 0xFFFF); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/session/Jt808ChannelRegistry.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/session/Jt808ChannelRegistry.java deleted file mode 100644 index 98130f7c..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/session/Jt808ChannelRegistry.java +++ /dev/null @@ -1,67 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.session; - -import io.netty.channel.Channel; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.util.Optional; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ConcurrentMap; -import java.util.concurrent.atomic.AtomicInteger; - -/** - * JT808 终端 Channel 注册表。按 {@code phone} 索引，一个 phone 对应一个活跃 Channel。 - * - *

    {@link Jt808CommandDispatcher} 通过本类查 Channel；{@code Jt808ChannelHandler} 在 - * 连接建立 / 断开 / 鉴权完成时维护映射。 - * - *

    线程安全：{@link ConcurrentHashMap}。 - */ -public final class Jt808ChannelRegistry { - - private static final Logger log = LoggerFactory.getLogger(Jt808ChannelRegistry.class); - - private final ConcurrentMap byPhone = new ConcurrentHashMap<>(); - private final ConcurrentMap reverse = new ConcurrentHashMap<>(); - private final ConcurrentMap serials = new ConcurrentHashMap<>(); - - public void bind(String phone, Channel channel) { - if (phone == null || phone.isBlank() || channel == null) return; - Channel old = byPhone.put(phone, channel); - reverse.put(channel, phone); - if (old != null && old != channel) { - reverse.remove(old); - // 同一 phone 新连接上线时关闭旧 channel，避免下行命令写到已被终端替换的连接。 - old.close(); - log.info("channel rebound phone={} oldChannel={} newChannel={}", phone, old.id(), channel.id()); - } - } - - public Optional find(String phone) { - if (phone == null) return Optional.empty(); - Channel c = byPhone.get(phone); - if (c == null || !c.isActive()) { - if (c != null) unbind(c); - return Optional.empty(); - } - return Optional.of(c); - } - - public Optional unbind(Channel channel) { - String phone = reverse.remove(channel); - if (phone != null) { - byPhone.remove(phone, channel); - } - return Optional.ofNullable(phone); - } - - public int nextSerial(String phone) { - // JT808 流水号 16 bit，无符号回绕到 0；pending 匹配也使用这个序号。 - return serials.computeIfAbsent(phone, k -> new AtomicInteger()) - .updateAndGet(i -> (i + 1) & 0xFFFF); - } - - public int size() { - return byPhone.size(); - } -} diff --git a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/session/Jt808PendingRequests.java b/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/session/Jt808PendingRequests.java deleted file mode 100644 index 8d03b852..00000000 --- a/modules/protocols/protocol-jt808/src/main/java/com/lingniu/ingest/protocol/jt808/session/Jt808PendingRequests.java +++ /dev/null @@ -1,52 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.session; - -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ConcurrentMap; - -/** - * 下行命令的请求/应答同步匹配表。 - * - *

    调用 {@link #await(String, int)} 时注册一个 {@link CompletableFuture}， - * 对应 key = {@code phone + ':' + serial}；当 {@code Jt808ChannelHandler} 收到 0x0001 通用应答 - * 或其它同步应答消息时调用 {@link #complete(String, int, Jt808Message)} 触发 future 完成。 - */ -public final class Jt808PendingRequests { - - private static final Logger log = LoggerFactory.getLogger(Jt808PendingRequests.class); - - private final ConcurrentMap> pending = new ConcurrentHashMap<>(); - - public CompletableFuture await(String phone, int serial) { - CompletableFuture cf = new CompletableFuture<>(); - // JT808 平台下行流水号按终端 phone 独立递增，因此 phone+serial 才能唯一定位一次请求。 - pending.put(key(phone, serial), cf); - return cf; - } - - public void complete(String phone, int serial, Jt808Message response) { - CompletableFuture cf = pending.remove(key(phone, serial)); - if (cf != null) { - cf.complete(response); - } else { - log.debug("no pending request for phone={} serial={}", phone, serial); - } - } - - public void cancel(String phone, int serial) { - CompletableFuture cf = pending.remove(key(phone, serial)); - if (cf != null) cf.cancel(true); - } - - public int size() { - return pending.size(); - } - - private static String key(String phone, int serial) { - return phone + ":" + serial; - } -} diff --git a/modules/protocols/protocol-jt808/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/protocols/protocol-jt808/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index 9af2ea7e..00000000 --- a/modules/protocols/protocol-jt808/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.protocol.jt808.config.Jt808AutoConfiguration diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/TestSessionStore.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/TestSessionStore.java deleted file mode 100644 index dc1be186..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/TestSessionStore.java +++ /dev/null @@ -1,74 +0,0 @@ -package com.lingniu.ingest.protocol.jt808; - -import com.lingniu.ingest.session.DeviceSession; -import com.lingniu.ingest.session.SessionStore; - -import java.util.Optional; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ConcurrentMap; -import java.util.function.UnaryOperator; - -public final class TestSessionStore implements SessionStore { - - private final ConcurrentMap byId = new ConcurrentHashMap<>(); - private final ConcurrentMap vinToId = new ConcurrentHashMap<>(); - private final ConcurrentMap phoneToId = new ConcurrentHashMap<>(); - - @Override - public void put(DeviceSession session) { - byId.put(session.sessionId(), session); - if (session.vin() != null) { - vinToId.put(session.vin(), session.sessionId()); - } - if (session.phone() != null) { - phoneToId.put(session.phone(), session.sessionId()); - } - } - - @Override - public Optional findBySessionId(String sessionId) { - return Optional.ofNullable(byId.get(sessionId)); - } - - @Override - public Optional findByVin(String vin) { - String sessionId = vinToId.get(vin); - return sessionId == null ? Optional.empty() : findBySessionId(sessionId); - } - - @Override - public Optional findByPhone(String phone) { - String sessionId = phoneToId.get(phone); - return sessionId == null ? Optional.empty() : findBySessionId(sessionId); - } - - @Override - public synchronized Optional update(String sessionId, UnaryOperator updater) { - DeviceSession current = byId.get(sessionId); - if (current == null) { - return Optional.empty(); - } - DeviceSession next = updater.apply(current); - put(next); - return Optional.of(next); - } - - @Override - public void remove(String sessionId) { - DeviceSession removed = byId.remove(sessionId); - if (removed == null) { - return; - } - if (removed.vin() != null) { - vinToId.remove(removed.vin(), sessionId); - } - if (removed.phone() != null) { - phoneToId.remove(removed.phone(), sessionId); - } - } - - @Override - public int size() { - return byId.size(); - } -} diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808DecoderGoldenTest.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808DecoderGoldenTest.java deleted file mode 100644 index 94c366fb..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808DecoderGoldenTest.java +++ /dev/null @@ -1,79 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -import com.lingniu.ingest.protocol.jt808.codec.parser.HeartbeatBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.LocationBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.RegisterBodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import org.junit.jupiter.api.DynamicTest; -import org.junit.jupiter.api.TestFactory; - -import java.io.IOException; -import java.net.URISyntaxException; -import java.nio.ByteBuffer; -import java.nio.file.Files; -import java.nio.file.Path; -import java.nio.file.Paths; -import java.util.Collection; -import java.util.List; -import java.util.stream.Collectors; -import java.util.stream.Stream; - -import static org.assertj.core.api.Assertions.assertThat; - -/** - * JT808 黄金样本集回放测试：遍历 {@code src/test/resources/samples/jt808/*.hex}。 - * - *

    样本文件格式：一行十六进制（完整 0x7e 包裹帧），脚本脱敏产出。 - * 测试会去掉首尾 0x7e，执行反转义后调用 {@link Jt808MessageDecoder}。 - */ -class Jt808DecoderGoldenTest { - - private final Jt808MessageDecoder decoder = new Jt808MessageDecoder( - new BodyParserRegistry(List.of( - new RegisterBodyParser(), - new LocationBodyParser(), - new HeartbeatBodyParser()))); - - @TestFactory - Collection replaySamples() throws URISyntaxException, IOException { - var url = getClass().getClassLoader().getResource("samples/jt808"); - if (url == null) return List.of(); - Path dir = Paths.get(url.toURI()); - try (Stream s = Files.list(dir)) { - return s.filter(p -> p.toString().endsWith(".hex")) - .sorted() - .map(this::toTest) - .collect(Collectors.toList()); - } - } - - private DynamicTest toTest(Path sample) { - return DynamicTest.dynamicTest(sample.getFileName().toString(), () -> { - byte[] framed = readHex(sample); - assertThat(framed[0]).as("must start with 0x7e").isEqualTo((byte) 0x7e); - assertThat(framed[framed.length - 1]).as("must end with 0x7e").isEqualTo((byte) 0x7e); - - byte[] unescaped = Jt808Escape.unescape(framed, 1, framed.length - 2); - Jt808Message msg = decoder.decode(ByteBuffer.wrap(unescaped)); - - assertThat(msg.header().messageId()).isNotNegative(); - assertThat(msg.header().phone()).isNotEmpty(); - }); - } - - private static byte[] readHex(Path path) throws IOException { - StringBuilder sb = new StringBuilder(); - for (String line : Files.readAllLines(path)) { - String trimmed = line.trim(); - if (trimmed.isEmpty() || trimmed.startsWith("#")) continue; - sb.append(trimmed); - } - String hex = sb.toString(); - int len = hex.length() / 2; - byte[] out = new byte[len]; - for (int i = 0; i < len; i++) { - out[i] = (byte) Integer.parseInt(hex.substring(i * 2, i * 2 + 2), 16); - } - return out; - } -} diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808DecoderTest.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808DecoderTest.java deleted file mode 100644 index 198430f9..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808DecoderTest.java +++ /dev/null @@ -1,274 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.codec.BcdCodec; -import com.lingniu.ingest.protocol.jt808.codec.parser.HeartbeatBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.AuthBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.LocationBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.LocationBatchBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.MediaEventBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.MediaUploadBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.PassthroughBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.TerminalAttrsBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.TerminalParamsReportBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.UnregisterBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.RegisterBodyParser; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808LocationAdditionalInfo; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayOutputStream; -import java.nio.ByteBuffer; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt808DecoderTest { - - private final Jt808MessageDecoder decoder = new Jt808MessageDecoder( - new BodyParserRegistry(List.of( - new RegisterBodyParser(), - new AuthBodyParser(), - new LocationBodyParser(), - new LocationBatchBodyParser(), - new UnregisterBodyParser(), - new TerminalParamsReportBodyParser(), - new TerminalAttrsBodyParser(), - new MediaEventBodyParser(), - new MediaUploadBodyParser(), - new PassthroughBodyParser(), - new HeartbeatBodyParser()))); - - @Test - void decodesSyntheticLocationFrame() { - byte[] body = buildLocationBody(); - byte[] frame = buildFrame(Jt808MessageId.TERMINAL_LOCATION, "123456789012", 0x0001, body); - Jt808Message msg = decoder.decode(ByteBuffer.wrap(frame)); - - assertThat(msg.header().messageId()).isEqualTo(Jt808MessageId.TERMINAL_LOCATION); - assertThat(msg.header().phone()).isEqualTo("123456789012"); - assertThat(msg.header().serialNo()).isEqualTo(1); - - assertThat(msg.body()).isInstanceOf(Jt808Body.Location.class); - Jt808Body.Location loc = (Jt808Body.Location) msg.body(); - assertThat(loc.longitude()).isEqualTo(116.397128, org.assertj.core.data.Offset.offset(0.000001)); - assertThat(loc.latitude()).isEqualTo(39.916527, org.assertj.core.data.Offset.offset(0.000001)); - assertThat(loc.speedKmh()).isEqualTo(52.3, org.assertj.core.data.Offset.offset(0.01)); - } - - @Test - void decodesLocationExtensionItems() { - byte[] body = bytes(os -> { - os.write(buildLocationBody(), 0, buildLocationBody().length); - os.write(0x01); // mileage, DWORD, 0.1 km - os.write(4); - writeU32(os, 12345); - os.write(0x30); // wireless signal strength - os.write(1); - os.write(88); - }); - - Jt808Body.Location loc = (Jt808Body.Location) decoder.decode(ByteBuffer.wrap(buildFrame( - Jt808MessageId.TERMINAL_LOCATION, "123456789012", 9, body))).body(); - - assertThat(loc.extensionItems()).containsOnlyKeys(0x01, 0x30); - assertThat(loc.extensionItems().get(0x01)).containsExactly(0x00, 0x00, 0x30, 0x39); - assertThat(loc.extensionItems().get(0x30)).containsExactly(88); - } - - @Test - void decodesForwardedLocationFrameGpsMileageExtension() { - byte[] framed = hex(""" - 7E020000320133077954250001000000000048000301D2C4C707376139000A00E6004F260630162357 - 01040001900C2504000000000202000030011F31010F867E - """); - byte[] unescaped = Jt808Escape.unescape(framed, 1, framed.length - 2); - - Jt808Body.Location loc = (Jt808Body.Location) decoder.decode(ByteBuffer.wrap(unescaped)).body(); - - assertThat(loc.longitude()).isEqualTo(121.069881); - assertThat(loc.latitude()).isEqualTo(30.590151); - assertThat(loc.speedKmh()).isEqualTo(23.0); - assertThat(loc.deviceTime().toString()).isEqualTo("2026-06-30T08:23:57Z"); - assertThat(loc.extensionItems()).containsOnlyKeys(0x01, 0x25, 0x02, 0x30, 0x31); - assertThat(loc.extensionItems().get(0x01)).containsExactly(0x00, 0x01, (byte) 0x90, 0x0C); - assertThat(Jt808LocationAdditionalInfo.mileageKm(loc.extensionItems())).isEqualTo(10241.2); - assertThat(Jt808LocationAdditionalInfo.decode(loc.extensionItems())) - .containsEntry("jt808.extra.mileage_km", "10241.2") - .containsEntry("jt808.extra.vehicle_signal_raw", "0") - .containsEntry("jt808.extra.fuel_l", "0") - .containsEntry("jt808.extra.network_signal_strength", "31") - .containsEntry("jt808.extra.gnss_satellite_count", "15"); - } - - @Test - void decodesSyntheticHeartbeatFrame() { - byte[] frame = buildFrame(Jt808MessageId.TERMINAL_HEARTBEAT, "123456789012", 0x0002, new byte[0]); - Jt808Message msg = decoder.decode(ByteBuffer.wrap(frame)); - assertThat(msg.header().messageId()).isEqualTo(Jt808MessageId.TERMINAL_HEARTBEAT); - assertThat(msg.body()).isInstanceOf(Jt808Body.Heartbeat.class); - } - - @Test - void decodesAuthFrameAndExtractsImeiWhenPresent() { - byte[] authBody = "AUTH-CODE,123456789012345,SW-1".getBytes(java.nio.charset.StandardCharsets.US_ASCII); - - Jt808Body.Auth auth = (Jt808Body.Auth) decoder.decode(ByteBuffer.wrap(buildFrame( - Jt808MessageId.TERMINAL_AUTH, "123456789012", 10, authBody))).body(); - - assertThat(auth.token()).isEqualTo("AUTH-CODE,123456789012345,SW-1"); - assertThat(auth.imei()).isEqualTo("123456789012345"); - assertThat(auth.softwareVersion()).isEqualTo("SW-1"); - } - - @Test - void decodesLogoutParamsAttrsMediaAndPassthroughFrames() { - assertThat(decoder.decode(ByteBuffer.wrap(buildFrame( - Jt808MessageId.TERMINAL_UNREGISTER, "123456789012", 3, new byte[0]))).body()) - .isInstanceOf(Jt808Body.Unregister.class); - - byte[] paramsBody = bytes(os -> { - writeU16(os, 7); - os.write(1); - writeU32(os, 0x00000080L); - os.write(1); - os.write(5); - }); - Jt808Body.ParamsReport params = (Jt808Body.ParamsReport) decoder.decode(ByteBuffer.wrap(buildFrame( - Jt808MessageId.TERMINAL_PARAMS_REPORT, "123456789012", 4, paramsBody))).body(); - assertThat(params.responseSerialNo()).isEqualTo(7); - assertThat(params.parameters()).containsKey(0x00000080L); - - byte[] attrsBody = bytes(os -> { - writeU16(os, 0x1234); - writeAscii(os, "MAKER", 5); - writeAscii(os, "MODEL-X", 20); - writeAscii(os, "DEV1234", 7); - os.write(BcdCodec.encode("89860012345678901234"), 0, 10); - writeAscii(os, "HW1", 10); - writeAscii(os, "SW1", 10); - os.write(1); - os.write(2); - }); - Jt808Body.TerminalAttrs attrs = (Jt808Body.TerminalAttrs) decoder.decode(ByteBuffer.wrap(buildFrame( - Jt808MessageId.TERMINAL_ATTRS_REPORT, "123456789012", 5, attrsBody))).body(); - assertThat(attrs.maker()).isEqualTo("MAKER"); - assertThat(attrs.terminalId()).isEqualTo("DEV1234"); - - byte[] mediaEventBody = bytes(os -> { - writeU32(os, 42); - os.write(0); - os.write(1); - os.write(2); - os.write(3); - }); - assertThat(decoder.decode(ByteBuffer.wrap(buildFrame( - Jt808MessageId.TERMINAL_MEDIA_EVENT, "123456789012", 6, mediaEventBody))).body()) - .isInstanceOf(Jt808Body.MediaEvent.class); - - byte[] passthroughBody = bytes(os -> { - os.write(0x41); - os.write(new byte[]{0x01, 0x02, 0x03}, 0, 3); - }); - Jt808Body.Passthrough passthrough = (Jt808Body.Passthrough) decoder.decode(ByteBuffer.wrap(buildFrame( - Jt808MessageId.TERMINAL_PASSTHROUGH, "123456789012", 7, passthroughBody))).body(); - assertThat(passthrough.passthroughType()).isEqualTo(0x41); - assertThat(passthrough.data()).containsExactly(0x01, 0x02, 0x03); - } - - @Test - void decodesLocationBatchFrame() { - byte[] location = buildLocationBody(); - byte[] batchBody = bytes(os -> { - writeU16(os, 2); - os.write(0); - writeU16(os, location.length); - os.write(location, 0, location.length); - writeU16(os, location.length); - os.write(location, 0, location.length); - }); - - Jt808Body.LocationBatch batch = (Jt808Body.LocationBatch) decoder.decode(ByteBuffer.wrap(buildFrame( - Jt808MessageId.TERMINAL_LOCATION_BATCH, "123456789012", 8, batchBody))).body(); - - assertThat(batch.batchType()).isEqualTo(0); - assertThat(batch.locations()).hasSize(2); - assertThat(batch.locations().getFirst().longitude()).isEqualTo(116.397128); - } - - // ===== helpers ===== - - private static byte[] buildLocationBody() { - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeU32(os, 0); // alarmFlag - writeU32(os, 0x00030000); // statusFlag - writeU32(os, 39_916_527L); // lat - writeU32(os, 116_397_128L); // lon - writeU16(os, 50); // altitude - writeU16(os, 523); // speed 52.3 - writeU16(os, 90); // direction - byte[] bcd = BcdCodec.encode("240102030405"); - os.write(bcd, 0, 6); - return os.toByteArray(); - } - - private static byte[] bytes(IoConsumer writer) { - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writer.accept(os); - return os.toByteArray(); - } - - private static void writeAscii(ByteArrayOutputStream os, String value, int len) { - byte[] out = new byte[len]; - byte[] raw = value.getBytes(java.nio.charset.StandardCharsets.US_ASCII); - System.arraycopy(raw, 0, out, 0, Math.min(raw.length, out.length)); - os.write(out, 0, out.length); - } - - @FunctionalInterface - private interface IoConsumer { - void accept(T value); - } - - private static byte[] buildFrame(int msgId, String phone, int serial, byte[] body) { - // header: msgId(2) attrs(2) phone(6 BCD) serial(2) - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeU16(os, msgId); - int attrs = body.length & 0x03FF; // 2013 版，无子包、无加密 - writeU16(os, attrs); - byte[] phoneBcd = BcdCodec.encode(phone); - os.write(phoneBcd, 0, 6); - writeU16(os, serial); - os.write(body, 0, body.length); - byte[] raw = os.toByteArray(); - byte bcc = BccChecksum.compute(raw, 0, raw.length); - - ByteArrayOutputStream framed = new ByteArrayOutputStream(); - framed.write(raw, 0, raw.length); - framed.write(bcc & 0xFF); - return framed.toByteArray(); - } - - private static byte[] hex(String value) { - String compact = value.replaceAll("\\s+", ""); - byte[] out = new byte[compact.length() / 2]; - for (int i = 0; i < out.length; i++) { - out[i] = (byte) Integer.parseInt(compact.substring(i * 2, i * 2 + 2), 16); - } - return out; - } - - private static void writeU16(ByteArrayOutputStream os, int v) { - os.write((v >> 8) & 0xFF); - os.write(v & 0xFF); - } - - private static void writeU32(ByteArrayOutputStream os, long v) { - os.write((int) ((v >> 24) & 0xFF)); - os.write((int) ((v >> 16) & 0xFF)); - os.write((int) ((v >> 8) & 0xFF)); - os.write((int) (v & 0xFF)); - } -} diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808EscapeTest.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808EscapeTest.java deleted file mode 100644 index 677e8276..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808EscapeTest.java +++ /dev/null @@ -1,25 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt808EscapeTest { - - @Test - void roundTrip() { - byte[] raw = {0x30, 0x7e, 0x08, 0x7d, (byte) 0xFF}; - byte[] escaped = Jt808Escape.escape(raw); - assertThat(escaped).containsExactly(0x30, 0x7d, 0x02, 0x08, 0x7d, 0x01, 0xFF); - - byte[] back = Jt808Escape.unescape(escaped, 0, escaped.length); - assertThat(back).containsExactly(raw[0], raw[1], raw[2], raw[3], raw[4]); - } - - @Test - void noEscapeNeeded() { - byte[] raw = {0x01, 0x02, 0x03}; - assertThat(Jt808Escape.escape(raw)).containsExactly(0x01, 0x02, 0x03); - assertThat(Jt808Escape.unescape(raw, 0, raw.length)).containsExactly(0x01, 0x02, 0x03); - } -} diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameDecoderTest.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameDecoderTest.java deleted file mode 100644 index 8fcc2d0c..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameDecoderTest.java +++ /dev/null @@ -1,39 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -import io.netty.buffer.Unpooled; -import io.netty.channel.embedded.EmbeddedChannel; -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt808FrameDecoderTest { - - @Test - void garbageWithoutFrameBoundaryEmitsMalformedCandidate() { - EmbeddedChannel channel = new EmbeddedChannel(new Jt808FrameDecoder()); - - channel.writeInbound(Unpooled.wrappedBuffer(new byte[]{0x01, 0x02, 0x03})); - - Jt808MalformedFrame malformed = channel.readInbound(); - assertThat(malformed.rawBytes()).containsExactly(0x01, 0x02, 0x03); - assertThat(malformed.reason()).contains("missing frame boundary"); - assertThat(malformed.peer()).isNotNull(); - assertThat((Object) channel.readInbound()).isNull(); - } - - @Test - void oversizedUnclosedFrameEmitsMalformedCandidate() { - byte[] bytes = new byte[16 * 1024 + 2]; - bytes[0] = 0x7e; - bytes[1] = 0x01; - EmbeddedChannel channel = new EmbeddedChannel(new Jt808FrameDecoder()); - - channel.writeInbound(Unpooled.wrappedBuffer(bytes)); - - Jt808MalformedFrame malformed = channel.readInbound(); - assertThat(malformed.rawBytes()).containsExactly(bytes); - assertThat(malformed.reason()).contains("too large"); - assertThat(malformed.peer()).isNotNull(); - assertThat((Object) channel.readInbound()).isNull(); - } -} diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameEncoderTest.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameEncoderTest.java deleted file mode 100644 index a29d9e46..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/codec/Jt808FrameEncoderTest.java +++ /dev/null @@ -1,91 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.codec; - -import com.lingniu.ingest.protocol.jt808.codec.parser.HeartbeatBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.LocationBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.RegisterBodyParser; -import com.lingniu.ingest.protocol.jt808.downlink.Jt808Commands; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; -import org.junit.jupiter.api.Test; - -import java.nio.ByteBuffer; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -/** - * 端到端验证：encoder 生成的下行帧可以被 decoder 反向解析，且字段对称。 - */ -class Jt808FrameEncoderTest { - - private final Jt808MessageDecoder decoder = new Jt808MessageDecoder( - new BodyParserRegistry(List.of( - new RegisterBodyParser(), - new LocationBodyParser(), - new HeartbeatBodyParser()))); - - @Test - void encodeThenDecodePlatformAck() { - var cmd = Jt808Commands.platformAck(0x1234, 0x0200, 0); - byte[] framed = Jt808FrameEncoder.encode(cmd.messageId(), "123456789012", 7, cmd.body()); - - // strip outer 0x7e and unescape - byte[] unescaped = Jt808Escape.unescape(framed, 1, framed.length - 2); - Jt808Message msg = decoder.decode(ByteBuffer.wrap(unescaped)); - - assertThat(msg.header().messageId()).isEqualTo(Jt808MessageId.PLATFORM_GENERAL_RESPONSE); - assertThat(msg.header().phone()).isEqualTo("123456789012"); - assertThat(msg.header().serialNo()).isEqualTo(7); - assertThat(msg.body()).isInstanceOf(Jt808Body.Raw.class); - Jt808Body.Raw raw = (Jt808Body.Raw) msg.body(); - // body: ackSerial(2) + ackMsgId(2) + result(1) = 5 bytes - assertThat(raw.bytes()).hasSize(5); - } - - @Test - void encodeQueryLocationProducesFramedWith7eBoundary() { - byte[] framed = Jt808FrameEncoder.encode( - Jt808MessageId.PLATFORM_QUERY_LOCATION, "123456789012", 1, new byte[0]); - assertThat(framed[0]).isEqualTo((byte) 0x7e); - assertThat(framed[framed.length - 1]).isEqualTo((byte) 0x7e); - // 中间不应出现裸 0x7e - for (int i = 1; i < framed.length - 1; i++) { - assertThat(framed[i]).isNotEqualTo((byte) 0x7e); - } - } - - @Test - void rejectsOversizedBodyWhenSubpackageIsNotUsed() { - assertThatThrownBy(() -> Jt808FrameEncoder.encode( - Jt808MessageId.PLATFORM_SET_PARAMS, "123456789012", 1, new byte[1024])) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("jt808 body too large"); - } - - @Test - void encodesOversizedBodyAsSubpackages() { - byte[] body = new byte[2048]; - for (int i = 0; i < body.length; i++) { - body[i] = (byte) i; - } - - List frames = Jt808FrameEncoder.encodeSubpackages( - Jt808MessageId.PLATFORM_SET_PARAMS, "123456789012", 9, body); - - assertThat(frames).hasSize(3); - java.io.ByteArrayOutputStream joined = new java.io.ByteArrayOutputStream(); - for (int i = 0; i < frames.size(); i++) { - byte[] unescaped = Jt808Escape.unescape(frames.get(i), 1, frames.get(i).length - 2); - Jt808Message msg = decoder.decode(ByteBuffer.wrap(unescaped)); - assertThat(msg.header().subpacket()).isTrue(); - assertThat(msg.header().totalPackets()).isEqualTo(3); - assertThat(msg.header().packetSeq()).isEqualTo(i + 1); - assertThat(msg.header().serialNo()).isEqualTo(9); - assertThat(msg.body()).isInstanceOf(Jt808Body.Raw.class); - joined.writeBytes(((Jt808Body.Raw) msg.body()).bytes()); - } - assertThat(joined.toByteArray()).containsExactly(body); - } -} diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/config/Jt808PropertiesTest.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/config/Jt808PropertiesTest.java deleted file mode 100644 index 0a77d6e4..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/config/Jt808PropertiesTest.java +++ /dev/null @@ -1,34 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.config; - -import org.junit.jupiter.api.Test; - -import java.io.IOException; -import java.nio.file.Files; -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt808PropertiesTest { - - @Test - void defaultsToCurrentProductionReceivePort() throws IOException { - assertThat(new Jt808Properties().getPort()).isEqualTo(808); - - String pom = Files.readString(repositoryRoot() - .resolve("modules/protocols/protocol-jt808/pom.xml")); - assertThat(pom) - .contains("默认 TCP 808") - .doesNotContain("10808"); - } - - private static Path repositoryRoot() { - Path current = Path.of(System.getProperty("user.dir")).toAbsolutePath(); - while (current != null) { - if (Files.exists(current.resolve("pom.xml")) && Files.exists(current.resolve("modules"))) { - return current; - } - current = current.getParent(); - } - throw new IllegalStateException("repository root not found"); - } -} diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808CommandDispatcherTest.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808CommandDispatcherTest.java deleted file mode 100644 index 94a92435..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808CommandDispatcherTest.java +++ /dev/null @@ -1,56 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.downlink; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.protocol.jt808.codec.BodyParserRegistry; -import com.lingniu.ingest.protocol.jt808.codec.Jt808Escape; -import com.lingniu.ingest.protocol.jt808.codec.Jt808MessageDecoder; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; -import com.lingniu.ingest.protocol.jt808.session.Jt808ChannelRegistry; -import com.lingniu.ingest.protocol.jt808.session.Jt808PendingRequests; -import com.lingniu.ingest.session.DeviceSession; -import com.lingniu.ingest.protocol.jt808.TestSessionStore; -import io.netty.buffer.ByteBuf; -import io.netty.channel.embedded.EmbeddedChannel; -import org.junit.jupiter.api.Test; - -import java.nio.ByteBuffer; -import java.time.Instant; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt808CommandDispatcherTest { - - @Test - void notifyWritesSubpackageFramesForLargeBody() { - TestSessionStore sessions = new TestSessionStore(); - sessions.put(new DeviceSession("sid-1", ProtocolId.JT808, "LNVIN000000000303", - "123456789012", "", "127.0.0.1:10000", Instant.now(), Instant.now(), Map.of())); - Jt808ChannelRegistry channels = new Jt808ChannelRegistry(); - EmbeddedChannel channel = new EmbeddedChannel(); - channels.bind("123456789012", channel); - Jt808CommandDispatcher dispatcher = new Jt808CommandDispatcher( - sessions, channels, new Jt808PendingRequests()); - byte[] body = new byte[2048]; - - dispatcher.notify("sid-1", new Jt808Commands.DownlinkCommand( - Jt808MessageId.PLATFORM_SET_PARAMS, body)).join(); - - Jt808MessageDecoder decoder = new Jt808MessageDecoder(new BodyParserRegistry(List.of())); - for (int i = 1; i <= 3; i++) { - ByteBuf outbound = channel.readOutbound(); - byte[] frame = new byte[outbound.readableBytes()]; - outbound.readBytes(frame); - byte[] unescaped = Jt808Escape.unescape(frame, 1, frame.length - 2); - Jt808Message decoded = decoder.decode(ByteBuffer.wrap(unescaped)); - assertThat(decoded.header().subpacket()).isTrue(); - assertThat(decoded.header().totalPackets()).isEqualTo(3); - assertThat(decoded.header().packetSeq()).isEqualTo(i); - assertThat(decoded.body()).isInstanceOf(Jt808Body.Raw.class); - } - assertThat((Object) channel.readOutbound()).isNull(); - } -} diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808CommandsTest.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808CommandsTest.java deleted file mode 100644 index f5a5dc6f..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/downlink/Jt808CommandsTest.java +++ /dev/null @@ -1,21 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.downlink; - -import org.junit.jupiter.api.Test; - -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt808CommandsTest { - - @Test - void deleteAreaEncodesCountAndDwordAreaIds() { - var command = Jt808Commands.deleteArea(List.of(1L, 0x01020304L)); - - assertThat(command.messageId()).isEqualTo(0x8601); - assertThat(command.body()).containsExactly( - 0x02, - 0x00, 0x00, 0x00, 0x01, - 0x01, 0x02, 0x03, 0x04); - } -} diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/handler/Jt808LocationHandlerTest.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/handler/Jt808LocationHandlerTest.java deleted file mode 100644 index 03c05fd9..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/handler/Jt808LocationHandlerTest.java +++ /dev/null @@ -1,48 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.handler; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.pipeline.IngestContext; -import com.lingniu.ingest.api.pipeline.RawFrame; -import com.lingniu.ingest.core.dispatcher.AnnotationHandlerBeanPostProcessor; -import com.lingniu.ingest.core.dispatcher.HandlerInvoker; -import com.lingniu.ingest.core.dispatcher.HandlerRegistry; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.protocol.jt808.mapper.Jt808EventMapper; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt808LocationHandlerTest { - - @Test - void unknownRawMessageRoutesToPassthroughHandler() { - Jt808LocationHandler handler = new Jt808LocationHandler( - new Jt808EventMapper(new InMemoryVehicleIdentityService())); - HandlerRegistry registry = new HandlerRegistry(); - new AnnotationHandlerBeanPostProcessor(registry).postProcessAfterInitialization(handler, "jt808Handler"); - Jt808Message message = new Jt808Message( - new Jt808Header(0x0F01, 3, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 7, 0, 0), - new Jt808Body.Raw(0x0F01, new byte[]{0x01, 0x02, 0x03})); - - var definitions = registry.resolve(ProtocolId.JT808, 0x0F01, 0); - - assertThat(definitions).hasSize(1); - var events = new HandlerInvoker().invoke(definitions.getFirst(), new RawFrame( - ProtocolId.JT808, 0x0F01, 0, message, new byte[]{0x7e}, - Map.of("phone", "123456789012"), Instant.now()), new IngestContext("trace-1")); - assertThat(events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.passthroughType()).isEqualTo(0x0F01); - assertThat(passthrough.metadata()).containsEntry("rawBody", "true"); - }); - } -} diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/inbound/Jt808ChannelHandlerTest.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/inbound/Jt808ChannelHandlerTest.java deleted file mode 100644 index 7f29f78b..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/inbound/Jt808ChannelHandlerTest.java +++ /dev/null @@ -1,857 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.inbound; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.sink.EventSink; -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.codec.BcdCodec; -import com.lingniu.ingest.core.concurrency.AsyncBatchExecutor; -import com.lingniu.ingest.core.concurrency.DisruptorEventBus; -import com.lingniu.ingest.core.dispatcher.Dispatcher; -import com.lingniu.ingest.core.dispatcher.HandlerDefinition; -import com.lingniu.ingest.core.dispatcher.HandlerInvoker; -import com.lingniu.ingest.core.dispatcher.HandlerRegistry; -import com.lingniu.ingest.core.pipeline.InterceptorChain; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.identity.VehicleIdentityBinding; -import com.lingniu.ingest.identity.VehicleIdentityLookup; -import com.lingniu.ingest.identity.VehicleIdentitySource; -import com.lingniu.ingest.protocol.jt808.codec.BodyParserRegistry; -import com.lingniu.ingest.protocol.jt808.codec.Jt808MalformedFrame; -import com.lingniu.ingest.protocol.jt808.codec.Jt808MessageDecoder; -import com.lingniu.ingest.protocol.jt808.codec.parser.AuthBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.LocationBodyParser; -import com.lingniu.ingest.protocol.jt808.codec.parser.RegisterBodyParser; -import com.lingniu.ingest.protocol.jt808.handler.Jt808LocationHandler; -import com.lingniu.ingest.protocol.jt808.mapper.Jt808EventMapper; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; -import com.lingniu.ingest.protocol.jt808.session.Jt808ChannelRegistry; -import com.lingniu.ingest.protocol.jt808.session.Jt808PendingRequests; -import com.lingniu.ingest.protocol.jt808.TestSessionStore; -import io.netty.buffer.ByteBuf; -import io.netty.channel.embedded.EmbeddedChannel; -import org.junit.jupiter.api.Test; - -import java.io.ByteArrayOutputStream; -import java.util.List; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.CopyOnWriteArrayList; -import java.util.concurrent.CountDownLatch; -import java.util.concurrent.TimeUnit; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.within; - -class Jt808ChannelHandlerTest { - - @Test - void registerSessionUsesResolvedInternalVinInsteadOfDeviceIdPlaceholder() { - TestSessionStore sessions = new TestSessionStore(); - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - identity.bind(new VehicleIdentityBinding( - ProtocolId.JT808, "LNVIN000000000808", "123456789012", "DEV808", "B80808")); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(new ImmediateKafkaSink())); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - new HandlerRegistry(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new RegisterBodyParser()))), - dispatcher, - sessions, - identity, - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - - channel.writeInbound(buildFrame( - Jt808MessageId.TERMINAL_REGISTER, - "123456789012", - 1, - buildRegisterBody("DEV808", "B80808"))); - - assertThat(sessions.findByPhone("123456789012")) - .get() - .extracting(session -> session.vin()) - .isEqualTo("LNVIN000000000808"); - assertThat(identity.resolve(new com.lingniu.ingest.identity.VehicleIdentityLookup( - ProtocolId.JT808, "", "123456789012", "", "")).vin()) - .isEqualTo("LNVIN000000000808"); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void unresolvedRegisterBindsGeneratedInternalVinToExternalIdentifiers() { - TestSessionStore sessions = new TestSessionStore(); - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(new ImmediateKafkaSink())); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - new HandlerRegistry(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new RegisterBodyParser()))), - dispatcher, - sessions, - identity, - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - - channel.writeInbound(buildFrame( - Jt808MessageId.TERMINAL_REGISTER, - "123456789012", - 1, - buildRegisterBody("DEV808", "B80808"))); - - assertThat(identity.resolve(new VehicleIdentityLookup( - ProtocolId.JT808, "", "123456789012", "", ""))) - .satisfies(resolved -> { - assertThat(resolved.vin()).startsWith("JT808"); - assertThat(resolved.resolved()).isTrue(); - assertThat(resolved.source()).isEqualTo(VehicleIdentitySource.BOUND_PHONE); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void authSessionUsesResolvedInternalVinFromImei() { - TestSessionStore sessions = new TestSessionStore(); - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - identity.bind(new VehicleIdentityBinding( - ProtocolId.JT808, "LNVIN000000AUTH01", "123456789012", "123456789012345", "")); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(new ImmediateKafkaSink())); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - new HandlerRegistry(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new AuthBodyParser()))), - dispatcher, - sessions, - identity, - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - - channel.writeInbound(buildFrame( - Jt808MessageId.TERMINAL_AUTH, - "123456789012", - 2, - "TOKEN,123456789012345,SW1".getBytes(java.nio.charset.StandardCharsets.US_ASCII))); - - assertThat(sessions.findByPhone("123456789012")) - .get() - .extracting(session -> session.vin()) - .isEqualTo("LNVIN000000AUTH01"); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void registerAckWaitsForDispatchDurability() { - ControlledSink sink = new ControlledSink(); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - new HandlerRegistry(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new RegisterBodyParser()))), - dispatcher, - new TestSessionStore(), - new InMemoryVehicleIdentityService(), - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - - channel.writeInbound(buildFrame( - Jt808MessageId.TERMINAL_REGISTER, - "123456789012", - 1, - buildRegisterBody("DEV808", "B80808"))); - - assertThat((Object) channel.readOutbound()).isNull(); - sink.awaitPublishCount(1); - sink.future(0).complete(null); - - ByteBuf ack = readOutboundAfterRunningTasks(channel); - assertThat(ack).isNotNull(); - ack.release(); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void inactiveChannelRemovesSessionStoreEntry() { - TestSessionStore sessions = new TestSessionStore(); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(new ImmediateKafkaSink())); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - new HandlerRegistry(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new AuthBodyParser()))), - dispatcher, - sessions, - new InMemoryVehicleIdentityService(), - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - - channel.writeInbound(buildFrame( - Jt808MessageId.TERMINAL_AUTH, - "123456789012", - 2, - "TOKEN,123456789012345,SW1".getBytes(java.nio.charset.StandardCharsets.US_ASCII))); - assertThat(sessions.findByPhone("123456789012")).isPresent(); - - channel.close(); - - assertThat(sessions.findByPhone("123456789012")).isEmpty(); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void successfulFrameArchiveUsesResolvedVehicleIdentity() throws Exception { - RecordingSink sink = new RecordingSink(1); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - new HandlerRegistry(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - identity.bind(new VehicleIdentityBinding( - ProtocolId.JT808, "LNVIN000000000808", "123456789012", "DEV808", "粤B80808")); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new LocationBodyParser()))), - dispatcher, - new TestSessionStore(), - identity, - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - byte[] frame = buildFrame(Jt808MessageId.TERMINAL_LOCATION, "123456789012", 1, buildLocationBody()); - - channel.writeInbound(frame); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.vin()).isEqualTo("LNVIN000000000808"); - assertThat(archive.metadata()) - .containsEntry("vin", "LNVIN000000000808") - .containsEntry("phone", "123456789012") - .containsEntry("messageId", "0x0200") - .containsEntry("messageName", "TERMINAL_LOCATION") - .containsEntry("reportType", "LOCATION") - .containsEntry("body.alarmFlag", "0") - .containsEntry("body.statusFlag", "196608") - .containsEntry("body.status.acc_on", "false") - .containsEntry("body.status.load_status", "EMPTY") - .containsEntry("body.status.door4_open", "true") - .containsEntry("body.status.door5_open", "true") - .containsEntry("body.alarm.overspeed", "false") - .containsEntry("body.longitude", "116.397128") - .containsEntry("body.latitude", "39.916527") - .containsEntry("body.altitudeM", "50") - .containsEntry("body.directionDeg", "90") - .containsEntry("body.deviceTime", "2024-01-01T19:04:05Z") - .containsEntry("body.extra.0x01", "00003039") - .containsEntry("body.extra.mileage_km", "1234.5") - .containsEntry("body.extra.0x30", "58") - .containsEntry("body.extra.network_signal_strength", "88") - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_PHONE"); - assertThat(Double.parseDouble(archive.metadata().get("body.speedKmh"))).isCloseTo(52.3, within(0.001)); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void registerArchiveIncludesParsedBodyFieldsForRawQuery() throws Exception { - RecordingSink sink = new RecordingSink(1); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - new HandlerRegistry(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - identity.bind(new VehicleIdentityBinding( - ProtocolId.JT808, "LNVIN000000000808", "123456789012", "DEV808", "B80808")); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new RegisterBodyParser()))), - dispatcher, - new TestSessionStore(), - identity, - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - byte[] frame = buildFrame( - Jt808MessageId.TERMINAL_REGISTER, - "123456789012", - 3, - buildRegisterBody("DEV808", "B80808")); - - channel.writeInbound(frame); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.metadata()) - .containsEntry("messageId", "0x0100") - .containsEntry("messageName", "TERMINAL_REGISTER") - .containsEntry("reportType", "REGISTER") - .containsEntry("body.province", "44") - .containsEntry("body.city", "4401") - .containsEntry("body.maker", "MAKER") - .containsEntry("body.deviceType", "TYPE-A") - .containsEntry("body.deviceId", "DEV808") - .containsEntry("body.plateColor", "1") - .containsEntry("body.plate", "B80808"); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void unboundRegisterCreatesVinBindingForRegisterAndFollowingRawFrames() throws Exception { - RecordingSink sink = new RecordingSink(2); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - new HandlerRegistry(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new RegisterBodyParser(), new LocationBodyParser()))), - dispatcher, - new TestSessionStore(), - identity, - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - - channel.writeInbound(buildFrame( - Jt808MessageId.TERMINAL_REGISTER, - "13079963289", - 3, - buildRegisterBody("9963289", "HUA00113F"))); - channel.writeInbound(buildFrame( - Jt808MessageId.TERMINAL_LOCATION, - "13079963289", - 4, - buildLocationBody())); - - assertThat(sink.await()).isTrue(); - VehicleEvent.RawArchive register = (VehicleEvent.RawArchive) sink.events.get(0); - VehicleEvent.RawArchive location = (VehicleEvent.RawArchive) sink.events.get(1); - assertThat(register.metadata()) - .containsEntry("reportType", "REGISTER") - .containsEntry("body.deviceId", "9963289") - .containsEntry("body.plate", "HUA00113F") - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "REGISTERED"); - assertThat(register.vin()) - .startsWith("JT808") - .hasSize(17); - assertThat(location.metadata()) - .containsEntry("reportType", "LOCATION") - .containsEntry("vin", register.vin()) - .containsEntry("identityResolved", "true") - .containsEntry("identitySource", "BOUND_PHONE"); - assertThat(location.vin()).isEqualTo(register.vin()); - assertThat(identity.resolve(new com.lingniu.ingest.identity.VehicleIdentityLookup( - ProtocolId.JT808, "", "13079963289", "", "")).vin()) - .isEqualTo(register.vin()); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void unboundLocationArchiveUsesUnknownVinInsteadOfPhoneFallback() throws Exception { - RecordingSink sink = new RecordingSink(1); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - new HandlerRegistry(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new LocationBodyParser()))), - dispatcher, - new TestSessionStore(), - identity, - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - byte[] frame = buildFrame(Jt808MessageId.TERMINAL_LOCATION, "123456789012", 1, buildLocationBody()); - - channel.writeInbound(frame); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.vin()).isEqualTo("unknown"); - assertThat(archive.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("phone", "123456789012") - .containsEntry("messageId", "0x0200") - .containsEntry("messageName", "TERMINAL_LOCATION") - .containsEntry("reportType", "LOCATION") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "FALLBACK_PHONE"); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void registerArchivePreservesRegistrationFieldsInMetadata() throws Exception { - RecordingSink sink = new RecordingSink(1); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - new HandlerRegistry(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new RegisterBodyParser()))), - dispatcher, - new TestSessionStore(), - identity, - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - byte[] frame = buildFrame( - Jt808MessageId.TERMINAL_REGISTER, - "123456789012", - 1, - buildRegisterBody("DEV808", "B80808")); - - channel.writeInbound(frame); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).singleElement().satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.command()).isEqualTo(Jt808MessageId.TERMINAL_REGISTER); - assertThat(archive.metadata()) - .containsEntry("body.province", "44") - .containsEntry("body.city", "4401") - .containsEntry("body.maker", "MAKER") - .containsEntry("body.deviceType", "TYPE-A") - .containsEntry("body.deviceId", "DEV808") - .containsEntry("body.plateColor", "1") - .containsEntry("body.plate", "B80808"); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void malformedFrameIsArchivedAndDispatchedAsPassthrough() throws Exception { - RecordingSink sink = new RecordingSink(2); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - registryWithRawHandler(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of())), - dispatcher, - new TestSessionStore(), - new InMemoryVehicleIdentityService(), - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - byte[] malformed = new byte[]{0x01, 0x02, 0x03}; - - channel.writeInbound(malformed); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive raw = (VehicleEvent.RawArchive) event; - assertThat(raw.source()).isEqualTo(ProtocolId.JT808); - assertThat(raw.command()).isZero(); - assertThat(raw.rawBytes()).containsExactly(malformed); - assertThat(raw.metadata()) - .containsEntry("parseError", "true") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - }); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(ProtocolId.JT808); - assertThat(passthrough.vin()).isEqualTo("unknown"); - assertThat(passthrough.metadata()) - .containsEntry("parseError", "true") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - assertThat(passthrough.data()).containsExactly(malformed); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void frameBoundaryMalformedCandidateIsArchivedAndDispatchedAsPassthrough() throws Exception { - RecordingSink sink = new RecordingSink(2); - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - registryWithRawHandler(), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of())), - dispatcher, - new TestSessionStore(), - new InMemoryVehicleIdentityService(), - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - byte[] raw = new byte[]{0x01, 0x02, 0x03}; - - channel.writeInbound(new Jt808MalformedFrame(raw, "unknown", "jt808 missing frame boundary")); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.source()).isEqualTo(ProtocolId.JT808); - assertThat(archive.rawBytes()).containsExactly(raw); - assertThat(archive.metadata()).containsEntry("frameError", "true") - .containsEntry("frameErrorMessage", "jt808 missing frame boundary") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - }); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.source()).isEqualTo(ProtocolId.JT808); - assertThat(passthrough.metadata()).containsEntry("frameError", "true") - .containsEntry("frameErrorMessage", "jt808 missing frame boundary") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN"); - assertThat(passthrough.data()).containsExactly(raw); - }); - - batchExecutor.close(); - eventBus.close(); - } - - @Test - void identityResolverFailureStillArchivesAndDispatchesDecodedLocationWithUnknownIdentity() throws Exception { - RecordingSink sink = new RecordingSink(2); - var failingIdentity = (com.lingniu.ingest.identity.VehicleIdentityResolver) lookup -> { - throw new IllegalStateException("identity backend unavailable"); - }; - DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink)); - AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish); - Dispatcher dispatcher = new Dispatcher( - registryWithLocationHandler(new DirectLocationHandler(new Jt808EventMapper(failingIdentity))), - new InterceptorChain(List.of()), - new HandlerInvoker(), - eventBus, - batchExecutor); - Jt808ChannelHandler handler = new Jt808ChannelHandler( - new Jt808MessageDecoder(new BodyParserRegistry(List.of(new LocationBodyParser()))), - dispatcher, - new TestSessionStore(), - new InMemoryVehicleIdentityService(), - failingIdentity, - new Jt808ChannelRegistry(), - new Jt808PendingRequests()); - EmbeddedChannel channel = new EmbeddedChannel(handler); - byte[] frame = buildFrame(Jt808MessageId.TERMINAL_LOCATION, "123456789012", 1, buildLocationBody()); - - channel.writeInbound(frame); - - assertThat(sink.await()).isTrue(); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.RawArchive.class); - VehicleEvent.RawArchive archive = (VehicleEvent.RawArchive) event; - assertThat(archive.source()).isEqualTo(ProtocolId.JT808); - assertThat(archive.command()).isEqualTo(Jt808MessageId.TERMINAL_LOCATION); - assertThat(archive.rawBytes()).containsExactly(frame); - assertThat(archive.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("phone", "123456789012") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN") - .containsEntry("identityError", "true") - .containsEntry("identityErrorMessage", "identity backend unavailable"); - }); - assertThat(sink.events).anySatisfy(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Location.class); - VehicleEvent.Location location = (VehicleEvent.Location) event; - assertThat(location.source()).isEqualTo(ProtocolId.JT808); - assertThat(location.vin()).isEqualTo("unknown"); - assertThat(location.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("phone", "123456789012") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN") - .containsEntry("identityError", "true") - .containsEntry("identityErrorMessage", "identity backend unavailable"); - }); - - batchExecutor.close(); - eventBus.close(); - } - - private static HandlerRegistry registryWithRawHandler() throws NoSuchMethodException { - HandlerRegistry registry = new HandlerRegistry(); - Jt808LocationHandler rawHandler = new Jt808LocationHandler( - new Jt808EventMapper(new InMemoryVehicleIdentityService())); - registry.register(new HandlerDefinition( - ProtocolId.JT808, - 0, - 0, - "test raw", - rawHandler, - Jt808LocationHandler.class.getMethod("onRaw", Jt808Message.class), - Jt808Message.class, - null, - null, - null)); - return registry; - } - - private static HandlerRegistry registryWithLocationHandler(DirectLocationHandler locationHandler) throws NoSuchMethodException { - HandlerRegistry registry = new HandlerRegistry(); - registry.register(new HandlerDefinition( - ProtocolId.JT808, - Jt808MessageId.TERMINAL_LOCATION, - 0, - "test location", - locationHandler, - DirectLocationHandler.class.getMethod("onLocation", Jt808Message.class), - Jt808Message.class, - null, - null, - null)); - return registry; - } - - public static final class DirectLocationHandler { - private final Jt808EventMapper mapper; - - private DirectLocationHandler(Jt808EventMapper mapper) { - this.mapper = mapper; - } - - public List onLocation(Jt808Message msg) { - return mapper.toEvents(msg); - } - } - - private static byte[] buildLocationBody() { - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeU32(os, 0); - writeU32(os, 0x00030000); - writeU32(os, 39_916_527L); - writeU32(os, 116_397_128L); - writeU16(os, 50); - writeU16(os, 523); - writeU16(os, 90); - byte[] bcd = BcdCodec.encode("240102030405"); - os.write(bcd, 0, 6); - os.write(0x01); - os.write(4); - writeU32(os, 12345); - os.write(0x30); - os.write(1); - os.write(0x58); - return os.toByteArray(); - } - - private static byte[] buildRegisterBody(String deviceId, String plate) { - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeU16(os, 44); - writeU16(os, 4401); - writeAscii(os, "MAKER", 5); - writeAscii(os, "TYPE-A", 20); - writeAscii(os, deviceId, 7); - os.write(1); - byte[] plateBytes = plate.getBytes(java.nio.charset.StandardCharsets.US_ASCII); - os.write(plateBytes, 0, plateBytes.length); - return os.toByteArray(); - } - - private static void writeAscii(ByteArrayOutputStream os, String value, int len) { - byte[] raw = value.getBytes(java.nio.charset.StandardCharsets.US_ASCII); - int copy = Math.min(raw.length, len); - os.write(raw, 0, copy); - for (int i = copy; i < len; i++) { - os.write(0x20); - } - } - - private static byte[] buildFrame(int msgId, String phone, int serial, byte[] body) { - ByteArrayOutputStream os = new ByteArrayOutputStream(); - writeU16(os, msgId); - writeU16(os, body.length & 0x03FF); - byte[] phoneBcd = BcdCodec.encode(phone); - os.write(phoneBcd, 0, 6); - writeU16(os, serial); - os.write(body, 0, body.length); - byte[] raw = os.toByteArray(); - byte bcc = BccChecksum.compute(raw, 0, raw.length); - - ByteArrayOutputStream framed = new ByteArrayOutputStream(); - framed.write(raw, 0, raw.length); - framed.write(bcc & 0xFF); - return framed.toByteArray(); - } - - private static void writeU16(ByteArrayOutputStream os, int v) { - os.write((v >> 8) & 0xFF); - os.write(v & 0xFF); - } - - private static void writeU32(ByteArrayOutputStream os, long v) { - os.write((int) ((v >> 24) & 0xFF)); - os.write((int) ((v >> 16) & 0xFF)); - os.write((int) ((v >> 8) & 0xFF)); - os.write((int) (v & 0xFF)); - } - - private static ByteBuf readOutboundAfterRunningTasks(EmbeddedChannel channel) { - long deadline = System.currentTimeMillis() + 3000; - while (System.currentTimeMillis() < deadline) { - channel.runPendingTasks(); - ByteBuf outbound = channel.readOutbound(); - if (outbound != null) { - return outbound; - } - Thread.onSpinWait(); - } - return null; - } - - private static final class ControlledSink implements EventSink { - private final List> futures = new CopyOnWriteArrayList<>(); - - @Override - public String name() { - return "kafka"; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - CompletableFuture future = new CompletableFuture<>(); - futures.add(future); - return future; - } - - private CompletableFuture future(int index) { - return futures.get(index); - } - - private void awaitPublishCount(int expected) { - long deadline = System.currentTimeMillis() + 3000; - while (System.currentTimeMillis() < deadline) { - if (futures.size() >= expected) { - return; - } - Thread.onSpinWait(); - } - throw new AssertionError("expected " + expected + " sink publishes, actual=" + futures.size()); - } - } - - private static final class ImmediateKafkaSink implements EventSink { - @Override - public String name() { - return "kafka"; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - return CompletableFuture.completedFuture(null); - } - } - - private static final class RecordingSink implements EventSink { - private final List events = new CopyOnWriteArrayList<>(); - private final CountDownLatch latch; - - private RecordingSink(int expectedEvents) { - this.latch = new CountDownLatch(expectedEvents); - } - - @Override - public String name() { - return "kafka"; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - events.add(event); - latch.countDown(); - return CompletableFuture.completedFuture(null); - } - - private boolean await() throws InterruptedException { - return latch.await(3, TimeUnit.SECONDS); - } - } -} diff --git a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/mapper/Jt808EventMapperTest.java b/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/mapper/Jt808EventMapperTest.java deleted file mode 100644 index 03279b1e..00000000 --- a/modules/protocols/protocol-jt808/src/test/java/com/lingniu/ingest/protocol/jt808/mapper/Jt808EventMapperTest.java +++ /dev/null @@ -1,326 +0,0 @@ -package com.lingniu.ingest.protocol.jt808.mapper; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.identity.InMemoryVehicleIdentityService; -import com.lingniu.ingest.identity.VehicleIdentityBinding; -import com.lingniu.ingest.protocol.jt808.model.Jt808Body; -import com.lingniu.ingest.protocol.jt808.model.Jt808Header; -import com.lingniu.ingest.protocol.jt808.model.Jt808Message; -import com.lingniu.ingest.protocol.jt808.model.Jt808MessageId; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt808EventMapperTest { - - private final InMemoryVehicleIdentityService identity = new InMemoryVehicleIdentityService(); - private final Jt808EventMapper mapper = new Jt808EventMapper(identity); - - @Test - void locationBodyProducesLocationEvent() { - var header = new Jt808Header( - Jt808MessageId.TERMINAL_LOCATION, 28, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 1, 0, 0); - var body = new Jt808Body.Location( - 0, 0, 116.397128, 39.916527, 50, 52.3, 90, Instant.parse("2024-01-02T03:04:05Z")); - var msg = new Jt808Message(header, body); - - List events = mapper.toEvents(msg); - - assertThat(events).hasSize(1); - assertThat(events.get(0)).isInstanceOf(VehicleEvent.Location.class); - assertThat(events.get(0).source()).isEqualTo(ProtocolId.JT808); - assertThat(events.get(0).vin()).isEqualTo("unknown"); - assertThat(events.get(0).metadata()) - .containsEntry("vin", "unknown") - .containsEntry("phone", "123456789012") - .containsEntry("jt808.status.acc_on", "false") - .containsEntry("jt808.status.positioned", "false") - .containsEntry("jt808.status.load_status", "EMPTY") - .containsEntry("jt808.alarm.overspeed", "false") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "FALLBACK_PHONE"); - } - - @Test - void locationExtensionItemsAreExposedAsMetadata() { - var header = new Jt808Header( - Jt808MessageId.TERMINAL_LOCATION, 35, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 1, 0, 0); - var body = new Jt808Body.Location( - 0, 0, 116.397128, 39.916527, 50, 52.3, 90, - Instant.parse("2024-01-02T03:04:05Z"), - java.util.Map.of(0x01, new byte[]{0x00, 0x00, 0x30, 0x39}, 0x30, new byte[]{0x58})); - - List events = mapper.toEvents(new Jt808Message(header, body)); - - assertThat(events).singleElement() - .extracting(VehicleEvent::metadata) - .satisfies(meta -> assertThat(meta) - .containsEntry("jt808.extra.0x01", "00003039") - .containsEntry("jt808.extra.0x30", "58") - .containsEntry("jt808.extra.mileage_km", "1234.5") - .containsEntry("jt808.extra.network_signal_strength", "88")); - } - - @Test - void locationAdditionalItemsAreDecodedAsTable27Fields() { - var header = new Jt808Header( - Jt808MessageId.TERMINAL_LOCATION, 64, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 1, 0, 0); - var body = new Jt808Body.Location( - 0, 0, 116.397128, 39.916527, 50, 52.3, 90, - Instant.parse("2024-01-02T03:04:05Z"), - java.util.Map.ofEntries( - java.util.Map.entry(0x01, new byte[]{0x00, 0x00, 0x30, 0x39}), - java.util.Map.entry(0x02, new byte[]{0x00, 0x64}), - java.util.Map.entry(0x03, new byte[]{0x01, (byte) 0xF4}), - java.util.Map.entry(0x04, new byte[]{0x00, 0x02}), - java.util.Map.entry(0x05, tirePressureBytes()), - java.util.Map.entry(0x06, new byte[]{(byte) 0x80, 0x05}), - java.util.Map.entry(0x11, new byte[]{0x01, 0x00, 0x00, 0x00, 0x2A}), - java.util.Map.entry(0x12, new byte[]{0x04, 0x00, 0x00, 0x00, 0x2B, 0x01}), - java.util.Map.entry(0x13, new byte[]{0x00, 0x00, 0x00, 0x2C, 0x00, 0x78, 0x01}), - java.util.Map.entry(0x25, new byte[]{0x00, 0x00, 0x00, 0x05}), - java.util.Map.entry(0x2A, new byte[]{0x00, 0x03}), - java.util.Map.entry(0x2B, new byte[]{0x12, 0x34, 0x56, 0x78}), - java.util.Map.entry(0x30, new byte[]{0x1F}), - java.util.Map.entry(0x31, new byte[]{0x0C}))); - - VehicleEvent.Location event = (VehicleEvent.Location) mapper.toEvents(new Jt808Message(header, body)).getFirst(); - - assertThat(event.payload().totalMileageKm()).isEqualTo(1234.5); - assertThat(event.metadata()) - .containsEntry("jt808.extra.mileage_km", "1234.5") - .containsEntry("jt808.extra.fuel_l", "10") - .containsEntry("jt808.extra.recorder_speed_kmh", "50") - .containsEntry("jt808.extra.manual_alarm_event_id", "2") - .containsEntry("jt808.extra.tire_pressure_pa_values", "32,33") - .containsEntry("jt808.extra.tire_pressure_invalid_indexes", "2-29") - .containsEntry("jt808.extra.compartment_temp_c", "-5") - .containsEntry("jt808.extra.overspeed.position_type", "1") - .containsEntry("jt808.extra.overspeed.area_route_id", "42") - .containsEntry("jt808.extra.area_route_alarm.position_type", "4") - .containsEntry("jt808.extra.area_route_alarm.area_route_id", "43") - .containsEntry("jt808.extra.area_route_alarm.direction", "1") - .containsEntry("jt808.extra.route_time.segment_id", "44") - .containsEntry("jt808.extra.route_time.duration_seconds", "120") - .containsEntry("jt808.extra.route_time.result", "1") - .containsEntry("jt808.extra.vehicle_signal_raw", "5") - .containsEntry("jt808.extra.vehicle_signal.low_beam", "true") - .containsEntry("jt808.extra.vehicle_signal.high_beam", "false") - .containsEntry("jt808.extra.vehicle_signal.right_turn", "true") - .containsEntry("jt808.extra.io_status_raw", "3") - .containsEntry("jt808.extra.io_status.deep_sleep", "true") - .containsEntry("jt808.extra.io_status.sleep", "true") - .containsEntry("jt808.extra.analog_raw", "305419896") - .containsEntry("jt808.extra.analog_ad0", "22136") - .containsEntry("jt808.extra.analog_ad1", "4660") - .containsEntry("jt808.extra.network_signal_strength", "31") - .containsEntry("jt808.extra.gnss_satellite_count", "12"); - } - - @Test - void locationMileageExtensionIsMappedToInternalMileageField() { - var header = new Jt808Header( - Jt808MessageId.TERMINAL_LOCATION, 34, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 1, 0, 0); - var body = new Jt808Body.Location( - 0, 0, 116.397128, 39.916527, 50, 52.3, 90, - Instant.parse("2024-01-02T03:04:05Z"), - java.util.Map.of(0x01, new byte[]{0x00, 0x00, 0x30, 0x39})); - - VehicleEvent.Location event = (VehicleEvent.Location) mapper.toEvents(new Jt808Message(header, body)).getFirst(); - - assertThat(event.payload().totalMileageKm()).isEqualTo(1234.5); - assertThat(event.metadata()) - .containsEntry("jt808.extra.0x01", "00003039") - .containsEntry("jt808.extra.mileage_km", "1234.5") - .containsEntry("total_mileage_km", "1234.5"); - } - - @Test - void locationBodyUsesBoundVehicleIdentity() { - identity.bind(new VehicleIdentityBinding(ProtocolId.JT808, - "LNVIN000000000009", "123456789012", "DEV009", "粤B99999")); - var header = new Jt808Header( - Jt808MessageId.TERMINAL_LOCATION, 28, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 1, 0, 0); - var body = new Jt808Body.Location( - 0, 0, 116.397128, 39.916527, 50, 52.3, 90, Instant.parse("2024-01-02T03:04:05Z")); - - List events = mapper.toEvents(new Jt808Message(header, body)); - - assertThat(events).hasSize(1); - assertThat(events.get(0).vin()).isEqualTo("LNVIN000000000009"); - assertThat(events.get(0).metadata()) - .containsEntry("vin", "LNVIN000000000009") - .containsEntry("identityResolved", "true"); - } - - @Test - void identityResolverFailureStillProducesLocationEventWithUnknownVin() { - Jt808EventMapper mapperWithFailingIdentity = new Jt808EventMapper(lookup -> { - throw new IllegalStateException("identity backend unavailable"); - }); - var header = new Jt808Header( - Jt808MessageId.TERMINAL_LOCATION, 28, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 1, 0, 0); - var body = new Jt808Body.Location( - 0, 0, 116.397128, 39.916527, 50, 52.3, 90, Instant.parse("2024-01-02T03:04:05Z")); - - List events = mapperWithFailingIdentity.toEvents(new Jt808Message(header, body)); - - assertThat(events).singleElement() - .satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Location.class); - assertThat(event.vin()).isEqualTo("unknown"); - assertThat(event.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("phone", "123456789012") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "UNKNOWN") - .containsEntry("identityError", "true") - .containsEntry("identityErrorMessage", "identity backend unavailable"); - }); - } - - @Test - void heartbeatBodyProducesHeartbeatEvent() { - var header = new Jt808Header( - Jt808MessageId.TERMINAL_HEARTBEAT, 0, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 1, 0, 0); - var msg = new Jt808Message(header, new Jt808Body.Heartbeat()); - assertThat(mapper.toEvents(msg)).hasSize(1) - .first().isInstanceOf(VehicleEvent.Heartbeat.class); - } - - @Test - void registerBodyPreservesRegistrationFieldsInMetadata() { - var header = new Jt808Header( - Jt808MessageId.TERMINAL_REGISTER, 37, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 1, 0, 0); - var body = new Jt808Body.Register( - 44, 1, "G7", "MODEL-X", "DEV123456", 2, "粤B12345"); - - List events = mapper.toEvents(new Jt808Message(header, body)); - - assertThat(events).singleElement() - .satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Login.class); - assertThat(event.metadata()) - .containsEntry("jt808.register.province", "44") - .containsEntry("jt808.register.city", "1") - .containsEntry("jt808.register.maker", "G7") - .containsEntry("jt808.register.deviceType", "MODEL-X") - .containsEntry("jt808.register.deviceId", "DEV123456") - .containsEntry("jt808.register.plateColor", "2") - .containsEntry("jt808.register.plate", "粤B12345"); - }); - } - - @Test - void unregisterBodyProducesLogoutEvent() { - var header = new Jt808Header( - Jt808MessageId.TERMINAL_UNREGISTER, 0, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 1, 0, 0); - - assertThat(mapper.toEvents(new Jt808Message(header, new Jt808Body.Unregister()))) - .singleElement() - .isInstanceOf(VehicleEvent.Logout.class); - } - - @Test - void locationBatchProducesOneLocationEventPerPoint() { - var header = new Jt808Header( - Jt808MessageId.TERMINAL_LOCATION_BATCH, 0, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 1, 0, 0); - var loc = new Jt808Body.Location( - 0, 0, 116.397128, 39.916527, 50, 52.3, 90, Instant.parse("2024-01-02T03:04:05Z")); - - List events = mapper.toEvents(new Jt808Message( - header, new Jt808Body.LocationBatch(0, List.of(loc, loc)))); - - assertThat(events).hasSize(2) - .allSatisfy(event -> assertThat(event).isInstanceOf(VehicleEvent.Location.class)); - } - - @Test - void mediaAndPassthroughBodiesProduceQueryableEvents() { - var header = new Jt808Header( - Jt808MessageId.TERMINAL_MEDIA_EVENT, 0, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 1, 0, 0); - assertThat(mapper.toEvents(new Jt808Message( - header, new Jt808Body.MediaEvent(42, 0, 1, 2, 3)))) - .singleElement() - .isInstanceOf(VehicleEvent.MediaMeta.class); - - var passHeader = new Jt808Header( - Jt808MessageId.TERMINAL_PASSTHROUGH, 0, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 2, 0, 0); - assertThat(mapper.toEvents(new Jt808Message( - passHeader, new Jt808Body.Passthrough(0x41, new byte[]{1, 2, 3})))) - .singleElement() - .satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - VehicleEvent.Passthrough passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.passthroughType()).isEqualTo(Jt808MessageId.TERMINAL_PASSTHROUGH); - assertThat(passthrough.metadata()).containsEntry("passthroughType", "0x41"); - assertThat(passthrough.data()).containsExactly(1, 2, 3); - }); - } - - @Test - void rawBodyProducesPassthroughEventSoUnknownMessagesRemainQueryable() { - var header = new Jt808Header( - 0x0F01, 3, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 9, 0, 0); - - assertThat(mapper.toEvents(new Jt808Message( - header, new Jt808Body.Raw(0x0F01, new byte[]{0x11, 0x22, 0x33})))) - .singleElement() - .satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - var passthrough = (VehicleEvent.Passthrough) event; - assertThat(passthrough.passthroughType()).isEqualTo(0x0F01); - assertThat(passthrough.data()).containsExactly(0x11, 0x22, 0x33); - assertThat(passthrough.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("phone", "123456789012") - .containsEntry("identityResolved", "false") - .containsEntry("identitySource", "FALLBACK_PHONE") - .containsEntry("rawBody", "true"); - }); - } - - @Test - void malformedBodyPassthroughMetadataUsesUnknownInternalVin() { - var header = new Jt808Header( - 0, 3, 0, false, - Jt808Header.ProtocolVersion.V2013, "123456789012", 9, 0, 0); - - assertThat(mapper.toEvents(new Jt808Message( - header, new Jt808Body.Malformed(new byte[]{0x01, 0x02, 0x03}, "127.0.0.1:7611", "decode failed")))) - .singleElement() - .satisfies(event -> { - assertThat(event).isInstanceOf(VehicleEvent.Passthrough.class); - assertThat(event.vin()).isEqualTo("unknown"); - assertThat(event.metadata()) - .containsEntry("vin", "unknown") - .containsEntry("identityResolved", "false") - .containsEntry("parseError", "true"); - }); - } - - private static byte[] tirePressureBytes() { - byte[] bytes = new byte[30]; - java.util.Arrays.fill(bytes, (byte) 0xFF); - bytes[0] = 32; - bytes[1] = 33; - return bytes; - } -} diff --git a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/auth_001.hex b/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/auth_001.hex deleted file mode 100644 index 48bcc762..00000000 --- a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/auth_001.hex +++ /dev/null @@ -1 +0,0 @@ -7e010200070000000000230000687569746f6e67417e diff --git a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/auth_002.hex b/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/auth_002.hex deleted file mode 100644 index 9ef924e7..00000000 --- a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/auth_002.hex +++ /dev/null @@ -1 +0,0 @@ -7e0102001800000000006500016e65382f51525a50664738596f7145575155577236513d3d607e diff --git a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_001.hex b/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_001.hex deleted file mode 100644 index c515a631..00000000 --- a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_001.hex +++ /dev/null @@ -1 +0,0 @@ -7e02000036000000000001072d000000000008000301ceb03b0732f26e000d02af0010260413123353010400052425020200000302000025040000000030011f3101192c7e diff --git a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_002.hex b/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_002.hex deleted file mode 100644 index 1832fb3a..00000000 --- a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_002.hex +++ /dev/null @@ -1 +0,0 @@ -7e020040460100000000000000000002002c00000000000c000301bb0e2107213304003f0000010d26041312335514040000000017020000010400006722030200002504000000002a020000300115310118ea0402008000fb7e diff --git a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_010.hex b/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_010.hex deleted file mode 100644 index 2819da10..00000000 --- a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_010.hex +++ /dev/null @@ -1 +0,0 @@ -7e020000360000000000100278000000000008000301d596aa0735b9910001004b01472604131234030104000a42c10202000003020000250400000000300115310117107e diff --git a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_100.hex b/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_100.hex deleted file mode 100644 index fa3a3be1..00000000 --- a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_100.hex +++ /dev/null @@ -1 +0,0 @@ -7e020000360000000001020443000000000008000301d2d74907376cde000f00000000260413123354010400078789020200000302000025040000000030011b310112db7e diff --git a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_200.hex b/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_200.hex deleted file mode 100644 index ce0d2a99..00000000 --- a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_200.hex +++ /dev/null @@ -1 +0,0 @@ -7e02004048010000000000000000020204d2000000000048000301cc1d4f073ed6640010033301492604131234001404000000001702000001040000f0542504000000002a02000030011f310110ea04020c8300ef0400000000d67e diff --git a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_400.hex b/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_400.hex deleted file mode 100644 index 3540edbf..00000000 --- a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_400.hex +++ /dev/null @@ -1 +0,0 @@ -7e02000036000000000413013d000000000008000201d392280736f1e30007000000002604131311220104000f334a020200000302000025040000000030011931010a097e diff --git a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_batch_001.hex b/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_batch_001.hex deleted file mode 100644 index 104ef178..00000000 --- a/modules/protocols/protocol-jt808/src/test/resources/samples/jt808/location_batch_001.hex +++ /dev/null @@ -1 +0,0 @@ -7e0704003b000000000058034e0001010036000000000008000301e26c120725c15100110000000726041313111301040006ffc6030200002504000000002a02000030010a31010c3c7e diff --git a/modules/services/event-history-service/pom.xml b/modules/services/event-history-service/pom.xml deleted file mode 100644 index e9065f59..00000000 --- a/modules/services/event-history-service/pom.xml +++ /dev/null @@ -1,65 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - event-history-service - event-history-service - Kafka full-field event consumer + TDengine historical query API. - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - tdengine-history-store - - - com.lingniu.ingest - sink-kafka - - - com.lingniu.ingest - protocol-gb32960 - - - org.springframework.boot - spring-boot-starter-web - - - org.springframework.boot - spring-boot-autoconfigure - - - io.swagger.core.v3 - swagger-annotations-jakarta - 2.2.47 - - - org.springframework.boot - spring-boot-configuration-processor - true - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.springframework.boot - spring-boot-starter-test - test - - - diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/ApiExceptionHandler.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/ApiExceptionHandler.java deleted file mode 100644 index 45f7b4d7..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/ApiExceptionHandler.java +++ /dev/null @@ -1,114 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import jakarta.servlet.http.HttpServletRequest; -import org.springframework.http.HttpStatus; -import org.springframework.http.ResponseEntity; -import org.springframework.web.bind.MissingServletRequestParameterException; -import org.springframework.web.bind.annotation.ExceptionHandler; -import org.springframework.web.bind.annotation.RestControllerAdvice; -import org.springframework.web.method.annotation.MethodArgumentTypeMismatchException; -import org.springframework.web.server.ResponseStatusException; -import org.springframework.web.servlet.resource.NoResourceFoundException; - -import java.io.IOException; -import java.time.OffsetDateTime; -import java.time.ZoneId; -import java.time.format.DateTimeParseException; - -/** - * 统一 API 错误响应。 - * - *

    业务接口抛出的参数错误、时间格式错误和 Spring 参数绑定错误都会转成包含具体原因的 - * JSON，而不是让前端只能看到默认 400/500。这里的 timestamp 仅表示错误发生时间， - * 按东八区展示；业务数据里的 eventTime/ingestTime 不在这里改时区。 - */ -@RestControllerAdvice -public final class ApiExceptionHandler { - - private static final ZoneId DEFAULT_TIME_ZONE = ZoneId.of("Asia/Shanghai"); - - @ExceptionHandler(MissingServletRequestParameterException.class) - public ResponseEntity missingRequestParameter(MissingServletRequestParameterException ex, - HttpServletRequest request) { - return error(HttpStatus.BAD_REQUEST, - "missing required query parameter: " + ex.getParameterName(), - request); - } - - @ExceptionHandler(MethodArgumentTypeMismatchException.class) - public ResponseEntity typeMismatch(MethodArgumentTypeMismatchException ex, - HttpServletRequest request) { - String message = "invalid query parameter: " + ex.getName(); - if (ex.getValue() != null) { - message += " value=" + ex.getValue(); - } - return error(HttpStatus.BAD_REQUEST, message, request); - } - - @ExceptionHandler(ResponseStatusException.class) - public ResponseEntity responseStatus(ResponseStatusException ex, - HttpServletRequest request) { - HttpStatus status = HttpStatus.resolve(ex.getStatusCode().value()); - if (status == null) { - status = HttpStatus.INTERNAL_SERVER_ERROR; - } - String message = ex.getReason(); - if (message == null || message.isBlank()) { - message = status.getReasonPhrase(); - } - return error(status, message, request); - } - - @ExceptionHandler(IllegalArgumentException.class) - public ResponseEntity illegalArgument(IllegalArgumentException ex, - HttpServletRequest request) { - return error(HttpStatus.BAD_REQUEST, safeMessage(ex, "bad request"), request); - } - - @ExceptionHandler(DateTimeParseException.class) - public ResponseEntity dateTimeParse(DateTimeParseException ex, - HttpServletRequest request) { - return error(HttpStatus.BAD_REQUEST, "invalid date/time parameter: " + ex.getParsedString(), request); - } - - @ExceptionHandler(NoResourceFoundException.class) - public ResponseEntity noResource(NoResourceFoundException ex, - HttpServletRequest request) { - return error(HttpStatus.NOT_FOUND, "resource not found: " + request.getRequestURI(), request); - } - - @ExceptionHandler(IOException.class) - public ResponseEntity io(IOException ex, - HttpServletRequest request) { - return error(HttpStatus.SERVICE_UNAVAILABLE, safeMessage(ex, "history storage unavailable"), request); - } - - @ExceptionHandler(Exception.class) - public ResponseEntity generic(Exception ex, - HttpServletRequest request) { - // 保留异常 message 方便联调定位；底层如果没有 message，则退回通用文案。 - return error(HttpStatus.INTERNAL_SERVER_ERROR, safeMessage(ex, "internal server error"), request); - } - - private static ResponseEntity error(HttpStatus status, String message, HttpServletRequest request) { - return ResponseEntity.status(status).body(new ApiError( - OffsetDateTime.now(DEFAULT_TIME_ZONE).toString(), - status.value(), - status.getReasonPhrase(), - message, - request.getRequestURI())); - } - - private static String safeMessage(Exception ex, String fallback) { - String message = ex.getMessage(); - return message == null || message.isBlank() ? fallback : message; - } - - public record ApiError( - String timestamp, - int status, - String error, - String message, - String path) { - } -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/DefaultProductionProtocols.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/DefaultProductionProtocols.java deleted file mode 100644 index eb532833..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/DefaultProductionProtocols.java +++ /dev/null @@ -1,27 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import org.springframework.http.HttpStatus; -import org.springframework.web.server.ResponseStatusException; - -import java.util.Locale; -import java.util.Set; - -final class DefaultProductionProtocols { - - private static final Set SUPPORTED = Set.of( - "GB32960", - "JT808", - "MQTT_YUTONG"); - private static final String SUPPORTED_MESSAGE = "protocol must be one of GB32960, JT808, MQTT_YUTONG"; - - private DefaultProductionProtocols() { - } - - static String requireSupported(String protocol) { - String normalized = protocol == null ? "" : protocol.trim().toUpperCase(Locale.ROOT); - if (!SUPPORTED.contains(normalized)) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, SUPPORTED_MESSAGE); - } - return normalized; - } -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/EventHistoryEnvelopeIngestor.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/EventHistoryEnvelopeIngestor.java deleted file mode 100644 index 0e8f596b..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/EventHistoryEnvelopeIngestor.java +++ /dev/null @@ -1,126 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.google.protobuf.InvalidProtocolBufferException; -import com.lingniu.ingest.api.consumer.EnvelopeBatchIngestor; -import com.lingniu.ingest.api.consumer.EnvelopeIngestResult; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import com.lingniu.ingest.tdenginehistory.TdengineEnvelopeRows; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryWriter; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.IOException; -import java.util.ArrayList; -import java.util.List; - -/** - * Consumer-side entry point that stores Kafka envelope values into TDengine history tables. - * - *

    当前 32960 本机运行配置没有启用 Kafka consumer；这个类保留给“其他服务把 envelope - * 写回历史库”的解耦部署方式。失败时返回结构化结果，由 Kafka worker 决定是否进 DLQ。 - */ -public final class EventHistoryEnvelopeIngestor implements EnvelopeBatchIngestor { - - private static final Logger log = LoggerFactory.getLogger(EventHistoryEnvelopeIngestor.class); - - private final TdengineHistoryWriter tdengineWriter; - - public EventHistoryEnvelopeIngestor(TdengineHistoryWriter tdengineWriter) { - if (tdengineWriter == null) { - throw new IllegalArgumentException("tdengineWriter must not be null"); - } - this.tdengineWriter = tdengineWriter; - } - - public void ingest(byte[] kafkaValue) throws IOException { - VehicleEnvelope envelope = parse(kafkaValue); - writeTdengineFacts(List.of(envelope)); - } - - @Override - public EnvelopeIngestResult tryIngest(byte[] kafkaValue) { - return tryIngestAll(List.of(kafkaValue)).getFirst(); - } - - @Override - public List tryIngestAll(List kafkaValues) { - if (kafkaValues == null || kafkaValues.isEmpty()) { - return List.of(); - } - List results = new ArrayList<>(kafkaValues.size()); - List valid = new ArrayList<>(kafkaValues.size()); - for (byte[] kafkaValue : kafkaValues) { - VehicleEnvelope envelope = null; - try { - envelope = parse(kafkaValue); - results.add(null); - valid.add(new BatchEntry(results.size() - 1, envelope)); - } catch (IllegalArgumentException ex) { - results.add(envelope == null - ? EnvelopeIngestResult.invalid(ex.getMessage()) - : EnvelopeIngestResult.skipped(envelope.getEventId(), envelope.getVin(), ex.getMessage())); - } - } - if (valid.isEmpty()) { - return List.copyOf(results); - } - try { - writeTdengineFacts(valid.stream().map(BatchEntry::envelope).toList()); - for (BatchEntry entry : valid) { - results.set(entry.index(), EnvelopeIngestResult.stored( - entry.eventId(), - entry.vin())); - } - } catch (IOException ex) { - log.error("event history TDengine ingest failed batchSize={} validSize={} firstEventId={} firstVin={}", - kafkaValues.size(), - valid.size(), - valid.getFirst().envelope().getEventId(), - valid.getFirst().envelope().getVin(), - ex); - for (BatchEntry entry : valid) { - results.set(entry.index(), EnvelopeIngestResult.failed( - entry.envelope().getEventId(), - entry.envelope().getVin(), - ex.getMessage())); - } - } - return List.copyOf(results); - } - - private record BatchEntry(int index, VehicleEnvelope envelope) { - private String eventId() { - return envelope.getEventId(); - } - - private String vin() { - return envelope.getVin(); - } - } - - private static VehicleEnvelope parse(byte[] kafkaValue) { - if (kafkaValue == null || kafkaValue.length == 0) { - throw new IllegalArgumentException("VehicleEnvelope bytes must not be empty"); - } - try { - return VehicleEnvelope.parseFrom(kafkaValue); - } catch (InvalidProtocolBufferException e) { - throw new IllegalArgumentException("failed to parse VehicleEnvelope", e); - } - } - - private void writeTdengineFacts(List envelopes) throws IOException { - var rawFrames = envelopes.stream() - .flatMap(envelope -> TdengineEnvelopeRows.rawFrame(envelope).stream()) - .toList(); - if (!rawFrames.isEmpty()) { - tdengineWriter.appendRawFrames(rawFrames); - } - var locations = envelopes.stream() - .flatMap(envelope -> TdengineEnvelopeRows.location(envelope).stream()) - .toList(); - if (!locations.isEmpty()) { - tdengineWriter.appendLocations(locations); - } - } -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Gb32960DecodedFrameService.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Gb32960DecodedFrameService.java deleted file mode 100644 index 562f21db..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Gb32960DecodedFrameService.java +++ /dev/null @@ -1,865 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.fasterxml.jackson.core.type.TypeReference; -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.model.Gb32960Message; -import com.lingniu.ingest.protocol.gb32960.model.InfoBlock; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdenginePage; -import com.lingniu.ingest.tdenginehistory.TdenginePageCursor; -import com.lingniu.ingest.tdenginehistory.TdengineQueryOrder; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameQuery; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameRow; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.io.IOException; -import java.math.BigDecimal; -import java.math.RoundingMode; -import java.nio.ByteBuffer; -import java.nio.file.Files; -import java.nio.file.NoSuchFileException; -import java.nio.file.Path; -import java.time.Instant; -import java.time.LocalDate; -import java.time.ZoneId; -import java.util.ArrayList; -import java.util.Comparator; -import java.util.LinkedHashMap; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Map; - -/** - * GB32960 RAW 历史帧查询和业务快照组装服务。 - * - *

    TDengine raw_frames 保存 archive:// URI。查询时先从 TDengine 找到 RAW 索引， - * 再读取本地 RAW .bin，用当前协议解析器即时解码。 - * 这种设计让写入路径保持轻量，也允许后续修复解析器后直接重放历史 RAW 得到新字段。 - * - *

    snapshot 的聚合单位是 {@code vin + eventTime}。同一时刻可能有多个 32960 子包， - * 比如广东燃料电池堆电压分帧；这里会把这些子包合并成前端可消费的单个逻辑快照。 - */ -public final class Gb32960DecodedFrameService { - - private static final Logger log = LoggerFactory.getLogger(Gb32960DecodedFrameService.class); - private static final TypeReference> MAP_TYPE = new TypeReference<>() {}; - private static final int OUTPUT_DOUBLE_SCALE = 6; - private static final ZoneId DEFAULT_ZONE = ZoneId.of("Asia/Shanghai"); - - private final TdengineHistoryReader tdengineReader; - private final Gb32960MessageDecoder decoder; - private final Path archiveRoot; - private final ObjectMapper objectMapper; - - public Gb32960DecodedFrameService(TdengineHistoryReader tdengineReader, - Gb32960MessageDecoder decoder, - Path archiveRoot, - ObjectMapper objectMapper) { - if (tdengineReader == null) { - log.info("gb32960 decoded frame service has no history index backend; rawUri direct lookup remains available"); - } else { - log.info("gb32960 decoded frame service using TDengine raw_frames index"); - } - if (decoder == null) { - throw new IllegalArgumentException("decoder must not be null"); - } - if (archiveRoot == null) { - throw new IllegalArgumentException("archiveRoot must not be null"); - } - this.tdengineReader = tdengineReader; - this.decoder = decoder; - this.archiveRoot = archiveRoot.toAbsolutePath().normalize(); - this.objectMapper = objectMapper == null ? new ObjectMapper() : objectMapper; - } - - public List query(LocalDate dateFrom, - LocalDate dateTo, - HistoryQueryOrder order, - int limit, - String vin, - String platformAccount) throws IOException { - return query(dateFrom, dateTo, null, null, order, limit, vin, platformAccount); - } - - public List query(LocalDate dateFrom, - LocalDate dateTo, - java.time.Instant eventTimeFrom, - java.time.Instant eventTimeTo, - HistoryQueryOrder order, - int limit, - String vin, - String platformAccount) throws IOException { - int frameLimit = Math.max(1, Math.min(limit, 1000)); - if (tdengineReader == null) { - return List.of(); - } - return queryFromTdengine(dateFrom, dateTo, eventTimeFrom, eventTimeTo, order, frameLimit, vin, platformAccount); - } - - public DecodedFramePage queryPage(QueryTimeRange range, - HistoryQueryOrder order, - int limit, - String vin, - String platformAccount, - TdenginePageCursor cursor) throws IOException { - int frameLimit = Math.max(1, Math.min(limit, 1000)); - if (tdengineReader == null || vin == null || vin.isBlank()) { - return new DecodedFramePage(List.of(), null); - } - return queryPageFromTdengine( - range.dateFrom(), - range.dateTo(), - range.eventTimeFrom(), - range.eventTimeTo(), - order, - frameLimit, - vin, - platformAccount, - cursor); - } - - private List queryFromTdengine(LocalDate dateFrom, - LocalDate dateTo, - java.time.Instant eventTimeFrom, - java.time.Instant eventTimeTo, - HistoryQueryOrder order, - int frameLimit, - String vin, - String platformAccount) throws IOException { - String vehicleKey = vin == null ? "" : vin.trim(); - TdenginePage page = tdengineReader.queryRawFrames(new TdengineRawFrameQuery( - "GB32960", - vehicleKey, - fromInstant(dateFrom, eventTimeFrom), - toExclusiveInstant(dateTo, eventTimeTo), - tdengineOrder(order), - frameLimit, - null)); - List out = new ArrayList<>(page.items().size()); - LinkedHashSet seenUris = new LinkedHashSet<>(); - for (TdengineRawFrameRow row : page.items()) { - String rawArchiveUri = row.rawUri(); - if (rawArchiveUri == null || rawArchiveUri.isBlank() || !seenUris.add(rawArchiveUri)) { - continue; - } - DecodedFrame frame = decodeIfAvailable(row, platformAccount); - if (frame != null) { - out.add(frame); - } - } - return out; - } - - private DecodedFramePage queryPageFromTdengine(LocalDate dateFrom, - LocalDate dateTo, - java.time.Instant eventTimeFrom, - java.time.Instant eventTimeTo, - HistoryQueryOrder order, - int frameLimit, - String vin, - String platformAccount, - TdenginePageCursor cursor) throws IOException { - TdenginePage page = tdengineReader.queryRawFrames(new TdengineRawFrameQuery( - "GB32960", - vin.trim(), - fromInstant(dateFrom, eventTimeFrom), - toExclusiveInstant(dateTo, eventTimeTo), - tdengineOrder(order), - frameLimit, - cursor)); - List out = new ArrayList<>(page.items().size()); - LinkedHashSet seenUris = new LinkedHashSet<>(); - for (TdengineRawFrameRow row : page.items()) { - String rawArchiveUri = row.rawUri(); - if (rawArchiveUri == null || rawArchiveUri.isBlank() || !seenUris.add(rawArchiveUri)) { - continue; - } - DecodedFrame frame = decodeIfAvailable(row, platformAccount); - if (frame != null) { - out.add(frame); - } - } - CursorResponse next = page.nextCursor() - .map(value -> new CursorResponse(value.ts().toString(), value.tieBreaker())) - .orElse(null); - return new DecodedFramePage(out, next); - } - - public List snapshots(QueryTimeRange range, - HistoryQueryOrder order, - int limit, - String vin, - String platformAccount) throws IOException { - return snapshots(range.dateFrom(), range.dateTo(), range.eventTimeFrom(), range.eventTimeTo(), - order, limit, vin, platformAccount); - } - - public List snapshotFields(QueryTimeRange range, - HistoryQueryOrder order, - int limit, - String vin, - String platformAccount, - String fields) throws IOException { - List selectedFields = parseSelectedFields(fields); - if (selectedFields.isEmpty()) { - throw new IllegalArgumentException("fields is required for gb32960 snapshot field query"); - } - validateSelectedFields(selectedFields); - return snapshots(range, order, limit, vin, platformAccount).stream() - .map(snapshot -> FieldSnapshot.from(snapshot, selectedFields)) - .toList(); - } - - public String snapshotFieldsCsv(QueryTimeRange range, - HistoryQueryOrder order, - int limit, - String vin, - String platformAccount, - String fields) throws IOException { - List selectedFields = parseSelectedFields(fields); - if (selectedFields.isEmpty()) { - throw new IllegalArgumentException("fields is required for gb32960 snapshot field query"); - } - validateSelectedFields(selectedFields); - List snapshots = snapshots(range, order, limit, vin, platformAccount).stream() - .map(snapshot -> FieldSnapshot.from(snapshot, selectedFields)) - .toList(); - Map headers = chineseHeadersByField(); - StringBuilder csv = new StringBuilder(256 + snapshots.size() * Math.max(128, selectedFields.size() * 32)); - csv.append("VIN,事件时间,首次接收时间,最后接收时间,原始包"); - for (String field : selectedFields) { - csv.append(',').append(csv(headers.getOrDefault(field, field))); - } - csv.append('\n'); - for (FieldSnapshot snapshot : snapshots) { - csv.append(csv(snapshot.vin())).append(',') - .append(csv(snapshot.eventTime())).append(',') - .append(csv(snapshot.ingestTimeFirst())).append(',') - .append(csv(snapshot.ingestTimeLast())).append(',') - .append(csv(sourceFrameUris(snapshot.sourceFrames()))); - for (String field : selectedFields) { - csv.append(',').append(csv(csvValue(snapshot.fields().get(field)))); - } - csv.append('\n'); - } - return csv.toString(); - } - - public DecodedFrame findByRawArchiveUri(String rawArchiveUri, String platformAccount) throws IOException { - if (rawArchiveUri == null || rawArchiveUri.isBlank()) { - return null; - } - return decodeWithoutIndex(rawArchiveUri, platformAccount); - } - - public List snapshots(LocalDate dateFrom, - LocalDate dateTo, - HistoryQueryOrder order, - int limit, - String vin, - String platformAccount) throws IOException { - if (vin == null || vin.isBlank()) { - throw new IllegalArgumentException("vin is required for gb32960 snapshots"); - } - return snapshots(dateFrom, dateTo, null, null, order, limit, vin, platformAccount); - } - - private List snapshots(LocalDate dateFrom, - LocalDate dateTo, - java.time.Instant eventTimeFrom, - java.time.Instant eventTimeTo, - HistoryQueryOrder order, - int limit, - String vin, - String platformAccount) throws IOException { - if (vin == null || vin.isBlank()) { - throw new IllegalArgumentException("vin is required for gb32960 snapshots"); - } - int snapshotLimit = Math.max(1, Math.min(limit, 1000)); - // 经验上一个完整 snapshot 可能由多个 RAW 子包组成，查询帧数需要放大后再按 snapshot 截断。 - List frames = query(dateFrom, dateTo, eventTimeFrom, eventTimeTo, - order, snapshotLimit * 30, vin, platformAccount); - LinkedHashMap builders = new LinkedHashMap<>(); - for (DecodedFrame frame : frames) { - if (frame.eventTime() == null || frame.eventTime().isBlank()) { - continue; - } - String key = frame.vin() + "|" + frame.eventTime(); - builders.computeIfAbsent(key, ignored -> new SnapshotBuilder(frame)).add(frame); - } - List out = new ArrayList<>(snapshotLimit); - for (SnapshotBuilder builder : builders.values()) { - out.add(builder.build()); - if (out.size() >= snapshotLimit) { - break; - } - } - return out; - } - - private DecodedFrame decodeIfAvailable(TdengineRawFrameRow row, String platformAccount) throws IOException { - try { - return decode(row, platformAccount); - } catch (NoSuchFileException e) { - log.warn("skip gb32960 frame because raw archive is missing frameId={} rawArchiveUri={}", - row.frameId(), row.rawUri()); - return null; - } - } - - private DecodedFrame decode(TdengineRawFrameRow row, String platformAccount) throws IOException { - Path rawPath = archivePath(row.rawUri()); - byte[] rawBytes = Files.readAllBytes(rawPath); - Gb32960Message message = decoder.decode(ByteBuffer.wrap(rawBytes), platformAccount(row, platformAccount)); - List> blocks = new ArrayList<>(); - for (InfoBlock block : message.infoBlocks()) { - blocks.add(blockJson(block)); - } - Instant ingestTime = row.receivedAt() == null ? Files.getLastModifiedTime(rawPath).toInstant() : row.receivedAt(); - return new DecodedFrame( - row.frameId(), - message.header().vin(), - message.header().command().name(), - message.header().responseFlag().name(), - message.header().protocolVersion().name(), - message.header().encryptType().name(), - message.header().dataLength(), - message.header().eventTime() == null ? null : message.header().eventTime().toString(), - ingestTime.toString(), - row.rawUri(), - rawBytes.length, - blocks); - } - - private DecodedFrame decodeWithoutIndex(String rawArchiveUri, String platformAccount) throws IOException { - Path rawPath = archivePath(rawArchiveUri); - byte[] rawBytes = Files.readAllBytes(rawPath); - Gb32960Message message = decoder.decode(ByteBuffer.wrap(rawBytes), platformAccount); - List> blocks = new ArrayList<>(); - for (InfoBlock block : message.infoBlocks()) { - blocks.add(blockJson(block)); - } - String ingestTime = Files.getLastModifiedTime(rawPath).toInstant().toString(); - return new DecodedFrame( - rawArchiveEventId(rawArchiveUri), - message.header().vin(), - message.header().command().name(), - message.header().responseFlag().name(), - message.header().protocolVersion().name(), - message.header().encryptType().name(), - message.header().dataLength(), - message.header().eventTime() == null ? null : message.header().eventTime().toString(), - ingestTime, - rawArchiveUri, - rawBytes.length, - blocks); - } - - private Map blockJson(InfoBlock block) { - Map json = new LinkedHashMap<>(); - json.put("type", block.type().name()); - json.put("className", block.getClass().getSimpleName()); - json.put("data", normalizeJsonValue(objectMapper.convertValue(block, MAP_TYPE))); - if (block instanceof InfoBlock.Raw raw) { - json.put("typeCode", raw.typeCode()); - json.put("bytesHex", hex(raw.bytes())); - } - return json; - } - - private static List parseSelectedFields(String fields) { - if (fields == null || fields.isBlank()) { - return List.of(); - } - List out = new ArrayList<>(); - for (String field : fields.split(",")) { - String normalized = field.trim(); - if (!normalized.isBlank()) { - out.add(normalized); - } - } - return List.copyOf(out); - } - - private static void validateSelectedFields(List selectedFields) { - LinkedHashSet knownFields = new LinkedHashSet<>(); - for (Gb32960FieldDictionary.Packet packet : Gb32960FieldDictionary.packets()) { - for (Gb32960FieldDictionary.Field field : packet.fields()) { - knownFields.add(packet.code() + "." + field.key()); - } - } - List unknown = selectedFields.stream() - .filter(field -> !knownFields.contains(field)) - .toList(); - if (!unknown.isEmpty()) { - throw new IllegalArgumentException("unknown gb32960 snapshot fields: " + String.join(",", unknown)); - } - } - - private static Map chineseHeadersByField() { - Map headers = new LinkedHashMap<>(); - for (Gb32960FieldDictionary.Packet packet : Gb32960FieldDictionary.packets()) { - for (Gb32960FieldDictionary.Field field : packet.fields()) { - String header = packet.nameZh() + "-" + field.nameZh(); - if (field.unit() != null && !field.unit().isBlank()) { - header += "(" + field.unit() + ")"; - } - headers.put(packet.code() + "." + field.key(), header); - } - } - return headers; - } - - private static String sourceFrameUris(List sourceFrames) { - if (sourceFrames == null || sourceFrames.isEmpty()) { - return ""; - } - return sourceFrames.stream() - .map(SourceFrame::rawArchiveUri) - .filter(uri -> uri != null && !uri.isBlank()) - .reduce((left, right) -> left + ";" + right) - .orElse(""); - } - - private static String csvValue(Object value) { - if (value == null) { - return ""; - } - if (value instanceof Iterable iterable) { - List values = new ArrayList<>(); - for (Object item : iterable) { - values.add(String.valueOf(item)); - } - return String.join(";", values); - } - return String.valueOf(value); - } - - private static String csv(String value) { - if (value == null) { - return ""; - } - boolean quote = value.indexOf(',') >= 0 - || value.indexOf('"') >= 0 - || value.indexOf('\n') >= 0 - || value.indexOf('\r') >= 0; - if (!quote) { - return value; - } - return "\"" + value.replace("\"", "\"\"") + "\""; - } - - static Object normalizeJsonValue(Object value) { - if (value instanceof Double d) { - return normalizeFloatingPoint(d); - } - if (value instanceof Float f) { - return normalizeFloatingPoint(f.doubleValue()); - } - if (value instanceof Map map) { - Map normalized = new LinkedHashMap<>(); - for (Map.Entry entry : map.entrySet()) { - normalized.put(String.valueOf(entry.getKey()), normalizeJsonValue(entry.getValue())); - } - return normalized; - } - if (value instanceof List list) { - List normalized = new ArrayList<>(list.size()); - for (Object item : list) { - normalized.add(normalizeJsonValue(item)); - } - return normalized; - } - return value; - } - - private static BigDecimal normalizeFloatingPoint(double value) { - if (!Double.isFinite(value)) { - return BigDecimal.ZERO; - } - // 协议缩放后的 double 容易出现 12.300000000000002；对外统一保留 6 位并去掉尾零。 - BigDecimal normalized = BigDecimal.valueOf(value) - .setScale(OUTPUT_DOUBLE_SCALE, RoundingMode.HALF_UP) - .stripTrailingZeros(); - return normalized.scale() < 0 ? normalized.setScale(0) : normalized; - } - - @SuppressWarnings("unchecked") - private Map deepCopy(Map value) { - return objectMapper.convertValue(value, MAP_TYPE); - } - - private Path archivePath(String rawArchiveUri) throws IOException { - String key = rawArchiveUri.startsWith("archive://") - ? rawArchiveUri.substring("archive://".length()) - : rawArchiveUri; - Path path = archiveRoot.resolve(key.replace("..", "_").replaceAll("^/+", "")).normalize(); - // archive:// 是外部输入，必须限制在 archiveRoot 内，避免通过 ../ 读取任意文件。 - if (!path.startsWith(archiveRoot)) { - throw new IOException("raw archive path escapes root: " + rawArchiveUri); - } - return path; - } - - private String platformAccount(TdengineRawFrameRow row, String override) { - if (override != null && !override.isBlank()) { - return override; - } - if (row.metadataJson() == null || row.metadataJson().isBlank()) { - return null; - } - try { - return objectMapper.readValue(row.metadataJson(), MAP_TYPE).getOrDefault("platformAccount", "").toString(); - } catch (IOException ignored) { - return null; - } - } - - private static Instant fromInstant(LocalDate dateFrom, Instant eventTimeFrom) { - return eventTimeFrom != null ? eventTimeFrom : dateFrom.atStartOfDay(DEFAULT_ZONE).toInstant(); - } - - private static Instant toExclusiveInstant(LocalDate dateTo, Instant eventTimeTo) { - Instant inclusive = eventTimeTo != null - ? eventTimeTo - : dateTo.plusDays(1).atStartOfDay(DEFAULT_ZONE).toInstant().minusMillis(1); - return inclusive.plusMillis(1); - } - - private static TdengineQueryOrder tdengineOrder(HistoryQueryOrder order) { - return order == HistoryQueryOrder.ASC ? TdengineQueryOrder.ASC : TdengineQueryOrder.DESC; - } - - private static String rawArchiveEventId(String rawArchiveUri) { - if (rawArchiveUri == null || rawArchiveUri.isBlank()) { - return ""; - } - String key = rawArchiveUri.startsWith("archive://") - ? rawArchiveUri.substring("archive://".length()) - : rawArchiveUri; - int slash = key.lastIndexOf('/'); - String fileName = slash >= 0 ? key.substring(slash + 1) : key; - return fileName.endsWith(".bin") ? fileName.substring(0, fileName.length() - 4) : fileName; - } - - private static String hex(byte[] bytes) { - if (bytes == null || bytes.length == 0) { - return ""; - } - StringBuilder out = new StringBuilder(bytes.length * 2); - for (byte b : bytes) { - out.append(Character.forDigit((b >> 4) & 0xF, 16)); - out.append(Character.forDigit(b & 0xF, 16)); - } - return out.toString(); - } - - public record DecodedFrame( - String eventId, - String vin, - String command, - String responseFlag, - String protocolVersion, - String encryptType, - int dataLength, - String eventTime, - String ingestTime, - String rawArchiveUri, - int rawSizeBytes, - List> blocks) { - } - - public record CursorResponse(String ts, String id) { - } - - public record DecodedFramePage( - List items, - CursorResponse nextCursor) { - } - - public record SourceFrame( - String eventId, - String ingestTime, - String rawArchiveUri, - int rawSizeBytes, - List blockTypes) { - } - - public record LogicalSnapshot( - String vin, - String command, - String responseFlag, - String protocolVersion, - String encryptType, - String eventTime, - String ingestTimeFirst, - String ingestTimeLast, - List sourceFrames, - List> blocks) { - } - - public record FieldSnapshot( - String vin, - String eventTime, - String ingestTimeFirst, - String ingestTimeLast, - List sourceFrames, - Map fields) { - - private static FieldSnapshot from(LogicalSnapshot snapshot, List selectedFields) { - Map values = new LinkedHashMap<>(); - for (String selectedField : selectedFields) { - values.put(selectedField, findField(snapshot.blocks(), selectedField)); - } - return new FieldSnapshot( - snapshot.vin(), - snapshot.eventTime(), - snapshot.ingestTimeFirst(), - snapshot.ingestTimeLast(), - snapshot.sourceFrames(), - values); - } - - private static Object findField(List> blocks, String selectedField) { - int dot = selectedField.indexOf('.'); - if (dot <= 0 || dot == selectedField.length() - 1) { - return null; - } - String type = selectedField.substring(0, dot); - String path = selectedField.substring(dot + 1); - for (Map block : blocks) { - if (type.equals(block.get("type"))) { - return valueAt(block.get("data"), path.split("\\.")); - } - } - return null; - } - - private static Object valueAt(Object current, String[] path) { - Object value = current; - for (String part : path) { - value = step(value, part); - if (value == null) { - return null; - } - } - return value; - } - - @SuppressWarnings("unchecked") - private static Object step(Object value, String part) { - if (value instanceof Map map) { - return map.get(part); - } - if (value instanceof List list) { - if (part.matches("\\d+")) { - int index = Integer.parseInt(part); - return index >= 0 && index < list.size() ? list.get(index) : null; - } - // 对数组字段支持 "stacks.cellVoltages" 这类投影，返回每个元素上对应属性的列表。 - List out = new ArrayList<>(list.size()); - for (Object item : list) { - if (item instanceof Map itemMap) { - out.add(((Map) itemMap).get(part)); - } - } - return out; - } - return null; - } - } - - private final class SnapshotBuilder { - private final DecodedFrame first; - private final List sourceFrames = new ArrayList<>(); - private final LinkedHashMap> blocks = new LinkedHashMap<>(); - - private SnapshotBuilder(DecodedFrame first) { - this.first = first; - } - - private void add(DecodedFrame frame) { - sourceFrames.add(new SourceFrame( - frame.eventId(), - frame.ingestTime(), - frame.rawArchiveUri(), - frame.rawSizeBytes(), - frame.blocks().stream().map(block -> String.valueOf(block.get("type"))).toList())); - for (Map block : frame.blocks()) { - String type = String.valueOf(block.get("type")); - if ("GD_FC_STACK".equals(type)) { - // 广东燃料电池堆电压可能按帧号拆成多个 RAW，这里按 cell 起始序号合并。 - mergeStack(block); - } else { - // 非分帧块同一 snapshot 内只保留第一次出现的块，避免后到空值覆盖前面完整值。 - blocks.putIfAbsent(type, deepCopy(block)); - } - } - } - - private void mergeStack(Map block) { - Map current = blocks.get("GD_FC_STACK"); - if (current == null) { - blocks.put("GD_FC_STACK", deepCopy(block)); - return; - } - Map currentData = map(current.get("data")); - Map incomingData = map(block.get("data")); - List> currentStacks = listOfMaps(currentData.get("stacks")); - List> incomingStacks = listOfMaps(incomingData.get("stacks")); - for (int i = 0; i < incomingStacks.size(); i++) { - if (i >= currentStacks.size()) { - currentStacks.add(deepCopy(incomingStacks.get(i))); - } else { - mergeStackItem(currentStacks.get(i), incomingStacks.get(i)); - } - } - currentData.put("stackCount", currentStacks.size()); - } - - private void mergeStackItem(Map target, Map incoming) { - int targetStart = intValue(target.get("frameCellStart"), 1); - int incomingStart = intValue(incoming.get("frameCellStart"), 1); - List targetVoltages = doubleList(target.get("frameCellVoltagesV")); - List incomingVoltages = doubleList(incoming.get("frameCellVoltagesV")); - // GB/T 分帧的起始序号是 1-based；内部 List 是 0-based，所以写入时统一减一。 - int cellCount = Math.max(intValue(target.get("cellCount"), 0), intValue(incoming.get("cellCount"), 0)); - int needed = Math.max(cellCount, Math.max( - targetStart - 1 + targetVoltages.size(), - incomingStart - 1 + incomingVoltages.size())); - List merged = new ArrayList<>(needed); - for (int i = 0; i < needed; i++) { - merged.add(null); - } - putVoltages(merged, targetStart, targetVoltages); - putVoltages(merged, incomingStart, incomingVoltages); - while (!merged.isEmpty() && merged.getLast() == null) { - merged.removeLast(); - } - target.put("frameCellStart", 1); - target.put("frameCellCount", merged.size()); - target.put("frameCellVoltagesV", merged); - target.put("cellCount", Math.max(cellCount, merged.size())); - normalizeStackItem(target); - } - - private LogicalSnapshot build() { - List orderedFrames = sourceFrames.stream() - .sorted(Comparator.comparing(SourceFrame::ingestTime)) - .toList(); - Map stackBlock = blocks.get("GD_FC_STACK"); - if (stackBlock != null) { - normalizeStackBlock(stackBlock); - } - List> orderedBlocks = new ArrayList<>(blocks.values()); - orderedBlocks.sort(Comparator.comparingInt(block -> blockOrder(String.valueOf(block.get("type"))))); - List> normalizedBlocks = orderedBlocks.stream() - .map(block -> map(normalizeJsonValue(block))) - .toList(); - return new LogicalSnapshot( - first.vin(), - first.command(), - first.responseFlag(), - first.protocolVersion(), - first.encryptType(), - first.eventTime(), - orderedFrames.isEmpty() ? null : orderedFrames.getFirst().ingestTime(), - orderedFrames.isEmpty() ? null : orderedFrames.getLast().ingestTime(), - sourceFrames, - normalizedBlocks); - } - } - - private static void normalizeStackBlock(Map stackBlock) { - Map data = map(stackBlock.get("data")); - for (Map stack : listOfMaps(data.get("stacks"))) { - normalizeStackItem(stack); - } - } - - private static void normalizeStackItem(Map target) { - int start = intValue(target.get("frameCellStart"), 1); - List voltages = doubleList(target.get("frameCellVoltagesV")); - int cellCount = intValue(target.get("cellCount"), 0); - int needed = Math.max(cellCount, start - 1 + voltages.size()); - List normalized = new ArrayList<>(needed); - for (int i = 0; i < needed; i++) { - normalized.add(null); - } - putVoltages(normalized, start, voltages); - while (cellCount == 0 && !normalized.isEmpty() && normalized.getLast() == null) { - normalized.removeLast(); - } - long missing = normalized.stream().filter(item -> item == null).count(); - // 给前端明确的完整性标记，避免只能靠数组长度猜测是否缺帧。 - target.put("frameCellStart", 1); - target.put("frameCellCount", normalized.size()); - target.put("frameCellVoltagesV", normalized); - target.put("cellCount", Math.max(cellCount, normalized.size())); - target.put("frameCellComplete", missing == 0 && (cellCount == 0 || normalized.size() >= cellCount)); - target.put("frameCellMissingCount", missing); - } - - @SuppressWarnings("unchecked") - private static Map map(Object value) { - return value instanceof Map m ? (Map) m : new LinkedHashMap<>(); - } - - @SuppressWarnings("unchecked") - private static List> listOfMaps(Object value) { - return value instanceof List list ? (List>) list : new ArrayList<>(); - } - - private static int intValue(Object value, int fallback) { - return value instanceof Number n ? n.intValue() : fallback; - } - - private static List doubleList(Object value) { - if (!(value instanceof List list)) { - return List.of(); - } - List out = new ArrayList<>(list.size()); - for (Object item : list) { - out.add(item instanceof Number n ? n.doubleValue() : null); - } - return out; - } - - private static void putVoltages(List target, int start, List values) { - int offset = Math.max(0, start - 1); - for (int i = 0; i < values.size(); i++) { - int index = offset + i; - if (index >= target.size()) { - target.add(values.get(i)); - } else if (values.get(i) != null) { - target.set(index, values.get(i)); - } - } - } - - private static int blockOrder(String type) { - return switch (type) { - case "VEHICLE" -> 10; - case "DRIVE_MOTOR" -> 20; - case "FUEL_CELL_V2016", "FUEL_CELL_V2025" -> 30; - case "ENGINE" -> 40; - case "POSITION_V2016", "POSITION_V2025" -> 50; - case "EXTREME_V2016" -> 60; - case "ALARM_V2016", "ALARM_V2025" -> 70; - case "VOLTAGE_V2016", "MIN_PARALLEL_VOLTAGE_V2025" -> 80; - case "TEMPERATURE_V2016", "BATTERY_TEMPERATURE_V2025" -> 90; - case "FUEL_CELL_STACK", "GD_FC_STACK" -> 100; - case "SUPER_CAPACITOR" -> 110; - case "SUPER_CAPACITOR_EXTREME" -> 120; - case "GD_FC_AUXILIARY" -> 130; - case "GD_FC_DCDC" -> 140; - case "GD_FC_AIR_CONDITIONER" -> 150; - case "GD_FC_VEHICLE_INFO" -> 160; - case "GD_FC_DEMO_EXTENSION" -> 170; - case "GD_FC_VENDOR_TLV" -> 180; - default -> 1000; - }; - } -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Gb32960FieldDictionary.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Gb32960FieldDictionary.java deleted file mode 100644 index f131a1d0..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Gb32960FieldDictionary.java +++ /dev/null @@ -1,207 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import java.util.List; - -/** - * GB32960 字段字典。 - * - *

    字典是前端字段选择器、中文表头、Tooltip、枚举/位图展示的单一来源。 - * 字段 key 必须和 {@link Gb32960DecodedFrameService} 输出的 block data 路径一致； - * 支持点号路径和数组投影路径，例如 {@code GD_FC_STACK.stacks.frameCellVoltagesV}。 - */ -public final class Gb32960FieldDictionary { - - private Gb32960FieldDictionary() { - } - - public static List packets() { - return PACKETS; - } - - private static final List PACKETS = List.of( - // 标准 32960 字段优先放前面；广东燃料电池扩展字段放后面，方便前端按常用程度展示。 - packet("VEHICLE", "整车数据", "GB/T 32960 整车运行状态、车速、里程、电压、电流、SOC 等核心字段。", - field("vehicleState", "车辆状态", "", "车辆运行、停止等状态编码。", - mappings( - mapping("01", "1", "启动", "车辆处于可行驶启动状态。"), - mapping("02", "2", "熄火", "车辆处于熄火状态。"), - mapping("03", "3", "其他", "车辆处于其他状态。"), - mapping("FE", "254", "异常", "协议异常值。"), - mapping("FF", "255", "无效", "协议无效值。"))), - field("chargingState", "充电状态", "", "车辆充电状态编码。", - mappings( - mapping("01", "1", "停车充电", "车辆停车充电。"), - mapping("02", "2", "行驶充电", "车辆行驶充电。"), - mapping("03", "3", "未充电", "车辆未充电。"), - mapping("04", "4", "充电完成", "车辆充电完成。"), - mapping("FE", "254", "异常", "协议异常值。"), - mapping("FF", "255", "无效", "协议无效值。"))), - field("runningMode", "运行模式", "", "纯电、混动、燃料电池等运行模式编码。", - mappings( - mapping("01", "1", "纯电", "纯电驱动模式。"), - mapping("02", "2", "混合动力", "混合动力驱动模式。"), - mapping("03", "3", "燃料电池", "燃料电池驱动模式。"), - mapping("FE", "254", "异常", "协议异常值。"), - mapping("FF", "255", "无效", "协议无效值。"))), - field("speedKmh", "车速", "km/h", "车辆当前速度。"), - field("totalMileageKm", "累计里程", "km", "车辆累计行驶里程。"), - field("totalVoltageV", "总电压", "V", "动力系统总电压。"), - field("totalCurrentA", "总电流", "A", "动力系统总电流。"), - field("socPercent", "SOC", "%", "动力电池荷电状态。"), - field("dcDcStatus", "DC/DC 状态", "", "DC/DC 工作状态编码。", - mappings( - mapping("01", "1", "工作", "DC/DC 处于工作状态。"), - mapping("02", "2", "断开", "DC/DC 处于断开状态。"), - mapping("FE", "254", "异常", "协议异常值。"), - mapping("FF", "255", "无效", "协议无效值。"))), - field("gearRaw", "挡位原始值", "", "挡位、驱动力和制动力组合原始编码。"), - field("insulationResistanceKohm", "绝缘电阻", "kOhm", "高压系统绝缘电阻。"), - field("acceleratorPedal", "加速踏板行程", "%", "2016 版字段，加速踏板开度。"), - field("brakePedal", "制动踏板状态", "%", "2016 版字段，制动踏板状态或开度。")), - packet("DRIVE_MOTOR", "驱动电机数据", "GB/T 32960 驱动电机控制器和电机运行状态。", - field("motors.serialNo", "驱动电机序号", "", "电机编号。"), - field("motors.state", "驱动电机状态", "", "电机工作状态编码。", - mappings( - mapping("01", "1", "耗电", "驱动电机耗电。"), - mapping("02", "2", "发电", "驱动电机发电。"), - mapping("03", "3", "关闭", "驱动电机关闭。"), - mapping("04", "4", "准备", "驱动电机准备。"), - mapping("FE", "254", "异常", "协议异常值。"), - mapping("FF", "255", "无效", "协议无效值。"))), - field("motors.controllerTempC", "控制器温度", "C", "驱动电机控制器温度。"), - field("motors.rpm", "转速", "rpm", "驱动电机转速。"), - field("motors.torqueNm", "转矩", "N.m", "驱动电机输出转矩。"), - field("motors.motorTempC", "电机温度", "C", "驱动电机温度。"), - field("motors.controllerInputVoltageV", "控制器输入电压", "V", "电机控制器直流母线输入电压。"), - field("motors.controllerDcCurrentA", "控制器直流母线电流", "A", "电机控制器直流母线电流。")), - packet("FUEL_CELL_V2016", "燃料电池数据（2016）", "GB/T 32960.3-2016 燃料电池电压、电流、氢耗、温度、浓度、压力等字段。", - field("fcVoltageV", "燃料电池电压", "V", "燃料电池系统输出电压。"), - field("fcCurrentA", "燃料电池电流", "A", "燃料电池系统输出电流。"), - field("hydrogenConsumptionKgPer100km", "氢耗", "kg/100km", "燃料电池氢气消耗率。"), - field("probeTempC", "探针温度", "C", "燃料电池温度探针列表。"), - field("maxHydrogenSystemTempC", "氢系统最高温度", "C", "氢系统最高温度。"), - field("maxHydrogenConcentrationPercent", "最高氢浓度", "%", "氢气浓度最高值。"), - field("maxHydrogenPressureMpa", "最高氢压力", "MPa", "氢系统最高压力。"), - field("hvDcDcStatus", "高压 DC/DC 状态", "", "燃料电池高压 DC/DC 状态。")), - packet("POSITION_V2016", "车辆位置数据（2016）", "GB/T 32960.3-2016 定位状态和经纬度。", - field("statusFlag", "定位状态标志", "", "定位有效性及经纬度方向标志。", - mappings( - mapping("bit0=0", "bit0=0", "有效定位", "定位状态有效。"), - mapping("bit0=1", "bit0=1", "无效定位", "定位状态无效。"), - mapping("bit1=0", "bit1=0", "北纬", "纬度方向为北纬。"), - mapping("bit1=1", "bit1=1", "南纬", "纬度方向为南纬。"), - mapping("bit2=0", "bit2=0", "东经", "经度方向为东经。"), - mapping("bit2=1", "bit2=1", "西经", "经度方向为西经。"))), - field("longitude", "经度", "deg", "车辆经度。"), - field("latitude", "纬度", "deg", "车辆纬度。")), - packet("POSITION_V2025", "车辆位置数据（2025）", "GB/T 32960.3-2025 定位状态、坐标系和经纬度。", - field("statusFlag", "定位状态标志", "", "定位有效性及经纬度方向标志。", - mappings( - mapping("bit0=0", "bit0=0", "有效定位", "定位状态有效。"), - mapping("bit0=1", "bit0=1", "无效定位", "定位状态无效。"), - mapping("bit1=0", "bit1=0", "北纬", "纬度方向为北纬。"), - mapping("bit1=1", "bit1=1", "南纬", "纬度方向为南纬。"), - mapping("bit2=0", "bit2=0", "东经", "经度方向为东经。"), - mapping("bit2=1", "bit2=1", "西经", "经度方向为西经。"))), - field("coordSystem", "坐标系", "", "2025 版新增坐标系编码。", - mappings( - mapping("01", "1", "WGS84", "WGS84 坐标系。"), - mapping("02", "2", "GCJ-02", "GCJ-02 坐标系。"), - mapping("03", "3", "其他", "其他坐标系。"))), - field("longitude", "经度", "deg", "车辆经度。"), - field("latitude", "纬度", "deg", "车辆纬度。")), - packet("ALARM_V2016", "报警数据（2016）", "GB/T 32960.3-2016 最高报警等级、通用报警位和故障码列表。", - field("maxLevel", "最高报警等级", "", "当前最高报警等级。", - mappings( - mapping("00", "0", "无故障", "当前无报警故障。"), - mapping("01", "1", "一级故障", "一级报警故障。"), - mapping("02", "2", "二级故障", "二级报警故障。"), - mapping("03", "3", "三级故障", "三级报警故障。"))), - field("generalAlarmFlag", "通用报警标志", "", "通用报警位图。"), - field("batteryFaults", "可充电储能装置故障码", "", "动力电池相关故障码列表。"), - field("motorFaults", "驱动电机故障码", "", "驱动电机相关故障码列表。"), - field("engineFaults", "发动机故障码", "", "发动机相关故障码列表。"), - field("otherFaults", "其他故障码", "", "其他故障码列表。")), - packet("GD_FC_STACK", "广东燃料电池电堆数据", "广东燃料电池汽车数据接入规范扩展，描述电堆状态、压力、温度和单体电压分段。", - field("stackCount", "电堆数量", "", "本包包含的电堆数量。"), - field("stacks.engineWorkState", "发动机工作状态", "", "燃料电池发动机工作状态。"), - field("stacks.stackWaterOutletTempC", "电堆出水温度", "C", "电堆冷却水出口温度。"), - field("stacks.hydrogenInletPressureKpa", "氢气入口压力", "kPa", "电堆氢气入口压力。"), - field("stacks.airInletPressureKpa", "空气入口压力", "kPa", "电堆空气入口压力。"), - field("stacks.airInletTempC", "空气入口温度", "C", "电堆空气入口温度。"), - field("stacks.maxCellVoltageV", "单体最高电压", "V", "本电堆单体最高电压。"), - field("stacks.minCellVoltageV", "单体最低电压", "V", "本电堆单体最低电压。"), - field("stacks.avgCellVoltageV", "单体平均电压", "V", "本电堆单体平均电压。"), - field("stacks.cellCount", "单体总数", "", "电堆单体总数量。"), - field("stacks.frameCellStart", "本帧单体起始序号", "", "当前分段电压起始单体序号。"), - field("stacks.frameCellCount", "本帧单体数量", "", "当前分段包含的单体数量。"), - field("stacks.frameCellVoltagesV", "本帧单体电压列表", "V", "当前分段的单体电压数组。"), - field("stacks.frameCellComplete", "单体电压是否完整", "", "snapshot 合并后是否已覆盖全部单体。"), - field("stacks.frameCellMissingCount", "缺失单体数量", "", "snapshot 合并后仍缺失的单体数量。")), - packet("GD_FC_DCDC", "广东燃料电池 DC/DC 数据", "广东燃料电池汽车数据接入规范扩展，描述 DC/DC 输入输出和控制器温度。", - field("inputVoltageV", "输入电压", "V", "DC/DC 输入电压。"), - field("inputCurrentA", "输入电流", "A", "DC/DC 输入电流。"), - field("outputVoltageV", "输出电压", "V", "DC/DC 输出电压。"), - field("outputCurrentA", "输出电流", "A", "DC/DC 输出电流。"), - field("controllerTempC", "控制器温度", "C", "DC/DC 控制器温度。")), - packet("GD_FC_AIR_CONDITIONER", "广东燃料电池空调数据", "广东燃料电池汽车数据接入规范扩展，描述空调状态、功率和压缩机输入电压。", - field("status", "空调状态", "", "空调工作状态。", - mappings( - mapping("00", "0", "关闭", "空调关闭。"), - mapping("01", "1", "启动", "空调启动。"), - mapping("FE", "254", "异常", "协议异常值。"), - mapping("FF", "255", "无效", "协议无效值。"))), - field("powerKw", "空调功率", "kW", "空调消耗功率。"), - field("compressorInputVoltageV", "压缩机输入电压", "V", "空调压缩机输入电压。")), - packet("GD_FC_VEHICLE_INFO", "广东燃料电池车辆信息数据", "广东燃料电池汽车数据接入规范扩展，描述碰撞报警、环境温度压力和车载氢量。", - field("collisionAlarm", "碰撞报警", "", "碰撞报警状态。", - mappings( - mapping("00", "0", "无碰撞报警", "当前无碰撞报警。"), - mapping("01", "1", "有碰撞报警", "当前有碰撞报警。"), - mapping("FE", "254", "异常", "协议异常值。"), - mapping("FF", "255", "无效", "协议无效值。"))), - field("ambientTempC", "环境温度", "C", "车辆环境温度。"), - field("ambientPressureKpa", "环境压力", "kPa", "车辆环境压力。"), - field("hydrogenMassKg", "车载氢量", "kg", "当前车载氢气质量。")) - ); - - private static Packet packet(String code, String nameZh, String description, Field... fields) { - return new Packet(code, nameZh, description, List.of(fields)); - } - - private static Field field(String key, String nameZh, String unit, String description) { - return field(key, nameZh, unit, description, List.of()); - } - - private static Field field(String key, String nameZh, String unit, String description, - List valueMappings) { - return new Field(key, nameZh, unit, description, valueMappings); - } - - private static List mappings(ValueMapping... mappings) { - return List.of(mappings); - } - - private static ValueMapping mapping(String code, String value, String nameZh, String description) { - return new ValueMapping(code, value, nameZh, description); - } - - /** - * 一个数据包/信息体类型的字段集合，例如 VEHICLE 或 GD_FC_DCDC。 - */ - public record Packet(String code, String nameZh, String description, List fields) { - } - - /** - * 可查询字段定义。key 是接口查询参数使用的英文路径，nameZh/unit/description 用于前端展示。 - */ - public record Field(String key, String nameZh, String unit, String description, - List valueMappings) { - } - - /** - * 协议枚举值或位图值的展示映射。code 保留原始协议表示，value 是解析后的业务值。 - */ - public record ValueMapping(String code, String value, String nameZh, String description) { - } -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Gb32960FrameController.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Gb32960FrameController.java deleted file mode 100644 index 9daf98a9..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Gb32960FrameController.java +++ /dev/null @@ -1,214 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.lingniu.ingest.tdenginehistory.TdenginePageCursor; -import io.swagger.v3.oas.annotations.Operation; -import io.swagger.v3.oas.annotations.Parameter; -import io.swagger.v3.oas.annotations.tags.Tag; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.http.HttpStatus; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.RequestParam; -import org.springframework.web.bind.annotation.RestController; -import org.springframework.web.server.ResponseStatusException; - -import java.io.IOException; -import java.time.Instant; -import java.util.List; - -/** - * GB32960 历史查询 HTTP 边界。 - * - *

    这个 Controller 只做参数校验、时间范围解析和 Swagger 说明，真正的 RAW 读取、 - * 协议解码、snapshot 合并、字段投影都放在 {@link Gb32960DecodedFrameService}。 - * 这样可以保证接口层足够薄，后续如果要把查询服务拆出去，只需要迁移 service。 - */ -@RestController -@ConditionalOnProperty(prefix = "lingniu.ingest.event-history", name = "enabled", havingValue = "true") -@ConditionalOnProperty(prefix = "lingniu.ingest.event-history.api", name = "specialized-enabled", havingValue = "true") -@ConditionalOnBean(Gb32960DecodedFrameService.class) -@RequestMapping("/api/event-history/gb32960") -@Tag(name = "gb-32960-frame-controller", description = "GB32960 专用历史帧、业务快照、字段查询和字段字典接口。") -public final class Gb32960FrameController { - - private final Gb32960DecodedFrameService service; - - public Gb32960FrameController(Gb32960DecodedFrameService service) { - if (service == null) { - throw new IllegalArgumentException("service must not be null"); - } - this.service = service; - } - - @GetMapping("/frames") - @Operation( - summary = "查询 GB32960 原始帧解析结果", - description = "按时间范围查询某辆车或账号下的 GB32960 原始帧，并返回每帧解析出的整包 blocks。主要用于排查原始报文、解析结果和子包内容。") - public List frames( - @Parameter(description = "开始时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-23T10:48:00") - @RequestParam String dateFrom, - @Parameter(description = "结束时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-23T10:49:00") - @RequestParam String dateTo, - @Parameter(description = "排序方向。", example = "DESC") - @RequestParam(defaultValue = "DESC") HistoryQueryOrder order, - @Parameter(description = "返回帧数量上限。", example = "10") - @RequestParam(defaultValue = "10") int limit, - @Parameter(description = "车辆 VIN；用于缩小到单车查询。", example = "LB9A32A20P0LS1257") - @RequestParam(required = false) String vin, - @Parameter(description = "平台账号；用于选择对应厂商扩展解析规则。", example = "Hyundai") - @RequestParam(required = false) String platformAccount) throws IOException { - QueryTimeRange range = QueryTimeRange.parse(dateFrom, dateTo); - return service.query( - range.dateFrom(), - range.dateTo(), - range.eventTimeFrom(), - range.eventTimeTo(), - order, - limit, - vin, - platformAccount); - } - - @GetMapping("/frames/page") - @Operation( - summary = "分页查询 GB32960 原始帧解析结果", - description = "按 VIN 和时间范围分页查询 TDengine raw_frames，并回读 archive 原始包即时解析。" - + "分页使用 cursorTs + cursorId，不使用 offset；适合生产热历史连续翻页。") - public Gb32960DecodedFrameService.DecodedFramePage framesPage( - @Parameter(description = "开始时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-23T10:48:00") - @RequestParam String dateFrom, - @Parameter(description = "结束时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-23T10:49:00") - @RequestParam String dateTo, - @Parameter(description = "排序方向。", example = "DESC") - @RequestParam(defaultValue = "DESC") HistoryQueryOrder order, - @Parameter(description = "返回帧数量上限，最大 1000。", example = "100") - @RequestParam(defaultValue = "100") int limit, - @Parameter(description = "车辆 VIN，必填；用于命中 TDengine raw child table。", required = true, example = "LB9A32A20P0LS1257") - @RequestParam String vin, - @Parameter(description = "平台账号；用于选择对应厂商扩展解析规则。", example = "Hyundai") - @RequestParam(required = false) String platformAccount, - @Parameter(description = "上一页返回的 nextCursor.ts。", example = "2026-06-29T05:00:01Z") - @RequestParam(required = false) String cursorTs, - @Parameter(description = "上一页返回的 nextCursor.id。", example = "gb32960-frame-xxx") - @RequestParam(required = false) String cursorId) throws IOException { - if (vin == null || vin.isBlank()) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "vin is required for gb32960 frame page query"); - } - return service.queryPage( - QueryTimeRange.parse(dateFrom, dateTo), - order, - limit, - vin, - platformAccount, - cursor(cursorTs, cursorId)); - } - - @GetMapping("/frame") - @Operation( - summary = "按 rawArchiveUri 查询单个 GB32960 原始帧", - description = "直接读取指定 archive://*.bin 原始包并返回解析结果。主要用于从 snapshots 的 sourceFrames 回查某个原始报文。") - public Gb32960DecodedFrameService.DecodedFrame frame( - @Parameter(description = "原始包归档地址。", example = "archive://2026/06/23/GB32960/LB9A32A20P0LS1257/1782182749928000.bin") - @RequestParam String rawArchiveUri, - @Parameter(description = "平台账号；用于选择对应厂商扩展解析规则。", example = "Hyundai") - @RequestParam(required = false) String platformAccount) throws IOException { - return service.findByRawArchiveUri(rawArchiveUri, platformAccount); - } - - private static TdenginePageCursor cursor(String cursorTs, String cursorId) { - boolean hasTs = cursorTs != null && !cursorTs.isBlank(); - boolean hasId = cursorId != null && !cursorId.isBlank(); - if (!hasTs && !hasId) { - return null; - } - if (!hasTs || !hasId) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "cursorTs and cursorId must be provided together"); - } - return new TdenginePageCursor(Instant.parse(cursorTs.trim()), cursorId.trim()); - } - - @GetMapping("/snapshots") - @Operation( - summary = "查询 GB32960 完整业务快照", - description = "按 VIN 和时间范围查询完整业务快照。会把同一车辆、同一 eventTime 的多个子包合并为一个 snapshot，适合车辆详情页和某一时刻全字段查看。VIN 为必填。") - public List snapshots( - @Parameter(description = "开始时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-23T10:48:00") - @RequestParam String dateFrom, - @Parameter(description = "结束时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-23T10:49:00") - @RequestParam String dateTo, - @Parameter(description = "排序方向。", example = "DESC") - @RequestParam(defaultValue = "DESC") HistoryQueryOrder order, - @Parameter(description = "返回 snapshot 数量上限。", example = "10") - @RequestParam(defaultValue = "10") int limit, - @Parameter(description = "车辆 VIN，必填；snapshot 查询只支持单车，避免跨车合并和低效扫描。", required = true, example = "LB9A32A20P0LS1257") - @RequestParam String vin, - @Parameter(description = "平台账号；用于选择对应厂商扩展解析规则。", example = "Hyundai") - @RequestParam(required = false) String platformAccount) throws IOException { - // snapshot 是按同一 VIN + eventTime 合并多个 RAW 子包，跨车查询会造成错误合并和低效扫描。 - if (vin == null || vin.isBlank()) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "vin is required for gb32960 snapshots"); - } - return service.snapshots(QueryTimeRange.parse(dateFrom, dateTo), order, limit, vin, platformAccount); - } - - @GetMapping("/snapshots/fields") - @Operation( - summary = "按所需字段查询 GB32960 快照", - description = "高频业务查询接口。按 VIN、时间范围和字段列表返回轻量快照，只包含 fields 指定字段；字段格式为 数据包代码.字段名，例如 VEHICLE.totalMileageKm,GD_FC_DCDC.inputVoltageV。字段名应来自 dictionary 接口。VIN 和 fields 为必填。") - public List snapshotFields( - @Parameter(description = "开始时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-23T10:48:00") - @RequestParam String dateFrom, - @Parameter(description = "结束时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-23T10:49:00") - @RequestParam String dateTo, - @Parameter(description = "排序方向。", example = "DESC") - @RequestParam(defaultValue = "DESC") HistoryQueryOrder order, - @Parameter(description = "返回 snapshot 数量上限。", example = "10") - @RequestParam(defaultValue = "10") int limit, - @Parameter(description = "车辆 VIN，必填；字段查询只支持单车。", required = true, example = "LB9A32A20P0LS1257") - @RequestParam String vin, - @Parameter(description = "逗号分隔的字段列表，字段来自 dictionary；格式为 数据包代码.字段名。", required = true, example = "VEHICLE.totalMileageKm,GD_FC_DCDC.inputVoltageV,GD_FC_DCDC.outputCurrentA") - @RequestParam String fields, - @Parameter(description = "平台账号；用于选择对应厂商扩展解析规则。", example = "Hyundai") - @RequestParam(required = false) String platformAccount) throws IOException { - // 字段投影接口是前端高频接口，强制单车查询可以命中 VIN 分区，避免全库扫描后再 join。 - if (vin == null || vin.isBlank()) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "vin is required for gb32960 snapshot field query"); - } - return service.snapshotFields(QueryTimeRange.parse(dateFrom, dateTo), order, limit, vin, platformAccount, fields); - } - - @GetMapping(value = "/snapshots/fields.csv", produces = "text/csv;charset=UTF-8") - @Operation( - summary = "导出 GB32960 指定字段快照 CSV", - description = "参数与 snapshots/fields 一致，返回 CSV 文本。基础列和字段列表表头均为中文，字段表头来自 dictionary，例如 整车数据-累计里程(km)。适合前端下载和临时报表。") - public String snapshotFieldsCsv( - @Parameter(description = "开始时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-23T10:48:00") - @RequestParam String dateFrom, - @Parameter(description = "结束时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-23T10:49:00") - @RequestParam String dateTo, - @Parameter(description = "排序方向。", example = "DESC") - @RequestParam(defaultValue = "DESC") HistoryQueryOrder order, - @Parameter(description = "返回 snapshot 数量上限。", example = "1000") - @RequestParam(defaultValue = "1000") int limit, - @Parameter(description = "车辆 VIN，必填；字段导出只支持单车。", required = true, example = "LB9A32A20P0LS1257") - @RequestParam String vin, - @Parameter(description = "逗号分隔的字段列表，字段来自 dictionary；格式为 数据包代码.字段名。", required = true, example = "VEHICLE.totalMileageKm,GD_FC_DCDC.inputVoltageV,GD_FC_DCDC.outputCurrentA") - @RequestParam String fields, - @Parameter(description = "平台账号；用于选择对应厂商扩展解析规则。", example = "Hyundai") - @RequestParam(required = false) String platformAccount) throws IOException { - // CSV 复用字段投影逻辑，表头中文化由 dictionary 提供，避免导出和页面展示出现两套字段定义。 - if (vin == null || vin.isBlank()) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "vin is required for gb32960 snapshot field csv export"); - } - return service.snapshotFieldsCsv(QueryTimeRange.parse(dateFrom, dateTo), order, limit, vin, platformAccount, fields); - } - - @GetMapping("/dictionary") - @Operation( - summary = "查询 GB32960 字段字典", - description = "返回数据包代码、数据包中文名、字段 key、中文名、单位、解释和枚举/位图值域对照。前端应用可用它渲染字段选择器、表头、Tooltip 和状态码中文展示。") - public List dictionary() { - return Gb32960FieldDictionary.packets(); - } -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/HistoryQueryOrder.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/HistoryQueryOrder.java deleted file mode 100644 index 8b41219a..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/HistoryQueryOrder.java +++ /dev/null @@ -1,6 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -public enum HistoryQueryOrder { - ASC, - DESC -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Jt808LocationHistoryController.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Jt808LocationHistoryController.java deleted file mode 100644 index 613b91ce..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Jt808LocationHistoryController.java +++ /dev/null @@ -1,148 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineLocationQuery; -import com.lingniu.ingest.tdenginehistory.TdengineLocationRow; -import com.lingniu.ingest.tdenginehistory.TdenginePage; -import com.lingniu.ingest.tdenginehistory.TdenginePageCursor; -import com.lingniu.ingest.tdenginehistory.TdengineQueryOrder; -import io.swagger.v3.oas.annotations.Operation; -import io.swagger.v3.oas.annotations.Parameter; -import io.swagger.v3.oas.annotations.tags.Tag; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.http.HttpStatus; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.RequestParam; -import org.springframework.web.bind.annotation.RestController; -import org.springframework.web.server.ResponseStatusException; - -import java.io.IOException; -import java.time.Instant; -import java.util.List; - -@RestController -@ConditionalOnProperty(prefix = "lingniu.ingest.event-history", name = "enabled", havingValue = "true") -@ConditionalOnProperty(prefix = "lingniu.ingest.event-history.api", name = "specialized-enabled", havingValue = "true") -@ConditionalOnBean(TdengineHistoryReader.class) -@RequestMapping("/api/event-history/jt808") -@Tag(name = "jt-808-location-controller", description = "JT808 位置历史分页查询接口。") -public final class Jt808LocationHistoryController { - - private final TdengineHistoryReader reader; - - public Jt808LocationHistoryController(TdengineHistoryReader reader) { - if (reader == null) { - throw new IllegalArgumentException("reader must not be null"); - } - this.reader = reader; - } - - @GetMapping("/locations") - @Operation( - summary = "查询 JT808 位置历史", - description = "按终端手机号和时间范围查询 TDengine 中的 JT808 位置点。分页使用 cursorTs + cursorId，不使用 offset，适合高并发历史轨迹查询。") - public LocationPageResponse locations( - @Parameter(description = "JT808 终端手机号或终端标识；内部会映射到 jt808: 车辆键。", required = true, example = "g7gps") - @RequestParam String phone, - @Parameter(description = "开始时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-29T13:00:00") - @RequestParam String dateFrom, - @Parameter(description = "结束时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-29T13:10:00") - @RequestParam String dateTo, - @Parameter(description = "排序方向。", example = "DESC") - @RequestParam(defaultValue = "DESC") TdengineQueryOrder order, - @Parameter(description = "返回位置点数量上限，最大 1000。", example = "100") - @RequestParam(defaultValue = "100") int limit, - @Parameter(description = "上一页返回的 nextCursor.ts。", example = "2026-06-29T05:00:01Z") - @RequestParam(required = false) String cursorTs, - @Parameter(description = "上一页返回的 nextCursor.id。", example = "jt808-location-xxx") - @RequestParam(required = false) String cursorId) throws IOException { - if (phone == null || phone.isBlank()) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "phone is required for jt808 location query"); - } - QueryTimeRange range = QueryTimeRange.parse(dateFrom, dateTo); - TdenginePage page = reader.queryLocations(new TdengineLocationQuery( - "JT808", - vehicleKey(phone), - range.eventTimeFrom(), - range.eventTimeTo().plusMillis(1), - order, - limit, - cursor(cursorTs, cursorId))); - return LocationPageResponse.from(page); - } - - private static String vehicleKey(String phone) { - String value = phone.trim(); - return value.startsWith("jt808:") ? value : "jt808:" + value; - } - - private static TdenginePageCursor cursor(String cursorTs, String cursorId) { - boolean hasTs = cursorTs != null && !cursorTs.isBlank(); - boolean hasId = cursorId != null && !cursorId.isBlank(); - if (!hasTs && !hasId) { - return null; - } - if (!hasTs || !hasId) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "cursorTs and cursorId must be provided together"); - } - return new TdenginePageCursor(Instant.parse(cursorTs.trim()), cursorId.trim()); - } - - public record LocationPageResponse( - List items, - CursorResponse nextCursor) { - - private static LocationPageResponse from(TdenginePage page) { - CursorResponse next = page.nextCursor() - .map(cursor -> new CursorResponse(cursor.ts().toString(), cursor.tieBreaker())) - .orElse(null); - return new LocationPageResponse( - page.items().stream().map(LocationResponse::from).toList(), - next); - } - } - - public record CursorResponse(String ts, String id) { - } - - public record LocationResponse( - String eventTime, - String receivedAt, - String factId, - String frameId, - String phone, - String vin, - String vehicleKey, - double longitude, - double latitude, - double altitudeM, - double speedKmh, - double directionDeg, - long alarmFlag, - long statusFlag, - Double totalMileageKm, - String rawUri) { - - private static LocationResponse from(TdengineLocationRow row) { - return new LocationResponse( - row.ts().toString(), - row.receivedAt().toString(), - row.factId(), - row.frameId(), - row.phone(), - row.vin(), - row.vehicleKey(), - row.longitude(), - row.latitude(), - row.altitudeM(), - row.speedKmh(), - row.directionDeg(), - row.alarmFlag(), - row.statusFlag(), - row.totalMileageKm(), - row.rawUri()); - } - } -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Jt808RawFrameHistoryController.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Jt808RawFrameHistoryController.java deleted file mode 100644 index b6d79f94..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/Jt808RawFrameHistoryController.java +++ /dev/null @@ -1,178 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdenginePage; -import com.lingniu.ingest.tdenginehistory.TdenginePageCursor; -import com.lingniu.ingest.tdenginehistory.TdengineQueryOrder; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameQuery; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameRow; -import io.swagger.v3.oas.annotations.Operation; -import io.swagger.v3.oas.annotations.Parameter; -import io.swagger.v3.oas.annotations.tags.Tag; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.http.HttpStatus; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.RequestParam; -import org.springframework.web.bind.annotation.RestController; -import org.springframework.web.server.ResponseStatusException; - -import java.io.IOException; -import java.time.Instant; -import java.util.List; -import java.util.Locale; - -@RestController -@ConditionalOnProperty(prefix = "lingniu.ingest.event-history", name = "enabled", havingValue = "true") -@ConditionalOnProperty(prefix = "lingniu.ingest.event-history.api", name = "specialized-enabled", havingValue = "true") -@ConditionalOnBean(TdengineHistoryReader.class) -@RequestMapping("/api/event-history/jt808") -@Tag(name = "jt-808-raw-frame-controller", description = "JT808 原始帧索引分页查询接口。") -public final class Jt808RawFrameHistoryController { - - private final TdengineHistoryReader reader; - - public Jt808RawFrameHistoryController(TdengineHistoryReader reader) { - if (reader == null) { - throw new IllegalArgumentException("reader must not be null"); - } - this.reader = reader; - } - - @GetMapping("/raw-frames") - @Operation( - summary = "查询 JT808 原始帧索引", - description = "按 phone、VIN、vehicleKey、messageId、parseStatus 和时间范围查询 TDengine raw_frames。" - + "返回完整 metadataJson，包含注册帧字段、身份解析来源、rawArchiveUri 等排障信息。" - + "分页使用 cursorTs + cursorId，不使用 offset。") - public RawFramePageResponse rawFrames( - @Parameter(description = "JT808 终端手机号或终端标识。", example = "13079963320") - @RequestParam(required = false) String phone, - @Parameter(description = "VIN。VIN 已反写后可用该字段查询。", example = "LNVIN000000000001") - @RequestParam(required = false) String vin, - @Parameter(description = "内部车辆键；传入后精确过滤 vehicle_key。", example = "jt808:13079963320") - @RequestParam(required = false) String vehicleKey, - @Parameter(description = "JT808 消息 ID，支持十进制，例如注册帧 256、位置帧 512。", example = "256") - @RequestParam(required = false) Integer messageId, - @Parameter(description = "解析状态，例如 SUCCEEDED、FAILED。", example = "SUCCEEDED") - @RequestParam(required = false) String parseStatus, - @Parameter(description = "开始时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-29T13:00:00") - @RequestParam String dateFrom, - @Parameter(description = "结束时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-29T13:10:00") - @RequestParam String dateTo, - @Parameter(description = "排序方向。", example = "DESC") - @RequestParam(defaultValue = "DESC") TdengineQueryOrder order, - @Parameter(description = "返回原始帧数量上限，最大 1000。", example = "100") - @RequestParam(defaultValue = "100") int limit, - @Parameter(description = "上一页返回的 nextCursor.ts。", example = "2026-06-29T05:00:01Z") - @RequestParam(required = false) String cursorTs, - @Parameter(description = "上一页返回的 nextCursor.id。", example = "jt808-frame-xxx") - @RequestParam(required = false) String cursorId) throws IOException { - if (allBlank(phone, vin, vehicleKey) && messageId == null && isBlank(parseStatus)) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, - "phone, vin, vehicleKey, messageId or parseStatus is required for jt808 raw frame query"); - } - QueryTimeRange range = QueryTimeRange.parse(dateFrom, dateTo); - TdenginePage page = reader.queryRawFrames(new TdengineRawFrameQuery( - "JT808", - normalize(vehicleKey), - normalize(vin), - normalize(phone), - messageId, - normalize(parseStatus).toUpperCase(Locale.ROOT), - range.eventTimeFrom(), - range.eventTimeTo().plusMillis(1), - order, - limit, - cursor(cursorTs, cursorId))); - return RawFramePageResponse.from(page); - } - - private static TdenginePageCursor cursor(String cursorTs, String cursorId) { - boolean hasTs = cursorTs != null && !cursorTs.isBlank(); - boolean hasId = cursorId != null && !cursorId.isBlank(); - if (!hasTs && !hasId) { - return null; - } - if (!hasTs || !hasId) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "cursorTs and cursorId must be provided together"); - } - return new TdenginePageCursor(Instant.parse(cursorTs.trim()), cursorId.trim()); - } - - private static boolean allBlank(String... values) { - for (String value : values) { - if (!isBlank(value)) { - return false; - } - } - return true; - } - - private static boolean isBlank(String value) { - return value == null || value.isBlank(); - } - - private static String normalize(String value) { - return value == null ? "" : value.trim(); - } - - public record RawFramePageResponse( - List items, - CursorResponse nextCursor) { - - private static RawFramePageResponse from(TdenginePage page) { - CursorResponse next = page.nextCursor() - .map(cursor -> new CursorResponse(cursor.ts().toString(), cursor.tieBreaker())) - .orElse(null); - return new RawFramePageResponse( - page.items().stream().map(RawFrameResponse::from).toList(), - next); - } - } - - public record CursorResponse(String ts, String id) { - } - - public record RawFrameResponse( - String eventTime, - String receivedAt, - String frameId, - int messageId, - String messageIdHex, - int subType, - String rawUri, - String checksum, - long rawSizeBytes, - String parseStatus, - String parseError, - String peer, - String metadataJson, - String protocol, - String vehicleKey, - String vin, - String phone) { - - private static RawFrameResponse from(TdengineRawFrameRow row) { - return new RawFrameResponse( - row.ts().toString(), - row.receivedAt().toString(), - row.frameId(), - row.messageId(), - "0x%04X".formatted(row.messageId()), - row.subType(), - row.rawUri(), - row.checksum(), - row.rawSizeBytes(), - row.parseStatus(), - row.parseError(), - row.peer(), - row.metadataJson(), - row.protocol(), - row.vehicleKey(), - row.vin(), - row.phone()); - } - } -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/LocationHistoryController.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/LocationHistoryController.java deleted file mode 100644 index f346a9d5..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/LocationHistoryController.java +++ /dev/null @@ -1,183 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineLocationQuery; -import com.lingniu.ingest.tdenginehistory.TdengineLocationRow; -import com.lingniu.ingest.tdenginehistory.TdenginePage; -import com.lingniu.ingest.tdenginehistory.TdenginePageCursor; -import com.lingniu.ingest.tdenginehistory.TdengineQueryOrder; -import io.swagger.v3.oas.annotations.Operation; -import io.swagger.v3.oas.annotations.Parameter; -import io.swagger.v3.oas.annotations.tags.Tag; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.http.HttpStatus; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.RequestParam; -import org.springframework.web.bind.annotation.RestController; -import org.springframework.web.server.ResponseStatusException; - -import java.io.IOException; -import java.time.Instant; -import java.util.List; - -@RestController -@ConditionalOnProperty(prefix = "lingniu.ingest.event-history", name = "enabled", havingValue = "true") -@ConditionalOnBean(TdengineHistoryReader.class) -@RequestMapping("/api/event-history") -@Tag(name = "location-history-controller", description = "通用位置历史分页查询接口。") -public final class LocationHistoryController { - - private final TdengineHistoryReader reader; - - public LocationHistoryController(TdengineHistoryReader reader) { - if (reader == null) { - throw new IllegalArgumentException("reader must not be null"); - } - this.reader = reader; - } - - @GetMapping("/locations") - @Operation( - summary = "查询车辆位置历史", - description = "按协议、车辆标识和时间范围查询 TDengine 位置点。分页使用 cursorTs + cursorId，不使用 offset。") - public LocationPageResponse locations( - @Parameter(description = "协议名，例如 GB32960、JT808、MQTT_YUTONG。", required = true, example = "MQTT_YUTONG") - @RequestParam String protocol, - @Parameter(description = "内部车辆键。传入后优先使用。", example = "LMRKH9AC2R1004087") - @RequestParam(required = false) String vehicleKey, - @Parameter(description = "VIN。GB32960、宇通通常可直接用 VIN。", example = "LMRKH9AC2R1004087") - @RequestParam(required = false) String vin, - @Parameter(description = "JT808 终端手机号或终端标识；JT808 会自动映射为 jt808:。", example = "g7gps") - @RequestParam(required = false) String phone, - @Parameter(description = "开始时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-29T13:00:00") - @RequestParam String dateFrom, - @Parameter(description = "结束时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-29T13:10:00") - @RequestParam String dateTo, - @Parameter(description = "排序方向。", example = "DESC") - @RequestParam(defaultValue = "DESC") TdengineQueryOrder order, - @Parameter(description = "返回位置点数量上限，最大 1000。", example = "100") - @RequestParam(defaultValue = "100") int limit, - @Parameter(description = "上一页返回的 nextCursor.ts。", example = "2026-06-29T05:00:01Z") - @RequestParam(required = false) String cursorTs, - @Parameter(description = "上一页返回的 nextCursor.id。", example = "location-xxx") - @RequestParam(required = false) String cursorId) throws IOException { - String normalizedProtocol = DefaultProductionProtocols.requireSupported( - require(protocol, "protocol is required for location query")); - QueryTimeRange range = QueryTimeRange.parse(dateFrom, dateTo); - TdenginePage page = reader.queryLocations(new TdengineLocationQuery( - normalizedProtocol, - resolveVehicleKey(normalizedProtocol, vehicleKey, vin, phone), - range.eventTimeFrom(), - range.eventTimeTo().plusMillis(1), - order, - limit, - cursor(cursorTs, cursorId))); - return LocationPageResponse.from(page); - } - - private static String resolveVehicleKey(String protocol, String vehicleKey, String vin, String phone) { - String explicitVehicleKey = trimToNull(vehicleKey); - if (explicitVehicleKey != null) { - return explicitVehicleKey; - } - String vinValue = trimToNull(vin); - if (vinValue != null) { - return vinValue; - } - String phoneValue = trimToNull(phone); - if (phoneValue != null) { - if ("JT808".equals(protocol) && !phoneValue.startsWith("jt808:")) { - return "jt808:" + phoneValue; - } - return phoneValue; - } - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "vehicleKey, vin or phone is required"); - } - - private static String require(String value, String message) { - String trimmed = trimToNull(value); - if (trimmed == null) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, message); - } - return trimmed; - } - - private static String trimToNull(String value) { - if (value == null || value.isBlank()) { - return null; - } - return value.trim(); - } - - private static TdenginePageCursor cursor(String cursorTs, String cursorId) { - boolean hasTs = cursorTs != null && !cursorTs.isBlank(); - boolean hasId = cursorId != null && !cursorId.isBlank(); - if (!hasTs && !hasId) { - return null; - } - if (!hasTs || !hasId) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "cursorTs and cursorId must be provided together"); - } - return new TdenginePageCursor(Instant.parse(cursorTs.trim()), cursorId.trim()); - } - - public record LocationPageResponse( - List items, - CursorResponse nextCursor) { - - private static LocationPageResponse from(TdenginePage page) { - CursorResponse next = page.nextCursor() - .map(cursor -> new CursorResponse(cursor.ts().toString(), cursor.tieBreaker())) - .orElse(null); - return new LocationPageResponse( - page.items().stream().map(LocationResponse::from).toList(), - next); - } - } - - public record CursorResponse(String ts, String id) { - } - - public record LocationResponse( - String eventTime, - String receivedAt, - String factId, - String frameId, - String phone, - String vin, - String vehicleKey, - double longitude, - double latitude, - double altitudeM, - double speedKmh, - double directionDeg, - long alarmFlag, - long statusFlag, - Double totalMileageKm, - String rawUri, - String protocol) { - - private static LocationResponse from(TdengineLocationRow row) { - return new LocationResponse( - row.ts().toString(), - row.receivedAt().toString(), - row.factId(), - row.frameId(), - row.phone(), - row.vin(), - row.vehicleKey(), - row.longitude(), - row.latitude(), - row.altitudeM(), - row.speedKmh(), - row.directionDeg(), - row.alarmFlag(), - row.statusFlag(), - row.totalMileageKm(), - row.rawUri(), - row.protocol()); - } - } -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/QueryTimeRange.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/QueryTimeRange.java deleted file mode 100644 index f71f2dc4..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/QueryTimeRange.java +++ /dev/null @@ -1,87 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import java.time.Instant; -import java.time.LocalDate; -import java.time.LocalDateTime; -import java.time.OffsetDateTime; -import java.time.ZoneId; - -/** - * 查询时间范围解析器。 - * - *

    HTTP 查询允许传日期、带时区时间或不带时区的本地时间： - * 日期用于分区裁剪，不带时区的时间统一按 Asia/Shanghai 转成 UTC Instant。 - * 返回值同时包含 LocalDate 分区边界和精确到秒/毫秒的 eventTime 边界。 - */ -final class QueryTimeRange { - - private static final ZoneId DEFAULT_ZONE = ZoneId.of("Asia/Shanghai"); - - private final LocalDate dateFrom; - private final LocalDate dateTo; - private final Instant eventTimeFrom; - private final Instant eventTimeTo; - - private QueryTimeRange(LocalDate dateFrom, LocalDate dateTo, Instant eventTimeFrom, Instant eventTimeTo) { - this.dateFrom = dateFrom; - this.dateTo = dateTo; - this.eventTimeFrom = eventTimeFrom; - this.eventTimeTo = eventTimeTo; - } - - static QueryTimeRange parse(String from, String to) { - Boundary start = parseBoundary(from, false); - Boundary end = parseBoundary(to, true); - if (end.instant() != null && start.instant() != null && end.instant().isBefore(start.instant())) { - throw new IllegalArgumentException("dateTo must not be before dateFrom"); - } - if (end.date().isBefore(start.date())) { - throw new IllegalArgumentException("dateTo must not be before dateFrom"); - } - return new QueryTimeRange(start.date(), end.date(), start.instant(), end.instant()); - } - - LocalDate dateFrom() { - return dateFrom; - } - - LocalDate dateTo() { - return dateTo; - } - - Instant eventTimeFrom() { - return eventTimeFrom; - } - - Instant eventTimeTo() { - return eventTimeTo; - } - - private static Boundary parseBoundary(String raw, boolean endOfRange) { - if (raw == null || raw.isBlank()) { - throw new IllegalArgumentException("date range must not be blank"); - } - String value = raw.trim(); - if (value.length() == 10) { - // 纯日期查询覆盖东八区自然日；结束日期取当天最后一毫秒，兼容 dateTo=2026-06-23。 - LocalDate date = LocalDate.parse(value); - Instant instant = endOfRange - ? date.plusDays(1).atStartOfDay(DEFAULT_ZONE).toInstant().minusMillis(1) - : date.atStartOfDay(DEFAULT_ZONE).toInstant(); - return new Boundary(date, instant); - } - try { - // 带 Z 或 +08:00 的时间按调用方显式时区解析。 - Instant instant = OffsetDateTime.parse(value).toInstant(); - return new Boundary(LocalDate.ofInstant(instant, DEFAULT_ZONE), instant); - } catch (RuntimeException ignored) { - // 不带时区的前端输入按东八区解释，接口返回的数据时间本身仍保持原始 UTC 字符串。 - LocalDateTime local = LocalDateTime.parse(value); - Instant instant = local.atZone(DEFAULT_ZONE).toInstant(); - return new Boundary(local.toLocalDate(), instant); - } - } - - private record Boundary(LocalDate date, Instant instant) { - } -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/RawFrameHistoryController.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/RawFrameHistoryController.java deleted file mode 100644 index 31008426..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/RawFrameHistoryController.java +++ /dev/null @@ -1,261 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.fasterxml.jackson.core.type.TypeReference; -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdenginePage; -import com.lingniu.ingest.tdenginehistory.TdenginePageCursor; -import com.lingniu.ingest.tdenginehistory.TdengineQueryOrder; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameQuery; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameRow; -import io.swagger.v3.oas.annotations.Operation; -import io.swagger.v3.oas.annotations.Parameter; -import io.swagger.v3.oas.annotations.tags.Tag; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.http.HttpStatus; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.RequestParam; -import org.springframework.web.bind.annotation.RestController; -import org.springframework.web.server.ResponseStatusException; - -import java.io.IOException; -import java.time.Instant; -import java.util.Collections; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Locale; -import java.util.Map; - -@RestController -@ConditionalOnProperty(prefix = "lingniu.ingest.event-history", name = "enabled", havingValue = "true") -@ConditionalOnBean(TdengineHistoryReader.class) -@RequestMapping("/api/event-history") -@Tag(name = "Raw Frame API", description = "原始帧索引分页查询接口。") -public final class RawFrameHistoryController { - - private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper(); - private static final TypeReference> MAP_TYPE = new TypeReference<>() {}; - private final TdengineHistoryReader reader; - - public RawFrameHistoryController(TdengineHistoryReader reader) { - if (reader == null) { - throw new IllegalArgumentException("reader must not be null"); - } - this.reader = reader; - } - - @GetMapping("/raw-frames") - @Operation( - summary = "查询原始帧索引", - description = "按 protocol、vehicleKey、VIN、phone、messageId、parseStatus 和时间范围查询 TDengine raw_frames。" - + "返回 metadataJson 与 rawUri，分页使用 cursorTs + cursorId。") - public RawFramePageResponse rawFrames( - @Parameter(description = "协议类型，例如 GB32960、JT808、MQTT_YUTONG。", example = "JT808") - @RequestParam String protocol, - @Parameter(description = "内部车辆键；传入后精确过滤 vehicle_key。", example = "jt808:g7gps") - @RequestParam(required = false) String vehicleKey, - @Parameter(description = "VIN。已完成身份反写的 raw 帧可用该字段查询。", example = "LB9A32A24P0LS1270") - @RequestParam(required = false) String vin, - @Parameter(description = "终端手机号或终端标识，主要用于 JT808。", example = "g7gps") - @RequestParam(required = false) String phone, - @Parameter(description = "消息 ID，支持十进制或 0x 十六进制，例如 512 或 0x0200。", example = "0x0200") - @RequestParam(required = false) String messageId, - @Parameter(description = "解析状态，例如 SUCCEEDED、FAILED、NOT_PARSED。", example = "SUCCEEDED") - @RequestParam(required = false) String parseStatus, - @Parameter(description = "开始时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-29T13:00:00") - @RequestParam String dateFrom, - @Parameter(description = "结束时间，支持日期或精确到秒的时间；无时区时按东八区解析。", example = "2026-06-29T13:10:00") - @RequestParam String dateTo, - @Parameter(description = "排序方向。", example = "DESC") - @RequestParam(defaultValue = "DESC") TdengineQueryOrder order, - @Parameter(description = "返回原始帧数量上限，最大 1000。", example = "100") - @RequestParam(defaultValue = "100") int limit, - @Parameter(description = "上一页返回的 nextCursor.ts。", example = "2026-06-29T05:00:01Z") - @RequestParam(required = false) String cursorTs, - @Parameter(description = "上一页返回的 nextCursor.id。", example = "frame-xxx") - @RequestParam(required = false) String cursorId) throws IOException { - Integer parsedMessageId = parseMessageId(messageId); - if (allBlank(vehicleKey, vin, phone) && parsedMessageId == null && isBlank(parseStatus)) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, - "vehicleKey, vin, phone, messageId or parseStatus is required for raw frame query"); - } - String normalizedProtocol = DefaultProductionProtocols.requireSupported(protocol); - QueryTimeRange range = QueryTimeRange.parse(dateFrom, dateTo); - TdenginePage page = reader.queryRawFrames(new TdengineRawFrameQuery( - normalizedProtocol, - normalize(vehicleKey), - normalize(vin), - normalize(phone), - parsedMessageId, - normalize(parseStatus).toUpperCase(Locale.ROOT), - range.eventTimeFrom(), - range.eventTimeTo().plusMillis(1), - order, - limit, - cursor(cursorTs, cursorId))); - return RawFramePageResponse.from(page); - } - - private static Integer parseMessageId(String messageId) { - String value = normalize(messageId); - if (value.isBlank()) { - return null; - } - try { - if (value.startsWith("0x") || value.startsWith("0X")) { - return Integer.parseInt(value.substring(2), 16); - } - return Integer.valueOf(value); - } catch (NumberFormatException ex) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "messageId must be decimal or 0x hex", ex); - } - } - - private static TdenginePageCursor cursor(String cursorTs, String cursorId) { - boolean hasTs = cursorTs != null && !cursorTs.isBlank(); - boolean hasId = cursorId != null && !cursorId.isBlank(); - if (!hasTs && !hasId) { - return null; - } - if (!hasTs || !hasId) { - throw new ResponseStatusException(HttpStatus.BAD_REQUEST, "cursorTs and cursorId must be provided together"); - } - return new TdenginePageCursor(Instant.parse(cursorTs.trim()), cursorId.trim()); - } - - private static boolean allBlank(String... values) { - for (String value : values) { - if (!isBlank(value)) { - return false; - } - } - return true; - } - - private static boolean isBlank(String value) { - return value == null || value.isBlank(); - } - - private static String normalize(String value) { - return value == null ? "" : value.trim(); - } - - public record RawFramePageResponse( - List items, - CursorResponse nextCursor) { - - private static RawFramePageResponse from(TdenginePage page) { - CursorResponse next = page.nextCursor() - .map(cursor -> new CursorResponse(cursor.ts().toString(), cursor.tieBreaker())) - .orElse(null); - List items = new java.util.ArrayList<>(page.items().size()); - for (TdengineRawFrameRow row : page.items()) { - items.add(RawFrameResponse.from(row)); - } - return new RawFramePageResponse( - items, - next); - } - } - - public record CursorResponse(String ts, String id) { - } - - public record RawFrameResponse( - String eventTime, - String receivedAt, - String frameId, - int messageId, - String messageIdHex, - int subType, - String rawUri, - String checksum, - long rawSizeBytes, - String parseStatus, - String parseError, - String peer, - String metadataJson, - String parsedJson, - Map metadata, - Map parsedFields, - String protocol, - String vehicleKey, - String vin, - String phone) { - - private static RawFrameResponse from(TdengineRawFrameRow row) { - Map metadata = metadata(row.metadataJson()); - return new RawFrameResponse( - row.ts().toString(), - row.receivedAt().toString(), - row.frameId(), - row.messageId(), - "0x%04X".formatted(row.messageId()), - row.subType(), - row.rawUri(), - row.checksum(), - row.rawSizeBytes(), - row.parseStatus(), - row.parseError(), - row.peer(), - row.metadataJson(), - row.parsedJson(), - metadata, - parsedFields(metadata, parsed(row.parsedJson())), - row.protocol(), - row.vehicleKey(), - row.vin(), - row.phone()); - } - - private static Map metadata(String metadataJson) { - if (metadataJson == null || metadataJson.isBlank()) { - return Map.of(); - } - try { - return OBJECT_MAPPER.readValue(metadataJson, MAP_TYPE); - } catch (Exception ignored) { - return Map.of("_raw", metadataJson); - } - } - - private static Map parsedFields(Map metadata, - Map parsed) { - Map out = new LinkedHashMap<>(); - if (parsed.isEmpty()) { - for (Map.Entry entry : metadata.entrySet()) { - String key = entry.getKey(); - if (key == null || key.isBlank()) { - continue; - } - if (key.startsWith("raw.") || key.startsWith("jt808.")) { - out.put(key, entry.getValue()); - } else { - out.put("raw." + key, entry.getValue()); - } - } - } else { - out.putAll(parsed); - } - return immutableMap(out); - } - - private static Map parsed(String parsedJson) { - if (parsedJson == null || parsedJson.isBlank()) { - return Map.of(); - } - try { - return OBJECT_MAPPER.readValue(parsedJson, MAP_TYPE); - } catch (Exception ignored) { - return Map.of("_raw", parsedJson); - } - } - - private static Map immutableMap(Map values) { - return Collections.unmodifiableMap(new LinkedHashMap<>(values)); - } - } - -} diff --git a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/config/EventHistoryAutoConfiguration.java b/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/config/EventHistoryAutoConfiguration.java deleted file mode 100644 index d2a8838e..00000000 --- a/modules/services/event-history-service/src/main/java/com/lingniu/ingest/eventhistory/config/EventHistoryAutoConfiguration.java +++ /dev/null @@ -1,198 +0,0 @@ -package com.lingniu.ingest.eventhistory.config; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.api.consumer.EnvelopeDeadLetterSink; -import com.lingniu.ingest.eventhistory.ApiExceptionHandler; -import com.lingniu.ingest.eventhistory.EventHistoryEnvelopeIngestor; -import com.lingniu.ingest.eventhistory.Gb32960DecodedFrameService; -import com.lingniu.ingest.eventhistory.Gb32960FrameController; -import com.lingniu.ingest.eventhistory.Jt808LocationHistoryController; -import com.lingniu.ingest.eventhistory.Jt808RawFrameHistoryController; -import com.lingniu.ingest.eventhistory.LocationHistoryController; -import com.lingniu.ingest.eventhistory.RawFrameHistoryController; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960BodyParser; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960CommandParser; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParserRegistry; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.AlarmV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.DriveMotorV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.EngineV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.ExtremeValueV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.FuelCellV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.PositionV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.TemperatureV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VoltageV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.AlarmV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.BatteryTemperatureV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.DriveMotorV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.EngineV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.FuelCellStackV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.FuelCellV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.MinParallelVoltageV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.PositionV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.SuperCapacitorExtremeV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.SuperCapacitorV2025BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2025.VehicleV2025BlockParser; -import com.lingniu.ingest.sink.kafka.KafkaSinkAutoConfiguration; -import com.lingniu.ingest.tdenginehistory.config.TdengineHistoryAutoConfiguration; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryWriter; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.AutoConfigureAfter; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.context.annotation.Bean; -import org.springframework.beans.factory.ObjectProvider; -import org.springframework.beans.factory.annotation.Value; - -import java.net.URI; -import java.nio.file.Path; -import java.util.List; - -/** - * Event History 查询/消费服务自动装配。 - * - *

    该模块有两类入口： - *

      - *
    • TDengine HTTP 查询入口：查询位置、RAW 帧和按需解码的 GB32960 帧。 - *
    • Kafka consumer 入口：消费 envelope 并写入 TDengine 历史表。 - *
    - * - *

    当前生产 history app 以 TDengine 为准；Kafka consumer 是否启动还取决于 - * {@code lingniu.ingest.sink.kafka.consumer.enabled=true}。 - */ -@AutoConfiguration -@AutoConfigureAfter({ - KafkaSinkAutoConfiguration.class, - TdengineHistoryAutoConfiguration.class -}) -@ConditionalOnProperty(prefix = "lingniu.ingest.event-history", name = "enabled", havingValue = "true") -public class EventHistoryAutoConfiguration { - - @Bean - @ConditionalOnProperty(prefix = "lingniu.ingest.tdengine-history", name = "telemetry-fields-enabled") - public Object rejectedRemovedTelemetryFieldsSwitch() { - throw new IllegalStateException( - "tdengine-history.telemetry-fields-enabled has been removed; history stores raw_frames and location rows only"); - } - - @Bean - @ConditionalOnBean(TdengineHistoryWriter.class) - @ConditionalOnMissingBean - public EventHistoryEnvelopeIngestor tdengineEventHistoryEnvelopeIngestor(TdengineHistoryWriter writer) { - return new EventHistoryEnvelopeIngestor(writer); - } - - @Bean - @ConditionalOnBean({EventHistoryEnvelopeIngestor.class, EnvelopeDeadLetterSink.class}) - @ConditionalOnMissingBean(name = "eventHistoryEnvelopeConsumerProcessor") - public EnvelopeConsumerProcessor eventHistoryEnvelopeConsumerProcessor(EventHistoryEnvelopeIngestor ingestor, - EnvelopeDeadLetterSink deadLetterSink) { - // 只注册 processor；真正拉 Kafka 的 runner 按 consumer.enabled 决定是否创建。 - return new EnvelopeConsumerProcessor("event-history", ingestor, deadLetterSink); - } - - @Bean - @ConditionalOnBean({EventHistoryEnvelopeIngestor.class, EnvelopeDeadLetterSink.class}) - @ConditionalOnMissingBean(name = "eventHistoryRawEnvelopeConsumerProcessor") - public EnvelopeConsumerProcessor eventHistoryRawEnvelopeConsumerProcessor(EventHistoryEnvelopeIngestor ingestor, - EnvelopeDeadLetterSink deadLetterSink) { - return new EnvelopeConsumerProcessor("event-history-raw", ingestor, deadLetterSink); - } - - @Bean - @ConditionalOnMissingBean - public ApiExceptionHandler apiExceptionHandler() { - return new ApiExceptionHandler(); - } - - @Bean - @ConditionalOnMissingBean - public Gb32960MessageDecoder gb32960HistoryMessageDecoder() { - List parsers = List.of( - new VehicleV2016BlockParser(), - new DriveMotorV2016BlockParser(), - new FuelCellV2016BlockParser(), - new EngineV2016BlockParser(), - new PositionV2016BlockParser(), - new ExtremeValueV2016BlockParser(), - new AlarmV2016BlockParser(), - new VoltageV2016BlockParser(), - new TemperatureV2016BlockParser(), - new VehicleV2025BlockParser(), - new DriveMotorV2025BlockParser(), - new FuelCellV2025BlockParser(), - new EngineV2025BlockParser(), - new PositionV2025BlockParser(), - new AlarmV2025BlockParser(), - new MinParallelVoltageV2025BlockParser(), - new BatteryTemperatureV2025BlockParser(), - new FuelCellStackV2025BlockParser(), - new SuperCapacitorV2025BlockParser(), - new SuperCapacitorExtremeV2025BlockParser()); - Gb32960BodyParser bodyParser = new Gb32960BodyParser(new InfoBlockParserRegistry(parsers)); - return new Gb32960MessageDecoder(bodyParser, new Gb32960CommandParser()); - } - - @Bean - @ConditionalOnBean(Gb32960MessageDecoder.class) - @ConditionalOnMissingBean - public Gb32960DecodedFrameService gb32960DecodedFrameService(ObjectProvider reader, - Gb32960MessageDecoder decoder, - @Value("${lingniu.ingest.event-history.archive-path:${SINK_ARCHIVE_PATH:./archive/}}") - String archivePath) { - return new Gb32960DecodedFrameService(reader.getIfAvailable(), decoder, archiveRoot(archivePath), null); - } - - @Bean - @ConditionalOnBean(Gb32960DecodedFrameService.class) - @ConditionalOnProperty(prefix = "lingniu.ingest.event-history.api", name = "specialized-enabled", havingValue = "true") - @ConditionalOnMissingBean - public Gb32960FrameController gb32960FrameController(Gb32960DecodedFrameService service) { - return new Gb32960FrameController(service); - } - - @Bean - @ConditionalOnBean(TdengineHistoryReader.class) - @ConditionalOnMissingBean - public LocationHistoryController locationHistoryController(TdengineHistoryReader reader) { - return new LocationHistoryController(reader); - } - - @Bean - @ConditionalOnBean(TdengineHistoryReader.class) - @ConditionalOnMissingBean - public RawFrameHistoryController rawFrameHistoryController(TdengineHistoryReader reader) { - return new RawFrameHistoryController(reader); - } - - @Bean - @ConditionalOnBean(TdengineHistoryReader.class) - @ConditionalOnProperty(prefix = "lingniu.ingest.event-history.api", name = "specialized-enabled", havingValue = "true") - @ConditionalOnMissingBean - public Jt808LocationHistoryController jt808LocationHistoryController(TdengineHistoryReader reader) { - return new Jt808LocationHistoryController(reader); - } - - @Bean - @ConditionalOnBean(TdengineHistoryReader.class) - @ConditionalOnProperty(prefix = "lingniu.ingest.event-history.api", name = "specialized-enabled", havingValue = "true") - @ConditionalOnMissingBean - public Jt808RawFrameHistoryController jt808RawFrameHistoryController(TdengineHistoryReader reader) { - return new Jt808RawFrameHistoryController(reader); - } - - static Path archiveRoot(String value) { - if (value == null || value.isBlank()) { - return Path.of(System.getProperty("java.io.tmpdir"), "lingniu-archive"); - } - // 支持 file:// URI，便于部署配置里统一用 URI 风格表达 archive 根路径。 - if (value.startsWith("file://")) { - return Path.of(URI.create(value)); - } - return Path.of(value); - } -} diff --git a/modules/services/event-history-service/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/services/event-history-service/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index 04b40140..00000000 --- a/modules/services/event-history-service/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.eventhistory.config.EventHistoryAutoConfiguration diff --git a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/ApiExceptionHandlerTest.java b/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/ApiExceptionHandlerTest.java deleted file mode 100644 index 9e0adbf1..00000000 --- a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/ApiExceptionHandlerTest.java +++ /dev/null @@ -1,101 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import jakarta.servlet.http.HttpServletRequest; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; -import org.springframework.http.HttpMethod; -import org.springframework.test.web.servlet.MockMvc; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.RestController; -import org.springframework.web.servlet.resource.NoResourceFoundException; - -import java.io.IOException; -import java.nio.file.Path; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.assertj.core.api.Assertions.assertThat; -import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; -import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; -import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; -import static org.springframework.test.web.servlet.setup.MockMvcBuilders.standaloneSetup; - -class ApiExceptionHandlerTest { - - @TempDir - Path archiveRoot; - - @Test - void missingRequiredQueryParameterReturnsConcreteReason() throws Exception { - MockMvc mvc = standaloneSetup(new Gb32960FrameController(service())) - .setControllerAdvice(new ApiExceptionHandler()) - .build(); - - mvc.perform(get("/api/event-history/gb32960/snapshots") - .param("dateFrom", "2026-06-23") - .param("dateTo", "2026-06-23") - .param("limit", "1")) - .andExpect(status().isBadRequest()) - .andExpect(jsonPath("$.message").value("missing required query parameter: vin")) - .andExpect(jsonPath("$.timestamp").value(org.hamcrest.Matchers.endsWith("+08:00"))) - .andExpect(jsonPath("$.path").value("/api/event-history/gb32960/snapshots")); - } - - @Test - void responseStatusExceptionReturnsReason() throws Exception { - MockMvc mvc = standaloneSetup(new Gb32960FrameController(service())) - .setControllerAdvice(new ApiExceptionHandler()) - .build(); - - mvc.perform(get("/api/event-history/gb32960/snapshots") - .param("dateFrom", "2026-06-23") - .param("dateTo", "2026-06-23") - .param("limit", "1") - .param("vin", "")) - .andExpect(status().isBadRequest()) - .andExpect(jsonPath("$.message").value("vin is required for gb32960 snapshots")); - } - - @Test - void ioExceptionReturnsServiceUnavailable() throws Exception { - MockMvc mvc = standaloneSetup(new IoFailureController()) - .setControllerAdvice(new ApiExceptionHandler()) - .build(); - - mvc.perform(get("/io")) - .andExpect(status().isServiceUnavailable()) - .andExpect(jsonPath("$.message").value("tdengine history query failed")) - .andExpect(jsonPath("$.path").value("/io")); - } - - @Test - void noResourceReturnsNotFound() { - HttpServletRequest request = mock(HttpServletRequest.class); - when(request.getRequestURI()).thenReturn("/missing"); - - var response = new ApiExceptionHandler().noResource( - new NoResourceFoundException(HttpMethod.GET, "/missing"), - request); - - assertThat(response.getStatusCode().value()).isEqualTo(404); - assertThat(response.getBody()).isNotNull(); - assertThat(response.getBody().message()).isEqualTo("resource not found: /missing"); - } - - private Gb32960DecodedFrameService service() { - return new Gb32960DecodedFrameService( - (TdengineHistoryReader) null, - mock(Gb32960MessageDecoder.class), - archiveRoot, - null); - } - - @RestController - private static final class IoFailureController { - @GetMapping("/io") - String io() throws IOException { - throw new IOException("tdengine history query failed"); - } - } -} diff --git a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/EventHistoryEnvelopeIngestorTest.java b/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/EventHistoryEnvelopeIngestorTest.java deleted file mode 100644 index d8534d69..00000000 --- a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/EventHistoryEnvelopeIngestorTest.java +++ /dev/null @@ -1,176 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.lingniu.ingest.api.consumer.EnvelopeIngestResult; -import com.lingniu.ingest.api.consumer.EnvelopeIngestor; -import com.lingniu.ingest.sink.kafka.proto.RawArchiveRef; -import com.lingniu.ingest.sink.kafka.proto.RawFrameFactPayload; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import com.lingniu.ingest.sink.kafka.proto.LocationPayload; -import com.lingniu.ingest.sink.kafka.proto.ParseStatusProto; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryWriter; -import com.lingniu.ingest.tdenginehistory.TdengineLocationRow; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameRow; -import org.junit.jupiter.api.Test; - -import java.util.ArrayList; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class EventHistoryEnvelopeIngestorTest { - - @Test - void rejectsInvalidEnvelopeBytes() { - CapturingTdengineWriter tdengineWriter = new CapturingTdengineWriter(); - EventHistoryEnvelopeIngestor ingestor = - new EventHistoryEnvelopeIngestor(tdengineWriter); - - assertThat(ingestor).isInstanceOf(EnvelopeIngestor.class); - assertThatThrownBy(() -> ingestor.ingest(new byte[]{0x01, 0x02})) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("VehicleEnvelope"); - } - - @Test - void tryIngestInvalidEnvelopeBytesReturnsInvalidWithoutWritingTdengine() { - CapturingTdengineWriter tdengineWriter = new CapturingTdengineWriter(); - EventHistoryEnvelopeIngestor ingestor = - new EventHistoryEnvelopeIngestor(tdengineWriter); - - var result = ingestor.tryIngest(new byte[]{0x01, 0x02}); - - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.INVALID_ENVELOPE); - assertThat(result.message()).contains("VehicleEnvelope"); - assertThat(tdengineWriter.rawFrames).isEmpty(); - assertThat(tdengineWriter.locations).isEmpty(); - } - - @Test - void tryIngestWritesRawAndLocationFactsToTdengineWhenWriterExists() throws Exception { - CapturingTdengineWriter tdengineWriter = new CapturingTdengineWriter(); - EventHistoryEnvelopeIngestor ingestor = new EventHistoryEnvelopeIngestor(tdengineWriter); - - VehicleEnvelope envelope = VehicleEnvelope.newBuilder() - .setSchemaVersion("1.0") - .setEventId("jt808-location-1") - .setVin("VINJT808001") - .setSource("JT808") - .setEventTimeMs(1_782_112_400_000L) - .setIngestTimeMs(1_782_112_401_000L) - .putMetadata("vehicle_key", "jt808:g7gps") - .putMetadata("frame_id", "frame-jt808-1") - .setRawArchive(RawArchiveRef.newBuilder() - .setUri("archive://jt808/2026/06/29/frame-jt808-1.bin") - .setSizeBytes(68)) - .setRawFrameFact(RawFrameFactPayload.newBuilder() - .setFrameId("frame-jt808-1") - .setVehicleKey("jt808:g7gps") - .setVin("VINJT808001") - .setPhone("013800000000") - .setMessageId(0x0200) - .setRawUri("archive://jt808/2026/06/29/frame-jt808-1.bin") - .setRawSizeBytes(68) - .setParseStatus(ParseStatusProto.PARSE_STATUS_SUCCEEDED)) - .setLocation(LocationPayload.newBuilder() - .setLongitude(113.12) - .setLatitude(23.45) - .setSpeedKmh(42.5)) - .build(); - - EnvelopeIngestResult result = ingestor.tryIngest(envelope.toByteArray()); - - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.STORED); - assertThat(tdengineWriter.rawFrames) - .extracting(TdengineRawFrameRow::frameId) - .containsExactly("frame-jt808-1"); - assertThat(tdengineWriter.locations) - .extracting(TdengineLocationRow::factId) - .containsExactly("jt808-location-1"); - assertThat(tdengineWriter.locations.getFirst().longitude()).isEqualTo(113.12); - } - - @Test - void tryIngestCanWriteTdengineFactsDirectly() { - CapturingTdengineWriter tdengineWriter = new CapturingTdengineWriter(); - EventHistoryEnvelopeIngestor ingestor = new EventHistoryEnvelopeIngestor(tdengineWriter); - - EnvelopeIngestResult result = ingestor.tryIngest( - jt808LocationEnvelope("jt808-location-1", "frame-jt808-1", "013800000001").toByteArray()); - - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.STORED); - assertThat(tdengineWriter.rawFrames).extracting(TdengineRawFrameRow::frameId) - .containsExactly("frame-jt808-1"); - assertThat(tdengineWriter.locations).extracting(TdengineLocationRow::factId) - .containsExactly("jt808-location-1"); - } - - @Test - void tryIngestAllBatchesFileStoreAndTdengineWrites() throws Exception { - CapturingTdengineWriter tdengineWriter = new CapturingTdengineWriter(); - EventHistoryEnvelopeIngestor ingestor = new EventHistoryEnvelopeIngestor(tdengineWriter); - - VehicleEnvelope first = jt808LocationEnvelope("jt808-location-1", "frame-jt808-1", "013800000001"); - VehicleEnvelope second = jt808LocationEnvelope("jt808-location-2", "frame-jt808-2", "013800000002"); - - List results = ingestor.tryIngestAll(List.of( - first.toByteArray(), - new byte[]{0x01, 0x02}, - second.toByteArray())); - - assertThat(results).extracting(EnvelopeIngestResult::status) - .containsExactly( - EnvelopeIngestResult.Status.STORED, - EnvelopeIngestResult.Status.INVALID_ENVELOPE, - EnvelopeIngestResult.Status.STORED); - assertThat(tdengineWriter.rawFrames).extracting(TdengineRawFrameRow::frameId) - .containsExactly("frame-jt808-1", "frame-jt808-2"); - assertThat(tdengineWriter.locations).extracting(TdengineLocationRow::factId) - .containsExactly("jt808-location-1", "jt808-location-2"); - } - - private static VehicleEnvelope jt808LocationEnvelope(String eventId, String frameId, String phone) { - return VehicleEnvelope.newBuilder() - .setSchemaVersion("1.0") - .setEventId(eventId) - .setVin("VINJT808001") - .setSource("JT808") - .setEventTimeMs(1_782_112_400_000L) - .setIngestTimeMs(1_782_112_401_000L) - .putMetadata("vehicle_key", "jt808:" + phone) - .putMetadata("frame_id", frameId) - .setRawArchive(RawArchiveRef.newBuilder() - .setUri("archive://jt808/2026/06/29/" + frameId + ".bin") - .setSizeBytes(68)) - .setRawFrameFact(RawFrameFactPayload.newBuilder() - .setFrameId(frameId) - .setVehicleKey("jt808:" + phone) - .setVin("VINJT808001") - .setPhone(phone) - .setMessageId(0x0200) - .setRawUri("archive://jt808/2026/06/29/" + frameId + ".bin") - .setRawSizeBytes(68) - .setParseStatus(ParseStatusProto.PARSE_STATUS_SUCCEEDED)) - .setLocation(LocationPayload.newBuilder() - .setLongitude(113.12) - .setLatitude(23.45) - .setSpeedKmh(42.5)) - .build(); - } - - private static final class CapturingTdengineWriter implements TdengineHistoryWriter { - private final List rawFrames = new ArrayList<>(); - private final List locations = new ArrayList<>(); - - @Override - public void appendRawFrames(List rows) { - rawFrames.addAll(rows); - } - - @Override - public void appendLocations(List rows) { - locations.addAll(rows); - } - - } -} diff --git a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Gb32960DecodedFrameServiceTest.java b/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Gb32960DecodedFrameServiceTest.java deleted file mode 100644 index 99162807..00000000 --- a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Gb32960DecodedFrameServiceTest.java +++ /dev/null @@ -1,685 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.lingniu.ingest.codec.BccChecksum; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960BodyParser; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.protocol.gb32960.codec.InfoBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcDcDcBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.parser.vendor.guangdong.GdFcStackBlockParser; -import com.lingniu.ingest.protocol.gb32960.codec.profile.Gb32960ProfileRegistry; -import com.lingniu.ingest.protocol.gb32960.codec.profile.VendorExtensionCatalog; -import com.lingniu.ingest.protocol.gb32960.codec.profile.VendorExtensionSelector; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineLocationQuery; -import com.lingniu.ingest.tdenginehistory.TdengineLocationRow; -import com.lingniu.ingest.tdenginehistory.TdenginePage; -import com.lingniu.ingest.tdenginehistory.TdenginePageCursor; -import com.lingniu.ingest.tdenginehistory.TdengineQueryOrder; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameQuery; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameRow; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.ByteArrayOutputStream; -import java.math.BigDecimal; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Path; -import java.time.Instant; -import java.time.LocalDate; -import java.util.List; -import java.util.Map; -import java.util.Set; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; -import static org.assertj.core.api.Assertions.within; - -class Gb32960DecodedFrameServiceTest { - - @TempDir - Path archiveRoot; - - @Test - void decodesFullGb32960BlocksFromRawArchive() throws Exception { - String key = "2026/06/22/GB32960/LNVFC000000000001/raw-1.bin"; - Path rawFile = archiveRoot.resolve(key); - Files.createDirectories(rawFile.getParent()); - Files.write(rawFile, realtimeFrame()); - String uri = "archive://" + key; - - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - new CapturingTdengineReader(List.of(rawRow("raw-1", uri))), - decoder(), - archiveRoot, - null); - - List frames = service.query( - LocalDate.parse("2026-06-22"), - LocalDate.parse("2026-06-22"), - HistoryQueryOrder.DESC, - 10, - null, - null); - - assertThat(frames).hasSize(1); - Gb32960DecodedFrameService.DecodedFrame frame = frames.getFirst(); - assertThat(frame.vin()).isEqualTo("LNVFC000000000001"); - assertThat(frame.command()).isEqualTo("REALTIME_REPORT"); - assertThat(frame.blocks()).extracting(block -> block.get("type")) - .containsExactly("VEHICLE", "GD_FC_DCDC"); - - @SuppressWarnings("unchecked") - Map vehicle = (Map) frames.getFirst().blocks().getFirst().get("data"); - assertThat(vehicle.get("totalMileageKm").toString()).isEqualTo("10000"); - assertThat(vehicle).containsEntry("socPercent", 85); - - @SuppressWarnings("unchecked") - Map dcdc = (Map) frames.getFirst().blocks().get(1).get("data"); - assertThat(dcdc.get("inputVoltageV").toString()).isEqualTo("300"); - assertThat(dcdc.get("outputCurrentA").toString()).isEqualTo("110"); - } - - @Test - void queryNormalizesFloatingPointNoiseForJsonOutput() throws Exception { - Object normalized = Gb32960DecodedFrameService.normalizeJsonValue(636.3000000000001); - - assertThat(normalized).isInstanceOf(BigDecimal.class); - assertThat(normalized.toString()).isEqualTo("636.3"); - } - - @Test - void findsSingleFrameDirectlyByRawArchiveUri() throws Exception { - String key = "2026/06/22/GB32960/LNVFC000000000001/1792527837000000.bin"; - writeArchive(key, realtimeFrame()); - - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - (TdengineHistoryReader) null, - decoder(), - archiveRoot, - null); - - Gb32960DecodedFrameService.DecodedFrame frame = - service.findByRawArchiveUri("archive://" + key, "Hyundai"); - - assertThat(frame.eventId()).isEqualTo("1792527837000000"); - assertThat(frame.vin()).isEqualTo("LNVFC000000000001"); - assertThat(frame.blocks()).extracting(block -> block.get("type")) - .containsExactly("VEHICLE", "GD_FC_DCDC"); - } - - @Test - void skipsMissingRawArchiveAndContinues() throws Exception { - String missingKey = "2026/06/22/GB32960/LNVFC000000000001/missing.bin"; - String availableKey = "2026/06/22/GB32960/LNVFC000000000001/raw-2.bin"; - Path rawFile = archiveRoot.resolve(availableKey); - Files.createDirectories(rawFile.getParent()); - Files.write(rawFile, realtimeFrame()); - - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - new CapturingTdengineReader(List.of( - rawRow("missing", "archive://" + missingKey), - rawRow("available", "archive://" + availableKey))), - decoder(), - archiveRoot, - null); - - List frames = service.query( - LocalDate.parse("2026-06-22"), - LocalDate.parse("2026-06-22"), - HistoryQueryOrder.DESC, - 10, - null, - null); - - assertThat(frames).hasSize(1); - assertThat(frames.getFirst().eventId()).isEqualTo("available"); - assertThat(frames.getFirst().blocks()).extracting(block -> block.get("type")) - .containsExactly("VEHICLE", "GD_FC_DCDC"); - } - - @Test - void snapshotsRequireVinSoSegmentsAreMergedOnlyWithinOneVehicle() { - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - (TdengineHistoryReader) null, - decoder(), - archiveRoot, - null); - - assertThatThrownBy(() -> service.snapshots( - LocalDate.parse("2026-06-22"), - LocalDate.parse("2026-06-22"), - HistoryQueryOrder.DESC, - 10, - null, - null)) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("vin is required"); - } - - @Test - void snapshotsMergeGb32960StackCellVoltageSegmentsByEventTime() throws Exception { - String vin = "LNVFC000000000001"; - String key1 = "2026/06/22/GB32960/" + vin + "/stack-1.bin"; - String key2 = "2026/06/22/GB32960/" + vin + "/stack-2.bin"; - String key3 = "2026/06/22/GB32960/" + vin + "/stack-3.bin"; - writeArchive(key1, stackFrame(1, 4, true)); - writeArchive(key2, stackFrame(5, 3, false)); - writeArchive(key3, stackFrame(8, 2, false)); - - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - new CapturingTdengineReader(List.of( - rawRow("segment-3", "archive://" + key3), - rawRow("segment-2", "archive://" + key2), - rawRow("segment-1", "archive://" + key1))), - decoder(), - archiveRoot, - null); - - List snapshots = service.snapshots( - LocalDate.parse("2026-06-22"), - LocalDate.parse("2026-06-22"), - HistoryQueryOrder.DESC, - 10, - vin, - null); - - assertThat(snapshots).hasSize(1); - Gb32960DecodedFrameService.LogicalSnapshot snapshot = snapshots.getFirst(); - assertThat(snapshot.vin()).isEqualTo(vin); - assertThat(snapshot.eventTime()).isEqualTo("2026-06-22T10:00:00Z"); - assertThat(snapshot.sourceFrames()).hasSize(3); - assertThat(snapshot.blocks()).extracting(block -> block.get("type")) - .contains("VEHICLE", "GD_FC_STACK"); - - @SuppressWarnings("unchecked") - Map stackBlock = (Map) snapshot.blocks().stream() - .filter(block -> "GD_FC_STACK".equals(block.get("type"))) - .findFirst() - .orElseThrow() - .get("data"); - @SuppressWarnings("unchecked") - List> stacks = (List>) stackBlock.get("stacks"); - assertThat(stacks).hasSize(1); - @SuppressWarnings("unchecked") - List mergedVoltages = (List) stacks.getFirst().get("frameCellVoltagesV"); - assertThat(stacks.getFirst()).containsEntry("frameCellStart", 1); - assertThat(stacks.getFirst()).containsEntry("frameCellCount", 9); - assertThat(stacks.getFirst()).containsEntry("frameCellComplete", true); - assertThat(stacks.getFirst()).containsEntry("frameCellMissingCount", 0L); - assertThat(mergedVoltages).hasSize(9); - List expected = List.of(1.001, 1.002, 1.003, 1.004, 1.005, 1.006, 1.007, 1.008, 1.009); - for (int i = 0; i < expected.size(); i++) { - assertThat(((Number) mergedVoltages.get(i)).doubleValue()).isCloseTo(expected.get(i), within(0.000001)); - } - } - - @Test - void snapshotsUseTdengineRawFrames() throws Exception { - String vin = "LNVFC000000000001"; - String key = "2026/06/22/GB32960/" + vin + "/tdengine-raw.bin"; - String uri = "archive://" + key; - writeArchive(key, realtimeFrame()); - CapturingTdengineReader reader = new CapturingTdengineReader(List.of(new TdengineRawFrameRow( - Instant.parse("2026-06-22T10:00:00Z"), - "tdengine-raw", - Instant.parse("2026-06-22T10:00:01Z"), - 0x02, - 0, - Instant.parse("2026-06-22T10:00:00Z"), - uri, - "sha256:test", - realtimeFrame().length, - "SUCCEEDED", - "", - "127.0.0.1:32960", - "{\"platformAccount\":\"Hyundai\"}", - "", - "GB32960", - vin, - vin, - ""))); - - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - reader, - decoder(), - archiveRoot, - null); - - List snapshots = service.snapshots( - QueryTimeRange.parse("2026-06-22T18:00:00", "2026-06-22T18:00:00"), - HistoryQueryOrder.DESC, - 10, - vin, - null); - - assertThat(reader.lastQuery.protocol()).isEqualTo("GB32960"); - assertThat(reader.lastQuery.vehicleKey()).isEqualTo(vin); - assertThat(snapshots).hasSize(1); - assertThat(snapshots.getFirst().sourceFrames()).extracting(Gb32960DecodedFrameService.SourceFrame::rawArchiveUri) - .containsExactly(uri); - assertThat(snapshots.getFirst().blocks()).extracting(block -> block.get("type")) - .containsExactly("VEHICLE", "GD_FC_DCDC"); - } - - @Test - void queryCanUseTdengineRawFramesWithoutVinForPlatformLoginFrames() throws Exception { - String key = "2026/06/22/GB32960/unknown-vin/platform-login.bin"; - String uri = "archive://" + key; - writeArchive(key, platformLoginFrame()); - CapturingTdengineReader reader = new CapturingTdengineReader(List.of(new TdengineRawFrameRow( - Instant.parse("2026-06-22T10:00:00Z"), - "tdengine-platform-login", - Instant.parse("2026-06-22T10:00:01Z"), - 0x05, - 0, - Instant.parse("2026-06-22T10:00:00Z"), - uri, - "sha256:test", - platformLoginFrame().length, - "SUCCEEDED", - "", - "8.134.95.166:37302", - "{\"platformAccount\":\"Hyundai\"}", - "", - "GB32960", - "unknown:GB32960:platform-login", - "", - ""))); - - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - reader, - decoder(), - archiveRoot, - null); - - List frames = service.query( - LocalDate.parse("2026-06-22"), - LocalDate.parse("2026-06-22"), - HistoryQueryOrder.DESC, - 10, - null, - "Hyundai"); - - assertThat(reader.lastQuery.protocol()).isEqualTo("GB32960"); - assertThat(reader.lastQuery.vehicleKey()).isBlank(); - assertThat(reader.lastQuery.vin()).isBlank(); - assertThat(frames).hasSize(1); - assertThat(frames.getFirst().command()).isEqualTo("PLATFORM_LOGIN"); - assertThat(frames.getFirst().rawArchiveUri()).isEqualTo(uri); - } - - @Test - void framePageUsesTdengineCursorAndReturnsNextCursor() throws Exception { - String vin = "LNVFC000000000001"; - String key = "2026/06/22/GB32960/" + vin + "/tdengine-page.bin"; - String uri = "archive://" + key; - writeArchive(key, realtimeFrame()); - TdenginePageCursor nextCursor = new TdenginePageCursor( - Instant.parse("2026-06-22T10:00:00Z"), "tdengine-page"); - CapturingTdengineReader reader = new CapturingTdengineReader(List.of(new TdengineRawFrameRow( - Instant.parse("2026-06-22T10:00:00Z"), - "tdengine-page", - Instant.parse("2026-06-22T10:00:01Z"), - 0x02, - 0, - Instant.parse("2026-06-22T10:00:00Z"), - uri, - "sha256:test", - realtimeFrame().length, - "SUCCEEDED", - "", - "127.0.0.1:32960", - "{\"platformAccount\":\"Hyundai\"}", - "", - "GB32960", - vin, - vin, - "")), nextCursor); - - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - reader, - decoder(), - archiveRoot, - null); - - Gb32960DecodedFrameService.DecodedFramePage page = service.queryPage( - QueryTimeRange.parse("2026-06-22T18:00:00", "2026-06-22T18:00:00"), - HistoryQueryOrder.ASC, - 10, - vin, - null, - new TdenginePageCursor(Instant.parse("2026-06-22T09:59:00Z"), "previous")); - - assertThat(reader.lastQuery.protocol()).isEqualTo("GB32960"); - assertThat(reader.lastQuery.vehicleKey()).isEqualTo(vin); - assertThat(reader.lastQuery.order()).isEqualTo(TdengineQueryOrder.ASC); - assertThat(reader.lastQuery.cursor()).isEqualTo(new TdenginePageCursor( - Instant.parse("2026-06-22T09:59:00Z"), "previous")); - assertThat(page.items()).hasSize(1); - assertThat(page.items().getFirst().rawArchiveUri()).isEqualTo(uri); - assertThat(page.nextCursor()).isEqualTo(new Gb32960DecodedFrameService.CursorResponse( - "2026-06-22T10:00:00Z", "tdengine-page")); - } - - @Test - void snapshotsKeepMissingStackCellsAsNullsUpToCellCount() throws Exception { - String vin = "LNVFC000000000001"; - String key = "2026/06/22/GB32960/" + vin + "/stack-tail-missing.bin"; - writeArchive(key, stackFrame(1, 4, true)); - - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - new CapturingTdengineReader(List.of(rawRow("partial", "archive://" + key))), - decoder(), - archiveRoot, - null); - - List snapshots = service.snapshots( - LocalDate.parse("2026-06-22"), - LocalDate.parse("2026-06-22"), - HistoryQueryOrder.DESC, - 10, - vin, - null); - - @SuppressWarnings("unchecked") - Map stackBlock = (Map) snapshots.getFirst().blocks().stream() - .filter(block -> "GD_FC_STACK".equals(block.get("type"))) - .findFirst() - .orElseThrow() - .get("data"); - @SuppressWarnings("unchecked") - List> stacks = (List>) stackBlock.get("stacks"); - @SuppressWarnings("unchecked") - List voltages = (List) stacks.getFirst().get("frameCellVoltagesV"); - - assertThat(stacks.getFirst()).containsEntry("frameCellStart", 1); - assertThat(stacks.getFirst()).containsEntry("frameCellCount", 9); - assertThat(stacks.getFirst()).containsEntry("frameCellComplete", false); - assertThat(stacks.getFirst()).containsEntry("frameCellMissingCount", 5L); - assertThat(voltages).hasSize(9); - assertThat(voltages.subList(4, 9)).containsOnlyNulls(); - } - - @Test - void snapshotFieldsReturnsOnlySelectedFields() throws Exception { - String vin = "LNVFC000000000001"; - String key = "2026/06/22/GB32960/" + vin + "/selected-fields.bin"; - writeArchive(key, realtimeFrame()); - - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - new CapturingTdengineReader(List.of(rawRow("selected", "archive://" + key))), - decoder(), - archiveRoot, - null); - - List snapshots = service.snapshotFields( - QueryTimeRange.parse("2026-06-22T18:00:00", "2026-06-22T18:00:00"), - HistoryQueryOrder.DESC, - 10, - vin, - null, - "VEHICLE.totalMileageKm,GD_FC_DCDC.inputVoltageV,GD_FC_DCDC.outputCurrentA"); - - assertThat(snapshots).hasSize(1); - assertThat(snapshots.getFirst().fields()) - .containsEntry("VEHICLE.totalMileageKm", new BigDecimal("10000")) - .containsEntry("GD_FC_DCDC.inputVoltageV", new BigDecimal("300")) - .containsEntry("GD_FC_DCDC.outputCurrentA", new BigDecimal("110")); - } - - @Test - void snapshotFieldsCsvUsesChineseDictionaryHeaders() throws Exception { - String vin = "LNVFC000000000001"; - String key = "2026/06/22/GB32960/" + vin + "/selected-fields-csv.bin"; - writeArchive(key, realtimeFrame()); - - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - new CapturingTdengineReader(List.of(rawRow("selected-csv", "archive://" + key))), - decoder(), - archiveRoot, - null); - - String csv = service.snapshotFieldsCsv( - QueryTimeRange.parse("2026-06-22T18:00:00", "2026-06-22T18:00:00"), - HistoryQueryOrder.DESC, - 10, - vin, - null, - "VEHICLE.totalMileageKm,GD_FC_DCDC.inputVoltageV"); - - assertThat(csv).startsWith("VIN,事件时间,首次接收时间,最后接收时间,原始包,整车数据-累计里程(km),广东燃料电池 DC/DC 数据-输入电压(V)\n"); - assertThat(csv).contains("LNVFC000000000001,2026-06-22T10:00:00Z"); - assertThat(csv).contains(",10000,300\n"); - } - - @Test - void snapshotFieldsRejectsUnknownDictionaryFields() throws Exception { - Gb32960DecodedFrameService service = new Gb32960DecodedFrameService( - (TdengineHistoryReader) null, - decoder(), - archiveRoot, - null); - - assertThatThrownBy(() -> service.snapshotFields( - QueryTimeRange.parse("2026-06-22T18:00:00", "2026-06-22T18:00:00"), - HistoryQueryOrder.DESC, - 10, - "LNVFC000000000001", - null, - "VEHICLE.totalMileageKm,VEHICLE.notExists")) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("unknown gb32960 snapshot fields: VEHICLE.notExists"); - } - - private static Gb32960MessageDecoder decoder() { - List standardParsers = List.of(new VehicleV2016BlockParser()); - VendorExtensionCatalog catalog = new VendorExtensionCatalog(Map.of( - "guangdong-fc", List.of(new GdFcDcDcBlockParser(), new GdFcStackBlockParser()))); - Gb32960ProfileRegistry registry = new Gb32960ProfileRegistry( - standardParsers, catalog, Set.of("guangdong-fc")); - VendorExtensionSelector selector = ctx -> "guangdong-fc"; - return new Gb32960MessageDecoder(new Gb32960BodyParser(registry, selector)); - } - - private static TdengineRawFrameRow rawRow(String eventId, String rawArchiveUri) { - return rawRow(eventId, rawArchiveUri, "LNVFC000000000001"); - } - - private static TdengineRawFrameRow rawRow(String eventId, String rawArchiveUri, String vin) { - return new TdengineRawFrameRow( - Instant.parse("2026-06-22T10:00:00Z"), - eventId, - Instant.parse("2026-06-22T10:00:01Z"), - 0x02, - 0, - Instant.parse("2026-06-22T10:00:00Z"), - rawArchiveUri, - "sha256:test", - 64, - "SUCCEEDED", - "", - "127.0.0.1:32960", - "{\"platformAccount\":\"Hyundai\"}", - "", - "GB32960", - vin, - vin, - ""); - } - - private static byte[] realtimeFrame() { - ByteArrayOutputStream body = new ByteArrayOutputStream(); - body.write(26); body.write(6); body.write(22); body.write(18); body.write(0); body.write(0); - - body.write(0x01); - body.write(0x01); - body.write(0x03); - body.write(0x02); - writeU16(body, 0); - writeU32(body, 100_000L); - writeU16(body, 5605); - writeU16(body, 10000); - body.write(85); - body.write(0x01); - body.write(0x00); - writeU16(body, 10000); - body.write(0x00); - body.write(0x00); - - body.write(0x32); - writeU16(body, 3000); - writeU16(body, 1200); - writeU16(body, 5600); - writeU16(body, 1100); - body.write(105); - - byte[] bodyBytes = body.toByteArray(); - ByteArrayOutputStream out = new ByteArrayOutputStream(); - out.write(0x23); out.write(0x23); - out.write(0x02); - out.write(0xFE); - byte[] vin = "LNVFC000000000001".getBytes(StandardCharsets.US_ASCII); - out.write(vin, 0, 17); - out.write(0x01); - writeU16(out, bodyBytes.length); - out.write(bodyBytes, 0, bodyBytes.length); - byte[] almost = out.toByteArray(); - out.write(BccChecksum.compute(almost, 2, almost.length - 2) & 0xFF); - return out.toByteArray(); - } - - private static byte[] platformLoginFrame() { - ByteArrayOutputStream body = new ByteArrayOutputStream(); - body.write(26); body.write(6); body.write(22); body.write(18); body.write(0); body.write(0); - writeU16(body, 1); - writeAsciiPadded(body, "Hyundai", 12); - writeAsciiPadded(body, "f2e3445d7cda409fb4f", 20); - body.write(0x01); - - byte[] bodyBytes = body.toByteArray(); - ByteArrayOutputStream out = new ByteArrayOutputStream(); - out.write(0x23); out.write(0x23); - out.write(0x05); - out.write(0xFE); - for (int i = 0; i < 17; i++) { - out.write(0); - } - out.write(0x01); - writeU16(out, bodyBytes.length); - out.write(bodyBytes, 0, bodyBytes.length); - byte[] almost = out.toByteArray(); - out.write(BccChecksum.compute(almost, 2, almost.length - 2) & 0xFF); - return out.toByteArray(); - } - - private static void writeAsciiPadded(ByteArrayOutputStream out, String value, int length) { - byte[] bytes = value.getBytes(StandardCharsets.US_ASCII); - int count = Math.min(bytes.length, length); - out.write(bytes, 0, count); - for (int i = count; i < length; i++) { - out.write(0); - } - } - - private void writeArchive(String key, byte[] frame) throws Exception { - Path rawFile = archiveRoot.resolve(key); - Files.createDirectories(rawFile.getParent()); - Files.write(rawFile, frame); - } - - private static byte[] stackFrame(int frameCellStart, int frameCellCount, boolean includeVehicle) { - ByteArrayOutputStream body = new ByteArrayOutputStream(); - body.write(26); body.write(6); body.write(22); body.write(18); body.write(0); body.write(0); - - if (includeVehicle) { - body.write(0x01); - body.write(0x01); - body.write(0x03); - body.write(0x02); - writeU16(body, 0); - writeU32(body, 100_000L); - writeU16(body, 5605); - writeU16(body, 10000); - body.write(85); - body.write(0x01); - body.write(0x00); - writeU16(body, 10000); - body.write(0x00); - body.write(0x00); - } - - body.write(0x30); - body.write(0x01); - body.write(0x01); - body.write(90); - writeU16(body, 1850); - writeU16(body, 1500); - body.write(70); - writeU16(body, 12); - writeU16(body, 88); - writeU16(body, 1009); - writeU16(body, 1001); - writeU16(body, 1005); - writeU16(body, 9); - writeU16(body, frameCellStart); - body.write(frameCellCount); - for (int i = 0; i < frameCellCount; i++) { - writeU16(body, 1000 + frameCellStart + i); - } - - byte[] bodyBytes = body.toByteArray(); - ByteArrayOutputStream out = new ByteArrayOutputStream(); - out.write(0x23); out.write(0x23); - out.write(0x02); - out.write(0xFE); - byte[] vin = "LNVFC000000000001".getBytes(StandardCharsets.US_ASCII); - out.write(vin, 0, 17); - out.write(0x01); - writeU16(out, bodyBytes.length); - out.write(bodyBytes, 0, bodyBytes.length); - byte[] almost = out.toByteArray(); - out.write(BccChecksum.compute(almost, 2, almost.length - 2) & 0xFF); - return out.toByteArray(); - } - - private static void writeU16(ByteArrayOutputStream os, int v) { - os.write((v >> 8) & 0xFF); - os.write(v & 0xFF); - } - - private static void writeU32(ByteArrayOutputStream os, long v) { - os.write((int) ((v >> 24) & 0xFF)); - os.write((int) ((v >> 16) & 0xFF)); - os.write((int) ((v >> 8) & 0xFF)); - os.write((int) (v & 0xFF)); - } - - private static final class CapturingTdengineReader implements TdengineHistoryReader { - private final List rows; - private final TdenginePageCursor nextCursor; - private TdengineRawFrameQuery lastQuery; - - private CapturingTdengineReader(List rows) { - this(rows, null); - } - - private CapturingTdengineReader(List rows, TdenginePageCursor nextCursor) { - this.rows = rows; - this.nextCursor = nextCursor; - } - - @Override - public TdenginePage queryRawFrames(TdengineRawFrameQuery query) { - lastQuery = query; - return new TdenginePage<>(rows, java.util.Optional.ofNullable(nextCursor)); - } - - @Override - public TdenginePage queryLocations(TdengineLocationQuery query) { - throw new UnsupportedOperationException(); - } - - } -} diff --git a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Gb32960FieldDictionaryTest.java b/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Gb32960FieldDictionaryTest.java deleted file mode 100644 index 250194de..00000000 --- a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Gb32960FieldDictionaryTest.java +++ /dev/null @@ -1,30 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import org.junit.jupiter.api.Test; - -import java.util.Map; - -import static java.util.stream.Collectors.toMap; -import static org.assertj.core.api.Assertions.assertThat; - -class Gb32960FieldDictionaryTest { - - @Test - void vehicleRunningModeIncludesProtocolValueMappings() { - Gb32960FieldDictionary.Field runningMode = Gb32960FieldDictionary.packets().stream() - .filter(packet -> packet.code().equals("VEHICLE")) - .flatMap(packet -> packet.fields().stream()) - .filter(field -> field.key().equals("runningMode")) - .findFirst() - .orElseThrow(); - - Map namesByCode = runningMode.valueMappings().stream() - .collect(toMap(Gb32960FieldDictionary.ValueMapping::code, - Gb32960FieldDictionary.ValueMapping::nameZh)); - - assertThat(namesByCode) - .containsEntry("01", "纯电") - .containsEntry("02", "混合动力") - .containsEntry("03", "燃料电池"); - } -} diff --git a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Jt808LocationHistoryControllerTest.java b/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Jt808LocationHistoryControllerTest.java deleted file mode 100644 index 489afc97..00000000 --- a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Jt808LocationHistoryControllerTest.java +++ /dev/null @@ -1,140 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineLocationQuery; -import com.lingniu.ingest.tdenginehistory.TdengineLocationRow; -import com.lingniu.ingest.tdenginehistory.TdenginePage; -import com.lingniu.ingest.tdenginehistory.TdenginePageCursor; -import com.lingniu.ingest.tdenginehistory.TdengineQueryOrder; -import org.junit.jupiter.api.Test; -import org.springframework.http.HttpStatus; -import org.springframework.web.server.ResponseStatusException; - -import java.io.IOException; -import java.time.Instant; -import java.util.List; -import java.util.Optional; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class Jt808LocationHistoryControllerTest { - - @Test - void queriesLocationsByPhoneWithKeysetCursor() throws Exception { - CapturingReader reader = new CapturingReader(new TdenginePage<>( - List.of(row("fact-1", "2026-06-29T05:00:01Z")), - Optional.of(new TdenginePageCursor(Instant.parse("2026-06-29T05:00:01Z"), "fact-1")))); - Jt808LocationHistoryController controller = new Jt808LocationHistoryController(reader); - - Jt808LocationHistoryController.LocationPageResponse response = controller.locations( - "g7gps", - "2026-06-29T13:00:00+08:00", - "2026-06-29T13:10:00+08:00", - TdengineQueryOrder.DESC, - 50, - "2026-06-29T05:00:00Z", - "fact-0"); - - assertThat(reader.query.protocol()).isEqualTo("JT808"); - assertThat(reader.query.vehicleKey()).isEqualTo("jt808:g7gps"); - assertThat(reader.query.order()).isEqualTo(TdengineQueryOrder.DESC); - assertThat(reader.query.limit()).isEqualTo(50); - assertThat(reader.query.cursor()).isEqualTo(new TdenginePageCursor( - Instant.parse("2026-06-29T05:00:00Z"), "fact-0")); - assertThat(response.items()) - .extracting(Jt808LocationHistoryController.LocationResponse::factId) - .containsExactly("fact-1"); - assertThat(response.items().getFirst().longitude()).isEqualTo(113.12); - assertThat(response.nextCursor()).isEqualTo(new Jt808LocationHistoryController.CursorResponse( - "2026-06-29T05:00:01Z", "fact-1")); - } - - @Test - void returnsLocationRawUriFromCoreColumn() throws Exception { - CapturingReader reader = new CapturingReader(new TdenginePage<>( - List.of(row( - "fact-blank-raw", - "2026-06-29T05:00:01Z", - "archive://jt808/core-frame.bin")), - Optional.empty())); - Jt808LocationHistoryController controller = new Jt808LocationHistoryController(reader); - - Jt808LocationHistoryController.LocationPageResponse response = controller.locations( - "g7gps", - "2026-06-29T13:00:00+08:00", - "2026-06-29T13:10:00+08:00", - TdengineQueryOrder.DESC, - 50, - null, - null); - - assertThat(response.items().getFirst().rawUri()) - .isEqualTo("archive://jt808/core-frame.bin"); - } - - @Test - void rejectsBlankPhone() { - Jt808LocationHistoryController controller = new Jt808LocationHistoryController( - new CapturingReader(new TdenginePage<>(List.of(), Optional.empty()))); - - assertThatThrownBy(() -> controller.locations( - " ", - "2026-06-29T13:00:00+08:00", - "2026-06-29T13:10:00+08:00", - TdengineQueryOrder.DESC, - 50, - null, - null)) - .isInstanceOfSatisfying(ResponseStatusException.class, ex -> - assertThat(ex.getStatusCode()).isEqualTo(HttpStatus.BAD_REQUEST)); - } - - private static TdengineLocationRow row(String factId, String ts) { - return row(factId, ts, "archive://jt808/frame-1.bin"); - } - - private static TdengineLocationRow row(String factId, String ts, String rawUri) { - Instant instant = Instant.parse(ts); - return new TdengineLocationRow( - instant, - factId, - "frame-1", - instant.plusMillis(500), - 113.12, - 23.45, - 8.0, - 42.5, - 90.0, - 1L, - 3L, - null, - rawUri, - "JT808", - "jt808:g7gps", - "", - "g7gps"); - } - - private static final class CapturingReader implements TdengineHistoryReader { - private final TdenginePage response; - private TdengineLocationQuery query; - - private CapturingReader(TdenginePage response) { - this.response = response; - } - - @Override - public TdenginePage queryRawFrames( - com.lingniu.ingest.tdenginehistory.TdengineRawFrameQuery query) { - throw new UnsupportedOperationException(); - } - - @Override - public TdenginePage queryLocations(TdengineLocationQuery query) throws IOException { - this.query = query; - return response; - } - - } -} diff --git a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Jt808RawFrameHistoryControllerTest.java b/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Jt808RawFrameHistoryControllerTest.java deleted file mode 100644 index 6cee6ebe..00000000 --- a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/Jt808RawFrameHistoryControllerTest.java +++ /dev/null @@ -1,124 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineLocationQuery; -import com.lingniu.ingest.tdenginehistory.TdengineLocationRow; -import com.lingniu.ingest.tdenginehistory.TdenginePage; -import com.lingniu.ingest.tdenginehistory.TdenginePageCursor; -import com.lingniu.ingest.tdenginehistory.TdengineQueryOrder; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameQuery; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameRow; -import org.junit.jupiter.api.Test; -import org.springframework.http.HttpStatus; -import org.springframework.web.server.ResponseStatusException; - -import java.io.IOException; -import java.time.Instant; -import java.util.List; -import java.util.Optional; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class Jt808RawFrameHistoryControllerTest { - - @Test - void queriesRawFramesByPhoneMessageIdAndCursor() throws Exception { - CapturingReader reader = new CapturingReader(new TdenginePage<>( - List.of(row("frame-1", "2026-06-29T12:00:01Z")), - Optional.of(new TdenginePageCursor(Instant.parse("2026-06-29T12:00:01Z"), "frame-1")))); - Jt808RawFrameHistoryController controller = new Jt808RawFrameHistoryController(reader); - - Jt808RawFrameHistoryController.RawFramePageResponse response = controller.rawFrames( - "13079963320", - null, - null, - 0x0100, - null, - "2026-06-29T00:00:00+08:00", - "2026-06-29T23:59:59+08:00", - TdengineQueryOrder.DESC, - 20, - "2026-06-29T12:00:00Z", - "frame-0"); - - assertThat(reader.query.protocol()).isEqualTo("JT808"); - assertThat(reader.query.vehicleKey()).isBlank(); - assertThat(reader.query.phone()).isEqualTo("13079963320"); - assertThat(reader.query.messageId()).isEqualTo(0x0100); - assertThat(reader.query.cursor()).isEqualTo(new TdenginePageCursor( - Instant.parse("2026-06-29T12:00:00Z"), "frame-0")); - assertThat(response.items()).hasSize(1); - Jt808RawFrameHistoryController.RawFrameResponse item = response.items().getFirst(); - assertThat(item.messageId()).isEqualTo(0x0100); - assertThat(item.messageIdHex()).isEqualTo("0x0100"); - assertThat(item.metadataJson()).contains("jt808.register.deviceId"); - assertThat(response.nextCursor()).isEqualTo(new Jt808RawFrameHistoryController.CursorResponse( - "2026-06-29T12:00:01Z", "frame-1")); - } - - @Test - void rejectsUnboundedRawFrameQuery() { - Jt808RawFrameHistoryController controller = new Jt808RawFrameHistoryController( - new CapturingReader(new TdenginePage<>(List.of(), Optional.empty()))); - - assertThatThrownBy(() -> controller.rawFrames( - null, - null, - null, - null, - null, - "2026-06-29", - "2026-06-29", - TdengineQueryOrder.DESC, - 20, - null, - null)) - .isInstanceOfSatisfying(ResponseStatusException.class, ex -> - assertThat(ex.getStatusCode()).isEqualTo(HttpStatus.BAD_REQUEST)); - } - - private static TdengineRawFrameRow row(String frameId, String ts) { - Instant instant = Instant.parse(ts); - return new TdengineRawFrameRow( - instant, - frameId, - instant.plusMillis(500), - 0x0100, - 0, - instant, - "archive://2026/06/29/JT808/unknown/" + frameId + ".bin", - "sha256:" + frameId, - 72, - "SUCCEEDED", - "", - "222.66.200.68:41000", - "{\"jt808.register.deviceId\":\"9963320\",\"jt808.register.plate\":\"沪A61559F\"}", - "", - "JT808", - "jt808:13079963320", - "unknown", - "13079963320"); - } - - private static final class CapturingReader implements TdengineHistoryReader { - private final TdenginePage response; - private TdengineRawFrameQuery query; - - private CapturingReader(TdenginePage response) { - this.response = response; - } - - @Override - public TdenginePage queryRawFrames(TdengineRawFrameQuery query) throws IOException { - this.query = query; - return response; - } - - @Override - public TdenginePage queryLocations(TdengineLocationQuery query) { - throw new UnsupportedOperationException(); - } - - } -} diff --git a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/LocationHistoryControllerTest.java b/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/LocationHistoryControllerTest.java deleted file mode 100644 index f572aa76..00000000 --- a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/LocationHistoryControllerTest.java +++ /dev/null @@ -1,132 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineLocationQuery; -import com.lingniu.ingest.tdenginehistory.TdengineLocationRow; -import com.lingniu.ingest.tdenginehistory.TdenginePage; -import com.lingniu.ingest.tdenginehistory.TdenginePageCursor; -import com.lingniu.ingest.tdenginehistory.TdengineQueryOrder; -import org.junit.jupiter.api.Test; -import org.springframework.http.HttpStatus; -import org.springframework.web.server.ResponseStatusException; - -import java.time.Instant; -import java.util.List; -import java.util.Optional; -import java.util.concurrent.atomic.AtomicReference; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class LocationHistoryControllerTest { - - @Test - void queriesGenericLocationHistoryWithCursor() throws Exception { - AtomicReference captured = new AtomicReference<>(); - TdengineHistoryReader reader = new StubReader(captured, new TdenginePage<>( - List.of(row("fact-1")), - Optional.of(new TdenginePageCursor(Instant.parse("2026-06-29T05:00:01Z"), "fact-1")))); - LocationHistoryController controller = new LocationHistoryController(reader); - - LocationHistoryController.LocationPageResponse response = controller.locations( - "mqtt_yutong", - null, - "LMRKH9AC2R1004087", - null, - "2026-06-29T13:00:00", - "2026-06-29T13:10:00", - TdengineQueryOrder.DESC, - 100, - null, - null); - - assertThat(captured.get().protocol()).isEqualTo("MQTT_YUTONG"); - assertThat(captured.get().vehicleKey()).isEqualTo("LMRKH9AC2R1004087"); - assertThat(response.items()).extracting(LocationHistoryController.LocationResponse::factId) - .containsExactly("fact-1"); - assertThat(response.nextCursor()).isEqualTo(new LocationHistoryController.CursorResponse( - "2026-06-29T05:00:01Z", "fact-1")); - } - - @Test - void mapsJt808PhoneToVehicleKey() throws Exception { - AtomicReference captured = new AtomicReference<>(); - LocationHistoryController controller = new LocationHistoryController( - new StubReader(captured, new TdenginePage<>(List.of(), null))); - - controller.locations( - "JT808", - null, - null, - "g7gps", - "2026-06-29T13:00:00", - "2026-06-29T13:10:00", - TdengineQueryOrder.DESC, - 100, - null, - null); - - assertThat(captured.get().vehicleKey()).isEqualTo("jt808:g7gps"); - } - - @Test - void rejectsRemovedXindaProtocolOnDefaultLocationApi() { - AtomicReference captured = new AtomicReference<>(); - LocationHistoryController controller = new LocationHistoryController( - new StubReader(captured, new TdenginePage<>(List.of(), null))); - - assertThatThrownBy(() -> controller.locations( - "XINDA_PUSH", - null, - "LB9A32A24P0LS1270", - null, - "2026-06-29T13:00:00", - "2026-06-29T13:10:00", - TdengineQueryOrder.DESC, - 100, - null, - null)) - .isInstanceOfSatisfying(ResponseStatusException.class, ex -> - assertThat(ex.getStatusCode()).isEqualTo(HttpStatus.BAD_REQUEST)); - assertThat(captured).hasValue(null); - } - - private static TdengineLocationRow row(String factId) { - Instant ts = Instant.parse("2026-06-29T05:00:00Z"); - return new TdengineLocationRow( - ts, - factId, - "frame-1", - ts, - 121.1, - 30.5, - 0.0, - 32.0, - 90.0, - 0, - 1, - null, - "", - "MQTT_YUTONG", - "LMRKH9AC2R1004087", - "LMRKH9AC2R1004087", - ""); - } - - private record StubReader(AtomicReference captured, - TdenginePage page) implements TdengineHistoryReader { - - @Override - public com.lingniu.ingest.tdenginehistory.TdenginePage queryRawFrames( - com.lingniu.ingest.tdenginehistory.TdengineRawFrameQuery query) { - throw new UnsupportedOperationException(); - } - - @Override - public TdenginePage queryLocations(TdengineLocationQuery query) { - captured.set(query); - return page; - } - - } -} diff --git a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/RawFrameHistoryControllerTest.java b/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/RawFrameHistoryControllerTest.java deleted file mode 100644 index 9d59ade1..00000000 --- a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/RawFrameHistoryControllerTest.java +++ /dev/null @@ -1,277 +0,0 @@ -package com.lingniu.ingest.eventhistory; - -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineLocationQuery; -import com.lingniu.ingest.tdenginehistory.TdengineLocationRow; -import com.lingniu.ingest.tdenginehistory.TdenginePage; -import com.lingniu.ingest.tdenginehistory.TdenginePageCursor; -import com.lingniu.ingest.tdenginehistory.TdengineQueryOrder; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameQuery; -import com.lingniu.ingest.tdenginehistory.TdengineRawFrameRow; -import org.junit.jupiter.api.Test; -import org.springframework.http.HttpStatus; -import org.springframework.web.server.ResponseStatusException; - -import java.io.IOException; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; -import java.util.Optional; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class RawFrameHistoryControllerTest { - - @Test - void queriesRawFramesByProtocolHexMessageIdAndCursor() throws Exception { - CapturingReader reader = new CapturingReader(new TdenginePage<>( - List.of(row("frame-1", "2026-06-29T12:00:01Z")), - Optional.of(new TdenginePageCursor(Instant.parse("2026-06-29T12:00:01Z"), "frame-1")))); - RawFrameHistoryController controller = new RawFrameHistoryController(reader); - - RawFrameHistoryController.RawFramePageResponse response = controller.rawFrames( - "mqtt_yutong", - null, - "LB9A32A24P0LS1270", - null, - "0x0200", - "not_parsed", - "2026-06-29T00:00:00+08:00", - "2026-06-29T23:59:59+08:00", - TdengineQueryOrder.DESC, - 20, - "2026-06-29T12:00:00Z", - "frame-0"); - - assertThat(reader.query.protocol()).isEqualTo("MQTT_YUTONG"); - assertThat(reader.query.vin()).isEqualTo("LB9A32A24P0LS1270"); - assertThat(reader.query.messageId()).isEqualTo(0x0200); - assertThat(reader.query.parseStatus()).isEqualTo("NOT_PARSED"); - assertThat(reader.query.cursor()).isEqualTo(new TdenginePageCursor( - Instant.parse("2026-06-29T12:00:00Z"), "frame-0")); - assertThat(response.items()).hasSize(1); - RawFrameHistoryController.RawFrameResponse item = response.items().getFirst(); - assertThat(item.messageId()).isEqualTo(0x0200); - assertThat(item.messageIdHex()).isEqualTo("0x0200"); - assertThat(item.rawUri()).contains("MQTT_YUTONG"); - assertThat(item.metadataJson()).contains("rawArchiveUri"); - assertThat(item.metadata()) - .containsEntry("rawArchiveUri", item.rawUri()) - .containsEntry("plateNo", "粤BD12345") - .containsEntry("rawJson", "{\"carName\":\"粤BD12345\"}"); - assertThat(item.parsedFields()) - .containsEntry("raw.rawArchiveUri", item.rawUri()) - .containsEntry("raw.plateNo", "粤BD12345"); - assertThat(reader.locationRawUri).isNull(); - assertThat(response.nextCursor()).isEqualTo(new RawFrameHistoryController.CursorResponse( - "2026-06-29T12:00:01Z", "frame-1")); - } - - @Test - void returnsParsedJsonWithoutQueryingDerivedTelemetryTables() throws Exception { - Instant ts = Instant.parse("2026-06-29T12:00:01Z"); - TdengineRawFrameRow raw = new TdengineRawFrameRow( - ts, - "frame-1", - ts.plusMillis(500), - 0x0200, - 0, - ts, - "archive://2026/06/29/JT808/VIN-JT808-1/frame-1.bin", - "sha256:frame-1", - 72, - "SUCCEEDED", - "", - "115.231.168.135:43625", - "{\"rawArchiveUri\":\"archive://2026/06/29/JT808/VIN-JT808-1/frame-1.bin\"}", - "{\"location\":{\"longitude\":121.312516,\"latitude\":30.832808,\"speedKmh\":25.0}," - + "\"extras\":{\"gpsMileageKm\":14504.446}}", - "JT808", - "VIN-JT808-1", - "VIN-JT808-1", - "g7gps"); - CapturingReader reader = new CapturingReader(new TdenginePage<>(List.of(raw), Optional.empty())); - reader.locationsByRawUri.add(new TdengineLocationRow( - ts, - "loc-1", - "frame-1", - ts.plusMillis(500), - 121.312516, - 30.832808, - 6.0, - 25.0, - 333.0, - 0, - 524290, - 14504.446, - raw.rawUri(), - "JT808", - "VIN-JT808-1", - "VIN-JT808-1", - "g7gps")); - RawFrameHistoryController controller = new RawFrameHistoryController(reader); - - RawFrameHistoryController.RawFramePageResponse response = controller.rawFrames( - "jt808", - null, - "VIN-JT808-1", - null, - "0x0200", - null, - "2026-06-29T00:00:00+08:00", - "2026-06-29T23:59:59+08:00", - TdengineQueryOrder.DESC, - 20, - null, - null); - - RawFrameHistoryController.RawFrameResponse item = response.items().getFirst(); - assertThat(reader.locationRawUri).isNull(); - assertThat(item.parsedFields()) - .containsKey("location") - .containsKey("extras"); - } - - @Test - void preservesNullValuesFromParsedJsonWithoutFailingResponseMapping() throws Exception { - Instant ts = Instant.parse("2026-06-29T12:00:01Z"); - TdengineRawFrameRow raw = new TdengineRawFrameRow( - ts, - "frame-with-null-json", - ts.plusMillis(500), - 0x0002, - 0, - ts, - "archive://2026/06/29/GB32960/LB9A32A24P0LS1270/frame-with-null-json.bin", - "sha256:null-json", - 128, - "NOT_PARSED", - "", - "8.134.95.166:49706", - "{\"rawArchiveUri\":\"archive://2026/06/29/GB32960/LB9A32A24P0LS1270/frame-with-null-json.bin\"}", - "{\"header\":{\"command\":\"REALTIME_REPORT\"},\"commandBody\":null,\"signature\":null}", - "GB32960", - "LB9A32A24P0LS1270", - "LB9A32A24P0LS1270", - ""); - RawFrameHistoryController controller = new RawFrameHistoryController( - new CapturingReader(new TdenginePage<>(List.of(raw), Optional.empty()))); - - RawFrameHistoryController.RawFramePageResponse response = controller.rawFrames( - "gb32960", - null, - "LB9A32A24P0LS1270", - null, - null, - "not_parsed", - "2026-06-29T00:00:00+08:00", - "2026-06-29T23:59:59+08:00", - TdengineQueryOrder.DESC, - 20, - null, - null); - - Map parsedFields = response.items().getFirst().parsedFields(); - assertThat(parsedFields).containsKeys("header", "commandBody", "signature"); - assertThat(parsedFields.get("commandBody")).isNull(); - assertThat(parsedFields.get("signature")).isNull(); - } - - @Test - void rejectsRawFrameQueryWithoutAnyNarrowingFilter() { - RawFrameHistoryController controller = new RawFrameHistoryController( - new CapturingReader(new TdenginePage<>(List.of(), Optional.empty()))); - - assertThatThrownBy(() -> controller.rawFrames( - "JT808", - null, - null, - null, - null, - null, - "2026-06-29", - "2026-06-30", - TdengineQueryOrder.DESC, - 20, - null, - null)) - .isInstanceOfSatisfying(ResponseStatusException.class, ex -> - assertThat(ex.getStatusCode()).isEqualTo(HttpStatus.BAD_REQUEST)); - } - - @Test - void rejectsRemovedXindaProtocolOnDefaultRawFrameApi() { - RawFrameHistoryController controller = new RawFrameHistoryController( - new CapturingReader(new TdenginePage<>(List.of(), Optional.empty()))); - - assertThatThrownBy(() -> controller.rawFrames( - "XINDA_PUSH", - null, - "LB9A32A24P0LS1270", - null, - null, - null, - "2026-06-29", - "2026-06-30", - TdengineQueryOrder.DESC, - 20, - null, - null)) - .isInstanceOfSatisfying(ResponseStatusException.class, ex -> - assertThat(ex.getStatusCode()).isEqualTo(HttpStatus.BAD_REQUEST)); - } - - private static TdengineRawFrameRow row(String frameId, String ts) { - Instant instant = Instant.parse(ts); - return new TdengineRawFrameRow( - instant, - frameId, - instant.plusMillis(500), - 0x0200, - 0, - instant, - "archive://2026/06/29/MQTT_YUTONG/VEHICLE-MQTT-1/" + frameId + ".bin", - "sha256:" + frameId, - 72, - "NOT_PARSED", - "", - "mqtt-yutong", - "{\"rawArchiveUri\":\"archive://2026/06/29/MQTT_YUTONG/VEHICLE-MQTT-1/" + frameId - + ".bin\",\"plateNo\":\"粤BD12345\",\"rawJson\":\"{\\\"carName\\\":\\\"粤BD12345\\\"}\"}", - "", - "MQTT_YUTONG", - "LB9A32A24P0LS1270", - "LB9A32A24P0LS1270", - ""); - } - - private static final class CapturingReader implements TdengineHistoryReader { - private final TdenginePage response; - private final List locationsByRawUri = new ArrayList<>(); - private TdengineRawFrameQuery query; - private String locationRawUri; - - private CapturingReader(TdenginePage response) { - this.response = response; - } - - @Override - public TdenginePage queryRawFrames(TdengineRawFrameQuery query) throws IOException { - this.query = query; - return response; - } - - @Override - public TdenginePage queryLocations(TdengineLocationQuery query) { - throw new UnsupportedOperationException(); - } - - @Override - public List queryLocationsByRawUri(String protocol, String rawUri, int limit) { - locationRawUri = rawUri; - return locationsByRawUri; - } - } -} diff --git a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/config/EventHistoryAutoConfigurationTest.java b/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/config/EventHistoryAutoConfigurationTest.java deleted file mode 100644 index b0905dce..00000000 --- a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/config/EventHistoryAutoConfigurationTest.java +++ /dev/null @@ -1,165 +0,0 @@ -package com.lingniu.ingest.eventhistory.config; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.api.consumer.EnvelopeDeadLetterSink; -import com.lingniu.ingest.eventhistory.ApiExceptionHandler; -import com.lingniu.ingest.eventhistory.EventHistoryEnvelopeIngestor; -import com.lingniu.ingest.eventhistory.Gb32960DecodedFrameService; -import com.lingniu.ingest.eventhistory.Gb32960FrameController; -import com.lingniu.ingest.eventhistory.Jt808LocationHistoryController; -import com.lingniu.ingest.eventhistory.Jt808RawFrameHistoryController; -import com.lingniu.ingest.eventhistory.LocationHistoryController; -import com.lingniu.ingest.eventhistory.RawFrameHistoryController; -import com.lingniu.ingest.protocol.gb32960.codec.Gb32960MessageDecoder; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryWriter; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; - -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.mock; - -class EventHistoryAutoConfigurationTest { - - private final ApplicationContextRunner contextRunner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(EventHistoryAutoConfiguration.class)) - .withBean(EnvelopeDeadLetterSink.class, () -> record -> {}) - .withBean(TdengineHistoryWriter.class, () -> mock(TdengineHistoryWriter.class)); - - @Test - void createsHistoryBeansWhenEnabled() { - contextRunner - .withPropertyValues( - "lingniu.ingest.event-history.enabled=true", - "lingniu.ingest.sink.kafka.consumer.enabled=true") - .run(context -> { - assertThat(context).hasSingleBean(EventHistoryEnvelopeIngestor.class); - assertThat(context.getBeansOfType(EnvelopeConsumerProcessor.class)) - .containsOnlyKeys( - "eventHistoryEnvelopeConsumerProcessor", - "eventHistoryRawEnvelopeConsumerProcessor"); - assertThat(context).hasSingleBean(Gb32960MessageDecoder.class); - assertThat(context).hasSingleBean(Gb32960DecodedFrameService.class); - assertThat(context).doesNotHaveBean(Gb32960FrameController.class); - }); - } - - @Test - void backsOffWhenDisabled() { - contextRunner.run(context -> { - assertThat(context).doesNotHaveBean(EventHistoryEnvelopeIngestor.class); - }); - } - - @Test - void createsGb32960FrameBeansWhenDecoderExists() { - contextRunner - .withPropertyValues( - "lingniu.ingest.event-history.enabled=true", - "lingniu.ingest.event-history.api.specialized-enabled=true", - "lingniu.ingest.event-history.archive-path=/tmp/lingniu-test-archive") - .withBean(Gb32960MessageDecoder.class, () -> mock(Gb32960MessageDecoder.class)) - .run(context -> { - assertThat(context).hasSingleBean(Gb32960DecodedFrameService.class); - assertThat(context).hasSingleBean(Gb32960FrameController.class); - }); - } - - @Test - void createsGb32960FrameBeansWithTdengineReader() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(EventHistoryAutoConfiguration.class)) - .withPropertyValues( - "lingniu.ingest.event-history.enabled=true", - "lingniu.ingest.event-history.api.specialized-enabled=true", - "lingniu.ingest.event-history.archive-path=/tmp/lingniu-test-archive") - .withBean(Gb32960MessageDecoder.class, () -> mock(Gb32960MessageDecoder.class)) - .withBean(TdengineHistoryReader.class, () -> mock(TdengineHistoryReader.class)) - .run(context -> { - assertThat(context).hasSingleBean(Gb32960DecodedFrameService.class); - assertThat(context).hasSingleBean(Gb32960FrameController.class); - }); - } - - @Test - void createsHistoryConsumerWithTdengineWriter() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(EventHistoryAutoConfiguration.class)) - .withBean(EnvelopeDeadLetterSink.class, () -> record -> {}) - .withBean(TdengineHistoryWriter.class, () -> mock(TdengineHistoryWriter.class)) - .withPropertyValues("lingniu.ingest.event-history.enabled=true") - .run(context -> { - assertThat(context).hasSingleBean(EventHistoryEnvelopeIngestor.class); - assertThat(context.getBeansOfType(EnvelopeConsumerProcessor.class)) - .containsOnlyKeys( - "eventHistoryEnvelopeConsumerProcessor", - "eventHistoryRawEnvelopeConsumerProcessor"); - }); - } - - @Test - void createsJt808LocationHistoryControllerWhenTdengineReaderExists() { - contextRunner - .withPropertyValues( - "lingniu.ingest.event-history.enabled=true", - "lingniu.ingest.event-history.api.specialized-enabled=true") - .withBean(TdengineHistoryReader.class, () -> mock(TdengineHistoryReader.class)) - .run(context -> assertThat(context).hasSingleBean(Jt808LocationHistoryController.class)); - } - - @Test - void createsGenericLocationAndRawControllersWithoutTelemetryFieldsByDefault() { - contextRunner - .withPropertyValues("lingniu.ingest.event-history.enabled=true") - .withBean(TdengineHistoryReader.class, () -> mock(TdengineHistoryReader.class)) - .run(context -> { - assertThat(context).hasSingleBean(LocationHistoryController.class); - assertThat(context).hasSingleBean(RawFrameHistoryController.class); - assertThat(context).hasSingleBean(ApiExceptionHandler.class); - assertThat(context).doesNotHaveBean("telemetryFieldHistoryController"); - assertThat(context).doesNotHaveBean(Jt808LocationHistoryController.class); - assertThat(context).doesNotHaveBean(Jt808RawFrameHistoryController.class); - }); - } - - @Test - void createsGenericRawFrameHistoryControllerWhenTdengineReaderExists() { - contextRunner - .withPropertyValues("lingniu.ingest.event-history.enabled=true") - .withBean(TdengineHistoryReader.class, () -> mock(TdengineHistoryReader.class)) - .run(context -> assertThat(context).hasSingleBean(RawFrameHistoryController.class)); - } - - @Test - void createsJt808RawFrameHistoryControllerWhenTdengineReaderExists() { - contextRunner - .withPropertyValues( - "lingniu.ingest.event-history.enabled=true", - "lingniu.ingest.event-history.api.specialized-enabled=true") - .withBean(TdengineHistoryReader.class, () -> mock(TdengineHistoryReader.class)) - .run(context -> assertThat(context).hasSingleBean(Jt808RawFrameHistoryController.class)); - } - - @Test - void rejectsRemovedTelemetryFieldsSwitch() { - contextRunner - .withPropertyValues( - "lingniu.ingest.event-history.enabled=true", - "lingniu.ingest.tdengine-history.telemetry-fields-enabled=true") - .withBean(TdengineHistoryReader.class, () -> mock(TdengineHistoryReader.class)) - .run(context -> assertThat(context) - .hasFailed() - .getFailure() - .hasMessageContaining("tdengine-history.telemetry-fields-enabled has been removed") - .hasMessageContaining("raw_frames")); - } - - @Test - void parsesFileUriArchiveRoot() { - assertThat(EventHistoryAutoConfiguration.archiveRoot("file:///tmp/lingniu-test-archive")) - .isEqualTo(Path.of("/tmp/lingniu-test-archive")); - } -} diff --git a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/config/EventHistoryPomBoundaryTest.java b/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/config/EventHistoryPomBoundaryTest.java deleted file mode 100644 index 388f073a..00000000 --- a/modules/services/event-history-service/src/test/java/com/lingniu/ingest/eventhistory/config/EventHistoryPomBoundaryTest.java +++ /dev/null @@ -1,91 +0,0 @@ -package com.lingniu.ingest.eventhistory.config; - -import org.junit.jupiter.api.Test; -import org.w3c.dom.Document; -import org.w3c.dom.Element; -import org.w3c.dom.NodeList; - -import javax.xml.parsers.DocumentBuilderFactory; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.ArrayList; -import java.util.List; - -import static org.assertj.core.api.Assertions.assertThat; - -class EventHistoryPomBoundaryTest { - - @Test - void tdengineJdbcDriverIsOwnedByTdengineHistoryStoreModule() throws Exception { - assertThat(directDependencies(modulePom())) - .doesNotContain("com.taosdata.jdbc:taos-jdbcdriver"); - } - - @Test - void kafkaClientsIsOwnedByKafkaSinkModule() throws Exception { - for (Path pom : servicePoms()) { - assertThat(directDependencies(readPom(pom))) - .as(pom.toString()) - .doesNotContain("org.apache.kafka:kafka-clients"); - } - } - - private static Document modulePom() throws Exception { - return readPom(repositoryRoot().resolve("modules/services/event-history-service/pom.xml")); - } - - private static Document readPom(Path pom) throws Exception { - DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance(); - factory.setFeature("http://apache.org/xml/features/disallow-doctype-decl", true); - return factory.newDocumentBuilder().parse(Files.newInputStream(pom)); - } - - private static List servicePoms() { - Path services = repositoryRoot().resolve("modules/services"); - return List.of( - services.resolve("event-history-service/pom.xml"), - services.resolve("vehicle-state-service/pom.xml"), - services.resolve("vehicle-stat-service/pom.xml")); - } - - private static List directDependencies(Document pom) { - Element dependencies = firstDirectChild(pom.getDocumentElement(), "dependencies"); - if (dependencies == null) { - return List.of(); - } - List coordinates = new ArrayList<>(); - NodeList children = dependencies.getChildNodes(); - for (int i = 0; i < children.getLength(); i++) { - if (!(children.item(i) instanceof Element dependency) || !"dependency".equals(dependency.getTagName())) { - continue; - } - Element groupId = firstDirectChild(dependency, "groupId"); - Element artifactId = firstDirectChild(dependency, "artifactId"); - if (groupId != null && artifactId != null) { - coordinates.add(groupId.getTextContent().trim() + ":" + artifactId.getTextContent().trim()); - } - } - return List.copyOf(coordinates); - } - - private static Element firstDirectChild(Element parent, String tagName) { - NodeList children = parent.getChildNodes(); - for (int i = 0; i < children.getLength(); i++) { - if (children.item(i) instanceof Element child && tagName.equals(child.getTagName())) { - return child; - } - } - return null; - } - - private static Path repositoryRoot() { - Path current = Path.of(System.getProperty("user.dir")).toAbsolutePath(); - while (current != null) { - if (Files.exists(current.resolve("pom.xml")) && Files.exists(current.resolve("modules"))) { - return current; - } - current = current.getParent(); - } - throw new IllegalStateException("repository root not found"); - } -} diff --git a/modules/services/vehicle-stat-service/pom.xml b/modules/services/vehicle-stat-service/pom.xml deleted file mode 100644 index 4855e022..00000000 --- a/modules/services/vehicle-stat-service/pom.xml +++ /dev/null @@ -1,66 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - vehicle-stat-service - vehicle-stat-service - Kafka full-field event consumer + configurable vehicle daily statistics. - - - - com.lingniu.ingest - sink-kafka - - - org.springframework.boot - spring-boot-autoconfigure - - - org.springframework.boot - spring-boot-starter-web - - - org.springframework - spring-jdbc - - - com.mysql - mysql-connector-j - - - org.springframework.boot - spring-boot-configuration-processor - true - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.mockito - mockito-core - test - - - org.springframework.boot - spring-boot-starter-test - test - - - com.h2database - h2 - test - - - diff --git a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/DailyMileageStrategy.java b/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/DailyMileageStrategy.java deleted file mode 100644 index 06eaf8c3..00000000 --- a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/DailyMileageStrategy.java +++ /dev/null @@ -1,13 +0,0 @@ -package com.lingniu.ingest.vehiclestat; - -public enum DailyMileageStrategy { - JT808_TOTAL_MILEAGE_DIFF; - - public static DailyMileageStrategy fromStorage(String value) { - if (value == null || value.isBlank()) { - return JT808_TOTAL_MILEAGE_DIFF; - } - String normalized = value.trim(); - return DailyMileageStrategy.valueOf(normalized); - } -} diff --git a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/JdbcVehicleStatMetricRepository.java b/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/JdbcVehicleStatMetricRepository.java deleted file mode 100644 index f95159a7..00000000 --- a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/JdbcVehicleStatMetricRepository.java +++ /dev/null @@ -1,184 +0,0 @@ -package com.lingniu.ingest.vehiclestat; - -import org.springframework.dao.DuplicateKeyException; -import org.springframework.jdbc.core.ConnectionCallback; -import org.springframework.jdbc.core.JdbcTemplate; - -import java.sql.Date; -import java.sql.ResultSet; -import java.sql.SQLException; -import java.time.LocalDate; -import java.util.List; -import java.util.Optional; -import java.util.OptionalDouble; - -public final class JdbcVehicleStatMetricRepository implements VehicleStatRepository { - - private static final String DAILY_MILEAGE_KEY = "daily_mileage_km"; - private static final String DAILY_MILEAGE_UNIT = "km"; - private static final String DAILY_MILEAGE_START_TOTAL_KEY = "daily_mileage_start_total_km"; - private static final String DAILY_MILEAGE_LATEST_TOTAL_KEY = "daily_mileage_latest_total_km"; - private static final String FIRST_TOTAL_COLUMN = "first_total_mileage_km"; - private static final String LATEST_TOTAL_COLUMN = "latest_total_mileage_km"; - - private final JdbcTemplate jdbcTemplate; - - public JdbcVehicleStatMetricRepository(JdbcTemplate jdbcTemplate) { - if (jdbcTemplate == null) { - throw new IllegalArgumentException("jdbcTemplate must not be null"); - } - this.jdbcTemplate = jdbcTemplate; - ensureSchema(); - } - - @Override - public Optional findDailyStat(String vin, LocalDate statDate) { - String normalizedVin = clean(vin); - List rows = jdbcTemplate.query(""" - SELECT metric_value, calculation_method - FROM vehicle_stat_metric - WHERE vin = ? AND stat_date = ? AND metric_key = ? - """, (rs, rowNum) -> new VehicleDailyStatResult( - normalizedVin, - statDate, - rs.getBigDecimal("metric_value") == null - ? OptionalDouble.empty() - : OptionalDouble.of(rs.getBigDecimal("metric_value").doubleValue()), - DailyMileageStrategy.fromStorage(rs.getString("calculation_method"))), - normalizedVin, Date.valueOf(statDate), DAILY_MILEAGE_KEY); - return rows.stream().findFirst(); - } - - @Override - public Optional recordDailyMileageSample(String vin, LocalDate statDate, double totalMileageKm) { - if (!Double.isFinite(totalMileageKm) || totalMileageKm < 0.0) { - return Optional.empty(); - } - String normalizedVin = clean(vin); - Date date = Date.valueOf(statDate); - String strategy = DailyMileageStrategy.JT808_TOTAL_MILEAGE_DIFF.name(); - DailyMileageState existingState = findDailyMileageState(normalizedVin, date).orElse(null); - double firstTotalMileage = existingState == null || !Double.isFinite(existingState.firstTotalMileageKm()) - ? totalMileageKm - : Math.min(existingState.firstTotalMileageKm(), totalMileageKm); - double latestTotalMileage = existingState == null || !Double.isFinite(existingState.latestTotalMileageKm()) - ? totalMileageKm - : Math.max(existingState.latestTotalMileageKm(), totalMileageKm); - double dailyMileageKm = latestTotalMileage - firstTotalMileage; - upsertDailyMileageMetric(normalizedVin, date, dailyMileageKm, DAILY_MILEAGE_UNIT, strategy, - firstTotalMileage, latestTotalMileage); - deleteLegacyMileageStateMetrics(normalizedVin, date); - VehicleDailyStatResult result = new VehicleDailyStatResult( - normalizedVin, - statDate, - OptionalDouble.of(dailyMileageKm), - DailyMileageStrategy.JT808_TOTAL_MILEAGE_DIFF); - return Optional.of(result); - } - - private Optional findDailyMileageState(String vin, Date statDate) { - List rows = jdbcTemplate.query(""" - SELECT first_total_mileage_km, latest_total_mileage_km - FROM vehicle_stat_metric - WHERE vin = ? AND stat_date = ? AND metric_key = ? - """, (rs, rowNum) -> new DailyMileageState( - nullableDouble(rs, FIRST_TOTAL_COLUMN), - nullableDouble(rs, LATEST_TOTAL_COLUMN)), vin, statDate, DAILY_MILEAGE_KEY); - return rows.stream() - .filter(DailyMileageState::hasCompleteState) - .findFirst(); - } - - private void upsertDailyMileageMetric(String vin, Date statDate, double value, String unit, String strategy, - double firstTotalMileageKm, double latestTotalMileageKm) { - try { - jdbcTemplate.update(""" - INSERT INTO vehicle_stat_metric - (vin, stat_date, metric_key, metric_value, metric_unit, calculation_method, - first_total_mileage_km, latest_total_mileage_km) - VALUES (?, ?, ?, ?, ?, ?, ?, ?) - """, vin, statDate, DAILY_MILEAGE_KEY, value, unit, strategy, - firstTotalMileageKm, latestTotalMileageKm); - } catch (DuplicateKeyException ex) { - jdbcTemplate.update(""" - UPDATE vehicle_stat_metric - SET metric_value = ?, - metric_unit = ?, - calculation_method = ?, - first_total_mileage_km = ?, - latest_total_mileage_km = ?, - updated_at = CURRENT_TIMESTAMP - WHERE vin = ? AND stat_date = ? AND metric_key = ? - """, value, unit, strategy, firstTotalMileageKm, latestTotalMileageKm, - vin, statDate, DAILY_MILEAGE_KEY); - } - } - - private void deleteLegacyMileageStateMetrics(String vin, Date statDate) { - jdbcTemplate.update(""" - DELETE FROM vehicle_stat_metric - WHERE vin = ? AND stat_date = ? AND metric_key IN (?, ?) - """, vin, statDate, DAILY_MILEAGE_START_TOTAL_KEY, DAILY_MILEAGE_LATEST_TOTAL_KEY); - } - - private void ensureSchema() { - jdbcTemplate.execute(""" - CREATE TABLE IF NOT EXISTS vehicle_stat_metric ( - vin VARCHAR(64) NOT NULL, - stat_date DATE NOT NULL, - metric_key VARCHAR(64) NOT NULL, - metric_value DECIMAL(18,6) NULL, - metric_unit VARCHAR(16) NOT NULL, - calculation_method VARCHAR(64) NOT NULL, - first_total_mileage_km DECIMAL(18,6) NULL, - latest_total_mileage_km DECIMAL(18,6) NULL, - created_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, - updated_at TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP, - PRIMARY KEY (vin, stat_date, metric_key) - ) - """); - ensureColumn(FIRST_TOTAL_COLUMN); - ensureColumn(LATEST_TOTAL_COLUMN); - } - - private void ensureColumn(String columnName) { - if (columnExists(columnName)) { - return; - } - jdbcTemplate.execute("ALTER TABLE vehicle_stat_metric ADD COLUMN " + columnName + " DECIMAL(18,6) NULL"); - } - - private boolean columnExists(String columnName) { - ConnectionCallback callback = connection -> { - try (ResultSet columns = connection.getMetaData().getColumns( - connection.getCatalog(), null, "vehicle_stat_metric", columnName)) { - if (columns.next()) { - return true; - } - } - try (ResultSet columns = connection.getMetaData().getColumns( - connection.getCatalog(), null, "VEHICLE_STAT_METRIC", columnName.toUpperCase())) { - return columns.next(); - } - }; - return Boolean.TRUE.equals(jdbcTemplate.execute(callback)); - } - - private static double nullableDouble(ResultSet rs, String columnName) throws SQLException { - double value = rs.getDouble(columnName); - return rs.wasNull() ? Double.NaN : value; - } - - private static String clean(String value) { - if (value == null || value.isBlank()) { - throw new IllegalArgumentException("vin must not be blank"); - } - return value.trim(); - } - - private record DailyMileageState(double firstTotalMileageKm, double latestTotalMileageKm) { - boolean hasCompleteState() { - return Double.isFinite(firstTotalMileageKm) && Double.isFinite(latestTotalMileageKm); - } - } -} diff --git a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleDailyStatResult.java b/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleDailyStatResult.java deleted file mode 100644 index d769900f..00000000 --- a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleDailyStatResult.java +++ /dev/null @@ -1,25 +0,0 @@ -package com.lingniu.ingest.vehiclestat; - -import java.time.LocalDate; -import java.util.OptionalDouble; - -public record VehicleDailyStatResult(String vin, - LocalDate statDate, - OptionalDouble dailyMileageKm, - DailyMileageStrategy dailyMileageStrategy) { - - public VehicleDailyStatResult { - if (vin == null || vin.isBlank()) { - throw new IllegalArgumentException("vin must not be blank"); - } - if (statDate == null) { - throw new IllegalArgumentException("statDate must not be null"); - } - if (dailyMileageKm == null) { - dailyMileageKm = OptionalDouble.empty(); - } - if (dailyMileageStrategy == null) { - dailyMileageStrategy = DailyMileageStrategy.JT808_TOTAL_MILEAGE_DIFF; - } - } -} diff --git a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleStatController.java b/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleStatController.java deleted file mode 100644 index f59b18a7..00000000 --- a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleStatController.java +++ /dev/null @@ -1,52 +0,0 @@ -package com.lingniu.ingest.vehiclestat; - -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.format.annotation.DateTimeFormat; -import org.springframework.http.MediaType; -import org.springframework.http.ResponseEntity; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.PathVariable; -import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.RequestParam; -import org.springframework.web.bind.annotation.RestController; - -import java.time.LocalDate; -import java.util.LinkedHashMap; -import java.util.Map; - -@RestController -@ConditionalOnProperty(prefix = "lingniu.ingest.vehicle-stat", name = "enabled", havingValue = "true") -@ConditionalOnBean(VehicleStatRepository.class) -@RequestMapping(path = "/api/vehicle-stat", produces = MediaType.APPLICATION_JSON_VALUE) -public final class VehicleStatController { - - private final VehicleStatRepository repository; - - public VehicleStatController(VehicleStatRepository repository) { - if (repository == null) { - throw new IllegalArgumentException("repository must not be null"); - } - this.repository = repository; - } - - @GetMapping("/{vin}/daily") - public ResponseEntity> daily( - @PathVariable String vin, - @RequestParam @DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate date) { - return repository.findDailyStat(vin, date) - .map(stat -> ResponseEntity.ok(toJson(stat))) - .orElseGet(() -> ResponseEntity.notFound().build()); - } - - private static Map toJson(VehicleDailyStatResult stat) { - Map json = new LinkedHashMap<>(); - json.put("vin", stat.vin()); - json.put("statDate", stat.statDate().toString()); - json.put("dailyMileageKm", stat.dailyMileageKm().isPresent() - ? stat.dailyMileageKm().getAsDouble() - : null); - json.put("dailyMileageStrategy", stat.dailyMileageStrategy().name()); - return json; - } -} diff --git a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleStatEnvelopeIngestor.java b/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleStatEnvelopeIngestor.java deleted file mode 100644 index 35936a5c..00000000 --- a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleStatEnvelopeIngestor.java +++ /dev/null @@ -1,62 +0,0 @@ -package com.lingniu.ingest.vehiclestat; - -import com.google.protobuf.InvalidProtocolBufferException; -import com.lingniu.ingest.api.consumer.EnvelopeIngestResult; -import com.lingniu.ingest.api.consumer.EnvelopeIngestor; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import com.lingniu.ingest.vehiclestat.jt808.Jt808MileageStreamProcessor; - -public final class VehicleStatEnvelopeIngestor implements EnvelopeIngestor { - - private final Jt808MileageStreamProcessor jt808MileageProcessor; - - public VehicleStatEnvelopeIngestor(Jt808MileageStreamProcessor jt808MileageProcessor) { - if (jt808MileageProcessor == null) { - throw new IllegalArgumentException("jt808MileageProcessor must not be null"); - } - this.jt808MileageProcessor = jt808MileageProcessor; - } - - public void ingest(byte[] kafkaValue) { - VehicleEnvelope envelope = parse(kafkaValue); - if (!envelope.hasTelemetrySnapshot()) { - return; - } - if ("JT808".equalsIgnoreCase(envelope.getSource())) { - jt808MileageProcessor.process(envelope); - } - } - - @Override - public EnvelopeIngestResult tryIngest(byte[] kafkaValue) { - VehicleEnvelope envelope = null; - try { - envelope = parse(kafkaValue); - if (!envelope.hasTelemetrySnapshot()) { - return EnvelopeIngestResult.skipped( - envelope.getEventId(), envelope.getVin(), "envelope telemetry_snapshot is required"); - } - if ("JT808".equalsIgnoreCase(envelope.getSource())) { - jt808MileageProcessor.process(envelope); - return EnvelopeIngestResult.processed(envelope.getEventId(), envelope.getVin()); - } - return EnvelopeIngestResult.skipped( - envelope.getEventId(), envelope.getVin(), "vehicle-stat only accepts JT808 telemetry"); - } catch (IllegalArgumentException ex) { - return envelope == null - ? EnvelopeIngestResult.invalid(ex.getMessage()) - : EnvelopeIngestResult.skipped(envelope.getEventId(), envelope.getVin(), ex.getMessage()); - } - } - - private static VehicleEnvelope parse(byte[] kafkaValue) { - if (kafkaValue == null || kafkaValue.length == 0) { - throw new IllegalArgumentException("VehicleEnvelope bytes must not be empty"); - } - try { - return VehicleEnvelope.parseFrom(kafkaValue); - } catch (InvalidProtocolBufferException ex) { - throw new IllegalArgumentException("VehicleEnvelope bytes are invalid", ex); - } - } -} diff --git a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleStatRepository.java b/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleStatRepository.java deleted file mode 100644 index 27f8eb46..00000000 --- a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/VehicleStatRepository.java +++ /dev/null @@ -1,11 +0,0 @@ -package com.lingniu.ingest.vehiclestat; - -import java.time.LocalDate; -import java.util.Optional; - -public interface VehicleStatRepository { - - Optional recordDailyMileageSample(String vin, LocalDate statDate, double totalMileageKm); - - Optional findDailyStat(String vin, LocalDate statDate); -} diff --git a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/config/VehicleStatAutoConfiguration.java b/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/config/VehicleStatAutoConfiguration.java deleted file mode 100644 index 8e1c069a..00000000 --- a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/config/VehicleStatAutoConfiguration.java +++ /dev/null @@ -1,84 +0,0 @@ -package com.lingniu.ingest.vehiclestat.config; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.api.consumer.EnvelopeDeadLetterSink; -import com.lingniu.ingest.vehiclestat.JdbcVehicleStatMetricRepository; -import com.lingniu.ingest.vehiclestat.VehicleStatController; -import com.lingniu.ingest.vehiclestat.VehicleStatEnvelopeIngestor; -import com.lingniu.ingest.vehiclestat.VehicleStatRepository; -import com.lingniu.ingest.vehiclestat.jt808.Jt808LocationPointExtractor; -import com.lingniu.ingest.vehiclestat.jt808.Jt808MileageStreamProcessor; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration; -import org.springframework.boot.autoconfigure.jdbc.JdbcTemplateAutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; -import org.springframework.jdbc.core.JdbcTemplate; - -import java.time.ZoneId; - -@AutoConfiguration(after = { - DataSourceAutoConfiguration.class, - JdbcTemplateAutoConfiguration.class -}) -@EnableConfigurationProperties(VehicleStatProperties.class) -@ConditionalOnProperty(prefix = "lingniu.ingest.vehicle-stat", name = "enabled", havingValue = "true") -public class VehicleStatAutoConfiguration { - - @Bean - @ConditionalOnProperty(prefix = "lingniu.ingest.vehicle-stat.jt808", name = "state-store") - public Object rejectedLegacyJt808MileageStateStore() { - throw new IllegalStateException( - "vehicle-stat.jt808.state-store has been removed; JT808 daily mileage uses vehicle_stat_metric only"); - } - - @Bean - @ConditionalOnBean(JdbcTemplate.class) - @ConditionalOnMissingBean - public VehicleStatRepository jdbcVehicleStatMetricRepository(JdbcTemplate jdbcTemplate) { - return new JdbcVehicleStatMetricRepository(jdbcTemplate); - } - - @Bean - @ConditionalOnProperty(prefix = "lingniu.ingest.vehicle-stat.jt808", name = "enabled", havingValue = "true") - @ConditionalOnMissingBean - public Jt808LocationPointExtractor jt808LocationPointExtractor() { - return new Jt808LocationPointExtractor(); - } - - @Bean - @ConditionalOnBean({Jt808LocationPointExtractor.class, VehicleStatRepository.class}) - @ConditionalOnMissingBean - public Jt808MileageStreamProcessor jt808MileageStreamProcessor(Jt808LocationPointExtractor extractor, - VehicleStatRepository repository, - VehicleStatProperties props) { - return new Jt808MileageStreamProcessor(extractor, repository, ZoneId.of(props.getZoneId())); - } - - @Bean - @ConditionalOnBean(Jt808MileageStreamProcessor.class) - @ConditionalOnMissingBean - public VehicleStatEnvelopeIngestor vehicleStatEnvelopeIngestor( - Jt808MileageStreamProcessor jt808MileageProcessor) { - return new VehicleStatEnvelopeIngestor(jt808MileageProcessor); - } - - @Bean - @ConditionalOnBean(VehicleStatRepository.class) - @ConditionalOnMissingBean - public VehicleStatController vehicleStatController(VehicleStatRepository repository) { - return new VehicleStatController(repository); - } - - @Bean - @ConditionalOnBean({VehicleStatEnvelopeIngestor.class, EnvelopeDeadLetterSink.class}) - @ConditionalOnMissingBean(name = "vehicleStatEnvelopeConsumerProcessor") - public EnvelopeConsumerProcessor vehicleStatEnvelopeConsumerProcessor(VehicleStatEnvelopeIngestor ingestor, - EnvelopeDeadLetterSink deadLetterSink) { - // Bean 名必须和 Kafka 默认 binding 对齐，KafkaEnvelopeConsumerFactory 才能自动创建 worker。 - return new EnvelopeConsumerProcessor("vehicle-stat", ingestor, deadLetterSink); - } -} diff --git a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/config/VehicleStatProperties.java b/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/config/VehicleStatProperties.java deleted file mode 100644 index 24b00c4f..00000000 --- a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/config/VehicleStatProperties.java +++ /dev/null @@ -1,40 +0,0 @@ -package com.lingniu.ingest.vehiclestat.config; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -@ConfigurationProperties(prefix = "lingniu.ingest.vehicle-stat") -public class VehicleStatProperties { - - /** 统计自然日口径，默认按国内业务使用东八区。 */ - private String zoneId = "Asia/Shanghai"; - - private Jt808 jt808 = new Jt808(); - - public String getZoneId() { - return zoneId; - } - - public void setZoneId(String zoneId) { - this.zoneId = zoneId; - } - - public Jt808 getJt808() { - return jt808; - } - - public void setJt808(Jt808 jt808) { - this.jt808 = jt808; - } - - public static class Jt808 { - private boolean enabled; - - public boolean isEnabled() { - return enabled; - } - - public void setEnabled(boolean enabled) { - this.enabled = enabled; - } - } -} diff --git a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/jt808/Jt808LocationPoint.java b/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/jt808/Jt808LocationPoint.java deleted file mode 100644 index dcccdb5f..00000000 --- a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/jt808/Jt808LocationPoint.java +++ /dev/null @@ -1,27 +0,0 @@ -package com.lingniu.ingest.vehiclestat.jt808; - -import java.time.Instant; - -public record Jt808LocationPoint( - String vehicleKey, - String vin, - String phone, - Instant eventTime, - double longitude, - double latitude, - Double speedKmh, - Long statusFlag, - Double totalMileageKm -) { - - public Jt808LocationPoint { - if (vehicleKey == null || vehicleKey.isBlank()) { - throw new IllegalArgumentException("vehicleKey must not be blank"); - } - if (eventTime == null) { - throw new IllegalArgumentException("eventTime must not be null"); - } - vin = vin == null ? "" : vin; - phone = phone == null ? "" : phone; - } -} diff --git a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/jt808/Jt808LocationPointExtractor.java b/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/jt808/Jt808LocationPointExtractor.java deleted file mode 100644 index 8883391d..00000000 --- a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/jt808/Jt808LocationPointExtractor.java +++ /dev/null @@ -1,97 +0,0 @@ -package com.lingniu.ingest.vehiclestat.jt808; - -import com.lingniu.ingest.sink.kafka.proto.TelemetryField; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; - -import java.time.Instant; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Optional; - -public final class Jt808LocationPointExtractor { - - public Optional extract(VehicleEnvelope envelope) { - if (envelope == null - || !"JT808".equalsIgnoreCase(envelope.getSource()) - || !envelope.hasTelemetrySnapshot() - || !"LOCATION".equalsIgnoreCase(envelope.getTelemetrySnapshot().getEventType())) { - return Optional.empty(); - } - Map fields = fields(envelope); - Double longitude = decimal(fields.get("longitude")); - Double latitude = decimal(fields.get("latitude")); - if (longitude == null || latitude == null || !validCoordinate(longitude, latitude)) { - return Optional.empty(); - } - - String vin = clean(envelope.getVin()); - String phone = clean(envelope.getMetadataMap().getOrDefault("phone", "")); - String vehicleKey = vehicleKey(vin, phone); - if (vehicleKey.isBlank()) { - return Optional.empty(); - } - return Optional.of(new Jt808LocationPoint( - vehicleKey, - vin, - phone, - Instant.ofEpochMilli(envelope.getEventTimeMs()), - longitude, - latitude, - decimal(fields.get("speed_kmh")), - integer(fields.get("location_status_raw")), - decimal(fields.get("total_mileage_km")))); - } - - private static Map fields(VehicleEnvelope envelope) { - Map out = new LinkedHashMap<>(); - for (TelemetryField field : envelope.getTelemetrySnapshot().getFieldsList()) { - if (!field.getKey().isBlank()) { - out.put(field.getKey(), field.getValue()); - } - } - return out; - } - - private static String vehicleKey(String vin, String phone) { - if (!vin.isBlank() && !"unknown".equalsIgnoreCase(vin)) { - return vin; - } - return phone.isBlank() ? "" : "jt808:" + phone; - } - - private static boolean validCoordinate(double longitude, double latitude) { - return Double.isFinite(longitude) - && Double.isFinite(latitude) - && longitude >= -180.0 - && longitude <= 180.0 - && latitude >= -90.0 - && latitude <= 90.0; - } - - private static Double decimal(String raw) { - if (raw == null || raw.isBlank()) { - return null; - } - try { - double value = Double.parseDouble(raw); - return Double.isFinite(value) ? value : null; - } catch (NumberFormatException ignored) { - return null; - } - } - - private static Long integer(String raw) { - if (raw == null || raw.isBlank()) { - return null; - } - try { - return Long.parseLong(raw); - } catch (NumberFormatException ignored) { - return null; - } - } - - private static String clean(String value) { - return value == null ? "" : value.trim(); - } -} diff --git a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/jt808/Jt808MileageStreamProcessor.java b/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/jt808/Jt808MileageStreamProcessor.java deleted file mode 100644 index 05fba77e..00000000 --- a/modules/services/vehicle-stat-service/src/main/java/com/lingniu/ingest/vehiclestat/jt808/Jt808MileageStreamProcessor.java +++ /dev/null @@ -1,42 +0,0 @@ -package com.lingniu.ingest.vehiclestat.jt808; - -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import com.lingniu.ingest.vehiclestat.VehicleStatRepository; - -import java.time.LocalDate; -import java.time.ZoneId; - -public final class Jt808MileageStreamProcessor { - - private final Jt808LocationPointExtractor extractor; - private final VehicleStatRepository repository; - private final ZoneId zoneId; - - public Jt808MileageStreamProcessor( - Jt808LocationPointExtractor extractor, - VehicleStatRepository repository, - ZoneId zoneId) { - this.extractor = extractor; - this.repository = repository; - this.zoneId = zoneId; - } - - public void process(VehicleEnvelope envelope) { - extractor.extract(envelope).ifPresent(this::processPoint); - } - - private void processPoint(Jt808LocationPoint point) { - if (point.totalMileageKm() == null || !Double.isFinite(point.totalMileageKm())) { - return; - } - LocalDate statDate = LocalDate.ofInstant(point.eventTime(), zoneId); - repository.recordDailyMileageSample(statVehicleId(point), statDate, point.totalMileageKm()); - } - - private static String statVehicleId(Jt808LocationPoint point) { - if (!point.vin().isBlank() && !"unknown".equalsIgnoreCase(point.vin())) { - return point.vin(); - } - return point.vehicleKey(); - } -} diff --git a/modules/services/vehicle-stat-service/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/services/vehicle-stat-service/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index 953cfaa1..00000000 --- a/modules/services/vehicle-stat-service/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.vehiclestat.config.VehicleStatAutoConfiguration diff --git a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/JdbcVehicleStatMetricRepositoryTest.java b/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/JdbcVehicleStatMetricRepositoryTest.java deleted file mode 100644 index a7480a80..00000000 --- a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/JdbcVehicleStatMetricRepositoryTest.java +++ /dev/null @@ -1,203 +0,0 @@ -package com.lingniu.ingest.vehiclestat; - -import org.junit.jupiter.api.Test; -import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.jdbc.datasource.DriverManagerDataSource; - -import java.time.LocalDate; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class JdbcVehicleStatMetricRepositoryTest { - - @Test - void recordsDailyMileageIntoCommonMetricTable() { - JdbcTemplate jdbcTemplate = new JdbcTemplate(new DriverManagerDataSource( - "jdbc:h2:mem:vehicle_stat_metric;MODE=MySQL;DATABASE_TO_LOWER=TRUE;DB_CLOSE_DELAY=-1", - "sa", - "")); - JdbcVehicleStatMetricRepository repository = new JdbcVehicleStatMetricRepository(jdbcTemplate); - - repository.recordDailyMileageSample("VIN001", LocalDate.of(2026, 6, 30), 100.0); - repository.recordDailyMileageSample("VIN001", LocalDate.of(2026, 6, 30), 115.5); - - assertThat(repository.findDailyStat("VIN001", LocalDate.of(2026, 6, 30))) - .isPresent() - .get() - .extracting(result -> result.dailyMileageKm().orElseThrow()) - .isEqualTo(15.5); - assertThat(jdbcTemplate.queryForObject("SELECT COUNT(*) FROM vehicle_stat_metric", Integer.class)) - .isOne(); - assertThat(jdbcTemplate.queryForObject( - "SELECT metric_key FROM vehicle_stat_metric WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km'", - String.class)).isEqualTo("daily_mileage_km"); - assertThat(jdbcTemplate.queryForObject( - "SELECT calculation_method FROM vehicle_stat_metric WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km'", - String.class)).isEqualTo("JT808_TOTAL_MILEAGE_DIFF"); - assertThat(jdbcTemplate.queryForObject(""" - SELECT first_total_mileage_km - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km' - """, Double.class)).isEqualTo(100.0); - assertThat(jdbcTemplate.queryForObject(""" - SELECT latest_total_mileage_km - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km' - """, Double.class)).isEqualTo(115.5); - assertThat(jdbcTemplate.queryForObject(""" - SELECT COUNT(*) - FROM information_schema.tables - WHERE table_name = 'vehicle_daily_stat' - """, Integer.class)).isZero(); - } - - @Test - void recordsJt808GpsMileageDifferenceInsideMetricTable() { - JdbcTemplate jdbcTemplate = new JdbcTemplate(new DriverManagerDataSource( - "jdbc:h2:mem:vehicle_stat_metric_jt808_diff;MODE=MySQL;DATABASE_TO_LOWER=TRUE;DB_CLOSE_DELAY=-1", - "sa", - "")); - JdbcVehicleStatMetricRepository repository = new JdbcVehicleStatMetricRepository(jdbcTemplate); - - VehicleDailyStatResult first = repository.recordDailyMileageSample( - "VIN001", LocalDate.of(2026, 7, 1), 1000.5).orElseThrow(); - VehicleDailyStatResult second = repository.recordDailyMileageSample( - "VIN001", LocalDate.of(2026, 7, 1), 1008.75).orElseThrow(); - - assertThat(first.dailyMileageKm()).hasValue(0.0); - assertThat(second.dailyMileageKm()).hasValue(8.25); - assertThat(jdbcTemplate.queryForObject( - "SELECT COUNT(*) FROM vehicle_stat_metric WHERE vin = 'VIN001' AND stat_date = DATE '2026-07-01'", - Integer.class)).isOne(); - assertThat(jdbcTemplate.queryForObject(""" - SELECT metric_value - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km' - """, Double.class)).isEqualTo(8.25); - assertThat(jdbcTemplate.queryForList( - "SELECT table_name FROM information_schema.tables WHERE table_name = 'jt808_daily_mileage'", - String.class)).isEmpty(); - } - - @Test - void recalculatesJt808MileageFromSingleMetricRowAfterRestart() { - JdbcTemplate jdbcTemplate = new JdbcTemplate(new DriverManagerDataSource( - "jdbc:h2:mem:vehicle_stat_metric_jt808_out_of_order;MODE=MySQL;DATABASE_TO_LOWER=TRUE;DB_CLOSE_DELAY=-1", - "sa", - "")); - JdbcVehicleStatMetricRepository repository = new JdbcVehicleStatMetricRepository(jdbcTemplate); - - repository.recordDailyMileageSample("VIN001", LocalDate.of(2026, 7, 1), 1000.5).orElseThrow(); - JdbcVehicleStatMetricRepository restartedRepository = new JdbcVehicleStatMetricRepository(jdbcTemplate); - VehicleDailyStatResult higherSample = restartedRepository.recordDailyMileageSample( - "VIN001", LocalDate.of(2026, 7, 1), 1010.0).orElseThrow(); - - assertThat(higherSample.dailyMileageKm()).hasValue(9.5); - assertThat(jdbcTemplate.queryForObject(""" - SELECT COUNT(*) - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND stat_date = DATE '2026-07-01' - """, Integer.class)).isOne(); - assertThat(jdbcTemplate.queryForObject(""" - SELECT first_total_mileage_km - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km' - """, Double.class)).isEqualTo(1000.5); - assertThat(jdbcTemplate.queryForObject(""" - SELECT latest_total_mileage_km - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km' - """, Double.class)).isEqualTo(1010.0); - assertThat(jdbcTemplate.queryForObject(""" - SELECT metric_value - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km' - """, Double.class)).isEqualTo(9.5); - } - - @Test - void recalculatesJt808MileageWhenEarlierLowerMileageArrivesAfterRestart() { - JdbcTemplate jdbcTemplate = new JdbcTemplate(new DriverManagerDataSource( - "jdbc:h2:mem:vehicle_stat_metric_jt808_replay_lower;MODE=MySQL;DATABASE_TO_LOWER=TRUE;DB_CLOSE_DELAY=-1", - "sa", - "")); - JdbcVehicleStatMetricRepository repository = new JdbcVehicleStatMetricRepository(jdbcTemplate); - - repository.recordDailyMileageSample("VIN001", LocalDate.of(2026, 7, 1), 1010.0).orElseThrow(); - JdbcVehicleStatMetricRepository restartedRepository = new JdbcVehicleStatMetricRepository(jdbcTemplate); - VehicleDailyStatResult replayedEarlierSample = restartedRepository.recordDailyMileageSample( - "VIN001", LocalDate.of(2026, 7, 1), 1000.5).orElseThrow(); - - assertThat(replayedEarlierSample.dailyMileageKm()).hasValue(9.5); - assertThat(jdbcTemplate.queryForObject(""" - SELECT first_total_mileage_km - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km' - """, Double.class)).isEqualTo(1000.5); - assertThat(jdbcTemplate.queryForObject(""" - SELECT latest_total_mileage_km - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km' - """, Double.class)).isEqualTo(1010.0); - assertThat(jdbcTemplate.queryForObject(""" - SELECT COUNT(*) - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND stat_date = DATE '2026-07-01' - """, Integer.class)).isOne(); - } - - @Test - void ignoresLegacySplitMileageStateMetricsWhenRecordingNewGpsTotalMileage() { - JdbcTemplate jdbcTemplate = new JdbcTemplate(new DriverManagerDataSource( - "jdbc:h2:mem:vehicle_stat_metric_ignore_legacy_state;MODE=MySQL;DATABASE_TO_LOWER=TRUE;DB_CLOSE_DELAY=-1", - "sa", - "")); - JdbcVehicleStatMetricRepository repository = new JdbcVehicleStatMetricRepository(jdbcTemplate); - jdbcTemplate.update(""" - INSERT INTO vehicle_stat_metric - (vin, stat_date, metric_key, metric_value, metric_unit, calculation_method) - VALUES - ('VIN001', DATE '2026-07-01', 'daily_mileage_start_total_km', 100.0, 'km', 'JT808_TOTAL_MILEAGE_DIFF'), - ('VIN001', DATE '2026-07-01', 'daily_mileage_latest_total_km', 105.0, 'km', 'JT808_TOTAL_MILEAGE_DIFF') - """); - - VehicleDailyStatResult result = repository.recordDailyMileageSample( - "VIN001", LocalDate.of(2026, 7, 1), 110.0).orElseThrow(); - - assertThat(result.dailyMileageKm()).hasValue(0.0); - assertThat(jdbcTemplate.queryForObject(""" - SELECT first_total_mileage_km - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km' - """, Double.class)).isEqualTo(110.0); - assertThat(jdbcTemplate.queryForObject(""" - SELECT latest_total_mileage_km - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND metric_key = 'daily_mileage_km' - """, Double.class)).isEqualTo(110.0); - assertThat(jdbcTemplate.queryForObject(""" - SELECT COUNT(*) - FROM vehicle_stat_metric - WHERE vin = 'VIN001' AND stat_date = DATE '2026-07-01' - """, Integer.class)).isOne(); - } - - @Test - void rejectsLegacyPreviousDayCalculationMethod() { - JdbcTemplate jdbcTemplate = new JdbcTemplate(new DriverManagerDataSource( - "jdbc:h2:mem:vehicle_stat_metric_legacy;MODE=MySQL;DATABASE_TO_LOWER=TRUE;DB_CLOSE_DELAY=-1", - "sa", - "")); - JdbcVehicleStatMetricRepository repository = new JdbcVehicleStatMetricRepository(jdbcTemplate); - jdbcTemplate.update(""" - INSERT INTO vehicle_stat_metric - (vin, stat_date, metric_key, metric_value, metric_unit, calculation_method) - VALUES ('VIN001', DATE '2026-06-30', 'daily_mileage_km', 8.5, 'km', 'CURRENT_LAST_MINUS_PREVIOUS_LAST') - """); - - assertThatThrownBy(() -> repository.findDailyStat("VIN001", LocalDate.of(2026, 6, 30))) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("CURRENT_LAST_MINUS_PREVIOUS_LAST"); - } -} diff --git a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/VehicleStatControllerTest.java b/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/VehicleStatControllerTest.java deleted file mode 100644 index 0cb9db8b..00000000 --- a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/VehicleStatControllerTest.java +++ /dev/null @@ -1,55 +0,0 @@ -package com.lingniu.ingest.vehiclestat; - -import org.junit.jupiter.api.Test; -import org.springframework.test.web.servlet.MockMvc; - -import java.time.LocalDate; -import java.util.Optional; -import java.util.OptionalDouble; - -import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; -import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.jsonPath; -import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; -import static org.springframework.test.web.servlet.setup.MockMvcBuilders.standaloneSetup; - -class VehicleStatControllerTest { - - @Test - void returnsDailyStatJson() throws Exception { - InMemoryVehicleStatRepository repository = new InMemoryVehicleStatRepository(); - repository.recordDailyMileageSample("VIN001", LocalDate.parse("2026-06-22"), 100.0); - repository.recordDailyMileageSample("VIN001", LocalDate.parse("2026-06-22"), 135.5); - MockMvc mvc = standaloneSetup(new VehicleStatController(repository)).build(); - - mvc.perform(get("/api/vehicle-stat/VIN001/daily").param("date", "2026-06-22")) - .andExpect(status().isOk()) - .andExpect(jsonPath("$.vin").value("VIN001")) - .andExpect(jsonPath("$.statDate").value("2026-06-22")) - .andExpect(jsonPath("$.dailyMileageKm").value(35.5)) - .andExpect(jsonPath("$.dailyMileageStrategy").value("JT808_TOTAL_MILEAGE_DIFF")); - } - - private static final class InMemoryVehicleStatRepository implements VehicleStatRepository { - private final java.util.Map results = new java.util.HashMap<>(); - private final java.util.Map firstTotals = new java.util.HashMap<>(); - - @Override - public Optional recordDailyMileageSample(String vin, LocalDate statDate, - double totalMileageKm) { - String key = vin + ":" + statDate; - double firstTotal = firstTotals.computeIfAbsent(key, ignored -> totalMileageKm); - VehicleDailyStatResult result = new VehicleDailyStatResult( - vin, - statDate, - OptionalDouble.of(totalMileageKm - firstTotal), - DailyMileageStrategy.JT808_TOTAL_MILEAGE_DIFF); - results.put(key, result); - return Optional.of(result); - } - - @Override - public Optional findDailyStat(String vin, LocalDate statDate) { - return Optional.ofNullable(results.get(vin + ":" + statDate)); - } - } -} diff --git a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/VehicleStatEnvelopeIngestorTest.java b/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/VehicleStatEnvelopeIngestorTest.java deleted file mode 100644 index e13d6051..00000000 --- a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/VehicleStatEnvelopeIngestorTest.java +++ /dev/null @@ -1,136 +0,0 @@ -package com.lingniu.ingest.vehiclestat; - -import com.lingniu.ingest.api.consumer.EnvelopeIngestResult; -import com.lingniu.ingest.api.consumer.EnvelopeIngestor; -import com.lingniu.ingest.sink.kafka.proto.RawArchiveRef; -import com.lingniu.ingest.sink.kafka.proto.TelemetryField; -import com.lingniu.ingest.sink.kafka.proto.TelemetrySnapshot; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import com.lingniu.ingest.vehiclestat.jt808.Jt808MileageStreamProcessor; -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.times; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.verifyNoInteractions; - -class VehicleStatEnvelopeIngestorTest { - - @Test - void ignoresNonJt808Telemetry() { - Jt808MileageStreamProcessor jt808Processor = mock(Jt808MileageStreamProcessor.class); - VehicleEnvelope envelope = envelope("GB32960"); - VehicleStatEnvelopeIngestor ingestor = new VehicleStatEnvelopeIngestor(jt808Processor); - - ingestor.ingest(envelope.toByteArray()); - var result = ingestor.tryIngest(envelope.toByteArray()); - - verifyNoInteractions(jt808Processor); - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.SKIPPED); - assertThat(result.message()).contains("only accepts JT808"); - } - - @Test - void routesJt808TelemetryOnlyToJt808MileageProcessorWhenAvailable() { - Jt808MileageStreamProcessor jt808Processor = mock(Jt808MileageStreamProcessor.class); - VehicleEnvelope envelope = envelope("JT808"); - VehicleStatEnvelopeIngestor ingestor = new VehicleStatEnvelopeIngestor(jt808Processor); - - assertThat(ingestor).isInstanceOf(EnvelopeIngestor.class); - ingestor.ingest(envelope.toByteArray()); - var result = ingestor.tryIngest(envelope.toByteArray()); - - verify(jt808Processor, times(2)).process(envelope); - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.PROCESSED); - } - - @Test - void rejectsInvalidEnvelopeBytes() { - VehicleStatEnvelopeIngestor ingestor = new VehicleStatEnvelopeIngestor(mock(Jt808MileageStreamProcessor.class)); - - assertThatThrownBy(() -> ingestor.ingest(new byte[]{0x01, 0x02})) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("VehicleEnvelope"); - } - - @Test - void tryIngestInvalidEnvelopeBytesReturnsInvalidWithoutProcessing() { - Jt808MileageStreamProcessor jt808Processor = mock(Jt808MileageStreamProcessor.class); - VehicleStatEnvelopeIngestor ingestor = new VehicleStatEnvelopeIngestor(jt808Processor); - - var result = ingestor.tryIngest(new byte[]{0x01, 0x02}); - - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.INVALID_ENVELOPE); - assertThat(result.message()).contains("VehicleEnvelope"); - verifyNoInteractions(jt808Processor); - } - - @Test - void tryIngestEnvelopeWithoutTelemetrySnapshotReturnsSkipped() { - Jt808MileageStreamProcessor jt808Processor = mock(Jt808MileageStreamProcessor.class); - VehicleStatEnvelopeIngestor ingestor = new VehicleStatEnvelopeIngestor(jt808Processor); - VehicleEnvelope envelope = VehicleEnvelope.newBuilder() - .setEventId("event-login-1") - .setVin("VIN001") - .setSource("GB32960") - .setEventTimeMs(1_782_112_400_000L) - .setIngestTimeMs(1_782_112_401_000L) - .build(); - - var result = ingestor.tryIngest(envelope.toByteArray()); - - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.SKIPPED); - assertThat(result.eventId()).isEqualTo("event-login-1"); - assertThat(result.message()).contains("telemetry_snapshot"); - verifyNoInteractions(jt808Processor); - } - - @Test - void rawArchiveEnvelopeIsIgnoredWithoutProcessing() { - Jt808MileageStreamProcessor jt808Processor = mock(Jt808MileageStreamProcessor.class); - VehicleStatEnvelopeIngestor ingestor = new VehicleStatEnvelopeIngestor(jt808Processor); - VehicleEnvelope envelope = rawArchiveEnvelope(); - - ingestor.ingest(envelope.toByteArray()); - var result = ingestor.tryIngest(envelope.toByteArray()); - - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.SKIPPED); - assertThat(result.eventId()).isEqualTo("raw-event-1"); - assertThat(result.vin()).isEqualTo("VIN001"); - assertThat(result.message()).contains("telemetry_snapshot"); - verifyNoInteractions(jt808Processor); - } - - private static VehicleEnvelope envelope(String source) { - return VehicleEnvelope.newBuilder() - .setEventId("event-1") - .setVin("VIN001") - .setSource(source) - .setEventTimeMs(1_782_112_400_000L) - .setIngestTimeMs(1_782_112_401_000L) - .setTelemetrySnapshot(TelemetrySnapshot.newBuilder() - .setEventType("REALTIME") - .addFields(TelemetryField.newBuilder() - .setKey("total_mileage_km") - .setValueType("DOUBLE") - .setValue("123.45") - .setQuality("GOOD"))) - .build(); - } - - private static VehicleEnvelope rawArchiveEnvelope() { - return VehicleEnvelope.newBuilder() - .setEventId("raw-event-1") - .setVin("VIN001") - .setSource("GB32960") - .setEventTimeMs(1_782_112_400_000L) - .setIngestTimeMs(1_782_112_401_000L) - .setRawArchive(RawArchiveRef.newBuilder() - .setUri("archive://2026/06/23/GB32960/VIN001/raw-event-1.bin") - .setSizeBytes(128) - .build()) - .build(); - } -} diff --git a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/VehicleStatRepositoryContractTest.java b/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/VehicleStatRepositoryContractTest.java deleted file mode 100644 index b1071c28..00000000 --- a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/VehicleStatRepositoryContractTest.java +++ /dev/null @@ -1,76 +0,0 @@ -package com.lingniu.ingest.vehiclestat; - -import org.junit.jupiter.api.Test; - -import java.io.IOException; -import java.lang.reflect.Method; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.Arrays; - -import static org.assertj.core.api.Assertions.assertThat; - -class VehicleStatRepositoryContractTest { - - @Test - void exposesOnlyMileageSampleAndQueryOperations() { - assertThat(Arrays.stream(VehicleStatRepository.class.getDeclaredMethods()) - .map(Method::getName)) - .containsExactlyInAnyOrder("recordDailyMileageSample", "findDailyStat"); - } - - @Test - void jt808MileagePlanDocumentsMetricTableDiffOnly() throws IOException { - String plan = Files.readString(repositoryRoot() - .resolve("docs/superpowers/plans/2026-06-30-kafka-streaming-mileage.md")); - - assertThat(plan) - .contains("Only supported message backbone: Kafka.") - .contains("Runtime state: none outside `vehicle_stat_metric`") - .contains("daily_mileage_km = max_total_mileage_km - min_total_mileage_km") - .contains("Do not create or write a protocol-specific JT808 daily-mileage table.") - .doesNotContain("Redis `Jt808MileageStateStore`") - .doesNotContain("VEHICLE_STAT_REPOSITORY_TYPE") - .doesNotContain("REDIS_HOST") - .doesNotContain("GPS segment distance") - .doesNotContain("speed integral"); - } - - @Test - void targetArchitectureDocumentsMetricTableAsOnlyJt808MileageRuntimeState() throws IOException { - String architecture = Files.readString(repositoryRoot().resolve("docs/target-architecture.md")); - - assertThat(architecture) - .contains("Only supported message backbone: Kafka.") - .contains("Runtime state: none outside `vehicle_stat_metric`") - .contains("Restart recovery reads the same metric row") - .contains("JT808 daily mileage is stored only in `vehicle_stat_metric`") - .doesNotContain("state-store") - .doesNotContain("Redis mileage state") - .doesNotContain("production memory"); - } - - @Test - void jt808MileageRunbookDocumentsMetricTableAsOnlyRuntimeState() throws IOException { - String runbook = Files.readString(repositoryRoot() - .resolve("docs/operations/jt808-daily-mileage-runbook.md")); - - assertThat(runbook) - .contains("Only supported message backbone: Kafka.") - .contains("Runtime state: none outside `vehicle_stat_metric`") - .contains("Restart recovery reads the same `daily_mileage_km` metric row") - .doesNotContain("Redis mileage state") - .doesNotContain("state-store"); - } - - private static Path repositoryRoot() { - Path path = Path.of("").toAbsolutePath(); - while (path != null && !Files.exists(path.resolve("docs/superpowers/plans"))) { - path = path.getParent(); - } - if (path == null) { - throw new IllegalStateException("repository root not found"); - } - return path; - } -} diff --git a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/config/VehicleStatAutoConfigurationTest.java b/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/config/VehicleStatAutoConfigurationTest.java deleted file mode 100644 index 059dbf03..00000000 --- a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/config/VehicleStatAutoConfigurationTest.java +++ /dev/null @@ -1,113 +0,0 @@ -package com.lingniu.ingest.vehiclestat.config; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.api.consumer.EnvelopeDeadLetterSink; -import com.lingniu.ingest.vehiclestat.JdbcVehicleStatMetricRepository; -import com.lingniu.ingest.vehiclestat.VehicleStatController; -import com.lingniu.ingest.vehiclestat.VehicleStatEnvelopeIngestor; -import com.lingniu.ingest.vehiclestat.VehicleStatRepository; -import com.lingniu.ingest.vehiclestat.jt808.Jt808MileageStreamProcessor; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.autoconfigure.jdbc.JdbcTemplateAutoConfiguration; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.jdbc.core.JdbcTemplate; -import org.springframework.jdbc.datasource.DriverManagerDataSource; - -import javax.sql.DataSource; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.mock; - -class VehicleStatAutoConfigurationTest { - - private final ApplicationContextRunner contextRunner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(VehicleStatAutoConfiguration.class)) - .withBean(EnvelopeDeadLetterSink.class, () -> record -> {}); - - @Test - void createsOnlyMetricRepositoryAndReadControllerWhenJt808MileageDisabled() { - contextRunner - .withBean(JdbcTemplate.class, () -> mock(JdbcTemplate.class)) - .withPropertyValues("lingniu.ingest.vehicle-stat.enabled=true") - .run(context -> { - assertThat(context).hasSingleBean(VehicleStatRepository.class); - assertThat(context).hasSingleBean(JdbcVehicleStatMetricRepository.class); - assertThat(context).hasSingleBean(VehicleStatController.class); - assertThat(context).doesNotHaveBean(Jt808MileageStreamProcessor.class); - assertThat(context).doesNotHaveBean(VehicleStatEnvelopeIngestor.class); - assertThat(context).doesNotHaveBean(EnvelopeConsumerProcessor.class); - }); - } - - @Test - void backsOffWhenDisabled() { - contextRunner.run(context -> { - assertThat(context).doesNotHaveBean(VehicleStatRepository.class); - assertThat(context).doesNotHaveBean(VehicleStatEnvelopeIngestor.class); - }); - } - - @Test - void doesNotCreateRepositoryWithoutJdbcMetricStore() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(VehicleStatAutoConfiguration.class)) - .withPropertyValues("lingniu.ingest.vehicle-stat.enabled=true") - .run(context -> { - assertThat(context).doesNotHaveBean(VehicleStatRepository.class); - assertThat(context).doesNotHaveBean(VehicleStatController.class); - }); - } - - @Test - void createsJt808MileageBeansWhenEnabledWithMetricRepositoryOnly() { - contextRunner - .withBean(JdbcTemplate.class, () -> mock(JdbcTemplate.class)) - .withPropertyValues( - "lingniu.ingest.vehicle-stat.enabled=true", - "lingniu.ingest.vehicle-stat.jt808.enabled=true") - .run(context -> { - assertThat(context).hasSingleBean(Jt808MileageStreamProcessor.class); - assertThat(context).hasSingleBean(VehicleStatEnvelopeIngestor.class); - assertThat(context).hasSingleBean(EnvelopeConsumerProcessor.class); - }); - } - - @Test - void rejectsLegacyStateStorePropertyInsteadOfCreatingHiddenRuntimeMode() { - contextRunner - .withBean(JdbcTemplate.class, () -> mock(JdbcTemplate.class)) - .withPropertyValues( - "lingniu.ingest.vehicle-stat.enabled=true", - "lingniu.ingest.vehicle-stat.jt808.enabled=true", - "lingniu.ingest.vehicle-stat.jt808.state-store=memory") - .run(context -> assertThat(context.getStartupFailure()) - .hasRootCauseInstanceOf(IllegalStateException.class) - .hasMessageContaining("vehicle-stat.jt808.state-store has been removed") - .hasMessageContaining("vehicle_stat_metric")); - } - - @Test - void createsJt808MileageBeansAfterJdbcAutoConfiguration() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of( - JdbcTemplateAutoConfiguration.class, - VehicleStatAutoConfiguration.class)) - .withBean(EnvelopeDeadLetterSink.class, () -> record -> {}) - .withBean(DataSource.class, () -> new DriverManagerDataSource( - "jdbc:h2:mem:vehicle_stat_auto;MODE=MySQL;DATABASE_TO_LOWER=TRUE;DB_CLOSE_DELAY=-1", - "sa", - "")) - .withPropertyValues( - "lingniu.ingest.vehicle-stat.enabled=true", - "lingniu.ingest.vehicle-stat.jt808.enabled=true") - .run(context -> { - assertThat(context).hasSingleBean(JdbcTemplate.class); - assertThat(context).hasSingleBean(JdbcVehicleStatMetricRepository.class); - assertThat(context).hasSingleBean(Jt808MileageStreamProcessor.class); - assertThat(context).hasSingleBean(VehicleStatEnvelopeIngestor.class); - assertThat(context).hasSingleBean(EnvelopeConsumerProcessor.class); - }); - } - -} diff --git a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/jt808/Jt808LocationPointExtractorTest.java b/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/jt808/Jt808LocationPointExtractorTest.java deleted file mode 100644 index 579c7135..00000000 --- a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/jt808/Jt808LocationPointExtractorTest.java +++ /dev/null @@ -1,99 +0,0 @@ -package com.lingniu.ingest.vehiclestat.jt808; - -import com.lingniu.ingest.sink.kafka.proto.TelemetryField; -import com.lingniu.ingest.sink.kafka.proto.TelemetrySnapshot; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import org.junit.jupiter.api.Test; - -import java.time.Instant; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt808LocationPointExtractorTest { - - private final Jt808LocationPointExtractor extractor = new Jt808LocationPointExtractor(); - - @Test - void extractsJt808LocationPointWithVin() { - var point = extractor.extract(envelope("JT808", "VIN001", "13900000001", - field("longitude", "118.901034"), - field("latitude", "31.946325"), - field("speed_kmh", "52.3"), - field("location_status_raw", "3"), - field("total_mileage_km", "1234.5"))).orElseThrow(); - - assertThat(point.vehicleKey()).isEqualTo("VIN001"); - assertThat(point.vin()).isEqualTo("VIN001"); - assertThat(point.phone()).isEqualTo("13900000001"); - assertThat(point.eventTime()).isEqualTo(Instant.ofEpochMilli(1_782_112_400_000L)); - assertThat(point.longitude()).isEqualTo(118.901034); - assertThat(point.latitude()).isEqualTo(31.946325); - assertThat(point.speedKmh()).isEqualTo(52.3); - assertThat(point.statusFlag()).isEqualTo(3L); - assertThat(point.totalMileageKm()).isEqualTo(1234.5); - } - - @Test - void usesPhoneFallbackWhenVinIsUnknown() { - var point = extractor.extract(envelope("JT808", "unknown", "40692934289", - field("longitude", "118.901034"), - field("latitude", "31.946325"))).orElseThrow(); - - assertThat(point.vehicleKey()).isEqualTo("jt808:40692934289"); - assertThat(point.vin()).isEqualTo("unknown"); - assertThat(point.phone()).isEqualTo("40692934289"); - } - - @Test - void ignoresNonJt808Envelope() { - assertThat(extractor.extract(envelope("GB32960", "VIN001", "", - field("longitude", "118.901034"), - field("latitude", "31.946325")))).isEmpty(); - } - - @Test - void ignoresEnvelopeWithoutCoordinates() { - assertThat(extractor.extract(envelope("JT808", "VIN001", "13900000001", - field("longitude", "118.901034")))).isEmpty(); - } - - @Test - void ignoresJt808NonLocationSnapshot() { - VehicleEnvelope envelope = envelope("JT808", "VIN001", "13900000001", - field("longitude", "118.901034"), - field("latitude", "31.946325")) - .toBuilder() - .setTelemetrySnapshot(TelemetrySnapshot.newBuilder() - .setEventType("REGISTER") - .addFields(field("longitude", "118.901034")) - .addFields(field("latitude", "31.946325"))) - .build(); - - assertThat(extractor.extract(envelope)).isEmpty(); - } - - private static VehicleEnvelope envelope(String source, String vin, String phone, TelemetryField... fields) { - TelemetrySnapshot.Builder snapshot = TelemetrySnapshot.newBuilder().setEventType("LOCATION"); - for (TelemetryField field : fields) { - snapshot.addFields(field); - } - return VehicleEnvelope.newBuilder() - .setEventId("event-1") - .setVin(vin) - .setSource(source) - .putMetadata("phone", phone) - .setEventTimeMs(1_782_112_400_000L) - .setIngestTimeMs(1_782_112_401_000L) - .setTelemetrySnapshot(snapshot) - .build(); - } - - private static TelemetryField field(String key, String value) { - return TelemetryField.newBuilder() - .setKey(key) - .setValueType("DOUBLE") - .setValue(value) - .setQuality("GOOD") - .build(); - } -} diff --git a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/jt808/Jt808MileageStreamProcessorTest.java b/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/jt808/Jt808MileageStreamProcessorTest.java deleted file mode 100644 index 6ef612e5..00000000 --- a/modules/services/vehicle-stat-service/src/test/java/com/lingniu/ingest/vehiclestat/jt808/Jt808MileageStreamProcessorTest.java +++ /dev/null @@ -1,135 +0,0 @@ -package com.lingniu.ingest.vehiclestat.jt808; - -import com.lingniu.ingest.sink.kafka.proto.TelemetryField; -import com.lingniu.ingest.sink.kafka.proto.TelemetrySnapshot; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import com.lingniu.ingest.vehiclestat.DailyMileageStrategy; -import com.lingniu.ingest.vehiclestat.VehicleDailyStatResult; -import com.lingniu.ingest.vehiclestat.VehicleStatRepository; -import org.junit.jupiter.api.Test; - -import java.lang.reflect.Modifier; -import java.time.Instant; -import java.time.LocalDate; -import java.time.ZoneId; -import java.util.ArrayList; -import java.util.List; -import java.util.Optional; -import java.util.OptionalDouble; - -import static org.assertj.core.api.Assertions.assertThat; - -class Jt808MileageStreamProcessorTest { - - @Test - void recordsDailyMileageSamplesToVehicleStatRepository() { - CapturingRepository repository = new CapturingRepository(); - Jt808MileageStreamProcessor processor = processor(repository); - - processor.process(envelope("2026-06-30T00:00:00Z", 120.0, 30.0, 100.0)); - processor.process(envelope("2026-06-30T00:01:00Z", 120.01, 30.0, 101.0)); - - assertThat(repository.samples).containsExactly(100.0, 101.0); - assertThat(repository.results).hasSize(2); - VehicleDailyStatResult latest = repository.results.getLast(); - assertThat(latest.vin()).isEqualTo("VIN001"); - assertThat(latest.statDate()).isEqualTo(LocalDate.of(2026, 6, 30)); - assertThat(latest.dailyMileageKm()).hasValue(1.0); - assertThat(latest.dailyMileageStrategy()).isEqualTo(DailyMileageStrategy.JT808_TOTAL_MILEAGE_DIFF); - } - - @Test - void usesSimpleTotalMileageDifferenceInsteadOfAccumulatedPlausibilityDeltas() { - CapturingRepository repository = new CapturingRepository(); - Jt808MileageStreamProcessor processor = processor(repository); - - processor.process(envelope("2026-06-30T00:00:00Z", 120.0, 30.0, 100.0)); - processor.process(envelope("2026-06-30T00:01:00Z", 120.0001, 30.0, 200.0)); - - VehicleDailyStatResult latest = repository.results.getLast(); - assertThat(latest.dailyMileageKm()).hasValue(100.0); - } - - @Test - void skipsNonJt808Envelope() { - CapturingRepository repository = new CapturingRepository(); - Jt808MileageStreamProcessor processor = processor(repository); - VehicleEnvelope envelope = envelope("2026-06-30T00:00:00Z", 120.0, 30.0, 100.0) - .toBuilder() - .setSource("GB32960") - .build(); - - processor.process(envelope); - - assertThat(repository.results).isEmpty(); - } - - @Test - void processDoesNotSerializeAllVehicleMileageUpdatesBehindOneMonitor() throws NoSuchMethodException { - int modifiers = Jt808MileageStreamProcessor.class - .getDeclaredMethod("process", VehicleEnvelope.class) - .getModifiers(); - - assertThat(Modifier.isSynchronized(modifiers)).isFalse(); - } - - private static Jt808MileageStreamProcessor processor(VehicleStatRepository repository) { - return new Jt808MileageStreamProcessor( - new Jt808LocationPointExtractor(), - repository, - ZoneId.of("Asia/Shanghai")); - } - - private static VehicleEnvelope envelope(String eventTime, double longitude, double latitude, double totalMileageKm) { - long eventTimeMs = Instant.parse(eventTime).toEpochMilli(); - return VehicleEnvelope.newBuilder() - .setEventId("event-" + eventTimeMs) - .setVin("VIN001") - .setSource("JT808") - .setEventTimeMs(eventTimeMs) - .putMetadata("phone", "13900000001") - .setTelemetrySnapshot(TelemetrySnapshot.newBuilder() - .setEventType("LOCATION") - .addFields(field("longitude", longitude)) - .addFields(field("latitude", latitude)) - .addFields(field("speed_kmh", 36.0)) - .addFields(field("total_mileage_km", totalMileageKm))) - .build(); - } - - private static TelemetryField field(String key, double value) { - return TelemetryField.newBuilder() - .setKey(key) - .setValueType("DOUBLE") - .setValue(Double.toString(value)) - .setQuality("GOOD") - .build(); - } - - private static final class CapturingRepository implements VehicleStatRepository { - private final List results = new ArrayList<>(); - private final List samples = new ArrayList<>(); - private double firstSample = Double.NaN; - - @Override - public Optional recordDailyMileageSample(String vin, LocalDate statDate, - double totalMileageKm) { - samples.add(totalMileageKm); - if (!Double.isFinite(firstSample)) { - firstSample = totalMileageKm; - } - VehicleDailyStatResult result = new VehicleDailyStatResult( - vin, - statDate, - OptionalDouble.of(totalMileageKm - firstSample), - DailyMileageStrategy.JT808_TOTAL_MILEAGE_DIFF); - results.add(result); - return Optional.of(result); - } - - @Override - public Optional findDailyStat(String vin, LocalDate statDate) { - return Optional.empty(); - } - } -} diff --git a/modules/services/vehicle-state-service/pom.xml b/modules/services/vehicle-state-service/pom.xml deleted file mode 100644 index 360dcd44..00000000 --- a/modules/services/vehicle-state-service/pom.xml +++ /dev/null @@ -1,57 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - vehicle-state-service - vehicle-state-service - Kafka full-field event consumer + Redis hot vehicle state query API. - - - - com.lingniu.ingest - sink-kafka - - - org.springframework.boot - spring-boot-starter-data-redis - - - org.springframework.boot - spring-boot-starter-web - - - org.springframework.boot - spring-boot-autoconfigure - - - org.springframework.boot - spring-boot-configuration-processor - true - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.mockito - mockito-core - test - - - org.springframework.boot - spring-boot-starter-test - test - - - diff --git a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/RedisVehicleStateRepository.java b/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/RedisVehicleStateRepository.java deleted file mode 100644 index e22888af..00000000 --- a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/RedisVehicleStateRepository.java +++ /dev/null @@ -1,73 +0,0 @@ -package com.lingniu.ingest.vehiclestate; - -import org.springframework.data.redis.core.StringRedisTemplate; - -import java.util.Optional; - -public final class RedisVehicleStateRepository implements VehicleStateRepository { - - private final StringRedisTemplate redis; - - public RedisVehicleStateRepository(StringRedisTemplate redis) { - if (redis == null) { - throw new IllegalArgumentException("redis must not be null"); - } - this.redis = redis; - } - - @Override - public void putState(String vin, String json) { - // 每辆车一个 Redis key，读最新状态时不需要扫描或 join 历史表。 - put(key("vehicle:state:", vin), json); - } - - @Override - public void putLocation(String vin, String json) { - put(key("vehicle:location:", vin), json); - } - - @Override - public void putSafety(String vin, String json) { - put(key("vehicle:safety:", vin), json); - } - - @Override - public void putLastEvent(String vin, String json) { - put(key("vehicle:event:last:", vin), json); - } - - @Override - public Optional getState(String vin) { - return get(key("vehicle:state:", vin)); - } - - @Override - public Optional getLocation(String vin) { - return get(key("vehicle:location:", vin)); - } - - @Override - public Optional getSafety(String vin) { - return get(key("vehicle:safety:", vin)); - } - - @Override - public Optional getLastEvent(String vin) { - return get(key("vehicle:event:last:", vin)); - } - - private void put(String key, String json) { - redis.opsForValue().set(key, json == null ? "{}" : json); - } - - private Optional get(String key) { - return Optional.ofNullable(redis.opsForValue().get(key)); - } - - private static String key(String prefix, String vin) { - if (vin == null || vin.isBlank()) { - throw new IllegalArgumentException("vin must not be blank"); - } - return prefix + vin; - } -} diff --git a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateController.java b/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateController.java deleted file mode 100644 index 507114c3..00000000 --- a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateController.java +++ /dev/null @@ -1,56 +0,0 @@ -package com.lingniu.ingest.vehiclestate; - -import org.springframework.http.MediaType; -import org.springframework.http.ResponseEntity; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.web.bind.annotation.GetMapping; -import org.springframework.web.bind.annotation.PathVariable; -import org.springframework.web.bind.annotation.RequestMapping; -import org.springframework.web.bind.annotation.RestController; - -import java.util.Optional; - -@RestController -@ConditionalOnProperty(prefix = "lingniu.ingest.vehicle-state", name = "enabled", havingValue = "true") -@ConditionalOnBean(VehicleStateRepository.class) -@RequestMapping(path = "/api/vehicle-state", produces = MediaType.APPLICATION_JSON_VALUE) -public final class VehicleStateController { - - private final VehicleStateRepository repository; - - public VehicleStateController(VehicleStateRepository repository) { - if (repository == null) { - throw new IllegalArgumentException("repository must not be null"); - } - this.repository = repository; - } - - @GetMapping("/{vin}") - public ResponseEntity state(@PathVariable String vin) { - // 查询的是 Redis 最新状态快照，不是历史明细库。 - return json(repository.getState(vin)); - } - - @GetMapping("/{vin}/location") - public ResponseEntity location(@PathVariable String vin) { - return json(repository.getLocation(vin)); - } - - @GetMapping("/{vin}/safety") - public ResponseEntity safety(@PathVariable String vin) { - return json(repository.getSafety(vin)); - } - - @GetMapping("/{vin}/last-event") - public ResponseEntity lastEvent(@PathVariable String vin) { - return json(repository.getLastEvent(vin)); - } - - private static ResponseEntity json(Optional value) { - return value.map(json -> ResponseEntity.ok() - .contentType(MediaType.APPLICATION_JSON) - .body(json)) - .orElseGet(() -> ResponseEntity.notFound().build()); - } -} diff --git a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateEnvelopeIngestor.java b/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateEnvelopeIngestor.java deleted file mode 100644 index 0411a863..00000000 --- a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateEnvelopeIngestor.java +++ /dev/null @@ -1,56 +0,0 @@ -package com.lingniu.ingest.vehiclestate; - -import com.google.protobuf.InvalidProtocolBufferException; -import com.lingniu.ingest.api.consumer.EnvelopeIngestResult; -import com.lingniu.ingest.api.consumer.EnvelopeIngestor; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; - -public final class VehicleStateEnvelopeIngestor implements EnvelopeIngestor { - - private final VehicleStateUpdater updater; - - public VehicleStateEnvelopeIngestor(VehicleStateUpdater updater) { - if (updater == null) { - throw new IllegalArgumentException("updater must not be null"); - } - this.updater = updater; - } - - public void ingest(byte[] kafkaValue) { - VehicleEnvelope envelope = parse(kafkaValue); - if (!envelope.hasTelemetrySnapshot()) { - return; - } - updater.update(envelope); - } - - @Override - public EnvelopeIngestResult tryIngest(byte[] kafkaValue) { - VehicleEnvelope envelope = null; - try { - envelope = parse(kafkaValue); - if (!envelope.hasTelemetrySnapshot()) { - return EnvelopeIngestResult.skipped( - envelope.getEventId(), envelope.getVin(), "envelope telemetry_snapshot is required"); - } - // Kafka 消费路径只接受标准 VehicleEnvelope；坏消息返回明确结果给处理器写 DLQ。 - updater.update(envelope); - return EnvelopeIngestResult.processed(envelope.getEventId(), envelope.getVin()); - } catch (IllegalArgumentException ex) { - return envelope == null - ? EnvelopeIngestResult.invalid(ex.getMessage()) - : EnvelopeIngestResult.skipped(envelope.getEventId(), envelope.getVin(), ex.getMessage()); - } - } - - private static VehicleEnvelope parse(byte[] kafkaValue) { - if (kafkaValue == null || kafkaValue.length == 0) { - throw new IllegalArgumentException("VehicleEnvelope bytes must not be empty"); - } - try { - return VehicleEnvelope.parseFrom(kafkaValue); - } catch (InvalidProtocolBufferException ex) { - throw new IllegalArgumentException("VehicleEnvelope bytes are invalid", ex); - } - } -} diff --git a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateRepository.java b/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateRepository.java deleted file mode 100644 index 0fc571bf..00000000 --- a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateRepository.java +++ /dev/null @@ -1,22 +0,0 @@ -package com.lingniu.ingest.vehiclestate; - -import java.util.Optional; - -public interface VehicleStateRepository { - - void putState(String vin, String json); - - void putLocation(String vin, String json); - - void putSafety(String vin, String json); - - void putLastEvent(String vin, String json); - - Optional getState(String vin); - - Optional getLocation(String vin); - - Optional getSafety(String vin); - - Optional getLastEvent(String vin); -} diff --git a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateUpdater.java b/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateUpdater.java deleted file mode 100644 index 8d6d6788..00000000 --- a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/VehicleStateUpdater.java +++ /dev/null @@ -1,145 +0,0 @@ -package com.lingniu.ingest.vehiclestate; - -import com.lingniu.ingest.sink.kafka.proto.TelemetryField; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; - -import java.util.LinkedHashMap; -import java.util.Map; - -public final class VehicleStateUpdater { - - private final VehicleStateRepository repository; - - public VehicleStateUpdater(VehicleStateRepository repository) { - if (repository == null) { - throw new IllegalArgumentException("repository must not be null"); - } - this.repository = repository; - } - - public void update(VehicleEnvelope envelope) { - if (envelope == null) { - throw new IllegalArgumentException("envelope must not be null"); - } - if (!envelope.hasTelemetrySnapshot()) { - throw new IllegalArgumentException("envelope telemetry_snapshot is required"); - } - Map fields = fields(envelope); - String vin = envelope.getVin(); - - // vehicle-state 只维护“最新状态”缓存，覆盖写 Redis，不承担历史查询或 RAW 回放职责。 - repository.putState(vin, json(base(envelope, fields))); - repository.putLastEvent(vin, json(lastEvent(envelope))); - - if (fields.containsKey("longitude") && fields.containsKey("latitude")) { - repository.putLocation(vin, json(location(envelope, fields))); - } - if (hasSafetyFields(fields)) { - repository.putSafety(vin, json(safety(envelope, fields))); - } - } - - private static Map fields(VehicleEnvelope envelope) { - Map fields = new LinkedHashMap<>(); - for (TelemetryField field : envelope.getTelemetrySnapshot().getFieldsList()) { - // 同名字段以后到者为准，和最新状态语义一致。 - fields.put(field.getKey(), field.getValue()); - } - return fields; - } - - private static Map base(VehicleEnvelope envelope, Map fields) { - Map map = eventIdentity(envelope); - map.put("eventType", envelope.getTelemetrySnapshot().getEventType()); - map.put("fields", fields); - return map; - } - - private static Map location(VehicleEnvelope envelope, Map fields) { - Map map = eventIdentity(envelope); - putIfPresent(map, fields, "longitude"); - putIfPresent(map, fields, "latitude"); - putIfPresent(map, fields, "altitude_m"); - putIfPresent(map, fields, "direction_deg"); - putIfPresent(map, fields, "speed_kmh"); - return map; - } - - private static Map safety(VehicleEnvelope envelope, Map fields) { - Map map = eventIdentity(envelope); - putIfPresent(map, fields, "safety_category"); - putIfPresent(map, fields, "hydrogen_leak_detected"); - putIfPresent(map, fields, "hydrogen_leak_level"); - putIfPresent(map, fields, "hydrogen_leak_action_required"); - putIfPresent(map, fields, "alarm_level"); - return map; - } - - private static Map lastEvent(VehicleEnvelope envelope) { - return eventIdentity(envelope); - } - - private static Map eventIdentity(VehicleEnvelope envelope) { - Map map = new LinkedHashMap<>(); - map.put("eventId", envelope.getEventId()); - map.put("traceId", envelope.getTraceId()); - map.put("vin", envelope.getVin()); - map.put("source", envelope.getSource()); - map.put("eventTimeMs", Long.toString(envelope.getEventTimeMs())); - map.put("ingestTimeMs", Long.toString(envelope.getIngestTimeMs())); - return map; - } - - private static boolean hasSafetyFields(Map fields) { - // safety 是从标准字段中筛出来的窄视图，字段缺失时不写对应 Redis key。 - return fields.containsKey("safety_category") - || fields.containsKey("hydrogen_leak_detected") - || fields.containsKey("hydrogen_leak_level") - || fields.containsKey("hydrogen_leak_action_required") - || fields.containsKey("alarm_level"); - } - - private static void putIfPresent(Map map, Map fields, String key) { - if (fields.containsKey(key)) { - map.put(key, fields.get(key)); - } - } - - private static String json(Map map) { - StringBuilder out = new StringBuilder("{"); - boolean first = true; - for (Map.Entry entry : map.entrySet()) { - if (!first) { - out.append(','); - } - first = false; - out.append('"').append(escape(entry.getKey())).append('"').append(':'); - Object value = entry.getValue(); - if (value instanceof Map nested) { - out.append(jsonObject(nested)); - } else { - out.append('"').append(escape(String.valueOf(value))).append('"'); - } - } - return out.append('}').toString(); - } - - private static String jsonObject(Map map) { - StringBuilder out = new StringBuilder("{"); - boolean first = true; - for (Map.Entry entry : map.entrySet()) { - if (!first) { - out.append(','); - } - first = false; - out.append('"').append(escape(String.valueOf(entry.getKey()))).append('"') - .append(':') - .append('"').append(escape(String.valueOf(entry.getValue()))).append('"'); - } - return out.append('}').toString(); - } - - private static String escape(String value) { - return value.replace("\\", "\\\\").replace("\"", "\\\""); - } -} diff --git a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/config/VehicleStateAutoConfiguration.java b/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/config/VehicleStateAutoConfiguration.java deleted file mode 100644 index c808e737..00000000 --- a/modules/services/vehicle-state-service/src/main/java/com/lingniu/ingest/vehiclestate/config/VehicleStateAutoConfiguration.java +++ /dev/null @@ -1,58 +0,0 @@ -package com.lingniu.ingest.vehiclestate.config; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.api.consumer.EnvelopeDeadLetterSink; -import com.lingniu.ingest.vehiclestate.RedisVehicleStateRepository; -import com.lingniu.ingest.vehiclestate.VehicleStateController; -import com.lingniu.ingest.vehiclestate.VehicleStateEnvelopeIngestor; -import com.lingniu.ingest.vehiclestate.VehicleStateRepository; -import com.lingniu.ingest.vehiclestate.VehicleStateUpdater; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.context.annotation.Bean; -import org.springframework.data.redis.core.StringRedisTemplate; - -@AutoConfiguration -@ConditionalOnProperty(prefix = "lingniu.ingest.vehicle-state", name = "enabled", havingValue = "true") -public class VehicleStateAutoConfiguration { - - @Bean - @ConditionalOnBean(StringRedisTemplate.class) - @ConditionalOnMissingBean - public VehicleStateRepository vehicleStateRepository(StringRedisTemplate redis) { - // 车辆状态模块依赖 Redis；没有 Redis Bean 时不自动启用，避免误以为它是历史库。 - return new RedisVehicleStateRepository(redis); - } - - @Bean - @ConditionalOnBean(VehicleStateRepository.class) - @ConditionalOnMissingBean - public VehicleStateUpdater vehicleStateUpdater(VehicleStateRepository repository) { - return new VehicleStateUpdater(repository); - } - - @Bean - @ConditionalOnBean(VehicleStateUpdater.class) - @ConditionalOnMissingBean - public VehicleStateEnvelopeIngestor vehicleStateEnvelopeIngestor(VehicleStateUpdater updater) { - return new VehicleStateEnvelopeIngestor(updater); - } - - @Bean - @ConditionalOnBean({VehicleStateEnvelopeIngestor.class, EnvelopeDeadLetterSink.class}) - @ConditionalOnMissingBean(name = "vehicleStateEnvelopeConsumerProcessor") - public EnvelopeConsumerProcessor vehicleStateEnvelopeConsumerProcessor(VehicleStateEnvelopeIngestor ingestor, - EnvelopeDeadLetterSink deadLetterSink) { - // Bean 名必须和 Kafka 默认 binding 对齐，KafkaEnvelopeConsumerFactory 才能自动创建 worker。 - return new EnvelopeConsumerProcessor("vehicle-state", ingestor, deadLetterSink); - } - - @Bean - @ConditionalOnBean(VehicleStateRepository.class) - @ConditionalOnMissingBean - public VehicleStateController vehicleStateController(VehicleStateRepository repository) { - return new VehicleStateController(repository); - } -} diff --git a/modules/services/vehicle-state-service/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/services/vehicle-state-service/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index e6d5607b..00000000 --- a/modules/services/vehicle-state-service/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.vehiclestate.config.VehicleStateAutoConfiguration diff --git a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/InMemoryVehicleStateRepository.java b/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/InMemoryVehicleStateRepository.java deleted file mode 100644 index 4419eab6..00000000 --- a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/InMemoryVehicleStateRepository.java +++ /dev/null @@ -1,50 +0,0 @@ -package com.lingniu.ingest.vehiclestate; - -import java.util.Optional; - -final class InMemoryVehicleStateRepository implements VehicleStateRepository { - String state; - String location; - String safety; - String lastEvent; - - @Override - public void putState(String vin, String json) { - this.state = json; - } - - @Override - public void putLocation(String vin, String json) { - this.location = json; - } - - @Override - public void putSafety(String vin, String json) { - this.safety = json; - } - - @Override - public void putLastEvent(String vin, String json) { - this.lastEvent = json; - } - - @Override - public Optional getState(String vin) { - return Optional.ofNullable(state); - } - - @Override - public Optional getLocation(String vin) { - return Optional.ofNullable(location); - } - - @Override - public Optional getSafety(String vin) { - return Optional.ofNullable(safety); - } - - @Override - public Optional getLastEvent(String vin) { - return Optional.ofNullable(lastEvent); - } -} diff --git a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/RedisVehicleStateRepositoryTest.java b/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/RedisVehicleStateRepositoryTest.java deleted file mode 100644 index 06f59006..00000000 --- a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/RedisVehicleStateRepositoryTest.java +++ /dev/null @@ -1,50 +0,0 @@ -package com.lingniu.ingest.vehiclestate; - -import org.junit.jupiter.api.Test; -import org.springframework.data.redis.core.StringRedisTemplate; -import org.springframework.data.redis.core.ValueOperations; - -import java.util.Optional; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.verify; -import static org.mockito.Mockito.when; - -class RedisVehicleStateRepositoryTest { - - @Test - void writesAndReadsHotStateJsonByStableRedisKey() { - StringRedisTemplate redis = mock(StringRedisTemplate.class); - @SuppressWarnings("unchecked") - ValueOperations ops = mock(ValueOperations.class); - when(redis.opsForValue()).thenReturn(ops); - when(ops.get("vehicle:state:VIN001")).thenReturn("{\"vin\":\"VIN001\"}"); - - RedisVehicleStateRepository repository = new RedisVehicleStateRepository(redis); - - repository.putState("VIN001", "{\"vin\":\"VIN001\"}"); - Optional value = repository.getState("VIN001"); - - verify(ops).set("vehicle:state:VIN001", "{\"vin\":\"VIN001\"}"); - assertThat(value).contains("{\"vin\":\"VIN001\"}"); - } - - @Test - void writesLocationSafetyAndLastEventToSeparateKeys() { - StringRedisTemplate redis = mock(StringRedisTemplate.class); - @SuppressWarnings("unchecked") - ValueOperations ops = mock(ValueOperations.class); - when(redis.opsForValue()).thenReturn(ops); - - RedisVehicleStateRepository repository = new RedisVehicleStateRepository(redis); - - repository.putLocation("VIN001", "{\"longitude\":113.12}"); - repository.putSafety("VIN001", "{\"hydrogen_leak_detected\":true}"); - repository.putLastEvent("VIN001", "{\"eventId\":\"event-1\"}"); - - verify(ops).set("vehicle:location:VIN001", "{\"longitude\":113.12}"); - verify(ops).set("vehicle:safety:VIN001", "{\"hydrogen_leak_detected\":true}"); - verify(ops).set("vehicle:event:last:VIN001", "{\"eventId\":\"event-1\"}"); - } -} diff --git a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/VehicleStateControllerTest.java b/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/VehicleStateControllerTest.java deleted file mode 100644 index 2aac4541..00000000 --- a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/VehicleStateControllerTest.java +++ /dev/null @@ -1,54 +0,0 @@ -package com.lingniu.ingest.vehiclestate; - -import org.junit.jupiter.api.Test; -import org.springframework.test.web.servlet.MockMvc; - -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; -import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; -import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.content; -import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.status; -import static org.springframework.test.web.servlet.setup.MockMvcBuilders.standaloneSetup; - -class VehicleStateControllerTest { - - @Test - void returnsLatestStateJson() throws Exception { - VehicleStateRepository repository = mock(VehicleStateRepository.class); - when(repository.getState("VIN001")).thenReturn(java.util.Optional.of("{\"speed_kmh\":\"80.5\"}")); - MockMvc mvc = standaloneSetup(new VehicleStateController(repository)).build(); - - mvc.perform(get("/api/vehicle-state/VIN001")) - .andExpect(status().isOk()) - .andExpect(content().json("{\"speed_kmh\":\"80.5\"}")); - } - - @Test - void returnsLocationSafetyAndLastEventJson() throws Exception { - VehicleStateRepository repository = mock(VehicleStateRepository.class); - when(repository.getLocation("VIN001")).thenReturn(java.util.Optional.of("{\"longitude\":\"113.12\"}")); - when(repository.getSafety("VIN001")).thenReturn(java.util.Optional.of("{\"hydrogen_leak_detected\":\"true\"}")); - when(repository.getLastEvent("VIN001")).thenReturn(java.util.Optional.of("{\"eventId\":\"event-1\"}")); - MockMvc mvc = standaloneSetup(new VehicleStateController(repository)).build(); - - mvc.perform(get("/api/vehicle-state/VIN001/location")) - .andExpect(status().isOk()) - .andExpect(content().json("{\"longitude\":\"113.12\"}")); - mvc.perform(get("/api/vehicle-state/VIN001/safety")) - .andExpect(status().isOk()) - .andExpect(content().json("{\"hydrogen_leak_detected\":\"true\"}")); - mvc.perform(get("/api/vehicle-state/VIN001/last-event")) - .andExpect(status().isOk()) - .andExpect(content().json("{\"eventId\":\"event-1\"}")); - } - - @Test - void returnsNotFoundWhenStateIsMissing() throws Exception { - VehicleStateRepository repository = mock(VehicleStateRepository.class); - when(repository.getState("VIN001")).thenReturn(java.util.Optional.empty()); - MockMvc mvc = standaloneSetup(new VehicleStateController(repository)).build(); - - mvc.perform(get("/api/vehicle-state/VIN001")) - .andExpect(status().isNotFound()); - } -} diff --git a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/VehicleStateEnvelopeIngestorTest.java b/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/VehicleStateEnvelopeIngestorTest.java deleted file mode 100644 index dbf9dd92..00000000 --- a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/VehicleStateEnvelopeIngestorTest.java +++ /dev/null @@ -1,111 +0,0 @@ -package com.lingniu.ingest.vehiclestate; - -import com.lingniu.ingest.api.consumer.EnvelopeIngestResult; -import com.lingniu.ingest.api.consumer.EnvelopeIngestor; -import com.lingniu.ingest.sink.kafka.proto.RawArchiveRef; -import com.lingniu.ingest.sink.kafka.proto.TelemetryField; -import com.lingniu.ingest.sink.kafka.proto.TelemetrySnapshot; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import org.junit.jupiter.api.Test; - -import java.util.Optional; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class VehicleStateEnvelopeIngestorTest { - - @Test - void parsesEnvelopeBytesAndUpdatesRepository() { - InMemoryVehicleStateRepository repository = new InMemoryVehicleStateRepository(); - VehicleStateEnvelopeIngestor ingestor = new VehicleStateEnvelopeIngestor(new VehicleStateUpdater(repository)); - - assertThat(ingestor).isInstanceOf(EnvelopeIngestor.class); - ingestor.ingest(envelope().toByteArray()); - - assertThat(repository.getState("VIN001")).hasValueSatisfying(json -> assertThat(json).contains("speed_kmh")); - } - - @Test - void rejectsInvalidEnvelopeBytes() { - VehicleStateEnvelopeIngestor ingestor = - new VehicleStateEnvelopeIngestor(new VehicleStateUpdater(new EmptyRepository())); - - assertThatThrownBy(() -> ingestor.ingest(new byte[]{0x01, 0x02})) - .isInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("VehicleEnvelope"); - } - - @Test - void tryIngestInvalidEnvelopeBytesReturnsInvalidWithoutUpdatingRepository() { - InMemoryVehicleStateRepository repository = new InMemoryVehicleStateRepository(); - VehicleStateEnvelopeIngestor ingestor = new VehicleStateEnvelopeIngestor(new VehicleStateUpdater(repository)); - - var result = ingestor.tryIngest(new byte[]{0x01, 0x02}); - - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.INVALID_ENVELOPE); - assertThat(result.message()).contains("VehicleEnvelope"); - assertThat(repository.getState("VIN001")).isEmpty(); - } - - @Test - void rawArchiveEnvelopeIsIgnoredWithoutUpdatingRepository() { - InMemoryVehicleStateRepository repository = new InMemoryVehicleStateRepository(); - VehicleStateEnvelopeIngestor ingestor = new VehicleStateEnvelopeIngestor(new VehicleStateUpdater(repository)); - VehicleEnvelope envelope = rawArchiveEnvelope(); - - ingestor.ingest(envelope.toByteArray()); - var result = ingestor.tryIngest(envelope.toByteArray()); - - assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.SKIPPED); - assertThat(result.eventId()).isEqualTo("raw-event-1"); - assertThat(result.vin()).isEqualTo("VIN001"); - assertThat(result.message()).contains("telemetry_snapshot"); - assertThat(repository.getState("VIN001")).isEmpty(); - assertThat(repository.getLocation("VIN001")).isEmpty(); - assertThat(repository.getSafety("VIN001")).isEmpty(); - assertThat(repository.getLastEvent("VIN001")).isEmpty(); - } - - private static VehicleEnvelope envelope() { - return VehicleEnvelope.newBuilder() - .setEventId("event-1") - .setVin("VIN001") - .setSource("GB32960") - .setEventTimeMs(1) - .setIngestTimeMs(2) - .setTelemetrySnapshot(TelemetrySnapshot.newBuilder() - .setEventType("REALTIME") - .addFields(TelemetryField.newBuilder() - .setKey("speed_kmh") - .setValueType("DOUBLE") - .setValue("80.5") - .setUnit("km/h"))) - .build(); - } - - private static VehicleEnvelope rawArchiveEnvelope() { - return VehicleEnvelope.newBuilder() - .setEventId("raw-event-1") - .setVin("VIN001") - .setSource("GB32960") - .setEventTimeMs(1_782_112_400_000L) - .setIngestTimeMs(1_782_112_401_000L) - .setRawArchive(RawArchiveRef.newBuilder() - .setUri("archive://2026/06/23/GB32960/VIN001/raw-event-1.bin") - .setSizeBytes(128) - .build()) - .build(); - } - - private static final class EmptyRepository implements VehicleStateRepository { - @Override public void putState(String vin, String json) {} - @Override public void putLocation(String vin, String json) {} - @Override public void putSafety(String vin, String json) {} - @Override public void putLastEvent(String vin, String json) {} - @Override public Optional getState(String vin) { return Optional.empty(); } - @Override public Optional getLocation(String vin) { return Optional.empty(); } - @Override public Optional getSafety(String vin) { return Optional.empty(); } - @Override public Optional getLastEvent(String vin) { return Optional.empty(); } - } -} diff --git a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/VehicleStateUpdaterTest.java b/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/VehicleStateUpdaterTest.java deleted file mode 100644 index c896c85b..00000000 --- a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/VehicleStateUpdaterTest.java +++ /dev/null @@ -1,80 +0,0 @@ -package com.lingniu.ingest.vehiclestate; - -import com.lingniu.ingest.sink.kafka.proto.TelemetryField; -import com.lingniu.ingest.sink.kafka.proto.TelemetrySnapshot; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; - -class VehicleStateUpdaterTest { - - @Test - void writesStateLocationSafetyAndLastEventFromTelemetrySnapshot() { - InMemoryVehicleStateRepository repository = new InMemoryVehicleStateRepository(); - VehicleStateUpdater updater = new VehicleStateUpdater(repository); - - updater.update(envelope() - .addField("speed_kmh", "DOUBLE", "80.5", "km/h") - .addField("longitude", "DOUBLE", "113.12", "deg") - .addField("latitude", "DOUBLE", "23.45", "deg") - .addField("hydrogen_leak_detected", "BOOLEAN", "true", "") - .addField("hydrogen_leak_level", "STRING", "CRITICAL", "") - .build()); - - assertThat(repository.state).contains("\"vin\":\"VIN001\""); - assertThat(repository.state).contains("\"speed_kmh\""); - assertThat(repository.location).contains("\"longitude\":\"113.12\""); - assertThat(repository.location).contains("\"latitude\":\"23.45\""); - assertThat(repository.safety).contains("\"hydrogen_leak_detected\":\"true\""); - assertThat(repository.lastEvent).contains("\"eventId\":\"event-1\""); - } - - @Test - void doesNotOverwriteLocationOrSafetyWhenFieldsAreMissing() { - InMemoryVehicleStateRepository repository = new InMemoryVehicleStateRepository(); - repository.location = "{\"longitude\":\"old\"}"; - repository.safety = "{\"hydrogen_leak_detected\":\"old\"}"; - - new VehicleStateUpdater(repository).update(envelope() - .addField("speed_kmh", "DOUBLE", "80.5", "km/h") - .build()); - - assertThat(repository.location).isEqualTo("{\"longitude\":\"old\"}"); - assertThat(repository.safety).isEqualTo("{\"hydrogen_leak_detected\":\"old\"}"); - assertThat(repository.state).contains("\"speed_kmh\""); - } - - private static EnvelopeBuilder envelope() { - return new EnvelopeBuilder(); - } - - private static final class EnvelopeBuilder { - private final TelemetrySnapshot.Builder snapshot = TelemetrySnapshot.newBuilder() - .setEventType("REALTIME") - .setRawArchiveUri("archive://raw/event-1.bin"); - - EnvelopeBuilder addField(String key, String valueType, String value, String unit) { - snapshot.addFields(TelemetryField.newBuilder() - .setKey(key) - .setValueType(valueType) - .setValue(value) - .setUnit(unit) - .setQuality("GOOD") - .setSourcePath("test")); - return this; - } - - VehicleEnvelope build() { - return VehicleEnvelope.newBuilder() - .setEventId("event-1") - .setTraceId("trace-1") - .setVin("VIN001") - .setSource("GB32960") - .setEventTimeMs(1_782_112_400_000L) - .setIngestTimeMs(1_782_112_401_000L) - .setTelemetrySnapshot(snapshot) - .build(); - } - } -} diff --git a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/config/VehicleStateAutoConfigurationTest.java b/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/config/VehicleStateAutoConfigurationTest.java deleted file mode 100644 index ba9cda84..00000000 --- a/modules/services/vehicle-state-service/src/test/java/com/lingniu/ingest/vehiclestate/config/VehicleStateAutoConfigurationTest.java +++ /dev/null @@ -1,56 +0,0 @@ -package com.lingniu.ingest.vehiclestate.config; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.api.consumer.EnvelopeDeadLetterSink; -import com.lingniu.ingest.vehiclestate.RedisVehicleStateRepository; -import com.lingniu.ingest.vehiclestate.VehicleStateController; -import com.lingniu.ingest.vehiclestate.VehicleStateEnvelopeIngestor; -import com.lingniu.ingest.vehiclestate.VehicleStateRepository; -import com.lingniu.ingest.vehiclestate.VehicleStateUpdater; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.data.redis.core.StringRedisTemplate; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.mock; - -class VehicleStateAutoConfigurationTest { - - private final ApplicationContextRunner contextRunner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(VehicleStateAutoConfiguration.class)) - .withBean(EnvelopeDeadLetterSink.class, () -> record -> {}) - .withBean(StringRedisTemplate.class, () -> mock(StringRedisTemplate.class)); - - @Test - void createsVehicleStateBeansWhenEnabled() { - contextRunner - .withPropertyValues("lingniu.ingest.vehicle-state.enabled=true") - .run(context -> { - assertThat(context).hasSingleBean(VehicleStateRepository.class); - assertThat(context).hasSingleBean(RedisVehicleStateRepository.class); - assertThat(context).hasSingleBean(VehicleStateUpdater.class); - assertThat(context).hasSingleBean(VehicleStateEnvelopeIngestor.class); - assertThat(context).hasSingleBean(EnvelopeConsumerProcessor.class); - assertThat(context).hasSingleBean(VehicleStateController.class); - }); - } - - @Test - void backsOffWhenDisabled() { - contextRunner.run(context -> { - assertThat(context).doesNotHaveBean(VehicleStateRepository.class); - assertThat(context).doesNotHaveBean(VehicleStateUpdater.class); - assertThat(context).doesNotHaveBean(VehicleStateEnvelopeIngestor.class); - assertThat(context).doesNotHaveBean(VehicleStateController.class); - }); - } - - @Test - void doesNotCreateRepositoryWithoutRedisTemplate() { - new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(VehicleStateAutoConfiguration.class)) - .withPropertyValues("lingniu.ingest.vehicle-state.enabled=true") - .run(context -> assertThat(context).doesNotHaveBean(VehicleStateRepository.class)); - } -} diff --git a/modules/sinks/sink-archive/pom.xml b/modules/sinks/sink-archive/pom.xml deleted file mode 100644 index 08d269e7..00000000 --- a/modules/sinks/sink-archive/pom.xml +++ /dev/null @@ -1,39 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - sink-archive - sink-archive - 原始报文冷存：本地文件系统实现。 - - - - com.lingniu.ingest - ingest-api - - - org.springframework.boot - spring-boot-starter - - - org.springframework.boot - spring-boot-configuration-processor - true - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - diff --git a/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/ArchiveStore.java b/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/ArchiveStore.java deleted file mode 100644 index 211d633c..00000000 --- a/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/ArchiveStore.java +++ /dev/null @@ -1,17 +0,0 @@ -package com.lingniu.ingest.sink.archive; - -import java.io.IOException; -import java.io.InputStream; - -public interface ArchiveStore { - - String put(String key, InputStream data, long length) throws IOException; - - String append(String key, byte[] chunk) throws IOException; - - InputStream get(String key) throws IOException; - - boolean exists(String key); - - long size(String key) throws IOException; -} diff --git a/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/LocalArchiveStore.java b/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/LocalArchiveStore.java deleted file mode 100644 index 89e6f04e..00000000 --- a/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/LocalArchiveStore.java +++ /dev/null @@ -1,150 +0,0 @@ -package com.lingniu.ingest.sink.archive; - -import java.io.IOException; -import java.io.InputStream; -import java.io.OutputStream; -import java.net.URI; -import java.nio.file.Files; -import java.nio.file.LinkOption; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.ConcurrentMap; - -public final class LocalArchiveStore implements ArchiveStore { - - private final Path root; - private final ConcurrentMap appendLocks = new ConcurrentHashMap<>(); - - public LocalArchiveStore(String root) { - this(rootPath(root)); - } - - public LocalArchiveStore(Path root) { - this.root = root.toAbsolutePath().normalize(); - } - - @Override - public String put(String key, InputStream data, long length) throws IOException { - if (data == null) { - throw new IllegalArgumentException("data must not be null"); - } - Path target = resolve(key); - createDirectoriesInsideRoot(target.getParent()); - rejectSymlinkPath(target); - try (OutputStream out = Files.newOutputStream( - target, - StandardOpenOption.CREATE, - StandardOpenOption.TRUNCATE_EXISTING, - StandardOpenOption.WRITE, - LinkOption.NOFOLLOW_LINKS)) { - data.transferTo(out); - } - return target.toUri().toString(); - } - - @Override - public String append(String key, byte[] chunk) throws IOException { - Path target = resolve(key); - synchronized (appendLocks.computeIfAbsent(target, ignored -> new Object())) { - createDirectoriesInsideRoot(target.getParent()); - rejectSymlinkPath(target); - Files.write( - target, - chunk == null ? new byte[0] : chunk, - StandardOpenOption.CREATE, - StandardOpenOption.APPEND, - StandardOpenOption.WRITE, - LinkOption.NOFOLLOW_LINKS); - } - return target.toUri().toString(); - } - - @Override - public InputStream get(String key) throws IOException { - Path target = resolveExisting(key); - return Files.newInputStream(target, StandardOpenOption.READ, LinkOption.NOFOLLOW_LINKS); - } - - @Override - public boolean exists(String key) { - try { - return Files.exists(resolveExisting(key), LinkOption.NOFOLLOW_LINKS); - } catch (IOException | IllegalArgumentException e) { - return false; - } - } - - @Override - public long size(String key) throws IOException { - return Files.size(resolveExisting(key)); - } - - private Path resolve(String key) { - if (key == null || key.isBlank()) { - throw new IllegalArgumentException("archive key must not be blank"); - } - Path target = root.resolve(key).normalize(); - if (!target.startsWith(root)) { - throw new IllegalArgumentException("archive key escapes root: " + key); - } - return target; - } - - private Path resolveExisting(String key) throws IOException { - Path target = resolve(key); - rejectSymlinkPath(target); - return target; - } - - private void createDirectoriesInsideRoot(Path directory) throws IOException { - rejectEscapedPath(directory); - if (Files.notExists(root, LinkOption.NOFOLLOW_LINKS)) { - Files.createDirectories(root); - } - Path current = root; - Path relative = root.relativize(directory); - for (Path segment : relative) { - current = current.resolve(segment); - if (Files.isSymbolicLink(current)) { - throw new IOException("archive path traverses symlink: " + current); - } - if (Files.notExists(current, LinkOption.NOFOLLOW_LINKS)) { - Files.createDirectory(current); - } else if (!Files.isDirectory(current, LinkOption.NOFOLLOW_LINKS)) { - throw new IOException("archive path segment is not a directory: " + current); - } - } - } - - private void rejectSymlinkPath(Path target) throws IOException { - rejectEscapedPath(target); - Path current = root; - Path relative = root.relativize(target); - for (Path segment : relative) { - current = current.resolve(segment); - if (Files.isSymbolicLink(current)) { - throw new IOException("archive path traverses symlink: " + current); - } - if (Files.notExists(current, LinkOption.NOFOLLOW_LINKS)) { - return; - } - } - } - - private void rejectEscapedPath(Path path) { - if (!path.normalize().startsWith(root)) { - throw new IllegalArgumentException("archive path escapes root: " + path); - } - } - - private static Path rootPath(String value) { - if (value == null || value.isBlank()) { - return Path.of(System.getProperty("java.io.tmpdir"), "lingniu-archive"); - } - if (value.startsWith("file://")) { - return Path.of(URI.create(value)); - } - return Path.of(value); - } -} diff --git a/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/RawArchiveEventSink.java b/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/RawArchiveEventSink.java deleted file mode 100644 index 5a1e9e8d..00000000 --- a/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/RawArchiveEventSink.java +++ /dev/null @@ -1,150 +0,0 @@ -package com.lingniu.ingest.sink.archive; - -import com.lingniu.ingest.api.event.RawArchiveKeys; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.sink.EventSink; - -import java.io.IOException; -import java.nio.file.FileAlreadyExistsException; -import java.nio.file.Files; -import java.nio.file.LinkOption; -import java.nio.file.Path; -import java.nio.file.StandardOpenOption; -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.util.Arrays; -import java.util.HexFormat; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.ExecutorService; -import java.util.concurrent.Executors; - -public final class RawArchiveEventSink implements EventSink, AutoCloseable { - - private final Path root; - private final ExecutorService executor; - - public RawArchiveEventSink(Path root) { - if (root == null) { - throw new IllegalArgumentException("root must not be null"); - } - this.root = root.toAbsolutePath().normalize(); - this.executor = Executors.newVirtualThreadPerTaskExecutor(); - } - - @Override - public String name() { - return "raw-archive"; - } - - @Override - public boolean accepts(VehicleEvent event) { - return event instanceof VehicleEvent.RawArchive; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - if (!(event instanceof VehicleEvent.RawArchive raw)) { - return CompletableFuture.completedFuture(null); - } - return CompletableFuture.runAsync(() -> write(raw), executor); - } - - @Override - public void close() { - executor.shutdown(); - } - - public Path root() { - return root; - } - - private void write(VehicleEvent.RawArchive raw) { - byte[] bytes = raw.rawBytes() == null ? new byte[0] : raw.rawBytes(); - if (bytes.length == 0) { - return; - } - String key = RawArchiveKeys.key(raw); - Path target = resolveKey(key); - try { - createDirectoriesInsideRoot(target.getParent()); - rejectSymlinkPath(target); - try { - Files.write(target, bytes, StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE); - } catch (FileAlreadyExistsException duplicate) { - byte[] existing = Files.readAllBytes(target); - if (!Arrays.equals(sha256(existing), sha256(bytes))) { - throw new IOException("raw archive duplicate key has different content: " + key, duplicate); - } - } - } catch (IOException e) { - throw new RawArchiveWriteException("failed to write raw archive " + RawArchiveKeys.logicalUri(key), e); - } - } - - private Path resolveKey(String key) { - Path target = root.resolve(key == null ? "" : key).normalize(); - if (!target.startsWith(root)) { - throw new IllegalArgumentException("raw archive key escapes root: " + key); - } - return target; - } - - private void createDirectoriesInsideRoot(Path directory) throws IOException { - rejectEscapedPath(directory); - if (Files.notExists(root, LinkOption.NOFOLLOW_LINKS)) { - Files.createDirectories(root); - } - Path current = root; - Path relative = root.relativize(directory); - for (Path segment : relative) { - current = current.resolve(segment); - if (Files.isSymbolicLink(current)) { - throw new IOException("raw archive path traverses symlink: " + current); - } - if (Files.notExists(current, LinkOption.NOFOLLOW_LINKS)) { - Files.createDirectory(current); - } else if (!Files.isDirectory(current, LinkOption.NOFOLLOW_LINKS)) { - throw new IOException("raw archive path segment is not a directory: " + current); - } - } - } - - private void rejectSymlinkPath(Path target) throws IOException { - rejectEscapedPath(target); - Path current = root; - Path relative = root.relativize(target); - for (Path segment : relative) { - current = current.resolve(segment); - if (Files.isSymbolicLink(current)) { - throw new IOException("raw archive path traverses symlink: " + current); - } - if (Files.notExists(current, LinkOption.NOFOLLOW_LINKS)) { - return; - } - } - } - - private void rejectEscapedPath(Path path) { - if (!path.normalize().startsWith(root)) { - throw new IllegalArgumentException("raw archive path escapes root: " + path); - } - } - - private static byte[] sha256(byte[] bytes) { - try { - return MessageDigest.getInstance("SHA-256").digest(bytes); - } catch (NoSuchAlgorithmException e) { - throw new IllegalStateException("SHA-256 is unavailable", e); - } - } - - static String checksum(byte[] bytes) { - return "sha256:" + HexFormat.of().formatHex(sha256(bytes == null ? new byte[0] : bytes)); - } - - public static final class RawArchiveWriteException extends RuntimeException { - public RawArchiveWriteException(String message, Throwable cause) { - super(message, cause); - } - } -} diff --git a/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/config/SinkArchiveAutoConfiguration.java b/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/config/SinkArchiveAutoConfiguration.java deleted file mode 100644 index 0bf04f4e..00000000 --- a/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/config/SinkArchiveAutoConfiguration.java +++ /dev/null @@ -1,32 +0,0 @@ -package com.lingniu.ingest.sink.archive.config; - -import com.lingniu.ingest.sink.archive.ArchiveStore; -import com.lingniu.ingest.sink.archive.LocalArchiveStore; -import com.lingniu.ingest.sink.archive.RawArchiveEventSink; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; - -import java.nio.file.Path; - -@AutoConfiguration -@EnableConfigurationProperties(SinkArchiveProperties.class) -@ConditionalOnProperty(prefix = "lingniu.ingest.sink.archive", name = "enabled", havingValue = "true", matchIfMissing = true) -public class SinkArchiveAutoConfiguration { - - @Bean - @ConditionalOnMissingBean - @ConditionalOnProperty(prefix = "lingniu.ingest.sink.archive", name = "type", havingValue = "local", matchIfMissing = true) - public ArchiveStore archiveStore(SinkArchiveProperties properties) { - return new LocalArchiveStore(Path.of(properties.getPath())); - } - - @Bean - @ConditionalOnMissingBean - @ConditionalOnProperty(prefix = "lingniu.ingest.sink.archive", name = "type", havingValue = "local", matchIfMissing = true) - public RawArchiveEventSink rawArchiveEventSink(SinkArchiveProperties properties) { - return new RawArchiveEventSink(Path.of(properties.getPath())); - } -} diff --git a/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/config/SinkArchiveProperties.java b/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/config/SinkArchiveProperties.java deleted file mode 100644 index 32668078..00000000 --- a/modules/sinks/sink-archive/src/main/java/com/lingniu/ingest/sink/archive/config/SinkArchiveProperties.java +++ /dev/null @@ -1,35 +0,0 @@ -package com.lingniu.ingest.sink.archive.config; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -@ConfigurationProperties(prefix = "lingniu.ingest.sink.archive") -public class SinkArchiveProperties { - - private boolean enabled = true; - private String type = "local"; - private String path = "./archive/"; - - public boolean isEnabled() { - return enabled; - } - - public void setEnabled(boolean enabled) { - this.enabled = enabled; - } - - public String getType() { - return type; - } - - public void setType(String type) { - this.type = type; - } - - public String getPath() { - return path; - } - - public void setPath(String path) { - this.path = path; - } -} diff --git a/modules/sinks/sink-archive/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/sinks/sink-archive/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index 4dcf699d..00000000 --- a/modules/sinks/sink-archive/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.sink.archive.config.SinkArchiveAutoConfiguration diff --git a/modules/sinks/sink-archive/src/test/java/com/lingniu/ingest/sink/archive/LocalArchiveStoreTest.java b/modules/sinks/sink-archive/src/test/java/com/lingniu/ingest/sink/archive/LocalArchiveStoreTest.java deleted file mode 100644 index f23e90c2..00000000 --- a/modules/sinks/sink-archive/src/test/java/com/lingniu/ingest/sink/archive/LocalArchiveStoreTest.java +++ /dev/null @@ -1,74 +0,0 @@ -package com.lingniu.ingest.sink.archive; - -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.io.ByteArrayInputStream; -import java.nio.charset.StandardCharsets; -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.ArrayList; -import java.util.List; -import java.util.concurrent.CompletableFuture; -import java.util.concurrent.Executors; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class LocalArchiveStoreTest { - - @TempDir - Path tempDir; - - @Test - void rejectsSymlinkTraversalOutsideRootForWriteReadAndSize() throws Exception { - Path root = tempDir.resolve("archive"); - Path outside = tempDir.resolve("outside"); - Files.createDirectories(root); - Files.createDirectories(outside); - Files.writeString(outside.resolve("existing.bin"), "secret", StandardCharsets.UTF_8); - Files.createSymbolicLink(root.resolve("link"), outside); - - LocalArchiveStore store = new LocalArchiveStore(root); - - assertThatThrownBy(() -> store.put( - "link/new.bin", - new ByteArrayInputStream("payload".getBytes(StandardCharsets.UTF_8)), - 7)) - .isInstanceOfAny(IllegalArgumentException.class, java.io.IOException.class); - assertThat(outside.resolve("new.bin")).doesNotExist(); - - assertThatThrownBy(() -> store.get("link/existing.bin")) - .isInstanceOfAny(IllegalArgumentException.class, java.io.IOException.class); - assertThatThrownBy(() -> store.size("link/existing.bin")) - .isInstanceOfAny(IllegalArgumentException.class, java.io.IOException.class); - } - - @Test - void appendsConcurrentChunksWithoutLosingData() throws Exception { - LocalArchiveStore store = new LocalArchiveStore(tempDir.resolve("archive")); - int chunks = 96; - - try (var executor = Executors.newFixedThreadPool(12)) { - List> futures = new ArrayList<>(); - for (int i = 0; i < chunks; i++) { - String chunk = "%03d\n".formatted(i); - futures.add(CompletableFuture.runAsync(() -> { - try { - store.append("same/key.bin", chunk.getBytes(StandardCharsets.UTF_8)); - } catch (Exception e) { - throw new RuntimeException(e); - } - }, executor)); - } - CompletableFuture.allOf(futures.toArray(CompletableFuture[]::new)).join(); - } - - String content = Files.readString(tempDir.resolve("archive/same/key.bin"), StandardCharsets.UTF_8); - assertThat(content).hasSize(chunks * 4); - assertThat(content.lines()).containsExactlyInAnyOrderElementsOf( - java.util.stream.IntStream.range(0, chunks) - .mapToObj("%03d"::formatted) - .toList()); - } -} diff --git a/modules/sinks/sink-archive/src/test/java/com/lingniu/ingest/sink/archive/RawArchiveEventSinkTest.java b/modules/sinks/sink-archive/src/test/java/com/lingniu/ingest/sink/archive/RawArchiveEventSinkTest.java deleted file mode 100644 index c6534851..00000000 --- a/modules/sinks/sink-archive/src/test/java/com/lingniu/ingest/sink/archive/RawArchiveEventSinkTest.java +++ /dev/null @@ -1,77 +0,0 @@ -package com.lingniu.ingest.sink.archive; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.RawArchiveKeys; -import com.lingniu.ingest.api.event.VehicleEvent; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.io.TempDir; - -import java.nio.file.Files; -import java.nio.file.Path; -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.assertj.core.api.Assertions.assertThatThrownBy; - -class RawArchiveEventSinkTest { - - @TempDir - Path tempDir; - - @Test - void writesRawBytesUsingSharedArchiveKey() throws Exception { - RawArchiveEventSink sink = new RawArchiveEventSink(tempDir); - VehicleEvent.RawArchive raw = rawArchive(Map.of(), new byte[]{0x23, 0x23, 0x01}); - - sink.publish(raw).join(); - - String key = RawArchiveKeys.key(raw); - assertThat(Files.readAllBytes(tempDir.resolve(key))).containsExactly(0x23, 0x23, 0x01); - assertThat(RawArchiveKeys.logicalUri(key)).isEqualTo("archive://" + key); - } - - @Test - void rejectsArchiveKeysThatEscapeRoot() { - RawArchiveEventSink sink = new RawArchiveEventSink(tempDir); - VehicleEvent.RawArchive raw = rawArchive( - Map.of(RawArchiveKeys.META_KEY, "../escape.bin"), - new byte[]{0x01}); - - assertThatThrownBy(() -> sink.publish(raw).join()) - .hasCauseInstanceOf(IllegalArgumentException.class) - .hasMessageContaining("raw archive key escapes root"); - } - - @Test - void rejectsSymlinkTraversalBeforeCreatingDirectories() throws Exception { - Path root = tempDir.resolve("archive"); - Path outside = tempDir.resolve("outside"); - Files.createDirectories(root); - Files.createDirectories(outside); - Files.createSymbolicLink(root.resolve("link"), outside); - - RawArchiveEventSink sink = new RawArchiveEventSink(root); - VehicleEvent.RawArchive raw = rawArchive( - Map.of(RawArchiveKeys.META_KEY, "link/nested/raw.bin"), - new byte[]{0x01}); - - assertThatThrownBy(() -> sink.publish(raw).join()) - .hasRootCauseMessage("raw archive path traverses symlink: " + root.resolve("link")); - assertThat(outside.resolve("nested")).doesNotExist(); - } - - private static VehicleEvent.RawArchive rawArchive(Map metadata, byte[] rawBytes) { - return new VehicleEvent.RawArchive( - "raw-1", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "trace-1", - metadata, - 0x02, - 0, - rawBytes); - } -} diff --git a/modules/sinks/sink-archive/src/test/java/com/lingniu/ingest/sink/archive/config/SinkArchiveAutoConfigurationTest.java b/modules/sinks/sink-archive/src/test/java/com/lingniu/ingest/sink/archive/config/SinkArchiveAutoConfigurationTest.java deleted file mode 100644 index 2156cb9e..00000000 --- a/modules/sinks/sink-archive/src/test/java/com/lingniu/ingest/sink/archive/config/SinkArchiveAutoConfigurationTest.java +++ /dev/null @@ -1,40 +0,0 @@ -package com.lingniu.ingest.sink.archive.config; - -import com.lingniu.ingest.sink.archive.ArchiveStore; -import com.lingniu.ingest.sink.archive.RawArchiveEventSink; -import org.junit.jupiter.api.Test; -import org.springframework.context.annotation.AnnotationConfigApplicationContext; -import org.springframework.core.env.MapPropertySource; - -import static org.assertj.core.api.Assertions.assertThat; - -class SinkArchiveAutoConfigurationTest { - - @Test - void localArchiveTypeCreatesStoreAndSink() { - try (AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext()) { - context.getEnvironment().getPropertySources().addFirst(new MapPropertySource( - "test", - java.util.Map.of("lingniu.ingest.sink.archive.path", "target/test-archive"))); - context.register(SinkArchiveAutoConfiguration.class); - context.refresh(); - - assertThat(context.getBeansOfType(ArchiveStore.class)).hasSize(1); - assertThat(context.getBeansOfType(RawArchiveEventSink.class)).hasSize(1); - } - } - - @Test - void nonLocalArchiveTypeDoesNotCreateSinkWithoutStore() { - try (AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext()) { - context.getEnvironment().getPropertySources().addFirst(new MapPropertySource( - "test", - java.util.Map.of("lingniu.ingest.sink.archive.type", "oss"))); - context.register(SinkArchiveAutoConfiguration.class); - context.refresh(); - - assertThat(context.getBeansOfType(ArchiveStore.class)).isEmpty(); - assertThat(context.getBeansOfType(RawArchiveEventSink.class)).isEmpty(); - } - } -} diff --git a/modules/sinks/sink-kafka/pom.xml b/modules/sinks/sink-kafka/pom.xml deleted file mode 100644 index aafb8d76..00000000 --- a/modules/sinks/sink-kafka/pom.xml +++ /dev/null @@ -1,96 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - sink-kafka - sink-kafka - Kafka producer + Protobuf Envelope + 重试/熔断/DLQ。 - - - - com.lingniu.ingest - ingest-api - - - com.lingniu.ingest - ingest-facts - - - org.springframework.boot - spring-boot-starter - - - org.springframework.boot - spring-boot-configuration-processor - true - - - org.apache.kafka - kafka-clients - - - com.google.protobuf - protobuf-java - - - com.google.protobuf - protobuf-java-util - - - io.github.resilience4j - resilience4j-retry - - - io.github.resilience4j - resilience4j-circuitbreaker - - - org.springframework.boot - spring-boot-starter-test - test - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - - - - - kr.motd.maven - os-maven-plugin - 1.7.1 - - - - - org.xolstice.maven.plugins - protobuf-maven-plugin - 0.6.1 - - com.google.protobuf:protoc:${protobuf.version}:exe:${os.detected.classifier} - - - - - compile - test-compile - - - - - - - diff --git a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/EnvelopeMapper.java b/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/EnvelopeMapper.java deleted file mode 100644 index 04cf415d..00000000 --- a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/EnvelopeMapper.java +++ /dev/null @@ -1,226 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.event.AlarmPayload; -import com.lingniu.ingest.api.event.LocationPayload; -import com.lingniu.ingest.api.event.RawArchiveKeys; -import com.lingniu.ingest.api.event.RealtimePayload; -import com.lingniu.ingest.api.event.TelemetryFieldValue; -import com.lingniu.ingest.api.event.TelemetrySnapshot; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.event.VehicleEventTelemetrySnapshotMapper; -import com.lingniu.ingest.facts.RawFrameIdentity; -import com.lingniu.ingest.sink.kafka.proto.ParseStatusProto; -import com.lingniu.ingest.sink.kafka.proto.TelemetryField; -import com.lingniu.ingest.sink.kafka.proto.TelemetrySnapshot.Builder; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; - -/** - * 领域事件 → Protobuf Envelope 的纯函数映射。通过 switch 模式匹配穷尽处理， - * 新增 {@link VehicleEvent} 子类型时编译期就会提示补齐分支。 - */ -public final class EnvelopeMapper { - - private static final String SCHEMA_VERSION = "1.0"; - private final String nodeId; - - public EnvelopeMapper(String nodeId) { - this.nodeId = nodeId; - } - - public VehicleEnvelope toEnvelope(VehicleEvent event) { - VehicleEnvelope.Builder b = VehicleEnvelope.newBuilder() - .setSchemaVersion(SCHEMA_VERSION) - .setEventId(event.eventId()) - .setTraceId(event.traceId() == null ? "" : event.traceId()) - .setVin(event.vin()) - .setSource(event.source().name()) - .setEventTimeMs(event.eventTime().toEpochMilli()) - .setIngestTimeMs(event.ingestTime().toEpochMilli()) - .setIngestNodeId(nodeId); - if (event.metadata() != null) b.putAllMetadata(event.metadata()); - // telemetrySnapshot 是跨事件类型的统一字段视图，消费者可先读它，必要时再读具体 payload。 - VehicleEventTelemetrySnapshotMapper.toSnapshot(event) - .map(EnvelopeMapper::buildTelemetrySnapshot) - .ifPresent(b::setTelemetrySnapshot); - - switch (event) { - case VehicleEvent.Realtime r -> b.setRealtime(buildRealtime(r.payload())); - case VehicleEvent.Location l -> b.setLocation(buildLocation(l.payload())); - case VehicleEvent.Alarm a -> b.setAlarm(buildAlarm(a.payload())); - case VehicleEvent.Login lg -> b.setLogin( - com.lingniu.ingest.sink.kafka.proto.LoginPayload.newBuilder() - .setIccid(nullToEmpty(lg.iccid())) - .setProtocolVersion(nullToEmpty(lg.protocolVersion())) - .build()); - case VehicleEvent.Logout ignored -> b.setLogout( - com.lingniu.ingest.sink.kafka.proto.LogoutPayload.getDefaultInstance()); - case VehicleEvent.Heartbeat ignored -> b.setHeartbeat( - com.lingniu.ingest.sink.kafka.proto.HeartbeatPayload.getDefaultInstance()); - case VehicleEvent.MediaMeta m -> b.setMediaMeta( - com.lingniu.ingest.sink.kafka.proto.MediaMetaPayload.newBuilder() - .setMediaId(nullToEmpty(m.mediaId())) - .setMediaType(nullToEmpty(m.mediaType())) - .setSizeBytes(m.sizeBytes()) - .setArchiveRef(nullToEmpty(m.archiveRef())) - .build()); - case VehicleEvent.Passthrough p -> b.setPassthrough( - com.lingniu.ingest.sink.kafka.proto.PassthroughPayload.newBuilder() - .setPassthroughType(p.passthroughType()) - .setData(com.google.protobuf.ByteString.copyFrom( - p.data() == null ? new byte[0] : p.data())) - .build()); - case VehicleEvent.RawArchive ra -> { - byte[] rawBytes = ra.rawBytes() == null ? new byte[0] : ra.rawBytes(); - int size = rawBytes.length; - String key = rawArchiveKey(ra); - String uri = rawArchiveUri(ra, key); - String phone = metadataValue(ra, "phone"); - RawFrameIdentity identity = RawFrameIdentity.derive( - ra.source(), ra.vin(), phone, ra.eventId(), ra.command(), ra.infoType(), - ra.ingestTime(), rawBytes); - // RAW envelope 只携带 URI/size 引用信息，不把完整原始字节塞进 Kafka，避免 topic 膨胀。 - b.putMetadata(RawArchiveKeys.META_KEY, key); - b.putMetadata(RawArchiveKeys.META_URI, uri); - b.putMetadata(RawArchiveKeys.META_EVENT_ID, ra.eventId()); - b.setRawArchive(com.lingniu.ingest.sink.kafka.proto.RawArchiveRef.newBuilder() - .setUri(uri) - .setChecksum(identity.checksum()) - .setSizeBytes(size) - .setParsedJson(nullToEmpty(ra.parsedJson())) - .build()); - b.setRawFrameFact(com.lingniu.ingest.sink.kafka.proto.RawFrameFactPayload.newBuilder() - .setFrameId(identity.frameId()) - .setVehicleKey(identity.vehicleKey()) - .setVin(nullToEmpty(ra.vin())) - .setPhone(phone) - .setMessageId(ra.command()) - .setSubType(ra.infoType()) - .setRawUri(uri) - .setChecksum(identity.checksum()) - .setRawSizeBytes(size) - .setParseStatus(parseStatus(ra)) - .setParseError(parseError(ra)) - .setPeer(metadataValue(ra, "peer")) - .putAllMetadata(ra.metadata() == null ? java.util.Map.of() : ra.metadata()) - .build()); - } - } - return b.build(); - } - - private static com.lingniu.ingest.sink.kafka.proto.TelemetrySnapshot buildTelemetrySnapshot(TelemetrySnapshot snapshot) { - Builder b = com.lingniu.ingest.sink.kafka.proto.TelemetrySnapshot.newBuilder() - .setEventType(snapshot.eventType()) - .setRawArchiveUri(snapshot.rawArchiveUri()); - for (TelemetryFieldValue field : snapshot.fields()) { - b.addFields(TelemetryField.newBuilder() - .setKey(field.key()) - .setValueType(field.valueType().name()) - .setValue(field.value()) - .setUnit(field.unit()) - .setQuality(field.quality().name()) - .setSourcePath(field.sourcePath()) - .build()); - } - return b.build(); - } - - private static com.lingniu.ingest.sink.kafka.proto.RealtimePayload buildRealtime(RealtimePayload p) { - var b = com.lingniu.ingest.sink.kafka.proto.RealtimePayload.newBuilder(); - if (p.speedKmh() != null) b.setSpeedKmh(p.speedKmh()); - if (p.totalMileageKm() != null) b.setTotalMileageKm(p.totalMileageKm()); - if (p.batterySoc() != null) b.setBatterySoc(p.batterySoc()); - if (p.batteryVoltageV() != null) b.setBatteryVoltageV(p.batteryVoltageV()); - if (p.batteryCurrentA() != null) b.setBatteryCurrentA(p.batteryCurrentA()); - if (p.fcVoltageV() != null) b.setFcVoltageV(p.fcVoltageV()); - if (p.fcCurrentA() != null) b.setFcCurrentA(p.fcCurrentA()); - if (p.fcTempC() != null) b.setFcTempC(p.fcTempC()); - if (p.hydrogenRemainingKg() != null) b.setHydrogenRemainingKg(p.hydrogenRemainingKg()); - if (p.hydrogenHighPressureMpa() != null) b.setHydrogenHighPressureMpa(p.hydrogenHighPressureMpa()); - if (p.hydrogenLowPressureMpa() != null) b.setHydrogenLowPressureMpa(p.hydrogenLowPressureMpa()); - if (p.vehicleState() != null) b.setVehicleState(p.vehicleState().name()); - if (p.chargingState() != null) b.setChargingState(p.chargingState().name()); - if (p.runningMode() != null) b.setRunningMode(p.runningMode().name()); - if (p.gearLevel() != null) b.setGearLevel(p.gearLevel()); - if (p.acceleratorPedal() != null) b.setAcceleratorPedal(p.acceleratorPedal()); - if (p.brakePedal() != null) b.setBrakePedal(p.brakePedal()); - if (p.longitude() != null) b.setLongitude(p.longitude()); - if (p.latitude() != null) b.setLatitude(p.latitude()); - if (p.altitudeM() != null) b.setAltitudeM(p.altitudeM()); - if (p.directionDeg() != null) b.setDirectionDeg(p.directionDeg()); - if (p.ambientTempC() != null) b.setAmbientTempC(p.ambientTempC()); - if (p.coolantTempC() != null) b.setCoolantTempC(p.coolantTempC()); - return b.build(); - } - - private static com.lingniu.ingest.sink.kafka.proto.LocationPayload buildLocation(LocationPayload p) { - return com.lingniu.ingest.sink.kafka.proto.LocationPayload.newBuilder() - .setLongitude(p.longitude()) - .setLatitude(p.latitude()) - .setAltitudeM(p.altitudeM()) - .setSpeedKmh(p.speedKmh()) - .setDirectionDeg(p.directionDeg()) - .setAlarmFlag(p.alarmFlag()) - .setStatusFlag(p.statusFlag()) - .build(); - } - - private static com.lingniu.ingest.sink.kafka.proto.AlarmPayload buildAlarm(AlarmPayload p) { - var b = com.lingniu.ingest.sink.kafka.proto.AlarmPayload.newBuilder() - .setLevel(p.level().name()) - .setAlarmTypeCode(p.alarmTypeCode()) - .setAlarmTypeName(nullToEmpty(p.alarmTypeName())); - if (p.faultCodes() != null) b.addAllFaultCodes(p.faultCodes()); - if (p.activeBits() != null) b.addAllActiveBits(p.activeBits()); - if (p.longitude() != null) b.setLongitude(p.longitude()); - if (p.latitude() != null) b.setLatitude(p.latitude()); - return b.build(); - } - - private static String nullToEmpty(String s) { - return s == null ? "" : s; - } - - private static String rawArchiveKey(VehicleEvent.RawArchive raw) { - String key = raw.metadata() == null ? "" : raw.metadata().getOrDefault(RawArchiveKeys.META_KEY, ""); - return key == null || key.isBlank() ? RawArchiveKeys.key(raw) : key; - } - - private static String rawArchiveUri(VehicleEvent.RawArchive raw, String key) { - String uri = raw.metadata() == null ? "" : raw.metadata().getOrDefault(RawArchiveKeys.META_URI, ""); - return uri == null || uri.isBlank() ? RawArchiveKeys.logicalUri(key) : uri; - } - - private static String metadataValue(VehicleEvent event, String key) { - if (event.metadata() == null) { - return ""; - } - return nullToEmpty(event.metadata().get(key)); - } - - private static ParseStatusProto parseStatus(VehicleEvent.RawArchive raw) { - if (hasTruthyMetadata(raw, "frameError") - || hasTruthyMetadata(raw, "parseError") - || hasTruthyMetadata(raw, "processingError")) { - return ParseStatusProto.PARSE_STATUS_FAILED; - } - return ParseStatusProto.PARSE_STATUS_NOT_PARSED; - } - - private static String parseError(VehicleEvent.RawArchive raw) { - String frameError = metadataValue(raw, "frameErrorMessage"); - if (!frameError.isBlank()) { - return frameError; - } - String parseError = metadataValue(raw, "parseErrorMessage"); - if (!parseError.isBlank()) { - return parseError; - } - return metadataValue(raw, "processingErrorMessage"); - } - - private static boolean hasTruthyMetadata(VehicleEvent event, String key) { - return Boolean.parseBoolean(metadataValue(event, key)); - } - -} diff --git a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerFactory.java b/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerFactory.java deleted file mode 100644 index ed83e261..00000000 --- a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerFactory.java +++ /dev/null @@ -1,142 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import org.apache.kafka.clients.consumer.ConsumerConfig; -import org.apache.kafka.clients.consumer.KafkaConsumer; -import org.apache.kafka.common.serialization.ByteArrayDeserializer; -import org.apache.kafka.common.serialization.StringDeserializer; - -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Map; -import java.util.Properties; -import java.util.function.Function; - -public final class KafkaEnvelopeConsumerFactory { - - private final Function> consumerFactory; - - public KafkaEnvelopeConsumerFactory() { - this(KafkaConsumer::new); - } - - KafkaEnvelopeConsumerFactory( - Function> consumerFactory) { - this.consumerFactory = consumerFactory; - } - - public List createWorkers(Map processors, - KafkaSinkProperties props) { - Map bindings = effectiveBindings(props); - List workers = new ArrayList<>(); - for (Map.Entry entry : bindings.entrySet()) { - String processorBeanName = entry.getKey(); - // binding 的 key 必须和 Spring Bean 名一致；这样配置只声明 topic/group， - // 实际处理逻辑仍由各业务模块自己的 EnvelopeConsumerProcessor 承接。 - EnvelopeConsumerProcessor processor = processors.get(processorBeanName); - KafkaSinkProperties.Binding binding = entry.getValue(); - if (processor == null || binding == null || !binding.isEnabled()) { - continue; - } - List topics = cleanTopics(binding.getTopics()); - if (topics.isEmpty()) { - continue; - } - int concurrency = Math.max(1, props.getConsumer().getConcurrency()); - for (int workerIndex = 0; workerIndex < concurrency; workerIndex++) { - KafkaEnvelopeConsumerWorker worker = new KafkaEnvelopeConsumerWorker( - consumerFactory.apply(consumerProperties(props, binding, processorBeanName, - workerIndex, concurrency)), - topicProcessors(topics, processor)); - worker.subscribe(topics); - workers.add(worker); - } - } - return workers; - } - - private Map effectiveBindings(KafkaSinkProperties props) { - Map configured = props.getConsumer().getBindings(); - if (configured != null && !configured.isEmpty()) { - return configured; - } - // 默认绑定仅给未显式配置 bindings 的轻量运行时兜底。 - // 生产 history app 会显式绑定各协议 event/raw topic，并写入 TDengine raw_frames/locations。 - KafkaSinkProperties.Topics topics = props.getTopics(); - Map defaults = new LinkedHashMap<>(); - defaults.put("eventHistoryEnvelopeConsumerProcessor", binding( - "vehicle-event-history", - topics.getRealtime(), topics.getLocation(), topics.getAlarm(), topics.getSession(), topics.getMediaMeta())); - defaults.put("vehicleStateEnvelopeConsumerProcessor", binding( - "vehicle-state", - topics.getRealtime(), topics.getLocation(), topics.getAlarm())); - defaults.put("vehicleStatEnvelopeConsumerProcessor", binding( - "vehicle-stat", - topics.getRealtime(), topics.getLocation())); - return defaults; - } - - private KafkaSinkProperties.Binding binding(String groupId, String... topics) { - KafkaSinkProperties.Binding binding = new KafkaSinkProperties.Binding(); - binding.setGroupId(groupId); - binding.setTopics(List.of(topics)); - return binding; - } - - private Map topicProcessors(List topics, EnvelopeConsumerProcessor processor) { - Map byTopic = new LinkedHashMap<>(); - for (String topic : topics) { - byTopic.put(topic, processor); - } - return byTopic; - } - - private List cleanTopics(List topics) { - if (topics == null || topics.isEmpty()) { - return List.of(); - } - LinkedHashSet clean = new LinkedHashSet<>(); - for (String topic : topics) { - if (topic != null && !topic.isBlank()) { - clean.add(topic); - } - } - return List.copyOf(clean); - } - - private Properties consumerProperties(KafkaSinkProperties props, - KafkaSinkProperties.Binding binding, - String processorBeanName, - int workerIndex, - int concurrency) { - KafkaSinkProperties.Consumer consumer = props.getConsumer(); - Properties p = new Properties(); - p.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, props.getBootstrapServers()); - p.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class.getName()); - p.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class.getName()); - p.put(ConsumerConfig.GROUP_ID_CONFIG, groupId(binding, processorBeanName)); - p.put(ConsumerConfig.CLIENT_ID_CONFIG, clientId(consumer.getClientIdPrefix(), processorBeanName, - workerIndex, concurrency)); - // 手动提交 offset：只有本批次至少有一条记录被处理器接收后才 commit， - // 避免轮询到空批次或未绑定 topic 时推进消费位点。 - p.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false); - p.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, consumer.getAutoOffsetReset()); - p.put(ConsumerConfig.MAX_POLL_RECORDS_CONFIG, consumer.getMaxPollRecords()); - p.put(ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG, consumer.getMaxPollIntervalMillis()); - return p; - } - - private String clientId(String prefix, String processorBeanName, int workerIndex, int concurrency) { - String base = prefix + "-" + processorBeanName; - return concurrency <= 1 ? base : base + "-" + workerIndex; - } - - private String groupId(KafkaSinkProperties.Binding binding, String processorBeanName) { - if (binding.getGroupId() != null && !binding.getGroupId().isBlank()) { - return binding.getGroupId(); - } - return processorBeanName.replace("EnvelopeConsumerProcessor", ""); - } -} diff --git a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerRunner.java b/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerRunner.java deleted file mode 100644 index 17e90726..00000000 --- a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerRunner.java +++ /dev/null @@ -1,182 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.context.SmartLifecycle; - -import java.time.Duration; -import java.util.List; -import java.util.concurrent.ExecutorService; -import java.util.concurrent.Executors; -import java.util.concurrent.TimeUnit; -import java.util.concurrent.atomic.AtomicBoolean; -import java.util.function.Supplier; - -public final class KafkaEnvelopeConsumerRunner implements SmartLifecycle, AutoCloseable { - - private static final Logger log = LoggerFactory.getLogger(KafkaEnvelopeConsumerRunner.class); - - private final Supplier> workersSupplier; - private final Duration pollTimeout; - private final Duration loopBackoff; - private final boolean autoStartup; - private final AtomicBoolean running = new AtomicBoolean(false); - private final AtomicBoolean closed = new AtomicBoolean(false); - private volatile List workers; - private ExecutorService executor; - - public KafkaEnvelopeConsumerRunner(List workers, - Duration pollTimeout, - Duration loopBackoff, - boolean autoStartup) { - if (workers == null || workers.isEmpty()) { - throw new IllegalArgumentException("workers must not be empty"); - } - this.workersSupplier = () -> List.copyOf(workers); - this.workers = List.copyOf(workers); - this.pollTimeout = pollTimeout == null ? Duration.ofSeconds(1) : pollTimeout; - this.loopBackoff = loopBackoff == null ? Duration.ofSeconds(1) : loopBackoff; - this.autoStartup = autoStartup; - } - - public KafkaEnvelopeConsumerRunner(Supplier> workersSupplier, - Duration pollTimeout, - Duration loopBackoff, - boolean autoStartup) { - if (workersSupplier == null) { - throw new IllegalArgumentException("workersSupplier must not be null"); - } - this.workersSupplier = workersSupplier; - this.pollTimeout = pollTimeout == null ? Duration.ofSeconds(1) : pollTimeout; - this.loopBackoff = loopBackoff == null ? Duration.ofSeconds(1) : loopBackoff; - this.autoStartup = autoStartup; - } - - public List workers() { - return workersOrCreate(); - } - - @Override - public void start() { - if (!running.compareAndSet(false, true)) { - return; - } - List activeWorkers = workersOrCreate(); - // 每个 worker 一条后台线程，避免某个处理器阻塞时拖慢其他消费组。 - executor = Executors.newFixedThreadPool(activeWorkers.size(), r -> { - Thread thread = new Thread(r, "kafka-envelope-consumer"); - thread.setDaemon(true); - return thread; - }); - for (KafkaEnvelopeConsumerWorker worker : activeWorkers) { - executor.submit(() -> pollLoop(worker)); - } - } - - private List workersOrCreate() { - List current = workers; - if (current != null) { - return current; - } - synchronized (this) { - if (workers == null) { - List created = workersSupplier.get(); - if (created == null || created.isEmpty()) { - throw new IllegalStateException("no kafka envelope consumer workers created; check consumer bindings"); - } - workers = List.copyOf(created); - } - return workers; - } - } - - private void pollLoop(KafkaEnvelopeConsumerWorker worker) { - while (running.get()) { - try { - worker.pollOnce(pollTimeout); - } catch (RuntimeException ex) { - if (!running.get()) { - break; - } - // Kafka/处理器异常不让 Spring 生命周期退出；退避后继续消费， - // 具体坏消息由 EnvelopeConsumerProcessor 写入 DLQ。 - log.warn("Kafka envelope consumer poll failed; the worker will retry after backoff", ex); - sleepBackoff(); - } - } - } - - private void sleepBackoff() { - try { - Thread.sleep(loopBackoff.toMillis()); - } catch (InterruptedException ex) { - Thread.currentThread().interrupt(); - } - } - - @Override - public void stop() { - if (!running.compareAndSet(true, false)) { - return; - } - wakeupWorkers(); - if (executor != null) { - executor.shutdown(); - try { - if (!executor.awaitTermination(5, TimeUnit.SECONDS)) { - executor.shutdownNow(); - executor.awaitTermination(5, TimeUnit.SECONDS); - } - } catch (InterruptedException ex) { - executor.shutdownNow(); - Thread.currentThread().interrupt(); - } - } - closeWorkers(); - } - - @Override - public void stop(Runnable callback) { - stop(); - callback.run(); - } - - @Override - public boolean isRunning() { - return running.get(); - } - - @Override - public boolean isAutoStartup() { - return autoStartup; - } - - @Override - public void close() { - stop(); - closeWorkers(); - } - - private void closeWorkers() { - if (!closed.compareAndSet(false, true)) { - return; - } - List current = workers; - if (current == null) { - return; - } - for (KafkaEnvelopeConsumerWorker worker : current) { - worker.close(); - } - } - - private void wakeupWorkers() { - List current = workers; - if (current == null) { - return; - } - for (KafkaEnvelopeConsumerWorker worker : current) { - worker.wakeup(); - } - } -} diff --git a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerWorker.java b/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerWorker.java deleted file mode 100644 index b1f9e9e3..00000000 --- a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerWorker.java +++ /dev/null @@ -1,75 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.api.consumer.EnvelopeConsumerRecord; -import org.apache.kafka.clients.consumer.Consumer; -import org.apache.kafka.clients.consumer.ConsumerRecord; -import org.apache.kafka.clients.consumer.ConsumerRecords; - -import java.time.Duration; -import java.util.ArrayList; -import java.util.Collection; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; - -public final class KafkaEnvelopeConsumerWorker implements AutoCloseable { - - private final Consumer consumer; - private final Map processorsByTopic; - - public KafkaEnvelopeConsumerWorker(Consumer consumer, - Map processorsByTopic) { - if (consumer == null) { - throw new IllegalArgumentException("consumer must not be null"); - } - if (processorsByTopic == null || processorsByTopic.isEmpty()) { - throw new IllegalArgumentException("processorsByTopic must not be empty"); - } - this.consumer = consumer; - this.processorsByTopic = Map.copyOf(processorsByTopic); - } - - public void subscribe(Collection topics) { - consumer.subscribe(topics); - } - - public int pollOnce(Duration timeout) { - ConsumerRecords records = consumer.poll(timeout == null ? Duration.ZERO : timeout); - Map> byProcessor = new LinkedHashMap<>(); - int processed = 0; - for (ConsumerRecord record : records) { - EnvelopeConsumerProcessor processor = processorsByTopic.get(record.topic()); - if (processor == null) { - // worker 可能订阅多个 topic；没有显式绑定处理器的 topic 不参与提交语义。 - continue; - } - byProcessor.computeIfAbsent(processor, ignored -> new ArrayList<>()).add(new EnvelopeConsumerRecord( - record.topic(), - record.partition(), - record.offset(), - record.key(), - record.value())); - processed++; - } - for (Map.Entry> entry : byProcessor.entrySet()) { - // EnvelopeConsumerProcessor 内部会把解析或业务错误转成 DLQ 记录， - // 这里保持 Kafka worker 的职责单一：轮询、分发、成功后提交 offset。 - entry.getKey().processBatch(entry.getValue()); - } - if (processed > 0) { - // commitSync 放在批次末尾，保证同一个 poll 批次内的消息按 Kafka offset 一起确认。 - consumer.commitSync(); - } - return processed; - } - - public void wakeup() { - consumer.wakeup(); - } - - @Override - public void close() { - consumer.close(); - } -} diff --git a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeDeadLetterSink.java b/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeDeadLetterSink.java deleted file mode 100644 index 4ba2bf2c..00000000 --- a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeDeadLetterSink.java +++ /dev/null @@ -1,53 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.consumer.EnvelopeDeadLetterRecord; -import com.lingniu.ingest.api.consumer.EnvelopeDeadLetterSink; -import org.apache.kafka.clients.producer.KafkaProducer; -import org.apache.kafka.clients.producer.Producer; -import org.apache.kafka.clients.producer.ProducerRecord; - -import java.nio.charset.StandardCharsets; - -public final class KafkaEnvelopeDeadLetterSink implements EnvelopeDeadLetterSink { - - private final Producer producer; - private final String topic; - - public KafkaEnvelopeDeadLetterSink(KafkaProducer producer, String topic) { - this((Producer) producer, topic); - } - - KafkaEnvelopeDeadLetterSink(Producer producer, String topic) { - if (producer == null) { - throw new IllegalArgumentException("producer must not be null"); - } - if (topic == null || topic.isBlank()) { - throw new IllegalArgumentException("topic must not be blank"); - } - this.producer = producer; - this.topic = topic; - } - - @Override - public void publish(EnvelopeDeadLetterRecord record) { - if (record == null) { - throw new IllegalArgumentException("record must not be null"); - } - ProducerRecord out = new ProducerRecord<>(topic, record.key(), record.payload()); - // DLQ payload 保持原始 Kafka value；定位信息全部放 header，便于后续重放或人工排查。 - header(out, "dlq-service", record.service()); - header(out, "dlq-source-topic", record.topic()); - header(out, "dlq-source-partition", Integer.toString(record.partition())); - header(out, "dlq-source-offset", Long.toString(record.offset())); - header(out, "dlq-status", record.status().name()); - header(out, "dlq-event-id", record.eventId()); - header(out, "dlq-vin", record.vin()); - header(out, "dlq-message", record.message()); - header(out, "dlq-created-at", record.createdAt().toString()); - producer.send(out); - } - - private static void header(ProducerRecord record, String key, String value) { - record.headers().add(key, value.getBytes(StandardCharsets.UTF_8)); - } -} diff --git a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEventSink.java b/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEventSink.java deleted file mode 100644 index cf879fce..00000000 --- a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaEventSink.java +++ /dev/null @@ -1,118 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.api.sink.EventSink; -import io.github.resilience4j.circuitbreaker.CircuitBreaker; -import org.apache.kafka.clients.producer.KafkaProducer; -import org.apache.kafka.clients.producer.ProducerRecord; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import java.util.concurrent.CompletableFuture; - -/** - * Kafka Sink：序列化为 Protobuf Envelope，按 vin 分区，异步发送。 - * - *

    失败处理：Resilience4j 熔断；熔断开启时事件转投 DLQ topic。 - */ -public final class KafkaEventSink implements EventSink, AutoCloseable { - - private static final Logger log = LoggerFactory.getLogger(KafkaEventSink.class); - - private final KafkaProducer producer; - private final EnvelopeMapper mapper; - private final TopicRouter router; - private final String dlqTopic; - private final CircuitBreaker breaker; - - public KafkaEventSink(KafkaProducer producer, - EnvelopeMapper mapper, - TopicRouter router, - String dlqTopic, - CircuitBreaker breaker) { - this.producer = producer; - this.mapper = mapper; - this.router = router; - this.dlqTopic = dlqTopic; - this.breaker = breaker; - } - - @Override - public String name() { - return "kafka"; - } - - /** - * Split-service mode uses Kafka for both normalized events and raw archive records. - */ - @Override - public boolean accepts(VehicleEvent event) { - return true; - } - - @Override - public CompletableFuture publish(VehicleEvent event) { - CompletableFuture cf = new CompletableFuture<>(); - byte[] payload; - try { - payload = mapper.toEnvelope(event).toByteArray(); - } catch (Exception e) { - log.error("envelope build failed eventId={} vin={}", event.eventId(), event.vin(), e); - cf.completeExceptionally(e); - return cf; - } - - String topic = breaker.tryAcquirePermission() ? router.route(event) : dlqTopic; - ProducerRecord record = new ProducerRecord<>(topic, partitionKey(event), payload); - record.headers().add("event-id", event.eventId().getBytes()); - record.headers().add("trace-id", event.traceId() == null ? new byte[0] : event.traceId().getBytes()); - record.headers().add("source", event.source().name().getBytes()); - - producer.send(record, (metadata, ex) -> { - if (ex != null) { - breaker.onError(0, java.util.concurrent.TimeUnit.MILLISECONDS, ex); - cf.completeExceptionally(ex); - } else { - breaker.onSuccess(0, java.util.concurrent.TimeUnit.MILLISECONDS); - cf.complete(null); - } - }); - return cf; - } - - private static String partitionKey(VehicleEvent event) { - String vin = event.vin(); - if (isKnown(vin)) { - return vin; - } - String vehicleKey = metadata(event, "vehicleKey"); - if (!isKnown(vehicleKey)) { - vehicleKey = metadata(event, "vehicle_key"); - } - if (isKnown(vehicleKey)) { - return "vehicleKey:" + vehicleKey; - } - String phone = metadata(event, "phone"); - if (isKnown(phone)) { - return "phone:" + phone; - } - return vin == null || vin.isBlank() ? "unknown" : vin; - } - - private static String metadata(VehicleEvent event, String key) { - if (event.metadata() == null) { - return ""; - } - return event.metadata().getOrDefault(key, "").trim(); - } - - private static boolean isKnown(String value) { - return value != null && !value.isBlank() && !"unknown".equalsIgnoreCase(value.trim()); - } - - @Override - public void close() { - producer.flush(); - producer.close(); - } -} diff --git a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkAutoConfiguration.java b/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkAutoConfiguration.java deleted file mode 100644 index 82a1dd91..00000000 --- a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkAutoConfiguration.java +++ /dev/null @@ -1,86 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import io.github.resilience4j.circuitbreaker.CircuitBreaker; -import io.github.resilience4j.circuitbreaker.CircuitBreakerConfig; -import org.apache.kafka.clients.producer.KafkaProducer; -import org.apache.kafka.clients.producer.ProducerConfig; -import org.apache.kafka.common.serialization.ByteArraySerializer; -import org.apache.kafka.common.serialization.StringSerializer; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; - -import java.time.Duration; -import java.util.Properties; - -/** - * Kafka Sink 自动装配。 - * - *

    {@code lingniu.ingest.sink.kafka.enabled=false} 时本模块完全不装配；Producer 和 Consumer - * 是两个独立开关，消费端还需要开启 {@code lingniu.ingest.sink.kafka.consumer.enabled=true} - * 并配置 bindings。 - */ -@AutoConfiguration -@EnableConfigurationProperties(KafkaSinkProperties.class) -@ConditionalOnProperty(prefix = "lingniu.ingest.sink.kafka", name = "enabled", havingValue = "true", matchIfMissing = true) -public class KafkaSinkAutoConfiguration { - - @Bean - @ConditionalOnMissingBean - public EnvelopeMapper envelopeMapper(KafkaSinkProperties props) { - return new EnvelopeMapper(props.getNodeId()); - } - - @Bean - @ConditionalOnMissingBean - public TopicRouter topicRouter(KafkaSinkProperties props) { - return new TopicRouter(props.getTopics()); - } - - @Bean - @ConditionalOnMissingBean - public CircuitBreaker kafkaSinkCircuitBreaker() { - return CircuitBreaker.of("kafka-sink", CircuitBreakerConfig.custom() - .slidingWindowSize(100) - .failureRateThreshold(50) - .waitDurationInOpenState(Duration.ofSeconds(10)) - .permittedNumberOfCallsInHalfOpenState(10) - .build()); - } - - @Bean(destroyMethod = "close") - @ConditionalOnMissingBean - public KafkaProducer kafkaProducer(KafkaSinkProperties props) { - Properties p = new Properties(); - p.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, props.getBootstrapServers()); - p.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName()); - p.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class.getName()); - p.put(ProducerConfig.COMPRESSION_TYPE_CONFIG, props.getCompressionType()); - p.put(ProducerConfig.LINGER_MS_CONFIG, props.getLingerMs()); - p.put(ProducerConfig.BATCH_SIZE_CONFIG, props.getBatchSize()); - p.put(ProducerConfig.ACKS_CONFIG, props.getAcks()); - p.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, props.isEnableIdempotence()); - return new KafkaProducer<>(p); - } - - @Bean(destroyMethod = "close") - @ConditionalOnMissingBean - public KafkaEventSink kafkaEventSink(KafkaProducer producer, - EnvelopeMapper mapper, - TopicRouter router, - KafkaSinkProperties props, - CircuitBreaker breaker) { - // KafkaEventSink 是 EventBus 的生产端出口；它不会启动任何 Kafka 消费线程。 - return new KafkaEventSink(producer, mapper, router, props.getTopics().getDlq(), breaker); - } - - @Bean - @ConditionalOnMissingBean - public KafkaEnvelopeDeadLetterSink kafkaEnvelopeDeadLetterSink(KafkaProducer producer, - KafkaSinkProperties props) { - return new KafkaEnvelopeDeadLetterSink(producer, props.deadLetterTopic()); - } - -} diff --git a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkConsumerAutoConfiguration.java b/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkConsumerAutoConfiguration.java deleted file mode 100644 index 602dc8ed..00000000 --- a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkConsumerAutoConfiguration.java +++ /dev/null @@ -1,51 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import org.springframework.beans.factory.ListableBeanFactory; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.AutoConfigureAfter; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; - -import java.time.Duration; -import java.util.List; -import java.util.Map; - -@AutoConfiguration(after = KafkaSinkAutoConfiguration.class) -@AutoConfigureAfter(name = { - "com.lingniu.ingest.eventhistory.config.EventHistoryAutoConfiguration", - "com.lingniu.ingest.vehiclestate.config.VehicleStateAutoConfiguration", - "com.lingniu.ingest.vehiclestat.config.VehicleStatAutoConfiguration" -}) -@EnableConfigurationProperties(KafkaSinkProperties.class) -@ConditionalOnProperty(prefix = "lingniu.ingest.sink.kafka", name = "enabled", havingValue = "true", matchIfMissing = true) -public class KafkaSinkConsumerAutoConfiguration { - - @Bean - @ConditionalOnMissingBean - public KafkaEnvelopeConsumerFactory kafkaEnvelopeConsumerFactory() { - return new KafkaEnvelopeConsumerFactory(); - } - - @Bean - @ConditionalOnMissingBean - @ConditionalOnProperty(prefix = "lingniu.ingest.sink.kafka.consumer", name = "enabled", havingValue = "true") - public KafkaEnvelopeConsumerRunner kafkaEnvelopeConsumerRunner(ListableBeanFactory beanFactory, - KafkaEnvelopeConsumerFactory consumerFactory, - KafkaSinkProperties props) { - return new KafkaEnvelopeConsumerRunner( - () -> createWorkers(beanFactory, consumerFactory, props), - Duration.ofMillis(props.getConsumer().getPollTimeoutMillis()), - Duration.ofMillis(props.getConsumer().getLoopBackoffMillis()), - props.getConsumer().isAutoStartup()); - } - - private List createWorkers(ListableBeanFactory beanFactory, - KafkaEnvelopeConsumerFactory consumerFactory, - KafkaSinkProperties props) { - Map processors = beanFactory.getBeansOfType(EnvelopeConsumerProcessor.class); - return consumerFactory.createWorkers(processors, props); - } -} diff --git a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkProperties.java b/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkProperties.java deleted file mode 100644 index 69d55521..00000000 --- a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkProperties.java +++ /dev/null @@ -1,151 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -import java.util.ArrayList; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; - -@ConfigurationProperties(prefix = "lingniu.ingest.sink.kafka") -public class KafkaSinkProperties { - - /** - * Kafka Sink 总开关。默认 {@code true}。 - * 设为 {@code false} 时 {@link KafkaSinkAutoConfiguration} 不装配 Kafka Producer、 - * EnvelopeMapper、TopicRouter、KafkaEventSink 等生产端组件。生产接入应用保持默认开启， - * 只有专门的查询/测试运行时才应关闭 Kafka sink。 - */ - private boolean enabled = true; - - /** Kafka bootstrap servers；生产环境应通过环境变量覆盖，不建议使用默认开发地址。 */ - private String bootstrapServers = "114.55.58.251:9092"; - private String compressionType = "zstd"; - private int lingerMs = 20; - private int batchSize = 65536; - private String acks = "all"; - private boolean enableIdempotence = true; - /** 写入 envelope 的节点标识，进入 Protobuf 字段 ingest_node_id，便于追踪多实例来源。 */ - private String nodeId = "ingest-local"; - private Topics topics = new Topics(); - private Consumer consumer = new Consumer(); - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public String getBootstrapServers() { return bootstrapServers; } - public void setBootstrapServers(String bootstrapServers) { this.bootstrapServers = bootstrapServers; } - public String getCompressionType() { return compressionType; } - public void setCompressionType(String compressionType) { this.compressionType = compressionType; } - public int getLingerMs() { return lingerMs; } - public void setLingerMs(int lingerMs) { this.lingerMs = lingerMs; } - public int getBatchSize() { return batchSize; } - public void setBatchSize(int batchSize) { this.batchSize = batchSize; } - public String getAcks() { return acks; } - public void setAcks(String acks) { this.acks = acks; } - public boolean isEnableIdempotence() { return enableIdempotence; } - public void setEnableIdempotence(boolean enableIdempotence) { this.enableIdempotence = enableIdempotence; } - public String getNodeId() { return nodeId; } - public void setNodeId(String nodeId) { this.nodeId = nodeId; } - public Topics getTopics() { return topics; } - public void setTopics(Topics topics) { this.topics = topics; } - public Consumer getConsumer() { return consumer; } - public void setConsumer(Consumer consumer) { this.consumer = consumer; } - - public String deadLetterTopic() { - String consumerDlq = consumer == null ? "" : consumer.getDlqTopic(); - if (consumerDlq != null && !consumerDlq.isBlank()) { - return consumerDlq; - } - return topics.getDlq(); - } - - public static class Topics { - /** 实时遥测事件 topic；GB32960 RAW-only 架构下可逐步弱化该 topic。 */ - private String realtime = "vehicle.event.gb32960.v1"; - /** 位置事件 topic；通常可由 realtime/RAW 派生。 */ - private String location = "vehicle.event.gb32960.v1"; - private String alarm = "vehicle.event.gb32960.v1"; - private String session = "vehicle.event.gb32960.v1"; - private String mediaMeta = "vehicle.media.meta.v1"; - /** RAW 归档引用 topic，payload 应携带 archive URI/size，不建议携带完整 raw bytes。 */ - private String rawArchive = "vehicle.raw.gb32960.v1"; - /** producer 熔断或 consumer 处理失败时的死信 topic。 */ - private String dlq = "vehicle.dlq.gb32960.v1"; - - public String getRealtime() { return realtime; } - public void setRealtime(String realtime) { this.realtime = realtime; } - public String getLocation() { return location; } - public void setLocation(String location) { this.location = location; } - public String getAlarm() { return alarm; } - public void setAlarm(String alarm) { this.alarm = alarm; } - public String getSession() { return session; } - public void setSession(String session) { this.session = session; } - public String getMediaMeta() { return mediaMeta; } - public void setMediaMeta(String mediaMeta) { this.mediaMeta = mediaMeta; } - public String getRawArchive() { return rawArchive; } - public void setRawArchive(String rawArchive) { this.rawArchive = rawArchive; } - public String getDlq() { return dlq; } - public void setDlq(String dlq) { this.dlq = dlq; } - } - - public static class Consumer { - /** Kafka 消费总开关。与 producer 总开关分离，默认 false，避免单体服务意外自消费。 */ - private boolean enabled = false; - private boolean autoStartup = true; - private String clientIdPrefix = "lingniu-envelope-consumer"; - private int pollTimeoutMillis = 1000; - private int loopBackoffMillis = 1000; - private String autoOffsetReset = "earliest"; - private int maxPollRecords = 500; - private int maxPollIntervalMillis = 1800000; - private int concurrency = 1; - /** 消费处理失败时的死信 topic；未配置时兼容使用 producer topics.dlq。 */ - private String dlqTopic = ""; - /** - * Processor bean name -> Kafka binding。 - * - *

    示例 key：{@code eventHistoryEnvelopeConsumerProcessor}、 - * {@code vehicleStateEnvelopeConsumerProcessor}、{@code vehicleStatEnvelopeConsumerProcessor}。 - */ - private Map bindings = new LinkedHashMap<>(); - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public boolean isAutoStartup() { return autoStartup; } - public void setAutoStartup(boolean autoStartup) { this.autoStartup = autoStartup; } - public String getClientIdPrefix() { return clientIdPrefix; } - public void setClientIdPrefix(String clientIdPrefix) { this.clientIdPrefix = clientIdPrefix; } - public int getPollTimeoutMillis() { return pollTimeoutMillis; } - public void setPollTimeoutMillis(int pollTimeoutMillis) { this.pollTimeoutMillis = pollTimeoutMillis; } - public int getLoopBackoffMillis() { return loopBackoffMillis; } - public void setLoopBackoffMillis(int loopBackoffMillis) { this.loopBackoffMillis = loopBackoffMillis; } - public String getAutoOffsetReset() { return autoOffsetReset; } - public void setAutoOffsetReset(String autoOffsetReset) { this.autoOffsetReset = autoOffsetReset; } - public int getMaxPollRecords() { return maxPollRecords; } - public void setMaxPollRecords(int maxPollRecords) { this.maxPollRecords = maxPollRecords; } - public int getMaxPollIntervalMillis() { return maxPollIntervalMillis; } - public void setMaxPollIntervalMillis(int maxPollIntervalMillis) { this.maxPollIntervalMillis = maxPollIntervalMillis; } - public int getConcurrency() { return concurrency; } - public void setConcurrency(int concurrency) { this.concurrency = concurrency; } - public String getDlqTopic() { return dlqTopic; } - public void setDlqTopic(String dlqTopic) { this.dlqTopic = dlqTopic; } - public Map getBindings() { return bindings; } - public void setBindings(Map bindings) { this.bindings = bindings; } - } - - public static class Binding { - /** 单个 processor binding 开关，用于临时停某个下游消费者而不关整个 consumer runner。 */ - private boolean enabled = true; - /** Kafka consumer group id。不同服务要独立消费同一 topic 时必须使用不同 group。 */ - private String groupId; - /** 该 processor 订阅的 topic 列表。 */ - private List topics = new ArrayList<>(); - - public boolean isEnabled() { return enabled; } - public void setEnabled(boolean enabled) { this.enabled = enabled; } - public String getGroupId() { return groupId; } - public void setGroupId(String groupId) { this.groupId = groupId; } - public List getTopics() { return topics; } - public void setTopics(List topics) { this.topics = topics; } - } -} diff --git a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/TopicRouter.java b/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/TopicRouter.java deleted file mode 100644 index f8c984c0..00000000 --- a/modules/sinks/sink-kafka/src/main/java/com/lingniu/ingest/sink/kafka/TopicRouter.java +++ /dev/null @@ -1,29 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.event.VehicleEvent; - -/** - * 事件 → Kafka topic 路由。集中管理避免散落。 - */ -public final class TopicRouter { - - private final KafkaSinkProperties.Topics topics; - - public TopicRouter(KafkaSinkProperties.Topics topics) { - this.topics = topics; - } - - public String route(VehicleEvent event) { - return switch (event) { - case VehicleEvent.Realtime _ -> topics.getRealtime(); - case VehicleEvent.Location _ -> topics.getLocation(); - case VehicleEvent.Alarm _ -> topics.getAlarm(); - case VehicleEvent.Login _, - VehicleEvent.Logout _, - VehicleEvent.Heartbeat _ -> topics.getSession(); - case VehicleEvent.MediaMeta _ -> topics.getMediaMeta(); - case VehicleEvent.Passthrough _ -> topics.getAlarm(); - case VehicleEvent.RawArchive _ -> topics.getRawArchive(); - }; - } -} diff --git a/modules/sinks/sink-kafka/src/main/proto/vehicle_envelope.proto b/modules/sinks/sink-kafka/src/main/proto/vehicle_envelope.proto deleted file mode 100644 index 57acbda1..00000000 --- a/modules/sinks/sink-kafka/src/main/proto/vehicle_envelope.proto +++ /dev/null @@ -1,168 +0,0 @@ -syntax = "proto3"; - -package com.lingniu.ingest.sink.kafka.proto; - -option java_multiple_files = true; -option java_package = "com.lingniu.ingest.sink.kafka.proto"; -option java_outer_classname = "VehicleEnvelopeProto"; - -// 统一消息外壳：所有 Topic 共用此 Envelope，payload 通过 oneof 区分具体事件类型。 -message VehicleEnvelope { - string schema_version = 1; - string event_id = 2; - string trace_id = 3; - string vin = 4; - string source = 5; // ProtocolId 字符串形式 - string protocol_version = 6; - int64 event_time_ms = 7; - int64 ingest_time_ms = 8; - string ingest_node_id = 9; - map metadata = 10; - - oneof payload { - RealtimePayload realtime = 20; - LocationPayload location = 21; - AlarmPayload alarm = 22; - LoginPayload login = 23; - LogoutPayload logout = 24; - HeartbeatPayload heartbeat = 25; - MediaMetaPayload media_meta = 26; - PassthroughPayload passthrough = 27; - } - - // 可选：原始字节指针 - RawArchiveRef raw_archive = 40; - - // 全字段内部遥测快照。新下游消费者优先读取这里，避免依赖协议字段名。 - TelemetrySnapshot telemetry_snapshot = 50; - - // 新一代事实模型：RAW 帧索引与解析事实，均不携带完整 raw bytes。 - RawFrameFactPayload raw_frame_fact = 60; - DecodedFactPayload decoded_fact = 61; -} - -message RawArchiveRef { - string uri = 1; - string checksum = 2; - int64 size_bytes = 3; - string parsed_json = 4; -} - -message TelemetrySnapshot { - string event_type = 1; - string raw_archive_uri = 2; - repeated TelemetryField fields = 3; -} - -message TelemetryField { - string key = 1; - string value_type = 2; - string value = 3; - string unit = 4; - string quality = 5; - string source_path = 6; -} - -enum ParseStatusProto { - PARSE_STATUS_UNSPECIFIED = 0; - PARSE_STATUS_NOT_PARSED = 1; - PARSE_STATUS_SUCCEEDED = 2; - PARSE_STATUS_FAILED = 3; -} - -message RawFrameFactPayload { - string frame_id = 1; - string vehicle_key = 2; - string vin = 3; - string phone = 4; - int32 message_id = 5; - int32 sub_type = 6; - string raw_uri = 7; - string checksum = 8; - int64 raw_size_bytes = 9; - ParseStatusProto parse_status = 10; - string parse_error = 11; - string peer = 12; - map metadata = 13; -} - -message DecodedFactPayload { - string fact_id = 1; - string frame_id = 2; - string fact_type = 3; - string vehicle_key = 4; - string vin = 5; - string phone = 6; - string raw_uri = 7; - map fields = 8; - map metadata = 9; -} - -message RealtimePayload { - optional double speed_kmh = 1; - optional double total_mileage_km = 2; - optional double battery_soc = 3; - optional double battery_voltage_v = 4; - optional double battery_current_a = 5; - optional double fc_voltage_v = 6; - optional double fc_current_a = 7; - optional double fc_temp_c = 8; - optional double hydrogen_remaining_kg = 9; - optional double hydrogen_high_pressure_mpa = 10; - optional double hydrogen_low_pressure_mpa = 11; - optional string vehicle_state = 12; - optional string charging_state = 13; - optional string running_mode = 14; - optional int32 gear_level = 15; - optional double accelerator_pedal = 16; - optional double brake_pedal = 17; - optional double longitude = 18; - optional double latitude = 19; - optional double altitude_m = 20; - optional double direction_deg = 21; - optional double ambient_temp_c = 22; - optional double coolant_temp_c = 23; -} - -message LocationPayload { - double longitude = 1; - double latitude = 2; - double altitude_m = 3; - double speed_kmh = 4; - double direction_deg = 5; - int64 alarm_flag = 6; - int64 status_flag = 7; -} - -message AlarmPayload { - string level = 1; - int32 alarm_type_code = 2; - string alarm_type_name = 3; - repeated string fault_codes = 4; - optional double longitude = 5; - optional double latitude = 6; - // 通用报警标志位（2016 版 0~15 / 2025 版 0~27），对照 GB/T 32960.3 表 24。 - // 例：SOC_LOW / BATTERY_HIGH_TEMP / HYDROGEN_LEAK - repeated string active_bits = 7; -} - -message LoginPayload { - string iccid = 1; - string protocol_version = 2; -} - -message LogoutPayload {} - -message HeartbeatPayload {} - -message MediaMetaPayload { - string media_id = 1; - string media_type = 2; - int64 size_bytes = 3; - string archive_ref = 4; -} - -message PassthroughPayload { - int32 passthrough_type = 1; - bytes data = 2; -} diff --git a/modules/sinks/sink-kafka/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/sinks/sink-kafka/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index 8c283ae7..00000000 --- a/modules/sinks/sink-kafka/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1,2 +0,0 @@ -com.lingniu.ingest.sink.kafka.KafkaSinkAutoConfiguration -com.lingniu.ingest.sink.kafka.KafkaSinkConsumerAutoConfiguration diff --git a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/EnvelopeMapperTelemetrySnapshotTest.java b/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/EnvelopeMapperTelemetrySnapshotTest.java deleted file mode 100644 index 5f700f70..00000000 --- a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/EnvelopeMapperTelemetrySnapshotTest.java +++ /dev/null @@ -1,102 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.RawArchiveKeys; -import com.lingniu.ingest.api.event.RealtimePayload; -import com.lingniu.ingest.api.event.VehicleEvent; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class EnvelopeMapperTelemetrySnapshotTest { - - @Test - void mapsVehicleEventToFullFieldTelemetrySnapshot() { - VehicleEvent.Realtime event = new VehicleEvent.Realtime( - "event-1", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "trace-1", - Map.of("protocolVersion", "V2016", "rawArchiveUri", "archive://raw/event-1.bin"), - new RealtimePayload( - 80.5, - 12345.6, - 55.0, - 610.0, - -20.5, - null, - null, - null, - 8.2, - 35.0, - null, - RealtimePayload.VehicleState.STARTED, - RealtimePayload.ChargingState.UNCHARGED, - RealtimePayload.RunningMode.FUEL, - 3, - null, - null, - 113.12, - 23.45, - null, - null, - null, - null - ) - ); - - VehicleEnvelope envelope = new EnvelopeMapper("node-1").toEnvelope(event); - - assertThat(envelope.getTelemetrySnapshot().getEventType()).isEqualTo("REALTIME"); - assertThat(envelope.getTelemetrySnapshot().getRawArchiveUri()).isEqualTo("archive://raw/event-1.bin"); - assertThat(envelope.getTelemetrySnapshot().getFieldsList()) - .anySatisfy(field -> { - assertThat(field.getKey()).isEqualTo("total_mileage_km"); - assertThat(field.getValueType()).isEqualTo("DOUBLE"); - assertThat(field.getValue()).isEqualTo("12345.6"); - assertThat(field.getUnit()).isEqualTo("km"); - }) - .anySatisfy(field -> { - assertThat(field.getKey()).isEqualTo("hydrogen_high_pressure_mpa"); - assertThat(field.getValue()).isEqualTo("35.0"); - assertThat(field.getUnit()).isEqualTo("MPa"); - }); - } - - @Test - void rawArchiveEnvelopeCarriesLogicalArchiveUri() { - VehicleEvent.RawArchive raw = new VehicleEvent.RawArchive( - "raw-808-1", - "VIN-JT808-001", - ProtocolId.JT808, - Instant.parse("2026-06-22T08:00:00Z"), - Instant.parse("2026-06-22T08:00:01Z"), - "trace-raw-1", - Map.of(), - 0x0200, - 0, - new byte[]{0x7e, 0x02, 0x00, 0x7e}); - - VehicleEnvelope envelope = new EnvelopeMapper("node-1").toEnvelope(raw); - - String key = RawArchiveKeys.key(raw); - assertThat(envelope.hasRawArchive()).isTrue(); - assertThat(envelope.getRawArchive().getUri()).isEqualTo(RawArchiveKeys.logicalUri(key)); - assertThat(envelope.getRawArchive().getSizeBytes()).isEqualTo(4); - assertThat(envelope.getMetadataMap()).containsEntry(RawArchiveKeys.META_URI, RawArchiveKeys.logicalUri(key)); - assertThat(envelope.hasRawFrameFact()).isTrue(); - assertThat(envelope.getRawFrameFact().getFrameId()).startsWith("rf_"); - assertThat(envelope.getRawFrameFact().getVehicleKey()).isEqualTo("VIN-JT808-001"); - assertThat(envelope.getRawFrameFact().getVin()).isEqualTo("VIN-JT808-001"); - assertThat(envelope.getRawFrameFact().getMessageId()).isEqualTo(0x0200); - assertThat(envelope.getRawFrameFact().getRawUri()).isEqualTo(RawArchiveKeys.logicalUri(key)); - assertThat(envelope.getRawFrameFact().getRawSizeBytes()).isEqualTo(4); - assertThat(envelope.getRawFrameFact().getChecksum()).startsWith("sha256:"); - } -} diff --git a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerFactoryTest.java b/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerFactoryTest.java deleted file mode 100644 index df0b9ef0..00000000 --- a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerFactoryTest.java +++ /dev/null @@ -1,79 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.api.consumer.EnvelopeIngestResult; -import org.apache.kafka.clients.consumer.ConsumerConfig; -import org.apache.kafka.clients.consumer.MockConsumer; -import org.apache.kafka.clients.consumer.OffsetResetStrategy; -import org.junit.jupiter.api.Test; - -import java.util.ArrayList; -import java.util.List; -import java.util.Map; -import java.util.Properties; - -import static org.assertj.core.api.Assertions.assertThat; - -class KafkaEnvelopeConsumerFactoryTest { - - @Test - void createsDefaultIndependentConsumersForKnownServiceProcessors() { - List created = new ArrayList<>(); - KafkaEnvelopeConsumerFactory factory = new KafkaEnvelopeConsumerFactory(props -> { - created.add(props); - return new MockConsumer<>(OffsetResetStrategy.EARLIEST); - }); - KafkaSinkProperties props = new KafkaSinkProperties(); - props.setBootstrapServers("kafka-1:9092"); - EnvelopeConsumerProcessor stateProcessor = processor(); - EnvelopeConsumerProcessor statProcessor = processor(); - - List workers = factory.createWorkers(Map.of( - "vehicleStateEnvelopeConsumerProcessor", stateProcessor, - "vehicleStatEnvelopeConsumerProcessor", statProcessor), props); - - assertThat(workers).hasSize(2); - assertThat(created) - .extracting(p -> p.getProperty(ConsumerConfig.GROUP_ID_CONFIG)) - .containsExactly("vehicle-state", "vehicle-stat"); - assertThat(created) - .allSatisfy(p -> { - assertThat(p.getProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG)).isEqualTo("kafka-1:9092"); - assertThat(p.get(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG)).isEqualTo(false); - assertThat(p.get(ConsumerConfig.MAX_POLL_INTERVAL_MS_CONFIG)).isEqualTo(1800000); - }); - } - - @Test - void createsParallelConsumersForEachBindingWhenConsumerConcurrencyIsConfigured() { - List created = new ArrayList<>(); - KafkaEnvelopeConsumerFactory factory = new KafkaEnvelopeConsumerFactory(props -> { - created.add(props); - return new MockConsumer<>(OffsetResetStrategy.EARLIEST); - }); - KafkaSinkProperties props = new KafkaSinkProperties(); - props.getConsumer().setConcurrency(3); - EnvelopeConsumerProcessor stateProcessor = processor(); - - List workers = factory.createWorkers(Map.of( - "vehicleStateEnvelopeConsumerProcessor", stateProcessor), props); - - assertThat(workers).hasSize(3); - assertThat(created) - .extracting(p -> p.getProperty(ConsumerConfig.GROUP_ID_CONFIG)) - .containsExactly("vehicle-state", "vehicle-state", "vehicle-state"); - assertThat(created) - .extracting(p -> p.getProperty(ConsumerConfig.CLIENT_ID_CONFIG)) - .containsExactly( - "lingniu-envelope-consumer-vehicleStateEnvelopeConsumerProcessor-0", - "lingniu-envelope-consumer-vehicleStateEnvelopeConsumerProcessor-1", - "lingniu-envelope-consumer-vehicleStateEnvelopeConsumerProcessor-2"); - } - - private EnvelopeConsumerProcessor processor() { - return new EnvelopeConsumerProcessor( - "service", - bytes -> EnvelopeIngestResult.processed("evt-1", "VIN001"), - record -> {}); - } -} diff --git a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerRunnerTest.java b/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerRunnerTest.java deleted file mode 100644 index 0a4d7f15..00000000 --- a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerRunnerTest.java +++ /dev/null @@ -1,53 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import org.junit.jupiter.api.Test; - -import java.time.Duration; -import java.util.List; -import java.util.concurrent.CountDownLatch; -import java.util.concurrent.TimeUnit; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.ArgumentMatchers.any; -import static org.mockito.Mockito.doAnswer; -import static org.mockito.Mockito.mock; - -class KafkaEnvelopeConsumerRunnerTest { - - @Test - void stopWakesPollLoopAndClosesWorkerAfterPollThreadExits() throws Exception { - KafkaEnvelopeConsumerWorker worker = mock(KafkaEnvelopeConsumerWorker.class); - CountDownLatch pollEntered = new CountDownLatch(1); - CountDownLatch wakeupCalled = new CountDownLatch(1); - CountDownLatch pollReturned = new CountDownLatch(1); - CountDownLatch closeCalled = new CountDownLatch(1); - - doAnswer(invocation -> { - pollEntered.countDown(); - assertThat(wakeupCalled.await(1, TimeUnit.SECONDS)).isTrue(); - pollReturned.countDown(); - return 0; - }).when(worker).pollOnce(any()); - doAnswer(invocation -> { - wakeupCalled.countDown(); - return null; - }).when(worker).wakeup(); - doAnswer(invocation -> { - assertThat(pollReturned.getCount()).isZero(); - closeCalled.countDown(); - return null; - }).when(worker).close(); - KafkaEnvelopeConsumerRunner runner = new KafkaEnvelopeConsumerRunner( - List.of(worker), - Duration.ofSeconds(30), - Duration.ofMillis(1), - false); - - runner.start(); - assertThat(pollEntered.await(1, TimeUnit.SECONDS)).isTrue(); - runner.stop(); - - assertThat(wakeupCalled.getCount()).isZero(); - assertThat(closeCalled.getCount()).isZero(); - } -} diff --git a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerWorkerTest.java b/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerWorkerTest.java deleted file mode 100644 index 9d1a1b02..00000000 --- a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeConsumerWorkerTest.java +++ /dev/null @@ -1,84 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerRecord; -import com.lingniu.ingest.api.consumer.EnvelopeBatchIngestor; -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.api.consumer.EnvelopeDeadLetterRecord; -import com.lingniu.ingest.api.consumer.EnvelopeIngestResult; -import org.apache.kafka.clients.consumer.ConsumerRecord; -import org.apache.kafka.clients.consumer.MockConsumer; -import org.apache.kafka.clients.consumer.OffsetResetStrategy; -import org.apache.kafka.common.TopicPartition; -import org.junit.jupiter.api.Test; - -import java.time.Duration; -import java.util.List; -import java.util.Map; -import java.util.concurrent.atomic.AtomicReference; -import java.util.concurrent.atomic.AtomicInteger; - -import static org.assertj.core.api.Assertions.assertThat; - -class KafkaEnvelopeConsumerWorkerTest { - - @Test - void pollsKafkaRecordsAndDispatchesThemToConfiguredTopicProcessor() { - MockConsumer consumer = new MockConsumer<>(OffsetResetStrategy.EARLIEST); - TopicPartition partition = new TopicPartition("vehicle.realtime", 0); - consumer.assign(List.of(partition)); - consumer.updateBeginningOffsets(Map.of(partition, 0L)); - consumer.addRecord(new ConsumerRecord<>("vehicle.realtime", 0, 12L, "VIN001", new byte[]{0x01, 0x02})); - AtomicReference captured = new AtomicReference<>(); - EnvelopeConsumerProcessor processor = new EnvelopeConsumerProcessor( - "vehicle-state", - bytes -> EnvelopeIngestResult.invalid("bad envelope"), - captured::set); - - int processed = new KafkaEnvelopeConsumerWorker( - consumer, Map.of("vehicle.realtime", processor)).pollOnce(Duration.ZERO); - - assertThat(processed).isEqualTo(1); - assertThat(captured.get()).satisfies(record -> { - assertThat(record.topic()).isEqualTo("vehicle.realtime"); - assertThat(record.partition()).isEqualTo(0); - assertThat(record.offset()).isEqualTo(12L); - assertThat(record.key()).isEqualTo("VIN001"); - assertThat(record.payload()).containsExactly(0x01, 0x02); - }); - } - - @Test - void pollsKafkaRecordsAndDispatchesBatchWhenProcessorSupportsIt() { - MockConsumer consumer = new MockConsumer<>(OffsetResetStrategy.EARLIEST); - TopicPartition partition = new TopicPartition("vehicle.raw", 0); - consumer.assign(List.of(partition)); - consumer.updateBeginningOffsets(Map.of(partition, 0L)); - consumer.addRecord(new ConsumerRecord<>("vehicle.raw", 0, 12L, "VIN001", new byte[]{0x01})); - consumer.addRecord(new ConsumerRecord<>("vehicle.raw", 0, 13L, "VIN002", new byte[]{0x02})); - AtomicInteger batchCalls = new AtomicInteger(); - EnvelopeConsumerProcessor processor = new EnvelopeConsumerProcessor( - "event-history", - new EnvelopeBatchIngestor() { - @Override - public EnvelopeIngestResult tryIngest(byte[] kafkaValue) { - throw new AssertionError("single-record ingest should not be used for a batch-capable processor"); - } - - @Override - public List tryIngestAll(List kafkaValues) { - batchCalls.incrementAndGet(); - assertThat(kafkaValues).hasSize(2); - return List.of( - EnvelopeIngestResult.stored("event-1", "VIN001"), - EnvelopeIngestResult.invalid("bad envelope")); - } - }, - record -> {}); - - int processed = new KafkaEnvelopeConsumerWorker( - consumer, Map.of("vehicle.raw", processor)).pollOnce(Duration.ZERO); - - assertThat(processed).isEqualTo(2); - assertThat(batchCalls).hasValue(1); - } -} diff --git a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeDeadLetterSinkTest.java b/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeDeadLetterSinkTest.java deleted file mode 100644 index 94f51d96..00000000 --- a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEnvelopeDeadLetterSinkTest.java +++ /dev/null @@ -1,54 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.consumer.EnvelopeDeadLetterRecord; -import com.lingniu.ingest.api.consumer.EnvelopeIngestResult; -import org.apache.kafka.clients.producer.MockProducer; -import org.apache.kafka.clients.producer.ProducerRecord; -import org.apache.kafka.common.serialization.ByteArraySerializer; -import org.apache.kafka.common.serialization.StringSerializer; -import org.junit.jupiter.api.Test; - -import java.nio.charset.StandardCharsets; -import java.time.Instant; - -import static org.assertj.core.api.Assertions.assertThat; - -class KafkaEnvelopeDeadLetterSinkTest { - - @Test - void publishesDeadLetterRecordToConfiguredDlqTopicWithDiagnosticHeaders() { - MockProducer producer = new MockProducer<>( - true, new StringSerializer(), new ByteArraySerializer()); - KafkaEnvelopeDeadLetterSink sink = new KafkaEnvelopeDeadLetterSink(producer, "vehicle.consumer.dlq"); - EnvelopeDeadLetterRecord record = new EnvelopeDeadLetterRecord( - "event-history", - "vehicle.realtime", - 3, - 99L, - "VIN001", - EnvelopeIngestResult.Status.INVALID_ENVELOPE, - "", - "", - "VehicleEnvelope bytes are invalid", - new byte[]{0x01, 0x02}, - Instant.parse("2026-06-22T08:00:00Z")); - - sink.publish(record); - - assertThat(producer.history()).hasSize(1); - ProducerRecord sent = producer.history().getFirst(); - assertThat(sent.topic()).isEqualTo("vehicle.consumer.dlq"); - assertThat(sent.key()).isEqualTo("VIN001"); - assertThat(sent.value()).containsExactly(0x01, 0x02); - assertThat(header(sent, "dlq-service")).isEqualTo("event-history"); - assertThat(header(sent, "dlq-source-topic")).isEqualTo("vehicle.realtime"); - assertThat(header(sent, "dlq-source-partition")).isEqualTo("3"); - assertThat(header(sent, "dlq-source-offset")).isEqualTo("99"); - assertThat(header(sent, "dlq-status")).isEqualTo("INVALID_ENVELOPE"); - assertThat(header(sent, "dlq-message")).isEqualTo("VehicleEnvelope bytes are invalid"); - } - - private static String header(ProducerRecord record, String key) { - return new String(record.headers().lastHeader(key).value(), StandardCharsets.UTF_8); - } -} diff --git a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEventSinkTest.java b/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEventSinkTest.java deleted file mode 100644 index 2aa77fc9..00000000 --- a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaEventSinkTest.java +++ /dev/null @@ -1,83 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.LocationPayload; -import com.lingniu.ingest.api.event.VehicleEvent; -import io.github.resilience4j.circuitbreaker.CircuitBreaker; -import org.apache.kafka.clients.producer.Callback; -import org.apache.kafka.clients.producer.KafkaProducer; -import org.apache.kafka.clients.producer.ProducerRecord; -import org.junit.jupiter.api.Test; -import org.mockito.ArgumentCaptor; - -import java.time.Instant; -import java.util.Map; -import java.util.concurrent.CompletableFuture; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.ArgumentMatchers.any; -import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.when; - -class KafkaEventSinkTest { - - @Test - void acceptsRawArchiveRecordsForSplitServiceKafkaBoundary() { - KafkaEventSink sink = new KafkaEventSink( - null, - new EnvelopeMapper("node-1"), - new TopicRouter(new KafkaSinkProperties.Topics()), - "vehicle.dlq.gb32960.v1", - CircuitBreaker.ofDefaults("kafka-test")); - - assertThat(sink.accepts(rawArchive())).isTrue(); - } - - @Test - void usesPhoneAsKafkaKeyWhenVinIsUnknown() { - @SuppressWarnings("unchecked") - KafkaProducer producer = mock(KafkaProducer.class); - @SuppressWarnings("unchecked") - ArgumentCaptor> sent = ArgumentCaptor.forClass(ProducerRecord.class); - when(producer.send(sent.capture(), any())).thenAnswer(invocation -> { - invocation.getArgument(1).onCompletion(null, null); - return CompletableFuture.completedFuture(null); - }); - KafkaEventSink sink = new KafkaEventSink( - producer, - new EnvelopeMapper("node-1"), - new TopicRouter(new KafkaSinkProperties.Topics()), - "vehicle.dlq.gb32960.v1", - CircuitBreaker.ofDefaults("kafka-test")); - - sink.publish(jt808Location()).join(); - - assertThat(sent.getValue().key()).isEqualTo("phone:13079979000"); - } - - private static VehicleEvent.RawArchive rawArchive() { - return new VehicleEvent.RawArchive( - "raw-1", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-23T07:00:00Z"), - Instant.parse("2026-06-23T07:00:01Z"), - "trace-raw-1", - Map.of("platformAccount", "Hyundai"), - 0x02, - 0, - new byte[] {0x23, 0x23}); - } - - private static VehicleEvent.Location jt808Location() { - return new VehicleEvent.Location( - "loc-1", - "unknown", - ProtocolId.JT808, - Instant.parse("2026-06-29T11:22:00Z"), - Instant.parse("2026-06-29T11:22:01Z"), - "trace-loc-1", - Map.of("phone", "13079979000"), - new LocationPayload(121.1, 31.2, 8.0, 40.0, 90.0, 0L, 1L, null)); - } -} diff --git a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaSinkConsumerAutoConfigurationTest.java b/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaSinkConsumerAutoConfigurationTest.java deleted file mode 100644 index b4d680c5..00000000 --- a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaSinkConsumerAutoConfigurationTest.java +++ /dev/null @@ -1,74 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.consumer.EnvelopeConsumerProcessor; -import com.lingniu.ingest.api.consumer.EnvelopeIngestResult; -import org.apache.kafka.clients.consumer.MockConsumer; -import org.apache.kafka.clients.consumer.OffsetResetStrategy; -import org.apache.kafka.clients.producer.KafkaProducer; -import org.junit.jupiter.api.Test; -import org.junit.jupiter.api.extension.ExtendWith; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; -import org.springframework.boot.test.system.CapturedOutput; -import org.springframework.boot.test.system.OutputCaptureExtension; - -import java.lang.reflect.Field; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.mock; - -@ExtendWith(OutputCaptureExtension.class) -class KafkaSinkConsumerAutoConfigurationTest { - - private final ApplicationContextRunner contextRunner = new ApplicationContextRunner() - .withUserConfiguration(KafkaSinkAutoConfiguration.class, KafkaSinkConsumerAutoConfiguration.class) - .withAllowBeanDefinitionOverriding(true) - .withBean("vehicleStateEnvelopeConsumerProcessor", EnvelopeConsumerProcessor.class, - () -> new EnvelopeConsumerProcessor( - "vehicle-state", - bytes -> EnvelopeIngestResult.processed("evt-1", "VIN001"), - record -> {})) - .withBean("kafkaProducer", KafkaProducer.class, KafkaSinkConsumerAutoConfigurationTest::kafkaProducer) - .withBean(KafkaEnvelopeConsumerFactory.class, - () -> new KafkaEnvelopeConsumerFactory(props -> new MockConsumer<>(OffsetResetStrategy.EARLIEST))) - .withPropertyValues( - "lingniu.ingest.sink.kafka.enabled=true", - "lingniu.ingest.sink.kafka.consumer.enabled=true", - "lingniu.ingest.sink.kafka.consumer.auto-startup=false", - "lingniu.ingest.sink.kafka.consumer.bindings.vehicleStateEnvelopeConsumerProcessor.group-id=vehicle-state", - "lingniu.ingest.sink.kafka.consumer.bindings.vehicleStateEnvelopeConsumerProcessor.topics[0]=vehicle.realtime"); - - @Test - void createsKafkaEnvelopeConsumerRunnerWhenConsumerBindingIsConfigured(CapturedOutput output) { - contextRunner.run(context -> { - assertThat(context).hasSingleBean(KafkaEnvelopeConsumerRunner.class); - assertThat(context.getBean(KafkaEnvelopeConsumerRunner.class).workers()).hasSize(1); - }); - assertThat(output).doesNotContain("114.55.58.251"); - } - - @Test - void consumerDlqTopicOverridesProducerTopicDefaultsForDeadLetterSink() { - contextRunner - .withPropertyValues("lingniu.ingest.sink.kafka.consumer.dlq-topic=vehicle.dlq.consumer.v1") - .run(context -> { - assertThat(context).hasSingleBean(KafkaEnvelopeDeadLetterSink.class); - assertThat(deadLetterTopic(context.getBean(KafkaEnvelopeDeadLetterSink.class))) - .isEqualTo("vehicle.dlq.consumer.v1"); - }); - } - - @SuppressWarnings("unchecked") - private static KafkaProducer kafkaProducer() { - return mock(KafkaProducer.class); - } - - private static String deadLetterTopic(KafkaEnvelopeDeadLetterSink sink) { - try { - Field field = KafkaEnvelopeDeadLetterSink.class.getDeclaredField("topic"); - field.setAccessible(true); - return (String) field.get(sink); - } catch (ReflectiveOperationException e) { - throw new AssertionError(e); - } - } -} diff --git a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaSinkPropertiesTest.java b/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaSinkPropertiesTest.java deleted file mode 100644 index 400f4f9f..00000000 --- a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/KafkaSinkPropertiesTest.java +++ /dev/null @@ -1,39 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import org.junit.jupiter.api.Test; - -import java.nio.file.Files; -import java.nio.file.Path; -import java.util.Arrays; - -import static org.assertj.core.api.Assertions.assertThat; - -class KafkaSinkPropertiesTest { - - @Test - void defaultKafkaBrokerUsesProductionAddressButCanStillBeOverriddenByConfigBinding() { - KafkaSinkProperties props = new KafkaSinkProperties(); - - assertThat(props.getBootstrapServers()).isEqualTo("114.55.58.251:9092"); - - props.setBootstrapServers("kafka.internal:9092"); - assertThat(props.getBootstrapServers()).isEqualTo("kafka.internal:9092"); - } - - @Test - void exposesNoBackendTypeBecauseKafkaIsTheOnlySink() { - assertThat(Arrays.stream(KafkaSinkProperties.class.getMethods()).map(method -> method.getName())) - .doesNotContain("getType", "setType"); - } - - @Test - void sourceCommentsDoNotDescribeKafkaDisabledAsNoopEventDropping() throws Exception { - String source = Files.readString(Path.of( - "src/main/java/com/lingniu/ingest/sink/kafka/KafkaSinkProperties.java")); - - assertThat(source) - .doesNotContain("Noop") - .doesNotContain("吞掉") - .doesNotContain("外部 sink"); - } -} diff --git a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/TopicRouterTest.java b/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/TopicRouterTest.java deleted file mode 100644 index 7642426d..00000000 --- a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/TopicRouterTest.java +++ /dev/null @@ -1,147 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.api.ProtocolId; -import com.lingniu.ingest.api.event.AlarmPayload; -import com.lingniu.ingest.api.event.LocationPayload; -import com.lingniu.ingest.api.event.RealtimePayload; -import com.lingniu.ingest.api.event.VehicleEvent; -import org.junit.jupiter.api.Test; - -import java.time.Instant; -import java.util.List; -import java.util.Map; -import java.util.Set; - -import static org.assertj.core.api.Assertions.assertThat; - -class TopicRouterTest { - - @Test - void rawArchiveRoutesToVersionedGb32960RawTopicByDefault() { - TopicRouter router = new TopicRouter(new KafkaSinkProperties.Topics()); - - assertThat(router.route(rawArchive())).isEqualTo("vehicle.raw.gb32960.v1"); - } - - @Test - void normalizedGb32960EventsRouteToVersionedEventTopicByDefault() { - TopicRouter router = new TopicRouter(new KafkaSinkProperties.Topics()); - - assertThat(router.route(realtime())).isEqualTo("vehicle.event.gb32960.v1"); - assertThat(router.route(location())).isEqualTo("vehicle.event.gb32960.v1"); - assertThat(router.route(alarm())).isEqualTo("vehicle.event.gb32960.v1"); - assertThat(router.route(login())).isEqualTo("vehicle.event.gb32960.v1"); - assertThat(router.route(logout())).isEqualTo("vehicle.event.gb32960.v1"); - assertThat(router.route(heartbeat())).isEqualTo("vehicle.event.gb32960.v1"); - } - - @Test - void dlqDefaultsToVersionedGb32960DlqTopic() { - KafkaSinkProperties.Topics topics = new KafkaSinkProperties.Topics(); - - assertThat(topics.getDlq()).isEqualTo("vehicle.dlq.gb32960.v1"); - } - - private static VehicleEvent.RawArchive rawArchive() { - return new VehicleEvent.RawArchive( - "raw-1", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-23T07:00:00Z"), - Instant.parse("2026-06-23T07:00:01Z"), - "trace-raw-1", - Map.of("platformAccount", "Hyundai"), - 0x02, - 0, - new byte[] {0x23, 0x23}); - } - - private static VehicleEvent.Realtime realtime() { - return new VehicleEvent.Realtime( - "event-1", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-23T07:00:00Z"), - Instant.parse("2026-06-23T07:00:01Z"), - "trace-1", - Map.of("platformAccount", "Hyundai"), - new RealtimePayload( - 42.1, 1234.5, 80.0, null, null, - null, null, null, null, null, null, - RealtimePayload.VehicleState.STARTED, - RealtimePayload.ChargingState.UNCHARGED, - RealtimePayload.RunningMode.ELECTRIC, - null, null, null, - 113.12, 23.45, null, null, null, null)); - } - - private static VehicleEvent.Location location() { - return new VehicleEvent.Location( - "event-2", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-23T07:00:00Z"), - Instant.parse("2026-06-23T07:00:01Z"), - "trace-2", - Map.of("platformAccount", "Hyundai"), - new LocationPayload(113.12, 23.45, 0, 42.1, 90, 0, 0)); - } - - private static VehicleEvent.Alarm alarm() { - return new VehicleEvent.Alarm( - "event-3", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-23T07:00:00Z"), - Instant.parse("2026-06-23T07:00:01Z"), - "trace-3", - Map.of("platformAccount", "Hyundai"), - new AlarmPayload( - AlarmPayload.AlarmLevel.MINOR, - 1, - "GB32960_ALARM", - List.of("1"), - Set.of("bit0"), - 113.12, - 23.45, - AlarmPayload.SafetyCategory.GENERAL, - false, - AlarmPayload.HydrogenLeakLevel.NONE, - false)); - } - - private static VehicleEvent.Login login() { - return new VehicleEvent.Login( - "event-4", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-23T07:00:00Z"), - Instant.parse("2026-06-23T07:00:01Z"), - "trace-4", - Map.of("platformAccount", "Hyundai"), - "iccid-1", - "V2016"); - } - - private static VehicleEvent.Logout logout() { - return new VehicleEvent.Logout( - "event-5", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-23T07:00:00Z"), - Instant.parse("2026-06-23T07:00:01Z"), - "trace-5", - Map.of("platformAccount", "Hyundai")); - } - - private static VehicleEvent.Heartbeat heartbeat() { - return new VehicleEvent.Heartbeat( - "event-6", - "VIN001", - ProtocolId.GB32960, - Instant.parse("2026-06-23T07:00:00Z"), - Instant.parse("2026-06-23T07:00:01Z"), - "trace-6", - Map.of("platformAccount", "Hyundai")); - } -} diff --git a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/VehicleEnvelopeProtoCompatibilityTest.java b/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/VehicleEnvelopeProtoCompatibilityTest.java deleted file mode 100644 index 405031e2..00000000 --- a/modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/VehicleEnvelopeProtoCompatibilityTest.java +++ /dev/null @@ -1,65 +0,0 @@ -package com.lingniu.ingest.sink.kafka; - -import com.lingniu.ingest.sink.kafka.proto.DecodedFactPayload; -import com.lingniu.ingest.sink.kafka.proto.ParseStatusProto; -import com.lingniu.ingest.sink.kafka.proto.RawFrameFactPayload; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import org.junit.jupiter.api.Test; - -import static org.assertj.core.api.Assertions.assertThat; - -class VehicleEnvelopeProtoCompatibilityTest { - - @Test - void rawFrameFactPayloadRoundTripsThroughEnvelope() throws Exception { - VehicleEnvelope envelope = VehicleEnvelope.newBuilder() - .setSchemaVersion("2.0") - .setEventId("frame-1") - .setVin("VIN001") - .setSource("GB32960") - .setEventTimeMs(1782662400000L) - .setIngestTimeMs(1782662400001L) - .setRawFrameFact(RawFrameFactPayload.newBuilder() - .setFrameId("frame-1") - .setVehicleKey("VIN001") - .setMessageId(2) - .setRawUri("archive://raw/2026/06/29/GB32960/VIN001/frame-1.bin") - .setChecksum("sha256:abc") - .setRawSizeBytes(128) - .setParseStatus(ParseStatusProto.PARSE_STATUS_SUCCEEDED) - .build()) - .build(); - - VehicleEnvelope parsed = VehicleEnvelope.parseFrom(envelope.toByteArray()); - - assertThat(parsed.hasRawFrameFact()).isTrue(); - assertThat(parsed.getRawFrameFact().getFrameId()).isEqualTo("frame-1"); - assertThat(parsed.getRawFrameFact().getParseStatus()).isEqualTo(ParseStatusProto.PARSE_STATUS_SUCCEEDED); - } - - @Test - void decodedFactPayloadRoundTripsThroughEnvelope() throws Exception { - VehicleEnvelope envelope = VehicleEnvelope.newBuilder() - .setSchemaVersion("2.0") - .setEventId("fact-1") - .setVin("VIN001") - .setSource("JT808") - .setEventTimeMs(1782662400000L) - .setIngestTimeMs(1782662400001L) - .setDecodedFact(DecodedFactPayload.newBuilder() - .setFactId("fact-1") - .setFrameId("frame-1") - .setFactType("location") - .setVehicleKey("jt808:013912345678") - .setRawUri("archive://raw/2026/06/29/JT808/jt808_013912345678/frame-1.bin") - .putFields("longitude", "120.1") - .putFields("latitude", "30.1") - .build()) - .build(); - - VehicleEnvelope parsed = VehicleEnvelope.parseFrom(envelope.toByteArray()); - - assertThat(parsed.hasDecodedFact()).isTrue(); - assertThat(parsed.getDecodedFact().getFieldsMap()).containsEntry("longitude", "120.1"); - } -} diff --git a/modules/sinks/tdengine-history-store/pom.xml b/modules/sinks/tdengine-history-store/pom.xml deleted file mode 100644 index be3ea689..00000000 --- a/modules/sinks/tdengine-history-store/pom.xml +++ /dev/null @@ -1,60 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - tdengine-history-store - tdengine-history-store - TDengine-backed hot history store schema and Kafka envelope mapping. - - - - com.lingniu.ingest - sink-kafka - - - org.springframework.boot - spring-boot-starter - - - org.springframework.boot - spring-boot-autoconfigure - - - com.taosdata.jdbc - taos-jdbcdriver - - - com.zaxxer - HikariCP - - - org.springframework.boot - spring-boot-configuration-processor - true - - - com.fasterxml.jackson.core - jackson-databind - - - org.junit.jupiter - junit-jupiter - test - - - org.assertj - assertj-core - test - - - org.springframework.boot - spring-boot-starter-test - test - - - diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineBatchStatement.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineBatchStatement.java deleted file mode 100644 index d634ae4d..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineBatchStatement.java +++ /dev/null @@ -1,15 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.util.List; -import java.util.ArrayList; -import java.util.Collections; - -public record TdengineBatchStatement( - String createChildTableSql, - String insertSql, - List values -) { - public TdengineBatchStatement { - values = values == null ? List.of() : Collections.unmodifiableList(new ArrayList<>(values)); - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineEnvelopeRows.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineEnvelopeRows.java deleted file mode 100644 index b0d11c03..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineEnvelopeRows.java +++ /dev/null @@ -1,164 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import com.fasterxml.jackson.core.JsonProcessingException; -import com.fasterxml.jackson.databind.ObjectMapper; -import com.lingniu.ingest.sink.kafka.proto.LocationPayload; -import com.lingniu.ingest.sink.kafka.proto.RawFrameFactPayload; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; - -import java.time.Instant; -import java.util.LinkedHashMap; -import java.util.Map; -import java.util.Optional; - -public final class TdengineEnvelopeRows { - - private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper(); - - private TdengineEnvelopeRows() { - } - - public static Optional rawFrame(VehicleEnvelope envelope) { - if (envelope == null || !envelope.hasRawFrameFact()) { - return Optional.empty(); - } - RawFrameFactPayload raw = envelope.getRawFrameFact(); - Map metadata = new LinkedHashMap<>(); - metadata.putAll(envelope.getMetadataMap()); - metadata.putAll(raw.getMetadataMap()); - return Optional.of(new TdengineRawFrameRow( - rawFrameTs(envelope, metadata), - raw.getFrameId(), - instant(envelope.getIngestTimeMs()), - raw.getMessageId(), - raw.getSubType(), - instant(envelope.getEventTimeMs()), - raw.getRawUri(), - raw.getChecksum(), - raw.getRawSizeBytes(), - raw.getParseStatus().name().replace("PARSE_STATUS_", ""), - raw.getParseError(), - raw.getPeer(), - json(metadata), - parsedJson(envelope, metadata), - protocol(envelope), - vehicleKey(envelope, raw.getVehicleKey(), raw.getPhone()), - firstNonBlank(raw.getVin(), envelope.getVin()), - raw.getPhone() - )); - } - - public static Optional location(VehicleEnvelope envelope) { - if (envelope == null || !envelope.hasLocation()) { - return Optional.empty(); - } - LocationPayload location = envelope.getLocation(); - String rawUri = firstNonBlank( - envelope.hasRawArchive() ? envelope.getRawArchive().getUri() : "", - envelope.getMetadataOrDefault("rawArchiveUri", "")); - String frameId = envelope.getMetadataOrDefault("frame_id", envelope.getEventId()); - return Optional.of(new TdengineLocationRow( - instant(envelope.getEventTimeMs()), - envelope.getEventId(), - frameId, - instant(envelope.getIngestTimeMs()), - location.getLongitude(), - location.getLatitude(), - location.getAltitudeM(), - location.getSpeedKmh(), - location.getDirectionDeg(), - location.getAlarmFlag(), - location.getStatusFlag(), - totalMileage(envelope), - rawUri, - protocol(envelope), - vehicleKey(envelope, envelope.getMetadataOrDefault("vehicle_key", ""), - envelope.getMetadataOrDefault("phone", "")), - envelope.getVin(), - envelope.getMetadataOrDefault("phone", "") - )); - } - - private static Instant instant(long epochMillis) { - return Instant.ofEpochMilli(epochMillis); - } - - private static Instant rawFrameTs(VehicleEnvelope envelope, Map metadata) { - String rawArchiveEventId = firstNonBlank(metadata.get("rawArchiveEventId"), envelope.getEventId()); - if (rawArchiveEventId == null || rawArchiveEventId.isBlank()) { - return instant(envelope.getEventTimeMs()); - } - try { - long sequenceId = Long.parseLong(rawArchiveEventId); - long baseMillis = sequenceId / 1000L; - long sameMillisecondOffset = sequenceId % 1000L; - if (Math.abs(baseMillis - envelope.getEventTimeMs()) > 60_000L) { - return instant(envelope.getEventTimeMs()); - } - return Instant.ofEpochMilli(baseMillis + sameMillisecondOffset); - } catch (NumberFormatException ignored) { - return instant(envelope.getEventTimeMs()); - } - } - - private static String protocol(VehicleEnvelope envelope) { - return firstNonBlank(envelope.getSource(), "UNKNOWN"); - } - - private static String vehicleKey(VehicleEnvelope envelope, String explicitVehicleKey, String phone) { - String key = firstKnownNonBlank(explicitVehicleKey, envelope.getMetadataOrDefault("vehicle_key", ""), - envelope.getVin()); - if (!key.isBlank()) { - return key; - } - if ("JT808".equalsIgnoreCase(protocol(envelope))) { - String phoneValue = firstKnownNonBlank(phone, envelope.getMetadataOrDefault("phone", "")); - if (!phoneValue.isBlank()) { - return phoneValue.startsWith("jt808:") ? phoneValue : "jt808:" + phoneValue; - } - } - return firstNonBlank(explicitVehicleKey, envelope.getMetadataOrDefault("vehicle_key", ""), - envelope.getVin(), phone); - } - - private static Double totalMileage(VehicleEnvelope envelope) { - String value = envelope.getMetadataOrDefault("total_mileage_km", ""); - if (value.isBlank()) { - return null; - } - return Double.parseDouble(value); - } - - private static String parsedJson(VehicleEnvelope envelope, Map metadata) { - if (envelope.hasRawArchive() && !envelope.getRawArchive().getParsedJson().isBlank()) { - return envelope.getRawArchive().getParsedJson(); - } - return firstNonBlank(metadata.get("parsedJson"), metadata.get("parsed_json")); - } - - private static String json(Map metadata) { - try { - return OBJECT_MAPPER.writeValueAsString(metadata == null ? Map.of() : metadata); - } catch (JsonProcessingException e) { - throw new IllegalArgumentException("metadata cannot be serialized as json", e); - } - } - - private static String firstNonBlank(String... values) { - for (String value : values) { - if (value != null && !value.isBlank()) { - return value; - } - } - return ""; - } - - private static String firstKnownNonBlank(String... values) { - for (String value : values) { - if (value != null && !value.isBlank() && !"unknown".equalsIgnoreCase(value.trim())) { - return value; - } - } - return ""; - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryQueries.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryQueries.java deleted file mode 100644 index 6bc889c3..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryQueries.java +++ /dev/null @@ -1,129 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.util.ArrayList; -import java.util.List; - -public final class TdengineHistoryQueries { - - private static final String RAW_COLUMNS = "ts, frame_id, received_at, message_id, sub_type, event_time, " - + "raw_uri, checksum, raw_size_bytes, parse_status, parse_error, peer, metadata_json, parsed_json, " - + "protocol, vehicle_key, vin, phone"; - private static final String LOCATION_COLUMNS = "ts, fact_id, frame_id, received_at, longitude, latitude, " - + "altitude_m, speed_kmh, direction_deg, alarm_flag, status_flag, total_mileage_km, raw_uri, " - + "protocol, vehicle_key, vin, phone"; - - private final TdengineHistorySchema schema; - - public TdengineHistoryQueries(TdengineHistorySchema schema) { - if (schema == null) { - throw new IllegalArgumentException("schema must not be null"); - } - this.schema = schema; - } - - public TdengineQueryStatement rawFrames(TdengineRawFrameQuery query) { - if (query.childTableQuery()) { - String table = schema.rawFrameTable(query.protocol(), query.vehicleKey()); - return query(table, RAW_COLUMNS, "frame_id", query.from(), query.to(), - query.order(), query.limit(), query.cursor()); - } - return rawFramesStable(query); - } - - public TdengineQueryStatement locations(TdengineLocationQuery query) { - String table = schema.locationTable(query.protocol(), query.vehicleKey()); - return query(table, LOCATION_COLUMNS, "fact_id", query.from(), query.to(), - query.order(), query.limit(), query.cursor()); - } - - public TdengineQueryStatement locationsByRawUri(String protocol, String rawUri, int limit) { - return byRawUri(schema.locationsStableTable(), LOCATION_COLUMNS, protocol, rawUri, "fact_id", limit); - } - - private static TdengineQueryStatement query(String table, - String columns, - String tieBreakerColumn, - Object from, - Object to, - TdengineQueryOrder order, - int limit, - TdenginePageCursor cursor) { - List values = new ArrayList<>(); - StringBuilder sql = new StringBuilder(256) - .append("SELECT ").append(columns) - .append(" FROM ").append(table) - .append(" WHERE ts >= ? AND ts < ?"); - values.add(from); - values.add(to); - if (cursor != null) { - String op = order == TdengineQueryOrder.ASC ? ">" : "<"; - sql.append(" AND (ts ").append(op).append(" ? OR (ts = ? AND ") - .append(tieBreakerColumn).append(' ').append(op).append(" ?))"); - values.add(cursor.ts()); - values.add(cursor.ts()); - values.add(cursor.tieBreaker()); - } - sql.append(" ORDER BY ts ").append(order.name()) - .append(", ").append(tieBreakerColumn).append(' ').append(order.name()) - .append(" LIMIT ?"); - values.add(limit); - return new TdengineQueryStatement(sql.toString(), values); - } - - private TdengineQueryStatement rawFramesStable(TdengineRawFrameQuery query) { - List values = new ArrayList<>(); - StringBuilder sql = new StringBuilder(384) - .append("SELECT ").append(RAW_COLUMNS) - .append(" FROM ").append(schema.rawFramesStableTable()) - .append(" WHERE ts >= ? AND ts < ?"); - values.add(query.from()); - values.add(query.to()); - addFilter(sql, values, "protocol", query.protocol()); - addFilter(sql, values, "vehicle_key", query.vehicleKey()); - addFilter(sql, values, "vin", query.vin()); - addFilter(sql, values, "phone", query.phone()); - if (query.messageId() != null) { - sql.append(" AND message_id = ?"); - values.add(query.messageId()); - } - addFilter(sql, values, "parse_status", query.parseStatus()); - if (query.cursor() != null) { - String op = query.order() == TdengineQueryOrder.ASC ? ">" : "<"; - sql.append(" AND (ts ").append(op).append(" ? OR (ts = ? AND frame_id ") - .append(op).append(" ?))"); - values.add(query.cursor().ts()); - values.add(query.cursor().ts()); - values.add(query.cursor().tieBreaker()); - } - sql.append(" ORDER BY ts ").append(query.order().name()) - .append(", frame_id ").append(query.order().name()) - .append(" LIMIT ?"); - values.add(query.limit()); - return new TdengineQueryStatement(sql.toString(), values); - } - - private static void addFilter(StringBuilder sql, List values, String column, String value) { - if (value != null && !value.isBlank()) { - sql.append(" AND ").append(column).append(" = ?"); - values.add(value); - } - } - - private static TdengineQueryStatement byRawUri(String table, - String columns, - String protocol, - String rawUri, - String tieBreakerColumn, - int limit) { - List values = new ArrayList<>(); - StringBuilder sql = new StringBuilder(256) - .append("SELECT ").append(columns) - .append(" FROM ").append(table) - .append(" WHERE raw_uri = ?"); - values.add(rawUri); - addFilter(sql, values, "protocol", protocol == null ? "" : protocol.trim().toUpperCase()); - sql.append(" ORDER BY ts DESC, ").append(tieBreakerColumn).append(" DESC LIMIT ?"); - values.add(Math.max(1, Math.min(limit, 1000))); - return new TdengineQueryStatement(sql.toString(), values); - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryReader.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryReader.java deleted file mode 100644 index 5044a28c..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryReader.java +++ /dev/null @@ -1,16 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.io.IOException; -import java.util.List; - -public interface TdengineHistoryReader { - - TdenginePage queryRawFrames(TdengineRawFrameQuery query) throws IOException; - - TdenginePage queryLocations(TdengineLocationQuery query) throws IOException; - - default List queryLocationsByRawUri(String protocol, String rawUri, int limit) - throws IOException { - return List.of(); - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistorySchema.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistorySchema.java deleted file mode 100644 index 4e135832..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistorySchema.java +++ /dev/null @@ -1,90 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.util.List; - -/** - * SQL model for the TDengine hot history store. - */ -public final class TdengineHistorySchema { - - private final String database; - - public TdengineHistorySchema(String database) { - this.database = TdengineIdentifier.database(database); - } - - public List bootstrapSql() { - return List.of( - "CREATE DATABASE IF NOT EXISTS " + database + " PRECISION 'ms'", - "USE " + database, - rawFramesStableSql(), - vehicleLocationsStableSql() - ); - } - - public String rawFrameTable(String protocol, String vehicleKey) { - return "raw_" + TdengineIdentifier.fragment(protocol) + "_" + TdengineIdentifier.hash16(vehicleKey); - } - - public String rawFramesStableTable() { - return "raw_frames"; - } - - public String locationsStableTable() { - return "vehicle_locations"; - } - - public String locationTable(String protocol, String vehicleKey) { - return "loc_" + TdengineIdentifier.fragment(protocol) + "_" + TdengineIdentifier.hash16(vehicleKey); - } - - private static String rawFramesStableSql() { - return """ - CREATE STABLE IF NOT EXISTS raw_frames ( - ts TIMESTAMP, - frame_id NCHAR(64), - received_at TIMESTAMP, - message_id INT, - sub_type INT, - event_time TIMESTAMP, - raw_uri NCHAR(512), - checksum NCHAR(128), - raw_size_bytes BIGINT, - parse_status NCHAR(16), - parse_error NCHAR(512), - peer NCHAR(128), - metadata_json NCHAR(4096), - parsed_json VARCHAR(16374) - ) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) - )"""; - } - - private static String vehicleLocationsStableSql() { - return """ - CREATE STABLE IF NOT EXISTS vehicle_locations ( - ts TIMESTAMP, - fact_id NCHAR(64), - frame_id NCHAR(64), - received_at TIMESTAMP, - longitude DOUBLE, - latitude DOUBLE, - altitude_m DOUBLE, - speed_kmh DOUBLE, - direction_deg DOUBLE, - alarm_flag BIGINT, - status_flag BIGINT, - total_mileage_km DOUBLE, - raw_uri NCHAR(512) - ) TAGS ( - protocol NCHAR(16), - vehicle_key NCHAR(128), - vin NCHAR(64), - phone NCHAR(32) - )"""; - } - -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryStatements.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryStatements.java deleted file mode 100644 index af5aaefe..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryStatements.java +++ /dev/null @@ -1,64 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.util.List; -import java.util.Arrays; - -public final class TdengineHistoryStatements { - - private static final String RAW_FRAME_COLUMNS = "ts, frame_id, received_at, message_id, sub_type, event_time, " - + "raw_uri, checksum, raw_size_bytes, parse_status, parse_error, peer, metadata_json, parsed_json"; - private static final String RAW_FRAME_PLACEHOLDERS = "?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?"; - - private static final String LOCATION_COLUMNS = "ts, fact_id, frame_id, received_at, longitude, latitude, " - + "altitude_m, speed_kmh, direction_deg, alarm_flag, status_flag, total_mileage_km, raw_uri"; - private static final String LOCATION_PLACEHOLDERS = "?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?"; - - private final TdengineHistorySchema schema; - - public TdengineHistoryStatements(TdengineHistorySchema schema) { - if (schema == null) { - throw new IllegalArgumentException("schema must not be null"); - } - this.schema = schema; - } - - public TdengineBatchStatement rawFrame(TdengineRawFrameRow row) { - String table = schema.rawFrameTable(row.protocol(), row.vehicleKey()); - return new TdengineBatchStatement( - "CREATE TABLE IF NOT EXISTS " + table + " USING raw_frames TAGS (" - + tags(row.protocol(), row.vehicleKey(), row.vin(), row.phone()) + ")", - "INSERT INTO " + table + " (" + RAW_FRAME_COLUMNS + ") VALUES (" + RAW_FRAME_PLACEHOLDERS + ")", - values( - row.ts(), row.frameId(), row.receivedAt(), row.messageId(), row.subType(), - row.eventTime(), row.rawUri(), row.checksum(), row.rawSizeBytes(), row.parseStatus(), - row.parseError(), row.peer(), row.metadataJson(), row.parsedJson() - ) - ); - } - - public TdengineBatchStatement location(TdengineLocationRow row) { - String table = schema.locationTable(row.protocol(), row.vehicleKey()); - return new TdengineBatchStatement( - "CREATE TABLE IF NOT EXISTS " + table + " USING vehicle_locations TAGS (" - + tags(row.protocol(), row.vehicleKey(), row.vin(), row.phone()) + ")", - "INSERT INTO " + table + " (" + LOCATION_COLUMNS + ") VALUES (" + LOCATION_PLACEHOLDERS + ")", - values( - row.ts(), row.factId(), row.frameId(), row.receivedAt(), row.longitude(), row.latitude(), - row.altitudeM(), row.speedKmh(), row.directionDeg(), row.alarmFlag(), row.statusFlag(), - row.totalMileageKm(), row.rawUri() - ) - ); - } - - private static String tags(String protocol, String vehicleKey, String vin, String phone) { - return quote(protocol) + ", " + quote(vehicleKey) + ", " + quote(vin) + ", " + quote(phone); - } - - private static String quote(String value) { - return "'" + (value == null ? "" : value.replace("'", "''")) + "'"; - } - - private static List values(Object... values) { - return Arrays.asList(values); - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryWriter.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryWriter.java deleted file mode 100644 index e8d51a7c..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryWriter.java +++ /dev/null @@ -1,19 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.io.IOException; -import java.util.List; - -public interface TdengineHistoryWriter { - - default void appendRawFrame(TdengineRawFrameRow row) throws IOException { - appendRawFrames(List.of(row)); - } - - void appendRawFrames(List rows) throws IOException; - - default void appendLocation(TdengineLocationRow row) throws IOException { - appendLocations(List.of(row)); - } - - void appendLocations(List rows) throws IOException; -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineIdentifier.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineIdentifier.java deleted file mode 100644 index 5f1d730a..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineIdentifier.java +++ /dev/null @@ -1,42 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.nio.charset.StandardCharsets; -import java.security.MessageDigest; -import java.security.NoSuchAlgorithmException; -import java.util.HexFormat; -import java.util.Locale; - -final class TdengineIdentifier { - - private TdengineIdentifier() { - } - - static String database(String value) { - String fragment = fragment(value); - if (fragment.isBlank()) { - throw new IllegalArgumentException("database must contain identifier characters"); - } - return fragment; - } - - static String fragment(String value) { - if (value == null) { - return "unknown"; - } - String normalized = value.trim().toLowerCase(Locale.ROOT) - .replaceAll("[^a-z0-9_]+", "_") - .replaceAll("_+", "_") - .replaceAll("^_|_$", ""); - return normalized.isBlank() ? "unknown" : normalized; - } - - static String hash16(String value) { - try { - byte[] digest = MessageDigest.getInstance("SHA-256") - .digest(String.valueOf(value).getBytes(StandardCharsets.UTF_8)); - return HexFormat.of().formatHex(digest, 0, 8); - } catch (NoSuchAlgorithmException e) { - throw new IllegalStateException("SHA-256 is not available", e); - } - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryReader.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryReader.java deleted file mode 100644 index cf3897e5..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryReader.java +++ /dev/null @@ -1,185 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import javax.sql.DataSource; -import java.io.IOException; -import java.sql.Connection; -import java.sql.PreparedStatement; -import java.sql.ResultSet; -import java.sql.SQLException; -import java.sql.Timestamp; -import java.time.Instant; -import java.util.ArrayList; -import java.util.List; -import java.util.Optional; -import java.util.function.Function; - -public final class TdengineJdbcHistoryReader implements TdengineHistoryReader { - - private final DataSource dataSource; - private final TdengineHistoryQueries queries; - - public TdengineJdbcHistoryReader(DataSource dataSource, TdengineHistorySchema schema) { - if (dataSource == null) { - throw new IllegalArgumentException("dataSource must not be null"); - } - this.dataSource = dataSource; - this.queries = new TdengineHistoryQueries(schema); - } - - @Override - public TdenginePage queryRawFrames(TdengineRawFrameQuery query) throws IOException { - int pageSize = Math.min(query.limit(), 1000); - TdengineQueryStatement statement = queries.rawFrames(query.withLimit(pageSize + 1)); - return readPage(statement, pageSize, this::rawFrame, row -> new TdenginePageCursor(row.ts(), row.frameId())); - } - - @Override - public TdenginePage queryLocations(TdengineLocationQuery query) throws IOException { - int pageSize = Math.min(query.limit(), 1000); - TdengineQueryStatement statement = queries.locations(query.withLimit(pageSize + 1)); - return readPage(statement, pageSize, this::location, row -> new TdenginePageCursor(row.ts(), row.factId())); - } - - @Override - public List queryLocationsByRawUri(String protocol, String rawUri, int limit) - throws IOException { - if (rawUri == null || rawUri.isBlank()) { - return List.of(); - } - TdengineQueryStatement statement = queries.locationsByRawUri(protocol, rawUri, limit); - return readList(statement, this::location); - } - - private TdenginePage readPage(TdengineQueryStatement statement, - int pageSize, - SqlRowMapper mapper, - Function cursor) throws IOException { - List rows = new ArrayList<>(); - try (Connection connection = dataSource.getConnection(); - PreparedStatement prepared = connection.prepareStatement(statement.sql())) { - bind(prepared, statement.values()); - try (ResultSet resultSet = prepared.executeQuery()) { - while (resultSet.next()) { - rows.add(mapper.map(resultSet)); - } - } - } catch (SQLException e) { - if (isTableMissing(e)) { - return new TdenginePage<>(List.of(), Optional.empty()); - } - throw new IOException("tdengine history query failed", e); - } - boolean hasMore = rows.size() > pageSize; - List items = hasMore ? List.copyOf(rows.subList(0, pageSize)) : List.copyOf(rows); - Optional next = hasMore && !items.isEmpty() - ? Optional.of(cursor.apply(items.getLast())) - : Optional.empty(); - return new TdenginePage<>(items, next); - } - - private List readList(TdengineQueryStatement statement, SqlRowMapper mapper) throws IOException { - List rows = new ArrayList<>(); - try (Connection connection = dataSource.getConnection(); - PreparedStatement prepared = connection.prepareStatement(statement.sql())) { - bind(prepared, statement.values()); - try (ResultSet resultSet = prepared.executeQuery()) { - while (resultSet.next()) { - rows.add(mapper.map(resultSet)); - } - } - } catch (SQLException e) { - if (isTableMissing(e)) { - return List.of(); - } - throw new IOException("tdengine history query failed", e); - } - return List.copyOf(rows); - } - - private static boolean isTableMissing(SQLException e) { - for (Throwable current = e; current != null; current = current.getCause()) { - String message = current.getMessage(); - if (message != null) { - String normalized = message.toLowerCase(); - if (normalized.contains("table does not exist") - || normalized.contains("0x2603") - || normalized.contains("table not exist")) { - return true; - } - } - } - return false; - } - - private static void bind(PreparedStatement prepared, List values) throws SQLException { - for (int i = 0; i < values.size(); i++) { - Object value = values.get(i); - int parameterIndex = i + 1; - if (value instanceof Instant instant) { - prepared.setTimestamp(parameterIndex, Timestamp.from(instant)); - } else { - prepared.setObject(parameterIndex, value); - } - } - } - - private TdengineRawFrameRow rawFrame(ResultSet rs) throws SQLException { - return new TdengineRawFrameRow( - instant(rs, "ts"), - rs.getString("frame_id"), - instant(rs, "received_at"), - rs.getInt("message_id"), - rs.getInt("sub_type"), - instant(rs, "event_time"), - rs.getString("raw_uri"), - rs.getString("checksum"), - rs.getLong("raw_size_bytes"), - rs.getString("parse_status"), - rs.getString("parse_error"), - rs.getString("peer"), - rs.getString("metadata_json"), - rs.getString("parsed_json"), - rs.getString("protocol"), - rs.getString("vehicle_key"), - rs.getString("vin"), - rs.getString("phone") - ); - } - - private TdengineLocationRow location(ResultSet rs) throws SQLException { - return new TdengineLocationRow( - instant(rs, "ts"), - rs.getString("fact_id"), - rs.getString("frame_id"), - instant(rs, "received_at"), - rs.getDouble("longitude"), - rs.getDouble("latitude"), - rs.getDouble("altitude_m"), - rs.getDouble("speed_kmh"), - rs.getDouble("direction_deg"), - rs.getLong("alarm_flag"), - rs.getLong("status_flag"), - nullableDouble(rs, "total_mileage_km"), - rs.getString("raw_uri"), - rs.getString("protocol"), - rs.getString("vehicle_key"), - rs.getString("vin"), - rs.getString("phone") - ); - } - - private static Instant instant(ResultSet rs, String column) throws SQLException { - Timestamp timestamp = rs.getTimestamp(column); - return timestamp == null ? null : timestamp.toInstant(); - } - - private static Double nullableDouble(ResultSet rs, String column) throws SQLException { - Object value = rs.getObject(column); - return value instanceof Number number ? number.doubleValue() : null; - } - - @FunctionalInterface - private interface SqlRowMapper { - T map(ResultSet rs) throws SQLException; - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryWriter.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryWriter.java deleted file mode 100644 index 5d4ed92e..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryWriter.java +++ /dev/null @@ -1,248 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import javax.sql.DataSource; -import java.io.IOException; -import java.sql.Connection; -import java.sql.SQLException; -import java.sql.Statement; -import java.sql.Timestamp; -import java.time.Instant; -import java.util.LinkedHashMap; -import java.util.LinkedHashSet; -import java.util.List; -import java.util.Map; -import java.util.Set; -import java.util.concurrent.ConcurrentHashMap; -import java.util.function.Function; - -public final class TdengineJdbcHistoryWriter implements TdengineHistoryWriter { - - private static final int MAX_LITERAL_ROWS_PER_STATEMENT = 200; - - private final DataSource dataSource; - private final TdengineHistorySchema schema; - private final TdengineHistoryStatements statements; - private final Object schemaInitializationMonitor = new Object(); - private final Map lastTimestampByInsertSql = new ConcurrentHashMap<>(); - private volatile boolean schemaInitialized; - - public TdengineJdbcHistoryWriter(DataSource dataSource, TdengineHistorySchema schema) { - if (dataSource == null) { - throw new IllegalArgumentException("dataSource must not be null"); - } - if (schema == null) { - throw new IllegalArgumentException("schema must not be null"); - } - this.dataSource = dataSource; - this.schema = schema; - this.statements = new TdengineHistoryStatements(schema); - } - - public void initializeSchema() throws IOException { - inTransaction(connection -> { - try (Statement statement = connection.createStatement()) { - for (String sql : schema.bootstrapSql()) { - statement.execute(sql); - } - } - }); - schemaInitialized = true; - } - - @Override - public void appendRawFrames(List rows) throws IOException { - append(rows, statements::rawFrame); - } - - @Override - public void appendLocations(List rows) throws IOException { - append(rows, statements::location); - } - - private void append(List rows, Function mapper) throws IOException { - if (rows == null || rows.isEmpty()) { - return; - } - if (!schemaInitialized) { - synchronized (schemaInitializationMonitor) { - if (!schemaInitialized) { - append(rows, mapper, true); - schemaInitialized = true; - return; - } - } - } - append(rows, mapper, false); - } - - private void append(List rows, - Function mapper, - boolean initializeSchema) throws IOException { - inTransaction(connection -> { - if (initializeSchema) { - try (Statement statement = connection.createStatement()) { - for (String sql : schema.bootstrapSql()) { - statement.execute(sql); - } - } - } - Set createdChildTables = new LinkedHashSet<>(); - Map literalBatches = new LinkedHashMap<>(); - for (T row : rows) { - if (row == null) { - continue; - } - TdengineBatchStatement batch = mapper.apply(row); - if (createdChildTables.add(batch.createChildTableSql())) { - try (Statement statement = connection.createStatement()) { - statement.execute(batch.createChildTableSql()); - } - } - literalBatches.computeIfAbsent(batch.insertSql(), LiteralBatch::new) - .add(uniqueTimestampValues(batch.insertSql(), batch.values())); - } - for (LiteralBatch batch : literalBatches.values()) { - executeLiteralBatch(connection, batch); - } - }); - } - - private List uniqueTimestampValues(String insertSql, List values) { - if (values == null || values.isEmpty() || !(values.getFirst() instanceof Instant candidate)) { - return values; - } - Instant uniqueTimestamp = lastTimestampByInsertSql.compute(insertSql, (key, lastTimestamp) -> { - if (lastTimestamp == null || candidate.isAfter(lastTimestamp)) { - return candidate; - } - return lastTimestamp.plusMillis(1); - }); - if (candidate.equals(uniqueTimestamp)) { - return values; - } - List copy = new java.util.ArrayList<>(values); - copy.set(0, uniqueTimestamp); - return copy; - } - - private static void executeLiteralBatch(Connection connection, LiteralBatch batch) throws SQLException { - try (Statement statement = connection.createStatement()) { - List> rows = batch.rows(); - for (int start = 0; start < rows.size(); start += MAX_LITERAL_ROWS_PER_STATEMENT) { - int end = Math.min(start + MAX_LITERAL_ROWS_PER_STATEMENT, rows.size()); - statement.execute(toLiteralInsertSql(batch.insertSql(), rows.subList(start, end))); - } - } - } - - private static String toLiteralInsertSql(String insertSql, List> rows) { - int valuesIndex = insertSql.lastIndexOf("VALUES"); - if (valuesIndex < 0) { - throw new IllegalArgumentException("insert SQL must contain VALUES: " + insertSql); - } - if (rows == null || rows.isEmpty()) { - throw new IllegalArgumentException("literal rows must not be empty"); - } - StringBuilder sql = new StringBuilder(insertSql.substring(0, valuesIndex)) - .append("VALUES "); - for (int rowIndex = 0; rowIndex < rows.size(); rowIndex++) { - if (rowIndex > 0) { - sql.append(' '); - } - List values = rows.get(rowIndex); - sql.append('('); - for (int i = 0; i < values.size(); i++) { - if (i > 0) { - sql.append(", "); - } - sql.append(literal(values.get(i))); - } - sql.append(')'); - } - return sql.toString(); - } - - private static String literal(Object value) { - if (value == null) { - return "NULL"; - } - if (value instanceof Instant instant) { - return Long.toString(instant.toEpochMilli()); - } - if (value instanceof Timestamp timestamp) { - return Long.toString(timestamp.toInstant().toEpochMilli()); - } - if (value instanceof Double doubleValue) { - return Double.isFinite(doubleValue) ? doubleValue.toString() : "NULL"; - } - if (value instanceof Float floatValue) { - return Float.isFinite(floatValue) ? floatValue.toString() : "NULL"; - } - if (value instanceof Number number) { - return number.toString(); - } - return "'" + value.toString().replace("'", "''") + "'"; - } - - private void inTransaction(SqlWork work) throws IOException { - try (Connection connection = dataSource.getConnection()) { - boolean autoCommit = connection.getAutoCommit(); - connection.setAutoCommit(false); - try { - work.execute(connection); - connection.commit(); - } catch (JdbcRuntimeException e) { - rollback(connection); - throw e.getCause(); - } catch (SQLException e) { - rollback(connection); - throw e; - } finally { - connection.setAutoCommit(autoCommit); - } - } catch (SQLException e) { - throw new IOException("tdengine history write failed", e); - } - } - - private static void rollback(Connection connection) throws SQLException { - connection.rollback(); - } - - @FunctionalInterface - private interface SqlWork { - void execute(Connection connection) throws SQLException; - } - - private static final class JdbcRuntimeException extends RuntimeException { - private JdbcRuntimeException(SQLException cause) { - super(cause); - } - - @Override - public synchronized SQLException getCause() { - return (SQLException) super.getCause(); - } - } - - private static final class LiteralBatch { - private final String insertSql; - private final List> rows = new java.util.ArrayList<>(); - - private LiteralBatch(String insertSql) { - this.insertSql = insertSql; - } - - private void add(List row) { - rows.add(new java.util.ArrayList<>(row)); - } - - private String insertSql() { - return insertSql; - } - - private List> rows() { - return rows; - } - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineLocationQuery.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineLocationQuery.java deleted file mode 100644 index 700484f8..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineLocationQuery.java +++ /dev/null @@ -1,31 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.time.Instant; - -public record TdengineLocationQuery( - String protocol, - String vehicleKey, - Instant from, - Instant to, - TdengineQueryOrder order, - int limit, - TdenginePageCursor cursor -) { - public TdengineLocationQuery { - if (protocol == null || protocol.isBlank()) { - throw new IllegalArgumentException("protocol must not be blank"); - } - if (vehicleKey == null || vehicleKey.isBlank()) { - throw new IllegalArgumentException("vehicleKey must not be blank"); - } - if (from == null || to == null || !from.isBefore(to)) { - throw new IllegalArgumentException("query time range must be valid"); - } - order = order == null ? TdengineQueryOrder.DESC : order; - limit = Math.max(1, Math.min(limit, 1001)); - } - - TdengineLocationQuery withLimit(int newLimit) { - return new TdengineLocationQuery(protocol, vehicleKey, from, to, order, newLimit, cursor); - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineLocationRow.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineLocationRow.java deleted file mode 100644 index 0b7f4567..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineLocationRow.java +++ /dev/null @@ -1,24 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.time.Instant; - -public record TdengineLocationRow( - Instant ts, - String factId, - String frameId, - Instant receivedAt, - double longitude, - double latitude, - double altitudeM, - double speedKmh, - double directionDeg, - long alarmFlag, - long statusFlag, - Double totalMileageKm, - String rawUri, - String protocol, - String vehicleKey, - String vin, - String phone -) { -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdenginePage.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdenginePage.java deleted file mode 100644 index 5447cf98..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdenginePage.java +++ /dev/null @@ -1,12 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.util.List; -import java.util.Optional; - -public record TdenginePage(List items, Optional nextCursor) { - - public TdenginePage { - items = items == null ? List.of() : List.copyOf(items); - nextCursor = nextCursor == null ? Optional.empty() : nextCursor; - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdenginePageCursor.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdenginePageCursor.java deleted file mode 100644 index 76a2a3e3..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdenginePageCursor.java +++ /dev/null @@ -1,13 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.time.Instant; - -public record TdenginePageCursor(Instant ts, String tieBreaker) { - - public TdenginePageCursor { - if (ts == null) { - throw new IllegalArgumentException("cursor ts must not be null"); - } - tieBreaker = tieBreaker == null ? "" : tieBreaker; - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineQueryOrder.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineQueryOrder.java deleted file mode 100644 index 3c1c0e71..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineQueryOrder.java +++ /dev/null @@ -1,6 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -public enum TdengineQueryOrder { - ASC, - DESC -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineQueryStatement.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineQueryStatement.java deleted file mode 100644 index 034fffd3..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineQueryStatement.java +++ /dev/null @@ -1,15 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.util.ArrayList; -import java.util.Collections; -import java.util.List; - -public record TdengineQueryStatement(String sql, List values) { - - public TdengineQueryStatement { - if (sql == null || sql.isBlank()) { - throw new IllegalArgumentException("sql must not be blank"); - } - values = values == null ? List.of() : Collections.unmodifiableList(new ArrayList<>(values)); - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineRawFrameQuery.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineRawFrameQuery.java deleted file mode 100644 index 87f040c4..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineRawFrameQuery.java +++ /dev/null @@ -1,60 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.time.Instant; - -public record TdengineRawFrameQuery( - String protocol, - String vehicleKey, - String vin, - String phone, - Integer messageId, - String parseStatus, - Instant from, - Instant to, - TdengineQueryOrder order, - int limit, - TdenginePageCursor cursor -) { - public TdengineRawFrameQuery(String protocol, - String vehicleKey, - Instant from, - Instant to, - TdengineQueryOrder order, - int limit, - TdenginePageCursor cursor) { - this(protocol, vehicleKey, "", "", null, "", from, to, order, limit, cursor); - } - - public TdengineRawFrameQuery { - if (protocol == null || protocol.isBlank()) { - throw new IllegalArgumentException("protocol must not be blank"); - } - if (from == null || to == null || !from.isBefore(to)) { - throw new IllegalArgumentException("query time range must be valid"); - } - protocol = protocol.trim().toUpperCase(); - vehicleKey = trim(vehicleKey); - vin = trim(vin); - phone = trim(phone); - parseStatus = trim(parseStatus); - order = order == null ? TdengineQueryOrder.DESC : order; - limit = Math.max(1, Math.min(limit, 1001)); - } - - TdengineRawFrameQuery withLimit(int newLimit) { - return new TdengineRawFrameQuery(protocol, vehicleKey, vin, phone, messageId, parseStatus, - from, to, order, newLimit, cursor); - } - - boolean childTableQuery() { - return !vehicleKey.isBlank() - && vin.isBlank() - && phone.isBlank() - && messageId == null - && parseStatus.isBlank(); - } - - private static String trim(String value) { - return value == null ? "" : value.trim(); - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineRawFrameRow.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineRawFrameRow.java deleted file mode 100644 index a965bcc0..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/TdengineRawFrameRow.java +++ /dev/null @@ -1,25 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import java.time.Instant; - -public record TdengineRawFrameRow( - Instant ts, - String frameId, - Instant receivedAt, - int messageId, - int subType, - Instant eventTime, - String rawUri, - String checksum, - long rawSizeBytes, - String parseStatus, - String parseError, - String peer, - String metadataJson, - String parsedJson, - String protocol, - String vehicleKey, - String vin, - String phone -) { -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/config/TdengineHistoryAutoConfiguration.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/config/TdengineHistoryAutoConfiguration.java deleted file mode 100644 index 2d612fb3..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/config/TdengineHistoryAutoConfiguration.java +++ /dev/null @@ -1,85 +0,0 @@ -package com.lingniu.ingest.tdenginehistory.config; - -import com.lingniu.ingest.tdenginehistory.TdengineHistorySchema; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryStatements; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryWriter; -import com.lingniu.ingest.tdenginehistory.TdengineJdbcHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineJdbcHistoryWriter; -import com.zaxxer.hikari.HikariConfig; -import com.zaxxer.hikari.HikariDataSource; -import org.springframework.boot.autoconfigure.AutoConfiguration; -import org.springframework.boot.autoconfigure.condition.ConditionalOnBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; -import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; -import org.springframework.boot.context.properties.EnableConfigurationProperties; -import org.springframework.context.annotation.Bean; - -import javax.sql.DataSource; - -@AutoConfiguration -@EnableConfigurationProperties(TdengineHistoryProperties.class) -public class TdengineHistoryAutoConfiguration { - - @Bean - @ConditionalOnMissingBean - @ConditionalOnProperty( - prefix = "lingniu.ingest.tdengine-history", - name = "enabled", - havingValue = "true") - public TdengineHistorySchema tdengineHistorySchema(TdengineHistoryProperties properties) { - return new TdengineHistorySchema(properties.getDatabase()); - } - - @Bean - @ConditionalOnMissingBean - @ConditionalOnProperty( - prefix = "lingniu.ingest.tdengine-history", - name = "enabled", - havingValue = "true") - public TdengineHistoryStatements tdengineHistoryStatements(TdengineHistorySchema schema) { - return new TdengineHistoryStatements(schema); - } - - @Bean - @ConditionalOnMissingBean(DataSource.class) - @ConditionalOnProperty( - prefix = "lingniu.ingest.tdengine-history", - name = "enabled", - havingValue = "true") - public DataSource tdengineDataSource(TdengineHistoryProperties properties) { - HikariConfig config = new HikariConfig(); - config.setPoolName("tdengine-history"); - config.setDriverClassName(properties.getDriverClassName()); - config.setJdbcUrl(properties.getJdbcUrl()); - config.setUsername(properties.getUsername()); - config.setPassword(properties.getPassword()); - config.setMaximumPoolSize(Math.max(1, properties.getMaximumPoolSize())); - config.setMinimumIdle(Math.max(0, Math.min(properties.getMinimumIdle(), properties.getMaximumPoolSize()))); - config.setConnectionTimeout(Math.max(250, properties.getConnectionTimeoutMillis())); - config.setInitializationFailTimeout(properties.getInitializationFailTimeoutMillis()); - return new HikariDataSource(config); - } - - @Bean - @ConditionalOnMissingBean - @ConditionalOnBean(DataSource.class) - @ConditionalOnProperty( - prefix = "lingniu.ingest.tdengine-history", - name = "enabled", - havingValue = "true") - public TdengineHistoryWriter tdengineHistoryWriter(DataSource dataSource, TdengineHistorySchema schema) { - return new TdengineJdbcHistoryWriter(dataSource, schema); - } - - @Bean - @ConditionalOnMissingBean - @ConditionalOnBean(DataSource.class) - @ConditionalOnProperty( - prefix = "lingniu.ingest.tdengine-history", - name = "enabled", - havingValue = "true") - public TdengineHistoryReader tdengineHistoryReader(DataSource dataSource, TdengineHistorySchema schema) { - return new TdengineJdbcHistoryReader(dataSource, schema); - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/config/TdengineHistoryProperties.java b/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/config/TdengineHistoryProperties.java deleted file mode 100644 index d19a28eb..00000000 --- a/modules/sinks/tdengine-history-store/src/main/java/com/lingniu/ingest/tdenginehistory/config/TdengineHistoryProperties.java +++ /dev/null @@ -1,119 +0,0 @@ -package com.lingniu.ingest.tdenginehistory.config; - -import org.springframework.boot.context.properties.ConfigurationProperties; - -@ConfigurationProperties(prefix = "lingniu.ingest.tdengine-history") -public class TdengineHistoryProperties { - - /** - * 默认关闭，避免未配置 TDengine 连接时影响现有历史查询服务启动。 - */ - private boolean enabled = false; - - /** - * TDengine 历史库名。 - */ - private String database = "vehicle_history"; - - /** - * TDengine RESTful JDBC 地址。默认走 6041，部署时可用 TDENGINE_JDBC_URL 覆盖。 - */ - private String jdbcUrl; - - private String username = "root"; - - private String password = "taosdata"; - - private String driverClassName = "com.taosdata.jdbc.rs.RestfulDriver"; - - private int maximumPoolSize = 16; - - private int minimumIdle = 0; - - private long connectionTimeoutMillis = 5000; - - private long initializationFailTimeoutMillis = -1; - - public boolean isEnabled() { - return enabled; - } - - public void setEnabled(boolean enabled) { - this.enabled = enabled; - } - - public String getDatabase() { - return database; - } - - public void setDatabase(String database) { - this.database = database; - } - - public String getJdbcUrl() { - if (jdbcUrl == null || jdbcUrl.isBlank()) { - return "jdbc:TAOS-RS://127.0.0.1:6041/" + database; - } - return jdbcUrl; - } - - public void setJdbcUrl(String jdbcUrl) { - this.jdbcUrl = jdbcUrl; - } - - public String getUsername() { - return username; - } - - public void setUsername(String username) { - this.username = username; - } - - public String getPassword() { - return password; - } - - public void setPassword(String password) { - this.password = password; - } - - public String getDriverClassName() { - return driverClassName; - } - - public void setDriverClassName(String driverClassName) { - this.driverClassName = driverClassName; - } - - public int getMaximumPoolSize() { - return maximumPoolSize; - } - - public void setMaximumPoolSize(int maximumPoolSize) { - this.maximumPoolSize = maximumPoolSize; - } - - public int getMinimumIdle() { - return minimumIdle; - } - - public void setMinimumIdle(int minimumIdle) { - this.minimumIdle = minimumIdle; - } - - public long getConnectionTimeoutMillis() { - return connectionTimeoutMillis; - } - - public void setConnectionTimeoutMillis(long connectionTimeoutMillis) { - this.connectionTimeoutMillis = connectionTimeoutMillis; - } - - public long getInitializationFailTimeoutMillis() { - return initializationFailTimeoutMillis; - } - - public void setInitializationFailTimeoutMillis(long initializationFailTimeoutMillis) { - this.initializationFailTimeoutMillis = initializationFailTimeoutMillis; - } -} diff --git a/modules/sinks/tdengine-history-store/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports b/modules/sinks/tdengine-history-store/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports deleted file mode 100644 index 2bd97e31..00000000 --- a/modules/sinks/tdengine-history-store/src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports +++ /dev/null @@ -1 +0,0 @@ -com.lingniu.ingest.tdenginehistory.config.TdengineHistoryAutoConfiguration diff --git a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineEnvelopeRowsTest.java b/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineEnvelopeRowsTest.java deleted file mode 100644 index dca1641f..00000000 --- a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineEnvelopeRowsTest.java +++ /dev/null @@ -1,173 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import com.lingniu.ingest.sink.kafka.proto.LocationPayload; -import com.lingniu.ingest.sink.kafka.proto.ParseStatusProto; -import com.lingniu.ingest.sink.kafka.proto.RawArchiveRef; -import com.lingniu.ingest.sink.kafka.proto.RawFrameFactPayload; -import com.lingniu.ingest.sink.kafka.proto.VehicleEnvelope; -import org.junit.jupiter.api.Test; - -import java.time.Instant; - -import static org.assertj.core.api.Assertions.assertThat; - -class TdengineEnvelopeRowsTest { - - @Test - void mapsRawFrameFactEnvelopeToRawFrameRow() { - VehicleEnvelope envelope = VehicleEnvelope.newBuilder() - .setEventId("evt-raw-1") - .setVin("VIN123") - .setSource("JT808") - .setEventTimeMs(1_772_000_001_234L) - .setIngestTimeMs(1_772_000_001_999L) - .putMetadata("channel", "tcp-808") - .setRawFrameFact(RawFrameFactPayload.newBuilder() - .setFrameId("frame-1") - .setVehicleKey("jt808:g7gps") - .setPhone("013800000000") - .setMessageId(0x0200) - .setSubType(0) - .setRawUri("archive://jt808/2026/06/29/frame-1.bin") - .setChecksum("sha256:abc") - .setRawSizeBytes(67) - .setParseStatus(ParseStatusProto.PARSE_STATUS_SUCCEEDED) - .setPeer("10.0.0.8:30001") - .putMetadata("auth", "passed")) - .build(); - - TdengineRawFrameRow row = TdengineEnvelopeRows.rawFrame(envelope).orElseThrow(); - - assertThat(row.ts()).isEqualTo(Instant.ofEpochMilli(1_772_000_001_234L)); - assertThat(row.receivedAt()).isEqualTo(Instant.ofEpochMilli(1_772_000_001_999L)); - assertThat(row.protocol()).isEqualTo("JT808"); - assertThat(row.vehicleKey()).isEqualTo("jt808:g7gps"); - assertThat(row.vin()).isEqualTo("VIN123"); - assertThat(row.phone()).isEqualTo("013800000000"); - assertThat(row.messageId()).isEqualTo(0x0200); - assertThat(row.rawUri()).isEqualTo("archive://jt808/2026/06/29/frame-1.bin"); - assertThat(row.metadataJson()).contains("\"channel\":\"tcp-808\"", "\"auth\":\"passed\""); - } - - @Test - void expandsRawFrameTsWithRawArchiveEventIdSequenceToAvoidSameMillisecondOverwrite() { - VehicleEnvelope first = rawFrameEnvelope("1782729152494000"); - VehicleEnvelope second = rawFrameEnvelope("1782729152494001"); - - TdengineRawFrameRow firstRow = TdengineEnvelopeRows.rawFrame(first).orElseThrow(); - TdengineRawFrameRow secondRow = TdengineEnvelopeRows.rawFrame(second).orElseThrow(); - - assertThat(firstRow.ts()).isEqualTo(Instant.ofEpochMilli(1_782_729_152_494L)); - assertThat(secondRow.ts()).isEqualTo(Instant.ofEpochMilli(1_782_729_152_495L)); - assertThat(firstRow.eventTime()).isEqualTo(Instant.ofEpochMilli(1_782_729_152_494L)); - assertThat(secondRow.eventTime()).isEqualTo(Instant.ofEpochMilli(1_782_729_152_494L)); - } - - @Test - void mapsLocationEnvelopeToVehicleLocationRow() { - VehicleEnvelope envelope = VehicleEnvelope.newBuilder() - .setEventId("evt-location-1") - .setVin("VIN32960") - .setSource("GB32960") - .setEventTimeMs(1_772_000_002_000L) - .setIngestTimeMs(1_772_000_002_321L) - .putMetadata("vehicle_key", "vin:VIN32960") - .putMetadata("frame_id", "frame-location-1") - .setRawArchive(RawArchiveRef.newBuilder() - .setUri("archive://gb32960/2026/06/29/frame-location-1.bin") - .setChecksum("sha256:def") - .setSizeBytes(98)) - .setLocation(LocationPayload.newBuilder() - .setLongitude(113.12) - .setLatitude(23.45) - .setAltitudeM(8.0) - .setSpeedKmh(42.5) - .setDirectionDeg(91.0) - .setAlarmFlag(1) - .setStatusFlag(3)) - .build(); - - TdengineLocationRow row = TdengineEnvelopeRows.location(envelope).orElseThrow(); - - assertThat(row.ts()).isEqualTo(Instant.ofEpochMilli(1_772_000_002_000L)); - assertThat(row.factId()).isEqualTo("evt-location-1"); - assertThat(row.frameId()).isEqualTo("frame-location-1"); - assertThat(row.protocol()).isEqualTo("GB32960"); - assertThat(row.vehicleKey()).isEqualTo("vin:VIN32960"); - assertThat(row.longitude()).isEqualTo(113.12); - assertThat(row.latitude()).isEqualTo(23.45); - assertThat(row.speedKmh()).isEqualTo(42.5); - assertThat(row.totalMileageKm()).isNull(); - assertThat(row.rawUri()).isEqualTo("archive://gb32960/2026/06/29/frame-location-1.bin"); - } - - @Test - void mapsLocationRawUriFromMetadataWhenRawArchiveRefIsAbsent() { - VehicleEnvelope envelope = VehicleEnvelope.newBuilder() - .setEventId("evt-location-metadata-raw") - .setVin("unknown") - .setSource("JT808") - .setEventTimeMs(1_772_000_002_000L) - .setIngestTimeMs(1_772_000_002_321L) - .putMetadata("phone", "13079963291") - .putMetadata("rawArchiveUri", "archive://jt808/metadata-only.bin") - .setLocation(LocationPayload.newBuilder() - .setLongitude(119.12) - .setLatitude(29.45)) - .build(); - - TdengineLocationRow row = TdengineEnvelopeRows.location(envelope).orElseThrow(); - - assertThat(row.rawUri()).isEqualTo("archive://jt808/metadata-only.bin"); - } - - @Test - void mapsJt808UnknownVinRowsToPhoneVehicleKey() { - VehicleEnvelope envelope = VehicleEnvelope.newBuilder() - .setEventId("evt-jt808-phone-1") - .setVin("unknown") - .setSource("JT808") - .setEventTimeMs(1_772_000_004_000L) - .setIngestTimeMs(1_772_000_004_500L) - .putMetadata("phone", "13079963291") - .putMetadata("frame_id", "frame-jt808-phone-1") - .setRawFrameFact(RawFrameFactPayload.newBuilder() - .setFrameId("frame-jt808-phone-1") - .setPhone("13079963291") - .setMessageId(0x0200) - .setRawUri("archive://jt808/frame-jt808-phone-1.bin") - .setChecksum("sha256:phone") - .setRawSizeBytes(88) - .setParseStatus(ParseStatusProto.PARSE_STATUS_SUCCEEDED)) - .setLocation(LocationPayload.newBuilder() - .setLongitude(119.1) - .setLatitude(29.2)) - .build(); - - assertThat(TdengineEnvelopeRows.rawFrame(envelope).orElseThrow().vehicleKey()) - .isEqualTo("jt808:13079963291"); - assertThat(TdengineEnvelopeRows.location(envelope).orElseThrow().vehicleKey()) - .isEqualTo("jt808:13079963291"); - } - - private static VehicleEnvelope rawFrameEnvelope(String rawArchiveEventId) { - return VehicleEnvelope.newBuilder() - .setEventId(rawArchiveEventId) - .setVin("unknown") - .setSource("JT808") - .setEventTimeMs(1_782_729_152_494L) - .setIngestTimeMs(1_782_729_152_539L) - .putMetadata("phone", "13079973200") - .putMetadata("rawArchiveEventId", rawArchiveEventId) - .setRawFrameFact(RawFrameFactPayload.newBuilder() - .setFrameId("rf-" + rawArchiveEventId) - .setPhone("13079973200") - .setMessageId(0x0200) - .setRawUri("archive://2026/06/29/JT808/unknown/" + rawArchiveEventId + ".bin") - .setChecksum("sha256:" + rawArchiveEventId) - .setRawSizeBytes(41) - .setParseStatus(ParseStatusProto.PARSE_STATUS_NOT_PARSED) - .putMetadata("rawArchiveEventId", rawArchiveEventId)) - .build(); - } -} diff --git a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryQueriesTest.java b/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryQueriesTest.java deleted file mode 100644 index 84f2b279..00000000 --- a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryQueriesTest.java +++ /dev/null @@ -1,95 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import org.junit.jupiter.api.Test; - -import java.time.Instant; - -import static org.assertj.core.api.Assertions.assertThat; - -class TdengineHistoryQueriesTest { - - private final TdengineHistorySchema schema = new TdengineHistorySchema("vehicle_history"); - private final TdengineHistoryQueries queries = new TdengineHistoryQueries(schema); - - @Test - void rawFrameQueryUsesVehicleChildTableAndKeysetPagination() { - TdengineRawFrameQuery query = new TdengineRawFrameQuery( - "JT808", - "jt808:g7gps", - Instant.parse("2026-06-29T00:00:00Z"), - Instant.parse("2026-06-30T00:00:00Z"), - TdengineQueryOrder.ASC, - 101, - new TdenginePageCursor(Instant.parse("2026-06-29T01:00:00Z"), "frame-100")); - - TdengineQueryStatement statement = queries.rawFrames(query); - - assertThat(statement.sql()) - .startsWith("SELECT ts, frame_id, received_at, message_id") - .contains(" FROM raw_jt808_") - .contains(" WHERE ts >= ? AND ts < ?") - .contains(" AND (ts > ? OR (ts = ? AND frame_id > ?))") - .contains(" ORDER BY ts ASC, frame_id ASC LIMIT ?"); - assertThat(statement.values()) - .containsExactly( - query.from(), query.to(), - query.cursor().ts(), query.cursor().ts(), query.cursor().tieBreaker(), - 101); - } - - @Test - void rawFrameQueryUsesStableWhenPhoneMessageIdOrParseStatusFiltersArePresent() { - TdengineRawFrameQuery query = new TdengineRawFrameQuery( - "JT808", - "", - "", - "13079963320", - 0x0100, - "SUCCEEDED", - Instant.parse("2026-06-29T00:00:00Z"), - Instant.parse("2026-06-30T00:00:00Z"), - TdengineQueryOrder.DESC, - 50, - new TdenginePageCursor(Instant.parse("2026-06-29T12:00:00Z"), "frame-100")); - - TdengineQueryStatement statement = queries.rawFrames(query); - - assertThat(statement.sql()) - .startsWith("SELECT ts, frame_id, received_at, message_id") - .contains(" FROM raw_frames") - .contains(" WHERE ts >= ? AND ts < ?") - .contains(" AND protocol = ?") - .contains(" AND phone = ?") - .contains(" AND message_id = ?") - .contains(" AND parse_status = ?") - .contains(" AND (ts < ? OR (ts = ? AND frame_id < ?))") - .contains(" ORDER BY ts DESC, frame_id DESC LIMIT ?"); - assertThat(statement.values()) - .containsExactly( - query.from(), query.to(), - "JT808", "13079963320", 0x0100, "SUCCEEDED", - query.cursor().ts(), query.cursor().ts(), query.cursor().tieBreaker(), - 50); - } - - @Test - void locationQueryCapsLimitAndUsesDescendingCursor() { - TdengineLocationQuery query = new TdengineLocationQuery( - "GB32960", - "vin:VIN123", - Instant.parse("2026-06-29T00:00:00Z"), - Instant.parse("2026-06-30T00:00:00Z"), - TdengineQueryOrder.DESC, - 5000, - new TdenginePageCursor(Instant.parse("2026-06-29T20:00:00Z"), "fact-200")); - - TdengineQueryStatement statement = queries.locations(query); - - assertThat(statement.sql()) - .contains(" FROM loc_gb32960_") - .contains(" AND (ts < ? OR (ts = ? AND fact_id < ?))") - .contains(" ORDER BY ts DESC, fact_id DESC LIMIT ?"); - assertThat(statement.values().getLast()).isEqualTo(1001); - } - -} diff --git a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineHistorySchemaTest.java b/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineHistorySchemaTest.java deleted file mode 100644 index 35931d48..00000000 --- a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineHistorySchemaTest.java +++ /dev/null @@ -1,66 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import org.junit.jupiter.api.Test; - -import java.nio.file.Files; -import java.nio.file.Path; - -import static org.assertj.core.api.Assertions.assertThat; - -class TdengineHistorySchemaTest { - - @Test - void bootstrapSqlCreatesOnlyRawAndLocationStablesFromSpec() { - TdengineHistorySchema schema = new TdengineHistorySchema("vehicle_history"); - - assertThat(schema.bootstrapSql()) - .first() - .isEqualTo("CREATE DATABASE IF NOT EXISTS vehicle_history PRECISION 'ms'"); - assertThat(schema.bootstrapSql()) - .contains("USE vehicle_history"); - assertThat(schema.bootstrapSql().get(2)) - .contains("CREATE STABLE IF NOT EXISTS raw_frames") - .contains("frame_id NCHAR(64)") - .contains("raw_uri NCHAR(512)") - .contains("metadata_json NCHAR(4096)") - .contains("parsed_json VARCHAR(16374)") - .contains("TAGS (") - .contains("protocol NCHAR(16)") - .contains("vehicle_key NCHAR(128)") - .contains("phone NCHAR(32)"); - assertThat(schema.bootstrapSql().get(3)) - .contains("CREATE STABLE IF NOT EXISTS vehicle_locations") - .contains("longitude DOUBLE") - .contains("total_mileage_km DOUBLE") - .contains("raw_uri NCHAR(512)"); - assertThat(schema.bootstrapSql()).hasSize(4); - assertThat(String.join("\n", schema.bootstrapSql())) - .doesNotContain("telemetry_fields") - .doesNotContain("field_key"); - } - - @Test - void childTableNamesAreStableAndSanitized() { - TdengineHistorySchema schema = new TdengineHistorySchema("vehicle_history"); - - assertThat(schema.rawFrameTable("JT808", "jt808:g7gps/013800000000")) - .matches("raw_jt808_[0-9a-f]{16}"); - assertThat(schema.locationTable("GB32960", "VIN WITH SPACE")) - .matches("loc_gb32960_[0-9a-f]{16}"); - } - - @Test - void historyStorePublicApiDoesNotExposeTelemetryFields() throws Exception { - String writer = Files.readString(Path.of( - "src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryWriter.java")); - String reader = Files.readString(Path.of( - "src/main/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryReader.java")); - - assertThat(writer) - .doesNotContain("TelemetryField") - .doesNotContain("appendTelemetry"); - assertThat(reader) - .doesNotContain("TelemetryField") - .doesNotContain("queryTelemetry"); - } -} diff --git a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryStatementsTest.java b/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryStatementsTest.java deleted file mode 100644 index af42941b..00000000 --- a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineHistoryStatementsTest.java +++ /dev/null @@ -1,85 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import org.junit.jupiter.api.Test; - -import java.time.Instant; - -import static org.assertj.core.api.Assertions.assertThat; - -class TdengineHistoryStatementsTest { - - private final TdengineHistorySchema schema = new TdengineHistorySchema("vehicle_history"); - private final TdengineHistoryStatements statements = new TdengineHistoryStatements(schema); - - @Test - void rawFrameBatchStatementCreatesChildTableAndParameterizedInsert() { - TdengineRawFrameRow row = new TdengineRawFrameRow( - Instant.parse("2026-06-29T05:00:01Z"), - "frame-1", - Instant.parse("2026-06-29T05:00:02Z"), - 0x0200, - 0, - Instant.parse("2026-06-29T05:00:01Z"), - "archive://jt808/frame-1.bin", - "sha256:abc", - 67, - "SUCCEEDED", - "", - "10.0.0.1:808", - "{\"auth\":\"passed\"}", - "{\"body\":{\"speedKmh\":42.0}}", - "JT808", - "jt808:g7gps", - "VIN123", - "013800000000"); - - TdengineBatchStatement batch = statements.rawFrame(row); - - assertThat(batch.createChildTableSql()) - .startsWith("CREATE TABLE IF NOT EXISTS raw_jt808_") - .contains(" USING raw_frames TAGS ('JT808', 'jt808:g7gps', 'VIN123', '013800000000')"); - assertThat(batch.insertSql()) - .startsWith("INSERT INTO raw_jt808_") - .contains("(ts, frame_id, received_at, message_id, sub_type, event_time, raw_uri, checksum, raw_size_bytes, parse_status, parse_error, peer, metadata_json, parsed_json)") - .endsWith("VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"); - assertThat(batch.values()) - .containsExactly( - row.ts(), row.frameId(), row.receivedAt(), row.messageId(), row.subType(), - row.eventTime(), row.rawUri(), row.checksum(), row.rawSizeBytes(), row.parseStatus(), - row.parseError(), row.peer(), row.metadataJson(), row.parsedJson()); - } - - @Test - void locationBatchStatementEscapesTags() { - TdengineLocationRow row = new TdengineLocationRow( - Instant.parse("2026-06-29T05:00:01Z"), - "fact-1", - "frame-1", - Instant.parse("2026-06-29T05:00:02Z"), - 113.1, - 23.4, - 8.0, - 42.0, - 90.0, - 1, - 3, - null, - "archive://gb32960/frame-1.bin", - "GB32960", - "vin:O'HARE", - "O'HARE", - ""); - - TdengineBatchStatement batch = statements.location(row); - - assertThat(batch.createChildTableSql()) - .startsWith("CREATE TABLE IF NOT EXISTS loc_gb32960_") - .contains(" USING vehicle_locations TAGS ('GB32960', 'vin:O''HARE', 'O''HARE', '')"); - assertThat(batch.values()) - .containsExactly( - row.ts(), row.factId(), row.frameId(), row.receivedAt(), row.longitude(), row.latitude(), - row.altitudeM(), row.speedKmh(), row.directionDeg(), row.alarmFlag(), row.statusFlag(), - row.totalMileageKm(), row.rawUri()); - } - -} diff --git a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryReaderTest.java b/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryReaderTest.java deleted file mode 100644 index 1ec60192..00000000 --- a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryReaderTest.java +++ /dev/null @@ -1,199 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import org.junit.jupiter.api.Test; - -import javax.sql.DataSource; -import java.lang.reflect.Proxy; -import java.sql.Connection; -import java.sql.PreparedStatement; -import java.sql.ResultSet; -import java.sql.SQLException; -import java.sql.Timestamp; -import java.time.Instant; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.LinkedHashMap; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class TdengineJdbcHistoryReaderTest { - - @Test - void readsRawFramesAndReturnsNextCursorWhenLimitHasMoreRow() throws Exception { - RecordingQueryJdbc jdbc = new RecordingQueryJdbc(); - jdbc.rows.add(rawFrameResult("frame-1", "2026-06-29T05:00:01Z")); - jdbc.rows.add(rawFrameResult("frame-2", "2026-06-29T05:00:02Z")); - TdengineJdbcHistoryReader reader = new TdengineJdbcHistoryReader( - jdbc.dataSource(), new TdengineHistorySchema("vehicle_history")); - - TdenginePage page = reader.queryRawFrames(new TdengineRawFrameQuery( - "JT808", - "jt808:g7gps", - Instant.parse("2026-06-29T00:00:00Z"), - Instant.parse("2026-06-30T00:00:00Z"), - TdengineQueryOrder.ASC, - 1, - null)); - - assertThat(page.items()) - .extracting(TdengineRawFrameRow::frameId) - .containsExactly("frame-1"); - assertThat(page.nextCursor()) - .contains(new TdenginePageCursor(Instant.parse("2026-06-29T05:00:01Z"), "frame-1")); - assertThat(jdbc.preparedSql).contains("LIMIT ?"); - assertThat(jdbc.boundValues.get(3)).isEqualTo(2); - } - - @Test - void readsLocationRowsWithoutNextCursorWhenPageIsComplete() throws Exception { - RecordingQueryJdbc jdbc = new RecordingQueryJdbc(); - jdbc.rows.add(locationResult("fact-1", "2026-06-29T05:00:01Z")); - TdengineJdbcHistoryReader reader = new TdengineJdbcHistoryReader( - jdbc.dataSource(), new TdengineHistorySchema("vehicle_history")); - - TdenginePage page = reader.queryLocations(new TdengineLocationQuery( - "GB32960", - "vin:VIN123", - Instant.parse("2026-06-29T00:00:00Z"), - Instant.parse("2026-06-30T00:00:00Z"), - TdengineQueryOrder.DESC, - 10, - null)); - - assertThat(page.items()) - .extracting(TdengineLocationRow::factId) - .containsExactly("fact-1"); - assertThat(page.nextCursor()).isEmpty(); - assertThat(page.items().getFirst().longitude()).isEqualTo(113.12); - } - - @Test - void returnsEmptyPageWhenTdengineChildTableDoesNotExistInNestedCause() throws Exception { - RecordingQueryJdbc jdbc = new RecordingQueryJdbc(); - jdbc.executeQueryFailure = new SQLException("wrapper", - new RuntimeException("TDengine error: table not exist")); - TdengineJdbcHistoryReader reader = new TdengineJdbcHistoryReader( - jdbc.dataSource(), new TdengineHistorySchema("vehicle_history")); - - TdenginePage page = reader.queryLocations(new TdengineLocationQuery( - "JT808", - "jt808:g7gps", - Instant.parse("2026-06-29T00:00:00Z"), - Instant.parse("2026-06-30T00:00:00Z"), - TdengineQueryOrder.ASC, - 10, - null)); - - assertThat(page.items()).isEmpty(); - assertThat(page.nextCursor()).isEmpty(); - } - - private static Map rawFrameResult(String frameId, String ts) { - Map row = new LinkedHashMap<>(); - row.put("ts", Timestamp.from(Instant.parse(ts))); - row.put("frame_id", frameId); - row.put("received_at", Timestamp.from(Instant.parse(ts).plusMillis(500))); - row.put("message_id", 0x0200); - row.put("sub_type", 0); - row.put("event_time", Timestamp.from(Instant.parse(ts))); - row.put("raw_uri", "archive://jt808/" + frameId + ".bin"); - row.put("checksum", "sha256:" + frameId); - row.put("raw_size_bytes", 67L); - row.put("parse_status", "SUCCEEDED"); - row.put("parse_error", ""); - row.put("peer", "10.0.0.1:808"); - row.put("metadata_json", "{}"); - row.put("protocol", "JT808"); - row.put("vehicle_key", "jt808:g7gps"); - row.put("vin", "VIN123"); - row.put("phone", "013800000000"); - return row; - } - - private static Map locationResult(String factId, String ts) { - Map row = new LinkedHashMap<>(); - row.put("ts", Timestamp.from(Instant.parse(ts))); - row.put("fact_id", factId); - row.put("frame_id", "frame-1"); - row.put("received_at", Timestamp.from(Instant.parse(ts).plusMillis(500))); - row.put("longitude", 113.12); - row.put("latitude", 23.45); - row.put("altitude_m", 8.0); - row.put("speed_kmh", 42.5); - row.put("direction_deg", 90.0); - row.put("alarm_flag", 1L); - row.put("status_flag", 3L); - row.put("total_mileage_km", null); - row.put("raw_uri", "archive://gb32960/frame-1.bin"); - row.put("metadata_json", "{}"); - row.put("protocol", "GB32960"); - row.put("vehicle_key", "vin:VIN123"); - row.put("vin", "VIN123"); - row.put("phone", ""); - return row; - } - - private static final class RecordingQueryJdbc { - private final List> rows = new ArrayList<>(); - private final Map boundValues = new HashMap<>(); - private SQLException executeQueryFailure; - private String preparedSql; - - DataSource dataSource() { - return (DataSource) Proxy.newProxyInstance( - getClass().getClassLoader(), - new Class[]{DataSource.class}, - (proxy, method, args) -> switch (method.getName()) { - case "getConnection" -> connection(); - default -> null; - }); - } - - private Connection connection() { - return (Connection) Proxy.newProxyInstance( - getClass().getClassLoader(), - new Class[]{Connection.class}, - (proxy, method, args) -> switch (method.getName()) { - case "prepareStatement" -> preparedStatement((String) args[0]); - case "getAutoCommit" -> true; - default -> null; - }); - } - - private PreparedStatement preparedStatement(String sql) { - preparedSql = sql; - return (PreparedStatement) Proxy.newProxyInstance( - getClass().getClassLoader(), - new Class[]{PreparedStatement.class}, - (proxy, method, args) -> switch (method.getName()) { - case "setTimestamp", "setObject" -> { - boundValues.put((Integer) args[0], args[1]); - yield null; - } - case "executeQuery" -> { - if (executeQueryFailure != null) { - throw executeQueryFailure; - } - yield resultSet(); - } - default -> null; - }); - } - - private ResultSet resultSet() { - final int[] index = {-1}; - return (ResultSet) Proxy.newProxyInstance( - getClass().getClassLoader(), - new Class[]{ResultSet.class}, - (proxy, method, args) -> switch (method.getName()) { - case "next" -> ++index[0] < rows.size(); - case "getTimestamp", "getString", "getInt", "getLong", "getDouble", "getObject" -> - rows.get(index[0]).get((String) args[0]); - case "wasNull" -> false; - default -> null; - }); - } - } -} diff --git a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryWriterTest.java b/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryWriterTest.java deleted file mode 100644 index bd91e249..00000000 --- a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/TdengineJdbcHistoryWriterTest.java +++ /dev/null @@ -1,307 +0,0 @@ -package com.lingniu.ingest.tdenginehistory; - -import org.junit.jupiter.api.Test; - -import javax.sql.DataSource; -import java.lang.reflect.Proxy; -import java.sql.Connection; -import java.sql.PreparedStatement; -import java.sql.SQLException; -import java.sql.Statement; -import java.sql.Timestamp; -import java.time.Instant; -import java.util.ArrayList; -import java.util.HashMap; -import java.util.List; -import java.util.Map; - -import static org.assertj.core.api.Assertions.assertThat; - -class TdengineJdbcHistoryWriterTest { - - private final TdengineHistorySchema schema = new TdengineHistorySchema("vehicle_history"); - - @Test - void initializesSchemaWithBootstrapSql() throws Exception { - RecordingJdbc jdbc = new RecordingJdbc(); - TdengineJdbcHistoryWriter writer = new TdengineJdbcHistoryWriter(jdbc.dataSource(), schema); - - writer.initializeSchema(); - - assertThat(jdbc.executedSql) - .containsExactlyElementsOf(schema.bootstrapSql()); - assertThat(jdbc.commits).isEqualTo(1); - } - - @Test - void writesRawFramesInBatchesPerChildTable() throws Exception { - RecordingJdbc jdbc = new RecordingJdbc(); - TdengineJdbcHistoryWriter writer = new TdengineJdbcHistoryWriter(jdbc.dataSource(), schema); - TdengineRawFrameRow first = rawFrame("frame-1", Instant.parse("2026-06-29T05:00:01Z"), "none"); - TdengineRawFrameRow second = rawFrame("frame-2", Instant.parse("2026-06-29T05:00:02Z"), "none"); - - writer.appendRawFrames(List.of(first, second)); - - assertThat(jdbc.executedSql).startsWith(schema.bootstrapSql().toArray(String[]::new)); - assertThat(jdbc.executedSql) - .filteredOn(sql -> sql.contains("USING raw_frames TAGS")) - .hasSize(1); - assertThat(jdbc.preparedBatches).isEmpty(); - assertThat(jdbc.executedSql) - .anyMatch(sql -> sql.startsWith("INSERT INTO raw_jt808_") - && sql.contains(Long.toString(first.ts().toEpochMilli())) - && sql.contains("frame-1") - && sql.contains("frame-2") - && sql.contains(") (")); - assertThat(jdbc.commits).isEqualTo(1); - } - - @Test - void writesLocationRowsWithNullableMileage() throws Exception { - RecordingJdbc jdbc = new RecordingJdbc(); - TdengineJdbcHistoryWriter writer = new TdengineJdbcHistoryWriter(jdbc.dataSource(), schema); - TdengineLocationRow location = new TdengineLocationRow( - Instant.parse("2026-06-29T05:00:01Z"), - "fact-1", - "frame-1", - Instant.parse("2026-06-29T05:00:02Z"), - 113.1, - 23.4, - 8.0, - 42.0, - 90.0, - 1, - 3, - null, - "archive://gb32960/frame-1.bin", - "GB32960", - "vin:VIN123", - "VIN123", - ""); - - writer.appendLocations(List.of(location)); - - assertThat(jdbc.executedSql).startsWith(schema.bootstrapSql().toArray(String[]::new)); - assertThat(jdbc.executedSql) - .anyMatch(sql -> sql.contains("USING vehicle_locations TAGS")); - assertThat(jdbc.preparedBatches).isEmpty(); - assertThat(jdbc.executedSql) - .anyMatch(sql -> sql.startsWith("INSERT INTO loc_gb32960_") - && sql.contains("fact-1") - && sql.contains("NULL") - && sql.contains("archive://gb32960/frame-1.bin")); - } - - @Test - void schemaBootstrapRunsOnlyOnceAcrossWrites() throws Exception { - RecordingJdbc jdbc = new RecordingJdbc(); - TdengineJdbcHistoryWriter writer = new TdengineJdbcHistoryWriter(jdbc.dataSource(), schema); - - writer.appendRawFrames(List.of(rawFrame("frame-1", Instant.parse("2026-06-29T05:00:01Z")))); - writer.appendRawFrames(List.of(rawFrame("frame-2", Instant.parse("2026-06-29T05:00:02Z")))); - - assertThat(jdbc.executedSql) - .filteredOn(sql -> sql.startsWith("CREATE DATABASE")) - .hasSize(1); - assertThat(jdbc.commits).isEqualTo(2); - } - - @Test - void writesLiteralInsertWithoutPreparedBatch() throws Exception { - RecordingJdbc jdbc = new RecordingJdbc(); - jdbc.failPreparedBatches = true; - TdengineJdbcHistoryWriter writer = new TdengineJdbcHistoryWriter(jdbc.dataSource(), schema); - - writer.appendRawFrames(List.of(rawFrame("frame-1", Instant.parse("2026-06-29T05:00:01Z"), "none"))); - - assertThat(jdbc.preparedBatches).isEmpty(); - assertThat(jdbc.executedSql) - .anyMatch(sql -> sql.startsWith("INSERT INTO raw_jt808_") - && sql.contains("parse_error") - && sql.contains("'SUCCEEDED', 'none', '10.0.0.1:808'")); - assertThat(jdbc.commits).isEqualTo(1); - } - - @Test - void usesLiteralInsertForRowsWithEmptyStrings() throws Exception { - RecordingJdbc jdbc = new RecordingJdbc(); - TdengineJdbcHistoryWriter writer = new TdengineJdbcHistoryWriter(jdbc.dataSource(), schema); - - writer.appendRawFrames(List.of(rawFrame("frame-1", Instant.parse("2026-06-29T05:00:01Z")))); - - assertThat(jdbc.preparedBatches).isEmpty(); - assertThat(jdbc.executedSql) - .anyMatch(sql -> sql.startsWith("INSERT INTO raw_jt808_") - && sql.contains("parse_error") - && sql.contains("'SUCCEEDED', '', '10.0.0.1:808'")); - assertThat(jdbc.commits).isEqualTo(1); - } - - @Test - void groupsLiteralRowsForSameChildTableIntoOneInsert() throws Exception { - RecordingJdbc jdbc = new RecordingJdbc(); - TdengineJdbcHistoryWriter writer = new TdengineJdbcHistoryWriter(jdbc.dataSource(), schema); - - writer.appendRawFrames(List.of( - rawFrame("frame-1", Instant.parse("2026-06-29T05:00:01Z")), - rawFrame("frame-2", Instant.parse("2026-06-29T05:00:02Z")))); - - List inserts = jdbc.executedSql.stream() - .filter(sql -> sql.startsWith("INSERT INTO raw_jt808_")) - .toList(); - assertThat(inserts).hasSize(1); - assertThat(inserts.getFirst()) - .contains("frame-1") - .contains("frame-2") - .contains(") ("); - } - - @Test - void bumpsDuplicateTimestampsForSameChildTable() throws Exception { - RecordingJdbc jdbc = new RecordingJdbc(); - TdengineJdbcHistoryWriter writer = new TdengineJdbcHistoryWriter(jdbc.dataSource(), schema); - Instant ts = Instant.parse("2026-06-29T05:00:01Z"); - - writer.appendRawFrames(List.of( - rawFrame("frame-1", ts), - rawFrame("frame-2", ts))); - - String insert = jdbc.executedSql.stream() - .filter(sql -> sql.startsWith("INSERT INTO raw_jt808_")) - .findFirst() - .orElseThrow(); - assertThat(insert) - .contains("(" + ts.toEpochMilli() + ", 'frame-1'") - .contains("(" + (ts.toEpochMilli() + 1) + ", 'frame-2'"); - } - - private static TdengineRawFrameRow rawFrame(String frameId, Instant ts) { - return rawFrame(frameId, ts, ""); - } - - private static TdengineRawFrameRow rawFrame(String frameId, Instant ts, String parseError) { - return new TdengineRawFrameRow( - ts, - frameId, - ts.plusSeconds(1), - 0x0200, - 0, - ts, - "archive://jt808/" + frameId + ".bin", - "sha256:" + frameId, - 67, - "SUCCEEDED", - parseError, - "10.0.0.1:808", - "{}", - "{\"frame\":\"" + frameId + "\"}", - "JT808", - "jt808:g7gps", - "VIN123", - "013800000000"); - } - - private static final class RecordingJdbc { - private final List executedSql = new ArrayList<>(); - private final Map preparedBatches = new HashMap<>(); - private boolean failPreparedBatches; - private int commits; - private int rollbacks; - - DataSource dataSource() { - return (DataSource) Proxy.newProxyInstance( - getClass().getClassLoader(), - new Class[]{DataSource.class}, - (proxy, method, args) -> switch (method.getName()) { - case "getConnection" -> connection(); - case "unwrap" -> null; - case "isWrapperFor" -> false; - default -> defaultValue(method.getReturnType()); - }); - } - - private Connection connection() { - return (Connection) Proxy.newProxyInstance( - getClass().getClassLoader(), - new Class[]{Connection.class}, - (proxy, method, args) -> switch (method.getName()) { - case "createStatement" -> statement(); - case "prepareStatement" -> preparedStatement((String) args[0]); - case "commit" -> { - commits++; - yield null; - } - case "rollback" -> { - rollbacks++; - yield null; - } - case "getAutoCommit" -> true; - case "unwrap" -> null; - case "isWrapperFor" -> false; - default -> defaultValue(method.getReturnType()); - }); - } - - private Statement statement() { - return (Statement) Proxy.newProxyInstance( - getClass().getClassLoader(), - new Class[]{Statement.class}, - (proxy, method, args) -> switch (method.getName()) { - case "execute", "executeUpdate" -> { - executedSql.add((String) args[0]); - yield method.getReturnType() == boolean.class ? true : 1; - } - case "unwrap" -> null; - case "isWrapperFor" -> false; - default -> defaultValue(method.getReturnType()); - }); - } - - private PreparedStatement preparedStatement(String sql) { - PreparedBatch batch = preparedBatches.computeIfAbsent(sql, PreparedBatch::new); - Map current = new HashMap<>(); - return (PreparedStatement) Proxy.newProxyInstance( - getClass().getClassLoader(), - new Class[]{PreparedStatement.class}, - (proxy, method, args) -> switch (method.getName()) { - case "setTimestamp", "setObject" -> { - current.put((Integer) args[0], args[1]); - yield null; - } - case "addBatch" -> { - batch.rows().add(new HashMap<>(current)); - current.clear(); - yield null; - } - case "executeBatch" -> { - if (failPreparedBatches) { - throw new SQLException("simulated prepared batch failure"); - } - yield new int[batch.rows().size()]; - } - case "unwrap" -> null; - case "isWrapperFor" -> false; - default -> defaultValue(method.getReturnType()); - }); - } - - private static Object defaultValue(Class type) { - if (type == boolean.class) { - return false; - } - if (type == int.class) { - return 0; - } - if (type == long.class) { - return 0L; - } - return null; - } - - record PreparedBatch(String sql, List> rows) { - PreparedBatch(String sql) { - this(sql, new ArrayList<>()); - } - } - } -} diff --git a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/config/TdengineHistoryAutoConfigurationTest.java b/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/config/TdengineHistoryAutoConfigurationTest.java deleted file mode 100644 index 8117f110..00000000 --- a/modules/sinks/tdengine-history-store/src/test/java/com/lingniu/ingest/tdenginehistory/config/TdengineHistoryAutoConfigurationTest.java +++ /dev/null @@ -1,81 +0,0 @@ -package com.lingniu.ingest.tdenginehistory.config; - -import com.lingniu.ingest.tdenginehistory.TdengineHistorySchema; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryReader; -import com.lingniu.ingest.tdenginehistory.TdengineHistoryWriter; -import com.zaxxer.hikari.HikariDataSource; -import org.junit.jupiter.api.Test; -import org.springframework.boot.autoconfigure.AutoConfigurations; -import org.springframework.boot.test.context.runner.ApplicationContextRunner; - -import javax.sql.DataSource; - -import static org.assertj.core.api.Assertions.assertThat; -import static org.mockito.Mockito.mock; - -class TdengineHistoryAutoConfigurationTest { - - private final ApplicationContextRunner contextRunner = new ApplicationContextRunner() - .withConfiguration(AutoConfigurations.of(TdengineHistoryAutoConfiguration.class)); - - @Test - void staysOffByDefault() { - contextRunner.run(context -> { - assertThat(context).doesNotHaveBean(TdengineHistorySchema.class); - assertThat(context).doesNotHaveBean(DataSource.class); - assertThat(context.getBean(TdengineHistoryProperties.class).isEnabled()).isFalse(); - }); - } - - @Test - void createsSchemaWhenEnabled() { - contextRunner - .withPropertyValues( - "lingniu.ingest.tdengine-history.enabled=true", - "lingniu.ingest.tdengine-history.database=vehicle_history_hot") - .run(context -> { - assertThat(context).hasSingleBean(TdengineHistorySchema.class); - assertThat(context.getBean(TdengineHistorySchema.class).bootstrapSql().getFirst()) - .isEqualTo("CREATE DATABASE IF NOT EXISTS vehicle_history_hot PRECISION 'ms'"); - }); - } - - @Test - void createsWriterWhenEnabledAndDataSourceExists() { - contextRunner - .withBean(DataSource.class, () -> mock(DataSource.class)) - .withPropertyValues("lingniu.ingest.tdengine-history.enabled=true") - .run(context -> assertThat(context).hasSingleBean(TdengineHistoryWriter.class)); - } - - @Test - void createsReaderWhenEnabledAndDataSourceExists() { - contextRunner - .withBean(DataSource.class, () -> mock(DataSource.class)) - .withPropertyValues("lingniu.ingest.tdengine-history.enabled=true") - .run(context -> assertThat(context).hasSingleBean(TdengineHistoryReader.class)); - } - - @Test - void createsTdengineDataSourceWhenEnabled() { - contextRunner - .withPropertyValues( - "lingniu.ingest.tdengine-history.enabled=true", - "lingniu.ingest.tdengine-history.jdbc-url=jdbc:TAOS-RS://tdengine:6041/vehicle_history", - "lingniu.ingest.tdengine-history.username=root", - "lingniu.ingest.tdengine-history.password=secret", - "lingniu.ingest.tdengine-history.maximum-pool-size=24", - "lingniu.ingest.tdengine-history.minimum-idle=3", - "lingniu.ingest.tdengine-history.initialization-fail-timeout-millis=-1") - .run(context -> { - assertThat(context).hasSingleBean(DataSource.class); - HikariDataSource dataSource = context.getBean(HikariDataSource.class); - assertThat(dataSource.getJdbcUrl()).isEqualTo("jdbc:TAOS-RS://tdengine:6041/vehicle_history"); - assertThat(dataSource.getUsername()).isEqualTo("root"); - assertThat(dataSource.getMaximumPoolSize()).isEqualTo(24); - assertThat(dataSource.getMinimumIdle()).isEqualTo(3); - assertThat(context).hasSingleBean(TdengineHistoryWriter.class); - assertThat(context).hasSingleBean(TdengineHistoryReader.class); - }); - } -} diff --git a/modules/testing/vehicle-identity-test-support/pom.xml b/modules/testing/vehicle-identity-test-support/pom.xml deleted file mode 100644 index 31fad0dc..00000000 --- a/modules/testing/vehicle-identity-test-support/pom.xml +++ /dev/null @@ -1,20 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - ../../../pom.xml - - vehicle-identity-test-support - vehicle-identity-test-support - 车辆身份解析测试夹具，仅供协议和接入模块测试依赖。 - - - - com.lingniu.ingest - vehicle-identity - - - diff --git a/modules/testing/vehicle-identity-test-support/src/main/java/com/lingniu/ingest/identity/InMemoryVehicleIdentityService.java b/modules/testing/vehicle-identity-test-support/src/main/java/com/lingniu/ingest/identity/InMemoryVehicleIdentityService.java deleted file mode 100644 index b9d97bc3..00000000 --- a/modules/testing/vehicle-identity-test-support/src/main/java/com/lingniu/ingest/identity/InMemoryVehicleIdentityService.java +++ /dev/null @@ -1,48 +0,0 @@ -package com.lingniu.ingest.identity; - -import java.util.List; -import java.util.concurrent.CopyOnWriteArrayList; - -public final class InMemoryVehicleIdentityService implements VehicleIdentityResolver, VehicleIdentityRegistry { - - private final List bindings = new CopyOnWriteArrayList<>(); - - @Override - public VehicleIdentity resolve(VehicleIdentityLookup lookup) { - if (lookup == null) { - return new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN); - } - if (!lookup.vin().isBlank() && !"unknown".equalsIgnoreCase(lookup.vin())) { - return new VehicleIdentity(lookup.vin(), true, VehicleIdentitySource.EXPLICIT_VIN); - } - for (VehicleIdentityBinding binding : bindings) { - if (binding.protocol() != lookup.protocol()) { - continue; - } - if (!lookup.phone().isBlank() && lookup.phone().equals(binding.phone())) { - return new VehicleIdentity(binding.vin(), true, VehicleIdentitySource.BOUND_PHONE); - } - if (!lookup.deviceId().isBlank() && lookup.deviceId().equals(binding.deviceId())) { - return new VehicleIdentity(binding.vin(), true, VehicleIdentitySource.BOUND_DEVICE_ID); - } - if (!lookup.plate().isBlank() && lookup.plate().equals(binding.plate())) { - return new VehicleIdentity(binding.vin(), true, VehicleIdentitySource.BOUND_PLATE); - } - } - if (!lookup.deviceId().isBlank()) { - return new VehicleIdentity(lookup.deviceId(), false, VehicleIdentitySource.FALLBACK_DEVICE_ID); - } - if (!lookup.phone().isBlank()) { - return new VehicleIdentity(lookup.phone(), false, VehicleIdentitySource.FALLBACK_PHONE); - } - if (!lookup.plate().isBlank()) { - return new VehicleIdentity(lookup.plate(), false, VehicleIdentitySource.FALLBACK_PLATE); - } - return new VehicleIdentity("unknown", false, VehicleIdentitySource.UNKNOWN); - } - - @Override - public void bind(VehicleIdentityBinding binding) { - bindings.add(binding); - } -} diff --git a/pom.xml b/pom.xml deleted file mode 100644 index 464b70c6..00000000 --- a/pom.xml +++ /dev/null @@ -1,379 +0,0 @@ - - - 4.0.0 - - com.lingniu.ingest - lingniu-vehicle-ingest - 0.1.0-SNAPSHOT - pom - - lingniu-vehicle-ingest - - 羚牛车辆数据接入平台 v2。 - 模块化原子能力 + 协议解耦 + 业务/实时解耦 + 生产链路只支持 Kafka。 - Java 25 + Spring Boot 3.5.3 + Netty 4.2.9.Final + Disruptor + Virtual Threads。 - - - - modules/core/ingest-api - modules/core/ingest-facts - modules/core/ingest-codec-common - modules/core/ingest-core - modules/core/session-core - modules/core/vehicle-identity - modules/testing/vehicle-identity-test-support - modules/core/observability - modules/sinks/sink-kafka - modules/sinks/sink-archive - modules/sinks/tdengine-history-store - modules/services/event-history-service - modules/services/vehicle-stat-service - modules/protocols/protocol-gb32960 - modules/protocols/protocol-jt808 - modules/inbound/inbound-mqtt - modules/apps/gb32960-ingest-app - modules/apps/jt808-ingest-app - modules/apps/yutong-mqtt-app - modules/apps/vehicle-history-app - modules/apps/vehicle-analytics-app - - - - - optional-command-gateway - - modules/protocols/protocol-jt1078 - modules/apps/command-gateway - - - - - com.lingniu.ingest - protocol-jt1078 - ${project.version} - - - com.lingniu.ingest - command-gateway - ${project.version} - - - - - - optional-attachments - - modules/protocols/protocol-jsatl12 - - - - - com.lingniu.ingest - protocol-jsatl12 - ${project.version} - - - - - - optional-latest-state - - modules/services/vehicle-state-service - - - - - com.lingniu.ingest - vehicle-state-service - ${project.version} - - - - - - - - UTF-8 - UTF-8 - 25 - 25 - 25 - 25 - - 3.5.3 - 2025.0.0.0 - 4.2.9.Final - 4.0.0 - 4.28.3 - 3.8.1 - 3.1.8 - 2.2.0 - 1.13.6 - 1.2.5 - 3.8.4 - 2.8.17 - 5.11.3 - 3.26.3 - 5.14.2 - - - - - - - - io.netty - netty-bom - ${netty.version} - pom - import - - - org.springframework.boot - spring-boot-dependencies - ${spring-boot.version} - pom - import - - - com.alibaba.cloud - spring-cloud-alibaba-dependencies - ${spring-cloud-alibaba.version} - pom - import - - - io.github.resilience4j - resilience4j-bom - ${resilience4j.version} - pom - import - - - io.micrometer - micrometer-bom - ${micrometer.version} - pom - import - - - org.junit - junit-bom - ${junit.version} - pom - import - - - - - com.lingniu.ingest - ingest-api - ${project.version} - - - com.lingniu.ingest - ingest-facts - ${project.version} - - - com.lingniu.ingest - ingest-core - ${project.version} - - - com.lingniu.ingest - ingest-codec-common - ${project.version} - - - com.lingniu.ingest - session-core - ${project.version} - - - com.lingniu.ingest - vehicle-identity - ${project.version} - - - com.lingniu.ingest - vehicle-identity-test-support - ${project.version} - - - com.lingniu.ingest - observability - ${project.version} - - - com.lingniu.ingest - sink-kafka - ${project.version} - - - com.lingniu.ingest - sink-archive - ${project.version} - - - com.lingniu.ingest - tdengine-history-store - ${project.version} - - - com.lingniu.ingest - event-history-service - ${project.version} - - - com.lingniu.ingest - vehicle-stat-service - ${project.version} - - - com.lingniu.ingest - protocol-gb32960 - ${project.version} - - - com.lingniu.ingest - protocol-jt808 - ${project.version} - - - com.lingniu.ingest - inbound-mqtt - ${project.version} - - - com.lingniu.ingest - gb32960-ingest-app - ${project.version} - - - com.lingniu.ingest - jt808-ingest-app - ${project.version} - - - com.lingniu.ingest - yutong-mqtt-app - ${project.version} - - - com.lingniu.ingest - vehicle-history-app - ${project.version} - - - com.lingniu.ingest - vehicle-analytics-app - ${project.version} - - - - - com.lmax - disruptor - ${disruptor.version} - - - com.google.protobuf - protobuf-java - ${protobuf.version} - - - com.google.protobuf - protobuf-java-util - ${protobuf.version} - - - org.apache.kafka - kafka-clients - ${kafka.version} - - - org.eclipse.paho - org.eclipse.paho.client.mqttv3 - ${paho-mqtt.version} - - - com.taosdata.jdbc - taos-jdbcdriver - ${taos-jdbcdriver.version} - - - commons-logging - commons-logging - - - - - org.springdoc - springdoc-openapi-starter-webmvc-ui - ${springdoc-openapi.version} - - - com.github.ben-manes.caffeine - caffeine - ${caffeine.version} - - - org.assertj - assertj-core - ${assertj.version} - test - - - org.mockito - mockito-core - ${mockito.version} - test - - - - - - - - - org.apache.maven.plugins - maven-compiler-plugin - 3.13.0 - - ${java.version} - true - - - - org.apache.maven.plugins - maven-surefire-plugin - 3.5.2 - - - org.springframework.boot - spring-boot-maven-plugin - ${spring-boot.version} - - --sun-misc-unsafe-memory-access=allow -Dio.netty.transport.noNative=true - - - - - repackage - build-info - - - - - - - - diff --git a/reference/GBT+32960.3-2016.pdf b/reference/GBT+32960.3-2016.pdf deleted file mode 100644 index 8b9a945e..00000000 Binary files a/reference/GBT+32960.3-2016.pdf and /dev/null differ diff --git a/reference/GBT+32960.3-2025.pdf b/reference/GBT+32960.3-2025.pdf deleted file mode 100644 index c29abb89..00000000 Binary files a/reference/GBT+32960.3-2025.pdf and /dev/null differ diff --git a/reference/广东燃料电池汽车示范应用城市群综合监管平台-燃料电池汽车数据接入技术规范v1.0.220822.pdf b/reference/广东燃料电池汽车示范应用城市群综合监管平台-燃料电池汽车数据接入技术规范v1.0.220822.pdf deleted file mode 100644 index ab50e811..00000000 Binary files a/reference/广东燃料电池汽车示范应用城市群综合监管平台-燃料电池汽车数据接入技术规范v1.0.220822.pdf and /dev/null differ diff --git a/tools/gb32960_e2e_smoke.py b/tools/gb32960_e2e_smoke.py deleted file mode 100644 index 8d884a5e..00000000 --- a/tools/gb32960_e2e_smoke.py +++ /dev/null @@ -1,358 +0,0 @@ -#!/usr/bin/env python3 -"""GB32960 TCP ingest to history/archive smoke verifier.""" - -from __future__ import annotations - -import argparse -import datetime as dt -import json -import os -import pathlib -import socket -import time -import urllib.parse -import urllib.request -from dataclasses import dataclass -from typing import Any - -try: - from tools import tdengine_smoke -except ModuleNotFoundError: - import tdengine_smoke - - -SHANGHAI = dt.timezone(dt.timedelta(hours=8)) -DEFAULT_SAMPLE = ( - "modules/protocols/protocol-gb32960/src/test/resources/samples/gb32960/realtime_001.hex" -) -DEFAULT_FIELD_KEYS = "VEHICLE.speedKmh,VEHICLE.totalMileageKm,POSITION_V2016.longitude,POSITION_V2016.latitude" - - -@dataclass(frozen=True) -class FrameInfo: - vin: str - command: int - event_time: dt.datetime | None - - -def read_hex_frame(path: pathlib.Path) -> bytes: - text = path.read_text(encoding="utf-8") - return bytes.fromhex("".join(text.split())) - - -def parse_frame_info(frame: bytes) -> FrameInfo: - if len(frame) < 25: - raise ValueError(f"gb32960 frame too short: {len(frame)}") - if frame[:2] not in (b"##", b"$$"): - raise ValueError("gb32960 frame must start with ## or $$") - data_len = int.from_bytes(frame[22:24], "big") - expected_len = 24 + data_len + 1 - if len(frame) < expected_len: - raise ValueError(f"gb32960 frame incomplete: len={len(frame)} expected={expected_len}") - command = frame[2] - vin = frame[4:21].decode("ascii", errors="ignore").strip() - event_time = None - if command in (0x01, 0x02, 0x03, 0x04, 0x05) and data_len >= 6: - event_time = parse_collect_time(frame[24:30]) - return FrameInfo(vin=vin, command=command, event_time=event_time) - - -def parse_collect_time(raw: bytes) -> dt.datetime: - if len(raw) != 6: - raise ValueError("gb32960 collect time must be 6 bytes") - year, month, day, hour, minute, second = raw - return dt.datetime(2000 + year, month, day, hour, minute, second, tzinfo=SHANGHAI) - - -def query_range(event_time: dt.datetime | None, minutes: int) -> tuple[str, str]: - center = event_time or dt.datetime.now(tz=SHANGHAI) - center = center.astimezone(SHANGHAI) - start = center - dt.timedelta(minutes=minutes) - end = center + dt.timedelta(minutes=minutes) - return start.isoformat(timespec="seconds"), end.isoformat(timespec="seconds") - - -def send_frame(host: str, port: int, frame: bytes, timeout: float) -> bytes: - with socket.create_connection((host, port), timeout=timeout) as sock: - sock.settimeout(timeout) - sock.sendall(frame) - try: - return sock.recv(4096) - except socket.timeout: - return b"" - - -def valid_ack(ack: bytes) -> bool: - return len(ack) >= 2 and ack[:2] in (b"##", b"$$") - - -def query_json(base_url: str, path: str, params: dict[str, Any], timeout: float) -> Any: - url = base_url.rstrip("/") + path + "?" + urllib.parse.urlencode(params) - request = urllib.request.Request(url, headers={"Accept": "application/json"}) - with urllib.request.urlopen(request, timeout=timeout) as response: - return json.loads(response.read().decode("utf-8")) - - -def wait_for_records( - history_base_url: str, - vin: str, - date_from: str, - date_to: str, - minimum: int, - timeout_seconds: float, - request_timeout: float, -) -> list[dict[str, Any]]: - deadline = time.monotonic() + timeout_seconds - params = { - "vin": vin, - "dateFrom": date_from, - "dateTo": date_to, - "limit": max(minimum, 10), - "order": "ASC", - } - last_items: list[dict[str, Any]] = [] - while time.monotonic() < deadline: - response = query_json(history_base_url, "/api/event-history/gb32960/snapshots", params, request_timeout) - last_items = response if isinstance(response, list) else [] - if len(last_items) >= minimum: - return last_items - time.sleep(1) - return last_items - - -def wait_for_fields( - history_base_url: str, - vin: str, - field_key: str, - date_from: str, - date_to: str, - timeout_seconds: float, - request_timeout: float, -) -> list[dict[str, Any]]: - deadline = time.monotonic() + timeout_seconds - params = { - "vin": vin, - "fields": field_key, - "dateFrom": date_from, - "dateTo": date_to, - "limit": 10, - "order": "ASC", - } - last_items: list[dict[str, Any]] = [] - while time.monotonic() < deadline: - response = query_json(history_base_url, "/api/event-history/gb32960/snapshots/fields", params, request_timeout) - last_items = response if isinstance(response, list) else [] - if any((item.get("fields") or {}).get(field_key) is not None for item in last_items): - return last_items - time.sleep(1) - return last_items - - -def wait_for_tdengine_raw_frames( - rest_url: str, - username: str, - password: str, - sql: str, - minimum: int, - timeout_seconds: float, - request_timeout: float, -) -> int: - deadline = time.monotonic() + timeout_seconds - last_count = 0 - while time.monotonic() < deadline: - last_count = tdengine_smoke.query_count(rest_url, username, password, sql, request_timeout) - if last_count >= minimum: - return last_count - time.sleep(1) - return last_count - - -def parse_field_keys(value: str) -> list[str]: - return [item.strip() for item in value.split(",") if item.strip()] - - -def archive_path(root: pathlib.Path, uri: str) -> pathlib.Path: - prefix = "archive://" - if not uri.startswith(prefix): - raise ValueError(f"unsupported archive uri: {uri}") - return root / uri[len(prefix):] - - -def check_archive(root: pathlib.Path, records: list[dict[str, Any]], limit: int) -> int: - checked = 0 - seen: set[str] = set() - for uri in raw_uris(records): - if uri in seen: - continue - seen.add(uri) - path = archive_path(root, uri) - if not path.exists(): - raise RuntimeError(f"raw archive missing: {path}") - checked += 1 - if checked >= limit: - return checked - return checked - - -def raw_frame_count_sql(info: FrameInfo, records: list[dict[str, Any]]) -> str: - sqls = raw_frame_count_sqls(info, records) - if not sqls: - raise ValueError("no raw archive uri found in gb32960 records") - return sqls[0] - - -def raw_frame_count_sqls(info: FrameInfo, records: list[dict[str, Any]]) -> list[str]: - seen: set[str] = set() - sqls: list[str] = [] - for uri in raw_uris(records): - if uri in seen: - continue - seen.add(uri) - sqls.append(tdengine_smoke.raw_frame_count_sql( - protocol="GB32960", - vehicle_key=info.vin, - vin=info.vin, - raw_uri=uri, - )) - return sqls - - -def verify_tdengine_raw_frames( - rest_url: str, - username: str, - password: str, - sqls: list[str], - timeout_seconds: float, - request_timeout: float, -) -> int: - if not sqls: - raise RuntimeError("no raw archive uri available for tdengine raw_frames verification") - verified = 0 - for sql in sqls: - count = wait_for_tdengine_raw_frames( - rest_url, username, password, sql, 1, timeout_seconds, request_timeout) - if count < 1: - raise RuntimeError(f"tdengine raw_frames row not visible for query: {sql}") - verified += 1 - return verified - - -def run(args: argparse.Namespace) -> dict[str, Any]: - frame = read_hex_frame(pathlib.Path(args.frame_hex_file)) - info = parse_frame_info(frame) - query_center = dt.datetime.now(tz=SHANGHAI) - date_from, date_to = query_range(query_center, args.window_minutes) - - ack = send_frame(args.tcp_host, args.tcp_port, frame, args.tcp_timeout) - if args.require_ack and not valid_ack(ack): - raise RuntimeError(f"gb32960 ack missing or invalid: {ack.hex()}") - - records = wait_for_records( - args.history_base_url, - info.vin, - date_from, - date_to, - args.expect_record_count, - args.history_timeout, - args.http_timeout, - ) - if len(records) < args.expect_record_count: - raise RuntimeError(f"gb32960 records not visible for {info.vin}: got {len(records)}") - - field_counts: dict[str, int] = {} - for field_key in parse_field_keys(args.field_keys): - field_items = wait_for_fields( - args.history_base_url, - info.vin, - field_key, - date_from, - date_to, - args.history_timeout, - args.http_timeout, - ) - field_counts[field_key] = len(field_items) - if not field_items: - raise RuntimeError(f"gb32960 telemetry field not visible for {info.vin}: {field_key}") - - archive_checked = 0 - if args.archive_root: - archive_checked = check_archive(pathlib.Path(args.archive_root), records, args.archive_check_limit) - - tdengine_raw_frames = None - if args.tdengine_rest_url: - tdengine_raw_frames = verify_tdengine_raw_frames( - args.tdengine_rest_url, - args.tdengine_username, - args.tdengine_password, - raw_frame_count_sqls(info, records), - args.history_timeout, - args.http_timeout, - ) - - return { - "tcpHost": args.tcp_host, - "tcpPort": args.tcp_port, - "vin": info.vin, - "command": f"0x{info.command:02x}", - "eventTime": info.event_time.isoformat() if info.event_time else None, - "queryTimeSource": "received", - "queryDateFrom": date_from, - "queryDateTo": date_to, - "ackBytesHex": ack.hex(), - "records": len(records), - "fieldCounts": field_counts, - "tdengineRawFrames": tdengine_raw_frames, - "archiveChecked": archive_checked, - "firstRawUri": first_raw_uri(records), - } - - -def first_raw_uri(records: list[dict[str, Any]]) -> str | None: - uris = raw_uris(records) - if uris: - return uris[0] - return None - - -def raw_uris(records: list[dict[str, Any]]) -> list[str]: - out: list[str] = [] - for record in records: - uri = record.get("rawArchiveUri") - if isinstance(uri, str) and uri: - out.append(uri) - for frame in record.get("sourceFrames") or []: - frame_uri = frame.get("rawArchiveUri") if isinstance(frame, dict) else "" - if frame_uri: - out.append(frame_uri) - return out - - -def parser() -> argparse.ArgumentParser: - p = argparse.ArgumentParser(description=__doc__) - p.add_argument("--tcp-host", default="127.0.0.1") - p.add_argument("--tcp-port", type=int, default=32960) - p.add_argument("--history-base-url", default="http://127.0.0.1:20200") - p.add_argument("--frame-hex-file", default=DEFAULT_SAMPLE) - p.add_argument("--field-keys", default=DEFAULT_FIELD_KEYS) - p.add_argument("--window-minutes", type=int, default=10) - p.add_argument("--expect-record-count", type=int, default=1) - p.add_argument("--archive-root", default="") - p.add_argument("--archive-check-limit", type=int, default=1) - p.add_argument("--tdengine-rest-url", default=os.environ.get("TDENGINE_REST_URL", "")) - p.add_argument("--tdengine-username", default=os.environ.get("TDENGINE_USERNAME", "root")) - p.add_argument("--tdengine-password", default=os.environ.get("TDENGINE_PASSWORD", "taosdata")) - p.add_argument("--require-ack", action="store_true", default=True) - p.add_argument("--tcp-timeout", type=float, default=5) - p.add_argument("--http-timeout", type=float, default=5) - p.add_argument("--history-timeout", type=float, default=60) - return p - - -def main() -> None: - args = parser().parse_args() - summary = run(args) - print(json.dumps(summary, ensure_ascii=False, indent=2)) - - -if __name__ == "__main__": - main() diff --git a/tools/go_kafka_prod_smoke.py b/tools/go_kafka_prod_smoke.py deleted file mode 100755 index 06e0c12a..00000000 --- a/tools/go_kafka_prod_smoke.py +++ /dev/null @@ -1,162 +0,0 @@ -#!/usr/bin/env python3 -"""Kafka topic and consumer lag smoke checks for the Go native production stack.""" - -from __future__ import annotations - -import argparse -import json -import subprocess -import sys -from dataclasses import asdict, dataclass - - -DEFAULT_HOST = "114.55.58.251" -DEFAULT_USER = "root" -DEFAULT_BOOTSTRAP = "127.0.0.1:9092" -DEFAULT_KAFKA_BIN = "/opt/kafka/current/bin" -DEFAULT_TOPICS = [ - "vehicle.raw.go.gb32960.v1", - "vehicle.raw.go.jt808.v1", - "vehicle.raw.go.yutong-mqtt.v1", - "vehicle.event.go.unified.v1", -] -DEFAULT_GROUPS = [ - "go-history-writer", - "go-stat-writer", - "go-realtime-api", -] - - -@dataclass(frozen=True) -class Check: - name: str - status: str - message: str - - -@dataclass(frozen=True) -class ConsumerLag: - group: str - topic: str - partition: int - current_offset: int | None - log_end_offset: int - lag: int | None - - -def parse_topic_list(output: str) -> set[str]: - return {line.strip() for line in output.splitlines() if line.strip()} - - -def topic_checks(existing: set[str], required: list[str]) -> list[Check]: - checks: list[Check] = [] - for topic in required: - if topic in existing: - checks.append(Check("topic." + topic, "pass", "present")) - else: - checks.append(Check("topic." + topic, "fail", "missing")) - return checks - - -def parse_consumer_group_describe(output: str) -> list[ConsumerLag]: - rows: list[ConsumerLag] = [] - for line in output.splitlines(): - parts = line.split() - if len(parts) < 6 or parts[0] == "GROUP": - continue - group, topic = parts[0], parts[1] - try: - partition = int(parts[2]) - current = parse_optional_int(parts[3]) - log_end = int(parts[4]) - lag = parse_optional_int(parts[5]) - except ValueError: - continue - rows.append(ConsumerLag(group, topic, partition, current, log_end, lag)) - return rows - - -def parse_optional_int(value: str) -> int | None: - value = value.strip() - if value == "-": - return None - return int(value) - - -def group_lag_check(group: str, rows: list[ConsumerLag], max_lag: int) -> Check: - group_rows = [row for row in rows if row.group == group] - if not group_rows: - return Check("group." + group, "fail", "no rows") - concrete_lags = [row.lag for row in group_rows if row.lag is not None] - if not concrete_lags: - return Check("group." + group, "fail", "no concrete lag rows") - max_seen = max(concrete_lags) - topics = sorted({row.topic for row in group_rows}) - status = "pass" if max_seen <= max_lag else "fail" - return Check( - "group." + group, - status, - "max_lag=" + str(max_seen) + "; threshold=" + str(max_lag) + "; topics=" + ",".join(topics), - ) - - -def ssh(host: str, user: str, command: str, timeout: float) -> str: - target = user + "@" + host if user else host - completed = subprocess.run( - ["ssh", "-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null", target, command], - check=True, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True, - timeout=timeout, - ) - return completed.stdout - - -def remote_topic_command(kafka_bin: str, bootstrap: str) -> str: - return kafka_bin.rstrip("/") + "/kafka-topics.sh --bootstrap-server " + bootstrap + " --list" - - -def remote_group_command(kafka_bin: str, bootstrap: str, group: str) -> str: - return kafka_bin.rstrip("/") + "/kafka-consumer-groups.sh --bootstrap-server " + bootstrap + " --describe --group " + group - - -def run(args: argparse.Namespace) -> tuple[str, list[Check]]: - topic_output = ssh(args.host, args.user, remote_topic_command(args.kafka_bin, args.bootstrap), args.timeout) - checks = topic_checks(parse_topic_list(topic_output), DEFAULT_TOPICS) - for group in DEFAULT_GROUPS: - group_output = ssh(args.host, args.user, remote_group_command(args.kafka_bin, args.bootstrap, group), args.timeout) - checks.append(group_lag_check(group, parse_consumer_group_describe(group_output), args.max_lag)) - status = "fail" if any(check.status == "fail" for check in checks) else "pass" - return status, checks - - -def parse_args(argv: list[str]) -> argparse.Namespace: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--host", default=DEFAULT_HOST) - parser.add_argument("--user", default=DEFAULT_USER) - parser.add_argument("--bootstrap", default=DEFAULT_BOOTSTRAP) - parser.add_argument("--kafka-bin", default=DEFAULT_KAFKA_BIN) - parser.add_argument("--max-lag", type=int, default=100) - parser.add_argument("--timeout", type=float, default=20.0) - return parser.parse_args(argv) - - -def main(argv: list[str]) -> int: - args = parse_args(argv) - try: - status, checks = run(args) - except Exception as exc: - status = "fail" - checks = [Check("kafka.ssh", "fail", str(exc))] - print(json.dumps({ - "status": status, - "host": args.host, - "bootstrap": args.bootstrap, - "checks": [asdict(check) for check in checks], - }, ensure_ascii=False, indent=2)) - return 0 if status == "pass" else 1 - - -if __name__ == "__main__": - raise SystemExit(main(sys.argv[1:])) diff --git a/tools/go_native_deploy.py b/tools/go_native_deploy.py deleted file mode 100644 index 8d7e5358..00000000 --- a/tools/go_native_deploy.py +++ /dev/null @@ -1,253 +0,0 @@ -#!/usr/bin/env python3 -"""Build and deploy the Go native vehicle gateway release to ECS.""" - -from __future__ import annotations - -import argparse -import os -import pathlib -import shutil -import subprocess -import sys -import tarfile -import tempfile -import time -from dataclasses import dataclass - - -DEFAULT_APP_HOST = "115.29.187.205" -DEFAULT_KAFKA_HOST = "114.55.58.251" -DEFAULT_USER = "root" -DEFAULT_RELEASE_ROOT = "/opt/lingniu-go-native" -DEFAULT_GO_DIR = "go/vehicle-gateway" -BINARIES = [ - ("gateway", "./cmd/gateway"), - ("history-writer", "./cmd/history-writer"), - ("stat-writer", "./cmd/stat-writer"), - ("realtime-api", "./cmd/realtime-api"), -] -SERVICES = [ - "lingniu-go-gateway", - "lingniu-go-history-writer", - "lingniu-go-stat-writer", - "lingniu-go-realtime-api", -] - - -@dataclass(frozen=True) -class BuildCommand: - output_name: str - argv: list[str] - env: list[str] - cwd: str - - -@dataclass(frozen=True) -class SpoolStatus: - files: int - recent: int - - -def git_short_sha(cwd: str) -> str: - return run_capture(["git", "rev-parse", "--short=7", "HEAD"], cwd=cwd).strip() - - -def build_commands(go_dir: str, output_dir: str) -> list[BuildCommand]: - commands: list[BuildCommand] = [] - env = ["CGO_ENABLED=0", "GOOS=linux", "GOARCH=amd64"] - for output_name, package in BINARIES: - commands.append(BuildCommand( - output_name=output_name, - argv=[ - "go", - "build", - "-trimpath", - "-ldflags=-s -w", - "-o", - str(pathlib.Path(output_dir) / output_name), - package, - ], - env=env, - cwd=go_dir, - )) - return commands - - -def build_release(go_dir: str, output_dir: str) -> None: - for command in build_commands(go_dir, output_dir): - env = os.environ.copy() - for item in command.env: - key, value = item.split("=", 1) - env[key] = value - run(command.argv, cwd=command.cwd, env=env) - - -def create_archive(source_dir: str, archive_path: str) -> None: - with tarfile.open(archive_path, "w:gz") as archive: - for binary, _ in BINARIES: - archive.add(pathlib.Path(source_dir) / binary, arcname=binary) - - -def remote_switch_command(release: str, release_root: str, remote_archive: str) -> str: - if not safe_release_name(release): - raise ValueError("release must contain only letters, numbers, dot, underscore, or dash") - release_dir = release_root.rstrip("/") + "/releases/" + release - root = release_root.rstrip("/") - commands = [ - "set -euo pipefail", - "mkdir -p " + shell_quote(release_dir), - "tar -xzf " + shell_quote(remote_archive) + " -C " + shell_quote(release_dir), - "chmod +x " + shell_quote(release_dir) + "/*", - "ln -sfn " + shell_quote(release_dir) + " " + shell_quote(root + "/current"), - "systemctl daemon-reload", - ] - commands.extend("systemctl restart " + service for service in SERVICES) - commands.append("rm -f " + shell_quote(remote_archive)) - return " && ".join(commands) - - -def acceptance_command(app_host: str, kafka_host: str, date: str, timeout: float) -> list[str]: - command = [ - "python3", - "tools/go_prod_acceptance.py", - "--app-host", - app_host, - "--kafka-host", - kafka_host, - "--timeout", - str(int(timeout) if timeout == int(timeout) else timeout), - ] - if date: - command.extend(["--date", date]) - return command - - -def deploy(args: argparse.Namespace) -> None: - repo = pathlib.Path(args.repo).resolve() - go_dir = str(repo / args.go_dir) - release = args.release or git_short_sha(str(repo)) - with tempfile.TemporaryDirectory(prefix="lingniu-go-native-") as tmp: - output_dir = pathlib.Path(tmp) / "out" - output_dir.mkdir() - archive_path = pathlib.Path(tmp) / ("lingniu-go-native-" + release + ".tar.gz") - build_release(go_dir, str(output_dir)) - create_archive(str(output_dir), str(archive_path)) - remote_archive = "/tmp/" + archive_path.name - target = ssh_target(args.user, args.app_host) - run(["scp", "-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null", str(archive_path), target + ":" + remote_archive]) - run([ - "ssh", - "-o", - "StrictHostKeyChecking=no", - "-o", - "UserKnownHostsFile=/dev/null", - target, - remote_switch_command(release, args.release_root, remote_archive), - ]) - wait_for_spool_drain(target, args.release_root, args.spool_drain_timeout, args.spool_poll_interval) - if not args.skip_acceptance: - run(acceptance_command(args.app_host, args.kafka_host, args.date, args.timeout), cwd=str(repo)) - - -def wait_for_spool_drain(target: str, release_root: str, timeout: float, poll_interval: float) -> None: - deadline = time.monotonic() + timeout - last = SpoolStatus(files=-1, recent=-1) - while time.monotonic() <= deadline: - output = run_capture([ - "ssh", - "-o", - "StrictHostKeyChecking=no", - "-o", - "UserKnownHostsFile=/dev/null", - target, - spool_status_command(release_root), - ]) - last = parse_spool_status(output) - print(f"spool files={last.files} recent={last.recent}", flush=True) - if last.files == 0: - return - time.sleep(max(1.0, poll_interval)) - raise RuntimeError(f"gateway spool did not drain: files={last.files} recent={last.recent}") - - -def spool_status_command(release_root: str) -> str: - spool_dir = release_root.rstrip("/") + "/spool/gateway" - return ( - "spool=" + shell_quote(spool_dir) + "; " - "files=$(find \"$spool\" -type f 2>/dev/null | wc -l); " - "recent=$(find \"$spool\" -type f -mmin -5 2>/dev/null | wc -l); " - "printf 'files=%s recent=%s\\n' \"$files\" \"$recent\"" - ) - - -def parse_spool_status(output: str) -> SpoolStatus: - values: dict[str, int] = {} - for part in output.split(): - if "=" not in part: - continue - key, value = part.split("=", 1) - try: - values[key] = int(value) - except ValueError: - continue - return SpoolStatus(files=values.get("files", -1), recent=values.get("recent", -1)) - - -def ssh_target(user: str, host: str) -> str: - return user + "@" + host if user else host - - -def shell_quote(value: str) -> str: - return "'" + value.replace("'", "'\"'\"'") + "'" - - -def safe_release_name(value: str) -> bool: - value = value.strip() - if not value: - return False - return all(r.isalnum() or r in "._-" for r in value) - - -def run(argv: list[str], cwd: str | None = None, env: dict[str, str] | None = None) -> None: - subprocess.run(argv, cwd=cwd, env=env, check=True) - - -def run_capture(argv: list[str], cwd: str | None = None) -> str: - return subprocess.run(argv, cwd=cwd, check=True, stdout=subprocess.PIPE, text=True).stdout - - -def parse_args(argv: list[str]) -> argparse.Namespace: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--repo", default=".") - parser.add_argument("--go-dir", default=DEFAULT_GO_DIR) - parser.add_argument("--app-host", default=DEFAULT_APP_HOST) - parser.add_argument("--kafka-host", default=DEFAULT_KAFKA_HOST) - parser.add_argument("--user", default=DEFAULT_USER) - parser.add_argument("--release-root", default=DEFAULT_RELEASE_ROOT) - parser.add_argument("--release", default="") - parser.add_argument("--date", default="") - parser.add_argument("--timeout", type=float, default=20.0) - parser.add_argument("--spool-drain-timeout", type=float, default=900.0) - parser.add_argument("--spool-poll-interval", type=float, default=10.0) - parser.add_argument("--skip-acceptance", action="store_true") - return parser.parse_args(argv) - - -def main(argv: list[str]) -> int: - args = parse_args(argv) - try: - deploy(args) - except subprocess.CalledProcessError as exc: - print("command failed: " + " ".join(exc.cmd), file=sys.stderr) - return exc.returncode or 1 - except (OSError, shutil.Error) as exc: - print("deploy failed: " + str(exc), file=sys.stderr) - return 1 - except RuntimeError as exc: - print("deploy failed: " + str(exc), file=sys.stderr) - return 1 - return 0 - - -if __name__ == "__main__": - raise SystemExit(main(sys.argv[1:])) diff --git a/tools/go_native_prod_smoke.py b/tools/go_native_prod_smoke.py deleted file mode 100755 index 77c11999..00000000 --- a/tools/go_native_prod_smoke.py +++ /dev/null @@ -1,544 +0,0 @@ -#!/usr/bin/env python3 -"""HTTP smoke checks for the Go native production vehicle ingest stack.""" - -from __future__ import annotations - -import argparse -import datetime as dt -import json -import sys -import urllib.parse -import urllib.request -from dataclasses import asdict, dataclass -from typing import Any - - -SHANGHAI = dt.timezone(dt.timedelta(hours=8)) -DEFAULT_BASE_URL = "http://115.29.187.205:20210" -DEFAULT_REALTIME_MAX_AGE_MINUTES = 15.0 - - -@dataclass(frozen=True) -class CheckSpec: - name: str - path: str - params: dict[str, Any] - minimum: int - kind: str = "total" - max_age_minutes: float | None = None - require_parsed_json: bool = False - require_frame_id: bool = False - require_metric_formula: bool = False - - -@dataclass(frozen=True) -class Check: - name: str - status: str - count: int - minimum: int - message: str - - -@dataclass(frozen=True) -class CheckWindow: - date_from: str - date_to: str - stat_date: str - max_raw_age_minutes: float | None - - -def shanghai_day_window(now: dt.datetime | None = None) -> tuple[str, str]: - now = now or dt.datetime.now(tz=SHANGHAI) - local = now.astimezone(SHANGHAI) - start = dt.datetime(local.year, local.month, local.day, tzinfo=SHANGHAI) - end = start + dt.timedelta(days=1) - return start.isoformat(timespec="seconds"), end.isoformat(timespec="seconds") - - -def resolve_check_window( - raw_date: str | None, - max_raw_age_minutes: float, - now: dt.datetime | None = None, -) -> CheckWindow: - current = (now or dt.datetime.now(tz=SHANGHAI)).astimezone(SHANGHAI) - today = current.date() - if raw_date: - day = dt.date.fromisoformat(raw_date) - start = dt.datetime(day.year, day.month, day.day, tzinfo=SHANGHAI) - date_from, date_to = shanghai_day_window(start) - freshness = max_raw_age_minutes if day == today else None - return CheckWindow(date_from, date_to, raw_date, freshness) - date_from, date_to = shanghai_day_window(current) - return CheckWindow(date_from, date_to, date_from[:10], max_raw_age_minutes) - - -def api_url(base_url: str, path: str, params: dict[str, Any]) -> str: - url = base_url.rstrip("/") + path - if not params: - return url - return url + "?" + urllib.parse.urlencode(params) - - -def query_json(base_url: str, path: str, params: dict[str, Any], timeout: float) -> dict[str, Any]: - request = urllib.request.Request( - api_url(base_url, path, params), - headers={"Accept": "application/json"}, - ) - with urllib.request.urlopen(request, timeout=timeout) as response: - return json.loads(response.read().decode("utf-8")) - - -def check_total( - name: str, - payload: dict[str, Any], - minimum: int, - max_age_minutes: float | None = None, - now: dt.datetime | None = None, - require_parsed_json: bool = False, - require_frame_id: bool = False, - require_metric_formula: bool = False, -) -> Check: - count = int(payload.get("total") or 0) - items = payload.get("items") or [] - sample = sample_summary(items) - age_message = latest_age_message(items, now) - if count >= minimum: - parsed_message = "" - if max_age_minutes is not None: - latest_age = latest_age_minutes(items, now) - if latest_age is None: - return Check(name, "fail", count, minimum, f"count={count}; missing latest ts; sample={sample}") - if latest_age > max_age_minutes: - return Check( - name, - "fail", - count, - minimum, - f"count={count}; latest_age_minutes={latest_age:.1f}; " - f"expected <= {max_age_minutes:.1f}; sample={sample}", - ) - if require_parsed_json: - parsed_message = parsed_json_message(items) - if not parsed_message.startswith("parsed_json=ok"): - return Check(name, "fail", count, minimum, f"count={count}; {parsed_message}; sample={sample}") - frame_message = "" - if require_frame_id: - frame_message = frame_id_message(items) - if not frame_message.startswith("frame_id=ok"): - return Check(name, "fail", count, minimum, f"count={count}; {frame_message}; sample={sample}") - metric_message = "" - if require_metric_formula: - metric_message = metric_formula_message(items) - if not metric_message.startswith("metric_formula=ok"): - return Check(name, "fail", count, minimum, f"count={count}; {metric_message}; sample={sample}") - details = [f"count={count}", age_message] - if parsed_message: - details.append(parsed_message) - if frame_message: - details.append(frame_message) - if metric_message: - details.append(metric_message) - return Check(name, "pass", count, minimum, "; ".join(details) + f"; sample={sample}") - return Check(name, "fail", count, minimum, f"count={count}; expected >= {minimum}; {age_message}; sample={sample}") - - -def parse_tdengine_utc_timestamp(value: str) -> dt.datetime: - value = str(value or "").strip() - for layout in ("%Y-%m-%d %H:%M:%S", "%Y-%m-%dT%H:%M:%S"): - try: - return dt.datetime.strptime(value[:19], layout).replace(tzinfo=dt.timezone.utc) - except ValueError: - continue - raise ValueError(f"unsupported timestamp: {value}") - - -def latest_age_minutes(items: list[Any], now: dt.datetime | None = None) -> float | None: - if not items or not isinstance(items[0], dict): - return None - timestamp = items[0].get("received_at") or items[0].get("ts") - if not timestamp: - return None - current = now or dt.datetime.now(tz=dt.timezone.utc) - if current.tzinfo is None: - current = current.replace(tzinfo=dt.timezone.utc) - latest = parse_tdengine_utc_timestamp(str(timestamp)) - return max(0.0, (current.astimezone(dt.timezone.utc) - latest).total_seconds() / 60) - - -def latest_age_message(items: list[Any], now: dt.datetime | None = None) -> str: - latest_age = latest_age_minutes(items, now) - if latest_age is None: - return "latest_age_minutes=unknown" - return f"latest_age_minutes={latest_age:.1f}" - - -def parsed_json_message(items: list[Any]) -> str: - if not items or not isinstance(items[0], dict): - return "missing parsed_json" - raw = items[0].get("parsed_json") - if isinstance(raw, dict) and raw: - return "parsed_json=ok" - if not isinstance(raw, str) or not raw.strip() or raw.strip().lower() == "null": - return "missing parsed_json" - try: - parsed = json.loads(raw) - except json.JSONDecodeError: - return "invalid parsed_json" - if isinstance(parsed, dict) and parsed: - return "parsed_json=ok" - return "invalid parsed_json" - - -def frame_id_message(items: list[Any]) -> str: - if not items or not isinstance(items[0], dict): - return "missing frame_id" - frame_id = str(items[0].get("frame_id") or "").strip() - if frame_id: - return "frame_id=ok" - return "missing frame_id" - - -def metric_formula_message(items: list[Any]) -> str: - if not items or not isinstance(items[0], dict): - return "missing metric row" - row = items[0] - try: - metric_value = float(row.get("metric_value")) - first_total = float(row.get("first_total_mileage_km")) - latest_total = float(row.get("latest_total_mileage_km")) - except (TypeError, ValueError): - return "missing metric formula fields" - metric_key = str(row.get("metric_key") or "") - if metric_key == "daily_mileage_km": - expected = max(0.0, latest_total - first_total) - elif metric_key == "daily_total_mileage_km": - expected = latest_total - else: - return "unsupported metric_key=" + metric_key - if abs(metric_value - expected) <= 0.001: - return "metric_formula=ok" - return f"metric_formula mismatch value={metric_value:.3f} expected={expected:.3f}" - - -def check_realtime( - name: str, - payload: dict[str, Any], - max_age_minutes: float | None = None, - now: dt.datetime | None = None, -) -> Check: - if payload.get("online") is False: - return Check(name, "fail", 0, 1, "online=false") - updated_message = "" - if max_age_minutes is not None: - updated_age = realtime_updated_age_minutes(payload, now) - if updated_age is None: - return Check(name, "fail", 0, 1, "missing updated_at_ms") - updated_message = f"updated_age_minutes={updated_age:.1f}" - if updated_age > max_age_minutes: - return Check( - name, - "fail", - 0, - 1, - updated_message + f"; expected <= {max_age_minutes:.1f}", - ) - identifier = payload.get("vin") or payload.get("vehicle_key") or "" - protocols = payload.get("protocols") or [] - protocol = payload.get("protocol") or "" - fields = payload.get("fields") or {} - message = json.dumps( - { - key: value - for key, value in { - "identifier": identifier, - "protocol": protocol, - "protocols": protocols, - "field_count": len(fields) if isinstance(fields, dict) else 0, - "online": payload.get("online"), - }.items() - if value not in (None, "", []) - }, - ensure_ascii=False, - sort_keys=True, - ) - if updated_message: - message = updated_message + "; " + message - return Check(name, "pass", 1, 1, message) - - -def realtime_updated_age_minutes(payload: dict[str, Any], now: dt.datetime | None = None) -> float | None: - try: - updated_ms = int(payload.get("updated_at_ms")) - except (TypeError, ValueError): - return None - current = now or dt.datetime.now(tz=dt.timezone.utc) - if current.tzinfo is None: - current = current.replace(tzinfo=dt.timezone.utc) - updated = dt.datetime.fromtimestamp(updated_ms / 1000, tz=dt.timezone.utc) - return max(0.0, (current.astimezone(dt.timezone.utc) - updated).total_seconds() / 60) - - -def sample_summary(items: list[Any]) -> str: - if not items: - return "{}" - first = items[0] - if not isinstance(first, dict): - return json.dumps(first, ensure_ascii=False, sort_keys=True) - keep = { - key: first.get(key) - for key in ( - "ts", - "stat_date", - "protocol", - "vehicle_key", - "vin", - "phone", - "message_id_hex", - "metric_key", - "metric_value", - "total_mileage_km", - "latest_total_mileage_km", - ) - if first.get(key) not in (None, "") - } - return json.dumps(keep, ensure_ascii=False, sort_keys=True) - - -def overall_status(checks: list[Check]) -> str: - return "fail" if any(check.status == "fail" for check in checks) else "pass" - - -def build_check_specs( - *, - date_from: str, - date_to: str, - stat_date: str, - min_raw: int, - min_history: int, - min_stat: int, - max_raw_age_minutes: float | None = None, -) -> list[CheckSpec]: - raw_params = {"limit": 1, "orderBy": "receivedAt", "includeTotal": "false"} - gb32960_history_params = {"protocol": "GB32960", "dateFrom": date_from, "dateTo": date_to, "limit": 1} - jt808_history_params = {"protocol": "JT808", "dateFrom": date_from, "dateTo": date_to, "limit": 1} - yutong_mqtt_history_params = {"protocol": "YUTONG_MQTT", "dateFrom": date_from, "dateTo": date_to, "limit": 1} - return [ - CheckSpec( - "gb32960.raw", - "/api/history/raw-frames", - {**raw_params, "protocol": "GB32960"}, - min_raw, - max_age_minutes=max_raw_age_minutes, - require_parsed_json=True, - ), - CheckSpec( - "jt808.raw", - "/api/history/raw-frames", - {**raw_params, "protocol": "JT808"}, - min_raw, - max_age_minutes=max_raw_age_minutes, - require_parsed_json=True, - ), - CheckSpec( - "yutong_mqtt.raw", - "/api/history/raw-frames", - {**raw_params, "protocol": "YUTONG_MQTT"}, - min_raw, - max_age_minutes=max_raw_age_minutes, - require_parsed_json=True, - ), - CheckSpec("gb32960.locations", "/api/history/locations", gb32960_history_params, min_history, require_frame_id=True), - CheckSpec("gb32960.mileage_points", "/api/history/mileage-points", gb32960_history_params, min_history, require_frame_id=True), - CheckSpec("jt808.locations", "/api/history/locations", jt808_history_params, min_history, require_frame_id=True), - CheckSpec("jt808.mileage_points", "/api/history/mileage-points", jt808_history_params, min_history, require_frame_id=True), - CheckSpec("yutong_mqtt.locations", "/api/history/locations", yutong_mqtt_history_params, min_history, require_frame_id=True), - CheckSpec("yutong_mqtt.mileage_points", "/api/history/mileage-points", yutong_mqtt_history_params, min_history, require_frame_id=True), - CheckSpec( - "gb32960.daily_mileage", - "/api/stats/daily-metrics", - { - "protocol": "GB32960", - "metricKey": "daily_mileage_km", - "dateFrom": stat_date, - "dateTo": stat_date, - "limit": 1, - }, - min_stat, - require_metric_formula=True, - ), - CheckSpec( - "gb32960.daily_total_mileage", - "/api/stats/daily-metrics", - { - "protocol": "GB32960", - "metricKey": "daily_total_mileage_km", - "dateFrom": stat_date, - "dateTo": stat_date, - "limit": 1, - }, - min_stat, - require_metric_formula=True, - ), - CheckSpec( - "jt808.daily_mileage", - "/api/stats/daily-metrics", - { - "protocol": "JT808", - "metricKey": "daily_mileage_km", - "dateFrom": stat_date, - "dateTo": stat_date, - "limit": 1, - }, - min_stat, - require_metric_formula=True, - ), - CheckSpec( - "jt808.daily_total_mileage", - "/api/stats/daily-metrics", - { - "protocol": "JT808", - "metricKey": "daily_total_mileage_km", - "dateFrom": stat_date, - "dateTo": stat_date, - "limit": 1, - }, - min_stat, - require_metric_formula=True, - ), - ] - - -def vehicle_identifier(item: dict[str, Any]) -> str: - for key in ("vin", "vehicle_key"): - value = str(item.get(key) or "").strip() - if value and value.lower() != "unknown": - return value - return "" - - -def build_realtime_specs(payloads: dict[str, dict[str, Any]]) -> list[CheckSpec]: - configs = [ - ("gb32960", "gb32960.raw", "GB32960"), - ("jt808", "jt808.raw", "JT808"), - ("yutong_mqtt", "yutong_mqtt.raw", "YUTONG_MQTT"), - ] - specs: list[CheckSpec] = [] - for prefix, source_name, protocol in configs: - items = payloads.get(source_name, {}).get("items") or [] - if not items or not isinstance(items[0], dict): - continue - identifier = vehicle_identifier(items[0]) - if not identifier: - continue - encoded = urllib.parse.quote(identifier, safe="") - base_path = "/api/realtime/vehicles/" + encoded - specs.extend([ - CheckSpec( - prefix + ".realtime", - base_path, - {}, - 1, - "realtime", - max_age_minutes=DEFAULT_REALTIME_MAX_AGE_MINUTES, - ), - CheckSpec(prefix + ".realtime_online", base_path + "/online", {}, 1, "realtime"), - CheckSpec( - prefix + ".realtime_protocol", - base_path + "/protocols/" + protocol, - {}, - 1, - "realtime", - max_age_minutes=DEFAULT_REALTIME_MAX_AGE_MINUTES, - ), - ]) - return specs - - -def run_checks(base_url: str, specs: list[CheckSpec], timeout: float) -> list[Check]: - checks: list[Check] = [] - for spec in specs: - try: - payload = query_json(base_url, spec.path, spec.params, timeout) - if spec.kind == "realtime": - checks.append(check_realtime(spec.name, payload, spec.max_age_minutes)) - else: - checks.append(check_total( - spec.name, - payload, - spec.minimum, - spec.max_age_minutes, - require_parsed_json=spec.require_parsed_json, - require_frame_id=spec.require_frame_id, - require_metric_formula=spec.require_metric_formula, - )) - except Exception as exc: # pragma: no cover - exercised by live failures. - checks.append(Check(spec.name, "fail", 0, spec.minimum, f"request failed: {exc}")) - return checks - - -def query_payloads(base_url: str, specs: list[CheckSpec], timeout: float) -> tuple[list[Check], dict[str, dict[str, Any]]]: - checks: list[Check] = [] - payloads: dict[str, dict[str, Any]] = {} - for spec in specs: - try: - payload = query_json(base_url, spec.path, spec.params, timeout) - payloads[spec.name] = payload - checks.append(check_total( - spec.name, - payload, - spec.minimum, - spec.max_age_minutes, - require_parsed_json=spec.require_parsed_json, - require_frame_id=spec.require_frame_id, - require_metric_formula=spec.require_metric_formula, - )) - except Exception as exc: - checks.append(Check(spec.name, "fail", 0, spec.minimum, f"request failed: {exc}")) - return checks, payloads - - -def parse_args(argv: list[str]) -> argparse.Namespace: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--base-url", default=DEFAULT_BASE_URL) - parser.add_argument("--date", help="Shanghai local date, YYYY-MM-DD. Defaults to today.") - parser.add_argument("--timeout", type=float, default=5.0) - parser.add_argument("--min-raw", type=int, default=1) - parser.add_argument("--min-history", type=int, default=1) - parser.add_argument("--min-stat", type=int, default=1) - parser.add_argument( - "--max-raw-age-minutes", - type=float, - default=15.0, - help="Require latest RAW sample age to be at most this many minutes when checking today's date.", - ) - return parser.parse_args(argv) - - -def main(argv: list[str]) -> int: - args = parse_args(argv) - window = resolve_check_window(args.date, args.max_raw_age_minutes) - specs = build_check_specs( - date_from=window.date_from, - date_to=window.date_to, - stat_date=window.stat_date, - min_raw=args.min_raw, - min_history=args.min_history, - min_stat=args.min_stat, - max_raw_age_minutes=window.max_raw_age_minutes, - ) - checks, payloads = query_payloads(args.base_url, specs, args.timeout) - checks.extend(run_checks(args.base_url, build_realtime_specs(payloads), args.timeout)) - status = overall_status(checks) - print(json.dumps({ - "status": status, - "baseUrl": args.base_url, - "dateFrom": window.date_from, - "dateTo": window.date_to, - "checks": [asdict(check) for check in checks], - }, ensure_ascii=False, indent=2)) - return 0 if status == "pass" else 1 - - -if __name__ == "__main__": - raise SystemExit(main(sys.argv[1:])) diff --git a/tools/go_prod_acceptance.py b/tools/go_prod_acceptance.py deleted file mode 100755 index d9a3c9de..00000000 --- a/tools/go_prod_acceptance.py +++ /dev/null @@ -1,136 +0,0 @@ -#!/usr/bin/env python3 -"""Run the Go native production acceptance smoke suite.""" - -from __future__ import annotations - -import argparse -import json -import subprocess -import sys -from dataclasses import asdict, dataclass -from typing import Any - - -DEFAULT_APP_HOST = "115.29.187.205" -DEFAULT_KAFKA_HOST = "114.55.58.251" -DEFAULT_USER = "root" -DEFAULT_BASE_URL = "http://115.29.187.205:20210" - - -@dataclass(frozen=True) -class ChildCommand: - name: str - argv: list[str] - - -@dataclass(frozen=True) -class ChildResult: - name: str - status: str - exit_code: int - payload: dict[str, Any] - - -def build_commands(args: argparse.Namespace) -> list[ChildCommand]: - timeout = str(args.timeout) - return [ - ChildCommand("systemd", [ - sys.executable, - "tools/go_systemd_prod_smoke.py", - "--host", - args.app_host, - "--user", - args.ssh_user, - "--timeout", - timeout, - ]), - ChildCommand("kafka", [ - sys.executable, - "tools/go_kafka_prod_smoke.py", - "--host", - args.kafka_host, - "--user", - args.ssh_user, - "--max-lag", - str(args.max_lag), - "--timeout", - timeout, - ]), - ChildCommand("http", [ - sys.executable, - "tools/go_native_prod_smoke.py", - "--base-url", - args.base_url, - "--timeout", - timeout, - ] + (["--date", args.date] if args.date else [])), - ] - - -def run_child(command: ChildCommand, timeout: float) -> ChildResult: - completed = subprocess.run( - command.argv, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True, - timeout=timeout, - ) - payload = parse_json_output(completed.stdout) - if completed.stderr.strip(): - payload.setdefault("stderr", completed.stderr.strip()) - status = payload.get("status") - if status not in {"pass", "fail"}: - status = "pass" if completed.returncode == 0 else "fail" - return ChildResult(command.name, status, completed.returncode, payload) - - -def parse_json_output(output: str) -> dict[str, Any]: - output = output.strip() - if not output: - return {} - start = output.find("{") - end = output.rfind("}") - if start < 0 or end < start: - return {"rawOutput": output} - try: - return json.loads(output[start:end + 1]) - except json.JSONDecodeError: - return {"rawOutput": output} - - -def overall_status(results: list[ChildResult]) -> str: - if not results: - return "fail" - return "fail" if any(result.status != "pass" or result.exit_code != 0 for result in results) else "pass" - - -def parse_args(argv: list[str]) -> argparse.Namespace: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--app-host", default=DEFAULT_APP_HOST) - parser.add_argument("--kafka-host", default=DEFAULT_KAFKA_HOST) - parser.add_argument("--ssh-user", default=DEFAULT_USER) - parser.add_argument("--base-url", default=DEFAULT_BASE_URL) - parser.add_argument("--date", help="Shanghai local date, YYYY-MM-DD. Defaults to today in go_native_prod_smoke.py.") - parser.add_argument("--timeout", type=float, default=20.0) - parser.add_argument("--max-lag", type=int, default=100) - return parser.parse_args(argv) - - -def main(argv: list[str]) -> int: - args = parse_args(argv) - results: list[ChildResult] = [] - for command in build_commands(args): - try: - results.append(run_child(command, args.timeout + 10)) - except Exception as exc: - results.append(ChildResult(command.name, "fail", 1, {"status": "fail", "error": str(exc)})) - status = overall_status(results) - print(json.dumps({ - "status": status, - "checks": [asdict(result) for result in results], - }, ensure_ascii=False, indent=2)) - return 0 if status == "pass" else 1 - - -if __name__ == "__main__": - raise SystemExit(main(sys.argv[1:])) diff --git a/tools/go_systemd_prod_smoke.py b/tools/go_systemd_prod_smoke.py deleted file mode 100755 index 9fe8f6cc..00000000 --- a/tools/go_systemd_prod_smoke.py +++ /dev/null @@ -1,210 +0,0 @@ -#!/usr/bin/env python3 -"""Systemd and port smoke checks for the Go native production ECS.""" - -from __future__ import annotations - -import argparse -import json -import subprocess -import sys -from dataclasses import asdict, dataclass - - -DEFAULT_HOST = "115.29.187.205" -DEFAULT_USER = "root" -DEFAULT_SERVICES = [ - "lingniu-go-gateway.service", - "lingniu-go-history-writer.service", - "lingniu-go-stat-writer.service", - "lingniu-go-realtime-api.service", -] -DEFAULT_PORTS = { - 808: "gateway", - 32960: "gateway", - 20210: "realtime-api", -} -DEFAULT_RELEASE_ROOT = "/opt/lingniu-go-native" -DEFAULT_SPOOL_DIR = "/opt/lingniu-go-native/spool/gateway" -DEFAULT_BINARIES = ["gateway", "history-writer", "stat-writer", "realtime-api"] - - -@dataclass(frozen=True) -class Check: - name: str - status: str - message: str - - -@dataclass(frozen=True) -class ServiceState: - active: str - enabled: str - - -def parse_service_states(output: str) -> dict[str, ServiceState]: - states: dict[str, ServiceState] = {} - for line in output.splitlines(): - parts = line.split() - if len(parts) >= 2: - enabled = parts[2] if len(parts) >= 3 else "enabled" - states[parts[0]] = ServiceState(parts[1], enabled) - return states - - -def service_checks(output: str, services: list[str]) -> list[Check]: - states = parse_service_states(output) - checks: list[Check] = [] - for service in services: - state = states.get(service, ServiceState("missing", "missing")) - status = "pass" if state.active == "active" and state.enabled == "enabled" else "fail" - checks.append(Check( - "service." + service, - status, - "state=" + state.active + "; enabled=" + state.enabled, - )) - return checks - - -def port_checks(output: str, ports: dict[int, str]) -> list[Check]: - lines = output.splitlines() - checks: list[Check] = [] - for port, process in sorted(ports.items()): - matching = [line.strip() for line in lines if has_port(line, port)] - if not matching: - checks.append(Check("port." + str(port), "fail", "not listening")) - continue - owned = [line for line in matching if process in line] - if owned: - checks.append(Check("port." + str(port), "pass", owned[0])) - else: - checks.append(Check("port." + str(port), "fail", matching[0])) - return checks - - -def has_port(line: str, port: int) -> bool: - needle = ":" + str(port) - return needle + " " in line or needle + "\t" in line - - -def spool_check(output: str) -> Check: - values = parse_key_values(output) - files = int(values.get("files", "-1")) - recent = int(values.get("recent", "-1")) - status = "pass" if files == 0 and recent == 0 else "fail" - return Check("spool.gateway", status, output.strip() or "missing") - - -def release_check(output: str) -> Check: - values = parse_key_values(output) - current = values.get("current", "") - binaries_present = [values.get(binary, "0") == "1" for binary in DEFAULT_BINARIES] - status = "pass" if current and all(binaries_present) else "fail" - return Check("release.current", status, output.strip() or "missing") - - -def parse_key_values(output: str) -> dict[str, str]: - values: dict[str, str] = {} - for part in output.split(): - if "=" not in part: - continue - key, value = part.split("=", 1) - values[key] = value - return values - - -def ssh(host: str, user: str, command: str, timeout: float) -> str: - target = user + "@" + host if user else host - completed = subprocess.run( - ["ssh", "-o", "StrictHostKeyChecking=no", "-o", "UserKnownHostsFile=/dev/null", target, command], - check=True, - stdout=subprocess.PIPE, - stderr=subprocess.PIPE, - text=True, - timeout=timeout, - ) - return completed.stdout - - -def remote_command(services: list[str]) -> str: - quoted = " ".join(services) - return ( - "for svc in " + quoted + "; do " - "printf '%s ' \"$svc\"; " - "printf '%s ' \"$(systemctl is-active \"$svc\" 2>/dev/null || true)\"; " - "systemctl is-enabled \"$svc\" 2>/dev/null || true; " - "done; " - "printf '\\n--PORTS--\\n'; " - "ss -lntp; " - "printf '\\n--SPOOL--\\n'; " - "spool='" + DEFAULT_SPOOL_DIR + "'; " - "if test -d \"$spool\"; then " - "files=$(find \"$spool\" -type f | wc -l); " - "bytes=$(du -sb \"$spool\" 2>/dev/null | awk '{print $1}'); " - "recent=$(find \"$spool\" -type f -mmin -5 | wc -l); " - "printf 'files=%s bytes=%s recent=%s\\n' \"$files\" \"${bytes:-0}\" \"$recent\"; " - "else printf 'files=-1 bytes=0 recent=-1 missing=%s\\n' \"$spool\"; fi; " - "printf '\\n--RELEASE--\\n'; " - "root='" + DEFAULT_RELEASE_ROOT + "'; " - "current=$(readlink -f \"$root/current\" 2>/dev/null || true); " - "printf 'current=%s' \"$current\"; " - "for bin in " + " ".join(DEFAULT_BINARIES) + "; do " - "if test -x \"$root/current/$bin\"; then present=1; else present=0; fi; " - "printf ' %s=%s' \"$bin\" \"$present\"; " - "done; printf '\\n'" - ) - - -def split_remote_output(output: str) -> tuple[str, str, str, str]: - ports_marker = "\n--PORTS--\n" - spool_marker = "\n--SPOOL--\n" - release_marker = "\n--RELEASE--\n" - if ports_marker not in output: - return output, "", "", "" - service_output, rest = output.split(ports_marker, 1) - if spool_marker not in rest: - return service_output, rest, "", "" - port_output, spool_output = rest.split(spool_marker, 1) - if release_marker not in spool_output: - return service_output, port_output, spool_output, "" - spool_output, release_output = spool_output.split(release_marker, 1) - return service_output, port_output, spool_output, release_output - - -def run(args: argparse.Namespace) -> tuple[str, list[Check]]: - output = ssh(args.host, args.user, remote_command(DEFAULT_SERVICES), args.timeout) - service_output, port_output, spool_output, release_output = split_remote_output(output) - checks = ( - service_checks(service_output, DEFAULT_SERVICES) - + port_checks(port_output, DEFAULT_PORTS) - + [spool_check(spool_output)] - + [release_check(release_output)] - ) - status = "fail" if any(check.status == "fail" for check in checks) else "pass" - return status, checks - - -def parse_args(argv: list[str]) -> argparse.Namespace: - parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument("--host", default=DEFAULT_HOST) - parser.add_argument("--user", default=DEFAULT_USER) - parser.add_argument("--timeout", type=float, default=20.0) - return parser.parse_args(argv) - - -def main(argv: list[str]) -> int: - args = parse_args(argv) - try: - status, checks = run(args) - except Exception as exc: - status = "fail" - checks = [Check("systemd.ssh", "fail", str(exc))] - print(json.dumps({ - "status": status, - "host": args.host, - "checks": [asdict(check) for check in checks], - }, ensure_ascii=False, indent=2)) - return 0 if status == "pass" else 1 - - -if __name__ == "__main__": - raise SystemExit(main(sys.argv[1:])) diff --git a/tools/jt808_e2e_smoke.py b/tools/jt808_e2e_smoke.py deleted file mode 100755 index c7e7116d..00000000 --- a/tools/jt808_e2e_smoke.py +++ /dev/null @@ -1,543 +0,0 @@ -#!/usr/bin/env python3 -"""JT808 TCP ingest to TDengine history smoke and light-load verifier.""" - -from __future__ import annotations - -import argparse -import concurrent.futures -import datetime as dt -import json -import os -import pathlib -import socket -import time -import urllib.parse -import urllib.request -from dataclasses import dataclass -from typing import Any - -try: - from tools import tdengine_smoke -except ModuleNotFoundError: - import tdengine_smoke - - -def bcd(value: str) -> bytes: - digits = "".join(ch for ch in value if ch.isdigit()) - if len(digits) % 2: - digits = "0" + digits - return bytes((int(digits[i]) << 4) | int(digits[i + 1]) for i in range(0, len(digits), 2)) - - -def write_u16(value: int) -> bytes: - return bytes([(value >> 8) & 0xFF, value & 0xFF]) - - -def write_u32(value: int) -> bytes: - return bytes([ - (value >> 24) & 0xFF, - (value >> 16) & 0xFF, - (value >> 8) & 0xFF, - value & 0xFF, - ]) - - -def ascii_fixed(value: str, length: int) -> bytes: - raw = value.encode("ascii", errors="ignore")[:length] - return raw + b" " * (length - len(raw)) - - -def bcc(data: bytes) -> int: - result = 0 - for byte in data: - result ^= byte - return result - - -def escape_payload(data: bytes) -> bytes: - out = bytearray() - for byte in data: - if byte == 0x7E: - out += b"\x7D\x02" - elif byte == 0x7D: - out += b"\x7D\x01" - else: - out.append(byte) - return bytes(out) - - -def unescape_payload(data: bytes) -> bytes: - out = bytearray() - index = 0 - while index < len(data): - byte = data[index] - if byte == 0x7D and index + 1 < len(data): - marker = data[index + 1] - if marker == 0x02: - out.append(0x7E) - index += 2 - continue - if marker == 0x01: - out.append(0x7D) - index += 2 - continue - out.append(byte) - index += 1 - return bytes(out) - - -def build_frame(message_id: int, phone: str, serial: int, body: bytes) -> bytes: - payload = ( - write_u16(message_id) - + write_u16(len(body) & 0x03FF) - + bcd(phone)[-6:] - + write_u16(serial & 0xFFFF) - + body - ) - payload += bytes([bcc(payload)]) - return b"\x7E" + escape_payload(payload) + b"\x7E" - - -def build_register_body(device_id: str, plate: str) -> bytes: - return ( - write_u16(44) - + write_u16(4401) - + ascii_fixed("MAKER", 5) - + ascii_fixed("TYPE-A", 20) - + ascii_fixed(device_id, 7) - + bytes([1]) - + plate.encode("ascii", errors="ignore") - ) - - -def build_location_body(event_time: dt.datetime, sequence: int = 0) -> bytes: - timestamp = event_time.strftime("%y%m%d%H%M%S") - return ( - write_u32(0) - + write_u32(0x00030000) - + write_u32(39916527 + sequence) - + write_u32(116397128 + sequence) - + write_u16(50) - + write_u16(523) - + write_u16(90) - + bcd(timestamp) - ) - - -def parse_event_time(value: str) -> dt.datetime: - parsed = dt.datetime.fromisoformat(value.replace("Z", "+00:00")) - if parsed.tzinfo is not None: - parsed = parsed.astimezone(dt.timezone.utc).replace(tzinfo=None) - return parsed - - -def send_frame(host: str, port: int, frame: bytes, timeout: float, read_response: bool, linger_ms: int = 0) -> bytes: - with socket.create_connection((host, port), timeout=timeout) as sock: - sock.settimeout(timeout) - sock.sendall(frame) - if not read_response: - if linger_ms > 0: - time.sleep(linger_ms / 1000) - return b"" - try: - return sock.recv(1024) - except socket.timeout: - return b"" - - -def send_register(host: str, port: int, phone: str, serial: int, timeout: float) -> bytes: - device_id = "D" + phone[-6:] - plate = "B" + phone[-5:] - return send_frame( - host, - port, - build_frame(0x0100, phone, serial, build_register_body(device_id, plate)), - timeout, - True, - ) - - -def send_location(host: str, - port: int, - phone: str, - serial: int, - event_time: dt.datetime, - timeout: float, - linger_ms: int) -> bytes: - return send_frame( - host, - port, - build_frame(0x0200, phone, serial, build_location_body(event_time, serial)), - timeout, - False, - linger_ms, - ) - - -def recv_optional(sock: socket.socket, timeout: float) -> bytes: - previous_timeout = sock.gettimeout() - sock.settimeout(timeout) - try: - return sock.recv(1024) - except socket.timeout: - return b"" - finally: - sock.settimeout(previous_timeout) - - -def send_phone_sequence( - host: str, - port: int, - phone: str, - phone_index: int, - frames: int, - event_time: dt.datetime, - timeout: float, - rate: float, - linger_ms: int, - post_register_ms: int, -) -> tuple[bytes, int]: - with socket.create_connection((host, port), timeout=timeout) as sock: - sock.settimeout(timeout) - device_id = "D" + phone[-6:] - plate = "B" + phone[-5:] - sock.sendall(build_frame(0x0100, phone, 100 + phone_index, build_register_body(device_id, plate))) - ack = recv_optional(sock, timeout) - if post_register_ms > 0: - time.sleep(post_register_ms / 1000) - sent = 0 - for frame_index in range(frames): - serial = 1000 + phone_index * frames + frame_index - sock.sendall(build_frame( - 0x0200, - phone, - serial, - build_location_body(event_time + dt.timedelta(seconds=frame_index), serial), - )) - sent += 1 - if rate > 0: - time.sleep(1 / rate) - if linger_ms > 0: - time.sleep(linger_ms / 1000) - return ack, sent - - -@dataclass(frozen=True) -class SendWorkloadResult: - sent_registers: int - sent_locations: int - elapsed_seconds: float - - -def send_one_phone(args: argparse.Namespace, - phone: str, - phone_index: int, - event_time: dt.datetime) -> tuple[bytes, int]: - if args.connection_mode == "session": - return send_phone_sequence( - args.tcp_host, - args.tcp_port, - phone, - phone_index, - args.frames, - event_time, - args.tcp_timeout, - args.rate, - args.linger_ms, - args.post_register_ms, - ) - - ack = send_register(args.tcp_host, args.tcp_port, phone, 100 + phone_index, args.tcp_timeout) - if args.post_register_ms > 0: - time.sleep(args.post_register_ms / 1000) - sent = 0 - for frame_index in range(args.frames): - serial = 1000 + phone_index * args.frames + frame_index - send_location( - args.tcp_host, - args.tcp_port, - phone, - serial, - event_time + dt.timedelta(seconds=frame_index), - args.tcp_timeout, - args.linger_ms, - ) - sent += 1 - if args.rate > 0: - time.sleep(1 / args.rate) - return ack, sent - - -def send_phone_workload(args: argparse.Namespace, - phones: list[str], - event_time: dt.datetime) -> SendWorkloadResult: - sent_locations = 0 - start = time.monotonic() - workers = max(1, int(getattr(args, "workers", 1))) - - def run_one(index_and_phone: tuple[int, str]) -> tuple[bytes, int]: - phone_index, phone = index_and_phone - return send_one_phone(args, phone, phone_index, event_time) - - indexed_phones = list(enumerate(phones)) - if workers == 1 or len(indexed_phones) <= 1: - results = [run_one(item) for item in indexed_phones] - else: - with concurrent.futures.ThreadPoolExecutor(max_workers=workers) as executor: - results = list(executor.map(run_one, indexed_phones)) - - for phone, (ack, sent) in zip(phones, results): - if args.require_register_ack and not ack: - raise RuntimeError(f"register ack timeout for {phone}") - sent_locations += sent - - elapsed = max(time.monotonic() - start, 0.001) - return SendWorkloadResult( - sent_registers=len(phones), - sent_locations=sent_locations, - elapsed_seconds=elapsed, - ) - - -def query_json(base_url: str, path: str, params: dict[str, Any], timeout: float) -> dict[str, Any]: - url = base_url.rstrip("/") + path + "?" + urllib.parse.urlencode(params) - request = urllib.request.Request(url, headers={"Accept": "application/json"}) - with urllib.request.urlopen(request, timeout=timeout) as response: - return json.loads(response.read().decode("utf-8")) - - -def location_params(phone: str, date_from: str, date_to: str, limit: int) -> dict[str, Any]: - return { - "phone": phone, - "dateFrom": date_from, - "dateTo": date_to, - "limit": limit, - "order": "ASC", - } - - -def next_page_params(params: dict[str, Any], page: dict[str, Any]) -> dict[str, Any]: - cursor = page.get("nextCursor") or {} - next_params = dict(params) - if cursor: - next_params["cursorTs"] = cursor["ts"] - next_params["cursorId"] = cursor["id"] - return next_params - - -def archive_path(root: pathlib.Path, uri: str) -> pathlib.Path: - prefix = "archive://" - if not uri.startswith(prefix): - raise ValueError(f"unsupported archive uri: {uri}") - return root / uri[len(prefix):] - - -def raw_frame_count_sql(phone: str, items: list[dict[str, Any]]) -> str: - sqls = raw_frame_count_sqls(phone, items) - if not sqls: - raise ValueError("no raw uri found in jt808 location items") - return sqls[0] - - -def raw_frame_count_sqls(phone: str, items: list[dict[str, Any]]) -> list[str]: - seen: set[str] = set() - sqls: list[str] = [] - for item in items: - raw_uri = item.get("rawUri") or "" - if not raw_uri or raw_uri in seen: - continue - seen.add(raw_uri) - sqls.append(tdengine_smoke.raw_frame_count_sql( - protocol="JT808", - vehicle_key=item.get("vehicleKey") or ("jt808:" + phone), - phone=phone, - raw_uri=raw_uri, - )) - return sqls - - -def verify_tdengine_raw_frames( - rest_url: str, - username: str, - password: str, - sqls: list[str], - timeout_seconds: float, - request_timeout: float, -) -> int: - if not sqls: - return 0 - verified = 0 - for sql in sqls: - count = wait_for_tdengine_raw_frames( - rest_url, username, password, sql, 1, timeout_seconds, request_timeout) - if count < 1: - raise RuntimeError(f"tdengine raw_frames row not visible for query: {sql}") - verified += 1 - return verified - - -def wait_for_tdengine_raw_frames( - rest_url: str, - username: str, - password: str, - sql: str, - minimum: int, - timeout_seconds: float, - request_timeout: float, -) -> int: - deadline = time.monotonic() + timeout_seconds - last_count = 0 - while time.monotonic() < deadline: - last_count = tdengine_smoke.query_count(rest_url, username, password, sql, request_timeout) - if last_count >= minimum: - return last_count - time.sleep(1) - return last_count - - -def wait_for_locations( - history_base_url: str, - phone: str, - date_from: str, - date_to: str, - minimum: int, - timeout_seconds: float, - request_timeout: float, -) -> dict[str, Any]: - deadline = time.monotonic() + timeout_seconds - params = location_params(phone, date_from, date_to, max(minimum, 10)) - last_page: dict[str, Any] = {} - while time.monotonic() < deadline: - last_page = query_json(history_base_url, "/api/event-history/jt808/locations", params, request_timeout) - if len(last_page.get("items") or []) >= minimum: - return last_page - time.sleep(1) - return last_page - - -def verify_pagination(history_base_url: str, phone: str, date_from: str, date_to: str, request_timeout: float) -> bool: - first_params = location_params(phone, date_from, date_to, 1) - first = query_json(history_base_url, "/api/event-history/jt808/locations", first_params, request_timeout) - if not first.get("items") or not first.get("nextCursor"): - return False - second = query_json( - history_base_url, - "/api/event-history/jt808/locations", - next_page_params(first_params, first), - request_timeout, - ) - return bool(second.get("items")) - - -def generated_phone(start_phone: str, offset: int) -> str: - return str(int(start_phone) + offset) - - -def expected_history_count(args: argparse.Namespace) -> int: - minimum = min(args.frames, args.expect_history_count) - if args.verify_pagination and args.frames > 1: - minimum = max(minimum, 2) - return minimum - - -def run(args: argparse.Namespace) -> dict[str, Any]: - event_time = parse_event_time(args.event_time) - date_from = (event_time - dt.timedelta(minutes=5)).isoformat() - date_to = (event_time + dt.timedelta(minutes=args.frames + 5)).isoformat() - phones = [generated_phone(args.start_phone, index) for index in range(args.phones)] - workload = send_phone_workload(args, phones, event_time) - first_phone = phones[0] - page = wait_for_locations( - args.history_base_url, - first_phone, - date_from, - date_to, - expected_history_count(args), - args.history_timeout, - args.http_timeout, - ) - items = page.get("items") or [] - if len(items) < expected_history_count(args): - raise RuntimeError(f"history rows not visible for {first_phone}: got {len(items)}") - - archive_checked = 0 - if args.archive_root: - root = pathlib.Path(args.archive_root) - for item in items[: args.archive_check_limit]: - path = archive_path(root, item["rawUri"]) - if not path.exists(): - raise RuntimeError(f"raw archive missing: {path}") - archive_checked += 1 - - tdengine_raw_frames = None - if args.tdengine_rest_url: - tdengine_raw_frames = verify_tdengine_raw_frames( - args.tdengine_rest_url, - args.tdengine_username, - args.tdengine_password, - raw_frame_count_sqls(first_phone, items), - args.history_timeout, - args.http_timeout, - ) - - pagination_ok = None - if args.verify_pagination and args.frames > 1: - pagination_ok = verify_pagination(args.history_base_url, first_phone, date_from, date_to, args.http_timeout) - if not pagination_ok: - raise RuntimeError("cursor pagination verification failed") - - return { - "tcpHost": args.tcp_host, - "tcpPort": args.tcp_port, - "connectionMode": args.connection_mode, - "workers": max(1, int(args.workers)), - "phones": phones, - "sentRegisters": workload.sent_registers, - "sentLocations": workload.sent_locations, - "sendElapsedSeconds": round(workload.elapsed_seconds, 3), - "sendLocationsPerSecond": round(workload.sent_locations / workload.elapsed_seconds, 2), - "historyRowsForFirstPhone": len(items), - "tdengineRawFrames": tdengine_raw_frames, - "archiveChecked": archive_checked, - "paginationOk": pagination_ok, - "firstFrameId": items[0].get("frameId") if items else None, - "firstRawUri": items[0].get("rawUri") if items else None, - } - - -def parser() -> argparse.ArgumentParser: - p = argparse.ArgumentParser(description=__doc__) - p.add_argument("--tcp-host", default="127.0.0.1") - p.add_argument("--tcp-port", type=int, default=808) - p.add_argument("--history-base-url", default="http://127.0.0.1:20200") - p.add_argument("--connection-mode", choices=["per-frame", "session"], default="per-frame") - p.add_argument("--start-phone", default="13079962000") - p.add_argument("--phones", type=int, default=1) - p.add_argument("--workers", type=int, default=1, help="parallel phone sessions; 1 keeps legacy serial behavior") - p.add_argument("--frames", type=int, default=3, help="location frames per phone") - p.add_argument("--rate", type=float, default=0, help="location send rate per second; 0 means unlimited") - p.add_argument("--linger-ms", type=int, default=2500, help="time to keep each terminal connection open after writes") - p.add_argument("--post-register-ms", type=int, default=1000, help="pause after register ack before location frames") - p.add_argument("--event-time", default="2024-01-02T03:04:05") - p.add_argument("--expect-history-count", type=int, default=1) - p.add_argument("--verify-pagination", action="store_true") - p.add_argument("--archive-root", default="") - p.add_argument("--archive-check-limit", type=int, default=3) - p.add_argument("--tdengine-rest-url", default=os.environ.get("TDENGINE_REST_URL", "")) - p.add_argument("--tdengine-username", default=os.environ.get("TDENGINE_USERNAME", "root")) - p.add_argument("--tdengine-password", default=os.environ.get("TDENGINE_PASSWORD", "taosdata")) - p.add_argument("--require-register-ack", action="store_true", default=True) - p.add_argument("--tcp-timeout", type=float, default=2) - p.add_argument("--http-timeout", type=float, default=5) - p.add_argument("--history-timeout", type=float, default=30) - return p - - -def main() -> None: - args = parser().parse_args() - summary = run(args) - print(json.dumps(summary, ensure_ascii=False, indent=2)) - - -if __name__ == "__main__": - main() diff --git a/tools/tdengine_smoke.py b/tools/tdengine_smoke.py deleted file mode 100644 index 06ad0524..00000000 --- a/tools/tdengine_smoke.py +++ /dev/null @@ -1,61 +0,0 @@ -#!/usr/bin/env python3 -"""Small TDengine REST helpers for local smoke verifiers.""" - -from __future__ import annotations - -import base64 -import json -import urllib.request -from typing import Any - - -def sql_literal(value: str) -> str: - return "'" + (value or "").replace("'", "''") + "'" - - -def raw_frame_count_sql( - *, - protocol: str, - vehicle_key: str = "", - vin: str = "", - phone: str = "", - raw_uri: str = "", -) -> str: - if not protocol or not protocol.strip(): - raise ValueError("protocol is required") - filters = ["protocol = " + sql_literal(protocol.strip().upper())] - optional_filters = { - "vehicle_key": vehicle_key, - "vin": vin, - "phone": phone, - "raw_uri": raw_uri, - } - present = False - for column, value in optional_filters.items(): - if value and value.strip(): - filters.append(column + " = " + sql_literal(value.strip())) - present = True - if not present: - raise ValueError("at least one identity or raw_uri filter is required") - return "SELECT COUNT(*) FROM raw_frames WHERE " + " AND ".join(filters) - - -def count_from_response(response: dict[str, Any]) -> int: - if response.get("code") != 0: - raise RuntimeError(f"tdengine query failed: {response}") - data = response.get("data") or [] - if not data or not data[0]: - return 0 - return int(data[0][0]) - - -def query_count(rest_url: str, username: str, password: str, sql: str, timeout: float) -> int: - request = urllib.request.Request(rest_url, data=sql.encode("utf-8"), method="POST") - if username: - token = base64.b64encode(f"{username}:{password}".encode("utf-8")).decode("ascii") - request.add_header("Authorization", "Basic " + token) - request.add_header("Content-Type", "text/plain; charset=utf-8") - request.add_header("Accept", "application/json") - with urllib.request.urlopen(request, timeout=timeout) as response: - payload = json.loads(response.read().decode("utf-8")) - return count_from_response(payload) diff --git a/tools/test_gb32960_e2e_smoke.py b/tools/test_gb32960_e2e_smoke.py deleted file mode 100644 index 66a16e0b..00000000 --- a/tools/test_gb32960_e2e_smoke.py +++ /dev/null @@ -1,107 +0,0 @@ -import datetime as dt -import pathlib -import tempfile -import unittest - -from tools import gb32960_e2e_smoke as smoke - - -class Gb32960E2ESmokeTest(unittest.TestCase): - def test_parse_frame_info_reads_vin_and_collect_time(self): - frame = bytes.fromhex( - "232302fe4c544553543230323630363239303030310100081a061d10060f" - "02002a" - ) - - info = smoke.parse_frame_info(frame) - - self.assertEqual(info.vin, "LTEST202606290001") - self.assertEqual(info.command, 0x02) - self.assertEqual(info.event_time, dt.datetime(2026, 6, 29, 16, 6, 15, tzinfo=smoke.SHANGHAI)) - - def test_query_range_surrounds_event_time_with_offset(self): - date_from, date_to = smoke.query_range( - dt.datetime(2026, 6, 29, 8, 6, 15, tzinfo=smoke.SHANGHAI), - minutes=5, - ) - - self.assertEqual(date_from, "2026-06-29T08:01:15+08:00") - self.assertEqual(date_to, "2026-06-29T08:11:15+08:00") - - def test_valid_ack_requires_frame_marker(self): - self.assertTrue(smoke.valid_ack(b"##\x02\x01")) - self.assertTrue(smoke.valid_ack(b"$$\x02\x01")) - self.assertFalse(smoke.valid_ack(b"")) - self.assertFalse(smoke.valid_ack(b"\x7e\x80")) - - def test_archive_path_maps_archive_uri_to_root(self): - path = smoke.archive_path( - pathlib.Path("/tmp/archive"), - "archive://2026/06/29/GB32960/LTEST202606290001/frame.bin", - ) - - self.assertEqual( - path, - pathlib.Path("/tmp/archive/2026/06/29/GB32960/LTEST202606290001/frame.bin"), - ) - - def test_parse_field_keys_trims_blank_items(self): - keys = smoke.parse_field_keys("speed_kmh, total_mileage_km, ,longitude") - - self.assertEqual(keys, ["speed_kmh", "total_mileage_km", "longitude"]) - - def test_first_raw_uri_reads_record_archive_uri(self): - uri = smoke.first_raw_uri([ - {"rawArchiveUri": ""}, - {"rawArchiveUri": "archive://2026/06/29/GB32960/LTEST/frame.bin"}, - ]) - - self.assertEqual(uri, "archive://2026/06/29/GB32960/LTEST/frame.bin") - - def test_raw_frame_count_sql_filters_by_vin_and_raw_uri(self): - info = smoke.FrameInfo("LTEST202606290001", 0x02, None) - sql = smoke.raw_frame_count_sql(info, [ - {"rawArchiveUri": "archive://2026/06/29/GB32960/LTEST202606290001/frame.bin"}, - ]) - - self.assertEqual( - sql, - "SELECT COUNT(*) FROM raw_frames WHERE protocol = 'GB32960' " - "AND vehicle_key = 'LTEST202606290001' " - "AND vin = 'LTEST202606290001' " - "AND raw_uri = 'archive://2026/06/29/GB32960/LTEST202606290001/frame.bin'", - ) - - def test_raw_frame_count_sqls_include_each_unique_record_raw_uri(self): - info = smoke.FrameInfo("LTEST202606290001", 0x02, None) - sqls = smoke.raw_frame_count_sqls(info, [ - {"rawArchiveUri": "archive://2026/06/29/GB32960/LTEST202606290001/frame-1.bin"}, - {"rawArchiveUri": "archive://2026/06/29/GB32960/LTEST202606290001/frame-2.bin"}, - {"rawArchiveUri": "archive://2026/06/29/GB32960/LTEST202606290001/frame-1.bin"}, - ]) - - self.assertEqual(len(sqls), 2) - self.assertTrue(sqls[0].endswith("frame-1.bin'")) - self.assertTrue(sqls[1].endswith("frame-2.bin'")) - - def test_check_archive_ignores_duplicate_uris(self): - with tempfile.TemporaryDirectory() as temp_dir: - root = pathlib.Path(temp_dir) - archive_file = root / "2026/06/29/GB32960/LTEST/frame.bin" - archive_file.parent.mkdir(parents=True) - archive_file.write_bytes(b"raw") - - checked = smoke.check_archive( - root, - [ - {"rawArchiveUri": "archive://2026/06/29/GB32960/LTEST/frame.bin"}, - {"rawArchiveUri": "archive://2026/06/29/GB32960/LTEST/frame.bin"}, - ], - limit=3, - ) - - self.assertEqual(checked, 1) - - -if __name__ == "__main__": - unittest.main() diff --git a/tools/test_go_kafka_prod_smoke.py b/tools/test_go_kafka_prod_smoke.py deleted file mode 100644 index 9b6d5b74..00000000 --- a/tools/test_go_kafka_prod_smoke.py +++ /dev/null @@ -1,62 +0,0 @@ -import unittest - -from tools import go_kafka_prod_smoke as smoke - - -class GoKafkaProdSmokeTest(unittest.TestCase): - def test_parse_topic_list_marks_required_topics_present(self): - topics = smoke.parse_topic_list(""" -vehicle.raw.go.gb32960.v1 -vehicle.raw.go.jt808.v1 -vehicle.raw.go.yutong-mqtt.v1 -vehicle.event.go.unified.v1 -""") - - checks = smoke.topic_checks(topics, smoke.DEFAULT_TOPICS) - - self.assertTrue(all(check.status == "pass" for check in checks)) - - def test_topic_checks_fail_when_required_topic_missing(self): - topics = {"vehicle.raw.go.gb32960.v1"} - - checks = smoke.topic_checks(topics, ["vehicle.raw.go.gb32960.v1", "vehicle.event.go.unified.v1"]) - - by_name = {check.name: check for check in checks} - self.assertEqual(by_name["topic.vehicle.event.go.unified.v1"].status, "fail") - - def test_parse_consumer_group_describe_reads_lag_rows(self): - rows = smoke.parse_consumer_group_describe(""" -GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG -go-realtime-api vehicle.event.go.unified.v1 0 15992 15992 0 -go-realtime-api vehicle.event.go.unified.v1 1 2036 2036 3 -""") - - self.assertEqual(rows[0].group, "go-realtime-api") - self.assertEqual(rows[1].topic, "vehicle.event.go.unified.v1") - self.assertEqual(rows[1].lag, 3) - - def test_lag_checks_fail_when_lag_exceeds_threshold(self): - rows = [ - smoke.ConsumerLag("go-history-writer", "vehicle.raw.go.jt808.v1", 0, 100, 100, 0), - smoke.ConsumerLag("go-history-writer", "vehicle.raw.go.jt808.v1", 1, 100, 110, 10), - ] - - check = smoke.group_lag_check("go-history-writer", rows, max_lag=5) - - self.assertEqual(check.status, "fail") - self.assertIn("max_lag=10", check.message) - - def test_lag_checks_pass_when_group_has_rows_and_lag_is_small(self): - rows = [ - smoke.ConsumerLag("go-stat-writer", "vehicle.raw.go.gb32960.v1", 0, 100, 100, 0), - smoke.ConsumerLag("go-stat-writer", "vehicle.raw.go.jt808.v1", 1, 100, 101, 1), - ] - - check = smoke.group_lag_check("go-stat-writer", rows, max_lag=5) - - self.assertEqual(check.status, "pass") - self.assertIn("max_lag=1", check.message) - - -if __name__ == "__main__": - unittest.main() diff --git a/tools/test_go_native_deploy.py b/tools/test_go_native_deploy.py deleted file mode 100644 index 24110410..00000000 --- a/tools/test_go_native_deploy.py +++ /dev/null @@ -1,63 +0,0 @@ -import unittest - -from tools import go_native_deploy as deploy - - -class GoNativeDeployTest(unittest.TestCase): - def test_build_commands_cover_four_production_binaries(self): - commands = deploy.build_commands("/repo/go/vehicle-gateway", "/tmp/out") - - self.assertEqual([command.output_name for command in commands], [ - "gateway", - "history-writer", - "stat-writer", - "realtime-api", - ]) - for command in commands: - self.assertEqual(command.argv[:3], ["go", "build", "-trimpath"]) - self.assertIn("GOOS=linux", command.env) - self.assertIn("GOARCH=amd64", command.env) - self.assertIn("CGO_ENABLED=0", command.env) - - def test_remote_switch_command_creates_release_and_restarts_all_services(self): - command = deploy.remote_switch_command("409f55b", "/opt/lingniu-go-native", "/tmp/lingniu-go-native-409f55b.tar.gz") - - self.assertIn("/opt/lingniu-go-native/releases/409f55b", command) - self.assertIn("ln -sfn", command) - self.assertIn("systemctl restart lingniu-go-gateway", command) - self.assertIn("systemctl restart lingniu-go-history-writer", command) - self.assertIn("systemctl restart lingniu-go-stat-writer", command) - self.assertIn("systemctl restart lingniu-go-realtime-api", command) - - def test_acceptance_command_uses_deployed_hosts_and_date(self): - command = deploy.acceptance_command( - app_host="115.29.187.205", - kafka_host="114.55.58.251", - date="2026-07-02", - timeout=20, - ) - - self.assertEqual(command[:2], ["python3", "tools/go_prod_acceptance.py"]) - self.assertIn("--app-host", command) - self.assertIn("115.29.187.205", command) - self.assertIn("--kafka-host", command) - self.assertIn("114.55.58.251", command) - self.assertIn("--date", command) - self.assertIn("2026-07-02", command) - - def test_parse_spool_status_reads_file_and_recent_counts(self): - status = deploy.parse_spool_status("files=42 recent=3") - - self.assertEqual(status.files, 42) - self.assertEqual(status.recent, 3) - - def test_spool_status_command_targets_gateway_spool_directory(self): - command = deploy.spool_status_command("/opt/lingniu-go-native") - - self.assertIn("/opt/lingniu-go-native/spool/gateway", command) - self.assertIn("files=", command) - self.assertIn("recent=", command) - - -if __name__ == "__main__": - unittest.main() diff --git a/tools/test_go_native_prod_smoke.py b/tools/test_go_native_prod_smoke.py deleted file mode 100644 index bbe968c9..00000000 --- a/tools/test_go_native_prod_smoke.py +++ /dev/null @@ -1,358 +0,0 @@ -import datetime as dt -import unittest - -from tools import go_native_prod_smoke as smoke - - -class GoNativeProdSmokeTest(unittest.TestCase): - def test_shanghai_day_window_uses_plus_eight_bounds(self): - now = dt.datetime(2026, 7, 2, 1, 35, tzinfo=smoke.SHANGHAI) - - date_from, date_to = smoke.shanghai_day_window(now) - - self.assertEqual(date_from, "2026-07-02T00:00:00+08:00") - self.assertEqual(date_to, "2026-07-03T00:00:00+08:00") - - def test_api_url_encodes_plus_eight_date_range(self): - url = smoke.api_url( - "http://example.test/base/", - "/api/history/raw-frames", - { - "protocol": "JT808", - "dateFrom": "2026-07-02T00:00:00+08:00", - "dateTo": "2026-07-03T00:00:00+08:00", - "limit": 1, - }, - ) - - self.assertEqual( - url, - "http://example.test/base/api/history/raw-frames?" - "protocol=JT808&dateFrom=2026-07-02T00%3A00%3A00%2B08%3A00" - "&dateTo=2026-07-03T00%3A00%3A00%2B08%3A00&limit=1", - ) - - def test_explicit_today_date_still_enforces_raw_freshness(self): - now = dt.datetime(2026, 7, 2, 9, 30, tzinfo=smoke.SHANGHAI) - - window = smoke.resolve_check_window("2026-07-02", 15.0, now) - - self.assertEqual(window.date_from, "2026-07-02T00:00:00+08:00") - self.assertEqual(window.date_to, "2026-07-03T00:00:00+08:00") - self.assertEqual(window.stat_date, "2026-07-02") - self.assertEqual(window.max_raw_age_minutes, 15.0) - - def test_explicit_historical_date_disables_raw_freshness(self): - now = dt.datetime(2026, 7, 2, 9, 30, tzinfo=smoke.SHANGHAI) - - window = smoke.resolve_check_window("2026-07-01", 15.0, now) - - self.assertEqual(window.date_from, "2026-07-01T00:00:00+08:00") - self.assertEqual(window.date_to, "2026-07-02T00:00:00+08:00") - self.assertEqual(window.stat_date, "2026-07-01") - self.assertIsNone(window.max_raw_age_minutes) - - def test_total_check_passes_when_total_reaches_minimum(self): - check = smoke.check_total( - "jt808.raw", - {"total": 2, "items": [{"vehicle_key": "JT808:013307811170"}]}, - minimum=1, - ) - - self.assertEqual(check.status, "pass") - self.assertEqual(check.count, 2) - self.assertIn("JT808:013307811170", check.message) - - def test_total_check_fails_when_total_is_below_minimum(self): - check = smoke.check_total("gb32960.raw", {"total": 0, "items": []}, minimum=1) - - self.assertEqual(check.status, "fail") - self.assertEqual(check.count, 0) - self.assertIn("expected >= 1", check.message) - - def test_overall_status_fails_if_any_check_fails(self): - checks = [ - smoke.Check("a", "pass", 1, 1, "ok"), - smoke.Check("b", "fail", 0, 1, "bad"), - ] - - self.assertEqual(smoke.overall_status(checks), "fail") - - def test_build_checks_cover_three_raw_protocols_and_two_daily_metric_protocols(self): - specs = smoke.build_check_specs( - date_from="2026-07-02T00:00:00+08:00", - date_to="2026-07-03T00:00:00+08:00", - stat_date="2026-07-02", - min_raw=1, - min_history=1, - min_stat=1, - ) - - names = [spec.name for spec in specs] - - self.assertIn("gb32960.raw", names) - self.assertIn("jt808.raw", names) - self.assertIn("yutong_mqtt.raw", names) - self.assertIn("gb32960.locations", names) - self.assertIn("gb32960.mileage_points", names) - self.assertIn("jt808.locations", names) - self.assertIn("jt808.mileage_points", names) - self.assertIn("yutong_mqtt.locations", names) - self.assertIn("yutong_mqtt.mileage_points", names) - self.assertIn("gb32960.daily_mileage", names) - self.assertIn("jt808.daily_total_mileage", names) - - def test_raw_checks_order_by_received_at_for_ingest_freshness(self): - specs = smoke.build_check_specs( - date_from="2026-07-02T00:00:00+08:00", - date_to="2026-07-03T00:00:00+08:00", - stat_date="2026-07-02", - min_raw=1, - min_history=1, - min_stat=1, - max_raw_age_minutes=15, - ) - - raw_specs = [spec for spec in specs if spec.name.endswith(".raw")] - - self.assertTrue(raw_specs) - for spec in raw_specs: - self.assertEqual(spec.params.get("orderBy"), "receivedAt") - self.assertEqual(spec.params.get("includeTotal"), "false") - self.assertNotIn("dateFrom", spec.params) - self.assertNotIn("dateTo", spec.params) - - def test_vehicle_identifier_prefers_vin_over_vehicle_key(self): - identifier = smoke.vehicle_identifier({ - "vin": "LB9A32A20R0LS1343", - "vehicle_key": "GB32960:ignored", - }) - - self.assertEqual(identifier, "LB9A32A20R0LS1343") - - def test_vehicle_identifier_falls_back_to_vehicle_key(self): - identifier = smoke.vehicle_identifier({ - "vin": "", - "vehicle_key": "JT808:013307811170", - }) - - self.assertEqual(identifier, "JT808:013307811170") - - def test_realtime_specs_are_built_from_raw_samples_with_vin(self): - payloads = { - "gb32960.raw": {"items": [{"vin": "LB9A32A20R0LS1343", "vehicle_key": "LB9A32A20R0LS1343"}]}, - "jt808.raw": {"items": [{"vin": "", "vehicle_key": "JT808:013307811170"}]}, - "yutong_mqtt.raw": {"items": [{"vin": "LMRKH9AC6R1004108"}]}, - } - - specs = smoke.build_realtime_specs(payloads) - - self.assertEqual( - [(spec.name, spec.path) for spec in specs], - [ - ("gb32960.realtime", "/api/realtime/vehicles/LB9A32A20R0LS1343"), - ("gb32960.realtime_online", "/api/realtime/vehicles/LB9A32A20R0LS1343/online"), - ("gb32960.realtime_protocol", "/api/realtime/vehicles/LB9A32A20R0LS1343/protocols/GB32960"), - ("jt808.realtime", "/api/realtime/vehicles/JT808%3A013307811170"), - ("jt808.realtime_online", "/api/realtime/vehicles/JT808%3A013307811170/online"), - ("jt808.realtime_protocol", "/api/realtime/vehicles/JT808%3A013307811170/protocols/JT808"), - ("yutong_mqtt.realtime", "/api/realtime/vehicles/LMRKH9AC6R1004108"), - ("yutong_mqtt.realtime_online", "/api/realtime/vehicles/LMRKH9AC6R1004108/online"), - ("yutong_mqtt.realtime_protocol", "/api/realtime/vehicles/LMRKH9AC6R1004108/protocols/YUTONG_MQTT"), - ], - ) - - def test_realtime_check_requires_online_true_when_field_exists(self): - check = smoke.check_realtime( - "gb32960.realtime_online", - {"vin": "LB9A32A20R0LS1343", "online": True, "protocols": ["GB32960"]}, - ) - - self.assertEqual(check.status, "pass") - self.assertEqual(check.count, 1) - - def test_realtime_check_fails_when_online_false(self): - check = smoke.check_realtime( - "gb32960.realtime_online", - {"vin": "LB9A32A20R0LS1343", "online": False}, - ) - - self.assertEqual(check.status, "fail") - self.assertIn("online=false", check.message) - - def test_realtime_check_fails_when_snapshot_is_stale(self): - now = dt.datetime(2026, 7, 2, 0, 30, 0, tzinfo=dt.timezone.utc) - - check = smoke.check_realtime( - "jt808.realtime", - {"vehicle_key": "JT808:013307811170", "updated_at_ms": 1782950400000}, - max_age_minutes=15, - now=now, - ) - - self.assertEqual(check.status, "fail") - self.assertIn("updated_age_minutes=30.0", check.message) - - def test_realtime_check_passes_when_snapshot_is_fresh(self): - now = dt.datetime(2026, 7, 2, 0, 30, 0, tzinfo=dt.timezone.utc) - - check = smoke.check_realtime( - "jt808.realtime", - {"vehicle_key": "JT808:013307811170", "updated_at_ms": 1782951600000}, - max_age_minutes=15, - now=now, - ) - - self.assertEqual(check.status, "pass") - self.assertIn("updated_age_minutes=10.0", check.message) - - def test_parse_tdengine_utc_timestamp_as_aware_utc(self): - parsed = smoke.parse_tdengine_utc_timestamp("2026-07-01 17:36:02") - - self.assertEqual(parsed, dt.datetime(2026, 7, 1, 17, 36, 2, tzinfo=dt.timezone.utc)) - - def test_total_check_fails_when_latest_sample_is_stale(self): - now = dt.datetime(2026, 7, 1, 18, 0, 0, tzinfo=dt.timezone.utc) - - check = smoke.check_total( - "gb32960.raw", - {"total": 1, "items": [{"ts": "2026-07-01 17:00:00"}]}, - minimum=1, - max_age_minutes=30, - now=now, - ) - - self.assertEqual(check.status, "fail") - self.assertIn("latest_age_minutes=60.0", check.message) - - def test_total_check_uses_received_at_for_raw_freshness_when_present(self): - now = dt.datetime(2026, 7, 1, 18, 0, 0, tzinfo=dt.timezone.utc) - - check = smoke.check_total( - "gb32960.raw", - { - "total": 1, - "items": [{ - "ts": "2026-07-01 17:00:00", - "received_at": "2026-07-01 17:58:00", - }], - }, - minimum=1, - max_age_minutes=15, - now=now, - ) - - self.assertEqual(check.status, "pass") - self.assertIn("latest_age_minutes=2.0", check.message) - - def test_total_check_passes_when_latest_sample_is_fresh(self): - now = dt.datetime(2026, 7, 1, 18, 0, 0, tzinfo=dt.timezone.utc) - - check = smoke.check_total( - "gb32960.raw", - {"total": 1, "items": [{"ts": "2026-07-01 17:45:00"}]}, - minimum=1, - max_age_minutes=30, - now=now, - ) - - self.assertEqual(check.status, "pass") - self.assertIn("latest_age_minutes=15.0", check.message) - - def test_raw_check_requires_structured_parsed_json(self): - check = smoke.check_total( - "jt808.raw", - { - "total": 1, - "items": [{ - "ts": "2026-07-01 17:45:00", - "parsed_json": "", - }], - }, - minimum=1, - require_parsed_json=True, - ) - - self.assertEqual(check.status, "fail") - self.assertIn("missing parsed_json", check.message) - - def test_raw_check_accepts_structured_parsed_json(self): - check = smoke.check_total( - "gb32960.raw", - { - "total": 1, - "items": [{ - "ts": "2026-07-01 17:45:00", - "parsed_json": "{\"header\":{\"vin\":\"LTEST\"}}", - }], - }, - minimum=1, - require_parsed_json=True, - ) - - self.assertEqual(check.status, "pass") - self.assertIn("parsed_json=ok", check.message) - - def test_history_check_requires_frame_id_backlink(self): - check = smoke.check_total( - "jt808.locations", - {"total": 1, "items": [{"ts": "2026-07-01 17:45:00", "frame_id": ""}]}, - minimum=1, - require_frame_id=True, - ) - - self.assertEqual(check.status, "fail") - self.assertIn("missing frame_id", check.message) - - def test_history_check_accepts_frame_id_backlink(self): - check = smoke.check_total( - "jt808.mileage_points", - {"total": 1, "items": [{"ts": "2026-07-01 17:45:00", "frame_id": "go_abc"}]}, - minimum=1, - require_frame_id=True, - ) - - self.assertEqual(check.status, "pass") - self.assertIn("frame_id=ok", check.message) - - def test_daily_mileage_formula_requires_latest_minus_first(self): - check = smoke.check_total( - "jt808.daily_mileage", - { - "total": 1, - "items": [{ - "metric_key": "daily_mileage_km", - "metric_value": 40.0, - "first_total_mileage_km": 4434.9, - "latest_total_mileage_km": 4481.0, - }], - }, - minimum=1, - require_metric_formula=True, - ) - - self.assertEqual(check.status, "fail") - self.assertIn("metric_formula mismatch", check.message) - - def test_daily_total_formula_requires_latest_total(self): - check = smoke.check_total( - "jt808.daily_total_mileage", - { - "total": 1, - "items": [{ - "metric_key": "daily_total_mileage_km", - "metric_value": 4481.0, - "first_total_mileage_km": 4434.9, - "latest_total_mileage_km": 4481.0, - }], - }, - minimum=1, - require_metric_formula=True, - ) - - self.assertEqual(check.status, "pass") - self.assertIn("metric_formula=ok", check.message) - - -if __name__ == "__main__": - unittest.main() diff --git a/tools/test_go_prod_acceptance.py b/tools/test_go_prod_acceptance.py deleted file mode 100644 index ac902d91..00000000 --- a/tools/test_go_prod_acceptance.py +++ /dev/null @@ -1,53 +0,0 @@ -import argparse -import sys -import unittest - -from tools import go_prod_acceptance as acceptance - - -class GoProdAcceptanceTest(unittest.TestCase): - def test_build_commands_runs_systemd_kafka_and_http_smokes(self): - args = argparse.Namespace( - app_host="115.29.187.205", - kafka_host="114.55.58.251", - ssh_user="root", - base_url="http://115.29.187.205:20210", - date="2026-07-02", - timeout=8.0, - max_lag=100, - ) - - commands = acceptance.build_commands(args) - - self.assertEqual(len(commands), 3) - self.assertEqual(commands[0].name, "systemd") - self.assertEqual(commands[0].argv[:2], [sys.executable, "tools/go_systemd_prod_smoke.py"]) - self.assertIn("--host", commands[0].argv) - self.assertIn("115.29.187.205", commands[0].argv) - self.assertEqual(commands[1].name, "kafka") - self.assertIn("tools/go_kafka_prod_smoke.py", commands[1].argv) - self.assertIn("--max-lag", commands[1].argv) - self.assertEqual(commands[2].name, "http") - self.assertIn("tools/go_native_prod_smoke.py", commands[2].argv) - self.assertIn("--base-url", commands[2].argv) - - def test_overall_status_fails_when_any_child_fails(self): - results = [ - acceptance.ChildResult("systemd", "pass", 0, {"status": "pass"}), - acceptance.ChildResult("kafka", "fail", 1, {"status": "fail"}), - ] - - self.assertEqual(acceptance.overall_status(results), "fail") - - def test_overall_status_passes_when_all_children_pass(self): - results = [ - acceptance.ChildResult("systemd", "pass", 0, {"status": "pass"}), - acceptance.ChildResult("kafka", "pass", 0, {"status": "pass"}), - acceptance.ChildResult("http", "pass", 0, {"status": "pass"}), - ] - - self.assertEqual(acceptance.overall_status(results), "pass") - - -if __name__ == "__main__": - unittest.main() diff --git a/tools/test_go_systemd_prod_smoke.py b/tools/test_go_systemd_prod_smoke.py deleted file mode 100644 index 30e4962e..00000000 --- a/tools/test_go_systemd_prod_smoke.py +++ /dev/null @@ -1,101 +0,0 @@ -import unittest - -from tools import go_systemd_prod_smoke as smoke - - -class GoSystemdProdSmokeTest(unittest.TestCase): - def test_service_checks_require_all_go_units_active(self): - output = """ -lingniu-go-gateway.service active enabled -lingniu-go-history-writer.service active enabled -lingniu-go-stat-writer.service active enabled -lingniu-go-realtime-api.service active enabled -""" - - checks = smoke.service_checks(output, smoke.DEFAULT_SERVICES) - - self.assertTrue(all(check.status == "pass" for check in checks)) - - def test_service_checks_fail_when_unit_is_missing_or_inactive(self): - output = """ -lingniu-go-gateway.service active enabled -lingniu-go-history-writer.service failed enabled -""" - - checks = smoke.service_checks(output, smoke.DEFAULT_SERVICES) - by_name = {check.name: check for check in checks} - - self.assertEqual(by_name["service.lingniu-go-history-writer.service"].status, "fail") - self.assertEqual(by_name["service.lingniu-go-stat-writer.service"].status, "fail") - - def test_service_checks_fail_when_unit_is_not_enabled(self): - output = """ -lingniu-go-gateway.service active disabled -lingniu-go-history-writer.service active enabled -lingniu-go-stat-writer.service active enabled -lingniu-go-realtime-api.service active enabled -""" - - checks = smoke.service_checks(output, smoke.DEFAULT_SERVICES) - by_name = {check.name: check for check in checks} - - self.assertEqual(by_name["service.lingniu-go-gateway.service"].status, "fail") - self.assertIn("enabled=disabled", by_name["service.lingniu-go-gateway.service"].message) - - def test_port_checks_require_expected_go_processes(self): - output = """ -LISTEN 0 4096 *:32960 *:* users:(("gateway",pid=10,fd=3)) -LISTEN 0 4096 *:808 *:* users:(("gateway",pid=10,fd=4)) -LISTEN 0 4096 *:20210 *:* users:(("realtime-api",pid=11,fd=3)) -""" - - checks = smoke.port_checks(output, smoke.DEFAULT_PORTS) - - self.assertTrue(all(check.status == "pass" for check in checks)) - - def test_port_checks_fail_when_java_owns_ingest_port(self): - output = """ -LISTEN 0 4096 *:808 *:* users:(("java",pid=20,fd=4)) -LISTEN 0 4096 *:32960 *:* users:(("gateway",pid=10,fd=3)) -LISTEN 0 4096 *:20210 *:* users:(("realtime-api",pid=11,fd=3)) -""" - - checks = smoke.port_checks(output, smoke.DEFAULT_PORTS) - by_name = {check.name: check for check in checks} - - self.assertEqual(by_name["port.808"].status, "fail") - self.assertIn("java", by_name["port.808"].message) - - def test_spool_check_passes_when_no_pending_files(self): - check = smoke.spool_check("files=0 bytes=208896 recent=0") - - self.assertEqual(check.status, "pass") - self.assertIn("files=0", check.message) - - def test_spool_check_fails_when_pending_or_recent_files_exist(self): - for output in ["files=2 bytes=300 recent=0", "files=0 bytes=208896 recent=1"]: - check = smoke.spool_check(output) - - self.assertEqual(check.status, "fail") - - def test_release_check_passes_when_current_release_has_all_binaries(self): - check = smoke.release_check( - "current=/opt/lingniu-go-native/releases/8def635 " - "gateway=1 history-writer=1 stat-writer=1 realtime-api=1" - ) - - self.assertEqual(check.status, "pass") - self.assertIn("8def635", check.message) - - def test_release_check_fails_when_binary_is_missing(self): - check = smoke.release_check( - "current=/opt/lingniu-go-native/releases/8def635 " - "gateway=1 history-writer=0 stat-writer=1 realtime-api=1" - ) - - self.assertEqual(check.status, "fail") - self.assertIn("history-writer=0", check.message) - - -if __name__ == "__main__": - unittest.main() diff --git a/tools/test_jt808_e2e_smoke.py b/tools/test_jt808_e2e_smoke.py deleted file mode 100644 index 9ed59263..00000000 --- a/tools/test_jt808_e2e_smoke.py +++ /dev/null @@ -1,134 +0,0 @@ -import pathlib -import types -import unittest -from unittest import mock - -from tools import jt808_e2e_smoke as smoke - - -class Jt808E2ESmokeTest(unittest.TestCase): - def test_build_frame_adds_boundaries_and_valid_checksum(self): - body = smoke.build_location_body(smoke.parse_event_time("2024-01-02T03:04:05")) - - frame = smoke.build_frame(0x0200, "13079961234", 1, body) - - self.assertEqual(frame[0], 0x7E) - self.assertEqual(frame[-1], 0x7E) - unescaped = smoke.unescape_payload(frame[1:-1]) - self.assertEqual(smoke.bcc(unescaped), 0) - - def test_next_page_params_include_cursor(self): - params = smoke.next_page_params( - {"phone": "13079961234", "limit": 1}, - {"nextCursor": {"ts": "2024-01-01T19:04:05Z", "id": "frame-1"}}, - ) - - self.assertEqual(params["cursorTs"], "2024-01-01T19:04:05Z") - self.assertEqual(params["cursorId"], "frame-1") - self.assertEqual(params["phone"], "13079961234") - - def test_archive_path_maps_archive_uri_to_root(self): - path = smoke.archive_path( - pathlib.Path("/tmp/archive"), - "archive://2026/06/29/JT808/unknown/frame.bin", - ) - - self.assertEqual(path, pathlib.Path("/tmp/archive/2026/06/29/JT808/unknown/frame.bin")) - - def test_raw_frame_count_sql_filters_by_vehicle_key_phone_and_raw_uri(self): - sql = smoke.raw_frame_count_sql("13079961234", [ - { - "vehicleKey": "jt808:13079961234", - "rawUri": "archive://2026/06/29/JT808/13079961234/frame.bin", - }, - ]) - - self.assertEqual( - sql, - "SELECT COUNT(*) FROM raw_frames WHERE protocol = 'JT808' " - "AND vehicle_key = 'jt808:13079961234' " - "AND phone = '13079961234' " - "AND raw_uri = 'archive://2026/06/29/JT808/13079961234/frame.bin'", - ) - - def test_raw_frame_count_sqls_include_each_unique_visible_raw_uri(self): - sqls = smoke.raw_frame_count_sqls("13079961234", [ - { - "vehicleKey": "jt808:13079961234", - "rawUri": "archive://2026/06/29/JT808/13079961234/frame-1.bin", - }, - { - "vehicleKey": "jt808:13079961234", - "rawUri": "archive://2026/06/29/JT808/13079961234/frame-2.bin", - }, - { - "vehicleKey": "jt808:13079961234", - "rawUri": "archive://2026/06/29/JT808/13079961234/frame-1.bin", - }, - ]) - - self.assertEqual(len(sqls), 2) - self.assertTrue(sqls[0].endswith("frame-1.bin'")) - self.assertTrue(sqls[1].endswith("frame-2.bin'")) - - def test_verify_tdengine_raw_frames_allows_probe_without_visible_raw_uri(self): - verified = smoke.verify_tdengine_raw_frames( - "http://localhost:6041/rest/sql/vehicle_ts", - "root", - "taosdata", - [], - 0, - 1, - ) - - self.assertEqual(verified, 0) - - def test_expected_history_count_waits_for_pagination_sample(self): - args = types.SimpleNamespace(frames=3, expect_history_count=1, verify_pagination=True) - - self.assertEqual(smoke.expected_history_count(args), 2) - - def test_send_phone_workload_can_run_session_phones_with_workers(self): - event_time = smoke.parse_event_time("2024-01-02T03:04:05") - args = types.SimpleNamespace( - connection_mode="session", - tcp_host="127.0.0.1", - tcp_port=808, - frames=3, - tcp_timeout=1, - rate=0, - linger_ms=0, - post_register_ms=0, - require_register_ack=True, - workers=2, - ) - calls = [] - - def fake_send_phone_sequence(host, port, phone, phone_index, frames, sent_event_time, - timeout, rate, linger_ms, post_register_ms): - calls.append((host, port, phone, phone_index, frames, sent_event_time, - timeout, rate, linger_ms, post_register_ms)) - return b"ack", frames - - with mock.patch.object(smoke, "send_phone_sequence", side_effect=fake_send_phone_sequence): - result = smoke.send_phone_workload( - args, - ["13079962000", "13079962001", "13079962002"], - event_time, - ) - - self.assertEqual(result.sent_registers, 3) - self.assertEqual(result.sent_locations, 9) - self.assertGreater(result.elapsed_seconds, 0) - self.assertEqual( - sorted((call[2], call[3], call[4]) for call in calls), - [ - ("13079962000", 0, 3), - ("13079962001", 1, 3), - ("13079962002", 2, 3), - ], - ) - - -if __name__ == "__main__": - unittest.main() diff --git a/tools/test_tdengine_smoke.py b/tools/test_tdengine_smoke.py deleted file mode 100644 index 4b89ff9c..00000000 --- a/tools/test_tdengine_smoke.py +++ /dev/null @@ -1,35 +0,0 @@ -import unittest - -from tools import tdengine_smoke - - -class TdengineSmokeTest(unittest.TestCase): - def test_sql_literal_escapes_single_quotes(self): - self.assertEqual(tdengine_smoke.sql_literal("VIN'001"), "'VIN''001'") - - def test_raw_frame_count_sql_filters_by_protocol_vehicle_and_raw_uri(self): - sql = tdengine_smoke.raw_frame_count_sql( - protocol="JT808", - vehicle_key="jt808:13079962000", - raw_uri="archive://2026/06/29/JT808/frame.bin", - ) - - self.assertEqual( - sql, - "SELECT COUNT(*) FROM raw_frames WHERE protocol = 'JT808' " - "AND vehicle_key = 'jt808:13079962000' " - "AND raw_uri = 'archive://2026/06/29/JT808/frame.bin'", - ) - - def test_raw_frame_count_sql_requires_at_least_one_identity_or_raw_uri_filter(self): - with self.assertRaises(ValueError): - tdengine_smoke.raw_frame_count_sql(protocol="GB32960") - - def test_count_from_response_reads_first_cell(self): - response = {"code": 0, "data": [[3]], "rows": 1} - - self.assertEqual(tdengine_smoke.count_from_response(response), 3) - - -if __name__ == "__main__": - unittest.main() diff --git a/tools/test_vehicle_ingest_live_verify.py b/tools/test_vehicle_ingest_live_verify.py deleted file mode 100644 index 7b9b5593..00000000 --- a/tools/test_vehicle_ingest_live_verify.py +++ /dev/null @@ -1,142 +0,0 @@ -import unittest - -from tools import vehicle_ingest_live_verify as live_verify - - -class VehicleIngestLiveVerifyTest(unittest.TestCase): - def test_warns_when_formal_gb32960_platform_login_exists_without_vehicle_frames(self): - checks = live_verify.evaluate_counts( - { - "jt808RawFrames": 36090, - "jt808Locations": 8783, - "gb32960PlatformLogins": 3, - "gb32960VehicleRealtimeFrames": 0, - }, - live_verify.Thresholds( - min_jt808_raw_frames=1, - min_jt808_locations=1, - min_gb32960_platform_logins=1, - min_gb32960_vehicle_realtime_frames=1, - require_gb32960_vehicle_realtime=False, - ), - ) - - by_name = {check.name: check for check in checks} - - self.assertEqual(live_verify.overall_status(checks), "warn") - self.assertEqual(by_name["gb32960.vehicle_realtime"].status, "warn") - self.assertIn("尚未收到正式车辆 0x02", by_name["gb32960.vehicle_realtime"].message) - - def test_fails_when_vehicle_frames_are_required_but_missing(self): - checks = live_verify.evaluate_counts( - { - "jt808RawFrames": 10, - "jt808Locations": 2, - "gb32960PlatformLogins": 1, - "gb32960VehicleRealtimeFrames": 0, - }, - live_verify.Thresholds(require_gb32960_vehicle_realtime=True), - ) - - by_name = {check.name: check for check in checks} - - self.assertEqual(live_verify.overall_status(checks), "fail") - self.assertEqual(by_name["gb32960.vehicle_realtime"].status, "fail") - - def test_gb32960_vehicle_realtime_sql_excludes_test_vins(self): - sql = live_verify.gb32960_vehicle_realtime_count_sql( - date_from="2026-06-29 00:00:00", - date_to="2026-06-29 23:59:59", - ) - - self.assertIn("message_id = 2", sql) - self.assertIn("vin <> ''", sql) - self.assertIn("vin NOT LIKE 'LTEST%'", sql) - self.assertIn("ts >= '2026-06-29 00:00:00'", sql) - self.assertIn("ts <= '2026-06-29 23:59:59'", sql) - - def test_gb32960_peer_filter_treats_plain_ip_as_contains_pattern(self): - sql = live_verify.gb32960_platform_login_count_sql( - date_from="2026-06-29 00:00:00", - date_to="2026-06-29 23:59:59", - peer_like="115.29.187.205", - ) - - self.assertIn("peer LIKE '%115.29.187.205%'", sql) - - def test_gb32960_peer_filter_preserves_explicit_like_pattern(self): - sql = live_verify.gb32960_platform_login_count_sql( - date_from="2026-06-29 00:00:00", - date_to="2026-06-29 23:59:59", - peer_like="115.29.187.%", - ) - - self.assertIn("peer LIKE '115.29.187.%'", sql) - - def test_gb32960_command_counts_sql_uses_same_peer_filter(self): - sql = live_verify.gb32960_command_count_sql( - date_from="2026-06-29 00:00:00", - date_to="2026-06-29 23:59:59", - peer_like="115.29.187.205", - ) - - self.assertIn("protocol = 'GB32960'", sql) - self.assertIn("peer LIKE '%115.29.187.205%'", sql) - self.assertIn("GROUP BY message_id", sql) - - def test_gb32960_command_count_rows_include_command_names(self): - response = { - "code": 0, - "data": [ - [2, 9], - [5, 3], - ], - } - - rows = live_verify.command_count_rows(response) - - self.assertEqual(rows, [ - {"messageId": 2, "command": "REALTIME_REPORT", "count": 9}, - {"messageId": 5, "command": "PLATFORM_LOGIN", "count": 3}, - ]) - - def test_gb32960_recent_frame_rows_surface_peer_and_raw_uri(self): - response = { - "code": 0, - "data": [[ - "2026-06-29T12:26:21.385Z", - 5, - "", - "unknown:GB32960:f729d788b943630e", - "115.29.187.205:59630", - "archive://frame.bin", - ]], - } - - rows = live_verify.recent_frame_rows(response) - - self.assertEqual(rows, [{ - "ts": "2026-06-29T12:26:21.385Z", - "messageId": 5, - "command": "PLATFORM_LOGIN", - "vin": "", - "vehicleKey": "unknown:GB32960:f729d788b943630e", - "peer": "115.29.187.205:59630", - "rawUri": "archive://frame.bin", - }]) - - def test_jt808_location_sql_uses_current_vehicle_locations_stable_with_protocol_filter(self): - sql = live_verify.jt808_location_count_sql( - date_from="2026-06-29 00:00:00", - date_to="2026-06-29 23:59:59", - ) - - self.assertEqual( - sql, - "SELECT COUNT(*) FROM vehicle_locations WHERE protocol = 'JT808' " - "AND ts >= '2026-06-29 00:00:00' AND ts <= '2026-06-29 23:59:59'", - ) - - -if __name__ == "__main__": - unittest.main() diff --git a/tools/vehicle_ingest_live_verify.py b/tools/vehicle_ingest_live_verify.py deleted file mode 100755 index dae81b33..00000000 --- a/tools/vehicle_ingest_live_verify.py +++ /dev/null @@ -1,392 +0,0 @@ -#!/usr/bin/env python3 -"""Verify the live 32960/JT808 ingest-to-history chain with production traffic.""" - -from __future__ import annotations - -import argparse -import base64 -import json -import os -import urllib.parse -import urllib.request -from dataclasses import asdict, dataclass -from typing import Any - -try: - from tools import tdengine_smoke -except ModuleNotFoundError: - import tdengine_smoke - - -@dataclass(frozen=True) -class Thresholds: - min_jt808_raw_frames: int = 1 - min_jt808_locations: int = 1 - min_gb32960_platform_logins: int = 1 - min_gb32960_vehicle_realtime_frames: int = 1 - require_gb32960_vehicle_realtime: bool = False - - -@dataclass(frozen=True) -class Check: - name: str - status: str - count: int - minimum: int - message: str - - -def count_sql(table: str, filters: list[str], date_from: str, date_to: str) -> str: - where = list(filters) - if date_from: - where.append("ts >= " + tdengine_smoke.sql_literal(date_from)) - if date_to: - where.append("ts <= " + tdengine_smoke.sql_literal(date_to)) - return "SELECT COUNT(*) FROM " + table + " WHERE " + " AND ".join(where) - - -def jt808_raw_count_sql(date_from: str, date_to: str, peer_like: str = "") -> str: - filters = ["protocol = 'JT808'"] - if peer_like: - filters.append("peer LIKE " + tdengine_smoke.sql_literal(peer_like)) - return count_sql("raw_frames", filters, date_from, date_to) - - -def jt808_location_count_sql(date_from: str, date_to: str) -> str: - return count_sql("vehicle_locations", ["protocol = 'JT808'"], date_from, date_to) - - -def gb32960_platform_login_count_sql(date_from: str, date_to: str, peer_like: str = "") -> str: - filters = ["protocol = 'GB32960'", "message_id = 5"] - if peer_like: - filters.append("peer LIKE " + tdengine_smoke.sql_literal(peer_like_pattern(peer_like))) - return count_sql("raw_frames", filters, date_from, date_to) - - -def gb32960_vehicle_realtime_count_sql(date_from: str, date_to: str, peer_like: str = "") -> str: - filters = [ - "protocol = 'GB32960'", - "message_id = 2", - "vin <> ''", - "vin NOT LIKE 'LTEST%'", - ] - if peer_like: - filters.append("peer LIKE " + tdengine_smoke.sql_literal(peer_like_pattern(peer_like))) - return count_sql("raw_frames", filters, date_from, date_to) - - -def latest_gb32960_platform_raw_uri_sql(date_from: str, date_to: str, peer_like: str = "") -> str: - filters = ["protocol = 'GB32960'", "message_id = 5", "raw_uri <> ''"] - if peer_like: - filters.append("peer LIKE " + tdengine_smoke.sql_literal(peer_like_pattern(peer_like))) - where = list(filters) - if date_from: - where.append("ts >= " + tdengine_smoke.sql_literal(date_from)) - if date_to: - where.append("ts <= " + tdengine_smoke.sql_literal(date_to)) - return "SELECT raw_uri FROM raw_frames WHERE " + " AND ".join(where) + " ORDER BY ts DESC LIMIT 1" - - -def gb32960_command_count_sql(date_from: str, date_to: str, peer_like: str = "") -> str: - filters = ["protocol = 'GB32960'"] - if peer_like: - filters.append("peer LIKE " + tdengine_smoke.sql_literal(peer_like_pattern(peer_like))) - where = list(filters) - if date_from: - where.append("ts >= " + tdengine_smoke.sql_literal(date_from)) - if date_to: - where.append("ts <= " + tdengine_smoke.sql_literal(date_to)) - return ( - "SELECT message_id,COUNT(*) FROM raw_frames WHERE " - + " AND ".join(where) - + " GROUP BY message_id ORDER BY message_id" - ) - - -def gb32960_recent_frame_sql(date_from: str, date_to: str, peer_like: str = "", limit: int = 10) -> str: - filters = ["protocol = 'GB32960'"] - if peer_like: - filters.append("peer LIKE " + tdengine_smoke.sql_literal(peer_like_pattern(peer_like))) - where = list(filters) - if date_from: - where.append("ts >= " + tdengine_smoke.sql_literal(date_from)) - if date_to: - where.append("ts <= " + tdengine_smoke.sql_literal(date_to)) - safe_limit = max(1, min(limit, 100)) - return ( - "SELECT ts,message_id,vin,vehicle_key,peer,raw_uri FROM raw_frames WHERE " - + " AND ".join(where) - + " ORDER BY ts DESC LIMIT " - + str(safe_limit) - ) - - -def peer_like_pattern(value: str) -> str: - value = value.strip() - if "%" in value or "_" in value: - return value - return "%" + value + "%" - - -def command_name(message_id: int) -> str: - return { - 1: "VEHICLE_LOGIN", - 2: "REALTIME_REPORT", - 3: "RESEND_REPORT", - 4: "VEHICLE_LOGOUT", - 5: "PLATFORM_LOGIN", - 6: "PLATFORM_LOGOUT", - 7: "HEARTBEAT", - 8: "TIME_CALIBRATION", - }.get(message_id, "UNKNOWN_0x" + format(message_id, "02X")) - - -def response_rows(response: dict[str, Any]) -> list[list[Any]]: - if response.get("code") != 0: - raise RuntimeError(f"tdengine query failed: {response}") - return response.get("data") or [] - - -def command_count_rows(response: dict[str, Any]) -> list[dict[str, Any]]: - rows = [] - for item in response_rows(response): - message_id = int(item[0]) - rows.append({ - "messageId": message_id, - "command": command_name(message_id), - "count": int(item[1]), - }) - return rows - - -def recent_frame_rows(response: dict[str, Any]) -> list[dict[str, Any]]: - rows = [] - for item in response_rows(response): - message_id = int(item[1]) - rows.append({ - "ts": item[0], - "messageId": message_id, - "command": command_name(message_id), - "vin": item[2], - "vehicleKey": item[3], - "peer": item[4], - "rawUri": item[5], - }) - return rows - - -def evaluate_counts(counts: dict[str, int], thresholds: Thresholds) -> list[Check]: - checks = [ - threshold_check( - "jt808.raw_frames", - counts.get("jt808RawFrames", 0), - thresholds.min_jt808_raw_frames, - "JT808 raw_frames 已有正式报文", - "JT808 raw_frames 未达到最低验收数量", - ), - threshold_check( - "jt808.locations", - counts.get("jt808Locations", 0), - thresholds.min_jt808_locations, - "JT808 位置历史已入 TDengine", - "JT808 位置历史未达到最低验收数量", - ), - threshold_check( - "gb32960.platform_login", - counts.get("gb32960PlatformLogins", 0), - thresholds.min_gb32960_platform_logins, - "GB32960 正式平台登录已入 TDengine", - "GB32960 正式平台登录未达到最低验收数量", - ), - ] - vehicle_count = counts.get("gb32960VehicleRealtimeFrames", 0) - vehicle_minimum = thresholds.min_gb32960_vehicle_realtime_frames - if vehicle_count >= vehicle_minimum: - checks.append(Check( - "gb32960.vehicle_realtime", - "pass", - vehicle_count, - vehicle_minimum, - "GB32960 正式车辆 0x02 实时上报已入 TDengine", - )) - else: - status = "fail" if thresholds.require_gb32960_vehicle_realtime else "warn" - checks.append(Check( - "gb32960.vehicle_realtime", - status, - vehicle_count, - vehicle_minimum, - "尚未收到正式车辆 0x02；平台登录在线不等于车辆数据在线", - )) - return checks - - -def threshold_check(name: str, count: int, minimum: int, ok_message: str, bad_message: str) -> Check: - if count >= minimum: - return Check(name, "pass", count, minimum, ok_message) - return Check(name, "fail", count, minimum, bad_message) - - -def overall_status(checks: list[Check]) -> str: - statuses = {check.status for check in checks} - if "fail" in statuses: - return "fail" - if "warn" in statuses: - return "warn" - return "pass" - - -def query_json(rest_url: str, username: str, password: str, sql: str, timeout: float) -> dict[str, Any]: - request = urllib.request.Request(rest_url, data=sql.encode("utf-8"), method="POST") - if username: - token = base64.b64encode(f"{username}:{password}".encode("utf-8")).decode("ascii") - request.add_header("Authorization", "Basic " + token) - request.add_header("Content-Type", "text/plain; charset=utf-8") - request.add_header("Accept", "application/json") - with urllib.request.urlopen(request, timeout=timeout) as response: - return json.loads(response.read().decode("utf-8")) - - -def first_cell(response: dict[str, Any]) -> Any: - if response.get("code") != 0: - raise RuntimeError(f"tdengine query failed: {response}") - data = response.get("data") or [] - if not data or not data[0]: - return None - return data[0][0] - - -def query_counts(args: argparse.Namespace) -> dict[str, int]: - queries = { - "jt808RawFrames": jt808_raw_count_sql(args.date_from, args.date_to, args.jt808_peer_like), - "jt808Locations": jt808_location_count_sql(args.date_from, args.date_to), - "gb32960PlatformLogins": gb32960_platform_login_count_sql( - args.date_from, args.date_to, args.gb32960_peer_like), - "gb32960VehicleRealtimeFrames": gb32960_vehicle_realtime_count_sql( - args.date_from, args.date_to, args.gb32960_peer_like), - } - out: dict[str, int] = {} - for key, sql in queries.items(): - out[key] = tdengine_smoke.query_count( - args.tdengine_rest_url, - args.tdengine_username, - args.tdengine_password, - sql, - args.http_timeout, - ) - return out - - -def query_latest_platform_raw_uri(args: argparse.Namespace) -> str: - sql = latest_gb32960_platform_raw_uri_sql(args.date_from, args.date_to, args.gb32960_peer_like) - response = query_json( - args.tdengine_rest_url, - args.tdengine_username, - args.tdengine_password, - sql, - args.http_timeout, - ) - value = first_cell(response) - return value if isinstance(value, str) else "" - - -def query_gb32960_command_counts(args: argparse.Namespace) -> list[dict[str, Any]]: - response = query_json( - args.tdengine_rest_url, - args.tdengine_username, - args.tdengine_password, - gb32960_command_count_sql(args.date_from, args.date_to, args.gb32960_peer_like), - args.http_timeout, - ) - return command_count_rows(response) - - -def query_gb32960_recent_frames(args: argparse.Namespace) -> list[dict[str, Any]]: - response = query_json( - args.tdengine_rest_url, - args.tdengine_username, - args.tdengine_password, - gb32960_recent_frame_sql( - args.date_from, - args.date_to, - args.gb32960_peer_like, - args.gb32960_recent_limit), - args.http_timeout, - ) - return recent_frame_rows(response) - - -def decode_frame(history_base_url: str, raw_uri: str, timeout: float) -> dict[str, Any]: - if not raw_uri: - return {} - params = urllib.parse.urlencode({"rawArchiveUri": raw_uri}) - request = urllib.request.Request( - history_base_url.rstrip("/") + "/api/event-history/gb32960/frame?" + params, - headers={"Accept": "application/json"}, - ) - with urllib.request.urlopen(request, timeout=timeout) as response: - return json.loads(response.read().decode("utf-8")) - - -def run(args: argparse.Namespace) -> dict[str, Any]: - counts = query_counts(args) - thresholds = Thresholds( - min_jt808_raw_frames=args.min_jt808_raw_frames, - min_jt808_locations=args.min_jt808_locations, - min_gb32960_platform_logins=args.min_gb32960_platform_logins, - min_gb32960_vehicle_realtime_frames=args.min_gb32960_vehicle_realtime_frames, - require_gb32960_vehicle_realtime=args.require_gb32960_vehicle_realtime, - ) - checks = evaluate_counts(counts, thresholds) - latest_raw_uri = "" - decoded_platform_frame: dict[str, Any] = {} - command_counts = query_gb32960_command_counts(args) - recent_frames = query_gb32960_recent_frames(args) - if args.history_base_url: - latest_raw_uri = query_latest_platform_raw_uri(args) - decoded_platform_frame = decode_frame(args.history_base_url, latest_raw_uri, args.http_timeout) - return { - "status": overall_status(checks), - "dateFrom": args.date_from, - "dateTo": args.date_to, - "counts": counts, - "checks": [asdict(check) for check in checks], - "gb32960CommandCounts": command_counts, - "gb32960RecentFrames": recent_frames, - "gb32960LatestPlatformRawUri": latest_raw_uri, - "gb32960LatestPlatformCommand": decoded_platform_frame.get("command"), - } - - -def parser() -> argparse.ArgumentParser: - p = argparse.ArgumentParser(description=__doc__) - p.add_argument("--tdengine-rest-url", default=os.environ.get("TDENGINE_REST_URL", "")) - p.add_argument("--tdengine-username", default=os.environ.get("TDENGINE_USERNAME", "root")) - p.add_argument("--tdengine-password", default=os.environ.get("TDENGINE_PASSWORD", "taosdata")) - p.add_argument("--history-base-url", default=os.environ.get("HISTORY_BASE_URL", "http://127.0.0.1:20200")) - p.add_argument("--date-from", required=True) - p.add_argument("--date-to", required=True) - p.add_argument("--jt808-peer-like", default=os.environ.get("JT808_PEER_LIKE", "")) - p.add_argument("--gb32960-peer-like", default=os.environ.get("GB32960_PEER_LIKE", "")) - p.add_argument("--min-jt808-raw-frames", type=int, default=1) - p.add_argument("--min-jt808-locations", type=int, default=1) - p.add_argument("--min-gb32960-platform-logins", type=int, default=1) - p.add_argument("--min-gb32960-vehicle-realtime-frames", type=int, default=1) - p.add_argument("--require-gb32960-vehicle-realtime", action="store_true") - p.add_argument("--gb32960-recent-limit", type=int, default=10) - p.add_argument("--http-timeout", type=float, default=5) - return p - - -def main() -> None: - args = parser().parse_args() - if not args.tdengine_rest_url: - raise SystemExit("--tdengine-rest-url is required") - result = run(args) - print(json.dumps(result, ensure_ascii=False, indent=2)) - if result["status"] == "fail": - raise SystemExit(1) - - -if __name__ == "__main__": - main() diff --git a/woodpecker.yml b/woodpecker.yml deleted file mode 100644 index 73eae03b..00000000 --- a/woodpecker.yml +++ /dev/null @@ -1,78 +0,0 @@ -steps: - - name: maven-build - image: maven:3.9.16-eclipse-temurin-25 - when: - event: - - push - - pull_request - - manual - branch: - - master - - develop - - main - environment: - ACTIVE_APPS: &active_apps "gb32960-ingest-app jt808-ingest-app yutong-mqtt-app vehicle-history-app vehicle-analytics-app" - commands: | - cd $CI_WORKSPACE - - ACTIVE_MODULES="" - for APP_NAME in $ACTIVE_APPS; do - if [ -z "$ACTIVE_MODULES" ]; then - ACTIVE_MODULES=":$APP_NAME" - else - ACTIVE_MODULES="$ACTIVE_MODULES,:$APP_NAME" - fi - done - - mvn -B -ntp -am -pl "$ACTIVE_MODULES" package -Dmaven.test.skip=true - - BRANCH_NAME=$(echo ${CI_COMMIT_BRANCH:-local} | tr / -) - PROJECT_VERSION=$(mvn -q -DforceStdout help:evaluate -Dexpression=project.version) - SHORT_SHA=$(echo ${CI_COMMIT_SHA:-manual} | cut -c1-8) - IMAGE_VERSION="$BRANCH_NAME-$PROJECT_VERSION-$SHORT_SHA" - - echo "Project version: $PROJECT_VERSION" - echo "Image version: $IMAGE_VERSION" - echo "$IMAGE_VERSION" > $CI_WORKSPACE/project_version.txt - - - name: docker-build-apps - image: docker:24.0.5-cli - when: - event: - - push - - pull_request - - manual - branch: - - master - - develop - - main - volumes: - - /var/run/docker.sock:/var/run/docker.sock - environment: - ACTIVE_APPS: *active_apps - ALIYUN_ACR_USERNAME: - from_secret: aliyun_acr_username - ALIYUN_ACR_PASSWORD: - from_secret: aliyun_acr_password - commands: | - cd $CI_WORKSPACE - APP_VERSION=$(cat $CI_WORKSPACE/project_version.txt) - REGISTRY=${LINGNIU_IMAGE_REGISTRY:-crpi-85r4m0ackrm3qpje.cn-shanghai.personal.cr.aliyuncs.com} - NAMESPACE=${LINGNIU_IMAGE_NAMESPACE:-oneos} - - printf '%s' "$ALIYUN_ACR_PASSWORD" | docker login \ - --username "$ALIYUN_ACR_USERNAME" \ - --password-stdin \ - "$REGISTRY" - - for APP_NAME in $ACTIVE_APPS; do - IMAGE=$REGISTRY/$NAMESPACE/$APP_NAME:$APP_VERSION - echo "Building and pushing $IMAGE" - - docker build \ - --build-arg APP_NAME=$APP_NAME \ - --build-arg APP_VERSION=$APP_VERSION \ - -t $IMAGE . - - docker push $IMAGE - done