Compare commits
10 Commits
f9b9c2a6a7
...
a096e4ce0e
| Author | SHA1 | Date | |
|---|---|---|---|
|
a096e4ce0e | ||
|
|
ba68ffe061 | ||
|
|
b14871ff1c | ||
|
|
1c1d400784 | ||
|
|
812b8877d4 | ||
|
|
dd838b4f73 | ||
|
|
bfc3c29c40 | ||
|
|
4d8dd54e59 | ||
|
|
da28a0c8d5 | ||
|
|
ce87c8ee26 |
1
.gitignore
vendored
1
.gitignore
vendored
@@ -22,6 +22,7 @@ logs/
|
||||
data/
|
||||
build/
|
||||
out/
|
||||
archive/
|
||||
|
||||
# Local config / secrets
|
||||
application-local.yml
|
||||
|
||||
35
CHANGELOG.md
35
CHANGELOG.md
@@ -10,6 +10,41 @@
|
||||
|
||||
---
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Added —— 原始报文冷存（Raw Archive）接入 EventSink 扇出
|
||||
- 新增 `VehicleEvent.RawArchive` sealed 变体（`ingest-api`），携带一帧入站的原始字节 +
|
||||
`command` / `infoType` 供按协议命令分片归档。
|
||||
- `Dispatcher.dispatch` 在 interceptor **之前**按 `RawFrame.rawBytes()` 产出一条
|
||||
`RawArchive` 事件，保证 dedup / rate-limit **不过滤** raw archive，满足"每帧必留痕"目标。
|
||||
- 新增 `ArchiveEventSink implements EventSink`（`sink-archive`）：
|
||||
- `accepts` 仅 `VehicleEvent.RawArchive`
|
||||
- 写入 key `yyyy/MM/dd/<source>/<vin>/<eventId>.bin`（UTC 日期分片）
|
||||
- 同 eventId 重复写按 put 语义覆盖
|
||||
- 失败 completeExceptionally，EventBus 打 WARN
|
||||
- AutoConfig 在 `ArchiveStore` bean 存在时自动装配
|
||||
- `KafkaEventSink.accepts(RawArchive)` 返回 `false`：本期 Kafka 不投递原始字节，
|
||||
只落 ArchiveStore。未来需要 Kafka 回填 URI 时再开（`TopicRouter` / `EnvelopeMapper`
|
||||
已补好 switch 分支）。
|
||||
- 覆盖测试 `ArchiveEventSinkTest` —— accepts 过滤 / 稳定 key 写入 / 幂等覆盖 3 case。
|
||||
|
||||
### Changed —— GB/T 32960 Body Parser 单块异常隔离
|
||||
- `Gb32960BodyParser` 主循环对单信息块的 parser 异常不再放弃整帧：
|
||||
- 固定长度块（`fixedLength ≥ 0`）：回滚 reader → 按 `fixedLen` 截取字节兜成
|
||||
`InfoBlock.Raw` → `continue` 继续解析后续块；
|
||||
- 变长块（`fixedLength == -1`）或剩余字节不足：剩余字节全部兜成 `Raw` 后终止循环；
|
||||
- `parser` 契约违反（声明 `fixedLen=X` 但实际读了 `Y`）仍抛 `DecodeException`，
|
||||
以暴露 parser bug，不走 lenient 兜底，避免静默误解析长期误判；
|
||||
- 只捕获 `DecodeException` / `BufferUnderflowException` / `IndexOutOfBoundsException`，
|
||||
`RuntimeException` 继续向上抛，保留 bug 可见性。
|
||||
- 新增配置 `lingniu.ingest.gb32960.parse.lenient-block-failure`（默认 `true`）；
|
||||
需要回退严格模式时设为 `false`。
|
||||
- `InfoBlock.Raw` record 结构**未变**，下游 `Gb32960EventMapper` 按 `findBlock(Class)`
|
||||
类型匹配，新增 Raw 不影响业务事件映射。
|
||||
- 新增测试 `Gb32960BodyParserIsolationTest` —— 3 种隔离场景 + 严格模式回退，共 4 case。
|
||||
|
||||
---
|
||||
|
||||
## [0.1.0] — 2026-04-15
|
||||
|
||||
初始基线。多模块 Spring Boot 车辆遥测接入服务。
|
||||
|
||||
57
README.md
57
README.md
@@ -21,7 +21,7 @@
|
||||
| 消息队列 | **Kafka**（vin 分区，保证单车有序） |
|
||||
| 线上消息格式 | **Protobuf**，调试走 JSON |
|
||||
| MQTT 客户端 | HiveMQ MQTT Client |
|
||||
| 会话缓存 | Caffeine + Redis 兜底 |
|
||||
| 会话缓存 | Redis 生产索引 + Caffeine 内存降级 |
|
||||
| 熔断/限流 | Resilience4j 2 |
|
||||
| 可观测 | Micrometer + Prometheus + OpenTelemetry |
|
||||
| 冷存 | S3 / OSS / 本地 |
|
||||
@@ -31,22 +31,35 @@
|
||||
|
||||
```
|
||||
lingniu-vehicle-ingest/
|
||||
├── bom/ 依赖版本统一
|
||||
├── ingest-api/ SPI + sealed 领域事件 + 注解
|
||||
├── ingest-codec-common/ 公共编解码工具（BCD/CRC/BCC/bit utils）
|
||||
├── ingest-core/ Pipeline / Dispatcher / Disruptor / Session 桥
|
||||
├── session-core/ 设备会话 + 鉴权 + Token
|
||||
├── observability/ metrics / tracing / health
|
||||
├── sink-mq/ Kafka producer + Protobuf Envelope
|
||||
├── sink-archive/ 原始报文冷存
|
||||
├── protocol-gb32960/ GB/T 32960
|
||||
├── protocol-jt808/ JT/T 808
|
||||
├── protocol-jt1078/ JT/T 1078
|
||||
├── protocol-jsatl12/ 苏标主动安全报警附件
|
||||
├── inbound-mqtt/ MQTT 接入（宇通等）
|
||||
├── inbound-xinda-push/ 信达 Push 接入（重写）
|
||||
├── command-gateway/ HTTP → 设备下行命令
|
||||
└── bootstrap-all/ 一体化启动
|
||||
├── modules/
|
||||
│ ├── core/
|
||||
│ │ ├── ingest-api/ SPI + sealed 领域事件 + 注解
|
||||
│ │ ├── ingest-codec-common/ 公共编解码工具（BCD/CRC/BCC/bit utils）
|
||||
│ │ ├── ingest-core/ Pipeline / Dispatcher / Disruptor / Session 桥
|
||||
│ │ ├── session-core/ 设备会话 + 鉴权 + Token + Redis/Memory SessionStore
|
||||
│ │ ├── vehicle-identity/ 跨协议车辆身份解析 + 外部标识绑定（memory/file）
|
||||
│ │ └── observability/ metrics / tracing / health
|
||||
│ ├── protocols/
|
||||
│ │ ├── protocol-gb32960/ GB/T 32960
|
||||
│ │ ├── protocol-jt808/ JT/T 808（统一身份映射 + 事件 metadata 内部 VIN + 注册/鉴权/心跳/注销/位置/批量位置/参数/属性/媒体/透传/未知上行和坏帧兜底/断链清理会话/下行分包）
|
||||
│ │ ├── protocol-jt1078/ JT/T 1078（808 信令按需桥接 + 常用下行信令编码 + TCP/UDP RTP 媒体流分段归档 + 事件 metadata 内部 VIN + 坏 RTP 统一 RawArchive/Passthrough + 归档失败兜底 + SIM 身份映射）
|
||||
│ │ └── protocol-jsatl12/ 苏标主动安全报警附件（真实 DataPacket 分帧 + 附件流归档 + T1210 文件清单身份继承 + 9212 补传应答 + MediaMeta 引用事件 + 坏帧/坏信令/归档失败兜底）
|
||||
│ ├── inbound/
|
||||
│ │ ├── inbound-mqtt/ MQTT 接入（endpoint 生命周期 + profile 注册扩展 + 统一身份映射 + PEM 双向 TLS + 未知 profile/解析失败/profile异常/连接订阅失败兜底 + 统一 UNKNOWN 身份 metadata）
|
||||
│ │ └── inbound-xinda-push/ 信达 Push 接入（统一身份映射 + 事件 metadata 内部 VIN + 未知业务 cmd/解析失败/启动配置失败/业务处理异常兜底）
|
||||
│ ├── sinks/
|
||||
│ │ ├── sink-mq/ Kafka producer + Protobuf Envelope
|
||||
│ │ ├── sink-archive/ 原始报文冷存
|
||||
│ │ └── event-file-store/ Parquet + DuckDB 文件型明细库
|
||||
│ ├── services/
|
||||
│ │ ├── event-history-service/ Kafka 全字段事件消费 + 历史查询/导出
|
||||
│ │ ├── vehicle-state-service/ Kafka 全字段事件消费 + Redis 热状态查询
|
||||
│ │ └── vehicle-stat-service/ Kafka 全字段事件消费 + 可配置日统计
|
||||
│ └── apps/
|
||||
│ ├── command-gateway/ HTTP → 设备下行命令（808 位置/参数/属性/控制/区域删除/报警确认 + 1078 音视频控制）
|
||||
│ └── bootstrap-all/ 一体化启动
|
||||
├── docs/ 架构文档、模块图、实施计划
|
||||
└── reference/ 参考资料
|
||||
```
|
||||
|
||||
## 快速开始
|
||||
@@ -55,17 +68,23 @@ lingniu-vehicle-ingest/
|
||||
# 要求：JDK 25, Maven 3.9+
|
||||
mvn -v
|
||||
mvn clean install -DskipTests
|
||||
cd bootstrap-all && mvn spring-boot:run
|
||||
mvn -pl :bootstrap-all spring-boot:run
|
||||
```
|
||||
|
||||
## 迁移说明
|
||||
|
||||
本项目是 `lingniu-vehicle-data-reception` 的 v2 重构，采用 strangler fig 渐进式迁移，旧项目保留只读参考。迁移路径与决策参见 `../REFRACTOR_PLAN.md`。
|
||||
|
||||
## 架构文档
|
||||
|
||||
- 目标架构：`docs/target-architecture.md`
|
||||
- 内部字段模型：`docs/vehicle-telemetry-internal-fields.md`
|
||||
- 模块与数据流：`docs/module-data-flow.html`
|
||||
|
||||
## 核心原则
|
||||
|
||||
1. **本服务不写业务库**：历史、行程、在线检测、里程统计全部由 Kafka 下游消费者实现
|
||||
2. **协议即插拔**：每个 `protocol-*` / `inbound-*` 都可独立开关（`lingniu.ingest.<name>.enabled`）
|
||||
3. **顺序保证**：同一 VIN 严格有序（Disruptor hash + Kafka 分区 key）
|
||||
4. **幂等消费**：Envelope 带 `eventId`，下游去重
|
||||
5. **原始可回放**：`vehicle.raw.archive` topic + 对象存储
|
||||
5. **原始可回放**：Dispatcher 会先发布 `RawArchive`，并把同一份 `rawArchiveKey/rawArchiveUri` 追加到业务事件 metadata；下游明细库、Kafka Envelope 和导出都使用 `archive://...` 逻辑 URI 追溯原始 bytes，实际文件由 `sink-archive`/`ArchiveStore` 管理；`event-file-store` 只额外保存 RawArchive 查询索引，不嵌入原始 bytes
|
||||
|
||||
@@ -1,153 +0,0 @@
|
||||
spring:
|
||||
application:
|
||||
name: lingniu-vehicle-ingest
|
||||
threads:
|
||||
virtual:
|
||||
enabled: true
|
||||
|
||||
server:
|
||||
port: 20000
|
||||
|
||||
lingniu:
|
||||
ingest:
|
||||
gb32960:
|
||||
enabled: true
|
||||
port: 9000
|
||||
boss-threads: 1
|
||||
worker-threads: 0 # 0 = Runtime.availableProcessors() * 2
|
||||
# VIN 白名单认证（内存配置）。GB/T 32960 设备没有账号密码，身份由 VIN 决定。
|
||||
auth:
|
||||
enabled: ${GB32960_AUTH_ENABLED:false} # true 启用白名单；false 放行所有 VIN
|
||||
case-sensitive: false # 默认大小写不敏感
|
||||
whitelist: # VIN 列表，未列入的 0x01 登入返回 0x04 VIN_NOT_EXIST
|
||||
# - LTEST000000000001
|
||||
# - LGWEF4A58RE123456
|
||||
# 平台登入 0x05 账号列表（GB/T 32960.3 表 29）。
|
||||
# 列表为空时放行所有平台登入（兼容旧行为），否则 username/password 必须匹配，
|
||||
# 失败返回 0x02 OTHER_ERROR 应答并关闭连接。
|
||||
platforms:
|
||||
- username: ${GB32960_PLATFORM_USER_1:lingniu}
|
||||
password: ${GB32960_PLATFORM_PWD_1:Lingniu.2024#}
|
||||
allowed-ips: # 可选：限制来源 IP，为空不限制
|
||||
description: 外部下级平台 (原旧服务客户端)
|
||||
# TLS 双向认证（可选）。启用后 Netty pipeline 前置 SslHandler 并强制客户端证书。
|
||||
tls:
|
||||
enabled: ${GB32960_TLS_ENABLED:false}
|
||||
cert-path: ${GB32960_TLS_CERT:} # 服务端证书 PEM 路径
|
||||
key-path: ${GB32960_TLS_KEY:} # 服务端私钥 PEM 路径（PKCS#8 未加密）
|
||||
trust-cert-path: ${GB32960_TLS_CA:} # 受信 CA 证书 PEM，用于校验客户端证书
|
||||
require-client-auth: true # 强制双向；false=单向 TLS
|
||||
# ====================================================================
|
||||
# 厂商扩展协议路由（vendor extensions）
|
||||
# 命中规则按 list 顺序首匹（first-match-wins），命中后用对应套件的 parser 集合
|
||||
# 在标准 0x01..0x09 之上叠加。已支持的套件：
|
||||
# - guangdong-fc：广东燃料电池汽车示范规范 0x30~0x34、0x80
|
||||
# 来源：reference/广东燃料电池汽车示范应用城市群综合监管平台-...pdf
|
||||
# ====================================================================
|
||||
vendor-extensions:
|
||||
- name: guangdong-fc
|
||||
match:
|
||||
# 平台账号匹配（精确匹配，case-insensitive）。这里和上面 auth.platforms
|
||||
# 中的 username 保持一致，确保 lingniu 账号登入后所有 0x02 帧都走广东 profile。
|
||||
platform-accounts:
|
||||
- lingniu
|
||||
# VIN 前缀匹配（startsWith）；空 list = 不参与匹配
|
||||
vin-prefixes: []
|
||||
# VIN 精确匹配
|
||||
vins: []
|
||||
# 报文解析容错。默认启用单块异常隔离：某个信息块解析失败时兜成
|
||||
# InfoBlock.Raw 后继续解析下一个块，不再整帧丢弃。仅在需要严格失败语义
|
||||
# （灰度回滚或抓虫）时设为 false。
|
||||
parse:
|
||||
lenient-block-failure: true
|
||||
jt808:
|
||||
enabled: false
|
||||
port: 10808
|
||||
max-frame-length: 1048
|
||||
jt1078:
|
||||
enabled: true
|
||||
jsatl12:
|
||||
enabled: true
|
||||
port: 7612
|
||||
file-store: file:///var/lingniu/alarm-files
|
||||
mqtt:
|
||||
enabled: false
|
||||
endpoints:
|
||||
- name: yutong
|
||||
uri: ${MQTT_URI:ssl://cpxlm.axxc.cn:38883}
|
||||
topic: ${MQTT_TOPIC:/ytforward/shln/+}
|
||||
qos: 1
|
||||
client-id: ${MQTT_CLIENT_ID:shlnClientId}
|
||||
username: ${MQTT_USER:shln}
|
||||
password: ${MQTT_PWD:}
|
||||
tls:
|
||||
ca-pem: ${MQTT_CA_PEM:}
|
||||
client-pem: ${MQTT_CLIENT_PEM:}
|
||||
client-key: ${MQTT_CLIENT_KEY:}
|
||||
xinda-push:
|
||||
enabled: false
|
||||
host: ${XINDA_HOST:115.231.168.135}
|
||||
port: ${XINDA_PORT:10100}
|
||||
username: ${XINDA_USER:浙江羚牛}
|
||||
password: ${XINDA_PWD:xd123456+}
|
||||
subscribe-msg-ids: 0200,0300,0401
|
||||
client-desc: 羚牛客户端
|
||||
|
||||
pipeline:
|
||||
disruptor:
|
||||
ring-buffer-size: 131072 # 2^17
|
||||
wait-strategy: yielding # blocking | sleeping | yielding | busy-spin
|
||||
producer-type: multi
|
||||
dedup:
|
||||
enabled: true
|
||||
cache-size: 200000
|
||||
ttl-seconds: 600
|
||||
rate-limit:
|
||||
per-vin-qps: 50
|
||||
|
||||
session:
|
||||
store: memory-and-redis
|
||||
token-ttl-seconds: 86400
|
||||
|
||||
sink:
|
||||
mq:
|
||||
# MQ Sink 总开关。true=装配 Kafka Producer 并投递事件；false=完全不装配（适合本地开发或 Kafka 不可达）。
|
||||
enabled: ${KAFKA_ENABLED:false}
|
||||
type: kafka # 后端选择：kafka | (rocketmq | pulsar 预留)
|
||||
bootstrap-servers: ${KAFKA_BROKERS:localhost:9092}
|
||||
compression-type: zstd
|
||||
linger-ms: 20
|
||||
batch-size: 65536
|
||||
acks: all
|
||||
enable-idempotence: true
|
||||
topics:
|
||||
realtime: vehicle.realtime
|
||||
location: vehicle.location
|
||||
alarm: vehicle.alarm
|
||||
session: vehicle.session
|
||||
media-meta: vehicle.media.meta
|
||||
raw-archive: vehicle.raw.archive
|
||||
dlq: vehicle.dlq
|
||||
archive:
|
||||
enabled: true
|
||||
type: local # local | s3 | oss
|
||||
path: ./archive/
|
||||
|
||||
management:
|
||||
endpoints:
|
||||
web:
|
||||
exposure:
|
||||
include: health,info,metrics,prometheus,env
|
||||
metrics:
|
||||
tags:
|
||||
application: lingniu-vehicle-ingest
|
||||
|
||||
logging:
|
||||
level:
|
||||
root: INFO
|
||||
com.lingniu.ingest: INFO
|
||||
com.lingniu.ingest.protocol.gb32960.inbound.Gb32960ChannelHandler: DEBUG
|
||||
com.lingniu.ingest.protocol.gb32960.codec.Gb32960FrameDecoder: DEBUG
|
||||
com.lingniu.ingest.protocol.gb32960.codec.Gb32960BodyParser: DEBUG
|
||||
# 信达 push 诊断：打开下面一行可以看到每一条收发的原始字节 hex dump
|
||||
com.lingniu.ingest.inbound.xinda: DEBUG
|
||||
@@ -1,145 +0,0 @@
|
||||
package com.lingniu.ingest.gateway;
|
||||
|
||||
import com.lingniu.ingest.protocol.jt808.downlink.Jt808Commands;
|
||||
import com.lingniu.ingest.protocol.jt808.model.Jt808Message;
|
||||
import com.lingniu.ingest.session.CommandDispatcher;
|
||||
import com.lingniu.ingest.session.DeviceSession;
|
||||
import com.lingniu.ingest.session.SessionStore;
|
||||
import org.springframework.web.bind.annotation.DeleteMapping;
|
||||
import org.springframework.web.bind.annotation.GetMapping;
|
||||
import org.springframework.web.bind.annotation.PostMapping;
|
||||
import org.springframework.web.bind.annotation.PutMapping;
|
||||
import org.springframework.web.bind.annotation.RequestBody;
|
||||
import org.springframework.web.bind.annotation.RequestMapping;
|
||||
import org.springframework.web.bind.annotation.RequestParam;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.Duration;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.CompletableFuture;
|
||||
import java.util.concurrent.CompletionException;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
/**
|
||||
* 下行命令 REST 入口。命令通过 {@link CommandDispatcher} 发往协议模块，
|
||||
* 由 protocol-jt808 的 {@code Jt808CommandDispatcher} 实际发送到终端。
|
||||
*
|
||||
* <p>路径参数 {@code clientId} 可以是 sessionId / phone / vin，
|
||||
* 由 {@link SessionStore} 三索引解析。
|
||||
*/
|
||||
@RestController
|
||||
@RequestMapping("/terminal")
|
||||
public class TerminalCommandController {
|
||||
|
||||
private static final Duration DEFAULT_TIMEOUT = Duration.ofSeconds(20);
|
||||
|
||||
private final SessionStore sessions;
|
||||
private final CommandDispatcher dispatcher;
|
||||
|
||||
public TerminalCommandController(SessionStore sessions, CommandDispatcher dispatcher) {
|
||||
this.sessions = sessions;
|
||||
this.dispatcher = dispatcher;
|
||||
}
|
||||
|
||||
@GetMapping("/session")
|
||||
public CommandResult session(@RequestParam String clientId) {
|
||||
Optional<DeviceSession> s = resolveSession(clientId);
|
||||
return s.<CommandResult>map(CommandResult::ok)
|
||||
.orElseGet(() -> CommandResult.failure(404, "session not found: " + clientId));
|
||||
}
|
||||
|
||||
@GetMapping("/location")
|
||||
public CommandResult queryLocation(@RequestParam String clientId) {
|
||||
return awaitSync(clientId, Jt808Commands.queryLocation(), "query-location");
|
||||
}
|
||||
|
||||
@GetMapping("/parameters")
|
||||
public CommandResult queryParameters(@RequestParam String clientId) {
|
||||
return awaitSync(clientId, Jt808Commands.queryParameters(), "query-parameters");
|
||||
}
|
||||
|
||||
@PutMapping("/parameters")
|
||||
public CommandResult setParameters(@RequestParam String clientId,
|
||||
@RequestBody Map<String, String> body) {
|
||||
List<Jt808Commands.ParamItem> items = body.entrySet().stream()
|
||||
.map(e -> new Jt808Commands.ParamItem(
|
||||
parseId(e.getKey()),
|
||||
e.getValue() == null ? new byte[0] : e.getValue().getBytes(StandardCharsets.UTF_8)))
|
||||
.collect(Collectors.toList());
|
||||
return awaitSync(clientId, Jt808Commands.setParameters(items), "set-parameters");
|
||||
}
|
||||
|
||||
@PostMapping("/control")
|
||||
public CommandResult terminalControl(@RequestParam String clientId,
|
||||
@RequestBody Map<String, Object> body) {
|
||||
int word = Integer.parseInt(body.getOrDefault("command", "0").toString());
|
||||
String params = body.getOrDefault("params", "").toString();
|
||||
return awaitSync(clientId, Jt808Commands.terminalControl(word, params), "terminal-control");
|
||||
}
|
||||
|
||||
@PostMapping("/ack")
|
||||
public CommandResult ack(@RequestParam String clientId, @RequestBody Map<String, Integer> body) {
|
||||
int ackSerial = body.getOrDefault("ackSerial", 0);
|
||||
int ackMsgId = body.getOrDefault("ackMsgId", 0);
|
||||
int result = body.getOrDefault("result", 0);
|
||||
return fireAndForget(clientId, Jt808Commands.platformAck(ackSerial, ackMsgId, result), "platform-ack");
|
||||
}
|
||||
|
||||
@GetMapping("/attributes")
|
||||
public CommandResult attributes(@RequestParam String clientId) {
|
||||
return CommandResult.notImplemented("0x8107 query-attributes");
|
||||
}
|
||||
|
||||
@DeleteMapping("/area")
|
||||
public CommandResult deleteArea(@RequestParam String clientId, @RequestParam String ids) {
|
||||
return CommandResult.notImplemented("0x8601 delete-area");
|
||||
}
|
||||
|
||||
@PostMapping("/alarm_ack")
|
||||
public CommandResult alarmAck(@RequestParam String clientId, @RequestBody Map<String, Object> body) {
|
||||
return CommandResult.notImplemented("0x8203 alarm-ack");
|
||||
}
|
||||
|
||||
// ===== helpers =====
|
||||
|
||||
private Optional<DeviceSession> resolveSession(String clientId) {
|
||||
return sessions.findBySessionId(clientId)
|
||||
.or(() -> sessions.findByPhone(clientId))
|
||||
.or(() -> sessions.findByVin(clientId));
|
||||
}
|
||||
|
||||
private CommandResult awaitSync(String clientId, Jt808Commands.DownlinkCommand cmd, String op) {
|
||||
CompletableFuture<Jt808Message> future =
|
||||
dispatcher.request(clientId, cmd, Jt808Message.class, DEFAULT_TIMEOUT);
|
||||
try {
|
||||
Jt808Message resp = future.join();
|
||||
return CommandResult.ok(Map.of(
|
||||
"messageId", "0x" + Integer.toHexString(resp.header().messageId()),
|
||||
"serial", resp.header().serialNo(),
|
||||
"phone", resp.header().phone()));
|
||||
} catch (CompletionException e) {
|
||||
Throwable cause = e.getCause() != null ? e.getCause() : e;
|
||||
return CommandResult.failure(500, op + " failed: " + cause.getMessage());
|
||||
} catch (Exception e) {
|
||||
return CommandResult.failure(500, op + " failed: " + e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
private CommandResult fireAndForget(String clientId, Jt808Commands.DownlinkCommand cmd, String op) {
|
||||
try {
|
||||
dispatcher.notify(clientId, cmd).join();
|
||||
return CommandResult.ok(Map.of("op", op));
|
||||
} catch (Exception e) {
|
||||
return CommandResult.failure(500, op + " failed: " + e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
private static long parseId(String key) {
|
||||
if (key == null || key.isBlank()) return 0;
|
||||
String k = key.startsWith("0x") ? key.substring(2) : key;
|
||||
return Long.parseLong(k, 16);
|
||||
}
|
||||
}
|
||||
733
docs/module-data-flow.html
Normal file
733
docs/module-data-flow.html
Normal file
@@ -0,0 +1,733 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>lingniu-vehicle-ingest 模块与数据流</title>
|
||||
<style>
|
||||
:root {
|
||||
--bg: #f7f8fa;
|
||||
--panel: #ffffff;
|
||||
--ink: #172033;
|
||||
--muted: #667085;
|
||||
--line: #d8dee8;
|
||||
--blue: #2563eb;
|
||||
--green: #0f766e;
|
||||
--amber: #b45309;
|
||||
--red: #b42318;
|
||||
--purple: #6d28d9;
|
||||
}
|
||||
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
body {
|
||||
margin: 0;
|
||||
background: var(--bg);
|
||||
color: var(--ink);
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", "PingFang SC", "Microsoft YaHei", sans-serif;
|
||||
line-height: 1.55;
|
||||
}
|
||||
|
||||
header {
|
||||
padding: 32px 40px 24px;
|
||||
background: #101828;
|
||||
color: #ffffff;
|
||||
}
|
||||
|
||||
header h1 {
|
||||
margin: 0 0 10px;
|
||||
font-size: 30px;
|
||||
line-height: 1.2;
|
||||
letter-spacing: 0;
|
||||
}
|
||||
|
||||
header p {
|
||||
margin: 0;
|
||||
max-width: 980px;
|
||||
color: #d0d5dd;
|
||||
font-size: 15px;
|
||||
}
|
||||
|
||||
main {
|
||||
max-width: 1420px;
|
||||
margin: 0 auto;
|
||||
padding: 28px 28px 44px;
|
||||
}
|
||||
|
||||
section {
|
||||
margin-bottom: 24px;
|
||||
padding: 24px;
|
||||
background: var(--panel);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 8px;
|
||||
box-shadow: 0 1px 2px rgba(16, 24, 40, 0.04);
|
||||
}
|
||||
|
||||
h2 {
|
||||
margin: 0 0 14px;
|
||||
font-size: 21px;
|
||||
letter-spacing: 0;
|
||||
}
|
||||
|
||||
h3 {
|
||||
margin: 22px 0 10px;
|
||||
font-size: 17px;
|
||||
}
|
||||
|
||||
p {
|
||||
margin: 0 0 12px;
|
||||
}
|
||||
|
||||
.summary-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(4, minmax(0, 1fr));
|
||||
gap: 12px;
|
||||
margin-top: 18px;
|
||||
}
|
||||
|
||||
.summary-card {
|
||||
padding: 14px 16px;
|
||||
border: 1px solid var(--line);
|
||||
border-left: 4px solid var(--blue);
|
||||
border-radius: 6px;
|
||||
background: #fbfcfe;
|
||||
}
|
||||
|
||||
.summary-card strong {
|
||||
display: block;
|
||||
margin-bottom: 4px;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
.summary-card span {
|
||||
color: var(--muted);
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.diagram {
|
||||
overflow-x: auto;
|
||||
padding: 12px;
|
||||
border: 1px solid var(--line);
|
||||
border-radius: 6px;
|
||||
background: #ffffff;
|
||||
}
|
||||
|
||||
.legend {
|
||||
display: flex;
|
||||
flex-wrap: wrap;
|
||||
gap: 10px;
|
||||
margin: 14px 0 0;
|
||||
color: var(--muted);
|
||||
font-size: 13px;
|
||||
}
|
||||
|
||||
.legend span {
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 6px;
|
||||
}
|
||||
|
||||
.dot {
|
||||
width: 10px;
|
||||
height: 10px;
|
||||
border-radius: 50%;
|
||||
display: inline-block;
|
||||
}
|
||||
|
||||
.dot.entry { background: var(--blue); }
|
||||
.dot.core { background: var(--green); }
|
||||
.dot.sink { background: var(--amber); }
|
||||
.dot.safety { background: var(--red); }
|
||||
.dot.command { background: var(--purple); }
|
||||
|
||||
table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
margin-top: 12px;
|
||||
font-size: 14px;
|
||||
}
|
||||
|
||||
th,
|
||||
td {
|
||||
padding: 10px 12px;
|
||||
border: 1px solid var(--line);
|
||||
text-align: left;
|
||||
vertical-align: top;
|
||||
}
|
||||
|
||||
th {
|
||||
background: #f2f4f7;
|
||||
color: #344054;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
td:first-child {
|
||||
width: 190px;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", monospace;
|
||||
font-size: 13px;
|
||||
color: #101828;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.tag {
|
||||
display: inline-block;
|
||||
margin: 0 6px 6px 0;
|
||||
padding: 2px 7px;
|
||||
border-radius: 999px;
|
||||
background: #eef2ff;
|
||||
color: #3730a3;
|
||||
font-size: 12px;
|
||||
white-space: nowrap;
|
||||
}
|
||||
|
||||
.tag.red {
|
||||
background: #fee4e2;
|
||||
color: #912018;
|
||||
}
|
||||
|
||||
.tag.green {
|
||||
background: #ccfbef;
|
||||
color: #134e48;
|
||||
}
|
||||
|
||||
.tag.amber {
|
||||
background: #fef0c7;
|
||||
color: #93370d;
|
||||
}
|
||||
|
||||
.callout {
|
||||
margin-top: 14px;
|
||||
padding: 14px 16px;
|
||||
border-left: 4px solid var(--red);
|
||||
border-radius: 6px;
|
||||
background: #fff7f5;
|
||||
}
|
||||
|
||||
code {
|
||||
padding: 1px 4px;
|
||||
border-radius: 4px;
|
||||
background: #eef2f6;
|
||||
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, "Liberation Mono", monospace;
|
||||
font-size: 0.95em;
|
||||
}
|
||||
|
||||
@media (max-width: 900px) {
|
||||
header {
|
||||
padding: 24px 20px 20px;
|
||||
}
|
||||
|
||||
main {
|
||||
padding: 18px;
|
||||
}
|
||||
|
||||
section {
|
||||
padding: 18px;
|
||||
}
|
||||
|
||||
.summary-grid {
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
}
|
||||
|
||||
table {
|
||||
display: block;
|
||||
overflow-x: auto;
|
||||
white-space: nowrap;
|
||||
}
|
||||
}
|
||||
|
||||
@media (max-width: 560px) {
|
||||
.summary-grid {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<h1>lingniu-vehicle-ingest 模块与数据流</h1>
|
||||
<p>
|
||||
本图基于当前 Maven 多模块项目整理，用于后续讨论协议接入、内部字段、氢能车辆运营统计、安全告警和下游消费边界。
|
||||
当前架构将接入层、历史明细、Redis 热状态和日统计拆成独立模块，通过统一 Kafka Envelope 解耦。
|
||||
</p>
|
||||
</header>
|
||||
|
||||
<main>
|
||||
<section>
|
||||
<h2>一、系统定位</h2>
|
||||
<p>
|
||||
当前项目不是业务库写入服务。它的核心职责是把 GB/T 32960、JT/T 808、JT/T 1078、JSATL12、
|
||||
MQTT、信达 Push 等不同来源的车辆数据统一转换为 <code>VehicleEvent</code> 和全字段 <code>TelemetrySnapshot</code>，
|
||||
再通过 Kafka、原始报文归档、Parquet 明细文件库、Redis 热状态和独立统计模块交给下游业务。
|
||||
</p>
|
||||
<div class="summary-grid">
|
||||
<div class="summary-card">
|
||||
<strong>接入方式</strong>
|
||||
<span>Netty TCP、MQTT、HTTP Push、后续可扩展更多 Inbound Adapter。</span>
|
||||
</div>
|
||||
<div class="summary-card">
|
||||
<strong>统一模型</strong>
|
||||
<span>所有协议归一到 <code>VehicleEvent</code> 和全字段内部 <code>TelemetrySnapshot</code>。</span>
|
||||
</div>
|
||||
<div class="summary-card">
|
||||
<strong>输出方式</strong>
|
||||
<span>Kafka Protobuf Envelope + 原始 bytes 冷存 + Parquet/DuckDB + Redis 热状态。</span>
|
||||
</div>
|
||||
<div class="summary-card">
|
||||
<strong>业务边界</strong>
|
||||
<span>日统计由 <code>vehicle-stat-service</code> 消费 Kafka 实现，告警工单、资产业务继续由下游消费。</span>
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>二、模块分层图</h2>
|
||||
<div class="diagram">
|
||||
<pre class="mermaid">
|
||||
flowchart TB
|
||||
subgraph external["外部来源"]
|
||||
vehicle["车辆终端<br/>GB/T 32960 / JT808 / JT1078"]
|
||||
mqttSource["车企或平台 MQTT"]
|
||||
xindaSource["信达 Push"]
|
||||
commandClient["业务系统 / 运维平台<br/>HTTP 下行命令"]
|
||||
end
|
||||
|
||||
subgraph protocolModules["协议模块 modules/protocols"]
|
||||
gb["protocol-gb32960<br/>32960 Netty 接入、鉴权、ACK、解析"]
|
||||
jt808["protocol-jt808<br/>808 Netty 接入、会话、位置/批量位置、媒体、透传、下行"]
|
||||
jt1078["protocol-jt1078<br/>1078 信令 + TCP/UDP RTP 媒体流归档"]
|
||||
jsatl12["protocol-jsatl12<br/>主动安全附件流接入"]
|
||||
end
|
||||
|
||||
subgraph inbound["入口适配 modules/inbound"]
|
||||
mqtt["inbound-mqtt<br/>MQTT endpoint 生命周期、PEM TLS"]
|
||||
mqttProfile["MqttProfileRegistry<br/>按 endpoint profile 解析/映射"]
|
||||
xinda["inbound-xinda-push<br/>HTTP Push 适配"]
|
||||
end
|
||||
|
||||
subgraph protocolBase["核心与公共能力 modules/core"]
|
||||
codec["ingest-codec-common<br/>BCD / CRC / BCC / bit 工具"]
|
||||
session["session-core<br/>设备会话、命令分发接口、会话存储"]
|
||||
identity["vehicle-identity<br/>跨协议身份解析、外部标识绑定、文件持久化"]
|
||||
obs["observability<br/>metrics / tracing / health"]
|
||||
api["ingest-api<br/>ProtocolId / RawFrame / VehicleEvent / 注解 SPI"]
|
||||
registry["HandlerRegistry<br/>按协议、命令、infoType 路由"]
|
||||
dispatcher["Dispatcher<br/>RawFrame 到 Handler 到 EventBus"]
|
||||
bus["DisruptorEventBus<br/>高吞吐事件发布"]
|
||||
end
|
||||
|
||||
subgraph sink["输出与明细存储 modules/sinks"]
|
||||
mq["sink-mq<br/>VehicleEvent 到 Protobuf Envelope 到 Kafka"]
|
||||
archive["sink-archive<br/>RawArchive 到本地/S3/OSS 冷存"]
|
||||
fileStore["event-file-store<br/>VehicleEvent/RawArchive 索引到 Parquet，DuckDB 按日查询"]
|
||||
end
|
||||
|
||||
subgraph services["消费服务 modules/services"]
|
||||
history["event-history-service<br/>Kafka 全字段事件到历史查询/导出"]
|
||||
state["vehicle-state-service<br/>Kafka 全字段事件到 Redis 热状态"]
|
||||
stat["vehicle-stat-service<br/>Kafka 全字段事件到可配置日统计"]
|
||||
end
|
||||
|
||||
subgraph apps["应用入口 modules/apps"]
|
||||
gateway["command-gateway<br/>HTTP 到设备命令入口"]
|
||||
bootall["bootstrap-all<br/>聚合所有 AutoConfiguration 一体启动"]
|
||||
end
|
||||
|
||||
vehicle --> gb
|
||||
vehicle --> jt808
|
||||
vehicle --> jt1078
|
||||
vehicle --> jsatl12
|
||||
mqttSource --> mqtt
|
||||
mqtt --> mqttProfile
|
||||
mqttProfile --> api
|
||||
xindaSource --> xinda
|
||||
commandClient --> gateway
|
||||
|
||||
gb --> codec
|
||||
jt808 --> codec
|
||||
jt1078 --> codec
|
||||
jsatl12 --> codec
|
||||
|
||||
gb --> api
|
||||
jt808 --> api
|
||||
xinda --> api
|
||||
xinda --> dispatcher
|
||||
jsatl12 --> archive
|
||||
jsatl12 --> jt808
|
||||
|
||||
api --> registry
|
||||
registry --> dispatcher
|
||||
dispatcher --> bus
|
||||
bus --> mq
|
||||
bus --> archive
|
||||
bus --> fileStore
|
||||
mq --> history
|
||||
mq --> state
|
||||
mq --> stat
|
||||
|
||||
gateway --> session
|
||||
session --> jt808
|
||||
session --> gb
|
||||
identity --> jt808
|
||||
identity --> mqttProfile
|
||||
identity --> xinda
|
||||
|
||||
obs -.监控.-> dispatcher
|
||||
obs -.监控.-> bus
|
||||
obs -.监控.-> mq
|
||||
bootall -.装配.-> gb
|
||||
bootall -.装配.-> jt808
|
||||
bootall -.装配.-> mqtt
|
||||
bootall -.装配.-> xinda
|
||||
bootall -.装配.-> mq
|
||||
bootall -.装配.-> archive
|
||||
bootall -.装配.-> fileStore
|
||||
bootall -.装配.-> history
|
||||
bootall -.装配.-> state
|
||||
bootall -.装配.-> stat
|
||||
|
||||
classDef entry fill:#eff6ff,stroke:#2563eb,color:#1e3a8a;
|
||||
classDef core fill:#ecfdf3,stroke:#0f766e,color:#134e48;
|
||||
classDef sink fill:#fffbeb,stroke:#b45309,color:#7c2d12;
|
||||
classDef command fill:#f5f3ff,stroke:#6d28d9,color:#4c1d95;
|
||||
classDef support fill:#f8fafc,stroke:#64748b,color:#334155;
|
||||
|
||||
class gb,jt808,jt1078,jsatl12,mqtt,mqttProfile,xinda entry;
|
||||
class api,registry,dispatcher,bus,identity core;
|
||||
class mq,archive,fileStore sink;
|
||||
class gateway,session command;
|
||||
class codec,obs,bootall support;
|
||||
</pre>
|
||||
</div>
|
||||
<div class="legend">
|
||||
<span><i class="dot entry"></i>入口/协议模块</span>
|
||||
<span><i class="dot core"></i>核心管线</span>
|
||||
<span><i class="dot sink"></i>输出层</span>
|
||||
<span><i class="dot command"></i>下行命令/会话</span>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>三、上行数据流转</h2>
|
||||
<div class="diagram">
|
||||
<pre class="mermaid">
|
||||
sequenceDiagram
|
||||
autonumber
|
||||
participant Device as 车辆/平台
|
||||
participant Inbound as Inbound Adapter<br/>Netty/MQTT/Push
|
||||
participant Decoder as FrameDecoder<br/>MessageDecoder
|
||||
participant Dispatcher as Dispatcher
|
||||
participant Handler as Protocol Handler<br/>Mapper
|
||||
participant EventBus as DisruptorEventBus
|
||||
participant MQ as sink-mq<br/>Kafka Envelope
|
||||
participant Archive as sink-archive<br/>ArchiveStore
|
||||
participant FileStore as event-file-store<br/>Parquet + DuckDB
|
||||
participant Consumer as 下游业务消费者
|
||||
|
||||
Device->>Inbound: 原始报文 bytes / MQTT message / HTTP payload
|
||||
Inbound->>Decoder: 粘包拆帧、校验、协议体解析
|
||||
Inbound->>Inbound: MQTT 按 endpoint profile 选择厂商解析器
|
||||
Decoder->>Dispatcher: RawFrame(protocol, command, infoType, payload, rawBytes)
|
||||
Dispatcher->>EventBus: 先发布 RawArchive 事件
|
||||
EventBus->>Archive: 保存原始 bytes，生成可回放材料
|
||||
Dispatcher->>Handler: 根据 protocol + command + infoType 路由
|
||||
Handler->>Handler: 协议字段映射为内部字段
|
||||
Dispatcher->>Handler: 绑定 rawArchiveKey / rawArchiveUri
|
||||
Handler->>EventBus: VehicleEvent(Realtime / Location / Alarm / Login ...)
|
||||
EventBus->>MQ: 投递规整后的业务事件
|
||||
EventBus->>FileStore: 写入解析后明细事件
|
||||
MQ->>MQ: EnvelopeMapper 转 Protobuf
|
||||
MQ->>Consumer: Kafka topic，key=vin，单车有序
|
||||
FileStore->>Consumer: 按日期范围正序/倒序查询和导出
|
||||
Consumer->>Consumer: 历史明细查询、Redis 热状态、可配置日统计、告警工单
|
||||
</pre>
|
||||
</div>
|
||||
<div class="callout">
|
||||
<strong>关键边界：</strong>
|
||||
协议字段只在协议模块内解释，出了 Mapper 以后都应该使用内部字段。统计每日里程、每日用电量、每日用氢量、储氢安全和氢泄露告警时，应消费全字段 TelemetrySnapshot，不要直接依赖某个协议的原始字段名。
|
||||
</div>
|
||||
<div class="callout">
|
||||
<strong>原始追溯：</strong>
|
||||
Dispatcher 为所有带 rawBytes 的 RawFrame 生成统一的 <code>rawArchiveKey</code>，业务事件 metadata 使用 <code>rawArchiveUri=archive://...</code> 指向同一份原始 bytes。归档文件的真实本地/S3/OSS 地址只属于 ArchiveStore，查询、导出和 Kafka Envelope 使用逻辑 URI 解耦存储实现。
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>四、32960 细化链路</h2>
|
||||
<div class="diagram">
|
||||
<pre class="mermaid">
|
||||
flowchart LR
|
||||
terminal["氢能车 TBOX<br/>GB/T 32960"] --> netty["Gb32960NettyServer<br/>TCP / TLS / Idle 检测"]
|
||||
netty --> access["Gb32960AccessService<br/>VIN 白名单 / 平台登录鉴权"]
|
||||
netty --> frame["Gb32960FrameDecoder<br/>拆帧、转义、校验"]
|
||||
frame --> decoder["Gb32960MessageDecoder<br/>Header + Body 解析"]
|
||||
decoder --> diag["Gb32960FrameDiagnostics<br/>首帧、Raw 块、异常诊断"]
|
||||
decoder --> handler["Gb32960ChannelHandler<br/>ACK / NACK / dispatch"]
|
||||
handler --> ack["Gb32960AckService<br/>登录应答、失败断开"]
|
||||
handler --> dispatcher["Dispatcher"]
|
||||
dispatcher --> rtHandler["Gb32960RealtimeHandler"]
|
||||
rtHandler --> mapper["Gb32960EventMapper<br/>协议字段到内部字段"]
|
||||
mapper --> realtime["RealtimePayload<br/>速度、里程、电量、氢量、压力、温度"]
|
||||
mapper --> alarm["AlarmPayload<br/>安全分类、氢泄露、告警等级"]
|
||||
mapper --> login["Login / Logout / Heartbeat"]
|
||||
realtime --> bus["DisruptorEventBus"]
|
||||
alarm --> bus
|
||||
login --> bus
|
||||
bus --> kafka["Kafka Envelope"]
|
||||
bus --> archive["RawArchive 冷存"]
|
||||
|
||||
classDef safety fill:#fff1f3,stroke:#b42318,color:#7a271a;
|
||||
classDef core fill:#ecfdf3,stroke:#0f766e,color:#134e48;
|
||||
classDef entry fill:#eff6ff,stroke:#2563eb,color:#1e3a8a;
|
||||
classDef sink fill:#fffbeb,stroke:#b45309,color:#7c2d12;
|
||||
|
||||
class terminal,netty,frame,decoder,handler entry;
|
||||
class access,diag,ack,dispatcher,rtHandler,mapper,bus core;
|
||||
class alarm safety;
|
||||
class kafka,archive sink;
|
||||
</pre>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>五、模块职责表</h2>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>模块</th>
|
||||
<th>职责</th>
|
||||
<th>主要输入</th>
|
||||
<th>主要输出</th>
|
||||
<th>业务开发关注点</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>modules/core/ingest-api</td>
|
||||
<td>定义系统边界：协议 ID、RawFrame、IngestContext、VehicleEvent、Payload、Handler 注解、Sink SPI；ProtocolId 提供 UNKNOWN 隔离桶，供消费侧兼容未来协议或脏数据，不作为正式接入协议使用；consumer 包定义 EnvelopeIngestor、EnvelopeConsumerProcessor 和 EnvelopeDeadLetterSink，统一 Kafka 消费结果与死信边界。</td>
|
||||
<td>无直接外部输入。</td>
|
||||
<td>全项目共享的接口和内部事件模型。</td>
|
||||
<td><span class="tag red">内部字段定义</span><span class="tag">新增事件类型</span><span class="tag">字段兼容性</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/core/ingest-codec-common</td>
|
||||
<td>公共编解码工具，承载 BCD、CRC、BCC、bit 操作等协议底层能力。</td>
|
||||
<td>协议模块传入的 byte、bit、校验材料。</td>
|
||||
<td>解析辅助结果。</td>
|
||||
<td><span class="tag">协议基础能力</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/core/ingest-core</td>
|
||||
<td>核心管线。扫描 Handler，按协议和命令路由，执行拦截器，发布 VehicleEvent 到 Disruptor。</td>
|
||||
<td>RawFrame。</td>
|
||||
<td>VehicleEvent、RawArchive。</td>
|
||||
<td><span class="tag green">吞吐</span><span class="tag green">路由</span><span class="tag amber">原始可回放</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/core/session-core</td>
|
||||
<td>设备会话、命令下发抽象、SessionStore、CommandDispatcher 默认实现。SessionStore 支持 <code>memory</code> 和 <code>redis</code> 两种后端；生产默认使用 Redis 保存 <code>sessionId</code>、VIN 和 phone 三索引及 TTL，内存实现作为本地开发/无 Redis 降级，协议模块只依赖 SPI，不感知存储后端。</td>
|
||||
<td>设备连接、命令请求。</td>
|
||||
<td>会话状态、命令分发结果、Redis 会话索引。</td>
|
||||
<td><span class="tag">在线状态</span><span class="tag">下行控制</span><span class="tag green">多实例会话索引</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/core/vehicle-identity</td>
|
||||
<td>跨协议车辆身份解析，维护 phone、deviceId、plate 到 VIN 的绑定关系；支持 memory 和 append-only JSONL file 后端，避免重启丢失绑定。</td>
|
||||
<td>协议模块传入的外部标识。</td>
|
||||
<td>稳定 VIN、解析来源、是否已命中绑定。</td>
|
||||
<td><span class="tag red">统计主键</span><span class="tag">多协议归一</span><span class="tag green">持久化绑定</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/core/observability</td>
|
||||
<td>监控、指标、追踪、健康检查等横切能力。</td>
|
||||
<td>核心管线和 Sink 运行状态。</td>
|
||||
<td>Micrometer 指标、健康信息。</td>
|
||||
<td><span class="tag green">可用性</span><span class="tag">延迟监控</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/protocols/protocol-gb32960</td>
|
||||
<td>GB/T 32960 接入、鉴权、ACK、报文解析、实时数据和告警映射。</td>
|
||||
<td>32960 TCP 报文。</td>
|
||||
<td>Realtime、Location、Alarm、Login、Logout、Heartbeat、RawFrame。</td>
|
||||
<td><span class="tag red">氢泄露</span><span class="tag red">储罐安全</span><span class="tag">电量/氢量</span><span class="tag">里程</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/protocols/protocol-jt808</td>
|
||||
<td>JT/T 808 接入、注册/鉴权/心跳/注销、位置/位置附加项、批量位置、参数/属性、多媒体、透传、未知上行兜底透传、分帧边界异常和协议解析异常坏帧兜底、下行命令和超长下行分包能力；0x0200 位置附加项 0x01 总里程按 0.1km 解码为内部字段 <code>total_mileage_km</code>，后续每日里程统计不依赖 808 原始字段；0x0102 鉴权帧解析会保留完整 token，并尽量提取 IMEI 和软件版本；注册和鉴权会话均使用共享身份解析后的内部 VIN，避免 deviceId/IMEI 污染后续会话和下行边界；终端连接断开时同步解绑 Channel 并清理 <code>SessionStore</code> 会话，避免 command-gateway 误判离线车辆仍可下行；正常上行 RawArchive 和业务事件 metadata 均使用共享身份解析后的内部 vin，并保留 phone、identityResolved、identitySource 便于冷存和业务事件按同一车辆查询；无法解析终端身份的坏帧 RawArchive 和 Passthrough 均按 vin=unknown、identityResolved=false、identitySource=UNKNOWN 标记；0x0900 透传事件按原始消息 ID 归类，透传类型写入 passthroughType metadata。</td>
|
||||
<td>808 TCP 报文。</td>
|
||||
<td>Location、Login、Logout、Heartbeat、MediaMeta、Passthrough、未知上行 Raw 兜底、坏帧 Passthrough、会话状态、下行响应。</td>
|
||||
<td><span class="tag">GPS 位置</span><span class="tag">在线状态</span><span class="tag">媒体证据</span><span class="tag">命令链路</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/protocols/protocol-jt1078</td>
|
||||
<td>JT/T 1078 音视频能力。信令在 JT808 mapper 存在时桥接复用 JT808 连接和包头；0x1005 乘客流量保留原始 body，同时结构化 channelId、startTime、endTime、passengerGetOn、passengerGetOff metadata 便于查询/导出；0x1205 文件列表保留原始 body，同时结构化 responseSerialNo、fileCount 摘要；下行信令编码由本模块提供，覆盖 0x1003 音视频属性查询、0x9101 实时预览、0x9102 实时控制、0x9201 历史回放、0x9202 回放控制、0x9205 资源列表查询、0x9206 文件上传和 0x9207 文件上传控制，命令发送仍由 command-gateway 经 CommandDispatcher 复用 JT808 在线通道；TCP/UDP RTP 媒体流独立端口接入，生产默认端口按旧接收服务对齐为 <code>11078</code>，按 VIN/通道/时间分段写 ArchiveStore，Kafka 只发送 MediaMeta 引用，事件 metadata 暴露内部 vin、sim、channelId、dataType、packetType、segment、sequence、segmentSizeBytes、archiveKey 和 archiveRef，MediaMeta.sizeBytes 同步当前片段大小，MediaMeta 按 archiveKey 去重，避免同一 SIM 运行中从 fallback 身份切换到内部 VIN 时漏发新归档引用，便于文件明细库追踪、展示和导出；超长/短包/坏魔数等 RTP 入口异常和媒体归档失败都会兜底产出 Passthrough 错误事件，坏 RTP 统一走 Dispatcher 产出 RawArchive 和 Passthrough，统一标记 parseError 和 parseErrorMessage，并保留内部 vin、原始 bytes、peer、长度、原因、RawArchive 引用、segment 和 archiveKey 便于排障；坏 RTP 头部可读时会提取 SIM 并通过共享身份服务映射 VIN。</td>
|
||||
<td>1078 信令报文、TCP/UDP RTP 媒体流。</td>
|
||||
<td>MediaMeta、乘客流量 Passthrough、文件列表 Passthrough、坏 RTP/归档失败 Passthrough、归档媒体分段。</td>
|
||||
<td><span class="tag">视频扩展</span><span class="tag green">信令事件化</span><span class="tag amber">大流冷存</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/protocols/protocol-jsatl12</td>
|
||||
<td>苏标主动安全报警附件接入。依赖 ArchiveStore 和 JT808 decoder；生产默认端口按旧接收服务对齐为 <code>7612</code>；附件 DataPacket 按旧服务真实布局 <code>01cd + 50B文件名 + offset + length + data</code> 分帧解析，不再把文件名前 4 字节误判为帧长；附件数据写 ArchiveStore 后通过 Dispatcher 产出 MediaMeta 引用事件，事件 metadata 保留 fileName、fileOffset、declaredChunkSizeBytes、archiveKey 和 archiveRef，便于附件明细查询、导出和补传诊断；内置附件状态机处理 T1210 文件清单、DataPacket 分块区间和 T1212 上传完成消息，数据分块会按 T1210 文件清单中的文件名继承手机号并通过共享身份解析器得到内部 VIN，入口 RawFrame、ArchivedChunk 和 MediaMeta 均保留 phone、vin、identityResolved、identitySource，只有找不到文件归属时才标记 UNKNOWN；按文件名合并无身份数据分块并返回 0x9212 完成/补传应答，补传应答包含缺失 offset/length 区间；附件归档失败会产出 JSATL12 Passthrough 错误事件并保留原始分块；JT_MESSAGE 信令复用 JT808 decoder 后进入 Dispatcher，桥接 RawFrame 使用共享身份解析后的内部 VIN并保留 phone、identityResolved、identitySource，坏 JT 信令入口 RawFrame 同样标记 UNKNOWN 身份；超长/畸形附件帧、未闭合超长 JT 信令和坏 JT 信令都以 JSATL12 Passthrough 兜底，附件帧异常同时标记 frameError 与统一 parseError/parseErrorMessage，保留原始字节、peer、长度和错误元数据，帧长和 worker 线程支持部署配置。</td>
|
||||
<td>主动安全附件流、JT808 信令帧。</td>
|
||||
<td>归档文件、MediaMeta、JT808 统一事件、坏帧/坏信令/归档失败 Passthrough。</td>
|
||||
<td><span class="tag amber">附件归档</span><span class="tag">报警证据</span><span class="tag">信令复用</span><span class="tag">错误隔离</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/inbound/inbound-mqtt</td>
|
||||
<td>MQTT 多 endpoint 生命周期管理，支持 PEM CA、客户端证书/私钥双向 TLS，按 endpoint profile 路由到厂商解析/映射实现，默认提供宇通 profile；生产默认接入口径参考旧接收服务：<code>ssl://cpxlm.axxc.cn:38883</code>、topic <code>/ytforward/shln/+</code>、QoS 2、cleanSession=false、keepAlive=20s、connectTimeout=10s，用户名和密码仍通过环境变量注入；入站 RawFrame 和事件映射均接入统一车辆身份解析，解析器保留 externalVin、phone、mqttDeviceId、plateNo，优先按设备号/终端 ID/IMEI 绑定解析内部 VIN，未绑定且字段形态为 VIN 时再作为显式 VIN 兜底，冷存和事件 metadata 均保留这些外部标识、identityResolved、identitySource；未知 profile、解析失败、profile 运行时异常、endpoint 初始化/连接/订阅失败均兜底为 Passthrough，解析失败诊断和 profile 异常诊断都会同步写入 RawArchive metadata，并在最终 Passthrough 事件保留 parseErrorMessage 或 operational reason，无法解析设备身份的兜底事件和运行类归档都按 vin=unknown、identityResolved=false、identitySource=UNKNOWN 标记，保留原始 payload 或 operational 错误信息便于排障和归档追溯。</td>
|
||||
<td>MQTT topic/message、endpoint profile、TLS PEM 配置。</td>
|
||||
<td>统一 RawFrame、Realtime、Location、Passthrough。</td>
|
||||
<td><span class="tag">车企平台接入</span><span class="tag green">profile 扩展</span><span class="tag green">身份映射</span><span class="tag green">双向 TLS</span><span class="tag green">错误隔离</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/inbound/inbound-xinda-push</td>
|
||||
<td>信达 Push 接入适配，基于 PushClient 接收真实协议帧并统一转换为 RawFrame；入站 RawFrame 和事件映射均接入统一车辆身份解析，冷存和事件 metadata 均携带内部 vin，并保留 externalVin、phone、deviceId、plateNo、identityResolved、identitySource；0200 映射 Location，并将 totalMileage、mileage 等里程别名归一为内部字段 <code>total_mileage_km</code>；0300 映射标准 AlarmPayload，氢气泄漏类报警统一提升为 HYDROGEN_LEAK/CRITICAL 并标记 hydrogenLeakDetected、hydrogenLeakActionRequired，报警元数据保留 alarmType、alarmName、alarmLevel；未知业务 cmd、非法十六进制 cmd、JSON 异常、身份解析/业务处理异常、启动配置缺失、启动异常、登录拒绝和坏登录应答均兜底为 Passthrough，运行类错误按 vin=unknown、identityResolved=false、identitySource=UNKNOWN 归档，非法 cmd 命令码归零并标记 malformedCommand，业务消息 JSON 异常会在 RawArchive metadata 和最终 Passthrough 事件同时保留 parseErrorMessage，业务处理异常会保留原始 payload 并标记 operationalError、phase=message 和 reason，便于归档追溯。</td>
|
||||
<td>信达 Push payload。</td>
|
||||
<td>RawFrame、Location、Passthrough、operational error。</td>
|
||||
<td><span class="tag">第三方平台接入</span><span class="tag green">统一 Dispatcher</span><span class="tag green">身份映射</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/sinks/sink-mq</td>
|
||||
<td>Kafka Sink。把 VehicleEvent 转成 Protobuf Envelope，按 VIN 分区投递；业务事件携带全字段 TelemetrySnapshot 和 rawArchiveUri，显式 RawArchive Envelope 会填充 archive:// 逻辑 URI 与 size，默认 Kafka Sink 仍不发送原始 bytes 本体；同时提供 KafkaEnvelopeConsumerFactory、KafkaEnvelopeConsumerRunner、KafkaEnvelopeConsumerWorker 和 KafkaEnvelopeDeadLetterSink，统一服务侧 Kafka 消费启动、独立 group 绑定、死信发布和 topic 到 EnvelopeConsumerProcessor 的分发。</td>
|
||||
<td>VehicleEvent。</td>
|
||||
<td>Kafka Protobuf 消息、消费侧 DLQ 记录。</td>
|
||||
<td><span class="tag green">单车有序</span><span class="tag">Schema 演进</span><span class="tag red">安全字段下发给消费者</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/sinks/sink-archive</td>
|
||||
<td>原始报文冷存，当前有本地文件系统实现，后续可替换 S3/OSS；写入键与业务事件 metadata 中的 <code>rawArchiveKey</code> 保持一致。</td>
|
||||
<td>RawArchive 事件、附件流。</td>
|
||||
<td>可回放原始文件、<code>archive://...</code> 逻辑引用。</td>
|
||||
<td><span class="tag amber">问题排查</span><span class="tag amber">合规留痕</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/sinks/event-file-store</td>
|
||||
<td>文件型明细库，使用 Parquet 存储解析后的事件明细和 RawArchive 可查询索引，使用 DuckDB 按日期范围正序/倒序查询；RawArchive 只存 archive:// 逻辑 URI、命令码、长度和 metadata，不嵌入原始 bytes，原始 bytes 本体仍由 sink-archive 管理；读取历史文件时未知 protocol 会降级为 UNKNOWN，避免查询中断。</td>
|
||||
<td>VehicleEvent 或 EventFileRecord。</td>
|
||||
<td>protocol/date 分区的 Parquet 文件和查询结果。</td>
|
||||
<td><span class="tag green">按日查询</span><span class="tag green">导出</span><span class="tag">字段配置</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/services/event-history-service</td>
|
||||
<td>消费 Kafka 全字段事件，写入文件型明细库，并提供历史查询和 CSV 导出边界；导出支持通过 <code>fields</code> 参数选择内部字段并平铺为 CSV 列，也支持 <code>metadata.xxx</code> 导出协议侧诊断、媒体归档、报警类型等 metadata 字段；未知 Envelope source 会进入 UNKNOWN 协议分区，并在 metadata 保留 <code>originalSource</code> 便于回溯；生产消费可使用 <code>tryIngest</code> 和自动装配的 <code>EnvelopeConsumerProcessor</code> 将坏 protobuf、缺快照和存储异常收敛为结构化结果，并通过 <code>EnvelopeDeadLetterSink</code> 发布死信，避免阻塞 Kafka 分区。</td>
|
||||
<td>Kafka Protobuf Envelope。</td>
|
||||
<td>EventFileRecord、查询结果、可配置字段 CSV。</td>
|
||||
<td><span class="tag green">历史明细</span><span class="tag green">多日导出</span><span class="tag">字段配置</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/services/vehicle-state-service</td>
|
||||
<td>消费 Kafka 全字段事件，更新车辆最新状态、位置、安全和最后事件到 Redis；生产消费可使用 <code>tryIngest</code> 和自动装配的 <code>EnvelopeConsumerProcessor</code> 隔离坏 protobuf 和缺快照 Envelope，并把失败记录交给死信出口。</td>
|
||||
<td>Kafka Protobuf Envelope。</td>
|
||||
<td><code>vehicle:state:{vin}</code>、<code>vehicle:location:{vin}</code>、<code>vehicle:safety:{vin}</code>。</td>
|
||||
<td><span class="tag green">毫秒级热查询</span><span class="tag red">氢泄露安全</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/services/vehicle-stat-service</td>
|
||||
<td>消费 Kafka 全字段事件，抽取统计源点，按车辆规则计算每日里程等日统计；生产消费可使用 <code>tryIngest</code> 和自动装配的 <code>EnvelopeConsumerProcessor</code> 隔离坏 protobuf，缺少 telemetry_snapshot 的消息会标记为 SKIPPED 并进入死信出口。</td>
|
||||
<td>Kafka Protobuf Envelope、车辆统计规则。</td>
|
||||
<td>按 <code>vin + stat_date</code> 保存的日统计结果。</td>
|
||||
<td><span class="tag green">每日里程</span><span class="tag">规则配置</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/apps/command-gateway</td>
|
||||
<td>HTTP 到设备下行命令入口。支持会话查询、808 位置查询、参数查询/设置、终端控制、平台通用应答、终端属性查询 0x8107、区域删除 0x8601、人工报警确认 0x8203，以及 1078 音视频属性查询、实时预览/控制、历史回放/控制、资源列表查询、文件上传/控制；通过 CommandDispatcher 复用 protocol-jt808 的在线通道、流水号和同步应答等待能力。</td>
|
||||
<td>业务系统命令请求。</td>
|
||||
<td>CommandDispatcher 调用、设备应答摘要。</td>
|
||||
<td><span class="tag">远程控制</span><span class="tag">参数设置</span><span class="tag">属性查询</span><span class="tag">区域删除</span><span class="tag">报警确认</span><span class="tag">视频控制</span></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>modules/apps/bootstrap-all</td>
|
||||
<td>一体化启动模块，聚合各模块 AutoConfiguration 和运行配置。</td>
|
||||
<td>application.yml、Spring Boot 启动。</td>
|
||||
<td>完整接入服务实例。</td>
|
||||
<td><span class="tag green">部署</span><span class="tag green">开关配置</span></td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
</section>
|
||||
|
||||
<section>
|
||||
<h2>六、氢能业务开发落点</h2>
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th>业务主题</th>
|
||||
<th>当前建议落点</th>
|
||||
<th>说明</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<td>车辆状态、速度、位置</td>
|
||||
<td><code>RealtimePayload</code>、<code>LocationPayload</code></td>
|
||||
<td>32960 和 808 都可能提供位置。统计侧需要明确优先级，例如优先 32960，缺失时用 808 补点。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>每日里程</td>
|
||||
<td>Kafka 下游统计服务</td>
|
||||
<td>接入层只投递总里程和实时点位；日增里程建议下游按 VIN、自然日、事件时间聚合，处理回补和乱序。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>每日用电量</td>
|
||||
<td>内部电池字段 + 下游统计</td>
|
||||
<td>接入层保留 SOC、电压、电流等瞬时字段；若有累计电耗字段，可加入内部字段模型后统一投递。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>每日用氢量</td>
|
||||
<td><code>hydrogenRemainingKg</code> + 下游统计</td>
|
||||
<td>优先使用累计氢耗字段；没有累计值时用氢余量差值估算，并在统计结果里标记估算口径。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>储氢安全</td>
|
||||
<td><code>RealtimePayload</code> 压力/温度 + <code>AlarmPayload.safetyCategory</code></td>
|
||||
<td>高压、低压、温度、异常 bit 都应统一归入储罐安全域，下游可做趋势、阈值和连续异常判断。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>氢气泄露</td>
|
||||
<td><code>AlarmPayload.hydrogenLeakDetected</code></td>
|
||||
<td>当前已作为高优先级安全事件。32960 中出现 <code>HYDROGEN_LEAK</code> 时强制映射为 <code>CRITICAL</code>。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>原始报文追溯</td>
|
||||
<td><code>RawArchive</code> + <code>rawArchiveUri</code> + <code>sink-archive</code></td>
|
||||
<td>业务统计出现争议时，用事件 metadata 中的 <code>archive://...</code> 逻辑 URI 回查原始 bytes，结合 eventId、traceId、VIN、时间复现解析链路。</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td>明细展示和导出</td>
|
||||
<td><code>event-file-store</code></td>
|
||||
<td>解析后的事件明细和 RawArchive 索引按 protocol/date 写 Parquet。DuckDB 查询时按日期范围读取，并按 event_time 正序或倒序返回。</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
<div class="callout">
|
||||
<strong>后续开发建议：</strong>
|
||||
当前 Kafka Protobuf 已加入全字段 <code>TelemetrySnapshot</code>。后续业务统计服务继续只依赖内部字段，不依赖 32960 原始字段名，这样接入 808、MQTT、车企私有协议时不会重写统计口径。
|
||||
</div>
|
||||
</section>
|
||||
</main>
|
||||
|
||||
<script type="module">
|
||||
import mermaid from "https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.esm.min.mjs";
|
||||
|
||||
mermaid.initialize({
|
||||
startOnLoad: true,
|
||||
theme: "base",
|
||||
securityLevel: "strict",
|
||||
flowchart: {
|
||||
htmlLabels: true,
|
||||
curve: "basis"
|
||||
},
|
||||
themeVariables: {
|
||||
fontFamily: "-apple-system, BlinkMacSystemFont, Segoe UI, PingFang SC, Microsoft YaHei, sans-serif",
|
||||
primaryColor: "#eff6ff",
|
||||
primaryTextColor: "#172033",
|
||||
primaryBorderColor: "#2563eb",
|
||||
lineColor: "#667085",
|
||||
secondaryColor: "#ecfdf3",
|
||||
tertiaryColor: "#fffbeb"
|
||||
}
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,600 @@
|
||||
# GB32960 Body Parser — Per-Block Exception Isolation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** `Gb32960BodyParser` 主循环对单信息块的解析异常实现**单块隔离**：一块解析失败不再整帧放弃，改为把失败块兜成 `InfoBlock.Raw` 并继续解析后续信息块。
|
||||
|
||||
**Architecture:** 在主循环每次 `parser.parse(body)` 外面套 try/catch（仅捕获 `DecodeException` / `BufferUnderflowException` / `IndexOutOfBoundsException`，放行 RuntimeException 以免掩盖 parser bug）。失败时：
|
||||
- 若 `fixedLen ≥ 0` 且剩余字节 ≥ fixedLen → 回滚 reader → 按 fixedLen 截取 Raw → position 前进 fixedLen → `continue` 循环
|
||||
- 否则（变长 parser 或剩余不足）→ 剩余字节全部兜成 Raw → `break` 循环（变长块无法安全找下一块边界）
|
||||
|
||||
通过 `Gb32960Properties.Parse.lenientBlockFailure`（默认 true）控制开关，可以回退到严格模式。**不改 `InfoBlock.Raw` record 结构**（避免 downstream 兼容风险——`Gb32960EventMapper` 按 `findBlock(Class)` 类型查找，新增 Raw 不影响其行为）。
|
||||
|
||||
**Tech Stack:** Java 25, JUnit 5, AssertJ, Spring Boot ConfigurationProperties, Maven。仅改动 `protocol-gb32960` 模块。
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
- **Modify** `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960Properties.java` — 新增 `Parse` 内嵌类 + `parse` 字段/getter/setter
|
||||
- **Modify** `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParser.java` — 主循环 try/catch + recovery 逻辑 + `lenientBlockFailure` 字段
|
||||
- **Modify** `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfiguration.java` — 把配置穿给 BodyParser
|
||||
- **Create** `protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParserIsolationTest.java` — 3 个隔离场景 + 1 个严格模式回退场景
|
||||
- **Modify** `bootstrap-all/src/main/resources/application.yml` — 添加 `parse:` 注释段示例
|
||||
- **Modify** `CHANGELOG.md` — 追加本次变更条目
|
||||
|
||||
---
|
||||
|
||||
## Task 1: 基线验证 —— 现有测试全绿
|
||||
|
||||
**Files:** 无
|
||||
|
||||
- [ ] **Step 1.1: 跑 protocol-gb32960 模块测试**
|
||||
|
||||
Run: `mvn -pl protocol-gb32960 test -q`
|
||||
Expected: BUILD SUCCESS，所有现有测试通过（`Gb32960DecoderGoldenTest`、`Gb32960FullBlocksTest`、`Gb32960DecoderTest`、`GuangdongFcEndToEndTest`、parser/* 各 Block Parser 单测、profile/* 选择器单测）。
|
||||
|
||||
如果基线红了，**先停下来修复基线**再进入 Task 2。
|
||||
|
||||
---
|
||||
|
||||
## Task 2: 添加 Parse 配置子节点
|
||||
|
||||
**Files:**
|
||||
- Modify: `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960Properties.java`
|
||||
|
||||
- [ ] **Step 2.1: 在 `Gb32960Properties` 类体中（在 `Auth` 字段后、`Tls` 字段前）新增 `parse` 字段**
|
||||
|
||||
位置参考：现在 L24 `private Auth auth = new Auth();` 下面一行，在 L27 `private Tls tls = new Tls();` 之前插入：
|
||||
|
||||
```java
|
||||
/**
|
||||
* 报文解析行为配置。
|
||||
*
|
||||
* <p>默认 {@code lenientBlockFailure=true}：单个信息块解析异常时兜成
|
||||
* {@link com.lingniu.ingest.protocol.gb32960.model.InfoBlock.Raw} 继续解析，
|
||||
* 不再整帧放弃。关闭后恢复旧行为（任意异常直接抛 {@code DecodeException}
|
||||
* 放弃整帧），仅在灰度回滚时使用。
|
||||
*/
|
||||
private Parse parse = new Parse();
|
||||
```
|
||||
|
||||
- [ ] **Step 2.2: 在类底部（`Tls` 静态类之后、`VendorExtension` 静态类之前）添加 `Parse` 静态内嵌类**
|
||||
|
||||
```java
|
||||
/**
|
||||
* 报文解析容错策略。
|
||||
*/
|
||||
public static class Parse {
|
||||
/**
|
||||
* 单块解析异常时是否兜底为 {@link com.lingniu.ingest.protocol.gb32960.model.InfoBlock.Raw}
|
||||
* 继续解析。默认开启。
|
||||
*
|
||||
* <ul>
|
||||
* <li>{@code true}（默认）：parser 抛 DecodeException / BufferUnderflowException /
|
||||
* IndexOutOfBoundsException 时，固定长度块按 fixedLen 截取 Raw 后 continue；
|
||||
* 变长块或剩余字节不足时，剩余全部兜成 Raw 后 break 循环。
|
||||
* <li>{@code false}：保留旧行为，任意异常直接抛出 DecodeException 放弃整帧。
|
||||
* </ul>
|
||||
*/
|
||||
private boolean lenientBlockFailure = true;
|
||||
|
||||
public boolean isLenientBlockFailure() { return lenientBlockFailure; }
|
||||
public void setLenientBlockFailure(boolean lenientBlockFailure) {
|
||||
this.lenientBlockFailure = lenientBlockFailure;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2.3: 在属性 getter/setter 区域（现有 `getTls` 之后）补充 `getParse` / `setParse`**
|
||||
|
||||
```java
|
||||
public Parse getParse() { return parse; }
|
||||
public void setParse(Parse parse) { this.parse = parse; }
|
||||
```
|
||||
|
||||
- [ ] **Step 2.4: 编译验证**
|
||||
|
||||
Run: `mvn -pl protocol-gb32960 compile -q`
|
||||
Expected: BUILD SUCCESS。
|
||||
|
||||
- [ ] **Step 2.5: 提交**
|
||||
|
||||
```bash
|
||||
git add protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960Properties.java
|
||||
git commit -m "$(cat <<'EOF'
|
||||
config(gb32960): add parse.lenientBlockFailure toggle
|
||||
|
||||
新增 lingniu.ingest.gb32960.parse.lenientBlockFailure 配置开关（默认 true），
|
||||
为 Gb32960BodyParser 的单块异常隔离行为做回退开关。实现在后续 commit。
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: RED —— 写三个失败测试覆盖隔离场景
|
||||
|
||||
**Files:**
|
||||
- Create: `protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParserIsolationTest.java`
|
||||
|
||||
- [ ] **Step 3.1: 创建测试文件**
|
||||
|
||||
完整内容：
|
||||
|
||||
```java
|
||||
package com.lingniu.ingest.protocol.gb32960.codec;
|
||||
|
||||
import com.lingniu.ingest.api.spi.DecodeException;
|
||||
import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.AlarmV2016BlockParser;
|
||||
import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.PositionV2016BlockParser;
|
||||
import com.lingniu.ingest.protocol.gb32960.codec.parser.v2016.VehicleV2016BlockParser;
|
||||
import com.lingniu.ingest.protocol.gb32960.model.InfoBlock;
|
||||
import com.lingniu.ingest.protocol.gb32960.model.InfoBlockType;
|
||||
import com.lingniu.ingest.protocol.gb32960.model.ProtocolVersion;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.nio.ByteBuffer;
|
||||
import java.util.List;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
/**
|
||||
* 验证 {@link Gb32960BodyParser} 单块异常隔离行为。
|
||||
*
|
||||
* <p>三个隔离场景：
|
||||
* <ol>
|
||||
* <li>固定长度块 parser 抛异常 → 失败块兜 Raw（按 fixedLen 截取）+ 后续块继续解析
|
||||
* <li>固定长度块剩余字节不足（截断帧） → 兜 Raw(剩余) + break 循环
|
||||
* <li>变长块 parser 抛异常 → 兜 Raw(从失败块起剩余全部) + break 循环
|
||||
* </ol>
|
||||
*
|
||||
* <p>外加一个严格模式回退测试：{@code lenientBlockFailure=false} 时任意异常应抛 DecodeException。
|
||||
*/
|
||||
class Gb32960BodyParserIsolationTest {
|
||||
|
||||
/** 构造一个"看起来合法但中途 parser 失败"用来注入失败的 Position parser。 */
|
||||
private static final InfoBlockParser EXPLODING_FIXED_LEN_POSITION = new InfoBlockParser() {
|
||||
@Override public ProtocolVersion version() { return ProtocolVersion.V2016; }
|
||||
@Override public int typeCode() { return 0x05; }
|
||||
@Override public int fixedLength() { return 9; }
|
||||
@Override public InfoBlock parse(ByteBuffer buffer) {
|
||||
// 模拟 parser 读了几字节后发现不合法就抛
|
||||
buffer.get(); buffer.get();
|
||||
throw new DecodeException("simulated parser failure in Position");
|
||||
}
|
||||
};
|
||||
|
||||
/** 变长块（fixedLength=-1），parse 时消费若干字节后抛。模拟 Alarm/Voltage 类列表读越界。 */
|
||||
private static final InfoBlockParser EXPLODING_VAR_LEN_ALARM = new InfoBlockParser() {
|
||||
@Override public ProtocolVersion version() { return ProtocolVersion.V2016; }
|
||||
@Override public int typeCode() { return 0x07; }
|
||||
@Override public int fixedLength() { return -1; }
|
||||
@Override public InfoBlock parse(ByteBuffer buffer) {
|
||||
// 假装读了个长度字段就爆
|
||||
buffer.get();
|
||||
throw new DecodeException("simulated parser failure in Alarm list-length read");
|
||||
}
|
||||
};
|
||||
|
||||
@Test
|
||||
void fixedLengthBlockFailure_isIsolated_subsequentBlocksStillParsed() {
|
||||
// 帧布局：Vehicle(0x01, 20B) + Position(0x05, 9B 但 parser 爆) + Engine(0x04) 不构造，
|
||||
// 简化为 Vehicle + FailingPosition + Vehicle 再次，验证"Position 之后还能继续"。
|
||||
InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of(
|
||||
new VehicleV2016BlockParser(),
|
||||
EXPLODING_FIXED_LEN_POSITION));
|
||||
|
||||
Gb32960BodyParser parser = new Gb32960BodyParser(registry);
|
||||
|
||||
ByteArrayOutputStream os = new ByteArrayOutputStream();
|
||||
writeValidVehicle(os); // 1+20 = 21B
|
||||
writePositionTypeAnd9ByteBody(os); // 1+9 = 10B (parser 会爆)
|
||||
writeValidVehicle(os); // 1+20 = 21B
|
||||
ByteBuffer body = ByteBuffer.wrap(os.toByteArray());
|
||||
|
||||
var result = parser.parse(ProtocolVersion.V2016, body);
|
||||
|
||||
assertThat(result.blocks()).hasSize(3);
|
||||
assertThat(result.blocks().get(0)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class);
|
||||
assertThat(result.blocks().get(1)).isInstanceOfSatisfying(InfoBlock.Raw.class, raw -> {
|
||||
assertThat(raw.typeCode()).isEqualTo(0x05);
|
||||
assertThat(raw.type()).isEqualTo(InfoBlockType.RAW);
|
||||
assertThat(raw.bytes()).hasSize(9); // 按 fixedLen 截取
|
||||
});
|
||||
assertThat(result.blocks().get(2)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class);
|
||||
assertThat(body.hasRemaining()).isFalse();
|
||||
}
|
||||
|
||||
@Test
|
||||
void truncatedFixedLengthBlock_isWrappedAsRaw_loopTerminates() {
|
||||
// Vehicle(21B) + Position(type 0x05)但 body 只给 3B —— 不足 9B
|
||||
InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of(
|
||||
new VehicleV2016BlockParser(),
|
||||
new PositionV2016BlockParser()));
|
||||
|
||||
Gb32960BodyParser parser = new Gb32960BodyParser(registry);
|
||||
|
||||
ByteArrayOutputStream os = new ByteArrayOutputStream();
|
||||
writeValidVehicle(os);
|
||||
os.write(0x05); // Position typeCode
|
||||
os.write(0); os.write(0); os.write(0); // 只 3B，远远不够 9B
|
||||
ByteBuffer body = ByteBuffer.wrap(os.toByteArray());
|
||||
|
||||
var result = parser.parse(ProtocolVersion.V2016, body);
|
||||
|
||||
assertThat(result.blocks()).hasSize(2);
|
||||
assertThat(result.blocks().get(0)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class);
|
||||
assertThat(result.blocks().get(1)).isInstanceOfSatisfying(InfoBlock.Raw.class, raw -> {
|
||||
assertThat(raw.typeCode()).isEqualTo(0x05);
|
||||
assertThat(raw.bytes()).hasSize(3); // 剩余字节全兜成 Raw
|
||||
});
|
||||
}
|
||||
|
||||
@Test
|
||||
void variableLengthBlockFailure_swallowsRemainderAsRaw_loopBreaks() {
|
||||
// Vehicle(21B) + FailingAlarm(0x07)后面还有一个 Vehicle，但因 Alarm 变长无法安全跳过，
|
||||
// 整段 Alarm 起的剩余字节都被兜成 Raw 后 break。
|
||||
InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of(
|
||||
new VehicleV2016BlockParser(),
|
||||
EXPLODING_VAR_LEN_ALARM));
|
||||
|
||||
Gb32960BodyParser parser = new Gb32960BodyParser(registry);
|
||||
|
||||
ByteArrayOutputStream os = new ByteArrayOutputStream();
|
||||
writeValidVehicle(os); // 21B
|
||||
os.write(0x07); // Alarm typeCode
|
||||
for (int i = 0; i < 10; i++) os.write(0xAA); // 10B 的 alarm body（parser 爆）
|
||||
writeValidVehicle(os); // 21B（应当不再被解析）
|
||||
ByteBuffer body = ByteBuffer.wrap(os.toByteArray());
|
||||
|
||||
var result = parser.parse(ProtocolVersion.V2016, body);
|
||||
|
||||
assertThat(result.blocks()).hasSize(2);
|
||||
assertThat(result.blocks().get(0)).isInstanceOf(InfoBlock.Gb32960V2016.Vehicle.class);
|
||||
assertThat(result.blocks().get(1)).isInstanceOfSatisfying(InfoBlock.Raw.class, raw -> {
|
||||
assertThat(raw.typeCode()).isEqualTo(0x07);
|
||||
// 剩余 10B alarm body + 1+20=21B 后续 Vehicle = 31B
|
||||
assertThat(raw.bytes()).hasSize(31);
|
||||
});
|
||||
}
|
||||
|
||||
@Test
|
||||
void strictMode_throwsOnAnyBlockFailure() {
|
||||
InfoBlockParserRegistry registry = new InfoBlockParserRegistry(List.of(
|
||||
new VehicleV2016BlockParser(),
|
||||
EXPLODING_FIXED_LEN_POSITION));
|
||||
|
||||
Gb32960BodyParser parser = new Gb32960BodyParser(registry);
|
||||
parser.setLenientBlockFailure(false);
|
||||
|
||||
ByteArrayOutputStream os = new ByteArrayOutputStream();
|
||||
writeValidVehicle(os);
|
||||
writePositionTypeAnd9ByteBody(os);
|
||||
ByteBuffer body = ByteBuffer.wrap(os.toByteArray());
|
||||
|
||||
assertThatThrownBy(() -> parser.parse(ProtocolVersion.V2016, body))
|
||||
.isInstanceOf(DecodeException.class);
|
||||
}
|
||||
|
||||
// ------------------------------------------------------------------------
|
||||
// helpers
|
||||
// ------------------------------------------------------------------------
|
||||
|
||||
/** 写一个完整的合法 V2016 Vehicle 块（typeCode + 20B body）。 */
|
||||
private static void writeValidVehicle(ByteArrayOutputStream os) {
|
||||
os.write(0x01); // typeCode Vehicle
|
||||
os.write(0x01); // vehicleState=1
|
||||
os.write(0x01); // chargingState=1
|
||||
os.write(0x01); // runningMode=1
|
||||
os.write(0); os.write(0); // speed=0
|
||||
os.write(0); os.write(0); os.write(0); os.write(0); // mileage=0
|
||||
os.write(0); os.write(0); // totalVoltage=0
|
||||
os.write(0); os.write(0); // totalCurrent=0（offset 1000，原值0）
|
||||
os.write(50); // soc=50%
|
||||
os.write(0x01); // dcdc
|
||||
os.write(0); // gear
|
||||
os.write(0); os.write(0); // insulation
|
||||
os.write(0); // accelerator
|
||||
os.write(0); // brake
|
||||
}
|
||||
|
||||
/** 写 Position typeCode(0x05) + 9 字节 body（内容不重要，parser 替换成 EXPLODING 的）。 */
|
||||
private static void writePositionTypeAnd9ByteBody(ByteArrayOutputStream os) {
|
||||
os.write(0x05);
|
||||
for (int i = 0; i < 9; i++) os.write(0);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 3.2: 跑测试验证 RED**
|
||||
|
||||
Run: `mvn -pl protocol-gb32960 test -Dtest=Gb32960BodyParserIsolationTest -q`
|
||||
Expected:
|
||||
- `fixedLengthBlockFailure_isIsolated_subsequentBlocksStillParsed` FAIL（当前会抛 DecodeException）
|
||||
- `truncatedFixedLengthBlock_isWrappedAsRaw_loopTerminates` FAIL（当前 L132 会抛 DecodeException）
|
||||
- `variableLengthBlockFailure_swallowsRemainderAsRaw_loopBreaks` FAIL（异常冒出整帧失败）
|
||||
- `strictMode_throwsOnAnyBlockFailure` FAIL（没有 `setLenientBlockFailure` 方法，编译就红）
|
||||
|
||||
**验证点**：编译失败（strictMode test 里 `setLenientBlockFailure` 尚未存在）是**预期** RED 信号——下一步实现。
|
||||
|
||||
---
|
||||
|
||||
## Task 4: GREEN —— 实现单块异常隔离
|
||||
|
||||
**Files:**
|
||||
- Modify: `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParser.java`
|
||||
|
||||
- [ ] **Step 4.1: 在类字段区新增 `lenientBlockFailure` 字段 + setter**
|
||||
|
||||
在 `private final VendorExtensionSelector selector;`（现 L54）下面插入：
|
||||
|
||||
```java
|
||||
/**
|
||||
* 单块解析异常时是否兜底为 Raw 后继续。由
|
||||
* {@link com.lingniu.ingest.protocol.gb32960.config.Gb32960Properties.Parse#isLenientBlockFailure()}
|
||||
* 注入；默认 true。
|
||||
*/
|
||||
private boolean lenientBlockFailure = true;
|
||||
|
||||
public void setLenientBlockFailure(boolean lenientBlockFailure) {
|
||||
this.lenientBlockFailure = lenientBlockFailure;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 4.2: 改造主循环：替换 L130~L150 的整个 "parse + 长度校验" 段**
|
||||
|
||||
**完整替换块**：以下代码替换原来从 `int fixedLen = parser.fixedLength();`（L130）开始到 `blocks.add(block);`（L150）结束的整块。
|
||||
|
||||
```java
|
||||
int fixedLen = parser.fixedLength();
|
||||
int posBefore = body.position();
|
||||
|
||||
// 旧行为：fixedLen 预检不足 → 直接抛。新行为（lenient 模式）：走统一恢复路径。
|
||||
if (fixedLen >= 0 && body.remaining() < fixedLen) {
|
||||
if (!lenientBlockFailure) {
|
||||
throw new DecodeException(
|
||||
"info block 0x" + Integer.toHexString(typeCode)
|
||||
+ " needs " + fixedLen + " bytes but got " + body.remaining());
|
||||
}
|
||||
// 剩余字节数不足 fixedLen —— 无法按块截取，整尾兜 Raw + break
|
||||
int remaining = body.remaining();
|
||||
byte[] tail = new byte[remaining];
|
||||
body.get(tail);
|
||||
log.warn("[gb32960] truncated block typeCode=0x{} declaredFixedLen={} remaining={} — wrapping tail as Raw",
|
||||
Integer.toHexString(typeCode), fixedLen, remaining);
|
||||
blocks.add(new InfoBlock.Raw(typeCode, InfoBlockType.RAW, tail));
|
||||
break;
|
||||
}
|
||||
|
||||
InfoBlock block;
|
||||
try {
|
||||
block = parser.parse(body);
|
||||
} catch (DecodeException | java.nio.BufferUnderflowException | IndexOutOfBoundsException e) {
|
||||
if (!lenientBlockFailure) {
|
||||
if (e instanceof DecodeException de) throw de;
|
||||
throw new DecodeException(
|
||||
"parser " + parser.getClass().getSimpleName() + " failed: " + e.getMessage(), e);
|
||||
}
|
||||
// 恢复路径：先回滚 position 到 parse 入口
|
||||
body.position(posBefore);
|
||||
if (fixedLen >= 0) {
|
||||
// 固定长度块：按 fixedLen 截取 → 兜 Raw → continue 循环
|
||||
byte[] corrupt = new byte[fixedLen];
|
||||
body.get(corrupt);
|
||||
log.warn("[gb32960] block parse failed typeCode=0x{} parser={} fixedLen={} — isolated as Raw, continuing",
|
||||
Integer.toHexString(typeCode), parser.getClass().getSimpleName(), fixedLen, e);
|
||||
blocks.add(new InfoBlock.Raw(typeCode, InfoBlockType.RAW, corrupt));
|
||||
continue;
|
||||
} else {
|
||||
// 变长块：无法安全找下一块边界 → 剩余整段兜 Raw + break
|
||||
int remaining = body.remaining();
|
||||
byte[] tail = new byte[remaining];
|
||||
body.get(tail);
|
||||
log.warn("[gb32960] variable-length block parse failed typeCode=0x{} parser={} remaining={} — wrapping remainder as Raw, stopping loop",
|
||||
Integer.toHexString(typeCode), parser.getClass().getSimpleName(), remaining, e);
|
||||
blocks.add(new InfoBlock.Raw(typeCode, InfoBlockType.RAW, tail));
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
int consumed = body.position() - posBefore;
|
||||
if (fixedLen >= 0 && consumed != fixedLen) {
|
||||
// 这是 parser 契约违反（声明 fixedLen=X 但实际读了 Y），保持抛异常以暴露 parser bug。
|
||||
throw new DecodeException(
|
||||
"parser " + parser.getClass().getSimpleName()
|
||||
+ " consumed " + consumed
|
||||
+ " bytes but declared " + fixedLen);
|
||||
}
|
||||
if (log.isDebugEnabled()) {
|
||||
log.debug("[gb32960] block typeCode=0x{} parser={} pos {}→{} consumed={} declaredFixed={}",
|
||||
Integer.toHexString(typeCode), parser.getClass().getSimpleName(),
|
||||
typeStartPos, body.position(), consumed, fixedLen);
|
||||
}
|
||||
blocks.add(block);
|
||||
```
|
||||
|
||||
- [ ] **Step 4.3: 运行隔离测试验证 GREEN**
|
||||
|
||||
Run: `mvn -pl protocol-gb32960 test -Dtest=Gb32960BodyParserIsolationTest -q`
|
||||
Expected: 4 个测试全部 PASS。
|
||||
|
||||
- [ ] **Step 4.4: 运行全模块测试检查回归**
|
||||
|
||||
Run: `mvn -pl protocol-gb32960 test -q`
|
||||
Expected: BUILD SUCCESS，所有原有测试（含 Golden、FullBlocks、GuangdongFcEndToEnd）依旧通过。
|
||||
|
||||
如果有原有测试红了：**停下来诊断**。最可能的原因是某个现有测试之前靠"抛异常"行为验证错误帧的，需要手动判断这是测试问题还是实现问题。
|
||||
|
||||
- [ ] **Step 4.5: 提交**
|
||||
|
||||
```bash
|
||||
git add protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParser.java \
|
||||
protocol-gb32960/src/test/java/com/lingniu/ingest/protocol/gb32960/codec/Gb32960BodyParserIsolationTest.java
|
||||
git commit -m "$(cat <<'EOF'
|
||||
feat(gb32960): isolate per-block parse failures in body parser
|
||||
|
||||
主循环改造：单信息块 parser 抛 DecodeException / BufferUnderflowException /
|
||||
IndexOutOfBoundsException 时不再放弃整帧。
|
||||
|
||||
- 固定长度块（fixedLen≥0）：回滚 reader，按 fixedLen 截取字节兜成 InfoBlock.Raw，
|
||||
position 前进 fixedLen，continue 循环继续解析后续块
|
||||
- 变长块（fixedLen=-1）或剩余不足 fixedLen：剩余字节全部兜成 Raw 后 break
|
||||
- parser 契约违反（consumed != declared fixedLen）仍抛异常，以暴露 parser bug
|
||||
- 通过 lingniu.ingest.gb32960.parse.lenientBlockFailure=false 可回退严格模式
|
||||
- 新增 Gb32960BodyParserIsolationTest 覆盖 3 种失败场景 + 1 严格模式
|
||||
|
||||
不改 InfoBlock.Raw record 结构；下游 Gb32960EventMapper 按 findBlock(Class) 类型
|
||||
查找，新增 Raw 不影响其行为。
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: 把配置注入 BodyParser（AutoConfiguration 连线）
|
||||
|
||||
**Files:**
|
||||
- Modify: `protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfiguration.java`
|
||||
|
||||
- [ ] **Step 5.1: 定位 `Gb32960BodyParser` bean 定义**
|
||||
|
||||
先查看：`grep -n "Gb32960BodyParser" protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfiguration.java`
|
||||
|
||||
根据返回的行号，在构造 `Gb32960BodyParser` 的 @Bean 方法里，构造完成后立即调用：
|
||||
|
||||
```java
|
||||
Gb32960BodyParser parser = new Gb32960BodyParser(profileRegistry, selector);
|
||||
parser.setLenientBlockFailure(properties.getParse().isLenientBlockFailure());
|
||||
return parser;
|
||||
```
|
||||
|
||||
（如果目前的构造返回一行表达式，改成先赋给 local 变量再设 flag 再 return。）
|
||||
|
||||
- [ ] **Step 5.2: 编译并运行全模块测试**
|
||||
|
||||
Run: `mvn -pl protocol-gb32960 test -q`
|
||||
Expected: BUILD SUCCESS。
|
||||
|
||||
- [ ] **Step 5.3: 提交**
|
||||
|
||||
```bash
|
||||
git add protocol-gb32960/src/main/java/com/lingniu/ingest/protocol/gb32960/config/Gb32960AutoConfiguration.java
|
||||
git commit -m "$(cat <<'EOF'
|
||||
config(gb32960): wire parse.lenientBlockFailure into BodyParser bean
|
||||
|
||||
Gb32960AutoConfiguration 在创建 Gb32960BodyParser bean 后，将
|
||||
Gb32960Properties.Parse.lenientBlockFailure 通过 setter 注入，让运行期配置生效。
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: bootstrap-all 配置文档化
|
||||
|
||||
**Files:**
|
||||
- Modify: `bootstrap-all/src/main/resources/application.yml`
|
||||
|
||||
- [ ] **Step 6.1: 在 `lingniu.ingest.gb32960` 节点的 `vendor-extensions` 段之后、`jt808` 之前，插入 parse 配置示例**
|
||||
|
||||
参考位置：现有 application.yml L57 末尾（`vendor-extensions` list 结束）。在 L58 `jt808:` 之前加入：
|
||||
|
||||
```yaml
|
||||
# 报文解析容错。默认启用单块异常隔离：某个信息块解析失败时兜成 Raw 后继续，
|
||||
# 不再整帧丢弃。仅在需要严格失败语义（灰度回滚或抓虫）时设为 false。
|
||||
parse:
|
||||
lenient-block-failure: true
|
||||
```
|
||||
|
||||
- [ ] **Step 6.2: 启动一次 bootstrap-all 的 dry-run 编译**
|
||||
|
||||
Run: `mvn -pl bootstrap-all compile -q`
|
||||
Expected: BUILD SUCCESS（只是配置段注释变化，不会影响编译）。
|
||||
|
||||
- [ ] **Step 6.3: 提交**
|
||||
|
||||
```bash
|
||||
git add bootstrap-all/src/main/resources/application.yml
|
||||
git commit -m "$(cat <<'EOF'
|
||||
config(gb32960): document parse.lenient-block-failure in application.yml
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: 全仓库回归测试
|
||||
|
||||
**Files:** 无（只是验证）
|
||||
|
||||
- [ ] **Step 7.1: 跑完整项目测试**
|
||||
|
||||
Run: `mvn test -q`
|
||||
Expected: BUILD SUCCESS。重点关注：
|
||||
- `protocol-gb32960` 模块全绿（Golden、FullBlocks、Isolation、GuangdongFcEndToEnd、Decoder、Mapper、所有 parser 单测、profile）
|
||||
- 其它模块（`ingest-core`、`sink-mq` 等）不受影响（无改动应自动通过）
|
||||
|
||||
若有红，停下来诊断。
|
||||
|
||||
---
|
||||
|
||||
## Task 8: 更新 CHANGELOG
|
||||
|
||||
**Files:**
|
||||
- Modify: `CHANGELOG.md`
|
||||
|
||||
- [ ] **Step 8.1: 在 CHANGELOG.md 文件顶部（在 `## [0.1.0] — 2026-04-15` 标题之前）新增一个 `## [Unreleased]` 段落；如已存在则追加**
|
||||
|
||||
新增段落内容（如已存在 Unreleased 段则在其 `### Added` / `### Changed` 内部追加）：
|
||||
|
||||
```markdown
|
||||
## [Unreleased]
|
||||
|
||||
### Changed —— GB/T 32960 Body Parser 单块异常隔离
|
||||
- `Gb32960BodyParser` 主循环对单信息块的 parser 异常不再放弃整帧：
|
||||
- 固定长度块（`fixedLength ≥ 0`）：回滚 reader → 按 fixedLen 截取字节兜成
|
||||
`InfoBlock.Raw` → 继续解析后续块；
|
||||
- 变长块或剩余字节不足：剩余字节全部兜成 `Raw` 后终止循环；
|
||||
- `parser` 契约违反（声明 fixedLen=X 但实际读了 Y）仍抛 `DecodeException`，
|
||||
以暴露 parser bug，不被静默吞掉；
|
||||
- 只捕获 `DecodeException` / `BufferUnderflowException` / `IndexOutOfBoundsException`，
|
||||
`RuntimeException` 继续向上抛，保留 bug 可见性。
|
||||
- 新增配置 `lingniu.ingest.gb32960.parse.lenient-block-failure`（默认 `true`）；
|
||||
回退严格模式用 `false`。
|
||||
- `InfoBlock.Raw` record 结构**未变**，下游 `Gb32960EventMapper` 按 `findBlock(Class)`
|
||||
类型匹配，新增 Raw 不影响业务事件映射。
|
||||
- 覆盖测试：`Gb32960BodyParserIsolationTest` —— 3 种隔离场景 + 严格模式回退。
|
||||
```
|
||||
|
||||
- [ ] **Step 8.2: 提交**
|
||||
|
||||
```bash
|
||||
git add CHANGELOG.md
|
||||
git commit -m "$(cat <<'EOF'
|
||||
docs(changelog): record gb32960 body parser block isolation
|
||||
|
||||
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||||
EOF
|
||||
)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-Review Checklist
|
||||
|
||||
- [x] **Spec coverage**: 原讨论的 A1~A7 全部覆盖——A1 状态机（Task 4 Step 4.2）、A2 三类错误策略（同）、A3 收窄异常种类（同，只 catch 三种）、A4 日志分类（warn 带 parser 名/typeCode/原因）、A5 配置开关（Task 2）、A6 合成帧测试（Task 3 三场景 + 严格回退）、A7 下游兼容（不改 Raw record，`findBlock` 类型匹配不受影响，plan 中已说明）。
|
||||
- [x] **Placeholder scan**: 无 TBD / TODO / "实现上面的" 等占位；每个 Step 含实际代码 or 命令。
|
||||
- [x] **Type consistency**: `lenientBlockFailure` 在 Properties / BodyParser 字段 / setter / 测试中拼写一致。`InfoBlockType.RAW` 枚举值依赖当前存在（已核对 InfoBlock.Raw record）。
|
||||
- [x] **测试命名一致**：`Gb32960BodyParserIsolationTest` 在 Task 3 创建、Task 4/7 引用一致。
|
||||
- [x] **未引入新枚举/proto 变更**：InfoBlock.Raw 沿用现有 `(int typeCode, InfoBlockType type, byte[] bytes)` 构造。
|
||||
270
docs/target-architecture.md
Normal file
270
docs/target-architecture.md
Normal file
@@ -0,0 +1,270 @@
|
||||
# Target Architecture
|
||||
|
||||
This document defines the target architecture for Lingniu vehicle ingest. The
|
||||
main goal is to keep protocol ingest, raw archive, historical detail storage,
|
||||
realtime state, and statistics as separate responsibilities connected by stable
|
||||
event contracts.
|
||||
|
||||
## Goals
|
||||
|
||||
- The ingest service receives protocol messages, archives raw frames, and
|
||||
publishes normalized full-field events to Kafka.
|
||||
- Historical detail storage consumes Kafka events and stores raw parsed event
|
||||
details in Parquet, queried through DuckDB.
|
||||
- Realtime vehicle state consumes Kafka events and writes hot state to Redis.
|
||||
- Statistics consumes Kafka events and calculates daily operational metrics by
|
||||
configurable rules.
|
||||
- Kafka consumers expose non-throwing ingest results for bad protobuf payloads,
|
||||
unsupported envelopes, and downstream failures so one bad record does not
|
||||
block a vehicle partition.
|
||||
- Consumer services expose `EnvelopeConsumerProcessor` beans. Kafka workers or
|
||||
framework listeners should pass topic, partition, offset, key, and value into
|
||||
the processor; `SKIPPED`, `INVALID_ENVELOPE`, and `FAILED` results are routed
|
||||
through `EnvelopeDeadLetterSink`.
|
||||
- `sink-mq` owns Kafka-specific consumer plumbing: `KafkaEnvelopeConsumerFactory`
|
||||
creates one independent Kafka consumer per service processor binding,
|
||||
`KafkaEnvelopeConsumerRunner` manages the long-running poll loop, and
|
||||
`KafkaEnvelopeDeadLetterSink` publishes failed records to the configured DLQ
|
||||
topic with diagnostic headers.
|
||||
- Business consumers depend on internal fields, not protocol-specific field
|
||||
names.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- The ingest service must not write business tables.
|
||||
- The ingest service must not serve historical detail queries.
|
||||
- Parquet history storage must not store derived daily statistics.
|
||||
- Redis hot state must not become the long-term historical store.
|
||||
- Statistics services must not parse GB/T 32960 frames directly.
|
||||
|
||||
## Service Boundaries
|
||||
|
||||
### Ingest Service
|
||||
|
||||
The ingest service owns protocol IO and event publication.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- Accept GB/T 32960, JT/T 808, MQTT, Xinda Push, and future inbound protocols.
|
||||
- Decode frames and acknowledge protocol messages where required.
|
||||
- Emit `RawArchive` before deduplication and rate limiting when raw bytes are
|
||||
available.
|
||||
- Write raw bytes through `sink-archive`.
|
||||
- Map protocol messages into internal telemetry events.
|
||||
- Publish internal events to Kafka with VIN as the partition key.
|
||||
|
||||
The ingest service must not depend on Redis, DuckDB query endpoints, or
|
||||
statistics rule repositories.
|
||||
|
||||
### Event Contract
|
||||
|
||||
The event contract is the only dependency shared by ingest, history, realtime
|
||||
state, and statistics.
|
||||
|
||||
Core event identity:
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `event_id` | Idempotency key for the parsed event |
|
||||
| `trace_id` | Cross-service trace id |
|
||||
| `vin` | Vehicle VIN and Kafka partition key |
|
||||
| `source_protocol` | Protocol source such as `GB32960`; consumers may map unknown future source names to `UNKNOWN` and preserve the original source in metadata |
|
||||
| `protocol_version` | Source protocol version |
|
||||
| `event_type` | Business event type, for example `REALTIME`, `LOCATION`, `ALARM` |
|
||||
| `event_time` | Device collection time |
|
||||
| `ingest_time` | Platform receive time |
|
||||
| `raw_archive_uri` | Optional reference to archived raw bytes |
|
||||
|
||||
Full-field telemetry is represented as internal field values:
|
||||
|
||||
| Field | Meaning |
|
||||
|---|---|
|
||||
| `field_key` | Stable internal field key such as `total_mileage_km` |
|
||||
| `value_type` | `STRING`, `DOUBLE`, `LONG`, `BOOLEAN`, `INSTANT`, `JSON` |
|
||||
| `value` | String representation used for durable serialization |
|
||||
| `unit` | Stable unit when applicable |
|
||||
| `quality` | `GOOD`, `ESTIMATED`, `INVALID`, `MISSING` |
|
||||
| `source_path` | Protocol source path, for traceability |
|
||||
|
||||
The internal field list is defined in
|
||||
`docs/vehicle-telemetry-internal-fields.md`. New protocol-specific fields start
|
||||
as extension fields and are promoted only after a business meaning and unit are
|
||||
stable.
|
||||
|
||||
### Historical Detail Service
|
||||
|
||||
The historical detail service owns raw parsed event detail storage and exports.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- Consume normalized telemetry events from Kafka.
|
||||
- Store events in Parquet partitioned by protocol and event date.
|
||||
- Query Parquet through DuckDB.
|
||||
- Provide HTTP APIs for date-range query and export.
|
||||
- Support ascending and descending time order.
|
||||
- Support multi-day concatenated export.
|
||||
|
||||
Storage partition:
|
||||
|
||||
```text
|
||||
<root>/
|
||||
protocol=GB32960/
|
||||
date=2026-06-22/
|
||||
part-*.parquet
|
||||
```
|
||||
|
||||
The service stores detail events and field snapshots only. It does not calculate
|
||||
or store daily mileage, daily hydrogen usage, or other derived metrics.
|
||||
|
||||
### Realtime State Service
|
||||
|
||||
The realtime state service owns hot vehicle state.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- Consume normalized telemetry events from Kafka.
|
||||
- Keep latest vehicle state by VIN in Redis.
|
||||
- Keep latest location by VIN in Redis.
|
||||
- Keep latest safety state and hydrogen leak state by VIN in Redis.
|
||||
- Optionally keep short recent windows for operations screens.
|
||||
|
||||
Suggested Redis keys:
|
||||
|
||||
| Key | Value |
|
||||
|---|---|
|
||||
| `vehicle:state:{vin}` | Latest full state snapshot |
|
||||
| `vehicle:location:{vin}` | Latest location snapshot |
|
||||
| `vehicle:safety:{vin}` | Latest safety snapshot |
|
||||
| `vehicle:event:last:{vin}` | Latest event identity and timestamps |
|
||||
|
||||
Redis is for millisecond-level reads and recent state. It is not the historical
|
||||
source of truth.
|
||||
|
||||
Current module: `vehicle-state-service`.
|
||||
|
||||
Implemented boundaries:
|
||||
|
||||
- `VehicleStateEnvelopeIngestor` parses Kafka envelope bytes.
|
||||
- `VehicleStateUpdater` maps full-field telemetry snapshots to Redis JSON
|
||||
payloads.
|
||||
- `VehicleStateRepository` isolates storage from event parsing.
|
||||
- `VehicleStateController` exposes read-only latest state APIs.
|
||||
|
||||
### Statistics Service
|
||||
|
||||
The statistics service owns derived operational metrics.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- Consume normalized telemetry events from Kafka.
|
||||
- Maintain day-level vehicle point state needed for statistics.
|
||||
- Run scheduled and event-triggered calculations.
|
||||
- Store daily results in a statistics database.
|
||||
- Support per-vehicle calculation rules.
|
||||
|
||||
Initial daily mileage strategies:
|
||||
|
||||
| Strategy | Formula |
|
||||
|---|---|
|
||||
| `CURRENT_LAST_MINUS_PREVIOUS_LAST` | Current day last valid total mileage minus previous day last valid total mileage |
|
||||
| `DAY_MAX_MINUS_DAY_MIN` | Current day max valid total mileage minus current day min valid total mileage |
|
||||
|
||||
Hydrogen and electricity statistics use internal fields from the same event
|
||||
contract. Refueling and charging resets must be handled in statistics logic, not
|
||||
in protocol mappers.
|
||||
|
||||
Current module: `vehicle-stat-service`.
|
||||
|
||||
Implemented boundaries:
|
||||
|
||||
- `VehicleStatEnvelopeIngestor` parses Kafka envelope bytes.
|
||||
- `VehicleStatEventProcessor` extracts statistic source points from internal
|
||||
fields such as `total_mileage_km`.
|
||||
- `VehicleStatRepository` isolates point/result persistence from calculation.
|
||||
- `VehicleStatRuleRepository` isolates per-vehicle calculation rules.
|
||||
- `DailyVehicleStatService` calculates and saves day-level results.
|
||||
|
||||
## Data Flow
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
vehicle["Vehicle / Platform"] --> ingest["ingest service"]
|
||||
ingest --> archive["sink-archive<br/>raw bytes"]
|
||||
ingest --> kafka["Kafka<br/>full-field telemetry events"]
|
||||
kafka --> history["event-history-service"]
|
||||
kafka --> state["vehicle-state-service"]
|
||||
kafka --> stat["vehicle-stat-service"]
|
||||
history --> parquet["Parquet files"]
|
||||
history --> duckdb["DuckDB query"]
|
||||
state --> redis["Redis hot state"]
|
||||
stat --> statdb["Statistics DB"]
|
||||
```
|
||||
|
||||
## Kafka Topic Boundaries
|
||||
|
||||
Suggested topics:
|
||||
|
||||
| Topic | Producer | Consumer |
|
||||
|---|---|---|
|
||||
| `vehicle.realtime` | protocol ingress | history, realtime state, statistics |
|
||||
| `vehicle.location` | protocol ingress | history, realtime state, statistics |
|
||||
| `vehicle.alarm` | protocol ingress | history, realtime state |
|
||||
| `vehicle.session` | protocol ingress | history |
|
||||
| `vehicle.media.meta` | protocol ingress | history, media tooling |
|
||||
| `vehicle.raw.archive` | optional archive publisher | replay or audit services |
|
||||
| `vehicle.dlq` | any consumer | operations and replay tooling |
|
||||
|
||||
All telemetry event topics use VIN as the partition key to preserve per-vehicle
|
||||
ordering.
|
||||
|
||||
## Failure Handling
|
||||
|
||||
- Ingest must keep receiving traffic when history, Redis, or statistics services
|
||||
are down.
|
||||
- Kafka publish failures go through the existing sink circuit breaker and DLQ
|
||||
strategy.
|
||||
- History consumer commits Kafka offsets only after Parquet append succeeds.
|
||||
- Redis state consumer can replay from Kafka to rebuild hot state.
|
||||
- Statistics consumer stores checkpoints and can recompute a day from historical
|
||||
detail events when rules change.
|
||||
- Raw archive failures are logged and alerted, but parsed event publication
|
||||
remains independent.
|
||||
|
||||
## Implementation Phases
|
||||
|
||||
### Phase 1: Full-Field Contract And Historical Detail
|
||||
|
||||
- Add a full-field telemetry contract to `ingest-api`.
|
||||
- Map GB/T 32960 parsed blocks into internal field snapshots.
|
||||
- Extend Kafka envelope schema for full-field events.
|
||||
- Make `event-file-store` store full-field telemetry records.
|
||||
- Add query and export APIs for date-range historical detail.
|
||||
|
||||
### Phase 2: Redis Hot State
|
||||
|
||||
- Add a Redis-backed realtime state module.
|
||||
- Consume full-field telemetry events.
|
||||
- Store latest state, location, and safety snapshots by VIN.
|
||||
- Provide query APIs for operations screens.
|
||||
|
||||
### Phase 3: Statistics Service
|
||||
|
||||
- Add a statistics module with rule configuration.
|
||||
- Consume full-field telemetry events.
|
||||
- Implement daily mileage strategies.
|
||||
- Add result persistence boundary and calculation tests.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Ingest has no compile-time dependency on history query controllers, Redis
|
||||
state repositories, or statistics calculators.
|
||||
- Every service boundary communicates through Kafka events or explicit public
|
||||
APIs, not shared mutable state.
|
||||
- GB/T 32960 detail fields are traceable from internal field key back to
|
||||
protocol source path.
|
||||
- Historical query works across multiple days with ascending and descending
|
||||
order.
|
||||
- Redis state can be rebuilt from Kafka replay.
|
||||
- Daily mileage can be calculated by at least the two initial strategies.
|
||||
- Tests cover mapper output, Kafka serialization, Parquet query order, Redis
|
||||
state update, and mileage strategy calculation.
|
||||
146
docs/vehicle-telemetry-internal-fields.md
Normal file
146
docs/vehicle-telemetry-internal-fields.md
Normal file
@@ -0,0 +1,146 @@
|
||||
# Vehicle Telemetry Internal Fields
|
||||
|
||||
This document defines the internal telemetry field model for Lingniu hydrogen
|
||||
vehicle operations. Protocol fields such as GB/T 32960, JT808, MQTT, and vendor
|
||||
extensions must be mapped into these fields before storage, statistics, or
|
||||
business reporting.
|
||||
|
||||
## Design Rules
|
||||
|
||||
- Business statistics use internal fields, not protocol field names.
|
||||
- Every internal field has one stable unit and one business meaning.
|
||||
- Protocol-specific raw values remain traceable through `source_protocol`,
|
||||
`protocol_version`, `metadata`, and raw archive references.
|
||||
- Core operational fields are stored as typed columns or typed payload fields.
|
||||
- Unstable vendor-specific fields can stay in extension metadata until promoted.
|
||||
- Hydrogen leak safety is a first-class critical safety event, not a generic
|
||||
alarm count.
|
||||
|
||||
## Identity And Time
|
||||
|
||||
| Internal field | Type | Unit | Meaning | GB/T 32960 source |
|
||||
|---|---:|---|---|---|
|
||||
| `vin` | string | - | Vehicle VIN and partition key | header VIN |
|
||||
| `event_time` | instant | - | Device collection time | command body timestamp |
|
||||
| `ingest_time` | instant | - | Platform receive time | ingest server clock |
|
||||
| `source_protocol` | enum | - | Source adapter | `GB32960` |
|
||||
| `protocol_version` | string | - | Source protocol version | `V2016` / `V2025` |
|
||||
|
||||
## Real-Time Vehicle State
|
||||
|
||||
| Internal field | Type | Unit | Meaning | GB/T 32960 source |
|
||||
|---|---:|---|---|---|
|
||||
| `vehicle_state` | enum | - | Started, shutdown, other, invalid | vehicle block `vehicleState` |
|
||||
| `charging_state` | enum | - | Charging state | vehicle block `chargingState` |
|
||||
| `running_mode` | enum | - | Electric, hybrid, fuel, other | vehicle block `runningMode` |
|
||||
| `speed_kmh` | double | km/h | Current speed | vehicle block `speedKmh` |
|
||||
| `total_mileage_km` | double | km | Odometer mileage | vehicle block `totalMileageKm` |
|
||||
| `gear_level` | integer | - | Gear value | vehicle block `gearRaw` low 4 bits |
|
||||
| `accelerator_pedal` | double | % | Accelerator pedal opening | V2016 vehicle block |
|
||||
| `brake_pedal` | double | % | Brake pedal opening | V2016 vehicle block |
|
||||
|
||||
## Location
|
||||
|
||||
| Internal field | Type | Unit | Meaning | GB/T 32960 source |
|
||||
|---|---:|---|---|---|
|
||||
| `longitude` | double | deg | Longitude | position block |
|
||||
| `latitude` | double | deg | Latitude | position block |
|
||||
| `altitude_m` | double | m | Altitude when available | extension or other protocol |
|
||||
| `direction_deg` | double | deg | Direction when available | extension or other protocol |
|
||||
| `location_status_raw` | long | - | Raw location status bits | position block `statusFlag` |
|
||||
|
||||
## Battery And Electricity
|
||||
|
||||
| Internal field | Type | Unit | Meaning | GB/T 32960 source |
|
||||
|---|---:|---|---|---|
|
||||
| `battery_soc` | double | % | Battery SOC | vehicle block `socPercent` |
|
||||
| `battery_voltage_v` | double | V | Battery total voltage | vehicle block `totalVoltageV` |
|
||||
| `battery_current_a` | double | A | Battery total current | vehicle block `totalCurrentA` |
|
||||
| `battery_power_kw` | double | kW | Derived instantaneous battery power | `voltage * current / 1000` in statistics |
|
||||
| `battery_temperature_max_c` | integer | C | Max battery temperature | temperature or extreme blocks |
|
||||
| `battery_temperature_min_c` | integer | C | Min battery temperature | temperature or extreme blocks |
|
||||
| `daily_electricity_kwh` | double | kWh | Daily electricity usage | statistics time integral |
|
||||
|
||||
Daily electricity must be computed in the statistics layer from typed telemetry
|
||||
points. Do not accumulate protocol raw values directly.
|
||||
|
||||
## Hydrogen And Fuel Cell
|
||||
|
||||
| Internal field | Type | Unit | Meaning | GB/T 32960 source |
|
||||
|---|---:|---|---|---|
|
||||
| `fc_voltage_v` | double | V | Fuel cell voltage | fuel cell block |
|
||||
| `fc_current_a` | double | A | Fuel cell current | fuel cell block |
|
||||
| `fc_temp_c` | double | C | Fuel cell or hydrogen system temperature | fuel cell block |
|
||||
| `hydrogen_remaining_kg` | double | kg | Remaining hydrogen mass | vendor extension or capacity conversion |
|
||||
| `hydrogen_remaining_percent` | double | % | Remaining hydrogen percent | V2025 fuel cell block |
|
||||
| `hydrogen_high_pressure_mpa` | double | MPa | Hydrogen high pressure | fuel cell block |
|
||||
| `hydrogen_low_pressure_mpa` | double | MPa | Hydrogen low pressure when available | extension or other protocol |
|
||||
| `daily_hydrogen_kg` | double | kg | Daily hydrogen consumed | statistics layer |
|
||||
| `hydrogen_mileage_efficiency` | double | km/kg | Mileage per kg hydrogen | statistics layer |
|
||||
|
||||
Daily hydrogen usage should be calculated from adjacent hydrogen remaining
|
||||
values. A rising value indicates refueling and must not be counted as negative
|
||||
consumption.
|
||||
|
||||
## Hydrogen Tank Safety
|
||||
|
||||
| Internal field | Type | Unit | Meaning | GB/T 32960 source |
|
||||
|---|---:|---|---|---|
|
||||
| `safety_category` | enum | - | `GENERAL`, `TANK_PRESSURE`, `TANK_TEMPERATURE`, `HYDROGEN_LEAK` | alarm bit classification |
|
||||
| `hydrogen_leak_detected` | boolean | - | Whether hydrogen leak is detected | general alarm bit `HYDROGEN_LEAK` |
|
||||
| `hydrogen_leak_level` | enum | - | `NONE`, `WARNING`, `CRITICAL`, `UNKNOWN` | internal rule |
|
||||
| `hydrogen_leak_action_required` | boolean | - | Whether immediate handling is required | internal rule |
|
||||
| `tank_pressure_status` | enum | - | Normal, warning, critical | hydrogen pressure bit or threshold rule |
|
||||
| `tank_temperature_status` | enum | - | Normal, warning, critical | hydrogen temp bit or threshold rule |
|
||||
|
||||
Safety rule:
|
||||
|
||||
```text
|
||||
If HYDROGEN_LEAK is present:
|
||||
alarm.level = CRITICAL
|
||||
safety_category = HYDROGEN_LEAK
|
||||
hydrogen_leak_detected = true
|
||||
hydrogen_leak_level = CRITICAL
|
||||
hydrogen_leak_action_required = true
|
||||
```
|
||||
|
||||
Hydrogen leak overrides otherwise normal vehicle, speed, battery, or hydrogen
|
||||
state. It must be counted independently from generic alarms.
|
||||
|
||||
## Daily Operation Statistics
|
||||
|
||||
Daily statistics are generated by `stat_date + vin`.
|
||||
|
||||
| Internal field | Type | Unit | Meaning |
|
||||
|---|---:|---|---|
|
||||
| `stat_date` | date | - | Statistics date |
|
||||
| `vin` | string | - | Vehicle VIN |
|
||||
| `first_event_time` | instant | - | First valid telemetry time |
|
||||
| `last_event_time` | instant | - | Last valid telemetry time |
|
||||
| `online_minutes` | double | min | Online duration |
|
||||
| `running_minutes` | double | min | Running duration |
|
||||
| `daily_mileage_km` | double | km | Daily mileage |
|
||||
| `daily_electricity_kwh` | double | kWh | Daily electricity usage |
|
||||
| `daily_hydrogen_kg` | double | kg | Daily hydrogen usage |
|
||||
| `hydrogen_added_kg` | double | kg | Refueled hydrogen |
|
||||
| `km_per_kg_hydrogen` | double | km/kg | Hydrogen efficiency |
|
||||
| `kwh_per_100km` | double | kWh/100km | Electricity efficiency |
|
||||
| `alarm_count` | integer | - | Generic alarm count |
|
||||
| `tank_safety_alarm_count` | integer | - | Tank safety alarm count |
|
||||
| `hydrogen_leak_alarm_count` | integer | - | Hydrogen leak alarm count |
|
||||
| `hydrogen_leak_duration_seconds` | long | s | Hydrogen leak duration |
|
||||
| `data_quality_level` | enum | - | Good, partial, bad |
|
||||
|
||||
## Implementation Status
|
||||
|
||||
- `RealtimePayload` already uses internal operational field names for state,
|
||||
location, battery, fuel cell, and hydrogen fields.
|
||||
- `AlarmPayload` carries internal hydrogen safety fields.
|
||||
- `Gb32960EventMapper` maps GB/T 32960 alarm bits into internal safety fields.
|
||||
- `TelemetrySnapshot` publishes full-field internal telemetry values through the
|
||||
Kafka protobuf envelope.
|
||||
- `event-history-service`, `vehicle-state-service`, and `vehicle-stat-service`
|
||||
consume the same full-field telemetry snapshot contract.
|
||||
- `event-history-service` CSV export can flatten selected internal fields with
|
||||
`fields=field_a,field_b`; media archive references are exposed through
|
||||
`rawArchiveUri` by falling back to `MediaMeta.archiveRef`.
|
||||
@@ -1,152 +0,0 @@
|
||||
package com.lingniu.ingest.inbound.mqtt.client;
|
||||
|
||||
import com.hivemq.client.mqtt.datatypes.MqttQos;
|
||||
import com.hivemq.client.mqtt.mqtt3.Mqtt3AsyncClient;
|
||||
import com.hivemq.client.mqtt.mqtt3.Mqtt3Client;
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
import com.lingniu.ingest.api.pipeline.RawFrame;
|
||||
import com.lingniu.ingest.core.dispatcher.Dispatcher;
|
||||
import com.lingniu.ingest.inbound.mqtt.config.MqttInboundProperties;
|
||||
import com.lingniu.ingest.inbound.mqtt.model.MqttPayload;
|
||||
import com.lingniu.ingest.inbound.mqtt.parser.MqttPayloadParser;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.net.URI;
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.HashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* MQTT 多 endpoint 生命周期管理。
|
||||
*
|
||||
* <p>为每个 endpoint 创建一个 HiveMQ 异步客户端 → 连接 → 订阅 → 注册回调 →
|
||||
* 回调里解析 payload → {@link Dispatcher#dispatch(RawFrame)}。
|
||||
*
|
||||
* <p>Netty EventLoop 回调不做业务处理，虚拟线程由 Dispatcher 侧的 Disruptor 承接。
|
||||
*/
|
||||
public final class MqttEndpointManager implements AutoCloseable {
|
||||
|
||||
private static final Logger log = LoggerFactory.getLogger(MqttEndpointManager.class);
|
||||
|
||||
private final MqttInboundProperties props;
|
||||
private final MqttPayloadParser payloadParser;
|
||||
private final Dispatcher dispatcher;
|
||||
private final List<Mqtt3AsyncClient> clients = new ArrayList<>();
|
||||
|
||||
public MqttEndpointManager(MqttInboundProperties props,
|
||||
MqttPayloadParser payloadParser,
|
||||
Dispatcher dispatcher) {
|
||||
this.props = props;
|
||||
this.payloadParser = payloadParser;
|
||||
this.dispatcher = dispatcher;
|
||||
}
|
||||
|
||||
public void start() {
|
||||
for (MqttInboundProperties.Endpoint ep : props.getEndpoints()) {
|
||||
try {
|
||||
Mqtt3AsyncClient client = buildClient(ep);
|
||||
clients.add(client);
|
||||
connectAndSubscribe(client, ep);
|
||||
} catch (Exception e) {
|
||||
log.error("mqtt endpoint [{}] init failed, skipped", ep.getName(), e);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private Mqtt3AsyncClient buildClient(MqttInboundProperties.Endpoint ep) {
|
||||
URI uri = URI.create(ep.getUri());
|
||||
int port = uri.getPort() > 0 ? uri.getPort() : defaultPort(uri.getScheme());
|
||||
var builder = Mqtt3Client.builder()
|
||||
.identifier(ep.getClientId() == null || ep.getClientId().isBlank()
|
||||
? "lingniu-" + UUID.randomUUID() : ep.getClientId())
|
||||
.serverHost(uri.getHost())
|
||||
.serverPort(port)
|
||||
.automaticReconnectWithDefaultConfig();
|
||||
|
||||
if ("ssl".equalsIgnoreCase(uri.getScheme()) || "tls".equalsIgnoreCase(uri.getScheme())
|
||||
|| "mqtts".equalsIgnoreCase(uri.getScheme())) {
|
||||
// TODO: 基于 ep.getTls() 的 caPem / clientPem / clientKey 构造自定义 SslConfig
|
||||
builder = builder.sslWithDefaultConfig();
|
||||
}
|
||||
return builder.buildAsync();
|
||||
}
|
||||
|
||||
private void connectAndSubscribe(Mqtt3AsyncClient client, MqttInboundProperties.Endpoint ep) {
|
||||
var connectBuilder = client.connectWith();
|
||||
if (ep.getUsername() != null && !ep.getUsername().isBlank()) {
|
||||
connectBuilder = connectBuilder.simpleAuth()
|
||||
.username(ep.getUsername())
|
||||
.password(ep.getPassword() == null ? new byte[0] : ep.getPassword().getBytes())
|
||||
.applySimpleAuth();
|
||||
}
|
||||
connectBuilder.send().whenComplete((ack, err) -> {
|
||||
if (err != null) {
|
||||
log.error("mqtt endpoint [{}] connect failed", ep.getName(), err);
|
||||
return;
|
||||
}
|
||||
log.info("mqtt endpoint [{}] connected code={}", ep.getName(), ack.getReturnCode());
|
||||
client.subscribeWith()
|
||||
.topicFilter(ep.getTopic())
|
||||
.qos(mapQos(ep.getQos()))
|
||||
.callback(publish -> onMessage(ep, publish.getTopic().toString(), publish.getPayloadAsBytes()))
|
||||
.send()
|
||||
.whenComplete((subAck, subErr) -> {
|
||||
if (subErr != null) {
|
||||
log.error("mqtt endpoint [{}] subscribe failed topic={}", ep.getName(), ep.getTopic(), subErr);
|
||||
} else {
|
||||
log.info("mqtt endpoint [{}] subscribed topic={} qos={}",
|
||||
ep.getName(), ep.getTopic(), ep.getQos());
|
||||
}
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
private void onMessage(MqttInboundProperties.Endpoint ep, String topic, byte[] payload) {
|
||||
try {
|
||||
MqttPayload parsed = payloadParser.parse(ep.getName(), ep.getProfile(), topic, payload);
|
||||
Map<String, String> meta = new HashMap<>(4);
|
||||
meta.put("vin", parsed.vin() == null ? "unknown" : parsed.vin());
|
||||
meta.put("endpoint", ep.getName());
|
||||
meta.put("topic", topic);
|
||||
RawFrame rf = new RawFrame(
|
||||
ProtocolId.MQTT_YUTONG, 0, 0,
|
||||
parsed, payload, meta, Instant.now());
|
||||
dispatcher.dispatch(rf);
|
||||
} catch (Exception e) {
|
||||
log.warn("mqtt endpoint [{}] message handling failed topic={} len={}",
|
||||
ep.getName(), topic, payload.length, e);
|
||||
}
|
||||
}
|
||||
|
||||
private static MqttQos mapQos(int level) {
|
||||
return switch (level) {
|
||||
case 0 -> MqttQos.AT_MOST_ONCE;
|
||||
case 2 -> MqttQos.EXACTLY_ONCE;
|
||||
default -> MqttQos.AT_LEAST_ONCE;
|
||||
};
|
||||
}
|
||||
|
||||
private static int defaultPort(String scheme) {
|
||||
if (scheme == null) return 1883;
|
||||
return switch (scheme.toLowerCase()) {
|
||||
case "ssl", "tls", "mqtts" -> 8883;
|
||||
default -> 1883;
|
||||
};
|
||||
}
|
||||
|
||||
@Override
|
||||
public void close() {
|
||||
for (Mqtt3AsyncClient c : clients) {
|
||||
try {
|
||||
c.disconnect();
|
||||
} catch (Exception ignored) {
|
||||
// best-effort
|
||||
}
|
||||
}
|
||||
log.info("mqtt endpoint manager stopped, clients={}", clients.size());
|
||||
}
|
||||
}
|
||||
@@ -1,99 +0,0 @@
|
||||
package com.lingniu.ingest.inbound.mqtt.mapper;
|
||||
|
||||
import com.fasterxml.jackson.databind.JsonNode;
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
import com.lingniu.ingest.api.event.LocationPayload;
|
||||
import com.lingniu.ingest.api.event.RealtimePayload;
|
||||
import com.lingniu.ingest.api.event.VehicleEvent;
|
||||
import com.lingniu.ingest.api.spi.EventMapper;
|
||||
import com.lingniu.ingest.inbound.mqtt.model.MqttPayload;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* 宇通 MQTT payload → 领域事件。
|
||||
*
|
||||
* <p>字段名取自旧项目的 {@code ReceptionDataVO}（大写 + 下划线），对字段映射一致性做出显式断言，
|
||||
* 便于与旧服务双跑对账。
|
||||
*/
|
||||
public final class YutongEventMapper implements EventMapper<MqttPayload> {
|
||||
|
||||
@Override
|
||||
public List<VehicleEvent> toEvents(MqttPayload payload) {
|
||||
JsonNode d = payload.data();
|
||||
if (d == null || !d.isObject()) return List.of();
|
||||
|
||||
Instant ingestTime = Instant.now();
|
||||
String vin = payload.vin();
|
||||
Map<String, String> meta = Map.of(
|
||||
"endpoint", payload.endpointName() == null ? "" : payload.endpointName(),
|
||||
"topic", payload.topic() == null ? "" : payload.topic(),
|
||||
"profile", payload.profile() == null ? "" : payload.profile());
|
||||
|
||||
Double speed = optDouble(d, "METER_SPEED");
|
||||
Double mileage = optDouble(d, "TOTAL_MILEAGE");
|
||||
Double soc = optDouble(d, "BATTERY_CAPACITY_SOC");
|
||||
Double fcVoltage = optDouble(d, "VOLTAGE_OF_FC");
|
||||
Double fcCurrent = optDouble(d, "CURRENT_OF_FC");
|
||||
Double fcTemp = optDouble(d, "TEMPT_OF_FC");
|
||||
Double leftH2 = optDouble(d, "LEFT_HYDROGEN");
|
||||
Double lowH2Pressure = optDouble(d, "HYDROGEN_LOW_PRESSURE");
|
||||
Double longitude = optCoord(d, "LONGITUDE");
|
||||
Double latitude = optCoord(d, "LATITUDE");
|
||||
Double direction = optDouble(d, "GPSDirection");
|
||||
Double acc = optDouble(d, "ACC_PEDAL_APT");
|
||||
Double brake = optDouble(d, "BRAKEPEDALAPT");
|
||||
Integer gear = optInt(d, "ON_GEAR");
|
||||
|
||||
RealtimePayload realtime = new RealtimePayload(
|
||||
speed, mileage, soc, null, null,
|
||||
fcVoltage, fcCurrent, fcTemp, leftH2, null, lowH2Pressure,
|
||||
null, null, null, gear, acc, brake,
|
||||
longitude, latitude, null, direction,
|
||||
null, null);
|
||||
|
||||
List<VehicleEvent> out = new ArrayList<>(2);
|
||||
out.add(new VehicleEvent.Realtime(
|
||||
UUID.randomUUID().toString(),
|
||||
vin, ProtocolId.MQTT_YUTONG, payload.deviceTime(), ingestTime,
|
||||
null, meta, realtime));
|
||||
|
||||
if (longitude != null && latitude != null) {
|
||||
LocationPayload loc = new LocationPayload(
|
||||
longitude, latitude, 0.0,
|
||||
speed == null ? 0.0 : speed,
|
||||
direction == null ? 0.0 : direction,
|
||||
0, 0);
|
||||
out.add(new VehicleEvent.Location(
|
||||
UUID.randomUUID().toString(),
|
||||
vin, ProtocolId.MQTT_YUTONG, payload.deviceTime(), ingestTime,
|
||||
null, meta, loc));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
private static Double optDouble(JsonNode n, String field) {
|
||||
JsonNode v = n.get(field);
|
||||
if (v == null || v.isNull() || !v.isNumber()) return null;
|
||||
return v.asDouble();
|
||||
}
|
||||
|
||||
private static Integer optInt(JsonNode n, String field) {
|
||||
JsonNode v = n.get(field);
|
||||
if (v == null || v.isNull() || !v.isNumber()) return null;
|
||||
return v.asInt();
|
||||
}
|
||||
|
||||
/** 宇通坐标约定：1e6 整数 或 已经是小数。同时过滤无效值 999999 / 0。 */
|
||||
private static Double optCoord(JsonNode n, String field) {
|
||||
Double v = optDouble(n, field);
|
||||
if (v == null) return null;
|
||||
if (v.intValue() == 999_999 || v == 0.0) return null;
|
||||
if (Math.abs(v) > 1000) return v / 1_000_000.0;
|
||||
return v;
|
||||
}
|
||||
}
|
||||
@@ -1,67 +0,0 @@
|
||||
package com.lingniu.ingest.inbound.mqtt;
|
||||
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
import com.lingniu.ingest.api.event.VehicleEvent;
|
||||
import com.lingniu.ingest.inbound.mqtt.mapper.YutongEventMapper;
|
||||
import com.lingniu.ingest.inbound.mqtt.model.MqttPayload;
|
||||
import com.lingniu.ingest.inbound.mqtt.parser.MqttPayloadParser;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
class YutongEventMapperTest {
|
||||
|
||||
private final ObjectMapper objectMapper = new ObjectMapper();
|
||||
private final MqttPayloadParser parser = new MqttPayloadParser(objectMapper);
|
||||
private final YutongEventMapper mapper = new YutongEventMapper();
|
||||
|
||||
@Test
|
||||
void parsesYutongSamplePayload() {
|
||||
String json = """
|
||||
{
|
||||
"id": "abc",
|
||||
"device": "LTEST000000000001",
|
||||
"code": "C1",
|
||||
"time": "20260413100000",
|
||||
"data": {
|
||||
"METER_SPEED": 52.3,
|
||||
"TOTAL_MILEAGE": 123456.7,
|
||||
"BATTERY_CAPACITY_SOC": 70,
|
||||
"LONGITUDE": 116397128,
|
||||
"LATITUDE": 39916527,
|
||||
"GPSDirection": 90,
|
||||
"LEFT_HYDROGEN": 5.5,
|
||||
"VOLTAGE_OF_FC": 600.0,
|
||||
"CURRENT_OF_FC": 120.0,
|
||||
"TEMPT_OF_FC": 65.0,
|
||||
"ON_GEAR": 3,
|
||||
"ACC_PEDAL_APT": 25,
|
||||
"BRAKEPEDALAPT": 0
|
||||
}
|
||||
}
|
||||
""";
|
||||
|
||||
MqttPayload payload = parser.parse("yutong", "yutong", "/ytforward/shln/dev1", json.getBytes());
|
||||
assertThat(payload.vin()).isEqualTo("LTEST000000000001");
|
||||
|
||||
List<VehicleEvent> events = mapper.toEvents(payload);
|
||||
assertThat(events).hasSize(2);
|
||||
|
||||
VehicleEvent.Realtime rt = (VehicleEvent.Realtime) events.stream()
|
||||
.filter(e -> e instanceof VehicleEvent.Realtime).findFirst().orElseThrow();
|
||||
assertThat(rt.source()).isEqualTo(ProtocolId.MQTT_YUTONG);
|
||||
assertThat(rt.vin()).isEqualTo("LTEST000000000001");
|
||||
assertThat(rt.payload().speedKmh()).isEqualTo(52.3);
|
||||
assertThat(rt.payload().batterySoc()).isEqualTo(70.0);
|
||||
assertThat(rt.payload().longitude()).isEqualTo(116.397128, org.assertj.core.data.Offset.offset(0.000001));
|
||||
assertThat(rt.payload().latitude()).isEqualTo(39.916527, org.assertj.core.data.Offset.offset(0.000001));
|
||||
assertThat(rt.payload().hydrogenRemainingKg()).isEqualTo(5.5);
|
||||
|
||||
VehicleEvent.Location loc = (VehicleEvent.Location) events.stream()
|
||||
.filter(e -> e instanceof VehicleEvent.Location).findFirst().orElseThrow();
|
||||
assertThat(loc.payload().longitude()).isEqualTo(116.397128, org.assertj.core.data.Offset.offset(0.000001));
|
||||
}
|
||||
}
|
||||
@@ -1,267 +0,0 @@
|
||||
package com.lingniu.ingest.inbound.xinda;
|
||||
|
||||
import com.alibaba.fastjson.JSON;
|
||||
import com.alibaba.fastjson.JSONObject;
|
||||
import com.gps31.push.netty.PushClient;
|
||||
import com.gps31.push.netty.PushMsg;
|
||||
import com.gps31.push.netty.client.TcpClient;
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
import com.lingniu.ingest.api.event.LocationPayload;
|
||||
import com.lingniu.ingest.api.event.VehicleEvent;
|
||||
import com.lingniu.ingest.core.concurrency.DisruptorEventBus;
|
||||
import com.lingniu.ingest.core.dispatcher.Dispatcher;
|
||||
import com.lingniu.ingest.inbound.xinda.config.XindaPushProperties;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.Instant;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* 信达 Push 接入客户端。基于 {@code org.lingniu:gps-push-client:1.0} 的 {@link PushClient}
|
||||
* 实现，完全使用信达真实协议帧格式（帧头 / 长度 / cmd / json / 帧尾 + 序列号 + 心跳）。
|
||||
*
|
||||
* <p>行为：
|
||||
* <ul>
|
||||
* <li>{@link #start()} 调用基类 {@code TcpClient.start()} 建立连接并自动重连
|
||||
* <li>基类在 {@code channelActive} 自动发送 {@code 0x0001 登录} 帧，携带 userName / pwd / csTag / desc
|
||||
* <li>登录成功收到应答后（由基类内部处理）会自动订阅 {@code subMsgIds}（默认空，需显式调用 setSubMsgIds）
|
||||
* <li>{@link #messageReceived(TcpClient, PushMsg)} 为业务入口：cmd 字符串区分消息类型
|
||||
* <li>位置 {@code 0200} → {@link VehicleEvent.Location}
|
||||
* <li>报警 {@code 0300} → {@link VehicleEvent.Alarm}（转为 Passthrough 保留原始 json 供下游分析）
|
||||
* <li>透传 {@code 0401} → {@link VehicleEvent.Passthrough}
|
||||
* </ul>
|
||||
*
|
||||
* <p>类继承基类 {@code PushClient}，不再需要手工处理帧编解码、心跳、重连、订阅等机制。
|
||||
*/
|
||||
public class XindaPushClient extends PushClient {
|
||||
|
||||
private static final Logger log = LoggerFactory.getLogger(XindaPushClient.class);
|
||||
|
||||
private final XindaPushProperties props;
|
||||
private final Dispatcher dispatcher;
|
||||
private final DisruptorEventBus eventBus;
|
||||
|
||||
public XindaPushClient(XindaPushProperties props, Dispatcher dispatcher, DisruptorEventBus eventBus) {
|
||||
super();
|
||||
this.props = props;
|
||||
this.dispatcher = dispatcher;
|
||||
this.eventBus = eventBus;
|
||||
this.isLog = false;
|
||||
}
|
||||
|
||||
/**
|
||||
* 启动客户端：填入 host/port/user/pwd/desc，配置订阅消息 ID，交给基类建立连接。
|
||||
*/
|
||||
public void start() {
|
||||
if (props.getHost() == null || props.getHost().isBlank()) {
|
||||
log.error("[xinda-push] host not configured, module stays idle");
|
||||
return;
|
||||
}
|
||||
if (props.getUsername() == null || props.getPassword() == null) {
|
||||
log.error("[xinda-push] credentials missing, module stays idle "
|
||||
+ "(set lingniu.ingest.xinda-push.username/password)");
|
||||
return;
|
||||
}
|
||||
setHost(props.getHost());
|
||||
setPort(props.getPort());
|
||||
setUserName(props.getUsername());
|
||||
setPwd(props.getPassword());
|
||||
setDesc(props.getClientDesc());
|
||||
// 订阅消息 ID：信达基类 PushClient.setSubMsgIds 用管道符 "|" 作为分隔符
|
||||
// （内部调用 StringUtil.splitStr(s, "|")）
|
||||
if (props.getSubscribeMsgIds() != null && !props.getSubscribeMsgIds().isEmpty()) {
|
||||
setSubMsgIds(String.join("|", props.getSubscribeMsgIds()));
|
||||
}
|
||||
// 基类 PushClient 构造器默认 isLog=false / isLogBytes=false；
|
||||
// 这里尊重默认值，不再覆盖为 true，避免 <<<Log发送 / >>>Log接收 每条帧都被 ERROR 级打印。
|
||||
// 真要打开 wire-level 诊断时，临时改为 setLog(true) 即可。
|
||||
try {
|
||||
log.info("[xinda-push] starting host={} port={} user={} subMsgIds={}",
|
||||
getHost(), getPort(), getUserName(), getSubMsgIds());
|
||||
super.start();
|
||||
} catch (Exception e) {
|
||||
log.error("[xinda-push] start failed", e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Spring 生命周期停止钩子。
|
||||
*/
|
||||
public void stop() {
|
||||
try {
|
||||
destroy();
|
||||
log.info("[xinda-push] stopped");
|
||||
} catch (Exception e) {
|
||||
log.warn("[xinda-push] stop error", e);
|
||||
}
|
||||
}
|
||||
|
||||
// ========================================================================
|
||||
// 业务回调：收到完整的 PushMsg 后分派事件
|
||||
// ========================================================================
|
||||
|
||||
@Override
|
||||
public void messageReceived(TcpClient client, PushMsg msg) throws Exception {
|
||||
super.messageReceived(client, msg);
|
||||
String cmd = msg.getCmd();
|
||||
if (cmd == null) return;
|
||||
try {
|
||||
switch (cmd) {
|
||||
case "8001" -> handleLoginAck(msg);
|
||||
case "8002" -> log.debug("[xinda-push] heartbeat ack");
|
||||
case "8003" -> log.info("[xinda-push] subscribe ack json={}", msg.getJson());
|
||||
case "0200" -> handleLocation(msg);
|
||||
case "0300" -> handleAlarm(msg);
|
||||
case "0401" -> handlePassthrough(msg);
|
||||
default -> log.debug("[xinda-push] rx cmd={} json={}", cmd, msg.getJson());
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.warn("[xinda-push] dispatch failed cmd={} json={}", cmd, msg.getJson(), e);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 收到 8001 登录应答后处理：若 {@code rspResult=0} 则主动发送 0003 订阅命令。
|
||||
*
|
||||
* <p>信达订阅命令的 body 格式（从旧 {@code PushClientHandler} 和 8003 错误提示反推得到）：
|
||||
* <pre>{@code
|
||||
* {
|
||||
* "seq": "1", // 序列号
|
||||
* "action": "add", // 操作: add 订阅 / del 取消
|
||||
* "msgIds": "[\"0200\",\"0300\",\"0401\"]" // JSON 数组字符串（注意是字符串，不是数组）
|
||||
* }
|
||||
* }</pre>
|
||||
* <p>
|
||||
* 其中 {@code msgIds} 的值来自基类 {@link PushClient#getSubCmdSet()}，内容由
|
||||
* {@link PushClient#setSubMsgIds(String)} 填充（分隔符是 {@code |}）。
|
||||
*/
|
||||
private void handleLoginAck(PushMsg msg) throws Exception {
|
||||
String json = msg.getJson() == null ? "" : msg.getJson();
|
||||
log.info("[xinda-push] login ack json={}", json);
|
||||
JSONObject parsed = JSON.parseObject(json);
|
||||
String rsp = parsed == null ? null : parsed.getString("rspResult");
|
||||
if (!"0".equals(rsp)) {
|
||||
log.error("[xinda-push] login REJECTED rspResult={}", rsp);
|
||||
return;
|
||||
}
|
||||
Map<String, Object> body = new java.util.HashMap<>();
|
||||
body.put("seq", "1");
|
||||
body.put("action", "add");
|
||||
body.put("msgIds", JSON.toJSONString(getSubCmdSet()));
|
||||
PushMsg subscribe = getInstance("0003", body);
|
||||
sendMsg(subscribe);
|
||||
log.info("[xinda-push] subscribe sent action=add msgIds={}", getSubCmdSet());
|
||||
}
|
||||
|
||||
@Override
|
||||
public void channelActive(TcpClient client) throws Exception {
|
||||
super.channelActive(client);
|
||||
log.info("[xinda-push] channel active, login frame sent");
|
||||
}
|
||||
|
||||
@Override
|
||||
public void channelInactive(TcpClient client) throws Exception {
|
||||
super.channelInactive(client);
|
||||
log.warn("[xinda-push] channel inactive, will auto-reconnect");
|
||||
}
|
||||
|
||||
// ========================================================================
|
||||
// 业务处理（0200/0300/0401 → VehicleEvent）
|
||||
// ========================================================================
|
||||
|
||||
/**
|
||||
* 0200 定位消息 → {@link VehicleEvent.Location}。
|
||||
* <p>信达 0200 常见字段（不同版本可能略有差异）：
|
||||
* <pre>
|
||||
* plateNo 车牌号
|
||||
* lat/lng WGS84 度 * 1e6 (整数)
|
||||
* speed km/h
|
||||
* drct 方向
|
||||
* height 海拔 m
|
||||
* alarmStts 报警状态位
|
||||
* statusStts 状态位
|
||||
* time yyyyMMddHHmmss
|
||||
* </pre>
|
||||
*/
|
||||
private void handleLocation(PushMsg msg) {
|
||||
JSONObject json = JSON.parseObject(msg.getJson());
|
||||
String vin = firstNonBlank(json, "vin", "plateNo", "carNo");
|
||||
double lon = asDouble(json, "lng", "lon", "longitude") / 1_000_000.0;
|
||||
double lat = asDouble(json, "lat", "latitude") / 1_000_000.0;
|
||||
// 有的实现已经是十进制度，做一次保护：如果 |lon| > 1000 显然是 1e6 单位再除一次
|
||||
if (Math.abs(lon) > 1000) lon /= 1.0; // already divided above
|
||||
double speed = asDouble(json, "speed");
|
||||
double direction = asDouble(json, "drct", "direction");
|
||||
long alarmFlag = (long) asDouble(json, "alarmStts", "alarmFlag");
|
||||
long statusFlag = (long) asDouble(json, "statusStts", "statusFlag");
|
||||
|
||||
VehicleEvent.Location loc = new VehicleEvent.Location(
|
||||
UUID.randomUUID().toString(),
|
||||
vin.isBlank() ? "unknown" : vin,
|
||||
ProtocolId.XINDA_PUSH,
|
||||
Instant.now(), Instant.now(),
|
||||
null,
|
||||
Map.of("source", "xinda-push", "cmd", "0200"),
|
||||
new LocationPayload(lon, lat, 0.0, speed, direction, alarmFlag, statusFlag));
|
||||
eventBus.publish(loc);
|
||||
log.debug("[xinda-push] 0200 location vin={} lon={} lat={} speed={}", vin, lon, lat, speed);
|
||||
}
|
||||
|
||||
private void handleAlarm(PushMsg msg) {
|
||||
emitPassthrough(0x0300, msg);
|
||||
log.info("[xinda-push] 0300 alarm json={}", msg.getJson());
|
||||
}
|
||||
|
||||
private void handlePassthrough(PushMsg msg) {
|
||||
emitPassthrough(0x0401, msg);
|
||||
log.debug("[xinda-push] 0401 passthrough json={}", msg.getJson());
|
||||
}
|
||||
|
||||
private void emitPassthrough(int type, PushMsg msg) {
|
||||
JSONObject json = JSON.parseObject(msg.getJson());
|
||||
String vin = firstNonBlank(json, "vin", "plateNo", "carNo");
|
||||
byte[] raw = msg.getJson() == null
|
||||
? new byte[0] : msg.getJson().getBytes(StandardCharsets.UTF_8);
|
||||
VehicleEvent.Passthrough pt = new VehicleEvent.Passthrough(
|
||||
UUID.randomUUID().toString(),
|
||||
vin.isBlank() ? "unknown" : vin,
|
||||
ProtocolId.XINDA_PUSH,
|
||||
Instant.now(), Instant.now(),
|
||||
null,
|
||||
Map.of("source", "xinda-push", "cmd", String.format("%04x", type),
|
||||
"rawJson", msg.getJson() == null ? "" : msg.getJson()),
|
||||
type, raw);
|
||||
eventBus.publish(pt);
|
||||
}
|
||||
|
||||
// ========================================================================
|
||||
// JSON helpers
|
||||
// ========================================================================
|
||||
|
||||
private static String firstNonBlank(JSONObject node, String... keys) {
|
||||
if (node == null) return "";
|
||||
for (String k : keys) {
|
||||
Object v = node.get(k);
|
||||
if (v != null && !v.toString().isBlank()) return v.toString();
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
private static double asDouble(JSONObject node, String... keys) {
|
||||
if (node == null) return 0.0;
|
||||
for (String k : keys) {
|
||||
Object v = node.get(k);
|
||||
if (v instanceof Number n) return n.doubleValue();
|
||||
if (v != null) {
|
||||
try {
|
||||
return Double.parseDouble(v.toString());
|
||||
} catch (NumberFormatException ignored) {
|
||||
}
|
||||
}
|
||||
}
|
||||
return 0.0;
|
||||
}
|
||||
}
|
||||
@@ -1,22 +0,0 @@
|
||||
package com.lingniu.ingest.api.annotation;
|
||||
|
||||
import java.lang.annotation.ElementType;
|
||||
import java.lang.annotation.Retention;
|
||||
import java.lang.annotation.RetentionPolicy;
|
||||
import java.lang.annotation.Target;
|
||||
|
||||
/**
|
||||
* 批量异步处理：框架把多次同类调用聚合成 {@code List} 交给方法。
|
||||
*
|
||||
* <p>处理方法签名需为 {@code void foo(List<T> list)} 或返回 {@code List<VehicleEvent>}。
|
||||
*/
|
||||
@Retention(RetentionPolicy.RUNTIME)
|
||||
@Target(ElementType.METHOD)
|
||||
public @interface AsyncBatch {
|
||||
|
||||
int size() default 1000;
|
||||
|
||||
long waitMs() default 500;
|
||||
|
||||
int poolSize() default 2;
|
||||
}
|
||||
@@ -1,12 +0,0 @@
|
||||
package com.lingniu.ingest.api.event;
|
||||
|
||||
/** 位置事件载荷。坐标已统一转换为 WGS84 十进制度。 */
|
||||
public record LocationPayload(
|
||||
double longitude,
|
||||
double latitude,
|
||||
double altitudeM,
|
||||
double speedKmh,
|
||||
double directionDeg,
|
||||
long alarmFlag,
|
||||
long statusFlag
|
||||
) {}
|
||||
@@ -1,79 +0,0 @@
|
||||
package com.lingniu.ingest.core.dispatcher;
|
||||
|
||||
import com.lingniu.ingest.api.event.VehicleEvent;
|
||||
import com.lingniu.ingest.api.pipeline.IngestContext;
|
||||
import com.lingniu.ingest.api.pipeline.RawFrame;
|
||||
import com.lingniu.ingest.core.concurrency.AsyncBatchExecutor;
|
||||
import com.lingniu.ingest.core.concurrency.DisruptorEventBus;
|
||||
import com.lingniu.ingest.core.pipeline.InterceptorChain;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.UUID;
|
||||
|
||||
/**
|
||||
* 核心分发器：RawFrame → 拦截链 → Handler → 事件 → Disruptor EventBus。
|
||||
*
|
||||
* <p>所有 Inbound Adapter（Netty / MQTT / PushClient）都调用 {@link #dispatch(RawFrame)}，
|
||||
* 协议差异在这里被彻底抹平。
|
||||
*
|
||||
* <p>对于带 {@code @AsyncBatch} 的 Handler，Dispatcher 不直接反射调用，而是把消息交给
|
||||
* {@link AsyncBatchExecutor} 进行聚合 → 批量调用 → 异步发布事件。
|
||||
*/
|
||||
public final class Dispatcher {
|
||||
|
||||
private static final Logger log = LoggerFactory.getLogger(Dispatcher.class);
|
||||
|
||||
private final HandlerRegistry registry;
|
||||
private final InterceptorChain interceptors;
|
||||
private final HandlerInvoker invoker;
|
||||
private final DisruptorEventBus eventBus;
|
||||
private final AsyncBatchExecutor batchExecutor;
|
||||
|
||||
public Dispatcher(HandlerRegistry registry,
|
||||
InterceptorChain interceptors,
|
||||
HandlerInvoker invoker,
|
||||
DisruptorEventBus eventBus,
|
||||
AsyncBatchExecutor batchExecutor) {
|
||||
this.registry = registry;
|
||||
this.interceptors = interceptors;
|
||||
this.invoker = invoker;
|
||||
this.eventBus = eventBus;
|
||||
this.batchExecutor = batchExecutor;
|
||||
}
|
||||
|
||||
public void dispatch(RawFrame frame) {
|
||||
IngestContext ctx = new IngestContext(UUID.randomUUID().toString());
|
||||
try {
|
||||
if (!interceptors.before(frame, ctx)) {
|
||||
log.debug("frame aborted: {}", ctx.abortReason());
|
||||
return;
|
||||
}
|
||||
|
||||
List<HandlerDefinition> handlers = registry.resolve(
|
||||
frame.protocolId(), frame.command(), frame.infoType());
|
||||
if (handlers.isEmpty()) {
|
||||
log.debug("no handler for {} cmd=0x{} info=0x{}",
|
||||
frame.protocolId(), Integer.toHexString(frame.command()),
|
||||
Integer.toHexString(frame.infoType()));
|
||||
return;
|
||||
}
|
||||
|
||||
for (HandlerDefinition def : handlers) {
|
||||
if (def.asyncBatch() != null) {
|
||||
batchExecutor.submit(def, frame.payload());
|
||||
continue;
|
||||
}
|
||||
List<VehicleEvent> events = invoker.invoke(def, frame, ctx);
|
||||
for (VehicleEvent e : events) {
|
||||
interceptors.after(e, ctx);
|
||||
eventBus.publish(e);
|
||||
}
|
||||
}
|
||||
} catch (Throwable t) {
|
||||
log.error("dispatch failure traceId={}", ctx.traceId(), t);
|
||||
interceptors.onError(t, ctx);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,7 @@
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>lingniu-vehicle-ingest</artifactId>
|
||||
<version>0.1.0-SNAPSHOT</version>
|
||||
<relativePath>../../../pom.xml</relativePath>
|
||||
</parent>
|
||||
<artifactId>bootstrap-all</artifactId>
|
||||
<name>bootstrap-all</name>
|
||||
@@ -34,6 +35,22 @@
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>sink-archive</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>event-file-store</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>event-history-service</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>vehicle-state-service</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>vehicle-stat-service</artifactId>
|
||||
</dependency>
|
||||
|
||||
<!-- Protocols -->
|
||||
<dependency>
|
||||
@@ -73,6 +90,10 @@
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springdoc</groupId>
|
||||
<artifactId>springdoc-openapi-starter-webmvc-ui</artifactId>
|
||||
</dependency>
|
||||
|
||||
<!-- 测试 -->
|
||||
<dependency>
|
||||
237
modules/apps/bootstrap-all/src/main/resources/application.yml
Normal file
237
modules/apps/bootstrap-all/src/main/resources/application.yml
Normal file
@@ -0,0 +1,237 @@
|
||||
spring:
|
||||
application:
|
||||
name: lingniu-vehicle-ingest
|
||||
threads:
|
||||
virtual:
|
||||
enabled: true
|
||||
|
||||
springdoc:
|
||||
swagger-ui:
|
||||
path: /swagger-ui.html
|
||||
api-docs:
|
||||
path: /v3/api-docs
|
||||
|
||||
server:
|
||||
port: 20000
|
||||
|
||||
lingniu:
|
||||
ingest:
|
||||
gb32960:
|
||||
enabled: true
|
||||
port: ${GB32960_PORT:32960}
|
||||
boss-threads: 1
|
||||
worker-threads: 0 # 0 = Runtime.availableProcessors() * 2
|
||||
# VIN 白名单认证（内存配置）。GB/T 32960 设备没有账号密码，身份由 VIN 决定。
|
||||
auth:
|
||||
enabled: ${GB32960_AUTH_ENABLED:false} # true 启用白名单；false 放行所有 VIN
|
||||
case-sensitive: false # 默认大小写不敏感
|
||||
whitelist: # VIN 列表，未列入的 0x01 登入返回 0x04 VIN_NOT_EXIST
|
||||
# - LTEST000000000001
|
||||
# - LGWEF4A58RE123456
|
||||
# 平台登入 0x05 账号列表（GB/T 32960.3 表 29）。
|
||||
# 列表为空时放行所有平台登入（兼容旧行为），否则 username/password 必须匹配，
|
||||
# 失败返回 0x02 OTHER_ERROR 应答并关闭连接。
|
||||
platforms:
|
||||
# 现代 (XIANDAI)
|
||||
- username: ${GB32960_PLATFORM_USER_1:lingniu}
|
||||
password: ${GB32960_PLATFORM_PWD_1:Lingniu.2024#}
|
||||
allowed-ips: # 可选：限制来源 IP，为空不限制
|
||||
description: 外部下级平台 - 现代 (原旧服务客户端)
|
||||
# 现代平台真实 GB32960 接入账号
|
||||
- username: ${GB32960_PLATFORM_USER_HYUNDAI:Hyundai}
|
||||
password: ${GB32960_PLATFORM_PWD_HYUNDAI:f2e3445d7cda409fb4f278f6fb890734}
|
||||
allowed-ips:
|
||||
- ${GB32960_PLATFORM_IP_HYUNDAI:8.134.95.166}
|
||||
description: 外部下级平台 - Hyundai
|
||||
# 飞驰 (FEICHI)
|
||||
- username: ${GB32960_PLATFORM_USER_2:ln@feichi}
|
||||
password: ${GB32960_PLATFORM_PWD_2:Lingniu.2024#}
|
||||
allowed-ips:
|
||||
description: 外部下级平台 - 飞驰
|
||||
# 海珀特 (HAIPOTE)
|
||||
- username: ${GB32960_PLATFORM_USER_3:hbt@lingniu}
|
||||
password: ${GB32960_PLATFORM_PWD_3:sgmi88b@&9}
|
||||
allowed-ips:
|
||||
description: 外部下级平台 - 海珀特
|
||||
# 跃进 (YUEJIN)
|
||||
- username: ${GB32960_PLATFORM_USER_4:yj01@lingniu}
|
||||
password: ${GB32960_PLATFORM_PWD_4:01234567890123456789}
|
||||
allowed-ips:
|
||||
description: 外部下级平台 - 跃进
|
||||
# TLS 双向认证（可选）。启用后 Netty pipeline 前置 SslHandler 并强制客户端证书。
|
||||
tls:
|
||||
enabled: ${GB32960_TLS_ENABLED:false}
|
||||
cert-path: ${GB32960_TLS_CERT:} # 服务端证书 PEM 路径
|
||||
key-path: ${GB32960_TLS_KEY:} # 服务端私钥 PEM 路径（PKCS#8 未加密）
|
||||
trust-cert-path: ${GB32960_TLS_CA:} # 受信 CA 证书 PEM，用于校验客户端证书
|
||||
require-client-auth: true # 强制双向；false=单向 TLS
|
||||
# ====================================================================
|
||||
# 厂商扩展协议路由（vendor extensions）
|
||||
# 命中规则按 list 顺序首匹（first-match-wins），命中后用对应套件的 parser 集合
|
||||
# 在标准 0x01..0x09 之上叠加。已支持的套件：
|
||||
# - guangdong-fc：广东燃料电池汽车示范规范 0x30~0x34、0x80
|
||||
# 来源：reference/广东燃料电池汽车示范应用城市群综合监管平台-...pdf
|
||||
# ====================================================================
|
||||
vendor-extensions:
|
||||
- name: guangdong-fc
|
||||
match:
|
||||
# 平台账号匹配（精确匹配，case-insensitive）。这里和上面 auth.platforms
|
||||
# 中的 username 保持一致，确保 lingniu 账号登入后所有 0x02 帧都走广东 profile。
|
||||
platform-accounts:
|
||||
- lingniu
|
||||
- Hyundai
|
||||
- ln@feichi
|
||||
# VIN 前缀匹配（startsWith）；空 list = 不参与匹配
|
||||
vin-prefixes: []
|
||||
# VIN 精确匹配
|
||||
vins: []
|
||||
# 报文解析容错。默认启用单块异常隔离：某个信息块解析失败时兜成
|
||||
# InfoBlock.Raw 后继续解析下一个块，不再整帧丢弃。仅在需要严格失败语义
|
||||
# （灰度回滚或抓虫）时设为 false。
|
||||
parse:
|
||||
lenient-block-failure: true
|
||||
diagnostics:
|
||||
max-logged-frame-keys-per-channel: 128
|
||||
jt808:
|
||||
# 1078 信令和 JSATL12 嵌入 JT808 信令都依赖本模块；一体化启动默认开启，
|
||||
# 需要只跑 GB32960 时可用 JT808_ENABLED=false 显式关闭。
|
||||
enabled: ${JT808_ENABLED:true}
|
||||
port: ${JT808_PORT:10808}
|
||||
max-frame-length: ${JT808_MAX_FRAME_LENGTH:1048}
|
||||
jt1078:
|
||||
# 1078 媒体流归档可独立启用；1078 信令事件复用 JT808 mapper，
|
||||
# 只有同时启用 jt808.enabled=true 时才会装配信令桥接 handler。
|
||||
enabled: ${JT1078_ENABLED:true}
|
||||
media-stream:
|
||||
enabled: ${JT1078_MEDIA_STREAM_ENABLED:true}
|
||||
tcp-enabled: ${JT1078_MEDIA_STREAM_TCP_ENABLED:true}
|
||||
udp-enabled: ${JT1078_MEDIA_STREAM_UDP_ENABLED:false}
|
||||
port: ${JT1078_MEDIA_STREAM_PORT:11078}
|
||||
udp-port: ${JT1078_MEDIA_STREAM_UDP_PORT:11078}
|
||||
worker-threads: ${JT1078_MEDIA_STREAM_WORKERS:0}
|
||||
max-packet-length: ${JT1078_MEDIA_STREAM_MAX_PACKET_LENGTH:1048576}
|
||||
segment-seconds: ${JT1078_MEDIA_STREAM_SEGMENT_SECONDS:300}
|
||||
jsatl12:
|
||||
# JSATL12 附件流依赖 ArchiveStore；嵌入的 JT808 信令解析依赖
|
||||
# jt808.enabled=true。缺任一依赖时不会装配附件接入 server。
|
||||
enabled: ${JSATL12_ENABLED:true}
|
||||
port: ${JSATL12_PORT:7612}
|
||||
worker-threads: ${JSATL12_WORKERS:0}
|
||||
max-frame-length: ${JSATL12_MAX_FRAME_LENGTH:65536}
|
||||
file-store: ${JSATL12_FILE_STORE:file:///var/lingniu/alarm-files}
|
||||
mqtt:
|
||||
enabled: ${MQTT_ENABLED:false}
|
||||
endpoints:
|
||||
- name: yutong
|
||||
uri: ${MQTT_URI:ssl://cpxlm.axxc.cn:38883}
|
||||
topic: ${MQTT_TOPIC:/ytforward/shln/+}
|
||||
qos: 2
|
||||
clean-session: false
|
||||
keep-alive-seconds: 20
|
||||
connection-timeout-seconds: 10
|
||||
client-id: ${MQTT_CLIENT_ID:shlnClientId}
|
||||
username: ${MQTT_USER:shln}
|
||||
password: ${MQTT_PWD:}
|
||||
tls:
|
||||
ca-pem: ${MQTT_CA_PEM:/opt/lingniuServices/certificate/yutong/vehicledatareception/ca.crt}
|
||||
client-pem: ${MQTT_CLIENT_PEM:/opt/lingniuServices/certificate/yutong/vehicledatareception/client.crt}
|
||||
client-key: ${MQTT_CLIENT_KEY:/opt/lingniuServices/certificate/yutong/vehicledatareception/client-key-pkcs8.pem}
|
||||
xinda-push:
|
||||
enabled: ${XINDA_PUSH_ENABLED:false}
|
||||
host: ${XINDA_HOST:}
|
||||
port: ${XINDA_PORT:10100}
|
||||
username: ${XINDA_USER:}
|
||||
password: ${XINDA_PWD:}
|
||||
subscribe-msg-ids: ${XINDA_SUBSCRIBE_MSG_IDS:0200,0300,0401}
|
||||
client-desc: ${XINDA_CLIENT_DESC:lingniu-ingest}
|
||||
|
||||
pipeline:
|
||||
disruptor:
|
||||
ring-buffer-size: 131072 # 2^17
|
||||
wait-strategy: yielding # blocking | sleeping | yielding | busy-spin
|
||||
producer-type: multi
|
||||
dedup:
|
||||
enabled: true
|
||||
cache-size: 200000
|
||||
ttl-seconds: 600
|
||||
rate-limit:
|
||||
per-vin-qps: 50
|
||||
|
||||
session:
|
||||
# memory | redis。生产默认 Redis，会话索引用于多实例/重启后的下行命令解析。
|
||||
store: ${SESSION_STORE:redis}
|
||||
ttl: ${SESSION_TTL:30m}
|
||||
|
||||
identity:
|
||||
# memory 适合本地开发；file 是生产最低可用持久化后端，后续可替换 DB/配置中心实现。
|
||||
store: ${VEHICLE_IDENTITY_STORE:file}
|
||||
file:
|
||||
path: ${VEHICLE_IDENTITY_FILE:./data/vehicle-identity.jsonl}
|
||||
|
||||
sink:
|
||||
mq:
|
||||
# MQ Sink 总开关。true=装配 Kafka Producer 并投递事件；false=完全不装配（适合本地开发或 Kafka 不可达）。
|
||||
enabled: ${KAFKA_ENABLED:false}
|
||||
type: kafka # 后端选择：kafka | (rocketmq | pulsar 预留)
|
||||
bootstrap-servers: ${KAFKA_BROKERS:114.55.58.251:9092}
|
||||
compression-type: zstd
|
||||
linger-ms: 20
|
||||
batch-size: 65536
|
||||
acks: all
|
||||
enable-idempotence: true
|
||||
topics:
|
||||
realtime: vehicle.realtime
|
||||
location: vehicle.location
|
||||
alarm: vehicle.alarm
|
||||
session: vehicle.session
|
||||
media-meta: vehicle.media.meta
|
||||
raw-archive: vehicle.raw.archive
|
||||
dlq: vehicle.dlq
|
||||
consumer:
|
||||
# true=启动服务侧 Kafka Envelope 消费 worker；默认绑定 event-history/state/stat 三类 processor。
|
||||
enabled: ${KAFKA_CONSUMER_ENABLED:false}
|
||||
auto-startup: true
|
||||
client-id-prefix: ${KAFKA_CONSUMER_CLIENT_ID_PREFIX:lingniu-envelope-consumer}
|
||||
poll-timeout-millis: ${KAFKA_CONSUMER_POLL_TIMEOUT_MILLIS:1000}
|
||||
loop-backoff-millis: ${KAFKA_CONSUMER_LOOP_BACKOFF_MILLIS:1000}
|
||||
auto-offset-reset: ${KAFKA_CONSUMER_AUTO_OFFSET_RESET:earliest}
|
||||
max-poll-records: ${KAFKA_CONSUMER_MAX_POLL_RECORDS:500}
|
||||
archive:
|
||||
enabled: true
|
||||
type: local # local | s3 | oss
|
||||
path: ./archive/
|
||||
event-file-store:
|
||||
enabled: ${EVENT_FILE_STORE_ENABLED:true}
|
||||
# 32960 历史索引库根路径。这里保存 Parquet 分区和 DuckDB sidecar 索引；
|
||||
# 全字段查询仍会按 rawArchiveUri 回读 archive 根目录下的 .bin 原始包。
|
||||
path: ${EVENT_FILE_STORE_PATH:./target/event-store/}
|
||||
zone-id: ${EVENT_FILE_STORE_ZONE_ID:Asia/Shanghai}
|
||||
batch-size: ${EVENT_FILE_STORE_BATCH_SIZE:500}
|
||||
flush-interval-millis: ${EVENT_FILE_STORE_FLUSH_INTERVAL_MILLIS:1000}
|
||||
event-history:
|
||||
enabled: ${EVENT_HISTORY_ENABLED:true}
|
||||
vehicle-state:
|
||||
enabled: ${VEHICLE_STATE_ENABLED:false}
|
||||
vehicle-stat:
|
||||
enabled: ${VEHICLE_STAT_ENABLED:true}
|
||||
file-path: ${VEHICLE_STAT_FILE_PATH:./target/vehicle-stat/}
|
||||
zone-id: ${VEHICLE_STAT_ZONE_ID:Asia/Shanghai}
|
||||
|
||||
management:
|
||||
endpoints:
|
||||
web:
|
||||
exposure:
|
||||
include: health,info,metrics,prometheus,env
|
||||
metrics:
|
||||
tags:
|
||||
application: lingniu-vehicle-ingest
|
||||
|
||||
logging:
|
||||
level:
|
||||
root: INFO
|
||||
com.lingniu.ingest: INFO
|
||||
com.lingniu.ingest.protocol.gb32960.inbound.Gb32960ChannelHandler: INFO
|
||||
com.lingniu.ingest.protocol.gb32960.codec.Gb32960FrameDecoder: INFO
|
||||
com.lingniu.ingest.protocol.gb32960.codec.Gb32960BodyParser: INFO
|
||||
# 信达 push 诊断：打开下面一行可以看到每一条收发的原始字节 hex dump
|
||||
com.lingniu.ingest.inbound.xinda: INFO
|
||||
@@ -0,0 +1,134 @@
|
||||
package com.lingniu.ingest.bootstrap;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.beans.factory.config.YamlPropertiesFactoryBean;
|
||||
import org.springframework.core.io.ClassPathResource;
|
||||
|
||||
import java.util.Properties;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
class BootstrapApplicationDefaultsTest {
|
||||
|
||||
@Test
|
||||
void defaultConfigEnablesJt808WhenDependentProtocolsAreEnabledByDefault() {
|
||||
Properties props = loadApplicationProperties();
|
||||
|
||||
assertThat(defaultEnabled(props.getProperty("lingniu.ingest.jsatl12.enabled"))).isTrue();
|
||||
assertThat(defaultEnabled(props.getProperty("lingniu.ingest.jt1078.enabled"))).isTrue();
|
||||
assertThat(defaultEnabled(props.getProperty("lingniu.ingest.jt808.enabled")))
|
||||
.as("JSATL12 and JT1078 signaling depend on JT808 decoder/mapper, so bootstrap defaults must not disable JT808")
|
||||
.isTrue();
|
||||
}
|
||||
|
||||
@Test
|
||||
void productionProtocolEntrypointsAreControlledByEnvironmentFlags() {
|
||||
Properties props = loadApplicationProperties();
|
||||
|
||||
assertThat(props.getProperty("lingniu.ingest.jt808.enabled")).isEqualTo("${JT808_ENABLED:true}");
|
||||
assertThat(props.getProperty("lingniu.ingest.jt1078.enabled")).isEqualTo("${JT1078_ENABLED:true}");
|
||||
assertThat(props.getProperty("lingniu.ingest.jsatl12.enabled")).isEqualTo("${JSATL12_ENABLED:true}");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.enabled")).isEqualTo("${MQTT_ENABLED:false}");
|
||||
assertThat(props.getProperty("lingniu.ingest.xinda-push.enabled")).isEqualTo("${XINDA_PUSH_ENABLED:false}");
|
||||
}
|
||||
|
||||
@Test
|
||||
void defaultConfigKeepsLegacyProductionIngressPortsAndYutongMqttShape() {
|
||||
Properties props = loadApplicationProperties();
|
||||
|
||||
assertThat(props.getProperty("lingniu.ingest.gb32960.port")).isEqualTo("${GB32960_PORT:32960}");
|
||||
assertThat(props.getProperty("lingniu.ingest.jt808.port")).isEqualTo("${JT808_PORT:10808}");
|
||||
assertThat(props.getProperty("lingniu.ingest.jsatl12.port")).isEqualTo("${JSATL12_PORT:7612}");
|
||||
assertThat(props.getProperty("lingniu.ingest.jt1078.media-stream.port"))
|
||||
.isEqualTo("${JT1078_MEDIA_STREAM_PORT:11078}");
|
||||
assertThat(props.getProperty("lingniu.ingest.jt1078.media-stream.udp-port"))
|
||||
.isEqualTo("${JT1078_MEDIA_STREAM_UDP_PORT:11078}");
|
||||
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].name")).isEqualTo("yutong");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].uri"))
|
||||
.isEqualTo("${MQTT_URI:ssl://cpxlm.axxc.cn:38883}");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].topic"))
|
||||
.isEqualTo("${MQTT_TOPIC:/ytforward/shln/+}");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].qos")).isEqualTo("2");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].clean-session")).isEqualTo("false");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].keep-alive-seconds")).isEqualTo("20");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].connection-timeout-seconds")).isEqualTo("10");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].client-id"))
|
||||
.isEqualTo("${MQTT_CLIENT_ID:shlnClientId}");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].username"))
|
||||
.isEqualTo("${MQTT_USER:shln}");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].password"))
|
||||
.isEqualTo("${MQTT_PWD:}");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].tls.ca-pem"))
|
||||
.isEqualTo("${MQTT_CA_PEM:/opt/lingniuServices/certificate/yutong/vehicledatareception/ca.crt}");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].tls.client-pem"))
|
||||
.isEqualTo("${MQTT_CLIENT_PEM:/opt/lingniuServices/certificate/yutong/vehicledatareception/client.crt}");
|
||||
assertThat(props.getProperty("lingniu.ingest.mqtt.endpoints[0].tls.client-key"))
|
||||
.isEqualTo("${MQTT_CLIENT_KEY:/opt/lingniuServices/certificate/yutong/vehicledatareception/client-key-pkcs8.pem}");
|
||||
|
||||
assertThat(props.getProperty("lingniu.ingest.sink.mq.bootstrap-servers"))
|
||||
.isEqualTo("${KAFKA_BROKERS:114.55.58.251:9092}");
|
||||
assertThat(props.getProperty("lingniu.ingest.session.store")).isEqualTo("${SESSION_STORE:redis}");
|
||||
assertThat(props.getProperty("lingniu.ingest.session.ttl")).isEqualTo("${SESSION_TTL:30m}");
|
||||
}
|
||||
|
||||
@Test
|
||||
void productionEventHistoryQueryApiIsEnabledByDefault() {
|
||||
Properties props = loadApplicationProperties();
|
||||
|
||||
assertThat(props.getProperty("lingniu.ingest.event-history.enabled"))
|
||||
.isEqualTo("${EVENT_HISTORY_ENABLED:true}");
|
||||
}
|
||||
|
||||
@Test
|
||||
void productionVehicleStatisticsAreEnabledByDefault() {
|
||||
Properties props = loadApplicationProperties();
|
||||
|
||||
assertThat(props.getProperty("lingniu.ingest.vehicle-stat.enabled"))
|
||||
.isEqualTo("${VEHICLE_STAT_ENABLED:true}");
|
||||
assertThat(props.getProperty("lingniu.ingest.vehicle-stat.file-path"))
|
||||
.isEqualTo("${VEHICLE_STAT_FILE_PATH:./target/vehicle-stat/}");
|
||||
assertThat(props.getProperty("lingniu.ingest.vehicle-stat.zone-id"))
|
||||
.isEqualTo("${VEHICLE_STAT_ZONE_ID:Asia/Shanghai}");
|
||||
}
|
||||
|
||||
@Test
|
||||
void swaggerUiAndOpenApiDocsUseStableDefaultPaths() {
|
||||
Properties props = loadApplicationProperties();
|
||||
|
||||
assertThat(props.getProperty("springdoc.swagger-ui.path")).isEqualTo("/swagger-ui.html");
|
||||
assertThat(props.getProperty("springdoc.api-docs.path")).isEqualTo("/v3/api-docs");
|
||||
}
|
||||
|
||||
@Test
|
||||
void hyundaiPlatformUsesGuangdongFuelCellVendorProfileByDefault() {
|
||||
Properties props = loadApplicationProperties();
|
||||
|
||||
assertThat(props.getProperty("lingniu.ingest.gb32960.vendor-extensions[0].name"))
|
||||
.isEqualTo("guangdong-fc");
|
||||
assertThat(props.getProperty("lingniu.ingest.gb32960.vendor-extensions[0].match.platform-accounts[1]"))
|
||||
.isEqualTo("Hyundai");
|
||||
}
|
||||
|
||||
private static Properties loadApplicationProperties() {
|
||||
YamlPropertiesFactoryBean yaml = new YamlPropertiesFactoryBean();
|
||||
yaml.setResources(new ClassPathResource("application.yml"));
|
||||
Properties props = yaml.getObject();
|
||||
assertThat(props).isNotNull();
|
||||
return props;
|
||||
}
|
||||
|
||||
private static boolean defaultEnabled(String value) {
|
||||
if (value == null) {
|
||||
return false;
|
||||
}
|
||||
String normalized = value.trim();
|
||||
if ("true".equalsIgnoreCase(normalized)) {
|
||||
return true;
|
||||
}
|
||||
if ("false".equalsIgnoreCase(normalized)) {
|
||||
return false;
|
||||
}
|
||||
return normalized.matches("\\$\\{[^:}]+:true}");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
package com.lingniu.ingest.bootstrap;
|
||||
|
||||
import com.lingniu.ingest.core.config.IngestCoreAutoConfiguration;
|
||||
import com.lingniu.ingest.identity.config.VehicleIdentityAutoConfiguration;
|
||||
import com.lingniu.ingest.inbound.mqtt.client.MqttEndpointManager;
|
||||
import com.lingniu.ingest.inbound.mqtt.config.MqttInboundAutoConfiguration;
|
||||
import com.lingniu.ingest.inbound.xinda.XindaPushClient;
|
||||
import com.lingniu.ingest.inbound.xinda.config.XindaPushAutoConfiguration;
|
||||
import com.lingniu.ingest.protocol.jsatl12.config.Jsatl12AutoConfiguration;
|
||||
import com.lingniu.ingest.protocol.jsatl12.inbound.Jsatl12NettyServer;
|
||||
import com.lingniu.ingest.protocol.jt1078.config.Jt1078AutoConfiguration;
|
||||
import com.lingniu.ingest.protocol.jt1078.config.Jt1078SignalAutoConfiguration;
|
||||
import com.lingniu.ingest.protocol.jt1078.handler.Jt1078MediaSignalHandler;
|
||||
import com.lingniu.ingest.protocol.jt1078.media.Jt1078MediaArchiveService;
|
||||
import com.lingniu.ingest.protocol.jt1078.media.Jt1078MediaStreamServer;
|
||||
import com.lingniu.ingest.protocol.jt808.config.Jt808AutoConfiguration;
|
||||
import com.lingniu.ingest.protocol.jt808.inbound.Jt808NettyServer;
|
||||
import com.lingniu.ingest.session.config.SessionCoreAutoConfiguration;
|
||||
import com.lingniu.ingest.sink.archive.config.SinkArchiveAutoConfiguration;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
import org.springframework.boot.autoconfigure.AutoConfigurations;
|
||||
import org.springframework.boot.test.context.runner.ApplicationContextRunner;
|
||||
|
||||
import java.nio.file.Path;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
class ProtocolAutoConfigurationCompositionTest {
|
||||
|
||||
@TempDir
|
||||
Path archiveRoot;
|
||||
|
||||
@Test
|
||||
void nonGb32960ProtocolAdaptersCanBeComposedWithoutExternalConnections() {
|
||||
new ApplicationContextRunner()
|
||||
.withConfiguration(AutoConfigurations.of(
|
||||
IngestCoreAutoConfiguration.class,
|
||||
SessionCoreAutoConfiguration.class,
|
||||
VehicleIdentityAutoConfiguration.class,
|
||||
SinkArchiveAutoConfiguration.class,
|
||||
Jt1078AutoConfiguration.class,
|
||||
Jt808AutoConfiguration.class,
|
||||
Jt1078SignalAutoConfiguration.class,
|
||||
Jsatl12AutoConfiguration.class,
|
||||
MqttInboundAutoConfiguration.class,
|
||||
XindaPushAutoConfiguration.class))
|
||||
.withPropertyValues(
|
||||
"lingniu.ingest.sink.archive.enabled=true",
|
||||
"lingniu.ingest.sink.archive.type=local",
|
||||
"lingniu.ingest.sink.archive.path=" + archiveRoot,
|
||||
"lingniu.ingest.jt808.enabled=true",
|
||||
"lingniu.ingest.jt808.port=0",
|
||||
"lingniu.ingest.jt1078.enabled=true",
|
||||
"lingniu.ingest.jt1078.media-stream.enabled=true",
|
||||
"lingniu.ingest.jt1078.media-stream.tcp-enabled=true",
|
||||
"lingniu.ingest.jt1078.media-stream.udp-enabled=false",
|
||||
"lingniu.ingest.jt1078.media-stream.port=0",
|
||||
"lingniu.ingest.jsatl12.enabled=true",
|
||||
"lingniu.ingest.jsatl12.port=0",
|
||||
"lingniu.ingest.mqtt.enabled=true",
|
||||
"lingniu.ingest.xinda-push.enabled=true")
|
||||
.run(context -> {
|
||||
assertThat(context).hasSingleBean(Jt808NettyServer.class);
|
||||
assertThat(context).hasSingleBean(Jt1078MediaSignalHandler.class);
|
||||
assertThat(context).hasSingleBean(Jt1078MediaArchiveService.class);
|
||||
assertThat(context).hasSingleBean(Jt1078MediaStreamServer.class);
|
||||
assertThat(context).hasSingleBean(Jsatl12NettyServer.class);
|
||||
assertThat(context).hasSingleBean(MqttEndpointManager.class);
|
||||
assertThat(context).hasSingleBean(XindaPushClient.class);
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,174 @@
|
||||
package com.lingniu.ingest.bootstrap;
|
||||
|
||||
import com.lingniu.ingest.codec.BccChecksum;
|
||||
import com.lingniu.ingest.protocol.gb32960.inbound.Gb32960NettyServer;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.junit.jupiter.api.io.TempDir;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.test.context.DynamicPropertyRegistry;
|
||||
import org.springframework.test.context.DynamicPropertySource;
|
||||
|
||||
import java.io.ByteArrayOutputStream;
|
||||
import java.net.Socket;
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.nio.file.Files;
|
||||
import java.nio.file.Path;
|
||||
import java.time.ZoneOffset;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.time.Instant;
|
||||
import java.util.List;
|
||||
import java.util.stream.Stream;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
/**
|
||||
* 端到端验证原始报文冷存链路：
|
||||
*
|
||||
* <pre>
|
||||
* TCP → Gb32960 Netty → Dispatcher 发 RawArchive 事件
|
||||
* → Disruptor 虚拟线程 → ArchiveEventSink.publish
|
||||
* → LocalArchiveStore.put → 文件系统落盘
|
||||
* </pre>
|
||||
*
|
||||
* <p>与 {@link IngestEndToEndTest} 互补：前者验证业务事件路径；这里验证 raw archive
|
||||
* 路径。两个测试跑在独立 Spring 上下文里（archive 是否启用的配置不同）。
|
||||
*/
|
||||
@SpringBootTest(
|
||||
classes = IngestApplication.class,
|
||||
webEnvironment = SpringBootTest.WebEnvironment.NONE,
|
||||
properties = {
|
||||
"lingniu.ingest.gb32960.enabled=true",
|
||||
"lingniu.ingest.gb32960.port=0",
|
||||
"lingniu.ingest.jt808.enabled=false",
|
||||
"lingniu.ingest.jt1078.enabled=false",
|
||||
"lingniu.ingest.jsatl12.enabled=false",
|
||||
"lingniu.ingest.mqtt.enabled=false",
|
||||
"lingniu.ingest.xinda-push.enabled=false",
|
||||
"lingniu.ingest.command-gateway.enabled=false",
|
||||
"lingniu.ingest.sink.mq.enabled=false",
|
||||
// 启用 archive sink，路径由 @DynamicPropertySource 注入
|
||||
"lingniu.ingest.sink.archive.enabled=true",
|
||||
"lingniu.ingest.sink.archive.type=local"
|
||||
})
|
||||
class RawArchiveEndToEndTest {
|
||||
|
||||
@TempDir
|
||||
static Path archiveRoot;
|
||||
|
||||
@DynamicPropertySource
|
||||
static void properties(DynamicPropertyRegistry registry) {
|
||||
registry.add("lingniu.ingest.sink.archive.path", archiveRoot::toString);
|
||||
}
|
||||
|
||||
@Autowired
|
||||
private Gb32960NettyServer server;
|
||||
|
||||
@Test
|
||||
void gb32960FrameIsArchivedToDisk() throws Exception {
|
||||
int port = server.getBoundPort();
|
||||
assertThat(port).isGreaterThan(0);
|
||||
|
||||
String vin = "LTEST0000ARCH0001";
|
||||
byte[] frame = buildRealtimeFrame(vin);
|
||||
|
||||
try (Socket socket = new Socket("127.0.0.1", port)) {
|
||||
socket.getOutputStream().write(frame);
|
||||
socket.getOutputStream().flush();
|
||||
}
|
||||
|
||||
// 预期落盘目录：<root>/yyyy/MM/dd/GB32960/<vin>/
|
||||
// ingestTime 在 Dispatcher 里用 receivedAt = Instant.now()，跨天边界理论上今天/昨天两个候选都可能命中，
|
||||
// 为避免边界脆弱，扫描 root 下所有 .bin 文件匹配 vin。
|
||||
Path file = awaitArchivedFile(archiveRoot, vin, 3000);
|
||||
assertThat(file)
|
||||
.as("expected an archived frame for vin=%s under %s within 3s", vin, archiveRoot)
|
||||
.isNotNull();
|
||||
assertThat(Files.readAllBytes(file)).containsExactly(frame);
|
||||
|
||||
// 额外：key 的日期分片应是 ingestTime 的 UTC 当天（允许跨天时两者都可接受）
|
||||
String today = DateTimeFormatter.ofPattern("yyyy/MM/dd").withZone(ZoneOffset.UTC).format(Instant.now());
|
||||
String yesterday = DateTimeFormatter.ofPattern("yyyy/MM/dd").withZone(ZoneOffset.UTC)
|
||||
.format(Instant.now().minusSeconds(86400));
|
||||
String rel = archiveRoot.relativize(file).toString().replace('\\', '/');
|
||||
assertThat(rel).satisfiesAnyOf(
|
||||
r -> assertThat(r).startsWith(today + "/GB32960/" + vin + "/"),
|
||||
r -> assertThat(r).startsWith(yesterday + "/GB32960/" + vin + "/"));
|
||||
assertThat(rel).endsWith(".bin");
|
||||
}
|
||||
|
||||
private static Path awaitArchivedFile(Path root, String vin, long timeoutMs) throws Exception {
|
||||
long deadline = System.currentTimeMillis() + timeoutMs;
|
||||
while (System.currentTimeMillis() < deadline) {
|
||||
Path hit = findFirst(root, vin);
|
||||
if (hit != null) return hit;
|
||||
Thread.sleep(50);
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
private static Path findFirst(Path root, String vin) throws Exception {
|
||||
if (!Files.exists(root)) return null;
|
||||
try (Stream<Path> walk = Files.walk(root)) {
|
||||
List<Path> hits = walk
|
||||
.filter(Files::isRegularFile)
|
||||
.filter(p -> p.toString().endsWith(".bin"))
|
||||
.filter(p -> p.toString().contains("/" + vin + "/") || p.toString().contains("\\" + vin + "\\"))
|
||||
.toList();
|
||||
return hits.isEmpty() ? null : hits.get(0);
|
||||
}
|
||||
}
|
||||
|
||||
// ===== frame builder (inlined copy of IngestEndToEndTest#buildRealtimeFrame) =====
|
||||
|
||||
private static byte[] buildRealtimeFrame(String vin) {
|
||||
ByteArrayOutputStream body = new ByteArrayOutputStream();
|
||||
body.write(24); body.write(1); body.write(2); body.write(3); body.write(4); body.write(5);
|
||||
|
||||
body.write(0x01);
|
||||
body.write(0x01); body.write(0x03); body.write(0x01);
|
||||
writeU16(body, 523);
|
||||
writeU32(body, 1234567);
|
||||
writeU16(body, 6000);
|
||||
writeU16(body, 1100);
|
||||
body.write(70);
|
||||
body.write(0x01);
|
||||
body.write(0x0F);
|
||||
writeU16(body, 500);
|
||||
body.write(30);
|
||||
body.write(0);
|
||||
|
||||
body.write(0x05);
|
||||
body.write(0x00);
|
||||
writeU32(body, 116_397_128L);
|
||||
writeU32(body, 39_916_527L);
|
||||
|
||||
byte[] bodyBytes = body.toByteArray();
|
||||
|
||||
ByteArrayOutputStream out = new ByteArrayOutputStream();
|
||||
out.write(0x23); out.write(0x23);
|
||||
out.write(0x02);
|
||||
out.write(0xFE);
|
||||
byte[] vinBytes = vin.getBytes(StandardCharsets.US_ASCII);
|
||||
out.write(vinBytes, 0, 17);
|
||||
out.write(0x01);
|
||||
writeU16(out, bodyBytes.length);
|
||||
out.write(bodyBytes, 0, bodyBytes.length);
|
||||
byte[] almost = out.toByteArray();
|
||||
byte bcc = BccChecksum.compute(almost, 2, almost.length - 2);
|
||||
out.write(bcc & 0xFF);
|
||||
return out.toByteArray();
|
||||
}
|
||||
|
||||
private static void writeU16(ByteArrayOutputStream os, int v) {
|
||||
os.write((v >> 8) & 0xFF);
|
||||
os.write(v & 0xFF);
|
||||
}
|
||||
|
||||
private static void writeU32(ByteArrayOutputStream os, long v) {
|
||||
os.write((int) ((v >> 24) & 0xFF));
|
||||
os.write((int) ((v >> 16) & 0xFF));
|
||||
os.write((int) ((v >> 8) & 0xFF));
|
||||
os.write((int) (v & 0xFF));
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,7 @@
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>lingniu-vehicle-ingest</artifactId>
|
||||
<version>0.1.0-SNAPSHOT</version>
|
||||
<relativePath>../../../pom.xml</relativePath>
|
||||
</parent>
|
||||
<artifactId>command-gateway</artifactId>
|
||||
<name>command-gateway</name>
|
||||
@@ -26,6 +27,10 @@
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>protocol-jt808</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>protocol-jt1078</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-web</artifactId>
|
||||
@@ -34,5 +39,15 @@
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-validation</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.junit.jupiter</groupId>
|
||||
<artifactId>junit-jupiter</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.assertj</groupId>
|
||||
<artifactId>assertj-core</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</project>
|
||||
@@ -11,4 +11,5 @@ import org.springframework.web.servlet.DispatcherServlet;
|
||||
@ConditionalOnProperty(prefix = "lingniu.ingest.command-gateway", name = "enabled", havingValue = "true", matchIfMissing = true)
|
||||
@ComponentScan(basePackageClasses = TerminalCommandController.class)
|
||||
public class CommandGatewayAutoConfiguration {
|
||||
// 只扫描 REST 下行命令网关；不参与 32960 接收、RAW 冷存或历史查询链路。
|
||||
}
|
||||
@@ -0,0 +1,281 @@
|
||||
package com.lingniu.ingest.gateway;
|
||||
|
||||
import com.lingniu.ingest.protocol.jt808.downlink.Jt808Commands;
|
||||
import com.lingniu.ingest.protocol.jt808.model.Jt808Message;
|
||||
import com.lingniu.ingest.protocol.jt1078.downlink.Jt1078Commands;
|
||||
import com.lingniu.ingest.session.CommandDispatcher;
|
||||
import com.lingniu.ingest.session.DeviceSession;
|
||||
import com.lingniu.ingest.session.SessionStore;
|
||||
import org.springframework.web.bind.annotation.DeleteMapping;
|
||||
import org.springframework.web.bind.annotation.GetMapping;
|
||||
import org.springframework.web.bind.annotation.PostMapping;
|
||||
import org.springframework.web.bind.annotation.PutMapping;
|
||||
import org.springframework.web.bind.annotation.RequestBody;
|
||||
import org.springframework.web.bind.annotation.RequestMapping;
|
||||
import org.springframework.web.bind.annotation.RequestParam;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
|
||||
import java.nio.charset.StandardCharsets;
|
||||
import java.time.Duration;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.CompletableFuture;
|
||||
import java.util.concurrent.CompletionException;
|
||||
import java.util.stream.Collectors;
|
||||
|
||||
/**
|
||||
* 下行命令 REST 入口。命令通过 {@link CommandDispatcher} 发往协议模块，
|
||||
* 由 protocol-jt808 的 {@code Jt808CommandDispatcher} 实际发送到终端。
|
||||
*
|
||||
* <p>路径参数 {@code clientId} 可以是 sessionId / phone / vin，
|
||||
* 由 {@link SessionStore} 三索引解析。
|
||||
*/
|
||||
@RestController
|
||||
@RequestMapping("/terminal")
|
||||
public class TerminalCommandController {
|
||||
|
||||
private static final Duration DEFAULT_TIMEOUT = Duration.ofSeconds(20);
|
||||
|
||||
private final SessionStore sessions;
|
||||
private final CommandDispatcher dispatcher;
|
||||
|
||||
public TerminalCommandController(SessionStore sessions, CommandDispatcher dispatcher) {
|
||||
this.sessions = sessions;
|
||||
this.dispatcher = dispatcher;
|
||||
}
|
||||
|
||||
@GetMapping("/session")
|
||||
public CommandResult session(@RequestParam String clientId) {
|
||||
Optional<DeviceSession> s = resolveSession(clientId);
|
||||
return s.<CommandResult>map(CommandResult::ok)
|
||||
.orElseGet(() -> CommandResult.failure(404, "session not found: " + clientId));
|
||||
}
|
||||
|
||||
@GetMapping("/location")
|
||||
public CommandResult queryLocation(@RequestParam String clientId) {
|
||||
return awaitSync(clientId, Jt808Commands.queryLocation(), "query-location");
|
||||
}
|
||||
|
||||
@GetMapping("/parameters")
|
||||
public CommandResult queryParameters(@RequestParam String clientId) {
|
||||
return awaitSync(clientId, Jt808Commands.queryParameters(), "query-parameters");
|
||||
}
|
||||
|
||||
@PutMapping("/parameters")
|
||||
public CommandResult setParameters(@RequestParam String clientId,
|
||||
@RequestBody Map<String, String> body) {
|
||||
List<Jt808Commands.ParamItem> items = body.entrySet().stream()
|
||||
.map(e -> new Jt808Commands.ParamItem(
|
||||
parseId(e.getKey()),
|
||||
e.getValue() == null ? new byte[0] : e.getValue().getBytes(StandardCharsets.UTF_8)))
|
||||
.collect(Collectors.toList());
|
||||
return awaitSync(clientId, Jt808Commands.setParameters(items), "set-parameters");
|
||||
}
|
||||
|
||||
@PostMapping("/control")
|
||||
public CommandResult terminalControl(@RequestParam String clientId,
|
||||
@RequestBody Map<String, Object> body) {
|
||||
int word = Integer.parseInt(body.getOrDefault("command", "0").toString());
|
||||
String params = body.getOrDefault("params", "").toString();
|
||||
return awaitSync(clientId, Jt808Commands.terminalControl(word, params), "terminal-control");
|
||||
}
|
||||
|
||||
@PostMapping("/ack")
|
||||
public CommandResult ack(@RequestParam String clientId, @RequestBody Map<String, Integer> body) {
|
||||
int ackSerial = body.getOrDefault("ackSerial", 0);
|
||||
int ackMsgId = body.getOrDefault("ackMsgId", 0);
|
||||
int result = body.getOrDefault("result", 0);
|
||||
return fireAndForget(clientId, Jt808Commands.platformAck(ackSerial, ackMsgId, result), "platform-ack");
|
||||
}
|
||||
|
||||
@GetMapping("/attributes")
|
||||
public CommandResult attributes(@RequestParam String clientId) {
|
||||
return awaitSync(clientId, Jt808Commands.queryAttributes(), "query-attributes");
|
||||
}
|
||||
|
||||
@DeleteMapping("/area")
|
||||
public CommandResult deleteArea(@RequestParam String clientId, @RequestParam String ids) {
|
||||
return awaitSync(clientId, Jt808Commands.deleteArea(parseIds(ids)), "delete-area");
|
||||
}
|
||||
|
||||
@PostMapping("/alarm_ack")
|
||||
public CommandResult alarmAck(@RequestParam String clientId, @RequestBody Map<String, Object> body) {
|
||||
int responseSerialNo = Integer.parseInt(body.getOrDefault("responseSerialNo", "0").toString());
|
||||
long type = parseLong(body.getOrDefault("type", "0").toString());
|
||||
return fireAndForget(clientId, Jt808Commands.alarmAck(responseSerialNo, type), "alarm-ack");
|
||||
}
|
||||
|
||||
@GetMapping("/jt1078/attributes")
|
||||
public CommandResult jt1078Attributes(@RequestParam String clientId) {
|
||||
return awaitSync(clientId, Jt1078Commands.queryMediaAttributes(), "jt1078-query-media-attributes");
|
||||
}
|
||||
|
||||
@PostMapping("/jt1078/realtime")
|
||||
public CommandResult jt1078RealtimePlay(@RequestParam String clientId,
|
||||
@RequestBody Map<String, Object> body) {
|
||||
return awaitSync(clientId, Jt1078Commands.realtimePlay(
|
||||
stringValue(body, "ip", ""),
|
||||
intValue(body, "tcpPort", 0),
|
||||
intValue(body, "udpPort", 0),
|
||||
intValue(body, "channelNo", 0),
|
||||
intValue(body, "mediaType", 0),
|
||||
intValue(body, "streamType", 0)), "jt1078-realtime-play");
|
||||
}
|
||||
|
||||
@PostMapping("/jt1078/realtime_control")
|
||||
public CommandResult jt1078RealtimeControl(@RequestParam String clientId,
|
||||
@RequestBody Map<String, Object> body) {
|
||||
return awaitSync(clientId, Jt1078Commands.realtimeControl(
|
||||
intValue(body, "channelNo", 0),
|
||||
intValue(body, "command", 0),
|
||||
intValue(body, "closeType", 0),
|
||||
intValue(body, "streamType", 0)), "jt1078-realtime-control");
|
||||
}
|
||||
|
||||
@PostMapping("/jt1078/history")
|
||||
public CommandResult jt1078HistoryPlay(@RequestParam String clientId,
|
||||
@RequestBody Map<String, Object> body) {
|
||||
return awaitSync(clientId, Jt1078Commands.historyPlay(
|
||||
stringValue(body, "ip", ""),
|
||||
intValue(body, "tcpPort", 0),
|
||||
intValue(body, "udpPort", 0),
|
||||
intValue(body, "channelNo", 0),
|
||||
intValue(body, "mediaType", 0),
|
||||
intValue(body, "streamType", 0),
|
||||
intValue(body, "storageType", 0),
|
||||
intValue(body, "playbackMode", 0),
|
||||
intValue(body, "playbackSpeed", 0),
|
||||
stringValue(body, "startTime", ""),
|
||||
stringValue(body, "endTime", "")), "jt1078-history-play");
|
||||
}
|
||||
|
||||
@PostMapping("/jt1078/history_control")
|
||||
public CommandResult jt1078HistoryControl(@RequestParam String clientId,
|
||||
@RequestBody Map<String, Object> body) {
|
||||
return awaitSync(clientId, Jt1078Commands.historyControl(
|
||||
intValue(body, "channelNo", 0),
|
||||
intValue(body, "playbackMode", 0),
|
||||
intValue(body, "playbackSpeed", 0),
|
||||
stringValue(body, "playbackTime", "")), "jt1078-history-control");
|
||||
}
|
||||
|
||||
@PostMapping("/jt1078/resource_search")
|
||||
public CommandResult jt1078ResourceSearch(@RequestParam String clientId,
|
||||
@RequestBody Map<String, Object> body) {
|
||||
return awaitSync(clientId, Jt1078Commands.resourceSearch(
|
||||
intValue(body, "channelNo", 0),
|
||||
stringValue(body, "startTime", ""),
|
||||
stringValue(body, "endTime", ""),
|
||||
longValue(body, "warnBit1", 0),
|
||||
longValue(body, "warnBit2", 0),
|
||||
intValue(body, "mediaType", 0),
|
||||
intValue(body, "streamType", 0),
|
||||
intValue(body, "storageType", 0)), "jt1078-resource-search");
|
||||
}
|
||||
|
||||
@PostMapping("/jt1078/file_upload")
|
||||
public CommandResult jt1078FileUpload(@RequestParam String clientId,
|
||||
@RequestBody Map<String, Object> body) {
|
||||
return awaitSync(clientId, Jt1078Commands.fileUpload(
|
||||
stringValue(body, "ip", ""),
|
||||
intValue(body, "port", 0),
|
||||
stringValue(body, "username", ""),
|
||||
stringValue(body, "password", ""),
|
||||
stringValue(body, "path", ""),
|
||||
intValue(body, "channelNo", 0),
|
||||
stringValue(body, "startTime", ""),
|
||||
stringValue(body, "endTime", ""),
|
||||
longValue(body, "warnBit1", 0),
|
||||
longValue(body, "warnBit2", 0),
|
||||
intValue(body, "mediaType", 0),
|
||||
intValue(body, "streamType", 0),
|
||||
intValue(body, "storageType", 0),
|
||||
intValue(body, "condition", 0)), "jt1078-file-upload");
|
||||
}
|
||||
|
||||
@PostMapping("/jt1078/file_upload_control")
|
||||
public CommandResult jt1078FileUploadControl(@RequestParam String clientId,
|
||||
@RequestBody Map<String, Object> body) {
|
||||
return awaitSync(clientId, Jt1078Commands.fileUploadControl(
|
||||
intValue(body, "responseSerialNo", 0),
|
||||
intValue(body, "command", 0)), "jt1078-file-upload-control");
|
||||
}
|
||||
|
||||
// ===== helpers =====
|
||||
|
||||
private Optional<DeviceSession> resolveSession(String clientId) {
|
||||
// 运维侧常拿到 VIN、手机号或 sessionId 中任意一个；这里按三索引兜底解析到在线连接。
|
||||
return sessions.findBySessionId(clientId)
|
||||
.or(() -> sessions.findByPhone(clientId))
|
||||
.or(() -> sessions.findByVin(clientId));
|
||||
}
|
||||
|
||||
private CommandResult awaitSync(String clientId, Jt808Commands.DownlinkCommand cmd, String op) {
|
||||
// 需要终端应答的命令走 request-response，超时时间由网关统一兜底，避免 HTTP 线程无限等待。
|
||||
CompletableFuture<Jt808Message> future =
|
||||
dispatcher.request(clientId, cmd, Jt808Message.class, DEFAULT_TIMEOUT);
|
||||
try {
|
||||
Jt808Message resp = future.join();
|
||||
return CommandResult.ok(Map.of(
|
||||
"messageId", "0x" + Integer.toHexString(resp.header().messageId()),
|
||||
"serial", resp.header().serialNo(),
|
||||
"phone", resp.header().phone()));
|
||||
} catch (CompletionException e) {
|
||||
Throwable cause = e.getCause() != null ? e.getCause() : e;
|
||||
return CommandResult.failure(500, op + " failed: " + cause.getMessage());
|
||||
} catch (Exception e) {
|
||||
return CommandResult.failure(500, op + " failed: " + e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
private CommandResult fireAndForget(String clientId, Jt808Commands.DownlinkCommand cmd, String op) {
|
||||
try {
|
||||
// ACK 类命令不等待终端回包；只确认写入在线 session 对应的 channel。
|
||||
dispatcher.notify(clientId, cmd).join();
|
||||
return CommandResult.ok(Map.of("op", op));
|
||||
} catch (Exception e) {
|
||||
return CommandResult.failure(500, op + " failed: " + e.getMessage());
|
||||
}
|
||||
}
|
||||
|
||||
private static long parseId(String key) {
|
||||
if (key == null || key.isBlank()) return 0;
|
||||
String k = key.startsWith("0x") ? key.substring(2) : key;
|
||||
return Long.parseLong(k, 16);
|
||||
}
|
||||
|
||||
private static List<Long> parseIds(String ids) {
|
||||
if (ids == null || ids.isBlank()) {
|
||||
return List.of();
|
||||
}
|
||||
return java.util.Arrays.stream(ids.split(","))
|
||||
.map(String::trim)
|
||||
.filter(s -> !s.isBlank())
|
||||
.map(TerminalCommandController::parseLong)
|
||||
.toList();
|
||||
}
|
||||
|
||||
private static long parseLong(String value) {
|
||||
if (value == null || value.isBlank()) return 0;
|
||||
String v = value.startsWith("0x") || value.startsWith("0X") ? value.substring(2) : value;
|
||||
return value.startsWith("0x") || value.startsWith("0X")
|
||||
? Long.parseLong(v, 16)
|
||||
: Long.parseLong(v);
|
||||
}
|
||||
|
||||
private static String stringValue(Map<String, Object> body, String key, String defaultValue) {
|
||||
Object value = body.get(key);
|
||||
return value == null ? defaultValue : value.toString();
|
||||
}
|
||||
|
||||
private static int intValue(Map<String, Object> body, String key, int defaultValue) {
|
||||
Object value = body.get(key);
|
||||
return value == null ? defaultValue : (int) parseLong(value.toString());
|
||||
}
|
||||
|
||||
private static long longValue(Map<String, Object> body, String key, long defaultValue) {
|
||||
Object value = body.get(key);
|
||||
return value == null ? defaultValue : parseLong(value.toString());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,147 @@
|
||||
package com.lingniu.ingest.gateway;
|
||||
|
||||
import com.lingniu.ingest.protocol.jt808.downlink.Jt808Commands;
|
||||
import com.lingniu.ingest.protocol.jt808.model.Jt808Header;
|
||||
import com.lingniu.ingest.protocol.jt808.model.Jt808Message;
|
||||
import com.lingniu.ingest.session.CommandDispatcher;
|
||||
import com.lingniu.ingest.session.DeviceSession;
|
||||
import com.lingniu.ingest.session.SessionStore;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.CompletableFuture;
|
||||
import java.util.function.UnaryOperator;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
class TerminalCommandControllerTest {
|
||||
|
||||
@Test
|
||||
void queryAttributesDispatchesJt8088107InsteadOfNotImplemented() {
|
||||
RecordingDispatcher dispatcher = new RecordingDispatcher();
|
||||
TerminalCommandController controller = new TerminalCommandController(new EmptySessionStore(), dispatcher);
|
||||
|
||||
CommandResult result = controller.attributes("13800138000");
|
||||
|
||||
assertThat(result.success()).isTrue();
|
||||
assertThat(dispatcher.lastRequestCommand.messageId()).isEqualTo(0x8107);
|
||||
assertThat(dispatcher.lastRequestCommand.body()).isEmpty();
|
||||
}
|
||||
|
||||
@Test
|
||||
void alarmAckDispatchesJt8088203WithResponseSerialAndAlarmType() {
|
||||
RecordingDispatcher dispatcher = new RecordingDispatcher();
|
||||
TerminalCommandController controller = new TerminalCommandController(new EmptySessionStore(), dispatcher);
|
||||
|
||||
CommandResult result = controller.alarmAck("13800138000", Map.of(
|
||||
"responseSerialNo", 0x1234,
|
||||
"type", 0x0000_0001));
|
||||
|
||||
assertThat(result.success()).isTrue();
|
||||
assertThat(dispatcher.lastNotifyCommand.messageId()).isEqualTo(0x8203);
|
||||
assertThat(dispatcher.lastNotifyCommand.body()).containsExactly(
|
||||
0x12, 0x34,
|
||||
0x00, 0x00, 0x00, 0x01);
|
||||
}
|
||||
|
||||
@Test
|
||||
void deleteAreaDispatchesJt8088601WithCommaSeparatedAreaIds() {
|
||||
RecordingDispatcher dispatcher = new RecordingDispatcher();
|
||||
TerminalCommandController controller = new TerminalCommandController(new EmptySessionStore(), dispatcher);
|
||||
|
||||
CommandResult result = controller.deleteArea("13800138000", "1,0x01020304");
|
||||
|
||||
assertThat(result.success()).isTrue();
|
||||
assertThat(dispatcher.lastRequestCommand.messageId()).isEqualTo(0x8601);
|
||||
assertThat(dispatcher.lastRequestCommand.body()).containsExactly(
|
||||
0x02,
|
||||
0x00, 0x00, 0x00, 0x01,
|
||||
0x01, 0x02, 0x03, 0x04);
|
||||
}
|
||||
|
||||
@Test
|
||||
void jt1078RealtimePlayDispatches9101ThroughSharedDispatcher() {
|
||||
RecordingDispatcher dispatcher = new RecordingDispatcher();
|
||||
TerminalCommandController controller = new TerminalCommandController(new EmptySessionStore(), dispatcher);
|
||||
|
||||
CommandResult result = controller.jt1078RealtimePlay("13800138000", Map.of(
|
||||
"ip", "10.1.2.3",
|
||||
"tcpPort", 11078,
|
||||
"udpPort", 11079,
|
||||
"channelNo", 2,
|
||||
"mediaType", 1,
|
||||
"streamType", 0));
|
||||
|
||||
assertThat(result.success()).isTrue();
|
||||
assertThat(dispatcher.lastRequestCommand.messageId()).isEqualTo(0x9101);
|
||||
assertThat(dispatcher.lastRequestCommand.body()).containsExactly(
|
||||
0x08, '1', '0', '.', '1', '.', '2', '.', '3',
|
||||
0x2B, 0x46,
|
||||
0x2B, 0x47,
|
||||
0x02,
|
||||
0x01,
|
||||
0x00);
|
||||
}
|
||||
|
||||
@Test
|
||||
void jt1078ResourceSearchDispatches9205AndAwaitsFileListResponse() {
|
||||
RecordingDispatcher dispatcher = new RecordingDispatcher();
|
||||
TerminalCommandController controller = new TerminalCommandController(new EmptySessionStore(), dispatcher);
|
||||
|
||||
CommandResult result = controller.jt1078ResourceSearch("13800138000", Map.of(
|
||||
"channelNo", 3,
|
||||
"startTime", "2026-06-22 08:09:10",
|
||||
"endTime", "260622180000",
|
||||
"warnBit1", "0x01020304",
|
||||
"warnBit2", "0x05060708",
|
||||
"mediaType", 3,
|
||||
"streamType", 2,
|
||||
"storageType", 1));
|
||||
|
||||
assertThat(result.success()).isTrue();
|
||||
assertThat(dispatcher.lastRequestCommand.messageId()).isEqualTo(0x9205);
|
||||
assertThat(dispatcher.lastRequestCommand.body()).containsExactly(
|
||||
0x03,
|
||||
0x26, 0x06, 0x22, 0x08, 0x09, 0x10,
|
||||
0x26, 0x06, 0x22, 0x18, 0x00, 0x00,
|
||||
0x01, 0x02, 0x03, 0x04,
|
||||
0x05, 0x06, 0x07, 0x08,
|
||||
0x03,
|
||||
0x02,
|
||||
0x01);
|
||||
}
|
||||
|
||||
private static final class RecordingDispatcher implements CommandDispatcher {
|
||||
private Jt808Commands.DownlinkCommand lastNotifyCommand;
|
||||
private Jt808Commands.DownlinkCommand lastRequestCommand;
|
||||
|
||||
@Override
|
||||
public CompletableFuture<Void> notify(String sessionId, Object command) {
|
||||
this.lastNotifyCommand = (Jt808Commands.DownlinkCommand) command;
|
||||
return CompletableFuture.completedFuture(null);
|
||||
}
|
||||
|
||||
@Override
|
||||
@SuppressWarnings("unchecked")
|
||||
public <T> CompletableFuture<T> request(String sessionId, Object command, Class<T> responseType, Duration timeout) {
|
||||
this.lastRequestCommand = (Jt808Commands.DownlinkCommand) command;
|
||||
Jt808Message response = new Jt808Message(
|
||||
new Jt808Header(0x0107, 0, 0, false, Jt808Header.ProtocolVersion.V2013,
|
||||
"13800138000", 1, 0, 0),
|
||||
new com.lingniu.ingest.protocol.jt808.model.Jt808Body.Raw(0x0107, new byte[0]));
|
||||
return CompletableFuture.completedFuture((T) response);
|
||||
}
|
||||
}
|
||||
|
||||
private static final class EmptySessionStore implements SessionStore {
|
||||
@Override public void put(DeviceSession session) {}
|
||||
@Override public Optional<DeviceSession> findBySessionId(String sessionId) { return Optional.empty(); }
|
||||
@Override public Optional<DeviceSession> findByVin(String vin) { return Optional.empty(); }
|
||||
@Override public Optional<DeviceSession> findByPhone(String phone) { return Optional.empty(); }
|
||||
@Override public Optional<DeviceSession> update(String sessionId, UnaryOperator<DeviceSession> updater) { return Optional.empty(); }
|
||||
@Override public void remove(String sessionId) {}
|
||||
@Override public int size() { return 0; }
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,7 @@
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>lingniu-vehicle-ingest</artifactId>
|
||||
<version>0.1.0-SNAPSHOT</version>
|
||||
<relativePath>../../../pom.xml</relativePath>
|
||||
</parent>
|
||||
<artifactId>ingest-api</artifactId>
|
||||
<name>ingest-api</name>
|
||||
@@ -5,6 +5,7 @@ package com.lingniu.ingest.api;
|
||||
* 新增协议需要在此处登记，避免散落的字符串常量。
|
||||
*/
|
||||
public enum ProtocolId {
|
||||
UNKNOWN,
|
||||
GB32960,
|
||||
JT808,
|
||||
JT1078,
|
||||
@@ -0,0 +1,30 @@
|
||||
package com.lingniu.ingest.api.annotation;
|
||||
|
||||
import java.lang.annotation.ElementType;
|
||||
import java.lang.annotation.Retention;
|
||||
import java.lang.annotation.RetentionPolicy;
|
||||
import java.lang.annotation.Target;
|
||||
|
||||
/**
|
||||
* 批量异步处理：框架把多次同类调用聚合成 {@code List} 交给方法。
|
||||
*
|
||||
* <p>处理方法签名需接收 {@code List<T>}，返回 {@code List<VehicleEvent>} 或单个
|
||||
* {@code VehicleEvent}。Dispatcher 不直接调用目标方法，而是交给
|
||||
* {@code AsyncBatchExecutor} 按 size/waitMs 聚合后调用。
|
||||
*
|
||||
* <p>注意：当调用方需要给每条事件补不同的 rawArchiveUri 等逐条元数据时，执行器会按单条 flush，
|
||||
* 以保证事件与原始帧一一对应。
|
||||
*/
|
||||
@Retention(RetentionPolicy.RUNTIME)
|
||||
@Target(ElementType.METHOD)
|
||||
public @interface AsyncBatch {
|
||||
|
||||
/** 达到该批量大小后立即 flush。 */
|
||||
int size() default 1000;
|
||||
|
||||
/** 第一条消息入队后最多等待毫秒数，避免低流量车辆长期不出批。 */
|
||||
long waitMs() default 500;
|
||||
|
||||
/** 每个 Handler 方法对应的批处理 worker 数。 */
|
||||
int poolSize() default 2;
|
||||
}
|
||||
@@ -9,6 +9,8 @@ import java.lang.annotation.Target;
|
||||
* 声明幂等键，由 Dedup 拦截器使用。支持 SpEL 表达式，上下文根对象为 Handler 参数。
|
||||
*
|
||||
* <p>示例：{@code @IdempotentKey("#msg.vin + ':' + #msg.seq + ':' + #msg.eventTime")}
|
||||
*
|
||||
* <p>当前内置去重主要基于 RawFrame sourceMeta/raw bytes；该注解保留给更细粒度 Handler 级去重。
|
||||
*/
|
||||
@Retention(RetentionPolicy.RUNTIME)
|
||||
@Target(ElementType.METHOD)
|
||||
@@ -14,14 +14,20 @@ import java.lang.annotation.Target;
|
||||
* <li>JT/T 808：{@code command} = 消息 ID（0x0100 / 0x0200 / ...）；{@code infoType} 留空
|
||||
* <li>MQTT：{@code command} 可为 topic hash 或忽略
|
||||
* </ul>
|
||||
*
|
||||
* <p>同一个方法可以声明多个 command/infoType。Dispatcher 会按协议、command、infoType 精确路由；
|
||||
* 32960 实时上报解析后通常由 0x02/0x03 主命令加信息体 ID 决定落到哪个 Handler。
|
||||
*/
|
||||
@Retention(RetentionPolicy.RUNTIME)
|
||||
@Target(ElementType.METHOD)
|
||||
public @interface MessageMapping {
|
||||
|
||||
/** 协议主命令或消息 ID，空数组表示该维度不限制。 */
|
||||
int[] command() default {};
|
||||
|
||||
/** 协议子类型，GB32960 中对应信息体 ID；空数组表示该维度不限制。 */
|
||||
int[] infoType() default {};
|
||||
|
||||
/** 仅用于日志/Swagger/排障展示，不参与路由。 */
|
||||
String desc() default "";
|
||||
}
|
||||
@@ -6,7 +6,10 @@ import java.lang.annotation.RetentionPolicy;
|
||||
import java.lang.annotation.Target;
|
||||
|
||||
/**
|
||||
* 单 VIN 速率限制。超限消息直接进 DLQ 并打点。
|
||||
* 单 VIN 速率限制声明。
|
||||
*
|
||||
* <p>当前内置限流器在 Pipeline before 阶段按 VIN 中止处理，超限帧不会进入 Handler 和下游存储。
|
||||
* 如果需要把超限帧写入 DLQ，应在拦截器或入口层显式增加对应 sink。
|
||||
*/
|
||||
@Retention(RetentionPolicy.RUNTIME)
|
||||
@Target(ElementType.METHOD)
|
||||
@@ -0,0 +1,58 @@
|
||||
package com.lingniu.ingest.api.consumer;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.EnumSet;
|
||||
import java.util.Set;
|
||||
|
||||
public final class EnvelopeConsumerProcessor {
|
||||
|
||||
private static final Set<EnvelopeIngestResult.Status> DEAD_LETTER_STATUSES = EnumSet.of(
|
||||
EnvelopeIngestResult.Status.SKIPPED,
|
||||
EnvelopeIngestResult.Status.INVALID_ENVELOPE,
|
||||
EnvelopeIngestResult.Status.FAILED
|
||||
);
|
||||
|
||||
private final String service;
|
||||
private final EnvelopeIngestor ingestor;
|
||||
private final EnvelopeDeadLetterSink deadLetterSink;
|
||||
|
||||
public EnvelopeConsumerProcessor(String service, EnvelopeIngestor ingestor, EnvelopeDeadLetterSink deadLetterSink) {
|
||||
if (ingestor == null) {
|
||||
throw new IllegalArgumentException("ingestor must not be null");
|
||||
}
|
||||
if (deadLetterSink == null) {
|
||||
throw new IllegalArgumentException("deadLetterSink must not be null");
|
||||
}
|
||||
this.service = service == null || service.isBlank() ? "unknown" : service;
|
||||
this.ingestor = ingestor;
|
||||
this.deadLetterSink = deadLetterSink;
|
||||
}
|
||||
|
||||
public EnvelopeIngestResult process(EnvelopeConsumerRecord record) {
|
||||
if (record == null) {
|
||||
throw new IllegalArgumentException("record must not be null");
|
||||
}
|
||||
EnvelopeIngestResult result = ingestor.tryIngest(record.payload());
|
||||
// 派生消费者不在这里抛出业务异常给 Kafka worker；坏消息统一进入 DLQ，
|
||||
// worker 看到 process 正常返回后才能提交 offset，避免同一坏消息无限阻塞消费组。
|
||||
if (DEAD_LETTER_STATUSES.contains(result.status())) {
|
||||
deadLetterSink.publish(toDeadLetter(record, result));
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
private EnvelopeDeadLetterRecord toDeadLetter(EnvelopeConsumerRecord record, EnvelopeIngestResult result) {
|
||||
return new EnvelopeDeadLetterRecord(
|
||||
service,
|
||||
record.topic(),
|
||||
record.partition(),
|
||||
record.offset(),
|
||||
record.key(),
|
||||
result.status(),
|
||||
result.eventId(),
|
||||
result.vin(),
|
||||
result.message(),
|
||||
record.payload(),
|
||||
Instant.now());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
package com.lingniu.ingest.api.consumer;
|
||||
|
||||
public record EnvelopeConsumerRecord(
|
||||
String topic,
|
||||
int partition,
|
||||
long offset,
|
||||
String key,
|
||||
byte[] payload
|
||||
) {
|
||||
public EnvelopeConsumerRecord {
|
||||
topic = topic == null ? "" : topic;
|
||||
key = key == null ? "" : key;
|
||||
payload = payload == null ? new byte[0] : payload.clone();
|
||||
}
|
||||
|
||||
@Override
|
||||
public byte[] payload() {
|
||||
return payload.clone();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,36 @@
|
||||
package com.lingniu.ingest.api.consumer;
|
||||
|
||||
import java.time.Instant;
|
||||
|
||||
public record EnvelopeDeadLetterRecord(
|
||||
String service,
|
||||
String topic,
|
||||
int partition,
|
||||
long offset,
|
||||
String key,
|
||||
EnvelopeIngestResult.Status status,
|
||||
String eventId,
|
||||
String vin,
|
||||
String message,
|
||||
byte[] payload,
|
||||
Instant createdAt
|
||||
) {
|
||||
public EnvelopeDeadLetterRecord {
|
||||
service = service == null ? "" : service;
|
||||
topic = topic == null ? "" : topic;
|
||||
key = key == null ? "" : key;
|
||||
if (status == null) {
|
||||
throw new IllegalArgumentException("status must not be null");
|
||||
}
|
||||
eventId = eventId == null ? "" : eventId;
|
||||
vin = vin == null ? "" : vin;
|
||||
message = message == null ? "" : message;
|
||||
payload = payload == null ? new byte[0] : payload.clone();
|
||||
createdAt = createdAt == null ? Instant.now() : createdAt;
|
||||
}
|
||||
|
||||
@Override
|
||||
public byte[] payload() {
|
||||
return payload.clone();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package com.lingniu.ingest.api.consumer;
|
||||
|
||||
@FunctionalInterface
|
||||
public interface EnvelopeDeadLetterSink {
|
||||
void publish(EnvelopeDeadLetterRecord record);
|
||||
}
|
||||
@@ -0,0 +1,45 @@
|
||||
package com.lingniu.ingest.api.consumer;
|
||||
|
||||
/**
|
||||
* Non-throwing result for Kafka envelope consumers. It lets production
|
||||
* listeners isolate bad records without blocking the partition.
|
||||
*/
|
||||
public record EnvelopeIngestResult(Status status, String eventId, String vin, String message) {
|
||||
|
||||
public enum Status {
|
||||
STORED,
|
||||
PROCESSED,
|
||||
SKIPPED,
|
||||
INVALID_ENVELOPE,
|
||||
FAILED
|
||||
}
|
||||
|
||||
public EnvelopeIngestResult {
|
||||
if (status == null) {
|
||||
throw new IllegalArgumentException("status must not be null");
|
||||
}
|
||||
eventId = eventId == null ? "" : eventId;
|
||||
vin = vin == null ? "" : vin;
|
||||
message = message == null ? "" : message;
|
||||
}
|
||||
|
||||
public static EnvelopeIngestResult stored(String eventId, String vin) {
|
||||
return new EnvelopeIngestResult(Status.STORED, eventId, vin, "");
|
||||
}
|
||||
|
||||
public static EnvelopeIngestResult processed(String eventId, String vin) {
|
||||
return new EnvelopeIngestResult(Status.PROCESSED, eventId, vin, "");
|
||||
}
|
||||
|
||||
public static EnvelopeIngestResult skipped(String eventId, String vin, String message) {
|
||||
return new EnvelopeIngestResult(Status.SKIPPED, eventId, vin, message);
|
||||
}
|
||||
|
||||
public static EnvelopeIngestResult invalid(String message) {
|
||||
return new EnvelopeIngestResult(Status.INVALID_ENVELOPE, "", "", message);
|
||||
}
|
||||
|
||||
public static EnvelopeIngestResult failed(String eventId, String vin, String message) {
|
||||
return new EnvelopeIngestResult(Status.FAILED, eventId, vin, message);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package com.lingniu.ingest.api.consumer;
|
||||
|
||||
@FunctionalInterface
|
||||
public interface EnvelopeIngestor {
|
||||
EnvelopeIngestResult tryIngest(byte[] kafkaValue);
|
||||
}
|
||||
@@ -13,6 +13,10 @@ import java.util.Set;
|
||||
* @param activeBits 结构化的通用报警位集合（2016 版：0~15；2025 版：0~27），对照 GB/T 32960.3 表 24
|
||||
* @param longitude 经度（可选，若事件伴随位置）
|
||||
* @param latitude 纬度（可选）
|
||||
* @param safetyCategory 内部安全分类。氢气泄露独立成类，不与普通故障混用
|
||||
* @param hydrogenLeakDetected 是否检测到氢气泄露
|
||||
* @param hydrogenLeakLevel 氢气泄露等级
|
||||
* @param hydrogenLeakActionRequired 是否需要立即处置
|
||||
*/
|
||||
public record AlarmPayload(
|
||||
AlarmLevel level,
|
||||
@@ -21,7 +25,11 @@ public record AlarmPayload(
|
||||
List<String> faultCodes,
|
||||
Set<String> activeBits,
|
||||
Double longitude,
|
||||
Double latitude
|
||||
Double latitude,
|
||||
SafetyCategory safetyCategory,
|
||||
boolean hydrogenLeakDetected,
|
||||
HydrogenLeakLevel hydrogenLeakLevel,
|
||||
boolean hydrogenLeakActionRequired
|
||||
) {
|
||||
/**
|
||||
* 报警等级（归一化的跨协议分类）。
|
||||
@@ -34,4 +42,24 @@ public record AlarmPayload(
|
||||
* </ul>
|
||||
*/
|
||||
public enum AlarmLevel { INFO, MINOR, MAJOR, CRITICAL }
|
||||
|
||||
/**
|
||||
* 面向运营统计的内部安全分类。
|
||||
*/
|
||||
public enum SafetyCategory {
|
||||
GENERAL,
|
||||
TANK_PRESSURE,
|
||||
TANK_TEMPERATURE,
|
||||
HYDROGEN_LEAK
|
||||
}
|
||||
|
||||
/**
|
||||
* 氢气泄露内部等级。明确泄露信号一律视为 CRITICAL。
|
||||
*/
|
||||
public enum HydrogenLeakLevel {
|
||||
NONE,
|
||||
WARNING,
|
||||
CRITICAL,
|
||||
UNKNOWN
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
package com.lingniu.ingest.api.event;
|
||||
|
||||
/** 位置事件载荷。坐标已统一转换为 WGS84 十进制度。 */
|
||||
public record LocationPayload(
|
||||
double longitude,
|
||||
double latitude,
|
||||
double altitudeM,
|
||||
double speedKmh,
|
||||
double directionDeg,
|
||||
long alarmFlag,
|
||||
long statusFlag,
|
||||
Double totalMileageKm
|
||||
) {
|
||||
public LocationPayload(double longitude,
|
||||
double latitude,
|
||||
double altitudeM,
|
||||
double speedKmh,
|
||||
double directionDeg,
|
||||
long alarmFlag,
|
||||
long statusFlag) {
|
||||
this(longitude, latitude, altitudeM, speedKmh, directionDeg, alarmFlag, statusFlag, null);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
package com.lingniu.ingest.api.event;
|
||||
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.time.ZoneId;
|
||||
import java.time.format.DateTimeFormatter;
|
||||
import java.util.Map;
|
||||
|
||||
/**
|
||||
* Shared raw-archive key conventions used by Dispatcher, archive sink, and query services.
|
||||
*
|
||||
* <p>32960 生产链路里，RAW .bin 文件本体按 {@code 日期/协议/VIN/eventId.bin} 落在 archive 根目录，
|
||||
* DuckDB 只保存 {@code archive://...} 引用和查询索引。这个类集中维护 key 规则，避免接收端、落盘端、
|
||||
* snapshot 查询端对目录分区的理解不一致。</p>
|
||||
*/
|
||||
public final class RawArchiveKeys {
|
||||
|
||||
public static final String META_EVENT_ID = "rawArchiveEventId";
|
||||
public static final String META_KEY = "rawArchiveKey";
|
||||
public static final String META_URI = "rawArchiveUri";
|
||||
|
||||
private static final DateTimeFormatter DATE_KEY =
|
||||
DateTimeFormatter.ofPattern("yyyy/MM/dd").withZone(ZoneId.of("Asia/Shanghai"));
|
||||
|
||||
private RawArchiveKeys() {
|
||||
}
|
||||
|
||||
public static String key(Instant ingestTime, ProtocolId source, String vin, String eventId) {
|
||||
// 分区日期使用 ingestTime 的东八区自然日，和磁盘目录保持一致；eventTime 只代表车端上报时间。
|
||||
String dateKey = DATE_KEY.format(ingestTime == null ? Instant.EPOCH : ingestTime);
|
||||
String safeVin = vin == null || vin.isBlank() ? "unknown-vin" : vin;
|
||||
String safeEventId = eventId == null || eventId.isBlank()
|
||||
? Long.toHexString(System.nanoTime()) : eventId;
|
||||
String safeSource = source == null ? "UNKNOWN" : source.name();
|
||||
return dateKey + "/" + safeSource + "/" + safeVin + "/" + safeEventId + ".bin";
|
||||
}
|
||||
|
||||
public static String key(VehicleEvent.RawArchive raw) {
|
||||
String key = metadataValue(raw.metadata(), META_KEY);
|
||||
if (!key.isBlank()) {
|
||||
return key;
|
||||
}
|
||||
return key(raw.ingestTime(), raw.source(), raw.vin(), raw.eventId());
|
||||
}
|
||||
|
||||
public static String logicalUri(String key) {
|
||||
return key == null || key.isBlank() ? "" : "archive://" + key;
|
||||
}
|
||||
|
||||
private static String metadataValue(Map<String, String> metadata, String key) {
|
||||
if (metadata == null || key == null) {
|
||||
return "";
|
||||
}
|
||||
String value = metadata.get(key);
|
||||
return value == null ? "" : value;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
package com.lingniu.ingest.api.event;
|
||||
|
||||
/**
|
||||
* One normalized telemetry field value.
|
||||
*
|
||||
* <p>Protocol mappers use stable internal field keys so Kafka, Parquet, Redis,
|
||||
* and statistics do not depend on protocol-specific names.
|
||||
*
|
||||
* <p>{@code value} 统一用字符串承载，真实类型由 {@code valueType} 表达。这样 Kafka
|
||||
* protobuf、CSV 导出、DuckDB JSON 字段和前端展示可以共用同一份字段结构；数值精度/格式化
|
||||
* 由查询层或前端按 {@code valueType + unit} 决定。
|
||||
*/
|
||||
public record TelemetryFieldValue(
|
||||
String key,
|
||||
ValueType valueType,
|
||||
String value,
|
||||
String unit,
|
||||
Quality quality,
|
||||
String sourcePath
|
||||
) {
|
||||
|
||||
public TelemetryFieldValue {
|
||||
if (key == null || key.isBlank()) {
|
||||
throw new IllegalArgumentException("key must not be blank");
|
||||
}
|
||||
if (valueType == null) {
|
||||
throw new IllegalArgumentException("valueType must not be null");
|
||||
}
|
||||
value = value == null ? "" : value;
|
||||
unit = unit == null ? "" : unit;
|
||||
// 默认 GOOD，只有解析器明确知道异常/无效/缺失时才降级，避免调用方到处补质量字段。
|
||||
quality = quality == null ? Quality.GOOD : quality;
|
||||
sourcePath = sourcePath == null ? "" : sourcePath;
|
||||
}
|
||||
|
||||
public static TelemetryFieldValue stringValue(String key, String value, String unit, String sourcePath) {
|
||||
return new TelemetryFieldValue(key, ValueType.STRING, value, unit, Quality.GOOD, sourcePath);
|
||||
}
|
||||
|
||||
public static TelemetryFieldValue doubleValue(String key, double value, String unit, String sourcePath) {
|
||||
return new TelemetryFieldValue(key, ValueType.DOUBLE, Double.toString(value), unit, Quality.GOOD, sourcePath);
|
||||
}
|
||||
|
||||
public static TelemetryFieldValue longValue(String key, long value, String unit, String sourcePath) {
|
||||
return new TelemetryFieldValue(key, ValueType.LONG, Long.toString(value), unit, Quality.GOOD, sourcePath);
|
||||
}
|
||||
|
||||
public static TelemetryFieldValue booleanValue(String key, boolean value, String unit, String sourcePath) {
|
||||
return new TelemetryFieldValue(key, ValueType.BOOLEAN, Boolean.toString(value), unit, Quality.GOOD, sourcePath);
|
||||
}
|
||||
|
||||
public static TelemetryFieldValue instantValue(String key, String value, String unit, String sourcePath) {
|
||||
return new TelemetryFieldValue(key, ValueType.INSTANT, value, unit, Quality.GOOD, sourcePath);
|
||||
}
|
||||
|
||||
public static TelemetryFieldValue jsonValue(String key, String value, String unit, String sourcePath) {
|
||||
return new TelemetryFieldValue(key, ValueType.JSON, value, unit, Quality.GOOD, sourcePath);
|
||||
}
|
||||
|
||||
public static TelemetryFieldValue missing(String key, String unit, String sourcePath) {
|
||||
return new TelemetryFieldValue(key, ValueType.STRING, "", unit, Quality.MISSING, sourcePath);
|
||||
}
|
||||
|
||||
public enum ValueType {
|
||||
STRING,
|
||||
DOUBLE,
|
||||
LONG,
|
||||
BOOLEAN,
|
||||
INSTANT,
|
||||
JSON
|
||||
}
|
||||
|
||||
public enum Quality {
|
||||
GOOD,
|
||||
ESTIMATED,
|
||||
INVALID,
|
||||
MISSING
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,147 @@
|
||||
package com.lingniu.ingest.api.event;
|
||||
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Normalized full-field telemetry snapshot for one parsed vehicle event.
|
||||
*
|
||||
* <p>This is the shared contract for Kafka, Parquet history, Redis hot state,
|
||||
* and statistics. It intentionally lives in {@code ingest-api} and has no
|
||||
* Spring, Kafka, Redis, or DuckDB dependency.
|
||||
*/
|
||||
public record TelemetrySnapshot(
|
||||
String eventId,
|
||||
String vin,
|
||||
ProtocolId protocol,
|
||||
String eventType,
|
||||
Instant eventTime,
|
||||
Instant ingestTime,
|
||||
String rawArchiveUri,
|
||||
Map<String, String> metadata,
|
||||
List<TelemetryFieldValue> fields
|
||||
) {
|
||||
|
||||
public TelemetrySnapshot {
|
||||
if (eventId == null || eventId.isBlank()) {
|
||||
throw new IllegalArgumentException("eventId must not be blank");
|
||||
}
|
||||
if (vin == null || vin.isBlank()) {
|
||||
throw new IllegalArgumentException("vin must not be blank");
|
||||
}
|
||||
if (protocol == null) {
|
||||
throw new IllegalArgumentException("protocol must not be null");
|
||||
}
|
||||
if (eventType == null || eventType.isBlank()) {
|
||||
throw new IllegalArgumentException("eventType must not be blank");
|
||||
}
|
||||
if (eventTime == null) {
|
||||
throw new IllegalArgumentException("eventTime must not be null");
|
||||
}
|
||||
if (ingestTime == null) {
|
||||
throw new IllegalArgumentException("ingestTime must not be null");
|
||||
}
|
||||
rawArchiveUri = rawArchiveUri == null ? "" : rawArchiveUri;
|
||||
metadata = Map.copyOf(metadata == null ? Map.of() : metadata);
|
||||
fields = List.copyOf(fields == null ? List.of() : fields);
|
||||
// snapshot/fields 查询按 key 投影和导出，重复 key 会导致前端列和 CSV 表头不稳定，构造期直接拒绝。
|
||||
validateUniqueKeys(fields);
|
||||
}
|
||||
|
||||
public Optional<TelemetryFieldValue> field(String key) {
|
||||
if (key == null || key.isBlank()) {
|
||||
return Optional.empty();
|
||||
}
|
||||
// fields 保持插入顺序，单字段查询走线性查找；批量查询应使用 fieldsByKey 建索引。
|
||||
for (TelemetryFieldValue field : fields) {
|
||||
if (field.key().equals(key)) {
|
||||
return Optional.of(field);
|
||||
}
|
||||
}
|
||||
return Optional.empty();
|
||||
}
|
||||
|
||||
public Map<String, TelemetryFieldValue> fieldsByKey() {
|
||||
Map<String, TelemetryFieldValue> index = new LinkedHashMap<>();
|
||||
for (TelemetryFieldValue field : fields) {
|
||||
index.put(field.key(), field);
|
||||
}
|
||||
// LinkedHashMap 保留原始字段顺序，CSV 导出和 Swagger 示例能稳定复现解析器顺序。
|
||||
return Map.copyOf(index);
|
||||
}
|
||||
|
||||
public static Builder builder(String eventId,
|
||||
String vin,
|
||||
ProtocolId protocol,
|
||||
String eventType,
|
||||
Instant eventTime,
|
||||
Instant ingestTime) {
|
||||
return new Builder(eventId, vin, protocol, eventType, eventTime, ingestTime);
|
||||
}
|
||||
|
||||
private static void validateUniqueKeys(List<TelemetryFieldValue> fields) {
|
||||
Map<String, TelemetryFieldValue> seen = new LinkedHashMap<>();
|
||||
for (TelemetryFieldValue field : fields) {
|
||||
if (field == null) {
|
||||
throw new IllegalArgumentException("field must not be null");
|
||||
}
|
||||
TelemetryFieldValue previous = seen.put(field.key(), field);
|
||||
if (previous != null) {
|
||||
throw new IllegalArgumentException("duplicate telemetry field key: " + field.key());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
public static final class Builder {
|
||||
private final String eventId;
|
||||
private final String vin;
|
||||
private final ProtocolId protocol;
|
||||
private final String eventType;
|
||||
private final Instant eventTime;
|
||||
private final Instant ingestTime;
|
||||
private String rawArchiveUri = "";
|
||||
private Map<String, String> metadata = Map.of();
|
||||
private final List<TelemetryFieldValue> fields = new ArrayList<>();
|
||||
|
||||
private Builder(String eventId,
|
||||
String vin,
|
||||
ProtocolId protocol,
|
||||
String eventType,
|
||||
Instant eventTime,
|
||||
Instant ingestTime) {
|
||||
this.eventId = eventId;
|
||||
this.vin = vin;
|
||||
this.protocol = protocol;
|
||||
this.eventType = eventType;
|
||||
this.eventTime = eventTime;
|
||||
this.ingestTime = ingestTime;
|
||||
}
|
||||
|
||||
public Builder rawArchiveUri(String rawArchiveUri) {
|
||||
this.rawArchiveUri = rawArchiveUri;
|
||||
return this;
|
||||
}
|
||||
|
||||
public Builder metadata(Map<String, String> metadata) {
|
||||
this.metadata = metadata == null ? Map.of() : Map.copyOf(metadata);
|
||||
return this;
|
||||
}
|
||||
|
||||
public Builder addField(TelemetryFieldValue field) {
|
||||
this.fields.add(field);
|
||||
return this;
|
||||
}
|
||||
|
||||
public TelemetrySnapshot build() {
|
||||
return new TelemetrySnapshot(
|
||||
eventId, vin, protocol, eventType, eventTime, ingestTime,
|
||||
rawArchiveUri, metadata, fields);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -26,7 +26,8 @@ public sealed interface VehicleEvent
|
||||
VehicleEvent.Logout,
|
||||
VehicleEvent.Heartbeat,
|
||||
VehicleEvent.MediaMeta,
|
||||
VehicleEvent.Passthrough {
|
||||
VehicleEvent.Passthrough,
|
||||
VehicleEvent.RawArchive {
|
||||
|
||||
String eventId();
|
||||
String vin();
|
||||
@@ -138,4 +139,28 @@ public sealed interface VehicleEvent
|
||||
int passthroughType,
|
||||
byte[] data
|
||||
) implements VehicleEvent {}
|
||||
|
||||
/**
|
||||
* 原始报文归档事件：每条成功解码的入站帧由 Dispatcher 产出一条，携带原始字节和归档索引信息。
|
||||
* 32960 历史全字段查询依赖它对应的 {@code archive://...} 文件能够被回读。
|
||||
*
|
||||
* <p>当前 {@code KafkaEventSink} 默认不发送本类型，{@code EventFileStoreSink} 只保存索引不保存
|
||||
* bytes；因此启用新 raw 落盘/转发实现时，必须同时保证 {@link RawArchiveKeys} 的 key 规则一致。
|
||||
*
|
||||
* @param command 协议主命令码（如 32960 0x02/0x03 等），冗余在事件里便于按命令分片归档
|
||||
* @param infoType 协议子类型（可为 0），同上
|
||||
* @param rawBytes 原始入站字节，不可为 null
|
||||
*/
|
||||
record RawArchive(
|
||||
String eventId,
|
||||
String vin,
|
||||
ProtocolId source,
|
||||
Instant eventTime,
|
||||
Instant ingestTime,
|
||||
String traceId,
|
||||
Map<String, String> metadata,
|
||||
int command,
|
||||
int infoType,
|
||||
byte[] rawBytes
|
||||
) implements VehicleEvent {}
|
||||
}
|
||||
@@ -0,0 +1,206 @@
|
||||
package com.lingniu.ingest.api.event;
|
||||
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Converts current {@link VehicleEvent} variants into normalized full-field
|
||||
* telemetry snapshots.
|
||||
*
|
||||
* <p>Protocol-specific mappers can add richer fields later, but all downstream
|
||||
* modules should consume {@link TelemetrySnapshot} instead of protocol objects.
|
||||
*/
|
||||
public final class VehicleEventTelemetrySnapshotMapper {
|
||||
|
||||
private VehicleEventTelemetrySnapshotMapper() {
|
||||
}
|
||||
|
||||
public static Optional<TelemetrySnapshot> toSnapshot(VehicleEvent event) {
|
||||
if (event == null || event instanceof VehicleEvent.RawArchive) {
|
||||
return Optional.empty();
|
||||
}
|
||||
TelemetrySnapshot.Builder builder = TelemetrySnapshot.builder(
|
||||
event.eventId(),
|
||||
event.vin(),
|
||||
event.source(),
|
||||
eventType(event),
|
||||
event.eventTime(),
|
||||
event.ingestTime())
|
||||
.metadata(event.metadata())
|
||||
.rawArchiveUri(rawArchiveUri(event));
|
||||
|
||||
switch (event) {
|
||||
case VehicleEvent.Realtime realtime -> addRealtimeFields(builder, realtime.payload());
|
||||
case VehicleEvent.Location location -> addLocationFields(builder, location.payload());
|
||||
case VehicleEvent.Alarm alarm -> addAlarmFields(builder, alarm.payload());
|
||||
case VehicleEvent.Login login -> {
|
||||
addString(builder, "iccid", login.iccid(), "", "event.login.iccid");
|
||||
addString(builder, "protocol_version", login.protocolVersion(), "", "event.login.protocolVersion");
|
||||
}
|
||||
case VehicleEvent.Logout ignored -> {
|
||||
}
|
||||
case VehicleEvent.Heartbeat ignored -> {
|
||||
}
|
||||
case VehicleEvent.MediaMeta media -> {
|
||||
addString(builder, "media_id", media.mediaId(), "", "event.media.mediaId");
|
||||
addString(builder, "media_type", media.mediaType(), "", "event.media.mediaType");
|
||||
builder.addField(TelemetryFieldValue.longValue(
|
||||
"media_size_bytes", media.sizeBytes(), "bytes", "event.media.sizeBytes"));
|
||||
addString(builder, "media_archive_ref", media.archiveRef(), "", "event.media.archiveRef");
|
||||
}
|
||||
case VehicleEvent.Passthrough passthrough -> {
|
||||
builder.addField(TelemetryFieldValue.longValue(
|
||||
"passthrough_type", passthrough.passthroughType(), "", "event.passthrough.type"));
|
||||
builder.addField(TelemetryFieldValue.longValue(
|
||||
"passthrough_size_bytes",
|
||||
passthrough.data() == null ? 0 : passthrough.data().length,
|
||||
"bytes",
|
||||
"event.passthrough.data"));
|
||||
}
|
||||
case VehicleEvent.RawArchive ignored -> {
|
||||
return Optional.empty();
|
||||
}
|
||||
}
|
||||
return Optional.of(builder.build());
|
||||
}
|
||||
|
||||
private static void addRealtimeFields(TelemetrySnapshot.Builder builder, RealtimePayload p) {
|
||||
if (p == null) return;
|
||||
addDouble(builder, "speed_kmh", p.speedKmh(), "km/h", "event.realtime.speedKmh");
|
||||
addDouble(builder, "total_mileage_km", p.totalMileageKm(), "km", "event.realtime.totalMileageKm");
|
||||
addDouble(builder, "battery_soc", p.batterySoc(), "%", "event.realtime.batterySoc");
|
||||
addDouble(builder, "battery_voltage_v", p.batteryVoltageV(), "V", "event.realtime.batteryVoltageV");
|
||||
addDouble(builder, "battery_current_a", p.batteryCurrentA(), "A", "event.realtime.batteryCurrentA");
|
||||
addDouble(builder, "fc_voltage_v", p.fcVoltageV(), "V", "event.realtime.fcVoltageV");
|
||||
addDouble(builder, "fc_current_a", p.fcCurrentA(), "A", "event.realtime.fcCurrentA");
|
||||
addDouble(builder, "fc_temp_c", p.fcTempC(), "C", "event.realtime.fcTempC");
|
||||
addDouble(builder, "hydrogen_remaining_kg", p.hydrogenRemainingKg(), "kg", "event.realtime.hydrogenRemainingKg");
|
||||
addDouble(builder, "hydrogen_high_pressure_mpa", p.hydrogenHighPressureMpa(), "MPa", "event.realtime.hydrogenHighPressureMpa");
|
||||
addDouble(builder, "hydrogen_low_pressure_mpa", p.hydrogenLowPressureMpa(), "MPa", "event.realtime.hydrogenLowPressureMpa");
|
||||
addEnum(builder, "vehicle_state", p.vehicleState(), "", "event.realtime.vehicleState");
|
||||
addEnum(builder, "charging_state", p.chargingState(), "", "event.realtime.chargingState");
|
||||
addEnum(builder, "running_mode", p.runningMode(), "", "event.realtime.runningMode");
|
||||
addLong(builder, "gear_level", p.gearLevel(), "", "event.realtime.gearLevel");
|
||||
addDouble(builder, "accelerator_pedal", p.acceleratorPedal(), "%", "event.realtime.acceleratorPedal");
|
||||
addDouble(builder, "brake_pedal", p.brakePedal(), "%", "event.realtime.brakePedal");
|
||||
addDouble(builder, "longitude", p.longitude(), "deg", "event.realtime.longitude");
|
||||
addDouble(builder, "latitude", p.latitude(), "deg", "event.realtime.latitude");
|
||||
addDouble(builder, "altitude_m", p.altitudeM(), "m", "event.realtime.altitudeM");
|
||||
addDouble(builder, "direction_deg", p.directionDeg(), "deg", "event.realtime.directionDeg");
|
||||
addDouble(builder, "ambient_temp_c", p.ambientTempC(), "C", "event.realtime.ambientTempC");
|
||||
addDouble(builder, "coolant_temp_c", p.coolantTempC(), "C", "event.realtime.coolantTempC");
|
||||
}
|
||||
|
||||
private static void addLocationFields(TelemetrySnapshot.Builder builder, LocationPayload p) {
|
||||
if (p == null) return;
|
||||
builder.addField(TelemetryFieldValue.doubleValue("longitude", p.longitude(), "deg", "event.location.longitude"));
|
||||
builder.addField(TelemetryFieldValue.doubleValue("latitude", p.latitude(), "deg", "event.location.latitude"));
|
||||
builder.addField(TelemetryFieldValue.doubleValue("altitude_m", p.altitudeM(), "m", "event.location.altitudeM"));
|
||||
builder.addField(TelemetryFieldValue.doubleValue("speed_kmh", p.speedKmh(), "km/h", "event.location.speedKmh"));
|
||||
addDouble(builder, "total_mileage_km", p.totalMileageKm(), "km", "event.location.totalMileageKm");
|
||||
builder.addField(TelemetryFieldValue.doubleValue("direction_deg", p.directionDeg(), "deg", "event.location.directionDeg"));
|
||||
builder.addField(TelemetryFieldValue.longValue("location_alarm_flag", p.alarmFlag(), "", "event.location.alarmFlag"));
|
||||
builder.addField(TelemetryFieldValue.longValue("location_status_raw", p.statusFlag(), "", "event.location.statusFlag"));
|
||||
}
|
||||
|
||||
private static void addAlarmFields(TelemetrySnapshot.Builder builder, AlarmPayload p) {
|
||||
if (p == null) return;
|
||||
addEnum(builder, "alarm_level", p.level(), "", "event.alarm.level");
|
||||
builder.addField(TelemetryFieldValue.longValue("alarm_type_code", p.alarmTypeCode(), "", "event.alarm.alarmTypeCode"));
|
||||
addString(builder, "alarm_type_name", p.alarmTypeName(), "", "event.alarm.alarmTypeName");
|
||||
if (p.faultCodes() != null && !p.faultCodes().isEmpty()) {
|
||||
builder.addField(TelemetryFieldValue.stringValue(
|
||||
"fault_codes", String.join(",", p.faultCodes()), "", "event.alarm.faultCodes"));
|
||||
}
|
||||
if (p.activeBits() != null && !p.activeBits().isEmpty()) {
|
||||
builder.addField(TelemetryFieldValue.stringValue(
|
||||
"active_alarm_bits", String.join(",", p.activeBits()), "", "event.alarm.activeBits"));
|
||||
}
|
||||
addDouble(builder, "longitude", p.longitude(), "deg", "event.alarm.longitude");
|
||||
addDouble(builder, "latitude", p.latitude(), "deg", "event.alarm.latitude");
|
||||
addEnum(builder, "safety_category", p.safetyCategory(), "", "event.alarm.safetyCategory");
|
||||
builder.addField(TelemetryFieldValue.booleanValue(
|
||||
"hydrogen_leak_detected", p.hydrogenLeakDetected(), "", "event.alarm.hydrogenLeakDetected"));
|
||||
addEnum(builder, "hydrogen_leak_level", p.hydrogenLeakLevel(), "", "event.alarm.hydrogenLeakLevel");
|
||||
builder.addField(TelemetryFieldValue.booleanValue(
|
||||
"hydrogen_leak_action_required",
|
||||
p.hydrogenLeakActionRequired(),
|
||||
"",
|
||||
"event.alarm.hydrogenLeakActionRequired"));
|
||||
}
|
||||
|
||||
private static String eventType(VehicleEvent event) {
|
||||
if (event instanceof VehicleEvent.Realtime) return "REALTIME";
|
||||
if (event instanceof VehicleEvent.Location) return "LOCATION";
|
||||
if (event instanceof VehicleEvent.Alarm) return "ALARM";
|
||||
if (event instanceof VehicleEvent.Login) return "LOGIN";
|
||||
if (event instanceof VehicleEvent.Logout) return "LOGOUT";
|
||||
if (event instanceof VehicleEvent.Heartbeat) return "HEARTBEAT";
|
||||
if (event instanceof VehicleEvent.MediaMeta) return "MEDIA_META";
|
||||
if (event instanceof VehicleEvent.Passthrough) return "PASSTHROUGH";
|
||||
return "UNKNOWN";
|
||||
}
|
||||
|
||||
private static String rawArchiveUri(VehicleEvent event) {
|
||||
Map<String, String> metadata = event.metadata();
|
||||
if (metadata == null) {
|
||||
return mediaArchiveRef(event);
|
||||
}
|
||||
String uri = metadata.getOrDefault(RawArchiveKeys.META_URI, "");
|
||||
if (uri != null && !uri.isBlank()) {
|
||||
return uri;
|
||||
}
|
||||
String key = metadata.getOrDefault(RawArchiveKeys.META_KEY, "");
|
||||
if (key != null && !key.isBlank()) {
|
||||
return RawArchiveKeys.logicalUri(key);
|
||||
}
|
||||
return mediaArchiveRef(event);
|
||||
}
|
||||
|
||||
private static String mediaArchiveRef(VehicleEvent event) {
|
||||
if (event instanceof VehicleEvent.MediaMeta media && media.archiveRef() != null) {
|
||||
return media.archiveRef();
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
private static void addDouble(TelemetrySnapshot.Builder builder,
|
||||
String key,
|
||||
Double value,
|
||||
String unit,
|
||||
String sourcePath) {
|
||||
if (value != null) {
|
||||
builder.addField(TelemetryFieldValue.doubleValue(key, value, unit, sourcePath));
|
||||
}
|
||||
}
|
||||
|
||||
private static void addLong(TelemetrySnapshot.Builder builder,
|
||||
String key,
|
||||
Integer value,
|
||||
String unit,
|
||||
String sourcePath) {
|
||||
if (value != null) {
|
||||
builder.addField(TelemetryFieldValue.longValue(key, value, unit, sourcePath));
|
||||
}
|
||||
}
|
||||
|
||||
private static void addString(TelemetrySnapshot.Builder builder,
|
||||
String key,
|
||||
String value,
|
||||
String unit,
|
||||
String sourcePath) {
|
||||
if (value != null && !value.isBlank()) {
|
||||
builder.addField(TelemetryFieldValue.stringValue(key, value, unit, sourcePath));
|
||||
}
|
||||
}
|
||||
|
||||
private static void addEnum(TelemetrySnapshot.Builder builder,
|
||||
String key,
|
||||
Enum<?> value,
|
||||
String unit,
|
||||
String sourcePath) {
|
||||
if (value != null) {
|
||||
builder.addField(TelemetryFieldValue.stringValue(key, value.name(), unit, sourcePath));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,9 @@ import java.util.Map;
|
||||
|
||||
/**
|
||||
* 一次消息处理的上下文，贯穿整个拦截链与 Handler。线程不共享，不需要同步。
|
||||
*
|
||||
* <p>attributes 用于在入口、拦截器、Dispatcher 之间传递轻量元数据，例如 traceId、归档 URI、
|
||||
* 鉴权结果等；不要放大对象或长期缓存，避免高频上报时放大内存占用。
|
||||
*/
|
||||
public final class IngestContext {
|
||||
|
||||
@@ -32,6 +35,7 @@ public final class IngestContext {
|
||||
}
|
||||
|
||||
public void abort(String reason) {
|
||||
// reason 会进入日志/诊断信息，保持短小、机器可读，方便定位是去重、限流还是鉴权失败。
|
||||
this.aborted = true;
|
||||
this.abortReason = reason;
|
||||
}
|
||||
@@ -15,6 +15,9 @@ import java.util.Map;
|
||||
* @param rawBytes 原始字节，用于冷存与排障（可能为 null，按配置开关）
|
||||
* @param sourceMeta 来源元数据：peer ip、端口、session id、topic 等
|
||||
* @param receivedAt 服务器接收时刻
|
||||
*
|
||||
* <p>RawFrame 是所有协议进入统一 Pipeline 的边界对象。32960 TCP 入口会把 VIN、seq、
|
||||
* rawArchiveUri 等关键索引放进 sourceMeta，后续去重、快照和历史查询都依赖这些字段保持稳定。
|
||||
*/
|
||||
public record RawFrame(
|
||||
ProtocolId protocolId,
|
||||
@@ -0,0 +1,55 @@
|
||||
package com.lingniu.ingest.api.consumer;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
class EnvelopeConsumerProcessorTest {
|
||||
|
||||
@Test
|
||||
void invalidEnvelopePublishesDeadLetterRecordWithKafkaPosition() {
|
||||
CapturingDeadLetterSink deadLetters = new CapturingDeadLetterSink();
|
||||
EnvelopeConsumerProcessor processor = new EnvelopeConsumerProcessor(
|
||||
"event-history", ignored -> EnvelopeIngestResult.invalid("VehicleEnvelope bytes are invalid"), deadLetters);
|
||||
|
||||
EnvelopeIngestResult result = processor.process(
|
||||
new EnvelopeConsumerRecord("vehicle.realtime", 2, 42L, "VIN001", new byte[]{0x01, 0x02}));
|
||||
|
||||
assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.INVALID_ENVELOPE);
|
||||
assertThat(deadLetters.records).singleElement().satisfies(record -> {
|
||||
assertThat(record.service()).isEqualTo("event-history");
|
||||
assertThat(record.topic()).isEqualTo("vehicle.realtime");
|
||||
assertThat(record.partition()).isEqualTo(2);
|
||||
assertThat(record.offset()).isEqualTo(42L);
|
||||
assertThat(record.key()).isEqualTo("VIN001");
|
||||
assertThat(record.status()).isEqualTo(EnvelopeIngestResult.Status.INVALID_ENVELOPE);
|
||||
assertThat(record.message()).contains("VehicleEnvelope");
|
||||
assertThat(record.payload()).containsExactly(0x01, 0x02);
|
||||
});
|
||||
}
|
||||
|
||||
@Test
|
||||
void processedEnvelopeDoesNotPublishDeadLetter() {
|
||||
CapturingDeadLetterSink deadLetters = new CapturingDeadLetterSink();
|
||||
EnvelopeConsumerProcessor processor = new EnvelopeConsumerProcessor(
|
||||
"vehicle-state", ignored -> EnvelopeIngestResult.processed("event-1", "VIN001"), deadLetters);
|
||||
|
||||
EnvelopeIngestResult result = processor.process(
|
||||
new EnvelopeConsumerRecord("vehicle.realtime", 0, 7L, "VIN001", new byte[]{0x0a}));
|
||||
|
||||
assertThat(result.status()).isEqualTo(EnvelopeIngestResult.Status.PROCESSED);
|
||||
assertThat(deadLetters.records).isEmpty();
|
||||
}
|
||||
|
||||
private static final class CapturingDeadLetterSink implements EnvelopeDeadLetterSink {
|
||||
private final List<EnvelopeDeadLetterRecord> records = new ArrayList<>();
|
||||
|
||||
@Override
|
||||
public void publish(EnvelopeDeadLetterRecord record) {
|
||||
records.add(record);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
package com.lingniu.ingest.api.event;
|
||||
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
class RawArchiveKeysTest {
|
||||
|
||||
@Test
|
||||
void keyDateUsesShanghaiBusinessDayInsteadOfUtcDay() {
|
||||
String key = RawArchiveKeys.key(
|
||||
Instant.parse("2026-06-22T16:30:00Z"),
|
||||
ProtocolId.GB32960,
|
||||
"VIN001",
|
||||
"event-1");
|
||||
|
||||
assertThat(key).isEqualTo("2026/06/23/GB32960/VIN001/event-1.bin");
|
||||
}
|
||||
|
||||
@Test
|
||||
void keyForRawArchiveUsesPrecomputedMetadataKey() {
|
||||
VehicleEvent.RawArchive raw = new VehicleEvent.RawArchive(
|
||||
"event-1",
|
||||
"VIN001",
|
||||
ProtocolId.GB32960,
|
||||
Instant.parse("2026-06-22T16:30:00Z"),
|
||||
Instant.parse("2026-06-22T16:30:00Z"),
|
||||
"trace-1",
|
||||
Map.of(RawArchiveKeys.META_KEY, "precomputed/key.bin"),
|
||||
0x02,
|
||||
0,
|
||||
new byte[]{0x23, 0x23});
|
||||
|
||||
assertThat(RawArchiveKeys.key(raw)).isEqualTo("precomputed/key.bin");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
package com.lingniu.ingest.api.event;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
class TelemetryFieldValueTest {
|
||||
|
||||
@Test
|
||||
void createsTypedDoubleFieldWithStableMetadata() {
|
||||
TelemetryFieldValue field = TelemetryFieldValue.doubleValue(
|
||||
"total_mileage_km", 12345.6, "km", "gb32960.vehicle.totalMileageKm");
|
||||
|
||||
assertThat(field.key()).isEqualTo("total_mileage_km");
|
||||
assertThat(field.valueType()).isEqualTo(TelemetryFieldValue.ValueType.DOUBLE);
|
||||
assertThat(field.value()).isEqualTo("12345.6");
|
||||
assertThat(field.unit()).isEqualTo("km");
|
||||
assertThat(field.quality()).isEqualTo(TelemetryFieldValue.Quality.GOOD);
|
||||
assertThat(field.sourcePath()).isEqualTo("gb32960.vehicle.totalMileageKm");
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsBlankFieldKey() {
|
||||
assertThatThrownBy(() -> TelemetryFieldValue.stringValue(" ", "STARTED", "", "gb32960.vehicle.state"))
|
||||
.isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessageContaining("key");
|
||||
}
|
||||
|
||||
@Test
|
||||
void marksMissingValueExplicitly() {
|
||||
TelemetryFieldValue field = TelemetryFieldValue.missing(
|
||||
"hydrogen_remaining_kg", "kg", "gb32960.vendor.hydrogenRemaining");
|
||||
|
||||
assertThat(field.valueType()).isEqualTo(TelemetryFieldValue.ValueType.STRING);
|
||||
assertThat(field.value()).isEmpty();
|
||||
assertThat(field.quality()).isEqualTo(TelemetryFieldValue.Quality.MISSING);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,58 @@
|
||||
package com.lingniu.ingest.api.event;
|
||||
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
import static org.assertj.core.api.Assertions.assertThatThrownBy;
|
||||
|
||||
class TelemetrySnapshotTest {
|
||||
|
||||
@Test
|
||||
void indexesFieldsByInternalKey() {
|
||||
TelemetrySnapshot snapshot = new TelemetrySnapshot(
|
||||
"event-1",
|
||||
"VIN001",
|
||||
ProtocolId.GB32960,
|
||||
"REALTIME",
|
||||
Instant.parse("2026-06-22T08:00:00Z"),
|
||||
Instant.parse("2026-06-22T08:00:01Z"),
|
||||
"archive://raw/event-1.bin",
|
||||
Map.of("protocolVersion", "V2016"),
|
||||
List.of(
|
||||
TelemetryFieldValue.doubleValue("total_mileage_km", 120.5, "km", "gb32960.vehicle.totalMileageKm"),
|
||||
TelemetryFieldValue.booleanValue("hydrogen_leak_detected", true, "", "gb32960.alarm.HYDROGEN_LEAK")
|
||||
)
|
||||
);
|
||||
|
||||
assertThat(snapshot.field("total_mileage_km")).isPresent();
|
||||
assertThat(snapshot.field("total_mileage_km").orElseThrow().value()).isEqualTo("120.5");
|
||||
assertThat(snapshot.field("hydrogen_leak_detected").orElseThrow().value()).isEqualTo("true");
|
||||
assertThat(snapshot.field("unknown")).isEmpty();
|
||||
assertThat(snapshot.metadata()).containsEntry("protocolVersion", "V2016");
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsDuplicateFieldKeys() {
|
||||
assertThatThrownBy(() -> new TelemetrySnapshot(
|
||||
"event-1",
|
||||
"VIN001",
|
||||
ProtocolId.GB32960,
|
||||
"REALTIME",
|
||||
Instant.parse("2026-06-22T08:00:00Z"),
|
||||
Instant.parse("2026-06-22T08:00:01Z"),
|
||||
"",
|
||||
Map.of(),
|
||||
List.of(
|
||||
TelemetryFieldValue.doubleValue("speed_kmh", 1.0, "km/h", "a"),
|
||||
TelemetryFieldValue.doubleValue("speed_kmh", 2.0, "km/h", "b")
|
||||
)
|
||||
))
|
||||
.isInstanceOf(IllegalArgumentException.class)
|
||||
.hasMessageContaining("duplicate");
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,175 @@
|
||||
package com.lingniu.ingest.api.event;
|
||||
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
class VehicleEventTelemetrySnapshotMapperTest {
|
||||
|
||||
@Test
|
||||
void mapsRealtimeEventToInternalTelemetryFields() {
|
||||
VehicleEvent.Realtime event = new VehicleEvent.Realtime(
|
||||
"event-1",
|
||||
"VIN001",
|
||||
ProtocolId.GB32960,
|
||||
Instant.parse("2026-06-22T08:00:00Z"),
|
||||
Instant.parse("2026-06-22T08:00:01Z"),
|
||||
"trace-1",
|
||||
Map.of("protocolVersion", "V2016", "rawArchiveUri", "archive://raw/event-1.bin"),
|
||||
new RealtimePayload(
|
||||
80.5,
|
||||
12345.6,
|
||||
55.0,
|
||||
610.0,
|
||||
-20.5,
|
||||
220.0,
|
||||
100.0,
|
||||
65.0,
|
||||
8.2,
|
||||
35.0,
|
||||
1.1,
|
||||
RealtimePayload.VehicleState.STARTED,
|
||||
RealtimePayload.ChargingState.UNCHARGED,
|
||||
RealtimePayload.RunningMode.FUEL,
|
||||
3,
|
||||
20.0,
|
||||
0.0,
|
||||
113.12,
|
||||
23.45,
|
||||
10.0,
|
||||
180.0,
|
||||
30.0,
|
||||
70.0
|
||||
)
|
||||
);
|
||||
|
||||
TelemetrySnapshot snapshot = VehicleEventTelemetrySnapshotMapper.toSnapshot(event).orElseThrow();
|
||||
|
||||
assertThat(snapshot.eventId()).isEqualTo("event-1");
|
||||
assertThat(snapshot.rawArchiveUri()).isEqualTo("archive://raw/event-1.bin");
|
||||
assertThat(snapshot.field("speed_kmh").orElseThrow().value()).isEqualTo("80.5");
|
||||
assertThat(snapshot.field("total_mileage_km").orElseThrow().value()).isEqualTo("12345.6");
|
||||
assertThat(snapshot.field("vehicle_state").orElseThrow().value()).isEqualTo("STARTED");
|
||||
assertThat(snapshot.field("hydrogen_high_pressure_mpa").orElseThrow().value()).isEqualTo("35.0");
|
||||
assertThat(snapshot.field("longitude").orElseThrow().value()).isEqualTo("113.12");
|
||||
}
|
||||
|
||||
@Test
|
||||
void mapsHydrogenLeakAlarmToInternalSafetyFields() {
|
||||
VehicleEvent.Alarm event = new VehicleEvent.Alarm(
|
||||
"event-2",
|
||||
"VIN001",
|
||||
ProtocolId.GB32960,
|
||||
Instant.parse("2026-06-22T08:00:00Z"),
|
||||
Instant.parse("2026-06-22T08:00:01Z"),
|
||||
"trace-2",
|
||||
Map.of("protocolVersion", "V2016"),
|
||||
new AlarmPayload(
|
||||
AlarmPayload.AlarmLevel.CRITICAL,
|
||||
1,
|
||||
"GB32960_ALARM",
|
||||
java.util.List.of("OTH-1"),
|
||||
java.util.Set.of("HYDROGEN_LEAK"),
|
||||
113.12,
|
||||
23.45,
|
||||
AlarmPayload.SafetyCategory.HYDROGEN_LEAK,
|
||||
true,
|
||||
AlarmPayload.HydrogenLeakLevel.CRITICAL,
|
||||
true
|
||||
)
|
||||
);
|
||||
|
||||
TelemetrySnapshot snapshot = VehicleEventTelemetrySnapshotMapper.toSnapshot(event).orElseThrow();
|
||||
|
||||
assertThat(snapshot.eventType()).isEqualTo("ALARM");
|
||||
assertThat(snapshot.field("hydrogen_leak_detected").orElseThrow().value()).isEqualTo("true");
|
||||
assertThat(snapshot.field("hydrogen_leak_level").orElseThrow().value()).isEqualTo("CRITICAL");
|
||||
assertThat(snapshot.field("hydrogen_leak_action_required").orElseThrow().value()).isEqualTo("true");
|
||||
assertThat(snapshot.field("safety_category").orElseThrow().value()).isEqualTo("HYDROGEN_LEAK");
|
||||
}
|
||||
|
||||
@Test
|
||||
void mapsLocationMileageToInternalTelemetryField() {
|
||||
VehicleEvent.Location event = new VehicleEvent.Location(
|
||||
"event-location-1",
|
||||
"VIN001",
|
||||
ProtocolId.JT808,
|
||||
Instant.parse("2026-06-22T08:00:00Z"),
|
||||
Instant.parse("2026-06-22T08:00:01Z"),
|
||||
"trace-location-1",
|
||||
Map.of("messageId", "0x200"),
|
||||
new LocationPayload(113.12, 23.45, 10.0, 60.0, 90.0, 0, 1, 1234.5)
|
||||
);
|
||||
|
||||
TelemetrySnapshot snapshot = VehicleEventTelemetrySnapshotMapper.toSnapshot(event).orElseThrow();
|
||||
|
||||
assertThat(snapshot.field("total_mileage_km").orElseThrow().value()).isEqualTo("1234.5");
|
||||
assertThat(snapshot.field("total_mileage_km").orElseThrow().sourcePath())
|
||||
.isEqualTo("event.location.totalMileageKm");
|
||||
}
|
||||
|
||||
@Test
|
||||
void mediaMetaUsesArchiveRefAsQueryableRawArchiveUriFallback() {
|
||||
VehicleEvent.MediaMeta event = new VehicleEvent.MediaMeta(
|
||||
"media-1",
|
||||
"VIN001",
|
||||
ProtocolId.JT1078,
|
||||
Instant.parse("2026-06-22T08:00:00Z"),
|
||||
Instant.parse("2026-06-22T08:00:01Z"),
|
||||
"trace-4",
|
||||
Map.of("channel", "1"),
|
||||
"media-001",
|
||||
"jt1078-video-h264-channel-1",
|
||||
1024,
|
||||
"file:///archive/jt1078/2026/06/22/VIN001/media-001.rtp"
|
||||
);
|
||||
|
||||
TelemetrySnapshot snapshot = VehicleEventTelemetrySnapshotMapper.toSnapshot(event).orElseThrow();
|
||||
|
||||
assertThat(snapshot.eventType()).isEqualTo("MEDIA_META");
|
||||
assertThat(snapshot.rawArchiveUri())
|
||||
.isEqualTo("file:///archive/jt1078/2026/06/22/VIN001/media-001.rtp");
|
||||
assertThat(snapshot.field("media_archive_ref").orElseThrow().value())
|
||||
.isEqualTo("file:///archive/jt1078/2026/06/22/VIN001/media-001.rtp");
|
||||
}
|
||||
|
||||
@Test
|
||||
void derivesRawArchiveUriFromRawArchiveKeyMetadata() {
|
||||
VehicleEvent.Heartbeat event = new VehicleEvent.Heartbeat(
|
||||
"heartbeat-1",
|
||||
"VIN001",
|
||||
ProtocolId.JT808,
|
||||
Instant.parse("2026-06-22T08:00:00Z"),
|
||||
Instant.parse("2026-06-22T08:00:01Z"),
|
||||
"trace-5",
|
||||
Map.of("rawArchiveKey", "2026/06/22/JT808/VIN001/raw-1.bin")
|
||||
);
|
||||
|
||||
TelemetrySnapshot snapshot = VehicleEventTelemetrySnapshotMapper.toSnapshot(event).orElseThrow();
|
||||
|
||||
assertThat(snapshot.rawArchiveUri())
|
||||
.isEqualTo("archive://2026/06/22/JT808/VIN001/raw-1.bin");
|
||||
}
|
||||
|
||||
@Test
|
||||
void skipsRawArchiveBecauseItIsStoredByArchiveSink() {
|
||||
VehicleEvent.RawArchive raw = new VehicleEvent.RawArchive(
|
||||
"raw-1",
|
||||
"VIN001",
|
||||
ProtocolId.GB32960,
|
||||
Instant.parse("2026-06-22T08:00:00Z"),
|
||||
Instant.parse("2026-06-22T08:00:01Z"),
|
||||
"trace-3",
|
||||
Map.of(),
|
||||
0x02,
|
||||
0,
|
||||
new byte[]{0x23, 0x23}
|
||||
);
|
||||
|
||||
assertThat(VehicleEventTelemetrySnapshotMapper.toSnapshot(raw)).isEmpty();
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,7 @@
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>lingniu-vehicle-ingest</artifactId>
|
||||
<version>0.1.0-SNAPSHOT</version>
|
||||
<relativePath>../../../pom.xml</relativePath>
|
||||
</parent>
|
||||
<artifactId>ingest-codec-common</artifactId>
|
||||
<name>ingest-codec-common</name>
|
||||
@@ -5,6 +5,7 @@
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>lingniu-vehicle-ingest</artifactId>
|
||||
<version>0.1.0-SNAPSHOT</version>
|
||||
<relativePath>../../../pom.xml</relativePath>
|
||||
</parent>
|
||||
<artifactId>ingest-core</artifactId>
|
||||
<name>ingest-core</name>
|
||||
@@ -16,6 +16,7 @@ import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.ConcurrentMap;
|
||||
import java.util.concurrent.TimeUnit;
|
||||
import java.util.function.Consumer;
|
||||
import java.util.function.UnaryOperator;
|
||||
|
||||
/**
|
||||
* {@link AsyncBatch} 注解的执行器：把 Handler 方法从"单条调用"变成"批量调用"。
|
||||
@@ -51,7 +52,12 @@ public final class AsyncBatchExecutor implements AutoCloseable {
|
||||
/** 供 Dispatcher 调用：把单条消息交给目标 Handler 的 batcher。 */
|
||||
public void submit(HandlerDefinition def, Object message) {
|
||||
Batcher b = batchers.computeIfAbsent(def.method(), m -> new Batcher(def, eventPublisher));
|
||||
b.offer(message);
|
||||
b.offer(new BatchItem(message, UnaryOperator.identity(), false));
|
||||
}
|
||||
|
||||
public void submit(HandlerDefinition def, Object message, UnaryOperator<VehicleEvent> eventTransformer) {
|
||||
Batcher b = batchers.computeIfAbsent(def.method(), m -> new Batcher(def, eventPublisher));
|
||||
b.offer(new BatchItem(message, eventTransformer == null ? UnaryOperator.identity() : eventTransformer, true));
|
||||
}
|
||||
|
||||
@Override
|
||||
@@ -68,7 +74,7 @@ public final class AsyncBatchExecutor implements AutoCloseable {
|
||||
private final Consumer<VehicleEvent> publisher;
|
||||
private final int batchSize;
|
||||
private final long waitMs;
|
||||
private final BlockingQueue<Object> queue;
|
||||
private final BlockingQueue<BatchItem> queue;
|
||||
private final Thread[] workers;
|
||||
private volatile boolean running = true;
|
||||
|
||||
@@ -92,7 +98,7 @@ public final class AsyncBatchExecutor implements AutoCloseable {
|
||||
def.method().getName(), batchSize, waitMs, pool);
|
||||
}
|
||||
|
||||
void offer(Object item) {
|
||||
void offer(BatchItem item) {
|
||||
try {
|
||||
if (!queue.offer(item, 100, TimeUnit.MILLISECONDS)) {
|
||||
log.warn("batcher queue full, dropping item for {}", def.method().getName());
|
||||
@@ -105,15 +111,15 @@ public final class AsyncBatchExecutor implements AutoCloseable {
|
||||
private void loop() {
|
||||
while (running) {
|
||||
try {
|
||||
Object first = queue.poll(200, TimeUnit.MILLISECONDS);
|
||||
BatchItem first = queue.poll(200, TimeUnit.MILLISECONDS);
|
||||
if (first == null) continue;
|
||||
List<Object> buf = new ArrayList<>(batchSize);
|
||||
List<BatchItem> buf = new ArrayList<>(batchSize);
|
||||
buf.add(first);
|
||||
long deadlineNanos = System.nanoTime() + waitMs * 1_000_000L;
|
||||
while (buf.size() < batchSize) {
|
||||
long remain = deadlineNanos - System.nanoTime();
|
||||
if (remain <= 0) break;
|
||||
Object next = queue.poll(remain, TimeUnit.NANOSECONDS);
|
||||
BatchItem next = queue.poll(remain, TimeUnit.NANOSECONDS);
|
||||
if (next == null) break;
|
||||
buf.add(next);
|
||||
}
|
||||
@@ -127,11 +133,38 @@ public final class AsyncBatchExecutor implements AutoCloseable {
|
||||
}
|
||||
}
|
||||
|
||||
private void flush(List<BatchItem> buf) {
|
||||
if (requiresPerItemTransform(buf)) {
|
||||
// rawArchiveUri/rawSize 等元数据是每帧唯一的。为了不把第一条帧的元数据错误复制给整批事件，
|
||||
// 一旦存在逐条 transformer，就按单条调用 Handler，再逐条发布。
|
||||
for (BatchItem item : buf) {
|
||||
flushTransformedItem(item);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
publishEvents(invoke(buf.stream().map(BatchItem::message).toList()), UnaryOperator.identity());
|
||||
}
|
||||
|
||||
private boolean requiresPerItemTransform(List<BatchItem> buf) {
|
||||
for (BatchItem item : buf) {
|
||||
if (item.transformRequired()) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
private void flushTransformedItem(BatchItem item) {
|
||||
publishEvents(invoke(List.of(item.message())), item.eventTransformer());
|
||||
}
|
||||
|
||||
@SuppressWarnings("unchecked")
|
||||
private void flush(List<Object> buf) {
|
||||
private List<VehicleEvent> invoke(List<Object> messages) {
|
||||
List<VehicleEvent> events;
|
||||
try {
|
||||
Object result = def.method().invoke(def.bean(), buf);
|
||||
// Handler 仍然只暴露一个批量签名；是否批满或超时由 Batcher 统一处理。
|
||||
Object result = def.method().invoke(def.bean(), messages);
|
||||
events = switch (result) {
|
||||
case null -> List.of();
|
||||
case List<?> list -> (List<VehicleEvent>) list;
|
||||
@@ -141,14 +174,18 @@ public final class AsyncBatchExecutor implements AutoCloseable {
|
||||
};
|
||||
} catch (InvocationTargetException e) {
|
||||
log.error("batcher invoke failed method={}", def.method().getName(), e.getCause());
|
||||
return;
|
||||
return List.of();
|
||||
} catch (IllegalAccessException e) {
|
||||
log.error("batcher access denied method={}", def.method().getName(), e);
|
||||
return;
|
||||
return List.of();
|
||||
}
|
||||
return events;
|
||||
}
|
||||
|
||||
private void publishEvents(List<VehicleEvent> events, UnaryOperator<VehicleEvent> eventTransformer) {
|
||||
for (VehicleEvent e : events) {
|
||||
try {
|
||||
publisher.accept(e);
|
||||
publisher.accept(eventTransformer.apply(e));
|
||||
} catch (Exception ex) {
|
||||
log.warn("batcher publish failed eventId={}", e.eventId(), ex);
|
||||
}
|
||||
@@ -161,4 +198,9 @@ public final class AsyncBatchExecutor implements AutoCloseable {
|
||||
for (Thread t : workers) t.interrupt();
|
||||
}
|
||||
}
|
||||
|
||||
private record BatchItem(Object message,
|
||||
UnaryOperator<VehicleEvent> eventTransformer,
|
||||
boolean transformRequired) {
|
||||
}
|
||||
}
|
||||
@@ -43,6 +43,7 @@ public final class DisruptorEventBus implements AutoCloseable {
|
||||
EventHandler<VehicleEventSlot>[] handlers = sinks.stream()
|
||||
.map(this::toHandler)
|
||||
.toArray(EventHandler[]::new);
|
||||
// 每个 sink 一个独立 handler，事件按扇出模式同时写 Kafka、event-file-store 等目标。
|
||||
disruptor.handleEventsWith(handlers);
|
||||
disruptor.start();
|
||||
log.info("DisruptorEventBus started ringBuffer={} wait={} sinks={}",
|
||||
@@ -66,6 +67,7 @@ public final class DisruptorEventBus implements AutoCloseable {
|
||||
if (e == null || !sink.accepts(e)) return;
|
||||
try {
|
||||
sink.publish(e).exceptionally(ex -> {
|
||||
// sink 异步失败只计数和打日志，不反向阻塞入站连接；生产侧靠 sink 自身重试/DLQ 保证可观测。
|
||||
failed.incrementAndGet();
|
||||
log.warn("sink {} publish failed", sink.name(), ex);
|
||||
return null;
|
||||
@@ -21,6 +21,10 @@ import java.util.List;
|
||||
|
||||
/**
|
||||
* ingest-core 的自动装配入口。所有 Bean 都是 {@code @ConditionalOnMissingBean}，便于下游替换。
|
||||
*
|
||||
* <p>生产链路顺序：入口收到 {@code RawFrame} → {@link InterceptorChain} 做准入 →
|
||||
* {@link Dispatcher} 定位协议 Handler → Handler 产出 {@code VehicleEvent} →
|
||||
* {@link DisruptorEventBus} 扇出到 Archive、DuckDB、Kafka 等 {@code EventSink}。
|
||||
*/
|
||||
@AutoConfiguration
|
||||
@EnableConfigurationProperties(IngestCoreProperties.class)
|
||||
@@ -46,6 +50,7 @@ public class IngestCoreAutoConfiguration {
|
||||
@Bean
|
||||
@ConditionalOnProperty(prefix = "lingniu.ingest.pipeline.dedup", name = "enabled", havingValue = "true", matchIfMissing = true)
|
||||
public DedupInterceptor dedupInterceptor(IngestCoreProperties props) {
|
||||
// 去重尽量放在限流前，重复包不应消耗后续每 VIN QPS 配额。
|
||||
return new DedupInterceptor(props.getDedup().getCacheSize(), props.getDedup().getTtlSeconds());
|
||||
}
|
||||
|
||||
@@ -19,7 +19,9 @@ public class IngestCoreProperties {
|
||||
public void setRateLimit(RateLimit rateLimit) { this.rateLimit = rateLimit; }
|
||||
|
||||
public static class Disruptor {
|
||||
/** 事件总线 ring buffer 大小；生产环境应保持 2 的幂，避免 Disruptor 初始化失败。 */
|
||||
private int ringBufferSize = 131072;
|
||||
/** waiting 策略直接影响延迟和 CPU 占用，默认 yielding 偏低延迟。 */
|
||||
private String waitStrategy = "yielding";
|
||||
private String producerType = "multi";
|
||||
|
||||
@@ -33,7 +35,9 @@ public class IngestCoreProperties {
|
||||
|
||||
public static class Dedup {
|
||||
private boolean enabled = true;
|
||||
/** 单实例缓存的幂等键数量上限，按 VIN 数和重发窗口估算。 */
|
||||
private int cacheSize = 200000;
|
||||
/** 幂等键保留窗口；过短会放过重发包，过长会增加内存压力。 */
|
||||
private long ttlSeconds = 600;
|
||||
|
||||
public boolean isEnabled() { return enabled; }
|
||||
@@ -45,7 +49,9 @@ public class IngestCoreProperties {
|
||||
}
|
||||
|
||||
public static class RateLimit {
|
||||
/** 每 VIN 每秒进入 Handler 的最大帧数，默认按 32960 高频实时上报预留。 */
|
||||
private int perVinQps = 50;
|
||||
/** VIN limiter 缓存上限；覆盖活跃车辆数即可，淘汰后会重新创建 limiter。 */
|
||||
private int maxVins = 100000;
|
||||
|
||||
public int getPerVinQps() { return perVinQps; }
|
||||
@@ -43,6 +43,7 @@ public class AnnotationHandlerBeanPostProcessor implements BeanPostProcessor {
|
||||
int[] commands = mapping.command().length == 0 ? new int[]{0} : mapping.command();
|
||||
int[] infoTypes = mapping.infoType().length == 0 ? new int[]{0} : mapping.infoType();
|
||||
|
||||
// 这些注解只声明策略，实际限流、幂等、异步批处理由 Dispatcher/Interceptor 层解释执行。
|
||||
RateLimited rl = AnnotatedElementUtils.findMergedAnnotation(method, RateLimited.class);
|
||||
IdempotentKey ik = AnnotatedElementUtils.findMergedAnnotation(method, IdempotentKey.class);
|
||||
AsyncBatch ab = AnnotatedElementUtils.findMergedAnnotation(method, AsyncBatch.class);
|
||||
@@ -0,0 +1,185 @@
|
||||
package com.lingniu.ingest.core.dispatcher;
|
||||
|
||||
import com.lingniu.ingest.api.event.RawArchiveKeys;
|
||||
import com.lingniu.ingest.api.event.VehicleEvent;
|
||||
import com.lingniu.ingest.api.pipeline.IngestContext;
|
||||
import com.lingniu.ingest.api.pipeline.RawFrame;
|
||||
import com.lingniu.ingest.core.concurrency.AsyncBatchExecutor;
|
||||
import com.lingniu.ingest.core.concurrency.DisruptorEventBus;
|
||||
import com.lingniu.ingest.core.pipeline.InterceptorChain;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.LinkedHashMap;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.UUID;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
|
||||
/**
|
||||
* 核心分发器：RawFrame → 拦截链 → Handler → 事件 → Disruptor EventBus。
|
||||
*
|
||||
* <p>所有 Inbound Adapter（Netty / MQTT / PushClient）都调用 {@link #dispatch(RawFrame)}，
|
||||
* 协议差异在这里被彻底抹平。
|
||||
*
|
||||
* <p>对于带 {@code @AsyncBatch} 的 Handler，Dispatcher 不直接反射调用，而是把消息交给
|
||||
* {@link AsyncBatchExecutor} 进行聚合 → 批量调用 → 异步发布事件。
|
||||
*/
|
||||
public final class Dispatcher {
|
||||
|
||||
private static final Logger log = LoggerFactory.getLogger(Dispatcher.class);
|
||||
private static final AtomicLong RAW_ARCHIVE_SEQUENCE = new AtomicLong();
|
||||
|
||||
private final HandlerRegistry registry;
|
||||
private final InterceptorChain interceptors;
|
||||
private final HandlerInvoker invoker;
|
||||
private final DisruptorEventBus eventBus;
|
||||
private final AsyncBatchExecutor batchExecutor;
|
||||
|
||||
public Dispatcher(HandlerRegistry registry,
|
||||
InterceptorChain interceptors,
|
||||
HandlerInvoker invoker,
|
||||
DisruptorEventBus eventBus,
|
||||
AsyncBatchExecutor batchExecutor) {
|
||||
this.registry = registry;
|
||||
this.interceptors = interceptors;
|
||||
this.invoker = invoker;
|
||||
this.eventBus = eventBus;
|
||||
this.batchExecutor = batchExecutor;
|
||||
}
|
||||
|
||||
public void dispatch(RawFrame frame) {
|
||||
IngestContext ctx = new IngestContext(UUID.randomUUID().toString());
|
||||
try {
|
||||
// 在 interceptor 之前发 RawArchive，保证原始字节被无条件落盘（dedup/rate-limit
|
||||
// 不会过滤它），满足"原始可回放"目标。archive 的写盘靠下游 ArchiveEventSink 消费。
|
||||
RawArchiveLookup rawArchive = emitRawArchive(frame, ctx);
|
||||
|
||||
if (!interceptors.before(frame, ctx)) {
|
||||
log.debug("frame aborted: {}", ctx.abortReason());
|
||||
return;
|
||||
}
|
||||
|
||||
List<HandlerDefinition> handlers = registry.resolve(
|
||||
frame.protocolId(), frame.command(), frame.infoType());
|
||||
if (handlers.isEmpty()) {
|
||||
log.debug("no handler for {} cmd=0x{} info=0x{}",
|
||||
frame.protocolId(), Integer.toHexString(frame.command()),
|
||||
Integer.toHexString(frame.infoType()));
|
||||
return;
|
||||
}
|
||||
|
||||
for (HandlerDefinition def : handlers) {
|
||||
if (def.asyncBatch() != null) {
|
||||
batchExecutor.submit(def, frame.payload(), event -> enrichWithRawArchive(event, rawArchive));
|
||||
continue;
|
||||
}
|
||||
List<VehicleEvent> events = invoker.invoke(def, frame, ctx);
|
||||
for (VehicleEvent e : events) {
|
||||
e = enrichWithRawArchive(e, rawArchive);
|
||||
interceptors.after(e, ctx);
|
||||
eventBus.publish(e);
|
||||
}
|
||||
}
|
||||
} catch (Throwable t) {
|
||||
log.error("dispatch failure traceId={}", ctx.traceId(), t);
|
||||
interceptors.onError(t, ctx);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* 从 {@link RawFrame} 构造一条 {@link VehicleEvent.RawArchive} 发到 EventBus。
|
||||
* 仅在 {@code rawBytes} 非空时发——有些入站适配器（未来）可能只传解析后的对象。
|
||||
*
|
||||
* <p>VIN 取自 sourceMeta 里的 {@code vin} key（由各入站适配器负责填充）；缺失时
|
||||
* 留空字符串，archive sink 会用 "unknown-vin" 占位保证 key 可解析。
|
||||
*/
|
||||
private RawArchiveLookup emitRawArchive(RawFrame frame, IngestContext ctx) {
|
||||
byte[] bytes = frame.rawBytes();
|
||||
if (bytes == null || bytes.length == 0) return RawArchiveLookup.empty();
|
||||
|
||||
Map<String, String> meta = frame.sourceMeta() == null ? Map.of() : frame.sourceMeta();
|
||||
String vin = meta.getOrDefault("vin", "");
|
||||
Instant ingestTime = frame.receivedAt() != null ? frame.receivedAt() : Instant.now();
|
||||
String eventId = nextRawArchiveEventId(ingestTime);
|
||||
String key = RawArchiveKeys.key(ingestTime, frame.protocolId(), vin, eventId);
|
||||
Map<String, String> archiveMeta = addRawArchiveMetadata(meta, eventId, key);
|
||||
|
||||
VehicleEvent.RawArchive raw = new VehicleEvent.RawArchive(
|
||||
eventId,
|
||||
vin,
|
||||
frame.protocolId(),
|
||||
ingestTime,
|
||||
ingestTime,
|
||||
ctx.traceId(),
|
||||
archiveMeta,
|
||||
frame.command(),
|
||||
frame.infoType(),
|
||||
bytes);
|
||||
eventBus.publish(raw);
|
||||
return new RawArchiveLookup(eventId, key, RawArchiveKeys.logicalUri(key));
|
||||
}
|
||||
|
||||
private static String nextRawArchiveEventId(Instant ingestTime) {
|
||||
// 文件名需要可排序且单进程内唯一：毫秒时间放大到微秒量级，再用 AtomicLong 递增兜底。
|
||||
long base = (ingestTime == null ? Instant.now() : ingestTime).toEpochMilli() * 1000L;
|
||||
long next = RAW_ARCHIVE_SEQUENCE.updateAndGet(previous -> Math.max(previous + 1, base));
|
||||
return Long.toString(next);
|
||||
}
|
||||
|
||||
private static Map<String, String> addRawArchiveMetadata(Map<String, String> metadata,
|
||||
String eventId,
|
||||
String key) {
|
||||
Map<String, String> out = new LinkedHashMap<>(metadata == null ? Map.of() : metadata);
|
||||
out.putIfAbsent(RawArchiveKeys.META_EVENT_ID, eventId);
|
||||
out.putIfAbsent(RawArchiveKeys.META_KEY, key);
|
||||
out.putIfAbsent(RawArchiveKeys.META_URI, RawArchiveKeys.logicalUri(key));
|
||||
return Map.copyOf(out);
|
||||
}
|
||||
|
||||
private static VehicleEvent enrichWithRawArchive(VehicleEvent event, RawArchiveLookup rawArchive) {
|
||||
if (event == null || rawArchive.isEmpty() || event instanceof VehicleEvent.RawArchive) {
|
||||
return event;
|
||||
}
|
||||
// 派生事件携带 rawArchiveUri，便于后续服务从 Kafka 事件回查原始报文。
|
||||
Map<String, String> metadata = addRawArchiveMetadata(event.metadata(), rawArchive.eventId(), rawArchive.key());
|
||||
return switch (event) {
|
||||
case VehicleEvent.Realtime e -> new VehicleEvent.Realtime(
|
||||
e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(),
|
||||
e.traceId(), metadata, e.payload());
|
||||
case VehicleEvent.Location e -> new VehicleEvent.Location(
|
||||
e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(),
|
||||
e.traceId(), metadata, e.payload());
|
||||
case VehicleEvent.Alarm e -> new VehicleEvent.Alarm(
|
||||
e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(),
|
||||
e.traceId(), metadata, e.payload());
|
||||
case VehicleEvent.Login e -> new VehicleEvent.Login(
|
||||
e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(),
|
||||
e.traceId(), metadata, e.iccid(), e.protocolVersion());
|
||||
case VehicleEvent.Logout e -> new VehicleEvent.Logout(
|
||||
e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(),
|
||||
e.traceId(), metadata);
|
||||
case VehicleEvent.Heartbeat e -> new VehicleEvent.Heartbeat(
|
||||
e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(),
|
||||
e.traceId(), metadata);
|
||||
case VehicleEvent.MediaMeta e -> new VehicleEvent.MediaMeta(
|
||||
e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(),
|
||||
e.traceId(), metadata, e.mediaId(), e.mediaType(), e.sizeBytes(), e.archiveRef());
|
||||
case VehicleEvent.Passthrough e -> new VehicleEvent.Passthrough(
|
||||
e.eventId(), e.vin(), e.source(), e.eventTime(), e.ingestTime(),
|
||||
e.traceId(), metadata, e.passthroughType(), e.data());
|
||||
case VehicleEvent.RawArchive e -> e;
|
||||
};
|
||||
}
|
||||
|
||||
private record RawArchiveLookup(String eventId, String key, String uri) {
|
||||
private static RawArchiveLookup empty() {
|
||||
return new RawArchiveLookup("", "", "");
|
||||
}
|
||||
|
||||
private boolean isEmpty() {
|
||||
return key == null || key.isBlank();
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -20,6 +20,7 @@ public class HandlerInvoker {
|
||||
return switch (result) {
|
||||
case null -> Collections.emptyList();
|
||||
case VehicleEvent e -> List.of(e);
|
||||
// Handler 可以一次产出多个领域事件，例如 GB32960 0x02 同时产出 Realtime/Location/Alarm。
|
||||
case List<?> list -> (List<VehicleEvent>) list;
|
||||
default -> throw new IllegalStateException(
|
||||
"Handler return type must be VehicleEvent or List<VehicleEvent>: " + def.method());
|
||||
@@ -34,6 +34,9 @@ public final class HandlerRegistry {
|
||||
if (d.matches(protocol, command, infoType)) result.add(d);
|
||||
}
|
||||
}
|
||||
if (!result.isEmpty()) {
|
||||
return result;
|
||||
}
|
||||
for (HandlerDefinition d : wildcardHandlers) {
|
||||
if (d.matches(protocol, command, infoType)) result.add(d);
|
||||
}
|
||||
@@ -11,6 +11,11 @@ import java.util.List;
|
||||
|
||||
/**
|
||||
* 顺序执行的拦截器链，通过 Spring {@code @Order} 或 {@code Ordered} 排序。
|
||||
*
|
||||
* <p>{@link #before(RawFrame, IngestContext)} 是接入链路的准入闸门：任何拦截器返回
|
||||
* {@code false} 或调用 {@link IngestContext#abort(String)} 都会停止后续协议解析/Handler 调用。
|
||||
* {@link #after(VehicleEvent, IngestContext)} 与 {@link #onError(Throwable, IngestContext)}
|
||||
* 则保持广播语义，让指标、审计等横切逻辑都能观察到同一条处理结果。
|
||||
*/
|
||||
public final class InterceptorChain {
|
||||
|
||||
@@ -24,6 +29,7 @@ public final class InterceptorChain {
|
||||
|
||||
public boolean before(RawFrame frame, IngestContext ctx) {
|
||||
for (IngestInterceptor i : interceptors) {
|
||||
// before 阶段短路是有意的：去重、限流失败后不应继续消耗解析和存储资源。
|
||||
if (!i.before(frame, ctx) || ctx.aborted()) {
|
||||
return false;
|
||||
}
|
||||
@@ -7,13 +7,18 @@ import com.lingniu.ingest.api.pipeline.IngestInterceptor;
|
||||
import com.lingniu.ingest.api.pipeline.RawFrame;
|
||||
import org.springframework.core.Ordered;
|
||||
|
||||
import java.util.Arrays;
|
||||
import java.time.Duration;
|
||||
|
||||
/**
|
||||
* 基于 Caffeine 的本地幂等去重。
|
||||
*
|
||||
* <p>Key 构造：{@code protocolId + command + sourceMeta.seq} —— 真实实现可以接入 Redis 做多节点一致性，
|
||||
* 本实现只覆盖单节点场景，满足第一阶段 PoC 需求。
|
||||
* <p>Key 构造：{@code protocolId + vin + command + seq/fingerprint}。如果上游已经在
|
||||
* {@link RawFrame#sourceMeta()} 里带了 {@code seq}，优先用真实流水号；否则退化为 raw bytes
|
||||
* fingerprint，避免设备重连或 TCP 重发导致同一帧重复进入 DuckDB/Archive。
|
||||
*
|
||||
* <p>当前缓存是进程内的，只能保证单实例去重。32960 若做多副本水平扩容，需要把这个位置替换成
|
||||
* Redis/集中式幂等键，否则不同实例仍可能各自接收一次相同原始包。
|
||||
*/
|
||||
public class DedupInterceptor implements IngestInterceptor, Ordered {
|
||||
|
||||
@@ -29,7 +34,11 @@ public class DedupInterceptor implements IngestInterceptor, Ordered {
|
||||
@Override
|
||||
public boolean before(RawFrame frame, IngestContext ctx) {
|
||||
String vin = frame.sourceMeta().getOrDefault("vin", "unknown");
|
||||
String seq = frame.sourceMeta().getOrDefault("seq", "0");
|
||||
String seq = frame.sourceMeta().get("seq");
|
||||
if (seq == null || seq.isBlank()) {
|
||||
// 某些入口没有硬件流水号，使用原始字节哈希作为保底键，至少能挡住同连接内的重复帧。
|
||||
seq = rawFingerprint(frame);
|
||||
}
|
||||
String key = frame.protocolId() + ":" + vin + ":" + frame.command() + ":" + seq;
|
||||
if (seen.asMap().putIfAbsent(key, Boolean.TRUE) != null) {
|
||||
ctx.abort("duplicate:" + key);
|
||||
@@ -42,4 +51,12 @@ public class DedupInterceptor implements IngestInterceptor, Ordered {
|
||||
public int getOrder() {
|
||||
return 100;
|
||||
}
|
||||
|
||||
private static String rawFingerprint(RawFrame frame) {
|
||||
byte[] rawBytes = frame.rawBytes();
|
||||
if (rawBytes != null && rawBytes.length > 0) {
|
||||
return "raw:" + Integer.toHexString(Arrays.hashCode(rawBytes));
|
||||
}
|
||||
return "receivedAt:" + frame.receivedAt();
|
||||
}
|
||||
}
|
||||
@@ -13,6 +13,9 @@ import java.time.Duration;
|
||||
|
||||
/**
|
||||
* 单 VIN 速率限制。每个 VIN 一个独立的 Resilience4j RateLimiter，由 Caffeine 按 LRU 管理。
|
||||
*
|
||||
* <p>这里限的是进入业务 Handler 前的原始帧，不区分命令类型；32960 生产侧如需允许登入/登出穿透，
|
||||
* 应在此处扩展白名单或改为按命令配置，而不是在存储层丢弃。
|
||||
*/
|
||||
public class RateLimitInterceptor implements IngestInterceptor, Ordered {
|
||||
|
||||
@@ -34,6 +37,7 @@ public class RateLimitInterceptor implements IngestInterceptor, Ordered {
|
||||
@Override
|
||||
public boolean before(RawFrame frame, IngestContext ctx) {
|
||||
String vin = frame.sourceMeta().getOrDefault("vin", "unknown");
|
||||
// VIN 粒度隔离，避免某一辆车的高频/异常上报挤占其他车辆处理额度。
|
||||
RateLimiter rl = limiters.get(vin, k -> RateLimiter.of("vin-" + k, defaultConfig));
|
||||
if (!rl.acquirePermission()) {
|
||||
ctx.abort("rate-limited:" + vin);
|
||||
@@ -0,0 +1,218 @@
|
||||
package com.lingniu.ingest.core.dispatcher;
|
||||
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
import com.lingniu.ingest.api.annotation.AsyncBatch;
|
||||
import com.lingniu.ingest.api.annotation.MessageMapping;
|
||||
import com.lingniu.ingest.api.annotation.ProtocolHandler;
|
||||
import com.lingniu.ingest.api.event.LocationPayload;
|
||||
import com.lingniu.ingest.api.event.VehicleEvent;
|
||||
import com.lingniu.ingest.api.pipeline.IngestContext;
|
||||
import com.lingniu.ingest.api.pipeline.RawFrame;
|
||||
import com.lingniu.ingest.api.sink.EventSink;
|
||||
import com.lingniu.ingest.core.concurrency.AsyncBatchExecutor;
|
||||
import com.lingniu.ingest.core.concurrency.DisruptorEventBus;
|
||||
import com.lingniu.ingest.core.pipeline.InterceptorChain;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.lang.reflect.Method;
|
||||
import java.time.Instant;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.CompletableFuture;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
class DispatcherRawArchiveMetadataTest {
|
||||
|
||||
@Test
|
||||
void enrichesBusinessEventsWithRawArchiveLookupMetadata() throws Exception {
|
||||
CollectingSink sink = new CollectingSink();
|
||||
DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink));
|
||||
AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish);
|
||||
try {
|
||||
Dispatcher dispatcher = new Dispatcher(
|
||||
registryWithLocationHandler(),
|
||||
new InterceptorChain(List.of()),
|
||||
new HandlerInvoker(),
|
||||
eventBus,
|
||||
batchExecutor);
|
||||
|
||||
dispatcher.dispatch(new RawFrame(
|
||||
ProtocolId.JT808,
|
||||
0x0200,
|
||||
0,
|
||||
new Object(),
|
||||
new byte[]{0x7e, 0x01, 0x7e},
|
||||
Map.of("vin", "VIN001", "peer", "127.0.0.1:10001"),
|
||||
Instant.parse("2026-06-22T08:00:00Z")));
|
||||
|
||||
VehicleEvent.RawArchive raw = awaitEvent(sink, VehicleEvent.RawArchive.class);
|
||||
VehicleEvent.Location location = awaitEvent(sink, VehicleEvent.Location.class);
|
||||
|
||||
String rawArchiveKey = raw.metadata().get("rawArchiveKey");
|
||||
assertThat(rawArchiveKey).isNotBlank();
|
||||
assertThat(raw.eventId()).containsOnlyDigits();
|
||||
assertThat(rawArchiveKey).endsWith("/" + raw.eventId() + ".bin");
|
||||
assertThat(location.metadata())
|
||||
.containsEntry("rawArchiveEventId", raw.eventId())
|
||||
.containsEntry("rawArchiveKey", rawArchiveKey)
|
||||
.containsEntry("rawArchiveUri", "archive://" + rawArchiveKey);
|
||||
} finally {
|
||||
batchExecutor.close();
|
||||
eventBus.close();
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void enrichesAsyncBatchEventsWithTheirRawArchiveLookupMetadata() throws Exception {
|
||||
CollectingSink sink = new CollectingSink();
|
||||
DisruptorEventBus eventBus = new DisruptorEventBus(1024, "blocking", List.of(sink));
|
||||
AsyncBatchExecutor batchExecutor = new AsyncBatchExecutor(eventBus::publish);
|
||||
try {
|
||||
Dispatcher dispatcher = new Dispatcher(
|
||||
registryWithAsyncLocationHandler(),
|
||||
new InterceptorChain(List.of()),
|
||||
new HandlerInvoker(),
|
||||
eventBus,
|
||||
batchExecutor);
|
||||
|
||||
dispatcher.dispatch(new RawFrame(
|
||||
ProtocolId.JT808,
|
||||
0x0201,
|
||||
0,
|
||||
"payload-1",
|
||||
new byte[]{0x7e, 0x02, 0x7e},
|
||||
Map.of("vin", "VIN002"),
|
||||
Instant.parse("2026-06-22T08:01:00Z")));
|
||||
|
||||
VehicleEvent.RawArchive raw = awaitEvent(sink, VehicleEvent.RawArchive.class);
|
||||
VehicleEvent.Location location = awaitLocationEvent(sink, "async-location-1");
|
||||
|
||||
String rawArchiveKey = raw.metadata().get("rawArchiveKey");
|
||||
assertThat(location.metadata())
|
||||
.containsEntry("rawArchiveEventId", raw.eventId())
|
||||
.containsEntry("rawArchiveKey", rawArchiveKey)
|
||||
.containsEntry("rawArchiveUri", "archive://" + rawArchiveKey);
|
||||
} finally {
|
||||
batchExecutor.close();
|
||||
eventBus.close();
|
||||
}
|
||||
}
|
||||
|
||||
private static HandlerRegistry registryWithLocationHandler() throws NoSuchMethodException {
|
||||
LocationHandler bean = new LocationHandler();
|
||||
Method method = LocationHandler.class.getDeclaredMethod("onLocation", Object.class);
|
||||
HandlerRegistry registry = new HandlerRegistry();
|
||||
registry.register(new HandlerDefinition(
|
||||
ProtocolId.JT808,
|
||||
0x0200,
|
||||
0,
|
||||
"test-location",
|
||||
bean,
|
||||
method,
|
||||
Object.class,
|
||||
null,
|
||||
null,
|
||||
null));
|
||||
return registry;
|
||||
}
|
||||
|
||||
private static HandlerRegistry registryWithAsyncLocationHandler() throws NoSuchMethodException {
|
||||
AsyncLocationHandler bean = new AsyncLocationHandler();
|
||||
Method method = AsyncLocationHandler.class.getDeclaredMethod("onLocationBatch", List.class);
|
||||
HandlerRegistry registry = new HandlerRegistry();
|
||||
registry.register(new HandlerDefinition(
|
||||
ProtocolId.JT808,
|
||||
0x0201,
|
||||
0,
|
||||
"test-async-location",
|
||||
bean,
|
||||
method,
|
||||
Object.class,
|
||||
null,
|
||||
null,
|
||||
method.getAnnotation(AsyncBatch.class)));
|
||||
return registry;
|
||||
}
|
||||
|
||||
private static <T extends VehicleEvent> T awaitEvent(CollectingSink sink, Class<T> type) throws InterruptedException {
|
||||
long deadline = System.currentTimeMillis() + 3000;
|
||||
while (System.currentTimeMillis() < deadline) {
|
||||
synchronized (sink.events) {
|
||||
for (VehicleEvent event : sink.events) {
|
||||
if (type.isInstance(event)) {
|
||||
return type.cast(event);
|
||||
}
|
||||
}
|
||||
}
|
||||
Thread.sleep(25);
|
||||
}
|
||||
throw new AssertionError("event not published: " + type.getSimpleName() + ", actual=" + sink.events);
|
||||
}
|
||||
|
||||
private static VehicleEvent.Location awaitLocationEvent(CollectingSink sink, String eventId) throws InterruptedException {
|
||||
long deadline = System.currentTimeMillis() + 3000;
|
||||
while (System.currentTimeMillis() < deadline) {
|
||||
synchronized (sink.events) {
|
||||
for (VehicleEvent event : sink.events) {
|
||||
if (event instanceof VehicleEvent.Location location && location.eventId().equals(eventId)) {
|
||||
return location;
|
||||
}
|
||||
}
|
||||
}
|
||||
Thread.sleep(25);
|
||||
}
|
||||
throw new AssertionError("location event not published: " + eventId + ", actual=" + sink.events);
|
||||
}
|
||||
|
||||
@ProtocolHandler(protocol = ProtocolId.JT808)
|
||||
static final class LocationHandler {
|
||||
@MessageMapping(command = 0x0200)
|
||||
VehicleEvent.Location onLocation(Object ignored) {
|
||||
return new VehicleEvent.Location(
|
||||
"location-1",
|
||||
"VIN001",
|
||||
ProtocolId.JT808,
|
||||
Instant.parse("2026-06-22T08:00:00Z"),
|
||||
Instant.parse("2026-06-22T08:00:01Z"),
|
||||
"trace-location",
|
||||
Map.of(),
|
||||
new LocationPayload(113.12, 23.45, 10.0, 60.0, 90.0, 0, 1));
|
||||
}
|
||||
}
|
||||
|
||||
@ProtocolHandler(protocol = ProtocolId.JT808)
|
||||
public static final class AsyncLocationHandler {
|
||||
@MessageMapping(command = 0x0201)
|
||||
@AsyncBatch(size = 2, waitMs = 10, poolSize = 1)
|
||||
public List<VehicleEvent> onLocationBatch(List<Object> batch) {
|
||||
return List.of(new VehicleEvent.Location(
|
||||
"async-location-1",
|
||||
"VIN002",
|
||||
ProtocolId.JT808,
|
||||
Instant.parse("2026-06-22T08:01:00Z"),
|
||||
Instant.parse("2026-06-22T08:01:01Z"),
|
||||
"trace-async-location",
|
||||
Map.of(),
|
||||
new LocationPayload(113.12, 23.45, 10.0, 60.0, 90.0, 0, 1)));
|
||||
}
|
||||
}
|
||||
|
||||
private static final class CollectingSink implements EventSink {
|
||||
private final List<VehicleEvent> events = new ArrayList<>();
|
||||
|
||||
@Override
|
||||
public String name() {
|
||||
return "collecting";
|
||||
}
|
||||
|
||||
@Override
|
||||
public CompletableFuture<Void> publish(VehicleEvent event) {
|
||||
synchronized (events) {
|
||||
events.add(event);
|
||||
}
|
||||
return CompletableFuture.completedFuture(null);
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
package com.lingniu.ingest.core.dispatcher;
|
||||
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.lang.reflect.Method;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
class HandlerRegistryTest {
|
||||
|
||||
@Test
|
||||
void exactCommandMatchTakesPrecedenceOverWildcardHandlers() throws Exception {
|
||||
HandlerRegistry registry = new HandlerRegistry();
|
||||
Object bean = new SampleHandlers();
|
||||
Method exact = SampleHandlers.class.getMethod("exact", Object.class);
|
||||
Method wildcard = SampleHandlers.class.getMethod("wildcard", Object.class);
|
||||
HandlerDefinition exactDef = definition(0x0200, "exact", bean, exact);
|
||||
HandlerDefinition wildcardDef = definition(0, "wildcard", bean, wildcard);
|
||||
|
||||
registry.register(exactDef);
|
||||
registry.register(wildcardDef);
|
||||
|
||||
assertThat(registry.resolve(ProtocolId.XINDA_PUSH, 0x0200, 0))
|
||||
.containsExactly(exactDef);
|
||||
assertThat(registry.resolve(ProtocolId.XINDA_PUSH, 0x0500, 0))
|
||||
.containsExactly(wildcardDef);
|
||||
}
|
||||
|
||||
private static HandlerDefinition definition(int command, String desc, Object bean, Method method) {
|
||||
return new HandlerDefinition(
|
||||
ProtocolId.XINDA_PUSH,
|
||||
command,
|
||||
0,
|
||||
desc,
|
||||
bean,
|
||||
method,
|
||||
Object.class,
|
||||
null,
|
||||
null,
|
||||
null);
|
||||
}
|
||||
|
||||
public static final class SampleHandlers {
|
||||
public void exact(Object ignored) {
|
||||
}
|
||||
|
||||
public void wildcard(Object ignored) {
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,40 @@
|
||||
package com.lingniu.ingest.core.pipeline.builtin;
|
||||
|
||||
import com.lingniu.ingest.api.ProtocolId;
|
||||
import com.lingniu.ingest.api.pipeline.IngestContext;
|
||||
import com.lingniu.ingest.api.pipeline.RawFrame;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.Map;
|
||||
|
||||
import static org.assertj.core.api.Assertions.assertThat;
|
||||
|
||||
class DedupInterceptorTest {
|
||||
|
||||
@Test
|
||||
void usesRawFingerprintWhenSequenceIsMissing() {
|
||||
DedupInterceptor interceptor = new DedupInterceptor(100, 60);
|
||||
IngestContext ctx = new IngestContext("trace-1");
|
||||
|
||||
RawFrame first = frame(new byte[]{0x23, 0x23, 0x02, 0x01});
|
||||
RawFrame differentPayload = frame(new byte[]{0x23, 0x23, 0x02, 0x02});
|
||||
RawFrame duplicate = frame(new byte[]{0x23, 0x23, 0x02, 0x01});
|
||||
|
||||
assertThat(interceptor.before(first, ctx)).isTrue();
|
||||
assertThat(interceptor.before(differentPayload, ctx)).isTrue();
|
||||
assertThat(interceptor.before(duplicate, ctx)).isFalse();
|
||||
assertThat(ctx.aborted()).isTrue();
|
||||
}
|
||||
|
||||
private static RawFrame frame(byte[] rawBytes) {
|
||||
return new RawFrame(
|
||||
ProtocolId.GB32960,
|
||||
0x02,
|
||||
0,
|
||||
new Object(),
|
||||
rawBytes,
|
||||
Map.of("vin", "TESTSTATVIN000001"),
|
||||
Instant.parse("2026-06-22T13:00:00Z"));
|
||||
}
|
||||
}
|
||||
@@ -5,6 +5,7 @@
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>lingniu-vehicle-ingest</artifactId>
|
||||
<version>0.1.0-SNAPSHOT</version>
|
||||
<relativePath>../../../pom.xml</relativePath>
|
||||
</parent>
|
||||
<artifactId>observability</artifactId>
|
||||
<name>observability</name>
|
||||
@@ -5,6 +5,7 @@
|
||||
<groupId>com.lingniu.ingest</groupId>
|
||||
<artifactId>lingniu-vehicle-ingest</artifactId>
|
||||
<version>0.1.0-SNAPSHOT</version>
|
||||
<relativePath>../../../pom.xml</relativePath>
|
||||
</parent>
|
||||
<artifactId>session-core</artifactId>
|
||||
<name>session-core</name>
|
||||
@@ -23,6 +24,10 @@
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-autoconfigure</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-data-redis</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.github.ben-manes.caffeine</groupId>
|
||||
<artifactId>caffeine</artifactId>
|
||||
@@ -37,5 +42,15 @@
|
||||
<artifactId>assertj-core</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.mockito</groupId>
|
||||
<artifactId>mockito-core</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-test</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
</project>
|
||||
@@ -35,6 +35,7 @@ public final class InMemorySessionStore implements SessionStore {
|
||||
@Override
|
||||
public void put(DeviceSession session) {
|
||||
byId.put(session.sessionId(), session);
|
||||
// 三索引用同一个 sessionId 汇聚；HTTP 下行可以拿 sessionId/VIN/phone 任意一种查在线连接。
|
||||
if (session.vin() != null) vinToId.put(session.vin(), session.sessionId());
|
||||
if (session.phone() != null) phoneToId.put(session.phone(), session.sessionId());
|
||||
}
|
||||
@@ -61,6 +62,7 @@ public final class InMemorySessionStore implements SessionStore {
|
||||
DeviceSession old = byId.getIfPresent(sessionId);
|
||||
if (old == null) return Optional.empty();
|
||||
DeviceSession next = updater.apply(old);
|
||||
// update 复用 put，保证 VIN/phone 二级索引和主会话对象一起刷新。
|
||||
put(next);
|
||||
return Optional.of(next);
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user