shishengliang/lingniu-vehicle-ingest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(go): batch nats fast writer tdengine writes

Browse Source
This commit is contained in:
lingniu
2026-07-03 19:17:02 +08:00
parent 2db05d100d
commit 3b694671fa
4 changed files with 127 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/ops/go-vehicle-ingest-memory.md
View File

@@ -421,6 +421,8 @@ vehicle_fast_writer_stage_duration_ms_histogram_sum{subject,stage,status}
`stage` 取值包括 `tdengine`、`redis`、`ack`，分别对应快速写 TDengine raw/location、Redis realtime 投影、NATS ack。带帧率压测时如果某个 stage 的 p99 持续上升，可以直接定位 fast path 是存储瓶颈还是 ack/网络瓶颈。
2026-07-03 后续优化：`nats-fast-writer` 已从“Fetch batch 后逐条写 TDengine”改为“Fetch batch 后先调用 `history.Writer.AppendAllBatch` 批量写 TDengine，再逐条更新 Redis 和 ack”。这样可以减少 TDengine round trip；Redis/ack 仍按单条执行，避免某条实时投影失败时误 ack。
### NATS Kafka bridge batch 指标
2026-07-03 已为 NATS -> Kafka bridge 增加 batch pending 和 duration histogram：