docs: document go native ecs deployment
This commit is contained in:
lingniu
183
docs/operations/current-ecs-deployment.md
Normal file
183
docs/operations/current-ecs-deployment.md
Normal file
@@ -0,0 +1,183 @@
|
|||||||
|
# 当前 ECS Go 原生部署说明
|
||||||
|
|
||||||
|
更新时间：2026-07-02 01:30 CST
|
||||||
|
|
||||||
|
本文档记录当前 `lingniu-vehicle-ingest` 的生产运行面。当前 goal 的接入链路已经切到 Go 原生 systemd 部署，不再以 Docker/Portainer 作为生产运行方式。密钥只维护在 ECS 环境文件或受控凭据中，不写入 Git。
|
||||||
|
|
||||||
|
## 部署范围
|
||||||
|
|
||||||
|
| systemd 服务 | 二进制 | 说明 | 对外端口 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `lingniu-go-gateway.service` | `gateway` | GB32960 TCP、JT808 TCP、宇通 MQTT 接入，解析后写 Kafka RAW 和统一事件 | TCP `32960`、TCP `808` |
|
||||||
|
| `lingniu-go-history-writer.service` | `history-writer` | 消费 Kafka RAW，写 TDengine RAW、位置点、里程点核心表 | 无 HTTP |
|
||||||
|
| `lingniu-go-stat-writer.service` | `stat-writer` | 消费 Kafka RAW，按总里程差值法写 MySQL 每日指标 | 无 HTTP |
|
||||||
|
| `lingniu-go-realtime-api.service` | `realtime-api` | 消费统一事件写 Redis，并提供实时、历史、统计查询 API | HTTP `20210` |
|
||||||
|
|
||||||
|
信达 Push 已废弃，不参与当前 Go 生产链路。旧 Java/Docker 服务不应占用 `808`、`32960`、`20210`。
|
||||||
|
|
||||||
|
## 应用 ECS
|
||||||
|
|
||||||
|
| 项 | 值 |
|
||||||
|
| --- | --- |
|
||||||
|
| 公网 IP | `115.29.187.205` |
|
||||||
|
| 登录用户 | `root` |
|
||||||
|
| 部署目录 | `/opt/lingniu-go-native` |
|
||||||
|
| 当前 release | `/opt/lingniu-go-native/current` |
|
||||||
|
| 环境文件目录 | `/opt/lingniu-go-native/env` |
|
||||||
|
| systemd unit 目录 | `/etc/systemd/system` |
|
||||||
|
| 对外服务 | `32960`、`808`、`20210` |
|
||||||
|
|
||||||
|
运行目录结构：
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/lingniu-go-native/
|
||||||
|
current -> /opt/lingniu-go-native/releases/<git-short-sha>
|
||||||
|
releases/<git-short-sha>/
|
||||||
|
gateway
|
||||||
|
history-writer
|
||||||
|
stat-writer
|
||||||
|
realtime-api
|
||||||
|
env/
|
||||||
|
gateway.env
|
||||||
|
history-writer.env
|
||||||
|
stat-writer.env
|
||||||
|
realtime-api.env
|
||||||
|
spool/gateway/
|
||||||
|
```
|
||||||
|
|
||||||
|
## 中间件
|
||||||
|
|
||||||
|
| 组件 | 内网地址 | 公网地址 | 用途 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| Kafka | `172.17.111.56:9092` | `114.55.58.251:9092` | RAW 和统一事件消息总线 |
|
||||||
|
| TDengine | `172.17.111.57:6041` | `115.29.185.82:6041` | RAW、位置、里程点时序热存储 |
|
||||||
|
| MySQL RDS | `rm-bp179zbv481rnw3e2.mysql.rds.aliyuncs.com:3306` | `rm-bp179zbv481rnw3e2no.mysql.rds.aliyuncs.com:3306` | 身份映射、JT808 注册、每日指标 |
|
||||||
|
| Redis RDS | `r-bp1u741kij7e51i481.redis.rds.aliyuncs.com:6379` | 无 | 准实时车辆状态缓存 |
|
||||||
|
|
||||||
|
生产应用之间访问中间件优先使用内网地址。RDS 白名单需要允许应用 ECS 私网访问。
|
||||||
|
|
||||||
|
## Kafka Topic
|
||||||
|
|
||||||
|
| Topic | 生产者 | 消费者 | 内容 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `vehicle.raw.gb32960.v1` | `gateway` | `history-writer`、`stat-writer` | GB32960 完整 RAW 记录 |
|
||||||
|
| `vehicle.raw.jt808.v1` | `gateway` | `history-writer`、`stat-writer` | JT808 完整 RAW 记录 |
|
||||||
|
| `vehicle.raw.yutong-mqtt.v1` | `gateway` | `history-writer`、`stat-writer` | 宇通 MQTT 完整 RAW 记录 |
|
||||||
|
| `vehicle.event.unified.v1` | `gateway` | `realtime-api` | 三类协议统一事件，用于 Redis 实时状态 |
|
||||||
|
|
||||||
|
`gateway` 配置了 Kafka retry；生产 env 中启用 `KAFKA_SPOOL_DIR` 时，本地 spool 可在 Kafka 短暂不可用后回放。
|
||||||
|
|
||||||
|
## TDengine 数据层
|
||||||
|
|
||||||
|
默认数据库：`lingniu_vehicle_ts`
|
||||||
|
|
||||||
|
| 表 | 类型 | 说明 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `raw_frames` | stable | 完整 RAW 帧，包含 `raw_hex`、`parsed_json`、`fields_json`、`source_endpoint` 和协议/车辆 tags |
|
||||||
|
| `vehicle_locations` | stable | 最小化位置历史，保留 `frame_id` 回链 RAW |
|
||||||
|
| `vehicle_mileage_points` | stable | 最小化总里程点，供历史查询和统计复核 |
|
||||||
|
|
||||||
|
设计原则：
|
||||||
|
|
||||||
|
- 完整结构化 payload 只落在 `raw_frames.parsed_json` / `fields_json`。
|
||||||
|
- `vehicle_locations` 和 `vehicle_mileage_points` 只保留核心查询字段，不重复保存完整 JSON。
|
||||||
|
- `telemetry_fields` 不由当前服务维护，字段配置解析由独立子服务处理。
|
||||||
|
- API 传入东八区时间时，服务会转换为 TDengine 当前存储使用的 UTC 字面量再查询。
|
||||||
|
|
||||||
|
## MySQL 数据层
|
||||||
|
|
||||||
|
默认业务库：`lingniu_vehicle_data`
|
||||||
|
|
||||||
|
| 表 | 维护方 | 说明 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `vehicle_identity_binding` | 人工/外部主数据 | VIN 映射主表，用 `phone`、`device_id`、`plate` 降级定位 VIN |
|
||||||
|
| `jt808_registration` | `gateway` 自动写入 | 仅 JT808 注册和鉴权记录，以 `phone` 作为主键，记录设备、车牌、厂家、鉴权、来源端点 |
|
||||||
|
| `vehicle_daily_metric` | `stat-writer` 自动写入 | 每日指标表，保存 `daily_mileage_km` 和 `daily_total_mileage_km` |
|
||||||
|
|
||||||
|
每日里程算法：
|
||||||
|
|
||||||
|
```text
|
||||||
|
daily_mileage_km = 当日最大 total_mileage_km - 当日最小 total_mileage_km
|
||||||
|
daily_total_mileage_km = 当日最大 total_mileage_km
|
||||||
|
```
|
||||||
|
|
||||||
|
统计按 `Asia/Shanghai` 自然日归档，当前支持 GB32960、JT808、宇通 MQTT 中能解析出 `total_mileage_km` 的数据。JT808 的总里程来自位置附加信息 `0x01`，单位按协议转换为 km；宇通 MQTT 的 `TOTAL_MILEAGE` 原始单位为米，入库前转换为 km。
|
||||||
|
|
||||||
|
## Redis 实时层
|
||||||
|
|
||||||
|
`realtime-api` 消费 `vehicle.event.unified.v1`，按 VIN/vehicle key 维护准实时快照。主要能力：
|
||||||
|
|
||||||
|
| API | 说明 |
|
||||||
|
| --- | --- |
|
||||||
|
| `/api/realtime/vehicles/{vin}` | 查询车辆合并实时快照 |
|
||||||
|
| `/api/realtime/vehicles/{vin}/online` | 查询车辆是否在线 |
|
||||||
|
| `/api/realtime/vehicles/{vin}/protocols/{protocol}` | 查询指定协议的实时快照 |
|
||||||
|
|
||||||
|
Redis 只作为实时缓存；历史和统计以 TDengine/MySQL 为准。
|
||||||
|
|
||||||
|
## API 入口
|
||||||
|
|
||||||
|
公网基础地址：
|
||||||
|
|
||||||
|
```text
|
||||||
|
http://115.29.187.205:20210
|
||||||
|
```
|
||||||
|
|
||||||
|
常用查询：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -sS 'http://115.29.187.205:20210/api/history/raw-frames?protocol=GB32960&dateFrom=2026-07-02T00:00:00%2B08:00&dateTo=2026-07-03T00:00:00%2B08:00&limit=1'
|
||||||
|
curl -sS 'http://115.29.187.205:20210/api/history/raw-frames?protocol=JT808&dateFrom=2026-07-02T00:00:00%2B08:00&dateTo=2026-07-03T00:00:00%2B08:00&limit=1'
|
||||||
|
curl -sS 'http://115.29.187.205:20210/api/history/raw-frames?protocol=YUTONG_MQTT&dateFrom=2026-07-02T00:00:00%2B08:00&dateTo=2026-07-03T00:00:00%2B08:00&limit=1'
|
||||||
|
curl -sS 'http://115.29.187.205:20210/api/history/locations?protocol=JT808&dateFrom=2026-07-02T00:00:00%2B08:00&dateTo=2026-07-03T00:00:00%2B08:00&limit=1'
|
||||||
|
curl -sS 'http://115.29.187.205:20210/api/history/mileage-points?protocol=JT808&dateFrom=2026-07-02T00:00:00%2B08:00&dateTo=2026-07-03T00:00:00%2B08:00&limit=1'
|
||||||
|
curl -sS 'http://115.29.187.205:20210/api/stats/daily-metrics?dateFrom=2026-07-02&dateTo=2026-07-02&limit=20'
|
||||||
|
```
|
||||||
|
|
||||||
|
分页参数统一使用 `limit` 和 `offset`。
|
||||||
|
|
||||||
|
## 部署命令
|
||||||
|
|
||||||
|
在开发机或 CI 构建 Linux amd64：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd go/vehicle-gateway
|
||||||
|
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' -o /tmp/lingniu-go-native/gateway ./cmd/gateway
|
||||||
|
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' -o /tmp/lingniu-go-native/history-writer ./cmd/history-writer
|
||||||
|
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' -o /tmp/lingniu-go-native/stat-writer ./cmd/stat-writer
|
||||||
|
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags='-s -w' -o /tmp/lingniu-go-native/realtime-api ./cmd/realtime-api
|
||||||
|
```
|
||||||
|
|
||||||
|
上传到 ECS 新 release 目录后切换：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl daemon-reload
|
||||||
|
systemctl enable lingniu-go-gateway lingniu-go-history-writer lingniu-go-stat-writer lingniu-go-realtime-api
|
||||||
|
systemctl restart lingniu-go-gateway lingniu-go-history-writer lingniu-go-stat-writer lingniu-go-realtime-api
|
||||||
|
```
|
||||||
|
|
||||||
|
只更新查询 API 时，可以仅替换 `/opt/lingniu-go-native/current/realtime-api` 并重启：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl restart lingniu-go-realtime-api.service
|
||||||
|
```
|
||||||
|
|
||||||
|
## 运行检查
|
||||||
|
|
||||||
|
```bash
|
||||||
|
systemctl is-active lingniu-go-gateway lingniu-go-history-writer lingniu-go-stat-writer lingniu-go-realtime-api
|
||||||
|
ss -lntp | egrep ':(808|32960|20210)\b'
|
||||||
|
journalctl -u lingniu-go-gateway --since '5 minutes ago' --no-pager
|
||||||
|
journalctl -u lingniu-go-history-writer --since '5 minutes ago' --no-pager
|
||||||
|
journalctl -u lingniu-go-stat-writer --since '5 minutes ago' --no-pager
|
||||||
|
journalctl -u lingniu-go-realtime-api --since '5 minutes ago' --no-pager
|
||||||
|
```
|
||||||
|
|
||||||
|
2026-07-02 01:30 CST 已验证：
|
||||||
|
|
||||||
|
- 四个 Go systemd 服务均为 `active`。
|
||||||
|
- `gateway` 正在监听 `32960` 和 `808`。
|
||||||
|
- `realtime-api` 正在监听 `20210`。
|
||||||
|
- GB32960、JT808、YUTONG_MQTT 的东八区 RAW 查询均可命中生产数据。
|
||||||
|
- `vehicle_daily_metric` 可查到 `daily_mileage_km` 和 `daily_total_mileage_km`。
|
||||||
|
|
||||||
Reference in New Issue
Block a user