shishengliang/lingniu-vehicle-ingest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(operations): add automated reconciliation center

Browse Source
This commit is contained in:
lingniu
2026-07-16 19:14:53 +08:00
parent bbab018d55
commit d67f42b4f4
25 changed files with 1697 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

25
vehicle-data-platform/docs/deployment.md
View File

@@ -14,6 +14,7 @@ GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -trimpath -ldflags='-s -w' -o ../
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -trimpath -ldflags='-s -w' -o ../../dist/alert-stream-evaluator ./cmd/alert-stream-evaluator
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -trimpath -ldflags='-s -w' -o ../../dist/platform-migrate ./cmd/platform-migrate
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -trimpath -ldflags='-s -w' -o ../../dist/oneos-scope-sync ./cmd/oneos-scope-sync
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -trimpath -ldflags='-s -w' -o ../../dist/reconciliation-evaluator ./cmd/reconciliation-evaluator
# Optional release/performance gate; this binary is run on demand, not installed as a service.
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -trimpath -ldflags='-s -w' -o ../../dist/alert-benchmark ./cmd/alert-benchmark
```
@@ -66,14 +67,15 @@ PREVIOUS_WEB=/opt/lingniu-vehicle-platform/releases/$PREVIOUS_RELEASE/web
"$PREVIOUS_WEB/.release-assets"
```
`deploy/install-web-release.sh` also accepts an optional third argument containing a newly built `platform-api` and an optional fourth argument containing a newly built `oneos-scope-sync`. When provided, it atomically publishes those binaries with the Web. The installer inherits every runtime binary that exists in the previous release (`alert-evaluator`, `alert-stream-evaluator`, `platform-migrate`, `oneos-scope-sync` and the optional benchmark) before switching the symlink, so a later systemd restart cannot fail because a Web-oriented release omitted an unchanged binary:
`deploy/install-web-release.sh` also accepts an optional third argument containing a newly built `platform-api`, an optional fourth argument containing a newly built `oneos-scope-sync`, and an optional fifth argument containing a newly built `reconciliation-evaluator`. When provided, it atomically publishes those binaries with the Web. The installer inherits every runtime binary that exists in the previous release (`alert-evaluator`, `alert-stream-evaluator`, `platform-migrate`, `oneos-scope-sync`, `reconciliation-evaluator` and the optional benchmark) before switching the symlink, so a later systemd restart cannot fail because a Web-oriented release omitted an unchanged binary:
```bash
deploy/install-web-release.sh \
"$PLATFORM_RELEASE" \
"/tmp/$PLATFORM_RELEASE-web.tar.gz" \
"/tmp/$PLATFORM_RELEASE-platform-api" \
"/tmp/$PLATFORM_RELEASE-oneos-scope-sync"
"/tmp/$PLATFORM_RELEASE-oneos-scope-sync" \
"/tmp/$PLATFORM_RELEASE-reconciliation-evaluator"
```
## Environment
@@ -108,6 +110,7 @@ ONEOS_SCOPE_API_MAX_ATTEMPTS=3
ONEOS_SCOPE_SYNC_TIMEOUT_SEC=60
ONEOS_SCOPE_MAX_REJECTED=100
ONEOS_SCOPE_MAX_REJECT_RATIO=0.10
RECONCILIATION_TIMEOUT_SEC=300
REQUEST_TIMEOUT_MS=5000
AMAP_WEB_JS_KEY=***
AMAP_SECURITY_JS_CODE=***
@@ -198,10 +201,13 @@ test -n "$MYSQL_DSN"
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/013_platform_identity_access.sql \
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/014_customer_vehicle_grant_time.sql \
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/015_vehicle_source_policy_audit.sql \
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/016_business_scope_dimensions.sql
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/016_business_scope_dimensions.sql \
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/017_reconciliation_center.sql
```
The API guards the access-threshold tables for compatibility, while alert APIs deliberately require the alert migrations to exist. Run every numbered migration explicitly before switching traffic so DDL permission and index creation failures are caught early. The migration journal records filename and SHA-256 and refuses a changed file; duplicate forward `ADD COLUMN` and `CREATE INDEX` statements are tolerated only when resuming partially executed MySQL DDL. Full-line SQL comments are removed before statement splitting, so punctuation in a comment cannot become executable SQL. Migration `008` adds forward-compatible access evidence columns and an index to the gateway-owned realtime snapshot table without changing its `(protocol, vin)` primary key. Migration `009` creates the per-group/topic/partition event-time checkpoint used to make MySQL effects authoritative before Kafka offsets are committed. Migration `012` creates the atomic, versioned business Scope projection; its reserved `row_number` column is quoted for production MySQL compatibility. Migration `014` backfills active vehicle-grant start times, creates the grant-interval history table and adds the active time lookup index; apply it before starting an API binary that writes grant history. Migration `015` adds platform-owned optimistic versions and immutable audits for per-vehicle location-source policy changes. It does not alter the gateway election SQL or expose the gateway `source_key`; the API resolves an opaque `sourceRef` server-side and the gateway applies the saved policy on the next valid vehicle report. Migration `016` adds customer name, department and responsible-person dimensions plus bounded lookup indexes to the platform-owned Scope projection.
The API guards the access-threshold tables for compatibility, while alert and reconciliation APIs deliberately require their migrations to exist. Run every numbered migration explicitly before switching traffic so DDL permission and index creation failures are caught early. The migration journal records filename and SHA-256 and refuses a changed file; duplicate forward `ADD COLUMN` and `CREATE INDEX` statements are tolerated only when resuming partially executed MySQL DDL. Full-line SQL comments are removed before statement splitting, so punctuation in a comment cannot become executable SQL. Migration `008` adds forward-compatible access evidence columns and an index to the gateway-owned realtime snapshot table without changing its `(protocol, vin)` primary key. Migration `009` creates the per-group/topic/partition event-time checkpoint used to make MySQL effects authoritative before Kafka offsets are committed. Migration `012` creates the atomic, versioned business Scope projection; its reserved `row_number` column is quoted for production MySQL compatibility. Migration `014` backfills active vehicle-grant start times, creates the grant-interval history table and adds the active time lookup index; apply it before starting an API binary that writes grant history. Migration `015` adds platform-owned optimistic versions and immutable audits for per-vehicle location-source policy changes. It does not alter the gateway election SQL or expose the gateway `source_key`; the API resolves an opaque `sourceRef` server-side and the gateway applies the saved policy on the next valid vehicle report. Migration `016` adds customer name, department and responsible-person dimensions plus bounded lookup indexes to the platform-owned Scope projection. Migration `017` creates the reconciliation run, issue and immutable action tables; the unique fingerprint index is the database-level duplicate-work-item guard.
Production release `reconciliation-summary-fix-20260716191153` applied migration `017`, deployed the API, Web and reconciliation evaluator, and enabled the daily timer. The initial run detected 1,622 active differences in about 1.43 seconds; an immediate repeat completed in about 0.88 seconds with `new=0`. The authenticated smoke verified summary/list/detail for administrator and operator, viewer denial, optimistic-version conflict handling, evidence-key redaction, 50-row pagination, responsive desktop/mobile rendering and no browser console errors.
Release `oneos-api-client-20260716184601` applied migrations `012` and `016` and deployed the switchable API/database sync binary. The projection is intentionally empty (`active_version` unset) until a verified OneOS endpoint is configured. A production fail-closed smoke with `ONEOS_SCOPE_SOURCE=api` and no endpoint exited nonzero without publishing any row. Do not install or enable the timer until the endpoint, service token, signing secret, source-IP restriction and a representative snapshot have passed the contract checks below.
@@ -214,6 +220,8 @@ sudo cp deploy/systemd/lingniu-vehicle-alert-evaluator.service /etc/systemd/syst
sudo cp deploy/systemd/lingniu-vehicle-alert-stream-evaluator.service /etc/systemd/system/
sudo cp deploy/systemd/lingniu-vehicle-oneos-scope-sync.service /etc/systemd/system/
sudo cp deploy/systemd/lingniu-vehicle-oneos-scope-sync.timer /etc/systemd/system/
sudo cp deploy/systemd/lingniu-vehicle-reconciliation-evaluator.service /etc/systemd/system/
sudo cp deploy/systemd/lingniu-vehicle-reconciliation-evaluator.timer /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now lingniu-vehicle-platform lingniu-vehicle-alert-evaluator lingniu-vehicle-alert-stream-evaluator
sudo systemctl status --no-pager lingniu-vehicle-alert-evaluator
@@ -224,6 +232,11 @@ sudo journalctl -u lingniu-vehicle-alert-stream-evaluator -n 100 --no-pager
sudo systemctl enable --now lingniu-vehicle-oneos-scope-sync.timer
sudo systemctl status --no-pager lingniu-vehicle-oneos-scope-sync.timer
sudo journalctl -u lingniu-vehicle-oneos-scope-sync.service -n 100 --no-pager
# Run once before enabling the timer; a failure must leave the timer disabled.
sudo systemctl start lingniu-vehicle-reconciliation-evaluator.service
sudo journalctl -u lingniu-vehicle-reconciliation-evaluator.service -n 100 --no-pager
sudo systemctl enable --now lingniu-vehicle-reconciliation-evaluator.timer
sudo systemctl list-timers lingniu-vehicle-reconciliation-evaluator.timer --no-pager
```
## Health
@@ -256,6 +269,10 @@ curl -fsS -X POST -H "$AUTH_HEADER" -H 'Content-Type: application/json' -d '{"li
http://127.0.0.1:20300/api/v2/alerts/events
curl -fsS -H "$AUTH_HEADER" http://127.0.0.1:20300/api/v2/alerts/rules
curl -fsS -H "$AUTH_HEADER" 'http://127.0.0.1:20300/api/v2/alerts/notifications?unreadOnly=true&limit=5'
curl -fsS -H "$AUTH_HEADER" 'http://127.0.0.1:20300/api/v2/reconciliation/summary?days=30'
curl -fsS -X POST -H "$AUTH_HEADER" -H 'Content-Type: application/json' \
-d '{"status":"active","limit":50,"offset":0}' \
http://127.0.0.1:20300/api/v2/reconciliation/issues
curl -fsS http://127.0.0.1:20300/app-config.js
curl -fsS http://127.0.0.1:20300/ | grep -q '<div id="root"></div>'
MAIN_ASSET="$(sed -n 's/.*src="\([^"]*\.js\)".*/\1/p' /opt/lingniu-vehicle-platform/current/web/index.html | head -1)"

14
vehicle-data-platform/docs/vehicle-data-platform-meeting-todo.md
View File

@@ -435,7 +435,19 @@
### P1-05 数据差异自动自查与对账
状态：`进行中`
状态：`已完成`
已上线结果（release `reconciliation-summary-fix-20260716191153`）：
- 新增独立的数据差异中心，自动检测重复车牌/终端、来源未绑定、主车辆无来源、多来源位置漂移、里程跳变/倒退、日里程来源差异、车队总数差异、OneOS 业务车辆未绑定及人工授权与业务范围不一致。OneOS 投影尚未启用时，业务范围规则不伪造结果，待正式快照发布后自动参与每日检测。
- 每条差异保存稳定指纹、规则、等级、车辆、来源、首次/最近发现时间、命中次数和结构化证据；原始 `source_key`、终端手机号、来源 IP 不进入接口响应。
- 支持待处理、确认来源 A、确认来源 B、无需处理、已修复和已恢复状态；非待处理结论必须填写说明，使用乐观版本避免并发覆盖，全部处理动作保留账号和时间履历。
- 相同指纹重复检测只增加命中次数，不产生重复工单；已修复/已恢复问题再次出现会重新打开，活跃问题不再命中时自动转为已恢复。
- 独立 `reconciliation-evaluator` 每日定时执行，不加入 API 高频轮询。生产定时器已启用，下一次运行时间为 `2026-07-17 02:19:54 +08:00`，单次全量检测约 0.88–1.43 秒。
- 首轮生产检测发现 1,622 条活跃差异：车队总数差异 1、里程跳变 112、里程倒退 6、多来源日里程差异 1,197、主车辆无来源 295、来源未绑定 11；第二轮同数据运行 `new=0`，证明指纹去重生效。
- 运维页面以 50 行差异队列为主体，提供关键词、状态、等级和规则筛选、30 天新增/存量/恢复趋势及按需证据抽屉。摘要缓存 60 秒、列表缓存 20 秒、详情按需读取，不加入 15 秒健康轮询。
- 管理员和运维员可读取、复核；只读查看者返回 403。生产验证中错误版本处置返回 409，防止覆盖其他人的结论。
- Go 全量测试、前端 49 个文件/259 项测试、TypeScript/Vite 构建和发布脚本测试通过。生产桌面首屏显示 1,622 条、50 行队列；390×844 移动端详情宽度不超过视窗，表格受控横向滚动；两种视图均无页面级横向溢出、运行时异常或控制台错误。
目标：