feat(operations): add auditable source provider maintenance
This commit is contained in:
lingniu
@@ -323,6 +323,28 @@ Access rows expose both event and receive time, `dataDelaySec`, `freshnessSec`,
|
||||
|
||||
Threshold configuration is stored in `vehicle_access_threshold_config`; every successful update increments `version`, requires the caller's previous version, and writes `vehicle_access_threshold_audit`. Stale updates return `ACCESS_THRESHOLD_VERSION_CONFLICT`. Valid ranges are bounded server-side.
|
||||
|
||||
### V2 Source Diagnostic and Provider Maintenance
|
||||
|
||||
```http
|
||||
GET /api/v2/operations/vehicles/{vin}/sources
|
||||
|
||||
PUT /api/v2/operations/vehicles/{vin}/sources/{sourceRef}
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"version": 2,
|
||||
"providerName": "G7s",
|
||||
"providerEvidence": "GPS 运维终端清单 2026-07-16",
|
||||
"enabled": true,
|
||||
"priority": 20,
|
||||
"remark": "保持当前优先级"
|
||||
}
|
||||
```
|
||||
|
||||
The diagnostic response exposes every current location candidate, including the current verified `providerOverride` and gateway-policy `policyRemark`, but never exposes the raw source key. `sourceRef` is a SHA-256 reference resolved only by the server. Operator principals may read this diagnostic; only administrators may write.
|
||||
|
||||
`providerEvidence` is mandatory whenever `providerName` changes, including removal. It is written to the immutable provider audit and is deliberately separate from `remark`, which belongs only to source enable/priority policy. A provider-only update must preserve the existing policy remark. Both change types share the vehicle-level optimistic `version`; stale writes return `SOURCE_POLICY_VERSION_CONFLICT`.
|
||||
|
||||
## Map Reverse Geocoding
|
||||
|
||||
`GET /api/map/reverse-geocode?longitude=<WGS-84>&latitude=<WGS-84>` is an authenticated server-side AMap Web Service adapter. It validates the source coordinate, converts WGS-84 to GCJ-02 exactly once, keeps the server API key out of the browser, and requests only the `base` reverse-geocode response documented by [AMap](https://lbs.amap.com/api/webservice/guide/api/georegeo).
|
||||
|
||||
@@ -203,10 +203,11 @@ test -n "$MYSQL_DSN"
|
||||
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/015_vehicle_source_policy_audit.sql \
|
||||
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/016_business_scope_dimensions.sql \
|
||||
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/017_reconciliation_center.sql \
|
||||
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/018_vehicle_oem_audit.sql
|
||||
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/018_vehicle_oem_audit.sql \
|
||||
/opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/019_vehicle_source_provider.sql
|
||||
```
|
||||
|
||||
The API guards the access-threshold tables for compatibility, while alert and reconciliation APIs deliberately require their migrations to exist. Run every numbered migration explicitly before switching traffic so DDL permission and index creation failures are caught early. The migration journal records filename and SHA-256 and refuses a changed file; duplicate forward `ADD COLUMN` and `CREATE INDEX` statements are tolerated only when resuming partially executed MySQL DDL. Full-line SQL comments are removed before statement splitting, so punctuation in a comment cannot become executable SQL. Migration `008` adds forward-compatible access evidence columns and an index to the gateway-owned realtime snapshot table without changing its `(protocol, vin)` primary key. Migration `009` creates the per-group/topic/partition event-time checkpoint used to make MySQL effects authoritative before Kafka offsets are committed. Migration `012` creates the atomic, versioned business Scope projection; its reserved `row_number` column is quoted for production MySQL compatibility. Migration `014` backfills active vehicle-grant start times, creates the grant-interval history table and adds the active time lookup index; apply it before starting an API binary that writes grant history. Migration `015` adds platform-owned optimistic versions and immutable audits for per-vehicle location-source policy changes. It does not alter the gateway election SQL or expose the gateway `source_key`; the API resolves an opaque `sourceRef` server-side and the gateway applies the saved policy on the next valid vehicle report. Migration `016` adds customer name, department and responsible-person dimensions plus bounded lookup indexes to the platform-owned Scope projection. Migration `017` creates the reconciliation run, issue and immutable action tables; the unique fingerprint index is the database-level duplicate-work-item guard. Migration `018` records every OneOS-sourced brand fill before the platform identity OEM field changes; it does not grant access to OneOS or run a cross-database sync.
|
||||
The API guards the access-threshold tables for compatibility, while alert and reconciliation APIs deliberately require their migrations to exist. Run every numbered migration explicitly before switching traffic so DDL permission and index creation failures are caught early. The migration journal records filename and SHA-256 and refuses a changed file; duplicate forward `ADD COLUMN` and `CREATE INDEX` statements are tolerated only when resuming partially executed MySQL DDL. Full-line SQL comments are removed before statement splitting, so punctuation in a comment cannot become executable SQL. Migration `008` adds forward-compatible access evidence columns and an index to the gateway-owned realtime snapshot table without changing its `(protocol, vin)` primary key. Migration `009` creates the per-group/topic/partition event-time checkpoint used to make MySQL effects authoritative before Kafka offsets are committed. Migration `012` creates the atomic, versioned business Scope projection; its reserved `row_number` column is quoted for production MySQL compatibility. Migration `014` backfills active vehicle-grant start times, creates the grant-interval history table and adds the active time lookup index; apply it before starting an API binary that writes grant history. Migration `015` adds platform-owned optimistic versions and immutable audits for per-vehicle location-source policy changes. It does not alter the gateway election SQL or expose the gateway `source_key`; the API resolves an opaque `sourceRef` server-side and the gateway applies the saved policy on the next valid vehicle report. Migration `016` adds customer name, department and responsible-person dimensions plus bounded lookup indexes to the platform-owned Scope projection. Migration `017` creates the reconciliation run, issue and immutable action tables; the unique fingerprint index is the database-level duplicate-work-item guard. Migration `018` records every OneOS-sourced brand fill before the platform identity OEM field changes; it does not grant access to OneOS or run a cross-database sync. Migration `019` stores administrator-maintained provider names by exact vehicle/protocol/source key and writes an immutable versioned audit. The browser still receives only the SHA-256 `sourceRef`; raw source keys remain server-side.
|
||||
|
||||
`docs/oneos-brand-backfill.sql` is an explicit one-time calibration, not a service dependency. It reads OneOS vehicle/model master data, inserts immutable source evidence, and fills only empty `vehicle_identity_binding.oem` values. Review the pre-run count, apply migration `018`, run the script once through the migration runner or a transaction-capable MySQL client, and verify its final audit/missing counts. Never overwrite a non-empty platform brand and never schedule this script; future updates belong in the formal OneOS API.
|
||||
|
||||
@@ -214,6 +215,8 @@ Production release `reconciliation-summary-fix-20260716191153` applied migration
|
||||
|
||||
Production release `vehicle-count-authority-exact-20260716193428` made `vehicle_identity_binding` the authoritative denominator for vehicle coverage, coverage summaries, missing-source statistics and service readiness. Authenticated production smoke returned 1,024 vehicles from vehicle query, monitor, access management and source readiness, while keeping the 11 unbound realtime-only VINs as a separate `identityRequiredVehicles` operational metric and 11 `UNBOUND_SOURCE` reconciliation issues. Do not reintroduce the realtime snapshot union into formal vehicle totals; an unbound source becomes a vehicle only after the identity-binding workflow succeeds.
|
||||
|
||||
Production release `source-provider-collation-hotfix-20260716195830` applied migration `019` and added platform-owned provider maintenance by opaque `sourceRef`. Provider changes require a separate authoritative evidence note, use the same optimistic vehicle-source version, preserve the existing gateway-policy remark, and write an immutable provider audit. Diagnostic and access projections prefer the verified override while raw `source_key` remains server-side. The first release exposed a production-only collation difference between gateway-owned source tables and platform-owned tables; the hotfix uses exact binary VIN/protocol/source-key joins, avoiding fuzzy matching and restoring both source diagnostic and access detail. Authenticated smoke on `LB9A32A21R0LS1464` verified rejected missing evidence without a version change, admin-only writes, operator read-only access, viewer denial, `v1→v2`, unchanged policy remark, immediate `JT808 / G7s` access evidence, and source-key redaction. The platform, both alert evaluators and reconciliation timer remained active.
|
||||
|
||||
Release `oneos-api-client-20260716184601` applied migrations `012` and `016` and deployed the switchable API/database sync binary. The projection is intentionally empty (`active_version` unset) until a verified OneOS endpoint is configured. A production fail-closed smoke with `ONEOS_SCOPE_SOURCE=api` and no endpoint exited nonzero without publishing any row. Do not install or enable the timer until the endpoint, service token, signing secret, source-IP restriction and a representative snapshot have passed the contract checks below.
|
||||
|
||||
Production release `source-diagnosis-stable-20260716173740` applied migration `015` before switching API traffic. The release gate verified 23 current assets and 42 compatibility assets. The authenticated diagnostic smoke used a real multi-source vehicle, confirmed that `source_key` was absent, and exercised the admin PUT route with values identical to the current policy; version, audit count and recommended source remained unchanged. Viewer/operator/admin access returned 403/200/200, median response time across 20 reads was approximately 70 ms (P95 79 ms), and the platform plus both alert evaluators remained active.
|
||||
|
||||
@@ -374,10 +374,21 @@
|
||||
- 11 个未绑定来源仍保留在数据差异中心的 11 条 `UNBOUND_SOURCE` 待处理工单中；它们只能通过身份绑定流程进入正式车辆集，不能继续造成 1,035 辆的错误口径。
|
||||
- Go 全量测试、前端 49 个文件/259 项测试、TypeScript/Vite 生产构建、Web 资产烟测及四个 systemd 单元状态检查通过。
|
||||
|
||||
提供方维护能力（release `source-provider-collation-hotfix-20260716195830`）：
|
||||
|
||||
- 运维诊断页允许管理员按具体车辆、协议和独立终端维护提供方名称；前端仍只提交 SHA-256 `sourceRef`，原始 `source_key` 仅在服务端解析和保存。
|
||||
- 提供方名称变更必须单独填写权威终端清单、厂商确认记录等核验依据。核验依据与启停/优先级的策略备注是两个独立字段和审计事实；只改提供方不会清空或改写已有策略备注。
|
||||
- 迁移 `019_vehicle_source_provider.sql` 已在生产应用，保存当前提供方投影和不可变版本审计。来源诊断与接入管理优先采用该人工核验值，再回退实时平台名或当前来源编码。
|
||||
- operator 可查看单车来源诊断但不能写入，viewer 被拒绝，只有 admin 可维护；缺少核验依据的请求返回 `SOURCE_PROVIDER_REASON_REQUIRED`，且乐观版本不会变化。
|
||||
- 首次发布在生产真实请求中发现网关来源表与平台表排序规则不同，已改为 VIN/协议/source key 的二进制精确关联；修复后单车诊断和接入明细均恢复，未通过模糊或降级匹配绕过。
|
||||
- 生产烟测以 `LB9A32A21R0LS1464` 的既有 `g7s` 识别结果写入同义规范值 `G7s`：策略版本 `v1→v2`，提供方审计包含核验依据，原策略备注保持不变，接入明细立即显示 `JT808 / G7s`，响应中不存在 `sourceKey`。平台、两类告警评估器和对账定时器均为 active。
|
||||
- Go 全量测试、前端 49 个文件/259 项测试、TypeScript/Vite 生产构建和 23 个当前 Web 资源门禁通过。
|
||||
|
||||
剩余业务资料依赖：
|
||||
|
||||
- 当前仍有 111 辆 JT808 提供方缺失。65 辆注册报文携带厂商码 `70504`，但生产中该编码同时对应 G7s、东方北斗和赛格，不能唯一映射；110 辆在 OneOS 只标记为“氢气智能管理平台”，这是聚合来源而非终端提供方；非当前来源中也没有额外 `source_code` 可补齐。
|
||||
- 因此这 111 辆继续留在待处理队列，需 GPS 运维/厂家提供 phone/终端到提供方的权威清单。平台不会根据终端号、IP、位置或厂商码猜测提供方。
|
||||
- 当前动态复核仍有 104 辆 JT808 提供方缺失。注册报文厂商码在生产中同时对应 G7s、东方北斗和赛格，不能唯一映射；OneOS 只标记为“氢气智能管理平台”，这是聚合来源而非终端提供方。
|
||||
- 进一步只读核对 `lingniu_prod` 后确认：旧 `tab_truck_device_info`、`view_addgpsdata`、`historical_data` 设备厂家表当前均无有效数据；仅存的 `v_vehicle_daily_stats.source=G7S/TBOX/NONE` 最晚截至 2025-12-08，属于历史里程来源，不能证明 2026 年当前终端提供方。
|
||||
- 因此这 104 辆继续留在待处理队列，需 GPS 运维/厂家提供当前 phone/终端到提供方的权威清单。平台不会根据终端号、IP、位置、旧里程来源或厂商码猜测提供方。
|
||||
- 另有 8 辆 OneOS 车型主数据本身无品牌，需资产资料责任人补录后再通过正式 API 同步。
|
||||
|
||||
目标：
|
||||
|
||||
Reference in New Issue
Block a user