diff --git a/vehicle-data-platform/docs/deployment.md b/vehicle-data-platform/docs/deployment.md index f2f9a290..0d51d560 100644 --- a/vehicle-data-platform/docs/deployment.md +++ b/vehicle-data-platform/docs/deployment.md @@ -66,13 +66,14 @@ PREVIOUS_WEB=/opt/lingniu-vehicle-platform/releases/$PREVIOUS_RELEASE/web "$PREVIOUS_WEB/.release-assets" ``` -`deploy/install-web-release.sh` also accepts an optional third argument containing a newly built `platform-api`. When provided, it atomically publishes the new API and Web together. The installer inherits every runtime binary that exists in the previous release (`alert-evaluator`, `alert-stream-evaluator`, `platform-migrate`, `oneos-scope-sync` and the optional benchmark) before switching the symlink, so a later systemd restart cannot fail because a Web-oriented release omitted an unchanged binary: +`deploy/install-web-release.sh` also accepts an optional third argument containing a newly built `platform-api` and an optional fourth argument containing a newly built `oneos-scope-sync`. When provided, it atomically publishes those binaries with the Web. The installer inherits every runtime binary that exists in the previous release (`alert-evaluator`, `alert-stream-evaluator`, `platform-migrate`, `oneos-scope-sync` and the optional benchmark) before switching the symlink, so a later systemd restart cannot fail because a Web-oriented release omitted an unchanged binary: ```bash deploy/install-web-release.sh \ "$PLATFORM_RELEASE" \ "/tmp/$PLATFORM_RELEASE-web.tar.gz" \ - "/tmp/$PLATFORM_RELEASE-platform-api" + "/tmp/$PLATFORM_RELEASE-platform-api" \ + "/tmp/$PLATFORM_RELEASE-oneos-scope-sync" ``` ## Environment @@ -98,6 +99,12 @@ BOOTSTRAP_ADMIN_USERNAME=admin BOOTSTRAP_ADMIN_PASSWORD= AUTH_SESSION_TTL_HOURS=12 ONEOS_MYSQL_DSN=vehicle_scope_reader:***@tcp(rm-bp179zbv481rnw3e2.mysql.rds.aliyuncs.com:3306)/ln_asset_management?parseTime=true&loc=Asia%2FShanghai +# 正式模式使用 OneOS HTTP 内部接口;database 仅用于接口上线前的只读影子核对。 +ONEOS_SCOPE_SOURCE=api +ONEOS_SCOPE_API_URL=https:///inner/v1/vehicle-data-platform/customer-vehicle-scopes +ONEOS_SCOPE_API_SERVICE_TOKEN= +ONEOS_SCOPE_API_SIGNING_SECRET= +ONEOS_SCOPE_API_MAX_ATTEMPTS=3 ONEOS_SCOPE_SYNC_TIMEOUT_SEC=60 ONEOS_SCOPE_MAX_REJECTED=100 ONEOS_SCOPE_MAX_REJECT_RATIO=0.10 @@ -158,7 +165,9 @@ Before a customer demonstration, run `deploy/verify-customer-demo.py` with the p After editing the environment file, run `chmod 600 /opt/lingniu-vehicle-platform/env/platform.env`. Never put a real token in Git, static JavaScript, shell history or deployment logs. -`ONEOS_MYSQL_DSN` must use a dedicated account with direct table-level `SELECT` grants only. The sync binary runs `SHOW GRANTS FOR CURRENT_USER` before every read and accepts only global `USAGE` plus `SELECT` on the seven tables used by its query. It refuses database/global reads, `ALL PRIVILEGES`, DML, DDL, PROCESS, replication, roles, `SHOW VIEW`, or any other privilege. It then opens a repeatable-read, read-only transaction, applies a 10-second statement timeout, classifies invalid customer/contract relationships, and atomically publishes a content-addressed local snapshot. Never point it at the existing `ln-bi` account: production audit showed that account still has broad write and replication privileges. +`ONEOS_SCOPE_SOURCE=api` is the intended production integration. The client sends a dedicated service token plus a timestamp/request-ID HMAC signature, requires one stable `scopeVersion` and `generatedAt` across every page, and fails closed on incomplete data, cursor loops, version drift, malformed IDs/timestamps, nonzero business codes, row/page limits or exhausted transient retries. HTTP is accepted only for literal private/loopback IP addresses; all other endpoints require HTTPS. Never log either secret. + +`ONEOS_SCOPE_SOURCE=database` is a temporary shadow/audit path before the OneOS team publishes the formal endpoint. In that mode, `ONEOS_MYSQL_DSN` must use a dedicated account with direct table-level `SELECT` grants only. The sync binary runs `SHOW GRANTS FOR CURRENT_USER` before every read and accepts only global `USAGE` plus `SELECT` on the seven tables used by its query. It refuses database/global reads, `ALL PRIVILEGES`, DML, DDL, PROCESS, replication, roles, `SHOW VIEW`, or any other privilege. It then opens a repeatable-read, read-only transaction, applies a 10-second statement timeout, classifies invalid customer/contract relationships, and atomically publishes a content-addressed local snapshot. Never point it at the existing `ln-bi` account: production audit showed that account still has broad write and replication privileges. Have an RDS administrator review and run `docs/oneos-scope-reader-provision.sql` separately. It restricts the login source to the verified ECS private address and grants `SELECT` on only the seven source tables used by the query. Do not include that DDL in application deployment or migration automation. Store `platform.env` as root-owned mode `0600`, and verify `SHOW GRANTS` before enabling the timer. @@ -188,10 +197,13 @@ test -n "$MYSQL_DSN" /opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/012_business_scope_projection.sql \ /opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/013_platform_identity_access.sql \ /opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/014_customer_vehicle_grant_time.sql \ - /opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/015_vehicle_source_policy_audit.sql + /opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/015_vehicle_source_policy_audit.sql \ + /opt/lingniu-vehicle-platform/releases/$PLATFORM_RELEASE/deploy/migrations/016_business_scope_dimensions.sql ``` -The API guards the access-threshold tables for compatibility, while alert APIs deliberately require the alert migrations to exist. Run every numbered migration explicitly before switching traffic so DDL permission and index creation failures are caught early. The migration journal records filename and SHA-256 and refuses a changed file; duplicate forward `ADD COLUMN` and `CREATE INDEX` statements are tolerated only when resuming partially executed MySQL DDL. Full-line SQL comments are removed before statement splitting, so punctuation in a comment cannot become executable SQL. Migration `008` adds forward-compatible access evidence columns and an index to the gateway-owned realtime snapshot table without changing its `(protocol, vin)` primary key. Migration `009` creates the per-group/topic/partition event-time checkpoint used to make MySQL effects authoritative before Kafka offsets are committed. Migration `014` backfills active vehicle-grant start times, creates the grant-interval history table and adds the active time lookup index; apply it before starting an API binary that writes grant history. Migration `015` adds platform-owned optimistic versions and immutable audits for per-vehicle location-source policy changes. It does not alter the gateway election SQL or expose the gateway `source_key`; the API resolves an opaque `sourceRef` server-side and the gateway applies the saved policy on the next valid vehicle report. +The API guards the access-threshold tables for compatibility, while alert APIs deliberately require the alert migrations to exist. Run every numbered migration explicitly before switching traffic so DDL permission and index creation failures are caught early. The migration journal records filename and SHA-256 and refuses a changed file; duplicate forward `ADD COLUMN` and `CREATE INDEX` statements are tolerated only when resuming partially executed MySQL DDL. Full-line SQL comments are removed before statement splitting, so punctuation in a comment cannot become executable SQL. Migration `008` adds forward-compatible access evidence columns and an index to the gateway-owned realtime snapshot table without changing its `(protocol, vin)` primary key. Migration `009` creates the per-group/topic/partition event-time checkpoint used to make MySQL effects authoritative before Kafka offsets are committed. Migration `012` creates the atomic, versioned business Scope projection; its reserved `row_number` column is quoted for production MySQL compatibility. Migration `014` backfills active vehicle-grant start times, creates the grant-interval history table and adds the active time lookup index; apply it before starting an API binary that writes grant history. Migration `015` adds platform-owned optimistic versions and immutable audits for per-vehicle location-source policy changes. It does not alter the gateway election SQL or expose the gateway `source_key`; the API resolves an opaque `sourceRef` server-side and the gateway applies the saved policy on the next valid vehicle report. Migration `016` adds customer name, department and responsible-person dimensions plus bounded lookup indexes to the platform-owned Scope projection. + +Release `oneos-api-client-20260716184601` applied migrations `012` and `016` and deployed the switchable API/database sync binary. The projection is intentionally empty (`active_version` unset) until a verified OneOS endpoint is configured. A production fail-closed smoke with `ONEOS_SCOPE_SOURCE=api` and no endpoint exited nonzero without publishing any row. Do not install or enable the timer until the endpoint, service token, signing secret, source-IP restriction and a representative snapshot have passed the contract checks below. Production release `source-diagnosis-stable-20260716173740` applied migration `015` before switching API traffic. The release gate verified 23 current assets and 42 compatibility assets. The authenticated diagnostic smoke used a real multi-source vehicle, confirmed that `source_key` was absent, and exercised the admin PUT route with values identical to the current policy; version, audit count and recommended source remained unchanged. Viewer/operator/admin access returned 403/200/200, median response time across 20 reads was approximately 70 ms (P95 79 ms), and the platform plus both alert evaluators remained active. @@ -203,11 +215,13 @@ sudo cp deploy/systemd/lingniu-vehicle-alert-stream-evaluator.service /etc/syste sudo cp deploy/systemd/lingniu-vehicle-oneos-scope-sync.service /etc/systemd/system/ sudo cp deploy/systemd/lingniu-vehicle-oneos-scope-sync.timer /etc/systemd/system/ sudo systemctl daemon-reload -sudo systemctl enable --now lingniu-vehicle-platform lingniu-vehicle-alert-evaluator lingniu-vehicle-alert-stream-evaluator lingniu-vehicle-oneos-scope-sync.timer +sudo systemctl enable --now lingniu-vehicle-platform lingniu-vehicle-alert-evaluator lingniu-vehicle-alert-stream-evaluator sudo systemctl status --no-pager lingniu-vehicle-alert-evaluator sudo systemctl status --no-pager lingniu-vehicle-alert-stream-evaluator sudo journalctl -u lingniu-vehicle-alert-evaluator -n 100 --no-pager sudo journalctl -u lingniu-vehicle-alert-stream-evaluator -n 100 --no-pager +# Only after a successful manual OneOS API sync and projection audit: +sudo systemctl enable --now lingniu-vehicle-oneos-scope-sync.timer sudo systemctl status --no-pager lingniu-vehicle-oneos-scope-sync.timer sudo journalctl -u lingniu-vehicle-oneos-scope-sync.service -n 100 --no-pager ``` diff --git a/vehicle-data-platform/docs/oneos-customer-vehicle-scope-contract.md b/vehicle-data-platform/docs/oneos-customer-vehicle-scope-contract.md index c5d8b458..ae5cfd4a 100644 --- a/vehicle-data-platform/docs/oneos-customer-vehicle-scope-contract.md +++ b/vehicle-data-platform/docs/oneos-customer-vehicle-scope-contract.md @@ -14,23 +14,32 @@ ## 2. 推荐接口 -### 查询客户车辆快照 +### 查询车辆中台全量业务范围快照 + +车辆数据中台的定时同步需要一次读取全部客户的当前车辆范围,避免先拉客户列表再产生 N+1 请求。推荐由 OneOS 提供专用平台快照: ```http -GET /inner/v1/customer-vehicle-scopes/{customerId}?cursor=&limit=500 +GET /inner/v1/vehicle-data-platform/customer-vehicle-scopes?cursor=&limit=500 Authorization: Service X-Request-Id: X-Request-Timestamp: X-Request-Signature: ``` +签名原文必须逐字节使用: + +```text +GET\n\n\n +``` + +双方使用独立共享密钥计算 HMAC-SHA256 小写十六进制;服务令牌与签名密钥不能相同。OneOS 应拒绝超过 60 秒的时间戳、重复 request ID、错误 audience 和外网来源。网关必须剥离外部传入的同名身份头。 + 响应: ```json { "code": 0, "data": { - "customerId": "1001", "scopeVersion": "2026-07-14T12:00:00.123Z/987654", "generatedAt": "2026-07-14T12:00:00.123Z", "complete": true, @@ -41,9 +50,14 @@ X-Request-Signature: "vin": "LXXXXXXXXXXXXXXXX", "plateNumber": "沪A00000", "customerId": "1001", + "customerName": "示例客户", "contractId": "3001", "contractCode": "HT20260001", "projectName": "示例项目", + "departmentId": "4001", + "departmentName": "运营一部", + "responsibleUserId": "5001", + "responsibleUserName": "示例负责人", "modelName": "示例车型", "brandName": "示例品牌", "operationStatus": "2", @@ -66,10 +80,25 @@ X-Request-Signature: - 雪花 ID 一律用 JSON 字符串传输,避免 JavaScript 数字精度丢失。 - `scopeVersion` 对同一业务快照稳定;分页中的每一页必须来自同一个快照。 -- `complete=false` 表示快照不完整,车辆中台不得发布该版本。 +- `generatedAt` 和 `scopeVersion` 在所有分页中必须完全一致。 +- 任意分页 `complete=false` 表示快照不完整,车辆中台不得发布该版本。 - `items` 内 VIN 必须非空、标准化为大写并在当前快照内唯一。 - `rejected` 只返回机器可读原因和内部车辆 ID,不返回敏感客户信息。 -- 建议 `ETag=scopeVersion`,支持 `If-None-Match` 降低同步流量。 +- 每页最多 500 条;全部快照最多 50,000 条、100 页。游标不能重复。 +- 客户、车辆、合同等雪花 ID 必须使用 JSON 字符串;部门和负责人允许为空,但不能伪造。 +- 建议 `ETag=scopeVersion`,后续支持 `If-None-Match` 降低同步流量。 + +车辆中台已实现该契约的客户端:总同步超时默认 60 秒,单请求超时 10 秒;网络错误、HTTP 429 和 5xx 最多重试 3 次,其他 4xx、业务 code 非 0、JSON 异常、分页版本漂移、重复游标、越界行数或不完整快照立即失败关闭。 + +### 查询单个客户车辆快照 + +如 OneOS 还需要为内部业务提供客户级接口,可保留: + +```http +GET /inner/v1/customer-vehicle-scopes/{customerId}?cursor=&limit=500 +``` + +该接口不作为车辆中台全量定时同步的唯一入口,以免形成客户级 N+1。 ### 批量检查客户车辆归属 diff --git a/vehicle-data-platform/docs/vehicle-data-platform-meeting-todo.md b/vehicle-data-platform/docs/vehicle-data-platform-meeting-todo.md index 7eddc484..fa734a42 100644 --- a/vehicle-data-platform/docs/vehicle-data-platform-meeting-todo.md +++ b/vehicle-data-platform/docs/vehicle-data-platform-meeting-todo.md @@ -402,7 +402,23 @@ ### P1-04 OneOS 业务接口对接 -状态:`进行中` +状态:`进行中(中台接入端已完成,等待 OneOS 正式接口联调)` + +已完成的中台侧结果(release `oneos-api-client-20260716184601`): + +- 新增可切换的 OneOS HTTP 范围快照客户端;正式模式为 `ONEOS_SCOPE_SOURCE=api`,原有数据库直读仅保留为接口上线前的只读影子核对,不修改 `ln-asset-management`。 +- 请求使用独立 Service Token、请求 ID、Unix 时间戳和 HMAC-SHA256 防重放签名;非私网字面 IP 强制 HTTPS,服务令牌与签名密钥均不写日志。 +- 支持每页 500 条的稳定游标分页,最多 100 页/50,000 条;要求所有分页 `scopeVersion`、`generatedAt` 完全一致且 `complete=true`。网络错误、429 和 5xx 最多重试 3 次,版本漂移、重复游标、业务错误、格式错误或越界立即失败关闭。 +- 客户、车辆、合同雪花 ID 按 JSON 字符串读取;投影已增加客户名称、部门、负责人、合同、项目、启用时间和来源更新时间,并继续以内容地址版本原子发布。 +- 修复旧迁移 012 中生产 MySQL 保留关键字 `row_number` 的兼容问题;生产已成功应用 012 和 016,投影字段及索引齐全。 +- 同步定时器继续保持未安装/未启用,当前 `active_version` 为空、accepted/rejected 均为 0。缺少正式接口配置时生产手工运行返回退出码 1,不发布空快照,也不退化成全量车辆。 +- 发布安装器支持同时原子替换新版 `oneos-scope-sync`;Go 全量测试、接口签名/分页/版本漂移/瞬时重试测试和安装器测试通过,ECS 三个现有服务均为 active。 + +尚待 OneOS 团队交付并联调: + +- 正式内网 URL、独立服务令牌、HMAC 密钥和允许访问的 ECS 私网源地址; +- 符合契约的全客户完整快照,包含稳定版本、客户/部门/负责人、合同/项目、正式交车启用时间及机器可读隔离原因; +- OneOS 侧完成时间戳窗口、request ID 去重、来源限制和接口监控。取得后先手工影子同步、核对抽样与总数,再启用两分钟定时器。 目标: