chore: commit mileage updates and accumulated project artifacts
This commit is contained in:
@@ -0,0 +1,68 @@
|
||||
# 里程对账修复与发布记录 — 2026-09-16
|
||||
|
||||
已完成代码修复、MySQL 全库一致性备份、生产数据修复、发布和验证。未改动工作区原有的用氢导出及地理统计开发内容。
|
||||
|
||||
## 已采用的口径
|
||||
|
||||
- 开放平台单日和区间接口共用 `ReconciledMileageRange`,直接使用可追溯终端累计读数。
|
||||
- 同一来源的日里程 = 当日累计值 − 前一天累计值,保留三位小数。缺报日累计值结转、日里程为 0,并标记 `CARRIED_FORWARD`;恢复日承接跨缺报期增量。这是对账日里程,不保证增量实际发生在恢复当天。
|
||||
- 按请求协议优先级选择截至该日已存在有效读数的协议;该协议缺报时沿用其历史读数,不自动切换到低优先级协议的另一累计基准。
|
||||
- 首次基线、来源变化、累计回退及异常累计记录返回 `DATA_ANOMALY`,`dailyMileageKm=null`,保留累计值与来源时间,避免伪造可连续相减的里程。
|
||||
- GPS 估算继续保存在内部独立来源记录中,不参与开放平台累计差日里程。日里程和累计值的时间证据来自同一来源。
|
||||
- 实时统计和历史回补接受更早的同来源基线,不再丢弃跨完整缺报日的增量。
|
||||
- 无协议依据的旧库迁移入口拒绝执行写入,禁止将旧库记录统一标为 GB32960。
|
||||
|
||||
## 备份
|
||||
|
||||
主机:115.29.187.205。数据库:`lingniu_vehicle_data`。
|
||||
|
||||
完整备份:`/opt/lingniu-go-native/backups/mileage-reconciliation-20260916/database.sql.gz`
|
||||
|
||||
- 一致性快照逻辑备份:全部表结构和数据;当前库无存储过程、无数据库事件。
|
||||
- 文件大小:499,964,151 bytes;目录权限 0700、备份文件权限 0600。
|
||||
- SHA-256:`5d1f86e062d3410316a6ea66775aebcd8061e2d4a959ca304b7bf6a93e9438c4`。
|
||||
- 完成时间:2026-09-16 11:51:31 +08:00;已通过 `gzip -t` 完整性检查。
|
||||
- 本目录 `backup-manifest.json` 保存逐表行数和备份校验信息。未把数据库凭证写入报告或日志。
|
||||
- 服务版本、配置备份和修复前基线另保存在同一服务器备份目录。
|
||||
|
||||
## 数据修复
|
||||
|
||||
1. 两类历史导入来源 `legacy-mysql.lingniu-prod`、`manual-lingniu-prod-day-mileage` 共 169,604 条、涉及 892 个 VIN,标为 `INVALID_DELTA / UNVERIFIED_LEGACY_PROTOCOL`,清除选中标记。原始来源记录和旧库数据保留。
|
||||
2. 移除由这些来源生成的 169,014 条错误 GB32960 日统计;未将其猜改成 MQTT 或 JT808。
|
||||
3. 修复截至 09-16 的 5,635 条当前日首帧基线记录,使用此前同来源最近有效累计值。保留累计读数本身,维持回退及按跨日长度计算的异常上限检查。
|
||||
4. 通过正式统计投影函数重建 10,079 个车辆/日期/协议统计,恢复被旧迁移记录遮盖的真实来源统计。
|
||||
5. 生产复核:误标来源对应的最终统计剩余 0 条;169,604 条来源记录均保留、未选中、已标记协议未确认。
|
||||
|
||||
影响边界:只有协议不明的历史导入、没有真实终端证据的日期,不再作为正常协议里程返回,可能表现为 NO_DATA。不能把这类变化当作车辆里程真实减少。来源变化等异常记录也不能直接把日里程 null 当作正常 0 后参与完整对账。
|
||||
|
||||
## 样本结果
|
||||
|
||||
| 车辆 | 日期 | 修复后日里程 km | 累计 km | 说明 |
|
||||
|---|---|---:|---:|---|
|
||||
| 沪A60591F | 09-13~09-15 | 0 | 51,416 | MQTT 历史结转,不再返回错误 GB32960 |
|
||||
| 粤AGQ8377 | 09-15 | 101.4 | 14,467.4 | 原 31.2,补回跨缺报期 70.2 |
|
||||
| 粤AGR6839 | 09-15 | 40.6 | 45,162.8 | 原 2.6,补回跨缺报期 38.0 |
|
||||
| 粤AGP3609 | 09-14 | 878.8 | 48,981.7 | 原 70.1,补回跨缺报期 808.7 |
|
||||
| 粤AGP3609 | 09-15 | 1,367.4 | 50,349.1 | 原正常日期保持一致 |
|
||||
|
||||
## 验证
|
||||
|
||||
- API Go 模块 `go test ./...` 通过;统计模块、回补命令、迁移命令相关测试通过。
|
||||
- 新增恢复日增量、协议缺报结转、来源替换、首次基线以及单日/区间切分一致性测试。
|
||||
- 使用与 HTTP 接口相同的生产 Repository,只读核对 1,024 个当前车辆绑定、09-11~09-15。
|
||||
- 返回 3,435 条有累计证据记录:3,399 条正常/结转、36 条来源变化标记;其余车辆日期无可用协议证据。
|
||||
- 正常/结转记录的相邻日累计差核对失败数:0;逐日查询与整段查询的字段差异数:0。
|
||||
- `verification-all.json`、`verification-samples.json`、`database-verification.json` 保存实际验证结果。验证覆盖真实 SQL 与计算路径;未使用客户密钥重放外部认证 HTTP 请求。
|
||||
- 发布后统计服务 `/readyz`、开放平台 `/healthz` 正常;三协议全部 Kafka 分区 lag=0,三个 worker 重试队列=0,相关服务近期无 error 级日志。
|
||||
|
||||
## 发布与恢复资料
|
||||
|
||||
发布标识:`mileage-reconciliation-20260916`。
|
||||
|
||||
- Go 发布目录:`/opt/lingniu-go-native/releases/mileage-reconciliation-20260916`,更新 stat-writer、stats-backfill、legacy-mileage-migrate,并包含显式列表重投影工具 mileage-reproject。
|
||||
- 开放平台发布目录:`/opt/lingniu-vehicle-open-platform/releases/mileage-reconciliation-20260916`。
|
||||
- 旧版本路径存于服务器备份目录 `previous-go-release.txt`、`previous-open-release.txt`;旧开放平台环境配置存于 `open-platform.env.before`。
|
||||
- 数据恢复应先将完整备份恢复到独立临时库,再按本次来源及重建清单恢复受影响记录;不要直接覆盖整个线上库,以免覆盖备份后新增业务数据。
|
||||
- `baseline-before.json`、`reproject.json`、`repair-result.json` 为本次修复清单和结果。`repair.py` 为本次一次性修复工具,默认仅预检。
|
||||
|
||||
下游 BI 历史缓存可能仍保留旧响应,需刷新或等待原有最长 6 小时缓存过期。本次没有修改下游 BI 或清空其缓存。
|
||||
@@ -0,0 +1,75 @@
|
||||
{
|
||||
"started_at": "2026-09-16T11:39:52+0800",
|
||||
"tables": {
|
||||
"business_customer_vehicle_scope": 0,
|
||||
"business_scope_rejection": 0,
|
||||
"business_scope_state": 1,
|
||||
"business_scope_sync_run": 0,
|
||||
"jt808_registration": 837,
|
||||
"platform_auth_audit": 1616,
|
||||
"platform_user": 13,
|
||||
"platform_user_business_scope": 0,
|
||||
"platform_user_menu": 48,
|
||||
"platform_user_session": 1363,
|
||||
"platform_user_vehicle": 2169,
|
||||
"platform_user_vehicle_grant_history": 2169,
|
||||
"platform_vehicle_source_policy_audit": 2,
|
||||
"platform_vehicle_source_policy_version": 103,
|
||||
"platform_vehicle_source_provider": 103,
|
||||
"platform_vehicle_source_provider_audit": 105,
|
||||
"vehicle": 940,
|
||||
"vehicle_access_identity_audit": 0,
|
||||
"vehicle_access_threshold_audit": 0,
|
||||
"vehicle_access_threshold_config": 1,
|
||||
"vehicle_alert_candidate": 11,
|
||||
"vehicle_alert_event": 25594,
|
||||
"vehicle_alert_event_action": 50114,
|
||||
"vehicle_alert_notification": 25594,
|
||||
"vehicle_alert_notification_retry_audit": 0,
|
||||
"vehicle_alert_rule": 12,
|
||||
"vehicle_alert_rule_audit": 100,
|
||||
"vehicle_alert_rule_state": 0,
|
||||
"vehicle_alert_stream_checkpoint": 9,
|
||||
"vehicle_daily_geography": 45609,
|
||||
"vehicle_daily_gps_mileage_state": 56017,
|
||||
"vehicle_daily_mileage": 1624735,
|
||||
"vehicle_daily_mileage_source": 1644263,
|
||||
"vehicle_data_source": 1804,
|
||||
"vehicle_geography_cache": 1206430,
|
||||
"vehicle_hydrogen_energy_parameter": 1141,
|
||||
"vehicle_hydrogen_tank_capacity": 1144,
|
||||
"vehicle_identifier": 2277,
|
||||
"vehicle_identity_binding": 1024,
|
||||
"vehicle_identity_oem_audit": 244,
|
||||
"vehicle_location_source_policy": 2,
|
||||
"vehicle_metric_definition": 28,
|
||||
"vehicle_metric_protocol_mapping": 40,
|
||||
"vehicle_open_admin_audit": 46,
|
||||
"vehicle_open_api_audit": 16212456,
|
||||
"vehicle_open_app": 5,
|
||||
"vehicle_open_app_vehicle": 4116,
|
||||
"vehicle_open_daily_energy": 29369,
|
||||
"vehicle_open_hydrogen_rate_stream_state": 19,
|
||||
"vehicle_open_hydrogen_segment_stream_state": 12558,
|
||||
"vehicle_open_hydrogen_stream_state": 17754,
|
||||
"vehicle_open_mileage_snapshot": 90,
|
||||
"vehicle_open_mileage_snapshot_vehicle": 90114,
|
||||
"vehicle_open_user": 3,
|
||||
"vehicle_open_user_app": 3,
|
||||
"vehicle_open_user_audit": 100,
|
||||
"vehicle_open_user_session": 61,
|
||||
"vehicle_platform_schema_migration": 45,
|
||||
"vehicle_profile": 1024,
|
||||
"vehicle_profile_audit": 1024,
|
||||
"vehicle_realtime_location": 1458,
|
||||
"vehicle_realtime_location_source": 811,
|
||||
"vehicle_realtime_snapshot": 1460,
|
||||
"vehicle_reconciliation_action": 20443,
|
||||
"vehicle_reconciliation_issue": 10648,
|
||||
"vehicle_reconciliation_run": 65
|
||||
},
|
||||
"database": "lingniu_vehicle_data",
|
||||
"sha256": "5d1f86e062d3410316a6ea66775aebcd8061e2d4a959ca304b7bf6a93e9438c4",
|
||||
"bytes": 499964151,
|
||||
"finished_at": "2026-09-16T11:51:31+0800"
|
||||
}
|
||||
@@ -0,0 +1,197 @@
|
||||
{
|
||||
"legacy_projection_remaining": [
|
||||
{
|
||||
"n": 0
|
||||
}
|
||||
],
|
||||
"legacy_quarantine": [
|
||||
{
|
||||
"quality_status": "INVALID_DELTA",
|
||||
"quality_reason": "UNVERIFIED_LEGACY_PROTOCOL",
|
||||
"is_selected": 0,
|
||||
"n": 169604
|
||||
}
|
||||
],
|
||||
"samples": [
|
||||
{
|
||||
"plate": "沪A60591F",
|
||||
"vin": "LMRKH9AC7R1004098",
|
||||
"stat_date": "2026-09-12",
|
||||
"protocol": "YUTONG_MQTT",
|
||||
"source_id": 1,
|
||||
"daily_mileage_km": "0.000",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "51416.000",
|
||||
"day_end_total_mileage_km": "51416.000",
|
||||
"updated_at": "2026-09-12 04:53:49"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGP3609",
|
||||
"vin": "LB9A32A23R0LS1661",
|
||||
"stat_date": "2026-09-12",
|
||||
"protocol": "GB32960",
|
||||
"source_id": 3,
|
||||
"daily_mileage_km": "1293.600",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "48102.900",
|
||||
"day_end_total_mileage_km": "48102.900",
|
||||
"updated_at": "2026-09-15 01:35:09"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGP3609",
|
||||
"vin": "LB9A32A23R0LS1661",
|
||||
"stat_date": "2026-09-12",
|
||||
"protocol": "JT808",
|
||||
"source_id": null,
|
||||
"daily_mileage_km": "0.148",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "0.148",
|
||||
"day_end_total_mileage_km": "37571.200",
|
||||
"updated_at": "2026-09-15 01:35:21"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGP3609",
|
||||
"vin": "LB9A32A23R0LS1661",
|
||||
"stat_date": "2026-09-14",
|
||||
"protocol": "GB32960",
|
||||
"source_id": 3,
|
||||
"daily_mileage_km": "878.800",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "48981.700",
|
||||
"day_end_total_mileage_km": "48981.700",
|
||||
"updated_at": "2026-09-16 11:54:43"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGP3609",
|
||||
"vin": "LB9A32A23R0LS1661",
|
||||
"stat_date": "2026-09-14",
|
||||
"protocol": "JT808",
|
||||
"source_id": null,
|
||||
"daily_mileage_km": "0.134",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "0.134",
|
||||
"day_end_total_mileage_km": "37571.100",
|
||||
"updated_at": "2026-09-16 11:54:43"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGP3609",
|
||||
"vin": "LB9A32A23R0LS1661",
|
||||
"stat_date": "2026-09-15",
|
||||
"protocol": "GB32960",
|
||||
"source_id": 3,
|
||||
"daily_mileage_km": "1367.400",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "50349.100",
|
||||
"day_end_total_mileage_km": "50349.100",
|
||||
"updated_at": "2026-09-16 01:34:49"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGP3609",
|
||||
"vin": "LB9A32A23R0LS1661",
|
||||
"stat_date": "2026-09-15",
|
||||
"protocol": "JT808",
|
||||
"source_id": 7,
|
||||
"daily_mileage_km": "29.400",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "37600.500",
|
||||
"day_end_total_mileage_km": "37600.500",
|
||||
"updated_at": "2026-09-16 01:35:31"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGQ8377",
|
||||
"vin": "LB9A32A27R0LS1419",
|
||||
"stat_date": "2026-09-12",
|
||||
"protocol": "GB32960",
|
||||
"source_id": 3,
|
||||
"daily_mileage_km": "31.800",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "14359.900",
|
||||
"day_end_total_mileage_km": "14359.900",
|
||||
"updated_at": "2026-09-16 11:54:50"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGQ8377",
|
||||
"vin": "LB9A32A27R0LS1419",
|
||||
"stat_date": "2026-09-12",
|
||||
"protocol": "JT808",
|
||||
"source_id": 7,
|
||||
"daily_mileage_km": "31.500",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "14422.700",
|
||||
"day_end_total_mileage_km": "14422.700",
|
||||
"updated_at": "2026-09-16 11:54:50"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGQ8377",
|
||||
"vin": "LB9A32A27R0LS1419",
|
||||
"stat_date": "2026-09-13",
|
||||
"protocol": "GB32960",
|
||||
"source_id": 3,
|
||||
"daily_mileage_km": "6.100",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "14366.000",
|
||||
"day_end_total_mileage_km": "14366.000",
|
||||
"updated_at": "2026-09-16 01:35:13"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGQ8377",
|
||||
"vin": "LB9A32A27R0LS1419",
|
||||
"stat_date": "2026-09-13",
|
||||
"protocol": "JT808",
|
||||
"source_id": 7,
|
||||
"daily_mileage_km": "76.300",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "14499.000",
|
||||
"day_end_total_mileage_km": "14499.000",
|
||||
"updated_at": "2026-09-16 01:35:19"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGQ8377",
|
||||
"vin": "LB9A32A27R0LS1419",
|
||||
"stat_date": "2026-09-15",
|
||||
"protocol": "GB32960",
|
||||
"source_id": 3,
|
||||
"daily_mileage_km": "101.400",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "14467.400",
|
||||
"day_end_total_mileage_km": "14467.400",
|
||||
"updated_at": "2026-09-16 11:54:50"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGQ8377",
|
||||
"vin": "LB9A32A27R0LS1419",
|
||||
"stat_date": "2026-09-15",
|
||||
"protocol": "JT808",
|
||||
"source_id": 7,
|
||||
"daily_mileage_km": "31.500",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "14530.500",
|
||||
"day_end_total_mileage_km": "14530.500",
|
||||
"updated_at": "2026-09-16 11:54:50"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGR6839",
|
||||
"vin": "LB9A32A23R0LS1434",
|
||||
"stat_date": "2026-09-15",
|
||||
"protocol": "GB32960",
|
||||
"source_id": 3,
|
||||
"daily_mileage_km": "40.600",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "45162.800",
|
||||
"day_end_total_mileage_km": "45162.800",
|
||||
"updated_at": "2026-09-16 11:54:42"
|
||||
},
|
||||
{
|
||||
"plate": "粤AGR6839",
|
||||
"vin": "LB9A32A23R0LS1434",
|
||||
"stat_date": "2026-09-15",
|
||||
"protocol": "JT808",
|
||||
"source_id": 7,
|
||||
"daily_mileage_km": "53.100",
|
||||
"pure_hydrogen_mileage_km": "0.000",
|
||||
"latest_total_mileage_km": "11138.800",
|
||||
"day_end_total_mileage_km": "11138.800",
|
||||
"updated_at": "2026-09-16 11:54:42"
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"legacy": {
|
||||
"n": 169604,
|
||||
"vehicles": 892
|
||||
},
|
||||
"legacy_projections": {
|
||||
"n": 169014
|
||||
},
|
||||
"baseline_candidates": 5635,
|
||||
"quarantined_source_rows": 169604,
|
||||
"removed_mislabelled_projections": 169014,
|
||||
"reproject_days": 10079
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
|
||||
{"elapsed":"2.236567803s","normal":3399,"qualities":{"":2333,"CARRIED_FORWARD":1066,"ODOMETER_SOURCE_CHANGED":36},"reconciliationMismatches":0,"rows":3435,"samples":{},"singleRangeDifferences":0,"vehicles":1024}
|
||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user