573f8397a6
输入文件：
- 租赁任务考核_2026年{1,2,3}月.xlsx (考核源数据)
- {1,2}月.xlsx (客户盈亏表)
- 车辆里程考核与奖金发放规则（V.1.2）.docx
输出文件：
- 里程任务考核_{1,2,3}月核算.xlsx (月度核算结果)
- 里程任务考核_Q1汇总.xlsx (含车辆台账)
- 3月客户盈亏表(待填写).xlsx (模版)
.claude_memory: 项目记忆（规则/偏好/架构/测试车辆）
.claude_plans: 历次计划文件
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
243 lines
12 KiB
Markdown
243 lines
12 KiB
Markdown
# 数据迁移计划：lingniu_prod → 新系统
|
||
|
||
## Context
|
||
|
||
老系统 `lingniu_prod`（单库 255 张表）需要全量迁移到新系统的 3 个库。新系统已做架构重构：拆分为 `ln_asset_management`（资产管理）、`ln_energy`（能源计费）、`ry-cloud`（系统核心/RuoYi 框架）。新库中现有数据为测试数据，迁移前清空。
|
||
|
||
## 源与目标
|
||
|
||
| | 源 | 目标 |
|
||
|---|---|---|
|
||
| **Host** | rm-uf65w5v2r77n674x2ko.mysql.rds.aliyuncs.com | 47.100.22.206:3306 |
|
||
| **用户** | oneos_read (只读) | root |
|
||
| **数据库** | lingniu_prod (255 表) | ln_asset_management (114), ln_energy (19), ry-cloud (29) |
|
||
|
||
## ID 策略
|
||
|
||
- `ln_asset_management` / `ln_energy`：`IdType.AUTO`（MySQL 自增），插入时不指定 ID，获取 `LAST_INSERT_ID`
|
||
- `ry-cloud`：雪花算法 ID（已有数据如 sys_user ID = 2038797628680523778）
|
||
|
||
所有迁移记录保存到 `ln_migration.id_mapping` 表，维护 `(old_table, old_id) → (new_table, new_id)` 映射。
|
||
|
||
## 通用字段转换规则
|
||
|
||
| 老字段 | 新字段 | 转换 |
|
||
|---|---|---|
|
||
| `is_deleted` (tinyint 0/1) | `del_flag` (char '0'/'1') | `str(val)` |
|
||
| `creater_id` (bigint) | `create_by` (bigint) | 查 user id_mapping |
|
||
| `updater_id` (bigint) | `update_by` (bigint) | 查 user id_mapping |
|
||
| int 枚举 (如 `truck_type`) | varchar 字典码 | 查 dict_mapping |
|
||
| `datetime` | `date` | 取日期部分 |
|
||
| `double` | `decimal` | `Decimal(str(val))` |
|
||
|
||
---
|
||
|
||
## 迁移分阶段执行
|
||
|
||
### Phase 0: 准备
|
||
|
||
1. 在目标库创建 `ln_migration` 数据库和 `id_mapping` 表
|
||
2. **清空**目标库 3 个库中所有业务表数据（保留 ry-cloud 框架种子数据如 sys_config, sys_oss_config, sys_tenant）
|
||
3. 读取老库 `tab_dic` 构建枚举映射字典 → `dict_mapping.py`
|
||
|
||
### Phase 1: 基础数据（无 FK 依赖）
|
||
|
||
| 老表 | 新表 (库) | 行数 | 说明 |
|
||
|---|---|---|---|
|
||
| `tab_dic` | `sys_dict_type` + `sys_dict_data` (ry-cloud) | 681 | 按 dic_type 分组为 type+data |
|
||
| `tab_region` | `common_district` (asset) | 3,359 | 直接映射 |
|
||
| `tab_vehicle_model` | `vehicle_model` (asset) | 39 | 字段对齐 |
|
||
| `tab_insurance_company` | `insurance_company` (asset) | 20 | 直接映射 |
|
||
| `tab_hydrogen_site` | `hydrogen_station` (asset) | 101 | int→varchar 枚举 |
|
||
| `tab_parking` | `parking_lot_info` (asset) | 65 | 直接映射 |
|
||
| `tab_maintain_site` | `repair_station` (asset) | 64 | 直接映射 |
|
||
| `tab_rescue_site` | `rescue_team` (asset) | ~14 | 直接映射 |
|
||
| `tab_annual_review_service_station` | `inspection_station` (asset) | 14 | 直接映射 |
|
||
| `tab_truck_check_item` | `vehicle_check_item` (asset) | 294 | 直接映射 |
|
||
| `tab_contract_templates` | `contract_template` (asset) | 13 | 直接映射 |
|
||
| `tab_charge_station` | `charging_station` (asset) | ~数条 | 直接映射 |
|
||
|
||
### Phase 2: 系统数据（用户/组织/角色）
|
||
|
||
| 老表 | 新表 (ry-cloud) | 行数 | 说明 |
|
||
|---|---|---|---|
|
||
| `tab_org` | `sys_dept` + `sys_organization` | 19 | 重建 ancestors 字段 |
|
||
| `tab_user` | `sys_user` | 405 | 密码需重置为 BCrypt 临时密码 |
|
||
| `tab_role` | `sys_role` | 69 | 雪花 ID |
|
||
| `tab_user_role` | `sys_user_role` | 201 | 查 user+role id_mapping |
|
||
| `tab_menu` | `sys_menu` | 464 | **评估**：新系统菜单已重新定义，可能跳过 |
|
||
| `tab_role_menu` | `sys_role_menu` | 6,236 | 依赖菜单决策 |
|
||
|
||
> **密码处理**：老系统使用自定义 salt+hash，新系统使用 BCrypt。迁移时统一设置临时密码（如 `Abc@123456` 的 BCrypt 值），首次登录强制修改。
|
||
|
||
### Phase 3: 核心业务实体
|
||
|
||
| 老表 | 新表 (asset) | 行数 | 复杂度 |
|
||
|---|---|---|---|
|
||
| `tab_truck` | `vehicle_info` | 1,203 | 高：47→29 列，大量字段重组 |
|
||
| `tab_truck_status_info` | `vehicle_status` | 1,385 | 中：int→varchar 枚举 |
|
||
| `tab_driver` | `driver_info` | 212 | 低 |
|
||
| `tab_customer` | `customer_info` | 310 | 中 |
|
||
| `tab_truck_licence` | `vehicle_license` | 1,712 | 中：truck_id→vehicle_id FK |
|
||
| `tab_truck_insure` | `insurance_procurement` | 740 | 中：15→32 列扩展 |
|
||
| `tab_equipment_info` | `aftermarket_device` | 1,562 | 中 |
|
||
| `tab_violation_management` | `traffic_violation` | 1,285 | 低 |
|
||
| `tab_accident` + `tab_accident_cost_bearing` | `accident_info` + `accident_expense` | 293+823 | 中 |
|
||
| `tab_failure` | `vehicle_fault_manage` | 5,140 | 低 |
|
||
| `tab_vehicle_annual_inspection` | `vehicle_annual_inspection` | 353 | 低 |
|
||
| `tab_vehicle_preparation` | `prepare_car` | 1,736 | 低 |
|
||
| `tab_customer_invoice` | `invoice_info` | 219 | 低 |
|
||
| `tab_truck_device_info` | `aftermarket_device` 或 `vehicle_realtime_location` | 1,227 | 评估 |
|
||
| `tab_training_materials` | `training_material` (asset) | 3 | 低 |
|
||
|
||
**vehicle_info 字段映射（核心）：**
|
||
```
|
||
tab_truck.plate_number → vehicle_info.plate_number
|
||
tab_truck.vin → vehicle_info.vin
|
||
tab_truck.truck_num → vehicle_info.vehicle_code
|
||
tab_truck.model (int) → vehicle_info.vehicle_model_id (FK, 查 vehicle_model id_mapping)
|
||
tab_truck.color → vehicle_info.body_color
|
||
tab_truck.buy_time → vehicle_info.purchase_date (datetime→date)
|
||
tab_truck.stock_area (int) → vehicle_info.packing_lot_id (FK, 查 parking_lot_info id_mapping)
|
||
tab_truck.mandatory_retirement_period → vehicle_info.mandatory_scrap_date
|
||
tab_truck.remarks → vehicle_info.remark
|
||
tab_truck.address → vehicle_info.province (提取省份)
|
||
```
|
||
|
||
### Phase 4: 合同域
|
||
|
||
| 老表 | 新表 (asset) | 行数 | 说明 |
|
||
|---|---|---|---|
|
||
| `tab_contract` | `vehicle_lease_contract_info` | 681 | 38→67 列，大量新字段 NULL |
|
||
| `tab_contract_rent_order` | `vehicle_lease_order` | 679 | 关联合同 |
|
||
| `tab_contract_rent_truck` | `vehicle_lease_order_detail` | 3,865 | 关联 order + vehicle |
|
||
| `tab_contract_rent_truck_service_cost` | `vehicle_lease_order_service_item` | 1,645 | 关联 detail |
|
||
| `tab_contract_thirty_party` | `contract_authorized_person` | 682 | 直接映射 |
|
||
| `tab_contract_authorizer_information` | `contract_authorized_person` | 666 | 合并 |
|
||
| `tab_contract_costs` | `template_*` 系列表 | 15,059 | 按费用类型拆分 |
|
||
| `tab_contract_hydrogen_fees` | `template_hydrogen_fee` | 13 | 直接映射 |
|
||
|
||
### Phase 5: 交付/退车/换车
|
||
|
||
| 老表 | 新表 (asset) | 行数 | 说明 |
|
||
|---|---|---|---|
|
||
| `tab_truck_rent_task` | `delivery_task_subject` | 3,104 | 任务容器 |
|
||
| `tab_truck_rent_take` | `delivery_order` + `delivery_vehicle` | 1,956 | 一拆多 |
|
||
| `tab_truck_rent_return` | `return_vehicle_task` | 1,079 | 重构 |
|
||
| `tab_truck_rent_return_cost` | `return_fees` | 73 | 直接映射 |
|
||
| `tab_truck_rent_return_dep_cost` | `return_settlement_*` 子表 | 4,634 | 按类型拆分 |
|
||
| `tab_truck_rent_replace` | `vehicle_replacement` | 247 | 直接映射 |
|
||
| `tab_standby_vehicle_main/detail` | `vehicle_abnormal_move` | 377+2,171 | 评估映射 |
|
||
|
||
### Phase 6: 账单/财务
|
||
|
||
| 老表 | 新表 (asset) | 行数 | 说明 |
|
||
|---|---|---|---|
|
||
| `tab_rent_contract_bill` | `bills` | 25,625 | 合同账单 |
|
||
| `tab_rent_contract_bill_truck` | `vehicle_bills` + `vehicle_bill_service_items` | **270,283** | **大表，批量处理** |
|
||
| `tab_rent_contract_bill_other_cost` | 合并到 `vehicle_bills` | ~少量 | |
|
||
| `tab_finance_receivable` | `receivable_subject` + `receivable_vehicle` | 10,308 | 拆分 |
|
||
| `tab_finance_deposit_receive` | `customer_payment_receipt` | 2,078 | 映射 |
|
||
| `tab_finance_deposit_deduction` | `customer_payment_item` | 52 | 映射 |
|
||
|
||
### Phase 7: 能源域 → ln_energy
|
||
|
||
| 老表 | 新表 (ln_energy) | 行数 | 说明 |
|
||
|---|---|---|---|
|
||
| `tab_energy_account` | `energy_account` | 201 | 结构重组 |
|
||
| `tab_energy_project_account` | `energy_account_project` | 307 | 直接映射 |
|
||
| `tab_energy_account_recharge` | `energy_recharge_order` | 942 | 字段扩展 |
|
||
| `tab_import_hydrogen_order` | `hydrogen_station_order` | 58,642 | 加氢原始订单 |
|
||
| `tab_energy_hydrogen_bill` | `energy_hydrogen_detail` | **58,554** | **大表** |
|
||
| `tab_import_ele_charge_order` | `electricity_charge_record` | 4,405 | 充电原始订单 |
|
||
| `tab_energy_electricity_bill` | `energy_bill_detail` | 4,355 | fee_type=electricity |
|
||
|
||
### Phase 8: 附件
|
||
|
||
| 老表 | 新表 (asset) | 行数 | 说明 |
|
||
|---|---|---|---|
|
||
| `tab_data_attachment` | `tab_data_attachment` | **247,447** | **最大表之一**，data_id 需 FK 重映射 |
|
||
| `tab_image_attachment` | 合并到 `tab_data_attachment` 或保留 | 51,516 | 评估 |
|
||
|
||
### Phase 9: 其他
|
||
|
||
| 老表 | 新表 | 行数 | 处理 |
|
||
|---|---|---|---|
|
||
| `tab_maintain_maintenance_project` | 评估 | 243,474 | 新系统无直接对应，可能跳过 |
|
||
| `tab_truck_rent_form_data` | 评估 | 456,061 | 表单数据，新系统结构不同 |
|
||
| `tab_preparation_form_data` | 评估 | 171,497 | 同上 |
|
||
| `tab_standby_vehicle_form_data` | 评估 | 143,034 | 同上 |
|
||
|
||
---
|
||
|
||
## 暂不迁移的表（无新系统对应）
|
||
|
||
- 审批流：`tab_approve_instance*`, `tab_approve_template_node`
|
||
- 工作流：`tab_flow_task*`, `tab_flow_template*`
|
||
- G7 车联网：`tab_g7s_org`, `tab_g7s_truck_mileage` (289K), `tab_g7s_in_out_event` (159K)
|
||
- 培训考试：`tab_train_*`, `tab_safety_training`
|
||
- 系统日志：`tab_api_access_log` (4.6M), `tab_user_log`, `tab_user_message`, `tab_short_message`
|
||
- 调度记录：`tab_schedule_execute_result` (347K), `tab_data_sync_task_record`
|
||
- 应用版本：`tab_app_version`, `tab_version_user_check`, `tab_release_version_log`
|
||
- 临时表：所有 `tab_aa_temp_*`, `*_copy*`, `temp_*`
|
||
- 视图：所有 `view_*`, `v_*`
|
||
- 汇总表：`truck_info`, `truck_equipment_info`, `truck_hydrogen_info`, `truck_mileage`, `truck_parking`
|
||
|
||
---
|
||
|
||
## 实现方案：Python 脚本
|
||
|
||
### 目录结构
|
||
|
||
```
|
||
/Users/kkfluous/Projects/lingniu/oneos-corp/migration/
|
||
config.py # 数据库连接配置
|
||
id_mapping.py # ID 映射表 CRUD
|
||
transform.py # 通用字段转换函数
|
||
dict_mapping.py # 老 int 枚举 → 新 varchar 编码映射
|
||
migrator.py # 基础迁移器（批量读写、mapping、日志）
|
||
|
||
phase0_prepare.py # 建 mapping 表、清空目标库
|
||
phase1_reference.py # 字典、区域、车型等基础数据
|
||
phase2_system.py # 组织、用户、角色
|
||
phase3_core.py # 车辆、司机、客户、证照、保险
|
||
phase4_contract.py # 合同、租赁订单
|
||
phase5_delivery.py # 交付、退车、换车
|
||
phase6_billing.py # 账单、财务
|
||
phase7_energy.py # 能源账户、加氢、充电
|
||
phase8_attachment.py # 附件
|
||
phase9_misc.py # 其他
|
||
|
||
verify.py # 行数校验、FK 完整性、抽样比对
|
||
run_all.py # 按顺序执行所有 phase
|
||
```
|
||
|
||
### 关键技术点
|
||
|
||
1. **批量处理**：大表（27 万+ 行）使用 `SSCursor` 服务端游标 + `executemany` 批量写入（每批 1000 行）
|
||
2. **ID 映射**：`ln_migration.id_mapping(source_table, source_id, target_db, target_table, target_id)`，FK 字段通过查映射表解析
|
||
3. **枚举映射**：从 `tab_dic` 读取所有字典项，预构建 `{(dic_type, int_value): new_dict_code}` 映射
|
||
4. **密码处理**：所有用户密码统一设为 BCrypt(`Abc@123456`)，首次登录强制修改
|
||
5. **事务**：每个 Phase 按表粒度 commit，失败可单表重试
|
||
|
||
## 验证策略
|
||
|
||
1. **行数校验**：每张表迁移后对比源/目标行数
|
||
2. **抽样比对**：每张核心表随机取 10 条，比对关键业务字段
|
||
3. **FK 完整性**：检查所有外键列无孤儿记录
|
||
4. **业务逻辑**：能源账户余额一致、合同-车辆关联完整
|
||
5. **登录测试**：使用临时密码验证 sys_user 登录
|
||
|
||
## 回滚方案
|
||
|
||
1. 迁移前对目标 3 库做 `mysqldump` 备份
|
||
2. 回滚 = truncate 所有目标表 + 从备份恢复 + drop `ln_migration`
|
||
3. 单 Phase 回滚 = 根据 `id_mapping` 删除该 Phase 写入的记录
|
||
|
||
## 关键文件
|
||
|
||
- `ln-asset-management/.../BaseEntity.java` — IdType.AUTO, del_flag char(1)
|
||
- `ln-asset-management/.../VehicleInfo.java` — 新车辆实体 29 字段
|
||
- `ln-cloud/ruoyi-common/.../BaseEntity.java` — RuoYi 基础实体
|
||
- 老后端代码 `/Users/kkfluous/Projects/lingniu/ln_asset/lingniu_asset_server/lingniu-manager/src/main/java/org/lingniu/manager/model/` — 所有老实体类
|