shishengliang/lingniu-vehicle-ingest
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor: rename kafka sink module

Browse Source
This commit is contained in:
lingniu
2026-07-01 10:56:33 +08:00
parent 849e8f8a8b
commit e80fa6f38c
43 changed files with 87 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
docs/superpowers/plans/2026-06-29-vehicle-ingest-redesign-phase1.md
View File

@@ -16,7 +16,7 @@ This plan implements only the first slice of the redesign:
- `modules/core/ingest-facts`: protocol-neutral fact records and helpers.
- `modules/sinks/raw-archive-store`: raw archive write/read contract and local URI-safe implementation skeleton.
- `modules/sinks/sink-mq`: protobuf messages for raw frame facts and decoded facts.
- `modules/sinks/sink-kafka`: protobuf messages for raw frame facts and decoded facts.
- Parent Maven wiring and focused tests.
It intentionally does not yet modify:
@@ -53,9 +53,9 @@ Create:
Modify:
- `pom.xml`: add modules and dependency management entries.
- `modules/sinks/sink-mq/pom.xml`: add dependency on `ingest-facts`.
- `modules/sinks/sink-mq/src/main/proto/vehicle_envelope.proto`: add `RawFrameFactPayload`, `DecodedFactPayload`, and related enums/messages.
- `modules/sinks/sink-mq/src/test/java/com/lingniu/ingest/sink/mq/VehicleEnvelopeProtoCompatibilityTest.java`: verify new proto messages can round-trip.
- `modules/sinks/sink-kafka/pom.xml`: add dependency on `ingest-facts`.
- `modules/sinks/sink-kafka/src/main/proto/vehicle_envelope.proto`: add `RawFrameFactPayload`, `DecodedFactPayload`, and related enums/messages.
- `modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/VehicleEnvelopeProtoCompatibilityTest.java`: verify new proto messages can round-trip.
---
@@ -1260,9 +1260,9 @@ git commit -m "feat: add raw archive store contract"
**Files:**
- Modify: `modules/sinks/sink-mq/pom.xml`
- Modify: `modules/sinks/sink-mq/src/main/proto/vehicle_envelope.proto`
- Create: `modules/sinks/sink-mq/src/test/java/com/lingniu/ingest/sink/mq/VehicleEnvelopeProtoCompatibilityTest.java`
- Modify: `modules/sinks/sink-kafka/pom.xml`
- Modify: `modules/sinks/sink-kafka/src/main/proto/vehicle_envelope.proto`
- Create: `modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/VehicleEnvelopeProtoCompatibilityTest.java`
- [ ] **Step 1: Write failing proto compatibility test**
@@ -1341,14 +1341,14 @@ class VehicleEnvelopeProtoCompatibilityTest {
Run:
```bash
mvn -pl modules/sinks/sink-mq -Dtest=VehicleEnvelopeProtoCompatibilityTest test
mvn -pl modules/sinks/sink-kafka -Dtest=VehicleEnvelopeProtoCompatibilityTest test
```
Expected: FAIL because proto generated classes do not exist.
- [ ] **Step 3: Add `ingest-facts` dependency to sink-mq**
- [ ] **Step 3: Add `ingest-facts` dependency to sink-kafka**
Add this dependency to `modules/sinks/sink-mq/pom.xml` after `ingest-api`:
Add this dependency to `modules/sinks/sink-kafka/pom.xml` after `ingest-api`:
```xml
<dependency>
@@ -1412,17 +1412,17 @@ message DecodedFactPayload {
Run:
```bash
mvn -pl modules/sinks/sink-mq -Dtest=VehicleEnvelopeProtoCompatibilityTest test
mvn -pl modules/sinks/sink-kafka -Dtest=VehicleEnvelopeProtoCompatibilityTest test
```
Expected: PASS.
- [ ] **Step 6: Run existing sink-mq tests**
- [ ] **Step 6: Run existing sink-kafka tests**
Run:
```bash
mvn -pl modules/sinks/sink-mq test
mvn -pl modules/sinks/sink-kafka test
```
Expected: PASS. Existing envelope tests must continue to pass because new proto fields are additive.
@@ -1430,9 +1430,9 @@ Expected: PASS. Existing envelope tests must continue to pass because new proto
- [ ] **Step 7: Commit**
```bash
git add modules/sinks/sink-mq/pom.xml \
modules/sinks/sink-mq/src/main/proto/vehicle_envelope.proto \
modules/sinks/sink-mq/src/test/java/com/lingniu/ingest/sink/mq/VehicleEnvelopeProtoCompatibilityTest.java
git add modules/sinks/sink-kafka/pom.xml \
modules/sinks/sink-kafka/src/main/proto/vehicle_envelope.proto \
modules/sinks/sink-kafka/src/test/java/com/lingniu/ingest/sink/kafka/VehicleEnvelopeProtoCompatibilityTest.java
git commit -m "feat: add fact payloads to kafka envelope"
```
@@ -1449,7 +1449,7 @@ git commit -m "feat: add fact payloads to kafka envelope"
Run:
```bash
mvn -pl modules/core/ingest-facts,modules/sinks/raw-archive-store,modules/sinks/sink-mq -am test
mvn -pl modules/core/ingest-facts,modules/sinks/raw-archive-store,modules/sinks/sink-kafka -am test
```
Expected: PASS.
@@ -1459,7 +1459,7 @@ Expected: PASS.
Run:
```bash
mvn -pl modules/sinks/sink-mq -am -DskipTests package
mvn -pl modules/sinks/sink-kafka -am -DskipTests package
```
Expected: PASS. Generated protobuf Java sources include `RawFrameFactPayload`, `DecodedFactPayload`, and `ParseStatusProto`.
@@ -1473,7 +1473,7 @@ rg -n "RawFrameFact|DecodedFact|RawArchiveWriter|vehicle.raw-frame.v1|vehicle.de
modules/apps modules/protocols modules/services modules/core modules/sinks
```
Expected: Matches only in the new modules, sink-mq proto/test, and planned references. No existing handler behavior should be changed in Phase 1.
Expected: Matches only in the new modules, sink-kafka proto/test, and planned references. No existing handler behavior should be changed in Phase 1.
- [ ] **Step 4: Commit final verification note if any docs changed**