# Target Architecture

This document records the production target for `lingniu-vehicle-ingest`.
The project should stay small at runtime: protocol apps ingest and publish,
history consumes and indexes, analytics derives metrics, and business systems
read through explicit APIs or Kafka.

Only supported message backbone: Kafka.

## Active Scope

Active production protocols:

| Protocol | Runtime app | Event topic | Raw topic |
|---|---|---|---|
| GB/T 32960 | `gb32960-ingest-app` | `vehicle.event.gb32960.v1` | `vehicle.raw.gb32960.v1` |
| JT/T 808 | `jt808-ingest-app` | `vehicle.event.jt808.v1` | `vehicle.raw.jt808.v1` |
| Yutong MQTT | `yutong-mqtt-app` | `vehicle.event.mqtt-yutong.v1` | `vehicle.raw.mqtt-yutong.v1` |

Xinda Push is legacy and is not part of the default production surface.
It may remain in source for reference, but new optimization work should target
the active protocols above.

JSATL12 attachment upload is also outside the default production reactor. Keep
it available through the `optional-attachments` profile until an attachment app
is explicitly deployed.

## Goals

- Keep protocol IO, raw archive, history, latest-state, and statistics as
  separate responsibilities.
- Publish normalized Kafka envelopes with VIN as the partition key whenever a
  VIN is known.
- Store complete raw payload metadata and parsed JSON in TDengine `raw_frames`.
- Store query-friendly location rows separately from raw payload JSON.
- Keep derived statistics in metric tables, not protocol-specific daily tables.
- Use Redis only for optional latest-state APIs that can be rebuilt from Kafka.
- Keep optional compatibility modules outside the production hot path.

## Non-Goals

- Ingest apps do not write business tables.
- Ingest apps do not serve history query APIs.
- Redis is not the long-term historical store.
- File-based event indexes are not part of the current build surface.
- `telemetry_fields` parsing and field-trend APIs are owned by a separate
  field parsing service. This project stores RAW parsed JSON plus compact
  location rows and does not expose a telemetry-fields history API.

## Runtime Responsibilities

### Protocol Apps

Apps:

- `gb32960-ingest-app`
- `jt808-ingest-app`
- `yutong-mqtt-app`

Responsibilities:

- Accept protocol traffic.
- Decode, authenticate, acknowledge, and maintain sessions where the protocol
  requires it.
- Archive raw bytes through `sink-archive` when bytes are available.
- Publish parsed event envelopes to Kafka.
- Publish raw envelopes to Kafka.
- Resolve vehicle identity through the shared MySQL identity binding table.

Protocol apps must stay independent from TDengine history readers, Redis state
repositories, and statistic calculators.

### Event Contract

The Kafka sink owns the protobuf envelope and consumer plumbing.
Consumers should use `EnvelopeConsumerProcessor` so invalid protobuf, skipped
envelopes, and downstream failures are converted to structured results and DLQ
records instead of blocking a partition.

Core identity fields:

| Field | Meaning |
|---|---|
| `event_id` | Parsed event idempotency key |
| `trace_id` | Cross-service trace id |
| `vin` | Vehicle id and Kafka partition key |
| `source` | Source protocol, for example `GB32960`, `JT808`, `YUTONG_MQTT` |
| `event_time_ms` | Device event time |
| `ingest_time_ms` | Platform receive time |
| `raw_uri` | `archive://...` reference to raw bytes when available |

Full-field telemetry uses `TelemetrySnapshot` fields with stable keys such as
`total_mileage_km`, `speed_kmh`, `longitude`, and `latitude`.

### History App

App: `vehicle-history-app`

Responsibilities:

- Consume active protocol event topics.
- Consume active protocol raw topics.
- Write raw frame rows into TDengine `raw_frames`.
- Write compact location rows into TDengine location tables.
- Keep `payloadJson.parsed` on raw rows for full raw inspection.
- Expose paged history APIs for raw frames and locations.
- Keep specialized APIs off by default unless explicitly enabled.

Production default:

- `TDENGINE_HISTORY_ENABLED=true` in deployment.
- `EVENT_FILE_STORE_ENABLED=false`.

File-based event indexes have been removed; this app should keep history
queries on TDengine and raw replay on `archive://...` references.

### Analytics App

App: `vehicle-analytics-app`

Responsibilities:

- Consume JT808 event envelopes.
- Calculate 808 daily mileage from the reported GPS total mileage only:

```text
daily_mileage_km = max_total_mileage_km - min_total_mileage_km
calculation_method = JT808_TOTAL_MILEAGE_DIFF
```

- Store the metric in `vehicle_stat_metric` with
  `metric_key = daily_mileage_km`; the local-day minimum and maximum GPS total
  mileage used for the subtraction stay on the same metric row as calculation
  source columns.

Runtime state: none outside `vehicle_stat_metric`. There is no JT808-specific
daily mileage table. Restart recovery reads the same metric row.

### Latest State

Module: `vehicle-state-service`

This module is outside the default production reactor. Build it with
`-Poptional-latest-state` when Redis latest-state APIs are explicitly needed.

Responsibilities:

- Consume normalized Kafka envelopes.
- Maintain latest state, location, and safety snapshots in Redis.
- Serve latest-state APIs for operational screens.

Redis state must be rebuildable from Kafka replay and should not be used as the
source of truth for historical queries.

## Data Flow

Default production flow:

```mermaid
flowchart LR
  vehicle["Vehicles / Platforms"] --> ingest["Protocol apps"]
  ingest --> archive["sink-archive<br/>raw bytes"]
  ingest --> kafka["Kafka<br/>event + raw envelopes"]
  kafka --> history["vehicle-history-app"]
  kafka --> analytics["vehicle-analytics-app"]
  history --> tdengine["TDengine<br/>raw_frames + locations"]
  analytics --> mysql["MySQL<br/>vehicle_stat_metric"]
```

Optional latest-state flow, enabled only with `-Poptional-latest-state`, consumes
the same Kafka envelopes and rebuilds Redis snapshots outside the default
production deployment.

## Failure Handling

- Protocol apps continue accepting traffic when history, analytics, or Redis
  consumers are unavailable.
- Kafka producer failures are handled by sink retry/circuit-breaker behavior and
  DLQ topics where configured.
- History consumers should commit offsets only after TDengine writes succeed.
- Analytics daily mileage state can recover from `vehicle_stat_metric` or Kafka
  replay.
- Raw archive failures must be observable, but event publication and history
  indexing remain separate concerns.

## Acceptance Criteria

- Active production deployment contains GB32960, JT808, Yutong MQTT,
  vehicle-history, and vehicle-analytics apps.
- Xinda Push is absent from default deployment and history consumer bindings.
- JSATL12 is absent from the default Maven reactor and available through
  `optional-attachments`.
- vehicle-state-service is absent from the default Maven reactor and available
  through `optional-latest-state`.
- raw-archive-store prototype has been removed; raw bytes are written through
  `sink-archive`.
- History APIs read from TDengine; file-based event indexes are not maintained.
- Raw frame queries can return complete parsed JSON through `payloadJson.parsed`.
- Location queries page over compact rows and reference raw records instead of
  duplicating full raw JSON.
- JT808 daily mileage is stored only in `vehicle_stat_metric`.
- Runtime state needed for 808 mileage is also in `vehicle_stat_metric`.
- Build validation covers the active modules and their composition tests.