# Target Architecture

This document records the production target for `lingniu-vehicle-ingest`.
The project should stay small at runtime: protocol apps ingest and publish,
history consumes and indexes, analytics derives metrics, and business systems
read through explicit APIs or Kafka.

## Active Scope

Active production protocols:

| Protocol | Runtime app | Event topic | Raw topic |
|---|---|---|---|
| GB/T 32960 | `gb32960-ingest-app` | `vehicle.event.gb32960.v1` | `vehicle.raw.gb32960.v1` |
| JT/T 808 | `jt808-ingest-app` | `vehicle.event.jt808.v1` | `vehicle.raw.jt808.v1` |
| Yutong MQTT | `yutong-mqtt-app` | `vehicle.event.mqtt-yutong.v1` | `vehicle.raw.mqtt-yutong.v1` |

Xinda Push is legacy and is not part of the default production surface.
It may remain in source for reference, but new optimization work should target
the active protocols above.

## Goals

- Keep protocol IO, raw archive, history, latest-state, and statistics as
  separate responsibilities.
- Publish normalized Kafka envelopes with VIN as the partition key whenever a
  VIN is known.
- Store complete raw payload metadata and parsed JSON in TDengine `raw_frames`.
- Store query-friendly location rows separately from raw payload JSON.
- Keep derived statistics in metric tables, not protocol-specific daily tables.
- Use Redis only for hot state or streaming calculation state that can be
  rebuilt from Kafka.
- Keep optional compatibility modules outside the production hot path.

## Non-Goals

- Ingest apps do not write business tables.
- Ingest apps do not serve history query APIs.
- Redis is not the long-term historical store.
- `event-file-store` Parquet/DuckDB is not the default history store.
- `telemetry_fields` parsing is not handled by this project when a separate
  field parsing service owns that job.

## Runtime Responsibilities

### Protocol Apps

Apps:

- `gb32960-ingest-app`
- `jt808-ingest-app`
- `yutong-mqtt-app`

Responsibilities:

- Accept protocol traffic.
- Decode, authenticate, acknowledge, and maintain sessions where the protocol
  requires it.
- Archive raw bytes through `sink-archive` when bytes are available.
- Publish parsed event envelopes to Kafka.
- Publish raw envelopes to Kafka.
- Resolve vehicle identity through the shared MySQL identity binding table.

Protocol apps must stay independent from TDengine history readers, Redis state
repositories, and statistic calculators.

### Event Contract

`sink-mq` owns the Kafka protobuf envelope and consumer plumbing.
Consumers should use `EnvelopeConsumerProcessor` so invalid protobuf, skipped
envelopes, and downstream failures are converted to structured results and DLQ
records instead of blocking a partition.

Core identity fields:

| Field | Meaning |
|---|---|
| `event_id` | Parsed event idempotency key |
| `trace_id` | Cross-service trace id |
| `vin` | Vehicle id and Kafka partition key |
| `source` | Source protocol, for example `GB32960`, `JT808`, `YUTONG_MQTT` |
| `event_time_ms` | Device event time |
| `ingest_time_ms` | Platform receive time |
| `raw_uri` | `archive://...` reference to raw bytes when available |

Full-field telemetry uses `TelemetrySnapshot` fields with stable keys such as
`total_mileage_km`, `speed_kmh`, `longitude`, and `latitude`.

### History App

App: `vehicle-history-app`

Responsibilities:

- Consume active protocol event topics.
- Consume active protocol raw topics.
- Write raw frame rows into TDengine `raw_frames`.
- Write compact location rows into TDengine location tables.
- Keep `payloadJson.parsed` on raw rows for full raw inspection.
- Expose paged history APIs for raw frames and locations.
- Keep specialized APIs off by default unless explicitly enabled.

Production default:

- `TDENGINE_HISTORY_ENABLED=true` in deployment.
- `EVENT_FILE_STORE_ENABLED=false`.
- `TDENGINE_TELEMETRY_FIELDS_ENABLED=false`.

`event-file-store` remains an optional compatibility path for old low-level
record APIs or local investigations. It should not be treated as the primary
history backend.

### Analytics App

App: `vehicle-analytics-app`

Responsibilities:

- Consume JT808 event envelopes.
- Calculate 808 daily mileage from the reported GPS total mileage only:

```text
daily_mileage_km = last_total_mileage_km - first_total_mileage_km
calculation_method = JT808_TOTAL_MILEAGE_DIFF
```

- Store the metric in `vehicle_stat_metric` with
  `metric_key = daily_mileage_km`; the first and latest GPS total mileage used
  for the subtraction stay on the same metric row as calculation source columns.

There is no JT808-specific daily mileage table and no in-memory production
state-store mode. Restart recovery reads the same metric row, so it does not
require a separate Redis mileage state.

### Latest State

Module: `vehicle-state-service`

Responsibilities:

- Consume normalized Kafka envelopes.
- Maintain latest state, location, and safety snapshots in Redis.
- Serve latest-state APIs for operational screens.

Redis state must be rebuildable from Kafka replay and should not be used as the
source of truth for historical queries.

## Data Flow

```mermaid
flowchart LR
  vehicle["Vehicles / Platforms"] --> ingest["Protocol apps"]
  ingest --> archive["sink-archive<br/>raw bytes"]
  ingest --> kafka["Kafka<br/>event + raw envelopes"]
  kafka --> history["vehicle-history-app"]
  kafka --> analytics["vehicle-analytics-app"]
  kafka --> state["vehicle-state-service"]
  history --> tdengine["TDengine<br/>raw_frames + locations"]
  analytics --> mysql["MySQL<br/>vehicle_stat_metric"]
  state --> redisState["Redis<br/>latest state"]
```

## Failure Handling

- Protocol apps continue accepting traffic when history, analytics, or Redis
  consumers are unavailable.
- Kafka producer failures are handled by sink retry/circuit-breaker behavior and
  DLQ topics where configured.
- History consumers should commit offsets only after TDengine writes succeed.
- Analytics daily mileage state can recover from `vehicle_stat_metric` or Kafka
  replay.
- Raw archive failures must be observable, but event publication and history
  indexing remain separate concerns.

## Acceptance Criteria

- Active production deployment contains GB32960, JT808, Yutong MQTT,
  vehicle-history, and vehicle-analytics apps.
- Xinda Push is absent from default deployment and history consumer bindings.
- History APIs read from TDengine by default; Parquet/DuckDB is optional
  compatibility only.
- Raw frame queries can return complete parsed JSON through `payloadJson.parsed`.
- Location queries page over compact rows and reference raw records instead of
  duplicating full raw JSON.
- JT808 daily mileage is stored only in `vehicle_stat_metric`.
- Runtime state needed for 808 mileage is also in `vehicle_stat_metric`, with no
  Redis or production memory state-store mode.
- Build validation covers the active modules and their composition tests.