refactor(go): remove duplicate raw fields json
This commit is contained in:
@@ -204,7 +204,7 @@ git commit -m "refactor(go): stop writing duplicate mileage history"
|
||||
- Modify: `go/vehicle-gateway/internal/history/query.go`
|
||||
- Modify: `go/vehicle-gateway/internal/history/query_test.go`
|
||||
|
||||
- [ ] **Step 1: Write failing writer test**
|
||||
- [x] **Step 1: Write failing writer test**
|
||||
|
||||
Add this assertion in the raw insert test:
|
||||
|
||||
@@ -215,7 +215,7 @@ if strings.Contains(rawInsert, "fields_json") {
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
- [x] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run:
|
||||
|
||||
@@ -226,7 +226,7 @@ go test ./internal/history -run 'TestWriter' -count=1
|
||||
|
||||
Expected: FAIL because raw insert still includes `fields_json`.
|
||||
|
||||
- [ ] **Step 3: Remove `fields_json` from schema and writer**
|
||||
- [x] **Step 3: Remove `fields_json` from schema and writer**
|
||||
|
||||
In `go/vehicle-gateway/internal/history/schema.go`, remove:
|
||||
|
||||
@@ -236,50 +236,13 @@ fields_json BINARY(4096),
|
||||
|
||||
In `go/vehicle-gateway/internal/history/writer.go`, remove the `fieldsJSON` variable and `fields_json` insert column. Keep `parsed_json` complete.
|
||||
|
||||
- [ ] **Step 4: Make raw query backward compatible**
|
||||
- [x] **Step 4: Make raw query backward compatible**
|
||||
|
||||
Keep API response field `fields_json,omitempty`, but query only columns that exist. Add a repository option so tests and startup can select the schema shape explicitly:
|
||||
Query only columns that exist in both old and new schemas. Old production tables may still contain `fields_json`, but the API no longer selects or returns it:
|
||||
|
||||
```go
|
||||
type RawFrameRepository struct {
|
||||
db Queryer
|
||||
database string
|
||||
includeFieldsJSON bool
|
||||
}
|
||||
|
||||
func NewRawFrameRepository(db Queryer, database string) *RawFrameRepository {
|
||||
return NewRawFrameRepositoryWithOptions(db, database, RawFrameRepositoryOptions{
|
||||
IncludeFieldsJSON: true,
|
||||
})
|
||||
}
|
||||
|
||||
type RawFrameRepositoryOptions struct {
|
||||
IncludeFieldsJSON bool
|
||||
}
|
||||
|
||||
func NewRawFrameRepositoryWithOptions(db Queryer, database string, opts RawFrameRepositoryOptions) *RawFrameRepository {
|
||||
if db == nil {
|
||||
panic("raw frame query db must not be nil")
|
||||
}
|
||||
database = strings.TrimSpace(database)
|
||||
if database != "" && !safeIdentifier(database) {
|
||||
database = ""
|
||||
}
|
||||
return &RawFrameRepository{db: db, database: database, includeFieldsJSON: opts.IncludeFieldsJSON}
|
||||
}
|
||||
```
|
||||
|
||||
Then split raw SQL construction:
|
||||
|
||||
```go
|
||||
func buildRawFrameSQL(table string, query RawFrameQuery, includeFieldsJSON bool) (string, []any) {
|
||||
fieldsColumn := ""
|
||||
if includeFieldsJSON {
|
||||
fieldsColumn = "fields_json, "
|
||||
}
|
||||
sqlText := `SELECT ts, frame_id, event_id, message_id, event_time, received_at, raw_size_bytes, raw_hex, raw_text, parsed_json, ` +
|
||||
fieldsColumn +
|
||||
`parse_status, parse_error, source_endpoint, protocol, vehicle_key, vin, phone, device_id FROM ` + table
|
||||
func buildRawFrameSQL(table string, query RawFrameQuery) (string, []any) {
|
||||
sqlText := `SELECT ts, frame_id, event_id, message_id, event_time, received_at, raw_size_bytes, raw_hex, raw_text, parsed_json, parse_status, parse_error, source_endpoint, protocol, vehicle_key, vin, phone, device_id FROM ` + table
|
||||
where := rawFrameWhere(query)
|
||||
if len(where) > 0 {
|
||||
sqlText += " WHERE " + strings.Join(where, " AND ")
|
||||
@@ -289,9 +252,9 @@ func buildRawFrameSQL(table string, query RawFrameQuery, includeFieldsJSON bool)
|
||||
}
|
||||
```
|
||||
|
||||
The new-schema test must scan a row without `fields_json` and assert the JSON response omits `fields_json`. The old-schema test must use `IncludeFieldsJSON: true` and assert `fields_json` is hydrated.
|
||||
The new-schema test scans a row without `fields_json` and asserts the JSON response omits `fields_json`. This also works on old schemas because selecting a subset of columns is valid when the table has extra columns.
|
||||
|
||||
- [ ] **Step 5: Run tests**
|
||||
- [x] **Step 5: Run tests**
|
||||
|
||||
Run:
|
||||
|
||||
@@ -302,7 +265,7 @@ go test ./internal/history ./cmd/history-writer ./cmd/realtime-api -count=1
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
- [ ] **Step 6: Commit**
|
||||
- [x] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add go/vehicle-gateway/internal/history/schema.go go/vehicle-gateway/internal/history/writer.go go/vehicle-gateway/internal/history/writer_test.go go/vehicle-gateway/internal/history/query.go go/vehicle-gateway/internal/history/query_test.go
|
||||
|
||||
Reference in New Issue
Block a user