refactor: trim unused stateful runtime settings
This commit is contained in:
@@ -6,13 +6,13 @@
|
|||||||
- `jt808-ingest-app`: TCP `808`,HTTP `20482`
|
- `jt808-ingest-app`: TCP `808`,HTTP `20482`
|
||||||
- `vehicle-history-app`: HTTP `20200`
|
- `vehicle-history-app`: HTTP `20200`
|
||||||
|
|
||||||
三个服务必须使用同一个 `SINK_ARCHIVE_PATH`。协议接入服务负责落原始 `.bin` 文件,Kafka raw envelope 只携带 `archive://...` 引用;history 服务按同一个 archive root 读取原始帧并提供查询。
|
`vehicle-history-app` 的热历史查询只依赖 Kafka 与 TDengine:RAW 帧的 `parsedJson`、`metadataJson`、`rawUri` 已进入 TDengine `raw_frames`。GB32960/JT808 接入服务可以继续保留 `SINK_ARCHIVE_PATH` 作为原始 `.bin` 冷备,history 本机模板不再要求共享 archive root。
|
||||||
|
|
||||||
`vehicle-history-app` 的生产高吞吐模板是 TDengine-only:
|
`vehicle-history-app` 的生产高吞吐模板是 TDengine-only:
|
||||||
|
|
||||||
- 不暴露 DuckDB/EventFileStore 启动变量,避免文件型明细库进入实时消费热路径。
|
- 不暴露 DuckDB/EventFileStore 启动变量,避免文件型明细库进入实时消费热路径。
|
||||||
- `TDENGINE_TELEMETRY_FIELDS_ENABLED=false`:默认只写 `raw_frames` 和协议位置表,避免逐字段写放大。
|
- `TDENGINE_TELEMETRY_FIELDS_ENABLED=false`:默认只写 `raw_frames` 和协议位置表,避免逐字段写放大。
|
||||||
- GB32960 snapshot/fields 接口默认通过 TDengine `raw_frames` 索引找到 `rawUri`,再回读共享 archive 中的原始 `.bin` 即时解析;后续解析器新增字段后可直接查询历史 raw。
|
- 通用 RAW 查询默认直接返回 TDengine `raw_frames` 中的 `parsedJson`/`parsedFields`;后续解析器新增字段后,优先通过 replay 或专用解析任务补写 TDengine,而不是让 history 服务依赖本地 archive 挂载。
|
||||||
- 如果需要兼容旧 `/api/event-history/records` 低层记录接口或逐字段趋势宽表,可以显式打开对应开关;开启后要重新压测 Kafka lag 和 TDengine 写入吞吐。
|
- 如果需要兼容旧 `/api/event-history/records` 低层记录接口或逐字段趋势宽表,可以显式打开对应开关;开启后要重新压测 Kafka lag 和 TDengine 写入吞吐。
|
||||||
|
|
||||||
## 生成 plist
|
## 生成 plist
|
||||||
@@ -22,7 +22,7 @@
|
|||||||
```bash
|
```bash
|
||||||
export PROJECT_ROOT="$(pwd)"
|
export PROJECT_ROOT="$(pwd)"
|
||||||
export JAVA_BIN="$(/usr/libexec/java_home 2>/dev/null)/bin/java"
|
export JAVA_BIN="$(/usr/libexec/java_home 2>/dev/null)/bin/java"
|
||||||
export SHARED_ARCHIVE_PATH="$PROJECT_ROOT/data/archive-live"
|
export INGEST_ARCHIVE_PATH="$PROJECT_ROOT/data/archive-live"
|
||||||
export TDENGINE_JDBC_URL='jdbc:TAOS-WS://<tdengine-host>:6041/vehicle_ts'
|
export TDENGINE_JDBC_URL='jdbc:TAOS-WS://<tdengine-host>:6041/vehicle_ts'
|
||||||
export TDENGINE_USERNAME='root'
|
export TDENGINE_USERNAME='root'
|
||||||
export TDENGINE_PASSWORD='<tdengine-password>'
|
export TDENGINE_PASSWORD='<tdengine-password>'
|
||||||
@@ -38,7 +38,7 @@ for service in gb32960 jt808 vehicle-history; do
|
|||||||
sed \
|
sed \
|
||||||
-e "s#__PROJECT_ROOT__#$PROJECT_ROOT#g" \
|
-e "s#__PROJECT_ROOT__#$PROJECT_ROOT#g" \
|
||||||
-e "s#__JAVA_BIN__#$JAVA_BIN#g" \
|
-e "s#__JAVA_BIN__#$JAVA_BIN#g" \
|
||||||
-e "s#__SHARED_ARCHIVE_PATH__#$SHARED_ARCHIVE_PATH#g" \
|
-e "s#__SHARED_ARCHIVE_PATH__#$INGEST_ARCHIVE_PATH#g" \
|
||||||
-e "s#__TDENGINE_JDBC_URL__#$TDENGINE_JDBC_URL#g" \
|
-e "s#__TDENGINE_JDBC_URL__#$TDENGINE_JDBC_URL#g" \
|
||||||
-e "s#__TDENGINE_USERNAME__#$TDENGINE_USERNAME#g" \
|
-e "s#__TDENGINE_USERNAME__#$TDENGINE_USERNAME#g" \
|
||||||
-e "s#__TDENGINE_PASSWORD__#$TDENGINE_PASSWORD#g" \
|
-e "s#__TDENGINE_PASSWORD__#$TDENGINE_PASSWORD#g" \
|
||||||
@@ -51,7 +51,7 @@ for service in gb32960 jt808 vehicle-history; do
|
|||||||
done
|
done
|
||||||
```
|
```
|
||||||
|
|
||||||
如果生产配置由 Nacos 下发,可以把模板里的 `NACOS_CONFIG_ENABLED` 改成 `true`,并补充 `NACOS_SERVER_ADDR`、`NACOS_NAMESPACE`、`NACOS_GROUP`、`NACOS_USERNAME`、`NACOS_PASSWORD`。端口、Kafka topic、archive root、TDengine 连接建议仍保留为启动环境变量,便于 Portainer、launchctl 和临时压测保持一致。
|
如果生产配置由 Nacos 下发,可以把模板里的 `NACOS_CONFIG_ENABLED` 改成 `true`,并补充 `NACOS_SERVER_ADDR`、`NACOS_NAMESPACE`、`NACOS_GROUP`、`NACOS_USERNAME`、`NACOS_PASSWORD`。端口、Kafka topic、接入服务冷备 archive 路径、TDengine 连接建议仍保留为启动环境变量,便于 Portainer、launchctl 和临时压测保持一致。
|
||||||
|
|
||||||
history 消费者会按协议和 raw/event 自动拆分 consumer group。设置 `KAFKA_GROUP_HISTORY=vehicle-history-live` 后,实际 group 为:
|
history 消费者会按协议和 raw/event 自动拆分 consumer group。设置 `KAFKA_GROUP_HISTORY=vehicle-history-live` 后,实际 group 为:
|
||||||
|
|
||||||
|
|||||||
@@ -38,8 +38,6 @@
|
|||||||
<string>vehicle.event.jt808.v1</string>
|
<string>vehicle.event.jt808.v1</string>
|
||||||
<key>KAFKA_TOPIC_JT808_RAW</key>
|
<key>KAFKA_TOPIC_JT808_RAW</key>
|
||||||
<string>vehicle.raw.jt808.v1</string>
|
<string>vehicle.raw.jt808.v1</string>
|
||||||
<key>SINK_ARCHIVE_PATH</key>
|
|
||||||
<string>__SHARED_ARCHIVE_PATH__</string>
|
|
||||||
<key>TDENGINE_HISTORY_ENABLED</key>
|
<key>TDENGINE_HISTORY_ENABLED</key>
|
||||||
<string>true</string>
|
<string>true</string>
|
||||||
<key>TDENGINE_HISTORY_DATABASE</key>
|
<key>TDENGINE_HISTORY_DATABASE</key>
|
||||||
|
|||||||
@@ -29,11 +29,6 @@ x-common-env: &common-env
|
|||||||
MYSQL_JDBC_URL: ${MYSQL_JDBC_URL:-}
|
MYSQL_JDBC_URL: ${MYSQL_JDBC_URL:-}
|
||||||
MYSQL_USERNAME: ${MYSQL_USERNAME:-}
|
MYSQL_USERNAME: ${MYSQL_USERNAME:-}
|
||||||
MYSQL_PASSWORD: ${MYSQL_PASSWORD:-}
|
MYSQL_PASSWORD: ${MYSQL_PASSWORD:-}
|
||||||
REDIS_HOST: ${REDIS_HOST:-r-bp1u741kij7e51i481pd.redis.rds.aliyuncs.com}
|
|
||||||
REDIS_PORT: ${REDIS_PORT:-6379}
|
|
||||||
REDIS_DATABASE: ${REDIS_DATABASE:-50}
|
|
||||||
REDIS_USERNAME: ${REDIS_USERNAME:-}
|
|
||||||
REDIS_PASSWORD: ${REDIS_PASSWORD:-}
|
|
||||||
|
|
||||||
services:
|
services:
|
||||||
gb32960-ingest-app:
|
gb32960-ingest-app:
|
||||||
@@ -56,6 +51,11 @@ services:
|
|||||||
VEHICLE_IDENTITY_MYSQL_JDBC_URL: ${MYSQL_JDBC_URL:-}
|
VEHICLE_IDENTITY_MYSQL_JDBC_URL: ${MYSQL_JDBC_URL:-}
|
||||||
VEHICLE_IDENTITY_MYSQL_USERNAME: ${MYSQL_USERNAME:-}
|
VEHICLE_IDENTITY_MYSQL_USERNAME: ${MYSQL_USERNAME:-}
|
||||||
VEHICLE_IDENTITY_MYSQL_PASSWORD: ${MYSQL_PASSWORD:-}
|
VEHICLE_IDENTITY_MYSQL_PASSWORD: ${MYSQL_PASSWORD:-}
|
||||||
|
REDIS_HOST: ${REDIS_HOST:-r-bp1u741kij7e51i481pd.redis.rds.aliyuncs.com}
|
||||||
|
REDIS_PORT: ${REDIS_PORT:-6379}
|
||||||
|
REDIS_DATABASE: ${REDIS_DATABASE:-50}
|
||||||
|
REDIS_USERNAME: ${REDIS_USERNAME:-}
|
||||||
|
REDIS_PASSWORD: ${REDIS_PASSWORD:-}
|
||||||
ports:
|
ports:
|
||||||
- "${GB32960_HTTP_PORT:-20100}:20100"
|
- "${GB32960_HTTP_PORT:-20100}:20100"
|
||||||
- "${GB32960_TCP_PORT:-32960}:32960"
|
- "${GB32960_TCP_PORT:-32960}:32960"
|
||||||
@@ -79,6 +79,11 @@ services:
|
|||||||
VEHICLE_IDENTITY_MYSQL_JDBC_URL: ${MYSQL_JDBC_URL:-}
|
VEHICLE_IDENTITY_MYSQL_JDBC_URL: ${MYSQL_JDBC_URL:-}
|
||||||
VEHICLE_IDENTITY_MYSQL_USERNAME: ${MYSQL_USERNAME:-}
|
VEHICLE_IDENTITY_MYSQL_USERNAME: ${MYSQL_USERNAME:-}
|
||||||
VEHICLE_IDENTITY_MYSQL_PASSWORD: ${MYSQL_PASSWORD:-}
|
VEHICLE_IDENTITY_MYSQL_PASSWORD: ${MYSQL_PASSWORD:-}
|
||||||
|
REDIS_HOST: ${REDIS_HOST:-r-bp1u741kij7e51i481pd.redis.rds.aliyuncs.com}
|
||||||
|
REDIS_PORT: ${REDIS_PORT:-6379}
|
||||||
|
REDIS_DATABASE: ${REDIS_DATABASE:-50}
|
||||||
|
REDIS_USERNAME: ${REDIS_USERNAME:-}
|
||||||
|
REDIS_PASSWORD: ${REDIS_PASSWORD:-}
|
||||||
ports:
|
ports:
|
||||||
- "${JT808_HTTP_PORT:-20400}:20400"
|
- "${JT808_HTTP_PORT:-20400}:20400"
|
||||||
- "${JT808_TCP_PORT:-808}:808"
|
- "${JT808_TCP_PORT:-808}:808"
|
||||||
|
|||||||
@@ -61,6 +61,20 @@ class PortainerComposeResourceLimitsTest {
|
|||||||
.doesNotContain("vehicle-history-archive:");
|
.doesNotContain("vehicle-history-archive:");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void localLaunchctlHistoryRuntimeDoesNotRequireSharedArchiveRoot() throws IOException {
|
||||||
|
Path launchctl = repositoryRoot().resolve("deploy/local/launchctl");
|
||||||
|
String historyTemplate = Files.readString(launchctl.resolve("com.lingniu.vehicle-history.plist.template"));
|
||||||
|
String readme = Files.readString(launchctl.resolve("README.md"));
|
||||||
|
|
||||||
|
assertThat(historyTemplate)
|
||||||
|
.doesNotContain("SINK_ARCHIVE_PATH")
|
||||||
|
.doesNotContain("__SHARED_ARCHIVE_PATH__");
|
||||||
|
assertThat(readme)
|
||||||
|
.doesNotContain("三个服务必须使用同一个 `SINK_ARCHIVE_PATH`")
|
||||||
|
.doesNotContain("history 服务按同一个 archive root 读取原始帧");
|
||||||
|
}
|
||||||
|
|
||||||
@Test
|
@Test
|
||||||
void vehicleAnalyticsComposeOnlyExposesStatConsumerEnvironment() throws IOException {
|
void vehicleAnalyticsComposeOnlyExposesStatConsumerEnvironment() throws IOException {
|
||||||
String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml"));
|
String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml"));
|
||||||
@@ -74,6 +88,21 @@ class PortainerComposeResourceLimitsTest {
|
|||||||
.doesNotContain("VEHICLE_STATE_ENABLED");
|
.doesNotContain("VEHICLE_STATE_ENABLED");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void vehicleAnalyticsDoesNotInheritRedisStateEnvironment() throws IOException {
|
||||||
|
String compose = Files.readString(repositoryRoot().resolve("deploy/portainer/docker-compose.yml"));
|
||||||
|
String commonEnv = commonEnvBlock(compose);
|
||||||
|
String analyticsService = serviceBlock(compose, "vehicle-analytics-app");
|
||||||
|
|
||||||
|
assertThat(analyticsService).contains("<<: *common-env");
|
||||||
|
assertThat(commonEnv)
|
||||||
|
.doesNotContain("REDIS_HOST")
|
||||||
|
.doesNotContain("REDIS_PORT")
|
||||||
|
.doesNotContain("REDIS_DATABASE")
|
||||||
|
.doesNotContain("REDIS_USERNAME")
|
||||||
|
.doesNotContain("REDIS_PASSWORD");
|
||||||
|
}
|
||||||
|
|
||||||
private static void assertServiceMemoryLimit(String compose,
|
private static void assertServiceMemoryLimit(String compose,
|
||||||
String serviceName,
|
String serviceName,
|
||||||
String envName,
|
String envName,
|
||||||
@@ -107,6 +136,19 @@ class PortainerComposeResourceLimitsTest {
|
|||||||
return compose.substring(start);
|
return compose.substring(start);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
private static String commonEnvBlock(String compose) {
|
||||||
|
String marker = "x-common-env: &common-env\n";
|
||||||
|
int start = compose.indexOf(marker);
|
||||||
|
if (start < 0) {
|
||||||
|
throw new IllegalArgumentException("common env block not found");
|
||||||
|
}
|
||||||
|
int end = compose.indexOf("\nservices:\n", start);
|
||||||
|
if (end < 0) {
|
||||||
|
throw new IllegalArgumentException("services block not found");
|
||||||
|
}
|
||||||
|
return compose.substring(start, end);
|
||||||
|
}
|
||||||
|
|
||||||
private static Path repositoryRoot() {
|
private static Path repositoryRoot() {
|
||||||
Path current = Path.of(System.getProperty("user.dir")).toAbsolutePath();
|
Path current = Path.of(System.getProperty("user.dir")).toAbsolutePath();
|
||||||
while (current != null) {
|
while (current != null) {
|
||||||
|
|||||||
Reference in New Issue
Block a user