ops: record 100k hardening ecs rollout

This commit is contained in:
lingniu
2026-07-03 17:59:44 +08:00
parent 79e591f864
commit c1fe1fbfb5
2 changed files with 42 additions and 5 deletions

View File

@@ -52,6 +52,7 @@ Go 服务使用 systemd 裸机部署,不使用 Docker。
最近 release
```text
/opt/lingniu-go-native/releases/100k-hardening-20260703175730
/opt/lingniu-go-native/releases/parsed-fields-once-20260703163821
```
@@ -332,6 +333,40 @@ journalctl -u lingniu-go-gateway.service --since "2 minutes ago" --no-pager | gr
'
```
## 2026-07-03 10W 接入优化部署记录
Release
```text
/opt/lingniu-go-native/releases/100k-hardening-20260703175730
```
包含:
- Gateway 默认 `TCP_MAX_CONNECTIONS=120000`
- 新增 `vehicle_gateway_connection_rejections_total`
- 新增 `/opt/lingniu-go-native/current/load-sim` 运维压测工具。
- Redis-first realtime projector 增加 MySQL async queue dropped 回归测试。
ECS 已落地内核参数:
```text
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.ip_local_port_range = 10000 65000
net.ipv4.tcp_tw_reuse = 1
```
注意:远端 `/etc/sysctl.conf` 原本有 `net.ipv4.tcp_max_syn_backlog = 1024`,已改为 `65535`,同时保留 `/etc/sysctl.d/zz-lingniu-vehicle-ingest.conf`
部署后验证:
- `current` 指向 `100k-hardening-20260703175730`
- `20211/20212/20213/20214/20200` readyz 均为 ok。
- `808``32960``20200``20211-20214` 监听 backlog 均显示 `65535`
- Gateway 重启后持续接收 GB32960、JT808、Yutong MQTT 帧。
- 重启完成后窗口未发现 gateway/realtime-api 持续 error/fatal 日志。
## 下次恢复上下文时先做
1. 读本文件。

View File

@@ -420,12 +420,12 @@ git commit -m "docs: design tdengine batch writer"
- Modify: `docs/ops/vehicle-ingest-runbook.md`
- Modify: `docs/ops/go-vehicle-ingest-memory.md`
- [ ] **Step 1: Apply sysctl changes on ECS**
- [x] **Step 1: Apply sysctl changes on ECS**
Run on ECS:
```bash
cat >/etc/sysctl.d/99-lingniu-vehicle-ingest.conf <<'EOF'
cat >/etc/sysctl.d/zz-lingniu-vehicle-ingest.conf <<'EOF'
net.core.somaxconn = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.ip_local_port_range = 10000 65000
@@ -434,7 +434,7 @@ EOF
sysctl --system
```
- [ ] **Step 2: Verify sysctl**
- [x] **Step 2: Verify sysctl**
Run:
@@ -451,7 +451,7 @@ net.ipv4.ip_local_port_range = 10000 65000
net.ipv4.tcp_tw_reuse = 1
```
- [ ] **Step 3: Restart gateway**
- [x] **Step 3: Restart gateway**
Run:
@@ -461,10 +461,12 @@ curl -fsS http://127.0.0.1:20211/readyz
ss -ltnp | grep -E ':(808|32960|20211) '
```
- [ ] **Step 4: Document rollout**
- [x] **Step 4: Document rollout**
Add the applied sysctl values and timestamp to `docs/ops/go-vehicle-ingest-memory.md`.
Actual release: `/opt/lingniu-go-native/releases/100k-hardening-20260703175730`.
- [ ] **Step 5: Commit**
```bash