ops: record 100k hardening ecs rollout
This commit is contained in:
@@ -52,6 +52,7 @@ Go 服务使用 systemd 裸机部署,不使用 Docker。
|
|||||||
最近 release:
|
最近 release:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
|
/opt/lingniu-go-native/releases/100k-hardening-20260703175730
|
||||||
/opt/lingniu-go-native/releases/parsed-fields-once-20260703163821
|
/opt/lingniu-go-native/releases/parsed-fields-once-20260703163821
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -332,6 +333,40 @@ journalctl -u lingniu-go-gateway.service --since "2 minutes ago" --no-pager | gr
|
|||||||
'
|
'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## 2026-07-03 10W 接入优化部署记录
|
||||||
|
|
||||||
|
Release:
|
||||||
|
|
||||||
|
```text
|
||||||
|
/opt/lingniu-go-native/releases/100k-hardening-20260703175730
|
||||||
|
```
|
||||||
|
|
||||||
|
包含:
|
||||||
|
|
||||||
|
- Gateway 默认 `TCP_MAX_CONNECTIONS=120000`。
|
||||||
|
- 新增 `vehicle_gateway_connection_rejections_total`。
|
||||||
|
- 新增 `/opt/lingniu-go-native/current/load-sim` 运维压测工具。
|
||||||
|
- Redis-first realtime projector 增加 MySQL async queue dropped 回归测试。
|
||||||
|
|
||||||
|
ECS 已落地内核参数:
|
||||||
|
|
||||||
|
```text
|
||||||
|
net.core.somaxconn = 65535
|
||||||
|
net.ipv4.tcp_max_syn_backlog = 65535
|
||||||
|
net.ipv4.ip_local_port_range = 10000 65000
|
||||||
|
net.ipv4.tcp_tw_reuse = 1
|
||||||
|
```
|
||||||
|
|
||||||
|
注意:远端 `/etc/sysctl.conf` 原本有 `net.ipv4.tcp_max_syn_backlog = 1024`,已改为 `65535`,同时保留 `/etc/sysctl.d/zz-lingniu-vehicle-ingest.conf`。
|
||||||
|
|
||||||
|
部署后验证:
|
||||||
|
|
||||||
|
- `current` 指向 `100k-hardening-20260703175730`。
|
||||||
|
- `20211/20212/20213/20214/20200` readyz 均为 ok。
|
||||||
|
- `808`、`32960`、`20200`、`20211-20214` 监听 backlog 均显示 `65535`。
|
||||||
|
- Gateway 重启后持续接收 GB32960、JT808、Yutong MQTT 帧。
|
||||||
|
- 重启完成后窗口未发现 gateway/realtime-api 持续 error/fatal 日志。
|
||||||
|
|
||||||
## 下次恢复上下文时先做
|
## 下次恢复上下文时先做
|
||||||
|
|
||||||
1. 读本文件。
|
1. 读本文件。
|
||||||
|
|||||||
@@ -420,12 +420,12 @@ git commit -m "docs: design tdengine batch writer"
|
|||||||
- Modify: `docs/ops/vehicle-ingest-runbook.md`
|
- Modify: `docs/ops/vehicle-ingest-runbook.md`
|
||||||
- Modify: `docs/ops/go-vehicle-ingest-memory.md`
|
- Modify: `docs/ops/go-vehicle-ingest-memory.md`
|
||||||
|
|
||||||
- [ ] **Step 1: Apply sysctl changes on ECS**
|
- [x] **Step 1: Apply sysctl changes on ECS**
|
||||||
|
|
||||||
Run on ECS:
|
Run on ECS:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
cat >/etc/sysctl.d/99-lingniu-vehicle-ingest.conf <<'EOF'
|
cat >/etc/sysctl.d/zz-lingniu-vehicle-ingest.conf <<'EOF'
|
||||||
net.core.somaxconn = 65535
|
net.core.somaxconn = 65535
|
||||||
net.ipv4.tcp_max_syn_backlog = 65535
|
net.ipv4.tcp_max_syn_backlog = 65535
|
||||||
net.ipv4.ip_local_port_range = 10000 65000
|
net.ipv4.ip_local_port_range = 10000 65000
|
||||||
@@ -434,7 +434,7 @@ EOF
|
|||||||
sysctl --system
|
sysctl --system
|
||||||
```
|
```
|
||||||
|
|
||||||
- [ ] **Step 2: Verify sysctl**
|
- [x] **Step 2: Verify sysctl**
|
||||||
|
|
||||||
Run:
|
Run:
|
||||||
|
|
||||||
@@ -451,7 +451,7 @@ net.ipv4.ip_local_port_range = 10000 65000
|
|||||||
net.ipv4.tcp_tw_reuse = 1
|
net.ipv4.tcp_tw_reuse = 1
|
||||||
```
|
```
|
||||||
|
|
||||||
- [ ] **Step 3: Restart gateway**
|
- [x] **Step 3: Restart gateway**
|
||||||
|
|
||||||
Run:
|
Run:
|
||||||
|
|
||||||
@@ -461,10 +461,12 @@ curl -fsS http://127.0.0.1:20211/readyz
|
|||||||
ss -ltnp | grep -E ':(808|32960|20211) '
|
ss -ltnp | grep -E ':(808|32960|20211) '
|
||||||
```
|
```
|
||||||
|
|
||||||
- [ ] **Step 4: Document rollout**
|
- [x] **Step 4: Document rollout**
|
||||||
|
|
||||||
Add the applied sysctl values and timestamp to `docs/ops/go-vehicle-ingest-memory.md`.
|
Add the applied sysctl values and timestamp to `docs/ops/go-vehicle-ingest-memory.md`.
|
||||||
|
|
||||||
|
Actual release: `/opt/lingniu-go-native/releases/100k-hardening-20260703175730`.
|
||||||
|
|
||||||
- [ ] **Step 5: Commit**
|
- [ ] **Step 5: Commit**
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
Reference in New Issue
Block a user