Make the offline RAG chain observable without widening credential access
Some checks failed
verify / verify (push) Has been cancelled

Expose the verified synthetic retrieval path through a typed React client and a non-root Nginx edge while keeping database credentials and data-network reachability out of the browser tier. A fixed-origin gateway preserves request boundaries and now closes upstream streams even when downstream disconnects before body iteration. The deployment ADR and runbooks record the four-network topology and its accepted Web edge-egress risk.

Constraint: The previously exposed Bailian key must be revoked and no live provider credential may enter Git, images, logs, or the browser.
Rejected: Connect Web directly to the data network | expands lateral reach to PostgreSQL.
Rejected: Publish the database-aware API on the edge network | gives a credential-bearing process a public default route.
Rejected: Buffer streaming responses in either proxy | prevents incremental chat delivery in the future.
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Do not mark Stage 1 or Stage 2 complete until the rotated-key live smoke and remaining stage gates pass.
Tested: make verify; 65 backend tests; 14 frontend tests; Docker image build; container health and isolation probes; real HTTP demo/status/search/docs/OpenAPI checks.
Not-tested: Live Bailian models, real document ingestion, business chat SSE generation, and final browser screenshot automation because the browser skill runtime was unavailable.
This commit is contained in:
2026-07-12 17:38:57 +08:00
parent 2fa27ae71c
commit c3bad0f186
60 changed files with 9062 additions and 82 deletions

View File

@@ -29,9 +29,10 @@ docker compose up -d --build
|---|---|---|---|
| `db` | 固定版本 pgvector 镜像 | 元数据、向量、任务、会话、评测 | 仅内部网络 |
| `migrate` | 与后端同镜像 | 运行 Alembic一次成功退出 | 无 |
| `api` | `backend/Dockerfile` | FastAPI、检索、问答、管理 API | 仅由 Web 代理 |
| `api` | `backend/Dockerfile` | FastAPI、检索、问答、管理 API | 仅 internal data 网络 |
| `gateway` | 与后端同镜像 | 固定上游、请求边界、脱敏错误和流式转发 | 仅 internal ingress/data 网络 |
| `worker` | 同一后端镜像 | 解析、向量化、评测后台任务 | 无 |
| `web` | `frontend/Dockerfile` | Nginx 静态资源和反向代理 | 宿主机 8080/HTTPS |
| `web` | `frontend/Dockerfile` | React 静态资源、Nginx 同源入口和 SSE 代理 | `127.0.0.1:8000`;生产部署 HTTPS |
| `ocr-worker` | 可选 profile | PaddleOCR 重型任务 | 无 |
| `prometheus/grafana` | 可选 profile | 本地观测 | 默认不对公网 |
@@ -116,26 +117,31 @@ services:
api:
build: ./backend
command: ["python", "-m", "app.main"]
command: ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
depends_on:
migrate:
condition: service_completed_successfully
environment:
<<: [*runtime-config, *rag-config]
APP_SECRET_KEY_FILE: /run/secrets/app_secret_key
environment: *runtime-config
secrets:
- postgres_app_password
- app_secret_key
- bailian_api_key
volumes:
- uploads:/data/uploads
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/api/v1/health/ready', timeout=2)"]
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health/ready', timeout=2)"]
interval: 10s
timeout: 3s
retries: 12
start_period: 10s
networks: [edge, data, egress]
networks: [data]
restart: unless-stopped
gateway:
build: ./backend
command: ["uvicorn", "app.gateway:app", "--host", "0.0.0.0", "--port", "8000"]
depends_on:
api:
condition: service_healthy
networks: [ingress, data]
restart: unless-stopped
worker:
@@ -183,11 +189,11 @@ services:
web:
build: ./frontend
depends_on:
api:
gateway:
condition: service_healthy
ports:
- "8080:80"
networks: [edge]
- "127.0.0.1:8000:8080"
networks: [edge, ingress]
restart: unless-stopped
volumes:
@@ -197,6 +203,9 @@ volumes:
networks:
edge:
driver: bridge
ingress:
driver: bridge
internal: true
data:
driver: bridge
internal: true
@@ -210,15 +219,13 @@ secrets:
file: ./secrets/postgres_migrator_password
postgres_app_password:
file: ./secrets/postgres_app_password
app_secret_key:
file: ./secrets/app_secret_key
bailian_api_key:
file: ./secrets/bailian_api_key
```
最终实现固定镜像 digest示意版本只说明可行结构。`migrate` 只有一个实例执行API 和 Worker 不在启动时并发自动迁移。首次初始化脚本只在 `db` 容器内使用 bootstrap 凭据创建 `vector` 扩展、无超级用户权限的 migrator/app 登录角色和专属 schemamigrator 拥有 schema/DDLapp 仅获运行期所需的 DML/sequence 权限及默认权限API/Worker 永远不挂载 bootstrap 或 migrator Secret。最后一个初始化脚本只有在全部 SQL 事务和授权成功后,才在 `PGDATA` 内用同文件系统 rename 原子写入 `.rag-bootstrap-complete`healthcheck 同时检查 `pg_isready` 与该哨兵,防止临时初始化 server 提前放行迁移。备份再使用独立只读角色。Compose 的 `.env` 仅做变量插值,不会自动注入容器,因此可配置的非敏感项和 `*_FILE` 路径通过 YAML anchor 在 `environment` 中完整声明。后端从各自的 `POSTGRES_PASSWORD_FILE` 读取密码后在内存中组装 DSN不把明文密码放进 `.env``DATABASE_URL`
网络分为层:`web + api` 位于 `edge``api + worker + migrate + db` 位于 `data`只有 `api + worker` 位于 `egress``data` 设置 `internal: true`,所以 Web 无法横向连接数据库;普通 bridge 本身不构成出站域名白名单,生产机仍需主机防火墙/出口代理放行百炼和对象存储域名。
网络分为层:`web` 位于普通 `edge` 和 internal `ingress``gateway` 位于 `ingress + data`API、迁移、Worker 和数据库位于 internal `data`只有明确需要云调用的一次性工具或未来 Worker 才能加入命名的 `egress`。因此浏览器入口不能横向连接数据库,数据库感知 API 也没有公网默认出口。只有无 Secret 的 Web 容器发布回环端口gateway 固定上游并继续执行请求边界和脱敏错误契约。详细理由和被否决方案见 [ADR-0004](adr/0004-secretless-web-ingress.md)。必须注意:普通 `edge` bridge 为 Web 提供宿主机端口发布时,也给 Web 留有技术上的默认公网出口;当前接受这一点的前提是 Web 无 Secret、无数据网络、无挂载且代理上游固定。若部署基线要求入口容器也完全禁网必须在宿主机防火墙/出口代理层阻断,不能把“未加入名为 egress 的网络”误写成“没有外网出口”。普通 bridge 本身不构成域名白名单,正式联网 Worker 仍需主机防火墙出口代理放行百炼和对象存储域名。
### 2.3 后台任务为何不用 Redis