Make the offline RAG chain observable without widening credential access
Some checks failed
verify / verify (push) Has been cancelled
Some checks failed
verify / verify (push) Has been cancelled
Expose the verified synthetic retrieval path through a typed React client and a non-root Nginx edge while keeping database credentials and data-network reachability out of the browser tier. A fixed-origin gateway preserves request boundaries and now closes upstream streams even when downstream disconnects before body iteration. The deployment ADR and runbooks record the four-network topology and its accepted Web edge-egress risk. Constraint: The previously exposed Bailian key must be revoked and no live provider credential may enter Git, images, logs, or the browser. Rejected: Connect Web directly to the data network | expands lateral reach to PostgreSQL. Rejected: Publish the database-aware API on the edge network | gives a credential-bearing process a public default route. Rejected: Buffer streaming responses in either proxy | prevents incremental chat delivery in the future. Confidence: high Scope-risk: moderate Reversibility: clean Directive: Do not mark Stage 1 or Stage 2 complete until the rotated-key live smoke and remaining stage gates pass. Tested: make verify; 65 backend tests; 14 frontend tests; Docker image build; container health and isolation probes; real HTTP demo/status/search/docs/OpenAPI checks. Not-tested: Live Bailian models, real document ingestion, business chat SSE generation, and final browser screenshot automation because the browser skill runtime was unavailable.
This commit is contained in:
@@ -29,9 +29,10 @@ docker compose up -d --build
|
||||
|---|---|---|---|
|
||||
| `db` | 固定版本 pgvector 镜像 | 元数据、向量、任务、会话、评测 | 仅内部网络 |
|
||||
| `migrate` | 与后端同镜像 | 运行 Alembic,一次成功退出 | 无 |
|
||||
| `api` | `backend/Dockerfile` | FastAPI、检索、问答、管理 API | 仅由 Web 代理 |
|
||||
| `api` | `backend/Dockerfile` | FastAPI、检索、问答、管理 API | 仅 internal data 网络 |
|
||||
| `gateway` | 与后端同镜像 | 固定上游、请求边界、脱敏错误和流式转发 | 仅 internal ingress/data 网络 |
|
||||
| `worker` | 同一后端镜像 | 解析、向量化、评测后台任务 | 无 |
|
||||
| `web` | `frontend/Dockerfile` | Nginx 静态资源和反向代理 | 宿主机 8080/HTTPS |
|
||||
| `web` | `frontend/Dockerfile` | React 静态资源、Nginx 同源入口和 SSE 代理 | `127.0.0.1:8000`;生产部署 HTTPS |
|
||||
| `ocr-worker` | 可选 profile | PaddleOCR 重型任务 | 无 |
|
||||
| `prometheus/grafana` | 可选 profile | 本地观测 | 默认不对公网 |
|
||||
|
||||
@@ -116,26 +117,31 @@ services:
|
||||
|
||||
api:
|
||||
build: ./backend
|
||||
command: ["python", "-m", "app.main"]
|
||||
command: ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
depends_on:
|
||||
migrate:
|
||||
condition: service_completed_successfully
|
||||
environment:
|
||||
<<: [*runtime-config, *rag-config]
|
||||
APP_SECRET_KEY_FILE: /run/secrets/app_secret_key
|
||||
environment: *runtime-config
|
||||
secrets:
|
||||
- postgres_app_password
|
||||
- app_secret_key
|
||||
- bailian_api_key
|
||||
volumes:
|
||||
- uploads:/data/uploads
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/api/v1/health/ready', timeout=2)"]
|
||||
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health/ready', timeout=2)"]
|
||||
interval: 10s
|
||||
timeout: 3s
|
||||
retries: 12
|
||||
start_period: 10s
|
||||
networks: [edge, data, egress]
|
||||
networks: [data]
|
||||
restart: unless-stopped
|
||||
|
||||
gateway:
|
||||
build: ./backend
|
||||
command: ["uvicorn", "app.gateway:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
depends_on:
|
||||
api:
|
||||
condition: service_healthy
|
||||
networks: [ingress, data]
|
||||
restart: unless-stopped
|
||||
|
||||
worker:
|
||||
@@ -183,11 +189,11 @@ services:
|
||||
web:
|
||||
build: ./frontend
|
||||
depends_on:
|
||||
api:
|
||||
gateway:
|
||||
condition: service_healthy
|
||||
ports:
|
||||
- "8080:80"
|
||||
networks: [edge]
|
||||
- "127.0.0.1:8000:8080"
|
||||
networks: [edge, ingress]
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
@@ -197,6 +203,9 @@ volumes:
|
||||
networks:
|
||||
edge:
|
||||
driver: bridge
|
||||
ingress:
|
||||
driver: bridge
|
||||
internal: true
|
||||
data:
|
||||
driver: bridge
|
||||
internal: true
|
||||
@@ -210,15 +219,13 @@ secrets:
|
||||
file: ./secrets/postgres_migrator_password
|
||||
postgres_app_password:
|
||||
file: ./secrets/postgres_app_password
|
||||
app_secret_key:
|
||||
file: ./secrets/app_secret_key
|
||||
bailian_api_key:
|
||||
file: ./secrets/bailian_api_key
|
||||
```
|
||||
|
||||
最终实现固定镜像 digest,示意版本只说明可行结构。`migrate` 只有一个实例执行;API 和 Worker 不在启动时并发自动迁移。首次初始化脚本只在 `db` 容器内使用 bootstrap 凭据创建 `vector` 扩展、无超级用户权限的 migrator/app 登录角色和专属 schema;migrator 拥有 schema/DDL,app 仅获运行期所需的 DML/sequence 权限及默认权限,API/Worker 永远不挂载 bootstrap 或 migrator Secret。最后一个初始化脚本只有在全部 SQL 事务和授权成功后,才在 `PGDATA` 内用同文件系统 rename 原子写入 `.rag-bootstrap-complete`;healthcheck 同时检查 `pg_isready` 与该哨兵,防止临时初始化 server 提前放行迁移。备份再使用独立只读角色。Compose 的 `.env` 仅做变量插值,不会自动注入容器,因此可配置的非敏感项和 `*_FILE` 路径通过 YAML anchor 在 `environment` 中完整声明。后端从各自的 `POSTGRES_PASSWORD_FILE` 读取密码后在内存中组装 DSN,不把明文密码放进 `.env` 或 `DATABASE_URL`。
|
||||
|
||||
网络分为三层:`web + api` 位于 `edge`,`api + worker + migrate + db` 位于 `data`,只有 `api + worker` 位于 `egress`。`data` 设置 `internal: true`,所以 Web 无法横向连接数据库;普通 bridge 本身不构成出站域名白名单,生产机仍需用主机防火墙/出口代理仅放行百炼和对象存储域名。
|
||||
网络分为四层:`web` 位于普通 `edge` 和 internal `ingress`;`gateway` 位于 `ingress + data`;API、迁移、Worker 和数据库位于 internal `data`;只有明确需要云调用的一次性工具或未来 Worker 才能加入命名的 `egress`。因此浏览器入口不能横向连接数据库,数据库感知 API 也没有公网默认出口。只有无 Secret 的 Web 容器发布回环端口;gateway 固定上游并继续执行请求边界和脱敏错误契约。详细理由和被否决方案见 [ADR-0004](adr/0004-secretless-web-ingress.md)。必须注意:普通 `edge` bridge 为 Web 提供宿主机端口发布时,也给 Web 留有技术上的默认公网出口;当前接受这一点的前提是 Web 无 Secret、无数据网络、无挂载且代理上游固定。若部署基线要求入口容器也完全禁网,必须在宿主机防火墙/出口代理层阻断,不能把“未加入名为 egress 的网络”误写成“没有外网出口”。普通 bridge 本身也不构成域名白名单,正式联网 Worker 仍需主机防火墙或出口代理只放行百炼和对象存储域名。
|
||||
|
||||
### 2.3 后台任务为何不用 Redis
|
||||
|
||||
|
||||
Reference in New Issue
Block a user