72a59f125c
- 新增迁移回滚、AppData 退场、小宝后台任务 runbook\n- 新增生产 readiness 证据清单和 runbook placeholder 扫描\n- 更新部署文档与路线图到 V2.8 运维闭环阶段\n\nCo-Authored-By: GPT-5 Codex <codex@openai.com>
56 lines
3.0 KiB
Markdown
56 lines
3.0 KiB
Markdown
# Xiaobao Background Jobs Runbook
|
|
|
|
Current V2.8 production monitoring supports Xiaobao staleness detection. The first production implementation is still page-triggered: opening `/xiaobao-warning` computes risk, saves snapshots, and lets V2.3 sync refresh summaries. A future scheduler must preserve the same idempotent data contract.
|
|
|
|
## Alert Triage
|
|
|
|
When `FtbXiaobaoSummaryStale` fires:
|
|
|
|
```bash
|
|
docker compose --env-file .env.production -f docker-compose.prod.yml exec postgres psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" -c "select version_id, dirty, updated_at, recomputed_at, risk_level, risk_score from xiaobao_risk_summaries where dirty = true or updated_at < now() - interval '6 hours' order by updated_at asc limit 20;"
|
|
docker compose --env-file .env.production -f docker-compose.prod.yml logs --tail=200 server | grep -E "xiaobao|AppData relation sync failed|Slow Prisma query"
|
|
```
|
|
|
|
Decision points:
|
|
|
|
- Rows are dirty after active version edits: ask a manager to open `/xiaobao-warning` once, then verify summaries refresh.
|
|
- Rows remain dirty and server logs show sync failures: treat as AppData relation sync incident and follow `migration-rollback.md`.
|
|
- Rows are stale but no product release is near: keep monitoring and schedule a manual refresh before the next release decision meeting.
|
|
- Rows are stale for a release due today: refresh manually and have the release owner review the resulting risk explanation before ship/no-ship decision.
|
|
|
|
## Manual Refresh Path
|
|
|
|
1. Log in as a user with `xiaobao.warning:manage`.
|
|
2. Open `/xiaobao-warning`.
|
|
3. Wait until AI interpretation status is no longer generating for high-risk versions.
|
|
4. Re-run the stale-summary query.
|
|
5. Run the release smoke test.
|
|
|
|
```bash
|
|
pnpm deploy:smoke -- --base-url http://localhost
|
|
```
|
|
|
|
## Future Scheduler Rules
|
|
|
|
When a background job is introduced, it must:
|
|
|
|
- Read unfinished versions by relation-table scope, not by full AppData document scan.
|
|
- Use one idempotency key per `versionId + riskSignature + snapshotDate`.
|
|
- Write snapshots append-only and upsert summaries by `version_id`.
|
|
- Mark failures with structured logs containing `xiaobao background job failed`.
|
|
- Retry transient AI failures with backoff and keep rule-based risk output even when AI interpretation fails.
|
|
- Never mutate Version, Requirement, DevTask, TestCase, Bug, or Member data.
|
|
|
|
## Monitoring Expectations
|
|
|
|
- `FtbXiaobaoSummaryStale` alerts on dirty or older-than-6-hour summaries.
|
|
- `FtbJobFailureLogBurst` alerts when sync or future job failure log counters increase.
|
|
- Grafana dashboard shows the stale summary count and matching server warning/error logs.
|
|
|
|
## Data Risks
|
|
|
|
- Recomputing Xiaobao summaries can change release risk badges and manager decisions; record manual refresh time in release notes.
|
|
- AI interpretation cache is explanatory only. Do not restore or delete business entities to fix a bad explanation.
|
|
- If stale summaries are caused by relation sync failure, refreshing the page can mask the symptom without fixing the underlying sync path. Preserve logs before restarting services.
|
|
|