ftb-project-management/deploy/monitoring

FTB Production Monitoring Baseline

This profile adds a deployable Prometheus/Grafana baseline for production operations. It is intentionally secret-free: no webhook URLs, API keys, SMTP passwords, or real alert receiver credentials are committed.

Start

docker compose --env-file .env.production -f docker-compose.prod.yml --profile monitoring up -d

Default local ports:

• Prometheus: http://localhost:9090
• Grafana: http://localhost:3002
• Loki: internal only

Set GRAFANA_ADMIN_USER and GRAFANA_ADMIN_PASSWORD in .env.production before exposing Grafana beyond localhost. Keep real alert receivers in the server environment or an untracked Alertmanager file.

Coverage

• DB availability: pg_up from postgres-exporter.
• Disk pressure: root filesystem availability from node-exporter.
• Slow API requests: Promtail turns Slow API request server logs into ftb_slow_api_log_total.
• Slow Prisma queries: Promtail turns Slow Prisma query server logs into ftb_slow_prisma_log_total.
• Job failures: Promtail turns AppData relation sync failed and AI call failure logs into ftb_job_failure_log_total.
• Xiaobao stale summaries: postgres-exporter custom query exposes ftb_xiaobao_stale_summary_count from xiaobao_risk_summaries.

Alerts

Prometheus loads prometheus/alert-rules.yml. The rules evaluate locally and are visible in Prometheus/Grafana. To send notifications, add Alertmanager outside git or mount an environment-specific receiver file; do not commit webhook URLs or tokens.

Baseline alert names:

• FtbPostgresDown
• FtbDiskPressure
• FtbSlowApiLogBurst
• FtbSlowPrismaLogBurst
• FtbJobFailureLogBurst
• FtbXiaobaoSummaryStale

Log Search

Promtail ships Docker logs to Loki with container labels. Grafana provisions both Prometheus and Loki data sources, so on-call checks can move from a firing alert to matching server logs without SSHing into the host.