# FTB Production Monitoring Baseline

This profile adds a deployable Prometheus/Grafana baseline for production operations. It is intentionally secret-free: no webhook URLs, API keys, SMTP passwords, or real alert receiver credentials are committed.

## Start

```bash
docker compose --env-file .env.production -f docker-compose.prod.yml --profile monitoring up -d
```

Default local ports:

- Prometheus: `http://localhost:9090`
- Grafana: `http://localhost:3002`
- Loki: internal only

Set `GRAFANA_ADMIN_USER` and `GRAFANA_ADMIN_PASSWORD` in `.env.production` before exposing Grafana beyond localhost. Keep real alert receivers in the server environment or an untracked Alertmanager file.

## Coverage

- DB availability: `pg_up` from postgres-exporter.
- Disk pressure: root filesystem availability from node-exporter.
- Slow API requests: Promtail turns `Slow API request` server logs into `ftb_slow_api_log_total`.
- Slow Prisma queries: Promtail turns `Slow Prisma query` server logs into `ftb_slow_prisma_log_total`.
- Job failures: Promtail turns `AppData relation sync failed` and AI call failure logs into `ftb_job_failure_log_total`.
- Xiaobao stale summaries: postgres-exporter custom query exposes `ftb_xiaobao_stale_summary_count` from `xiaobao_risk_summaries`.

## Alerts

Prometheus loads `prometheus/alert-rules.yml`. The rules evaluate locally and are visible in Prometheus/Grafana. To send notifications, add Alertmanager outside git or mount an environment-specific receiver file; do not commit webhook URLs or tokens.

Baseline alert names:

- `FtbPostgresDown`
- `FtbDiskPressure`
- `FtbSlowApiLogBurst`
- `FtbSlowPrismaLogBurst`
- `FtbJobFailureLogBurst`
- `FtbXiaobaoSummaryStale`

## Log Search

Promtail ships Docker logs to Loki with container labels. Grafana provisions both Prometheus and Loki data sources, so on-call checks can move from a firing alert to matching server logs without SSHing into the host.