Monitoring and health checks

This page covers what your own monitoring stack needs to watch a self-hosted orboto instance: the two health endpoints, the metrics endpoint, and log behaviour. It applies to self-hosted installs - on orboto Cloud, monitoring is the platform's job.

Health endpoints

Point your orchestrator's liveness probe at /health and its readiness probe at /ready. If you don't run an orchestrator, an external uptime monitor hitting /health on an interval is enough to know the process is alive; hit /ready as well if you want to be alerted specifically on a database or storage outage rather than a process crash.

Sanity-check both from the host right after you deploy:

curl -i https://your-domain.example.com/health   # expect: 200, body {"status":"ok"}
curl -i https://your-domain.example.com/ready    # expect: 200 once Postgres + storage are reachable

No monitoring stack yet? You don't need Prometheus to get basic coverage. Any free or paid external uptime-check service (the kind that just polls a URL on an interval and emails/pages you on failure) works fine pointed at /health - that alone catches "the whole instance is down", which is most of what actually goes wrong on a small deployment. Add /ready as a second check once you want to distinguish "the process crashed" from "the database is unreachable but the process is still up".

Metrics

GET /metrics exposes Prometheus-format metrics: process and HTTP-request metrics plus a handful of orboto-specific counters. It is gated in production so it never leaks route topology or request volume publicly - one of these must be set, or the endpoint returns 503:

• METRICS_ALLOWED_CIDRS - comma-separated CIDR allowlist (IPv4 and IPv6), e.g. 10.0.0.0/8,127.0.0.1/32, so your Prometheus host can scrape it.
• METRICS_BASIC_AUTH - user:pass; the scraper sends HTTP Basic auth.

In development, with both unset, /metrics is open on localhost.

# prometheus.yml
scrape_configs:
  - job_name: orboto-api
    metrics_path: /metrics
    # If you use METRICS_BASIC_AUTH instead of a CIDR allowlist:
    # basic_auth: { username: prom, password: <pass> }
    static_configs:
      - targets: ["orboto-api:3000"]

Metrics worth alerting on:

Any Prometheus-compatible dashboard tool can visualise these once scraping works - point it at the same Prometheus data source you configured above.

Logs

The API logs structured JSON (via Pino). Authorization headers and passwords are redacted automatically before a line is written, so logs are safe to ship to a third-party aggregator. Control verbosity with LOG_LEVEL (info in production, debug in development).

docker compose logs -f api

Error tracking (optional)

Set SENTRY_DSN on the API service to start shipping unhandled API exceptions and frontend crashes to a Sentry-compatible error tracker. The web bundle picks up the same DSN automatically. Leaving it unset simply turns this integration off - orboto's own in-app error log for administrators keeps working either way.

Troubleshooting

/metrics always returns 503. Cause: in production, neither METRICS_ALLOWED_CIDRS nor METRICS_BASIC_AUTH is set - the endpoint refuses to serve anyone. Fix: set one of the two.

My Prometheus host gets 401 or 403 from /metrics. Cause: its IP isn't inside METRICS_ALLOWED_CIDRS, or the Basic Auth credentials don't match METRICS_BASIC_AUTH. Fix: widen the CIDR to include the scraper's real source IP, or fix the credentials on either side.

/ready flaps between 200 and 503. Cause: Postgres or the S3-compatible storage is intermittently unreachable from the API container - a network issue, not an application bug. Fix: check the health of those two services and the network path between them and the API container.

Nothing shows up in my error tracker. Cause: SENTRY_DSN isn't set. Fix: set it on the API service; it takes effect on the next restart.