Observability

Every system we hand over arrives instrumented. This describes what is already there and how to extend it without breaking the dashboards.

Structured logs#

Logs are JSON, one object per line, and always carry a correlation id.

{
  "level": "info",
  "time": "2026-07-14T09:12:44.201Z",
  "msg": "order.created",
  "correlationId": "req_01H8X...",
  "userId": "usr_01H7...",
  "durationMs": 42
}

Never log request bodies, tokens or anything under a password, secret or token key — the log pipeline redacts known field names, but it cannot redact a token you have concatenated into a message string.

Metrics#

Four golden signals per service, plus whatever the domain needs:

• Latency — histogram, p50/p95/p99, by route and status class.
• Traffic — requests per second, by route.
• Errors — rate of 5xx and unhandled rejections.
• Saturation — CPU, memory, connection pool utilisation.

Business metrics go alongside them. A checkout service that reports latency but not conversion tells you the system is healthy while revenue falls.

Tracing#

Every inbound request starts a trace, propagated through W3C traceparent. Spans are sampled at 100% for errors and 5% otherwise, which keeps cost sane while guaranteeing you can always investigate a failure.

Alerting defaults#

Shipped defaults:

Every alert links to a runbook. If you add an alert without one, the person woken by it at 3am has to reverse-engineer your intent — which is the situation the runbook exists to prevent.