Exposing a /metrics endpoint takes five minutes. Getting the instrumentation itself right — metrics that stay cheap at scale, catch real problems, and don't lie under load. Here's a breakdown of the design decisions behind a recent metrics layer built for a Go API.
RED as the baseline, not the ceiling
Request rate, error ratio, and latency:
http_requests_total{method, path, status} for volume and error breakdown
http_request_duration_seconds as a histogram, buckets tuned to the actual latency distribution
