Cardinality safety¶
High cardinality is how a metrics system takes down the very target it monitors. It
is the single most important thing to get right in an instrumentation library, so
transport-metrics treats it as a first-class design concern rather than a caveat.
What "cardinality" means, and why it bites¶
Every unique combination of label values on a metric creates a distinct time
series. A counter http_requests_total{method, route, status} with 5 methods, 20
routes and 6 status classes is 600 series — fine. Swap the route label for the raw
URL path and every /users/42, /users/43, … becomes its own series. On a busy
service that is millions of series: the scrape balloons, the client-side registry
grows unbounded, and the Prometheus server falls over. The classic outage.
The rule: templates, not raw values¶
Label by the route template, never the raw URL. /users/{id} is one series;
/users/42 would be one per user. Phase 2's HTTP request instrumentation reads the
matched route pattern from http.Request.Pattern (Go 1.22+ ServeMux) so this is
the default, not something you must remember. A WithRouteLabelFunc override lets
non-stdlib routers supply their own template, and an unmatched request falls into a
single bounded <other> bucket rather than flooding with one series per stray path.
The same discipline applies everywhere a label value could be unbounded:
- method / status — naturally bounded (optionally normalise status to its class,
2xx). - gRPC method — the full method name is bounded by your service definition.
- user / request / trace ids — never a label. That is what exemplars are for: they attach a trace id to a sample, not to a label, so you get the drill-down without the series explosion.
Your collectors, your contract¶
When you register app collectors (custom collectors) or, in later phases, use the RED/USE helpers, the cardinality of any caller-supplied label is yours to bound. Every helper that accepts a label name documents its cardinality contract, and the guidance is always the same: label values must come from a small, fixed set. If you cannot enumerate a label's possible values ahead of time, it does not belong in a label.