Cardinality Limit

Analytics Metrics

Synopsis

Bounds the number of live metric series, per metric or globally, with sliding retention and an explicit policy for what happens past the limit.

Schema

- cardinality_limit:
    per: <enum>
    max_series: <numeric>
    max_metrics: <numeric>
    series_ttl_sec: <numeric>
    overflow: <enum>
    ignore_resource: <boolean>
    filter: <script>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>

Configuration

No field is required — the processor runs on its defaults.

Field	Required	Default	Description
per	N	"metric"	Scope of the series budget: metric gives each metric its own, global shares one across all of them
max_series	N	10000	Series allowed within the scope
max_metrics	N	1000	Distinct metric names tracked
series_ttl_sec	N	900	A series unseen for this long is forgotten, freeing its slot
overflow	N	"aggregate"	What happens past the limit: aggregate folds excess series into an overflow series, drop discards them
ignore_resource	N	false	Exclude resource attributes from the series identity
filter	N	-	Cribl-style JavaScript truthiness expression evaluated after if. A falsy result skips the processor
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	See Handling Failures. This does not mean "keep filtering and ignore errors" — see the warning below.
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
tag	N	-	Identifier
disabled	N	false	When true, the processor is skipped and the event continues to the next one. Lets you take a processor out of the path without removing its configuration

warning

ignore_failure: true silently turns this processor off. The drop is signalled to the pipeline as an error value, and the ignore_failure check runs before that value is inspected — so a matched event is kept instead of dropped, no error is logged, and the pipeline reports success. The processor appears to run normally while filtering nothing.

Use it only if you genuinely want a pass-through. To tolerate real errors without losing the filtering, leave ignore_failure unset and handle the failure with on_failure instead.

Details

Cardinality is the count of distinct attribute-value combinations, and each combination is a series a downstream store must keep. A single unbounded attribute can turn one metric into millions of series. This processor puts a ceiling on that, whatever the data does.

overflow is the decision that matters, and the two options lose different things:

Value	Behavior
aggregate	Series past the limit are folded into a single overflow series. The total stays right; the attribution is lost
drop	Series past the limit are discarded. What remains is exact; the excess is gone from the totals

The default is aggregate, on the reasoning that a wrong total is worse than a coarse one — a count that silently under-reports is harder to notice than an "overflow" bucket sitting in the results.

series_ttl_sec is what makes the limit a live one. Series are forgotten when unseen for the TTL, so a workload that churns through short-lived attribute values does not permanently exhaust the budget with series nothing is reporting any more.

Every numeric option treats 0 as "use the default", not "unlimited" — max_series: 0 gives 10000, not an unbounded budget. A negative value is a configuration error, rejected at load. per and overflow are also validated at load: any value other than the two listed fails the configuration.

ignore_resource removes resource attributes from the identity, so series differing only at resource level count as one. Use it when the resource attributes are incidental to what you are measuring.

To reduce cardinality by removing the attribute causing it rather than capping the result, use Drop Dimensions.

Examples

A Per-Metric Ceiling

Giving every metric its own budget...	- cardinality_limit: per: metric max_series: 5000 overflow: aggregate
so one runaway metric cannot starve the others:	# Series 5001+ of any single metric fold into # that metric's overflow series. Totals stay correct.

A Global Budget, Dropping Excess

One shared ceiling, discarding what does not fit...	- cardinality_limit: per: global max_series: 50000 overflow: drop series_ttl_sec: 300
with a shorter TTL, so idle series release their slots sooner:	# What survives is exact rather than partly bucketed — # at the cost of the excess being absent from totals.

Ignoring Resource Attributes

Where resource-level differences should not multiply the count...	- cardinality_limit: ignore_resource: true max_series: 10000
series differing only by resource attribute count as one:	# Useful when each pod reports the same metric and the # pod identity is not what you are measuring.