Regex Filter

Filter Cribl Compatible

Synopsis

Filters out events based on regular expression matches.

Schema

- regex_filter:
    field: <ident>
    regex: <string>
    regexes: <string[]>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
field	Y	-	Field to match against patterns
regex	Y	-	Single regex pattern to match
regexes	N	-	Additional patterns to match
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	Continue if matching fails. This does not mean "keep filtering and ignore errors" — see the warning below.
ignore_missing	N	false	Continue if source field doesn't exist
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
tag	N	-	Identifier
disabled	N	false	When true, the processor is skipped and the event continues to the next one. Lets you take a processor out of the path without removing its configuration

warning

ignore_failure: true silently turns this processor off. The drop is signalled to the pipeline as an error value, and the ignore_failure check runs before that value is inspected — so a matched event is kept instead of dropped, no error is logged, and the pipeline reports success. The processor appears to run normally while filtering nothing.

Use it only if you genuinely want a pass-through. To tolerate real errors without losing the filtering, leave ignore_failure unset and handle the failure with on_failure instead.

Details

The processor uses Golang regular expressions to match field content.

If either the primary regex or any pattern in regexes matches, the event is dropped from the pipeline. This is useful for filtering out unwanted events.

Multiple patterns can be specified, and any match will trigger the filter.

caution

Complex regular expressions may impact performance.

The processor can be nicely dove-tailed with conditional execution, field value validation, and error handling and success/failure processors.

If no patterns are specified, all events pass through. Empty field content is treated as non-matching. Matching success triggers event dropping.

warning

Invalid regex patterns and non-string field values will cause errors unless ignore_failure is set. Missing fields can be skipped by setting ignore_missing.

Regular Expression Semantics

Patterns use RE2 syntax, with lookahead ((?=…), (?!…)) and lookbehind ((?<=…), (?<!…)) added. Backreferences (\1, \k<name>), atomic groups and possessive quantifiers are not supported: a pattern that uses one fails to compile, and the processor fails the record — its on_failure processors run, and ignore_failure applies. A lookbehind cannot contain a capture group or another lookaround. \d, \w and \s match ASCII characters.

Matching has no per-match timeout, because none is needed: no pattern can make matching backtrack without bound.

Matching is unanchored and case-sensitive. A plain string matches anywhere in the field — anchor with ^ and $, and prefix the expression with (?i) for case-insensitive matching.

A pattern is rejected before it compiles when it:

• exceeds 1000 characters,
• uses more than 100 quantifiers (*, +, {n,m}),
• nests groups more than 10 levels deep, or
• takes longer than 100ms to compile.

Examples

Basic

Messages containing errors...	{ "message": "error occurred" } - regex_filter: field: message regex: "error.*"
are dropped when the event matches:	Event dropped from pipeline

Multi-Pattern

Filtering out both errors and warnings...	{ "message": "warning: system overload" } - regex_filter: field: message regexes: - "error.*" - "warning.*"
drops both the warning and the event:	Event dropped from pipeline

Non-Matching Event

Allowing non-matching messages through...	{ "message": "success message" } - regex_filter: field: message regex: "error.*"
lets the event to continue:	{ "message": "success message" }

Conditionals

Filtering only when a condition is met...	{ "message": "error message", "level": "info" } - regex_filter: if: "ctx.level == 'error'" field: message regex: "error.*"
skips the event upon match:	{ "message": "error message", "level": "info" }

Missing Fields

Handling missing fields gracefully...	{} - regex_filter: field: nonexistent regex: "error.*" ignore_missing: true
continues the execution:	{}

Non-String Fields

Handling non-string field values...	{ "message": 123 } - regex_filter: field: message regex: "error.*" ignore_failure: true
skips filtering upon invalid types:	{ "message": 123 }