Lookup

Enrich Cribl Compatible

Synopsis

Enriches log events by looking up values in a CSV file and adding corresponding fields to the event. Supports multiple matching modes, case-sensitive/insensitive matching, and various ways to handle multiple matches.

Schema

- lookup:
    lookup_file: <string>
    match_mode: <enum>
    match_type: <enum>
    lookup_fields: <LookupField[]>
    output_fields: <OutputField[]>
    ignore_case: <boolean>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
lookup_file	Y	-	Path to CSV file containing lookup data
match_mode	Y	-	Type of matching: exact, regex, or cidr
match_type	Y	-	How to handle multiple matches: first, most_specific, or all
lookup_fields	Y	-	Array of field mappings used for matching
output_fields	Y	-	Array of fields to add to event from matched lookup entries
ignore_case	N	false	Enable case-insensitive matching
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	Continue on lookup errors
ignore_missing	N	false	Continue if lookup fields don't exist
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
tag	N	-	Identifier

Lookup Field

Field	Required	Default	Description
event_field	Y	-	Field name in the event
lookup_field	N	event_field	Corresponding field name in lookup CSV

Output Field

Field	Required	Default	Description
lookup_field	Y	-	Field name in lookup CSV
event_field	N	lookup_field	Field name to add to event
default_value	N	-	Value to use if no match found

Match Modes

• exact: Exact string matching
• regex: Regular expression matching (pattern from lookup table)
• cidr: CIDR network matching (for IP addresses)

Match Types

• first: Use first matching entry found
• most_specific: Use most specific match (primarily for CIDR matching)
• all: Use all matching entries (creates array of values)

Details

The processor reads the lookup data from a CSV file where the first row contains header names. It matches event fields against lookup fields and adds corresponding output fields to the event.

note

• The processor caches lookup data in memory for performance.
• Malformed CSV rows (wrong number of fields) are skipped.
• For CIDR matching, more specific networks (longer prefixes) take precedence.
• When using match_type: all, matching fields become arrays containing all matched values.

warning

• When using regex matching, ensure patterns in the lookup table are valid regular expressions. Invalid patterns will be skipped.
• Be cautious with case sensitivity when field names in the CSV differ only by case.
• Large CSV files will be kept in memory, so consider memory usage for very large lookup tables.

Examples

Basic IP Lookup

Map IP addresses to locations...	- lookup: lookup_file: "ip_locations.csv" match_mode: cidr match_type: most_specific lookup_fields: - event_field: source.ip lookup_field: network output_fields: - lookup_field: location event_field: source.location - lookup_field: datacenter event_field: source.datacenter
where ip_locations.csv contains:	network,location,datacenter 10.0.0.0/8,Internal,DC1 192.168.0.0/16,Office,DC2

CIDR Network Zone Mapping

Map IPs to network zones with different security levels...	- lookup: lookup_file: "network_zones.csv" match_mode: cidr match_type: most_specific lookup_fields: - lookup_field: network event_field: client_ip output_fields: - lookup_field: zone event_field: network_zone - lookup_field: security_level event_field: security
where network_zones.csv contains overlapping networks:	network,zone,security_level 10.0.0.0/8,internal,low 10.1.0.0/16,dmz,medium 10.1.1.0/24,secure,high

An IP address like "10.1.1.50" will match all three networks but return the most specific match (/24), resulting in zone="secure" and security_level="high".

Case-Insensitive Matching

Match values regardless of case...	- lookup: lookup_file: "status_map.csv" match_mode: exact match_type: first ignore_case: true lookup_fields: - event_field: state lookup_field: Status output_fields: - event_field: status_desc lookup_field: Description
where status_map.csv contains:	Status,Description Active,Running INACTIVE,Stopped

With ignore_case: true, an event with state="ACTIVE" will match the "Active" entry and set status_desc="Running".

Multiple Field Regular Expression Matching

Match on multiple fields using regex patterns...	- lookup: lookup_file: "access_rules.csv" match_mode: regex match_type: all lookup_fields: - event_field: dept lookup_field: department - event_field: title lookup_field: access_level output_fields: - event_field: access lookup_field: permissions
where access_rules.csv contains:	department,access_level,permissions IT,.*,admin Engineering,developer.*,read-write Engineering,junior.*,read-only

When match_type: all is used with multiple matches, the output field becomes an array containing all matched values.

Default Values for Unmatched Events

Provide default values when no match is found...	- lookup: lookup_file: "roles.csv" match_mode: exact match_type: first lookup_fields: - event_field: username lookup_field: user output_fields: - event_field: user_role lookup_field: role default_value: guest
where roles.csv contains:	user,role alice,admin bob,user

When looking up a username not in the CSV (like "carol"), the default_value will be used.

Pattern Matching with Message Content

Match log messages against patterns for classification...	- lookup: lookup_file: "error_patterns.csv" match_mode: regex match_type: first lookup_fields: - event_field: message lookup_field: pattern output_fields: - event_field: alert_level lookup_field: severity - event_field: response lookup_field: action
where error_patterns.csv contains:	pattern,severity,action error.*,high,alert warn.*,medium,log info.*,low,ignore

The processor will match the message content against each pattern and enrich the event with the corresponding severity and action.