YARA-L

Security Threat Intelligence

Synopsis

Evaluates Google SecOps (Chronicle) YARA-L 2.0 detection rules against each event, annotating matches or filtering to them.

Schema

- yaral:
    library: <string[]>
    rules: <string[]>
    rule_files: <string[]>
    rule_dirs: <string[]>
    select: <YaralSelect>
    source_format: <enum>
    mark_field: <ident>
    mode: <enum>
    multi_event: <enum>
    min_severity: <enum>
    mitre_enrich: <boolean>
    reference_lists: <map>
    proto_defaults: <boolean>
    nocase: <boolean>
    include_rule: <boolean>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>

Configuration

Field	Required	Default	Description
library	Y*	-	Platform Library entry names. Accepts globs — * for every entry, win_* for a prefix
rules	Y*	-	Inline rule text
rule_files	Y*	-	Paths to rule files
rule_dirs	Y*	-	Paths to directories of rule files
select	N	-	Filters the loaded rules by metadata. See below
source_format	N	auto-detect	Pins the schema the rules' UDM paths resolve onto: udm, ecs, cim, asim, csl, cef, leef, ocsf, gelf or native
mark_field	N	_yaral	Field the match annotation is written under
mode	N	"annotate"	annotate enriches matching events and passes everything through; filter drops events that match no rule
multi_event	N	"skip"	What to do with correlation rules: skip or candidate. See below
min_severity	N	-	Drop matches below this severity: informational, low, medium, high, critical
mitre_enrich	N	true	Scan rule metadata for ATT&CK references and add them to the mark
reference_lists	N	-	Membership sets for the rules' %list_name references
proto_defaults	N	true under udm, else false	Treat an unpopulated scalar as its zero value, as UDM protobuf semantics do
nocase	N	false	Fold every string comparison, not only those written nocase
include_rule	N	false	Add the rule's source text to the mark. Rule bodies are large, so this is opt-in
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	See Handling Failures
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
tag	N	-	Identifier
disabled	N	false	When true, the processor is skipped and the event continues to the next one. Lets you take a processor out of the path without removing its configuration

* = At least one of library, rules, rule_files or rule_dirs is required. All four are unioned.

warning

ignore_failure: true silently turns this processor off. The drop is signalled to the pipeline as an error value, and the ignore_failure check runs before that value is inspected — so a matched event is kept instead of dropped, no error is logged, and the pipeline reports success. The processor appears to run normally while filtering nothing.

Use it only if you genuinely want a pass-through. To tolerate real errors without losing the filtering, leave ignore_failure unset and handle the failure with on_failure instead.

Select

All set criteria are AND-linked; within one criterion the list is OR-linked.

Field	Description
severities, priorities	Rule meta severity / priority is in the set
authors	Rule meta author is in the set
names	Rule name matches exactly
name_prefix, name_contains, name_regex	Rule-name matching
desc_contains	Substring of the rule's meta description
meta_contains	Any meta value contains any of these substrings, case-insensitively
meta	Exact key=value meta pairs, all of which must be present. Keys and values case-insensitive
single_event_only	Keep only rules decidable from a single event

meta_contains is the practical way to select by ATT&CK reference: rule packs spell the key at least nine different ways (mitre_attack_technique, mitre.technique, technique, …), and this searches the values instead of guessing the key.

Details

This is YARA-L, Google SecOps' detection language — not byte-pattern YARA. For file-content matching see YARA.

One event at a time

A streaming processor sees one event at a time, so only rules decidable from a single event are evaluated — one event variable, no match: section, a condition a single occurrence can satisfy. Correlation rules are handled by multi_event:

Value	Behavior
skip	Default. Correlation rules are not evaluated, and are reported once at load. Enrichment fails closed: a detection that cannot be proven is not stamped
candidate	The event is marked when it satisfies any of the rule's event-variable predicates — it could contribute to a future match

A candidate entry is explicitly not a detection, and the mark says so: the entry carries candidate: true and, where known, candidate_reason. That is deliberate, so a downstream consumer cannot mistake a triage hint for a proven match.

note

The same rule packs feed the Smart Engine processor, where the split is the mirror image: reduction fails open, because dropping a true detection is the worse error there. The difference is not an inconsistency — annotation and reduction have opposite safe defaults.

Reference lists

reference_lists supplies the %list_name sets a rule refers to. A list a rule uses but the map does not define is UNKNOWN, not empty — the rule then enriches nothing, failing closed. An empty list would look like a working rule that can never fire, which is the harder failure to notice.

A value of the form lookup:<name> expands to the first column of the named platform lookup table, so an existing threat-intel or asset CSV can back a reference list without being copied here. The expansion happens once, when the rule set is compiled.

proto_defaults

UDM protobuf semantics read an unpopulated scalar as its zero value, which makes $e.field != "x" true for an event that has no such field. That is faithful under UDM and defaults to on there — but under any other schema it would turn a rule's negative predicates into match-everything, so it defaults to off. Set it explicitly to override.

The mark

Field	Contents
_yaral.matched	true on a matching event
_yaral.rules	One entry per matched rule
_yaral.count	The number of matched rules

Each entry carries name, and — when the rule defines them — severity, priority, description, author and mitre. A correlation-rule candidate adds candidate and candidate_reason; include_rule adds the rule source as rule.

If mark_field collides with an existing non-object value the annotation cannot be written. The processor warns once, so the loss is discoverable rather than silent.

Examples

Annotating Matches

Running a Library rule pack against every event...	- yaral: library: ["win_*"] min_severity: medium
matching events are enriched, and everything passes through:	{ "message": "suspicious service install", "_yaral": { "matched": true, "count": 1, "rules": [ { "name": "win_service_install_suspicious", "severity": "high", "mitre": {"technique": "T1543.003"} } ] } }

Filtering to Detections

In filter mode, events matching no rule are dropped...	- yaral: library: ["*"] mode: filter min_severity: high
so only high-severity detections continue down the pipeline:	# Use this to route detections to a SIEM while # the raw stream goes to cheaper storage.

Selecting by ATT&CK Reference

meta_contains searches meta VALUES, so the key spelling does not matter...	- yaral: library: ["*"] select: meta_contains: ["T1053", "T1543"] severities: ["high", "critical"]
both criteria must hold, and within each the list is an OR:	# Keeps high or critical rules that reference # either technique, however the pack spells the meta key.

Backing a Reference List with a Lookup

Reusing an existing lookup table instead of copying its contents...	- yaral: library: ["*"] reference_lists: known_bad_domains: ["lookup:threat_intel_domains"] admin_accounts: ["svc_backup", "svc_deploy"]
the first column of the table becomes the set:	# A rule using %known_bad_domains resolves against the # lookup table. A rule using a list NOT defined here # enriches nothing rather than never matching.