Normalize

Transform

Synopsis

Converts log field names between different normalization formats (ECS, CIM, ASIM, CEF, LEEF, CSL).

Schema

normalize:
  - source_format: <string>
  - target_format: <string>
  - description: <text>
  - if: <script>
  - ignore_failure: <boolean>
  - on_failure: <processor[]>
  - on_success: <processor[]>
  - tag: <string>

Configuration

Field	Required	Default	Description
source_format	N	auto-detect	Source format (ecs, cim, asim, cef, leef, csl)
target_format	Y	-	Target format to convert to
description	N	-	Documentation note
if	N	-	Conditional expression
ignore_failure	N	false	Skip processing errors
on_failure	N	-	Error handling processors
on_success	N	-	Success handling processors
tag	N	-	Identifier for logging

Details

The processor automatically detects source formats when not specified and handles field name transformations while preserving values.

note

Format detection is case-insensitive - i.e. "ECS", "ecs", and "Ecs" are equivalent.

The processor automatically detects the source format based on characteristic fields. Some commonly used field mappings:

	ECS	CIM	ASIM
Network
	source.ip	src	SrcIp
	destination.ip	dest	DstIp
	network.direction	direction	NetworkDirection
Event
	@timestamp	_time	TimeGenerated
	event.type	action	EventType
	event.category	category	EventCategory
User
	user.name	user	ActorUsername
	user.id	user_id	ActorUserId
	user.domain	user_domain	ActorUserDomain

warning

Field mapping is non-reversible if the targeted format doesn't have equivalent fields. Test conversions with sample data.

Examples

ECS to CIM

ECS fields...	{ "source": { "ip": "128.232.110.120" }, "destination": { "ip": "192.168.1.1" }, "network": { "direction": "inbound" } } normalize: - source_format: ecs - target_format: cim
are mapped to CIM fields:	{ "src": "128.232.110.120", "dest": "192.168.1.1", "direction": "inbound" }

CIM to ECS

CIM fields...	{ "src": "128.232.110.120", "dest": "192.168.1.1", "direction": "outbound" } normalize: - source_format: cim - target_format: ecs
are mapped to ECS fields:	{ "source": { "ip": "128.232.110.120" }, "destination": { "ip": "192.168.1.1" }, "network": { "direction": "outbound" } }

Auto-detection

Auto-detection discovers CIM...	{ "_time": "2023-01-01T00:00:00Z", "src": "128.232.110.120", "dest": "192.168.1.1", "direction": "outbound" } normalize: - target_format: asim
and maps the fields to ASIM:	{ "TimeGenerated": "2023-01-01T00:00:00Z", "SrcIp": "128.232.110.120", "DstIp": "192.168.1.1", "NetworkDirection": "outbound" }

Error Handling

Handling conversion errors...	{ "source": { "invalid": true } } normalize: - target_format: cim - ignore_failure: true - on_failure: - set: - field: error - value: "Conversion failed"
captures the error information:	{ "source": { "invalid": true }, "error": "Conversion failed" }