Skip to main content

CEF to CSL

Transform

Synopsis​

Parses a CEF message and maps it directly to Microsoft Sentinel's Common Security Log (CommonSecurityLog) schema in one fused pass, replacing the manual cef + normalize + move (+ enforce_schema) processor chain.

note

For details of CEF, see Appendix.

Schema​

- cef_to_csl:
field: <ident>
use_table: <boolean>
use_kv_parser: <boolean>
description: <text>
if: <script>
tag: <string>
disabled: <boolean>
ignore_failure: <boolean>
on_failure: <processor[]>
on_success: <processor[]>

Configuration​

The following fields are used to define the processor:

FieldRequiredDefaultDescription
fieldYField containing the raw CEF message to convert. Must be a string starting with CEF:
use_tableNfalseWrite the mapped fields into the typed CommonSecurityLog virtual table instead of the event map. The table is created first if no create_table ran for it
use_kv_parserNfalseParse the CEF extension field with the generic key-value parser instead of the built-in CEF-aware extension scanner
descriptionNExplanatory note
ifNCondition to run
tagNIdentifier
disabledNfalseSkip this processor
ignore_failureNfalseSee Handling Failures
on_failureNSee Handling Failures
on_successNSee Handling Success

Details​

cef_to_csl parses the raw CEF message and hands the resulting fields straight to the internal cef→csl mapping in one pass, without ever materializing an intermediate cef object, a separate normalize step, or a move-to-root step. The result already reflects the same CSL schema enforcement that a manual cef + normalize + move + enforce_schema chain would produce—no separate enforce_schema step is needed afterward, and ignore_rules-style schema enforcement is not configurable on this processor; it always applies.

field must resolve to a string starting with CEF:. If the field is missing, is not a string, or does not start with CEF:, the processor fails. There is no ignore_missing option on this processor—unlike the plain cef parsing processor, a missing field is handled the same way as any other failure, through ignore_failure/on_failure.

use_table: false (default)​

The mapped CSL fields replace the entire event map. Every field that existed on the event before this processor ran—including the source field itself—is discarded, except system (_vmetric) fields and any virtual tables already created on the entry.

use_table: true​

Nothing is written to the event map. Instead, the mapped fields are written directly into the typed CommonSecurityLog virtual table, and the log entry's output source switches to that table. If no create_table ran for it, the processor creates the table first, exactly as a bare create_table would: empty, with no from_map. A parse that fails leaves no table behind that it did not find. Truncating and dropping the table remain the caller's job. The event map itself is left untouched. Field-for-field, the values written are identical to the use_table: false path—only the storage target differs.

use_kv_parser​

Selects a generic key=value parser for the CEF extension field (the part of the message after the seven pipe-delimited header fields) instead of the built-in CEF-aware extension scanner. leef_to_csl has no equivalent option—LEEF's body is always parsed by a single dedicated LEEF tokenizer.

Unmapped fields​

CEF extension fields with no corresponding CSL schema column are folded into AdditionalExtensions as key=value pairs joined with ;, in the order they appear in the source message.

Distinct from the cef processor​

cef_to_csl is not the CEF parser. The cef processor parses a CEF message into a structured object at target_field (leaving the rest of the event untouched) and supports target_field/ignore_missing; cef_to_csl skips that intermediate object entirely and converts straight to CSL, replacing the event map (or writing to a table) and supporting neither target_field nor ignore_missing. For LEEF, see the sibling LEEF to CSL processor.

Examples​

Basic Conversion​

Converting a raw CEF message straight to CSL fields...

{
"message": "CEF:0|Check Point|VPN-1 & FireWall-1|Check Point|Detect|Address spoofing|Unknown|act=Detect deviceDirection=0 msg=Address spoofing rt=171436202000 spt=50740 dpt=53 ifname=eth1-01.7 loguid={0x663270c7,0x3f,0xf76511ac,0x1b933721} origin=172.17.11.241 originsicname=CN\\=user ,O\\=user.com sequencenum=679 version=5 dst=172.17.101.41 product=VPN-1 & FireWall-1 proto=17 src=192.168.10.9"
}
- cef_to_csl:
field: message

replaces the entire event map with the mapped CSL fields; unmapped extensions fold into AdditionalExtensions...

{
"Activity": "Address spoofing",
"AdditionalExtensions": "ifname=eth1-01.7;loguid={0x663270c7,0x3f,0xf76511ac,0x1b933721};origin=172.17.11.241;originsicname=CN\\=user,O\\=user.com;sequencenum=679;version=5;product=VPN-1 & FireWall-1",
"CommunicationDirection": "0",
"DestinationIP": "172.17.101.41",
"DestinationPort": 53,
"DeviceAction": "Detect",
"DeviceEventClassID": "Detect",
"DeviceProduct": "VPN-1 & FireWall-1",
"DeviceVendor": "Check Point",
"DeviceVersion": "Check Point",
"LogSeverity": "0",
"Message": "Address spoofing",
"Protocol": "17",
"ReceiptTime": "171436202000",
"SimplifiedDeviceAction": "Detect",
"SourceIP": "192.168.10.9",
"SourcePort": 50740
}

Writing into the CommonSecurityLog Table​

Creating the typed table first, then converting with use_table: true...

{
"message": "CEF:0|Fortinet|Fortigate|v6.0.3|20503|utm:emailfilter smtp log-only|2|deviceExternalId=FGT5HD3915800610 FTNTFGTlogid=0508020503 cat=utm:emailfilter FTNTFGTsubtype=emailfilter FTNTFGTeventtype=smtp FTNTFGTlevel=information FTNTFGTvd=vdom1 FTNTFGTeventtime=1545939418 FTNTFGTpolicyid=1 externalId=1135 duser=bob src=10.1.100.11 spt=35969 deviceInboundInterface=port12 FTNTFGTsrcintfrole=undefined dst=172.18.62.158 dpt=25 deviceOutboundInterface=port11 FTNTFGTdstintfrole=undefined proto=6 app=SMTP FTNTFGTprofile=test-spam act=log-only suser=testpc1@qa.fortinet.com duser=test1@server88.qa.fortinet.com FTNTFGTsender=testpc1@qa.fortinet.com FTNTFGTrecipient=test1@server88.qa.fortinet.com deviceDirection=1 msg=general email log FTNTFGTsubject=hello_world2 FTNTFGTsize=216 FTNTFGTattachment=no"
}
- create_table:
name: CommonSecurityLog
- cef_to_csl:
field: message
use_table: true

writes the same mapped fields into the CommonSecurityLog table instead of the event map; the event map is left untouched and output now marshals from the table...

{
"Activity": "utm:emailfilter smtp log-only",
"AdditionalExtensions": "FTNTFGTlogid=0508020503;FTNTFGTsubtype=emailfilter;FTNTFGTeventtype=smtp;FTNTFGTlevel=information;FTNTFGTvd=vdom1;FTNTFGTeventtime=1545939418;FTNTFGTpolicyid=1;FTNTFGTsrcintfrole=undefined;FTNTFGTdstintfrole=undefined;FTNTFGTprofile=test-spam;FTNTFGTsender=testpc1@qa.fortinet.com;FTNTFGTrecipient=test1@server88.qa.fortinet.com;FTNTFGTsubject=hello_world2;FTNTFGTsize=216;FTNTFGTattachment=no",
"ApplicationProtocol": "SMTP",
"CommunicationDirection": "1",
"DestinationIP": "172.18.62.158",
"DestinationPort": 25,
"DestinationUserName": "test1@server88.qa.fortinet.com",
"DeviceAction": "log-only",
"DeviceEventCategory": "utm:emailfilter",
"DeviceEventClassID": "20503",
"DeviceExternalID": "FGT5HD3915800610",
"DeviceInboundInterface": "port12",
"DeviceOutboundInterface": "port11",
"DeviceProduct": "Fortigate",
"DeviceVendor": "Fortinet",
"DeviceVersion": "v6.0.3",
"ExtID": "1135",
"LogSeverity": "2",
"Message": "general email log",
"Protocol": "6",
"SimplifiedDeviceAction": "log-only",
"SourceIP": "10.1.100.11",
"SourcePort": 35969,
"SourceUserName": "testpc1@qa.fortinet.com"
}

Error Handling​

Handling a field that isn't a valid CEF message...

{
"message": "Invalid CEF message"
}
- cef_to_csl:
field: message
on_failure:
- append:
field: tags
value: cef_to_csl_error

leaves the event map untouched and adds an error tag instead of failing the pipeline...

{
"message": "Invalid CEF message",
"tags": ["cef_to_csl_error"]
}