UDM
The Unified Data Model is the normalization schema of Google Security Operations. A UDM record organizes an event around a set of nouns—the actor, the thing acted upon, the network path between them—rather than a flat field list.
DataStream converts to and from UDM with the Normalize processor.
UDM carries no schema version. The metadata.product_version field records the version of the reporting device, not of the model.
Event Types
metadata.event_type classifies the event. It is never mapped from a source field—it is derived, and the derivation runs in two stages.
First, the event action is matched against the following:
| Action value | Event type |
|---|---|
networksession, connection, connect | NETWORK_CONNECTION |
http | NETWORK_HTTP |
dns, query | NETWORK_DNS |
login, logon | USER_LOGIN |
logout, logoff | USER_LOGOUT |
launch, start, processcreated | PROCESS_LAUNCH |
terminate | PROCESS_TERMINATION |
create | FILE_CREATION |
delete | FILE_DELETION |
modify | FILE_MODIFICATION |
write | FILE_WRITE |
read | FILE_READ |
Matching is case-insensitive and exact.
Second, if no action matched, the type is inferred from which fields are present. The first condition that holds wins:
| Condition | Event type |
|---|---|
| DNS questions or answers present | NETWORK_DNS |
| An HTTP method or response code present | NETWORK_HTTP |
| Principal IP, target IP and principal port all present | NETWORK_CONNECTION |
| A principal process ID and executable path present | PROCESS_LAUNCH |
| A target file path present, and the product event type mentions a file | FILE_CREATION |
| A principal user present, and the product event type mentions a login, authentication or session | USER_LOGIN |
| A target registry key present | REGISTRY_MODIFICATION |
Anything unresolved becomes GENERIC_EVENT.
The schema defines 51 event types and DataStream accepts all of them on input, but only the 14 reachable through the two stages above can be produced when converting from another format.
Field Mapping
UDM groups fields under nouns. The tables below give the UDM field produced for each ECS field.
Metadata
| ECS Field | UDM Field |
|---|---|
@timestamp | metadata.event_timestamp |
observer.vendor | metadata.vendor_name |
observer.product | metadata.product_name |
observer.version | metadata.product_version |
event.id | metadata.id |
event.extid | metadata.correlation_id |
event.action | metadata.product_event_type |
event.created | metadata.collected_timestamp |
event.start | metadata.interval.start_time |
event.end | metadata.interval.end_time |
event.module | metadata.product_deployment_id |
event.dataset | metadata.log_type |
event.owner | metadata.collected_by |
message | metadata.description |
Principal
The principal is the actor, and is populated from the ECS source entity.
| ECS Field | UDM Field |
|---|---|
source.ip | principal.ip |
source.port | principal.port |
source.hostname | principal.hostname |
source.mac | principal.mac |
source.domain | principal.administrative_domain |
source.nat.ip | principal.nat_ip |
source.nat.port | principal.nat_port |
source.service.name | principal.asset.software.name |
source.interface | principal.asset.network_interfaces.name |
source.user.name | principal.user.userid |
source.user.id | principal.user.windows_sid |
source.user.email | principal.user.email_addresses |
source.user.group | principal.user.group_identifiers |
source.user.roles | principal.user.attribute.roles |
source.process.name | principal.process.file.full_path |
source.process.id | principal.process.pid |
source.geo.country | principal.location.country_or_region |
source.geo.city | principal.location.city |
source.geo.region | principal.location.state |
Target
The target is the thing acted upon, populated from the ECS destination entity and from file, URL and registry fields.
| ECS Field | UDM Field |
|---|---|
destination.ip | target.ip |
destination.port | target.port |
destination.hostname | target.hostname |
destination.mac | target.mac |
destination.domain | target.administrative_domain |
destination.nat.ip | target.nat_ip |
destination.nat.port | target.nat_port |
destination.service.name | target.asset.software.name |
destination.user.name | target.user.userid |
destination.user.id | target.user.windows_sid |
destination.user.email | target.user.email_addresses |
destination.process.name | target.process.file.full_path |
destination.process.id | target.process.pid |
destination.geo.country | target.location.country_or_region |
destination.geo.city | target.location.city |
destination.geo.region | target.location.state |
file.path | target.file.full_path |
file.name | target.file.names |
file.size | target.file.size |
file.hash.md5 | target.file.md5 |
file.hash.sha1 | target.file.sha1 |
file.hash.sha256 | target.file.sha256 |
file.created | target.file.creation_time |
file.mtime | target.file.last_modification_time |
file.mime_type | target.file.mime_type |
url.full | target.url |
registry.key | target.registry.registry_key |
registry.value | target.registry.registry_value_name |
registry.data | target.registry.registry_value_data |
Intermediary
The intermediary is the device that reported the event.
| ECS Field | UDM Field |
|---|---|
observer.hostname | intermediary.hostname |
observer.ip | intermediary.ip |
observer.mac | intermediary.mac |
observer.name | intermediary.asset.asset_id |
observer.domain | intermediary.administrative_domain |
observer.nat.address | intermediary.nat_ip |
observer.process.id | intermediary.process.pid |
observer.process.name | intermediary.process.file.full_path |
Network
| ECS Field | UDM Field |
|---|---|
network.transport | network.ip_protocol |
network.bytes | network.total_bytes |
network.direction | network.direction |
network.community_id | network.session_id |
network.forwarded | network.http.x_forwarded_for |
source.bytes | network.sent_bytes |
source.packets | network.sent_packets |
destination.bytes | network.received_bytes |
destination.packets | network.received_packets |
event.duration | network.session_duration |
http.request.method | network.http.method |
http.response.status_code | network.http.response_code |
http.request.referrer | network.http.referral_url |
user_agent.name | network.http.user_agent |
dns.question.name | network.dns.questions.name |
dns.question.type | network.dns.questions.type |
dns.question.class | network.dns.questions.class |
dns.answers.type | network.dns.answers.type |
dns.answers.class | network.dns.answers.class |
dns.answers.ttl | network.dns.answers.ttl |
dns.resolved_ip | network.dns.answers.data |
dns.response_code | network.dns.response_code |
Security Result
| ECS Field | UDM Field |
|---|---|
event.severity | security_result.severity |
event.reason | security_result.description |
rule.id | security_result.rule_id |
rule.name | security_result.rule_name |
rule.version | security_result.rule_version |
rule.category | security_result.category |
rule.reference | security_result.url_back_to_product |
threat.framework | security_result.threat_name |
threat.indicator.type | security_result.category |
url.category | security_result.category_details |
Additional Fields
Any field without a place in the model above is written under additional.fields, with its dots replaced by underscores. process.args becomes additional.fields.process_args, syslog.priority becomes additional.fields.syslog_priority. This accounts for the majority of fields on a typical record.
Enumerations
security_result.action takes one of UNKNOWN_ACTION, ALLOW, BLOCK or ALERT. Words such as permit, accept and success produce ALLOW; deny, drop, reject and failure produce BLOCK; alert and detect produce ALERT. Anything unrecognized produces UNKNOWN_ACTION.
security_result.severity takes UNKNOWN_SEVERITY, INFORMATIONAL, LOW, MEDIUM, HIGH or CRITICAL. FATAL maps to CRITICAL and ERROR to HIGH. A value matching none of these is passed through unchanged rather than normalized.
network.direction takes UNKNOWN_DIRECTION, INBOUND, OUTBOUND or INTERNAL. When the source does not supply one, it is derived by testing the principal and target addresses against the private address ranges.
network.ip_protocol accepts TCP, UDP, ICMP, ICMPV6, SCTP, GRE, ESP and AH. A value of IPV4 or IPV6 describes the address family rather than the protocol and is moved to additional.fields.ip_version; any other value is preserved under additional.fields.network_ip_protocol_raw.
Always Written
Three fields are present on every converted record regardless of the source: metadata.event_type, which falls back to GENERIC_EVENT; metadata.event_timestamp, which falls back to the time of conversion; and security_result.action, which falls back to UNKNOWN_ACTION.
metadata.vendor_name and metadata.product_name are required by the model but are only written when the source supplies an observer vendor and product.
Limitations
- The output is not restricted to UDM fields. A source field with no mapping is carried through under its original name, so a converted record can contain both UDM fields and leftover fields from the source format.
- Longitude is not written. The geographic mappings name a latitude and a longitude field, and only the first is written, so
principal.location.region_longitudeandtarget.location.region_longitudeare always absent. - Converting back is lossy where several fields share a target.
event.originalandmessageboth producemetadata.description;event.reasonandrule.descriptionboth producesecurity_result.description;rule.categoryandthreat.indicator.typeboth producesecurity_result.category. The reverse conversion recovers one source field per target.