Snowflake

Synopsis

Generates a unique Snowflake ID and stores it in a target field.

Schema

- snowflake:
    target_field: <ident>
    description: <text>
    if: <script>
    ignore_failure: <boolean>
    ignore_missing: <boolean>
    on_failure: <processor[]>
    on_success: <processor[]>
    tag: <string>

Configuration

The following fields are used to define the processor:

Field	Required	Default	Description
target_field	Y	-	Field to store the generated Snowflake ID
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	Continue processing if ID generation fails
ignore_missing	N	false	Skip if field doesn't exist
on_failure	N	-	Error handling processors
on_success	N	-	Success handling processors
tag	N	-	Identifier

Details

Generates a unique Snowflake ID and stores it in the specified target field. Snowflake IDs are 64-bit unique identifiers designed for distributed systems, combining a timestamp, worker ID, and sequence number to create globally unique values that are also roughly time-sortable.

note

Snowflake IDs are 64-bit integers typically represented as strings. They are time-sortable (IDs created later will generally be larger than IDs created earlier), which makes them useful for chronological sorting in databases.

The processor is useful for assigning unique identifiers to log entries, tracking events across distributed systems, and creating correlation IDs without coordination between nodes.

warning

While Snowflake IDs are designed to be unique across distributed systems, they should not be used for cryptographic purposes. If you need IDs for security-sensitive operations, consider using a dedicated cryptographic random generator.

Examples

Basic

Generating a basic Snowflake ID...	{ "event": { "type": "login" } } - snowflake: target_field: event.id
adds a unique identifier:	{ "event": { "type": "login", "id": "1637812502315610112" } }

Nested Fields

Generating ID in a nested structure...	{ "transaction": { "amount": 29.99, "currency": "USD" } } - snowflake: target_field: transaction.metadata.id
creates nested fields automatically:	{ "transaction": { "amount": 29.99, "currency": "USD", "metadata": { "id": "1637812635481661440" } } }

Correlation

Generating correlation ID for distributed tracing...	{ "service": { "name": "payment-gateway" }, "request": { "method": "POST", "path": "/api/v1/payments" } } - snowflake: target_field: trace.id if: "!hasField(logEntry, 'trace.id')"
adds tracking ID only if not already present:	{ "service": { "name": "payment-gateway" }, "request": { "method": "POST", "path": "/api/v1/payments" }, "trace": { "id": "1637812728377626624" } }

Multi-ID

Generating multiple IDs for different purposes...	{ "user": { "email": "[email protected]" }, "session": {} } - snowflake: target_field: session.id - snowflake: target_field: transaction.id - set: field: event.metadata.generated_at value: "{{@timestamp}}"
creates multiple unique identifiers:	{ "user": { "email": "[email protected]" }, "session": { "id": "1637812912861540352" }, "transaction": { "id": "1637812912907677696" }, "event": { "metadata": { "generated_at": "2023-07-30T15:42:19.123Z" } } }