Fingerprint

Security Elastic Compatible

Synopsis

Generates a cryptographic hash of specified document fields to create a unique fingerprint.

Schema

fingerprint:
  - fields: <ident[]>
  - method: <string>
  - target_field: <ident>
  - salt: <string>
  - description: <text>
  - if: <script>
  - ignore_failure: <boolean>
  - ignore_missing: <boolean>
  - on_failure: <processor[]>
  - on_success: <processor[]>
  - tag: <string>

Configuration

Field	Required	Default	Description
fields	Y	-	Array of field names to include in the fingerprint
method	N	SHA-1	Hash algorithm: MD5, SHA-1, SHA-256, or SHA-512
target_field	N	fingerprint	Field to store the generated hash
salt	N	-	Additional value to include in the hash calculation
description	N	-	Explanatory note
if	N	-	Condition to run
ignore_failure	N	false	If true, errors are ignored
ignore_missing	N	false	If true, missing fields are skipped
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
tag	N	-	Identifier

Details

The processor supports multiple hash algorithms and can handle complex data types including nested objects and arrays. It ensures consistent hashing by sorting the object keys before hashing, using delimiters between values and converting various data types to bytes uniformly:

Strings	Direct byte conversion
Numbers	Binary representation
Booleans	1 for true, 2 for false
Arrays	All elements in order
Objects	Keys sorted alphabetically
Dates	Uses the RFC3339Nano format
Null	Empty bytes

note

For consistent hashing, use the same method and salt across comparisons. Consider field presence/absence with ignore_missing, and keep in mind that object key order is normalized.

warning

Object keys are included in the hash calculation for nested objects, and field order in the fields array doesn't affect the hash. Missing fields with ignore_missing set to false cause processor failure.

Common use cases:

• Document Deduplication

  fingerprint:
    - fields: ["content", "metadata"]
    - method: SHA-256
    - target_field: content_hash

• Change Detection

  fingerprint:
    - fields: ["user.profile", "user.settings"]
    - method: SHA-1
    - target_field: profile_hash

• Data Integrity

  fingerprint:
    - fields: ["payload"]
    - method: SHA-512
    - salt: "verification-key"

Examples

Basic

Hashing simple field values...	{ "field1": "value1", "field2": "value2" } fingerprint: - fields: ["field1", "field2"] - method: SHA-256 - target_field: doc_hash
creates base64 encoding:	{ "field1": "value1", "field2": "value2", "doc_hash": "vjAQ3j4wF3T7LdCXFNgBYAlpYAFBH5qgy5najgPpEyE=" }

Nested Objects

Hashing nested object fields...	{ "user": { "name": "Smith", "first_name": "John", "date_of_birth": "1980-01-15" } } fingerprint: - fields: ["user"] - method: SHA-1 - target_field: user_hash
includes all the nested values:	{ "user": { "name": "Smith", "first_name": "John", "date_of_birth": "1980-01-15" }, "user_hash": "TNN0UCl8wwOsdXK3R24U+ARZo0g=" }

Salt

Adding salt to the hash calculation...	{ "email": "[email protected]", "timestamp": "2024-01-01T00:00:00Z" } fingerprint: - fields: ["email", "timestamp"] - method: SHA-512 - salt: "my-secret-salt"
creates a salted hash:	{ "email": "[email protected]", "timestamp": "2024-01-01T00:00:00Z", "fingerprint": "hX9cW+YKDvE/R1k..." # truncated for brevity }

Missing Fields

Skipping missing fields...	{ "field1": "value1" } fingerprint: - fields: ["field1", "field2"] - ignore_missing: true - method: SHA-256
creates the hash from the available fields:	{ "field1": "value1", "fingerprint": "19o6lchfZXPT5hG6xb40xHikt3EOnrCVWrmSOPVMy0U=" }