CSV

Parse Elastic Compatible

Synopsis

Parses CSV (Comma-Separated Values) data from a field and assigns the extracted values to specified target fields.

Schema

csv:
  - field: <ident>
  - target_fields: <ident[]>
  - description: <text>
  - empty_value: <string>
  - if: <script>
  - ignore_failure: <boolean>
  - ignore_missing: <boolean>
  - on_failure: <processor[]>
  - on_success: <processor[]>
  - quote: <char>
  - separator: <char>
  - tag: <string>
  - trim: <boolean>

Configuration

Field	Required	Default	Description
field	Y	-	Field containing the CSV string to parse
target_fields	Y	-	Array of field names to assign the parsed values to
description	N	-	Explanatory note
empty_value	N	-	Value to use for empty fields. If not specified, empty fields remain empty
if	N	-	Condition to run
ignore_failure	N	false	Continue processing if parsing fails
ignore_missing	N	false	Skip processing if source field doesn't exist
on_failure	N	-	See Handling Failures
on_success	N	-	See Handling Success
quote	N	"	Character used for quoting fields
separator	N	,	Character used to separate fields
tag	N	-	Identifier
trim	N	false	Trim whitespace from unquoted fields

Details

The processor reads a single line of CSV data and maps each field to the corresponding target field. It can handle quoted values, custom separators, quote characters, and provides options for trimming whitespace and handling empty fields.

note

The processor expects the CSV data to be a single line. For multi-line CSV processing, consider using other processors, or pre-processing the data first.

warning

If the number of fields in the CSV data is less than the number of target fields, the processor will fail unless ignore_missing is set to true.

Examples

Basic CSV Parsing

Parsing simple quoted CSV data...	{ "csv_data": "\"data1\", \"data2\", \"data3\"" } csv: - field: csv_data - target_fields: ["field1", "field2", "field3"] - trim: true
creates new fields:	{ "csv_data": "\"data1\", \"data2\", \"data3\"", "field1": "data1", "field2": "data2", "field3": "data3" }

Handling Empty Values

Filling empty fields with default values...	{ "csv_data": "data1,,data3" } csv: - field: csv_data - target_fields: ["field1", "field2", "field3"] - empty_value: "N/A"
result:	{ "csv_data": "data1,,data3", "field1": "data1", "field2": "N/A", "field3": "data3" }

Custom Separator

Using semicolon as the separator...	{ "csv_data": "data1;data2;data3" } csv: - field: csv_data - target_fields: ["field1", "field2", "field3"] - separator: ";"
parses the values:	{ "csv_data": "data1;data2;data3", "field1": "data1", "field2": "data2", "field3": "data3" }