Scripting: Overview

Scripting lets a pipeline do work no processor covers — a calculation, a conditional rewrite, a format-specific parse. DataStream runs scripts through the Script processor, which selects one of three engines with its lang field.

Engines

lang	Engine	Reference
javascript, js	Sandboxed JavaScript with a large built-in function library	JavaScript
golang, go	A sandboxed bytecode VM for a Go-like language	Go
vmetric	Named functions compiled into the product	Built-in Functions

lang has no default. An unrecognized value — including an empty string, and including painless, which earlier releases accepted — fails the processor with unsupported script language.

Choosing an engine

Reach for vmetric first: these functions are compiled into the product, skip script compilation entirely, and cover the transformations that recur across vendor pipelines. Where none fits, use javascript — it carries the widest function library and the most generous time budget. Use golang where a script is arithmetic-heavy and short; its 50-millisecond budget is twenty times tighter than the JavaScript one, but it compiles to bytecode rather than being interpreted.

Limits

Every engine is sandboxed. None of them can open a network connection, read a file of your choosing, or hold state between events.

Constraint	javascript	golang	vmetric
Time budget per event	1 second	50 milliseconds	Not applicable
Maximum script size	No limit	64 KiB	Not applicable
Allocation cap	None	50,000 objects	Not applicable
Module imports	None available	text, math, times, json, rand	Not applicable
State between events	None	None	None

The two script engines differ in what happens when a script fails partway through, and the difference is worth knowing before you write one:

• JavaScript writes to the event as the script runs. A script that modifies three fields and then throws leaves those three modifications in place.
• Go collects the event and writes it back only after the script completes. A script that fails or exceeds its budget leaves the event exactly as it arrived.

In both cases the event continues down the pipeline — a script failure does not drop it. The processor records the failure in the _ingest.on_failure_* fields and runs its on_failure chain, unless ignore_failure is set.

Where else scripting appears

Two other places accept JavaScript, both of them single expressions rather than script bodies:

• The Script processor's filter field — a truthiness test that decides whether the processor runs.
• The Eval processor's field values.

These use the expression form described under JavaScript, where a bare field name resolves against the event and the result is returned automatically.

warning

The if field is not JavaScript. Every processor's if uses DataStream's own expression language, described under Conditional Running. On the Script processor if and filter sit next to each other and take different languages.