Unit Tests

Synopsis

A unit test pins what a pipeline does to a known input. It is a set of log lines, the environment they run in, and the record each line is expected to produce. Running the test feeds every line through the pipeline on its own and compares the result against its expectation.

Tests are saved with the pipeline and shared with everyone working on it, so a test written once keeps holding after you have moved on. An organization can require that they all pass before a commit is merged — see Blocking a Merge on Tests.

Unit Tests is a section of the Pipeline Editor rail. Each test belongs to one pipeline tree.

Creating a Test

With no tests yet, the section offers two starting points:

• Create from debugger input — takes the lines and environment currently loaded in the debugger, so a session you have already set up becomes a test without retyping it
• Create an empty test — starts from nothing

Add new test adds further tests. A test carries:

Field	Notes
Test name	Names the test in the list and in the merge summary
Description	What the test covers
Entry pipeline	Which pipeline in the tree the lines are fed to. Defaults to the root
Environment	Device type, Definition ID, Request, and Enable system fields

The entry pipeline is stored as a name, not an identifier, so a test survives a re-import of the tree. If the named pipeline is no longer there, the field says so and asks you to choose again.

Input and Expectations

The test has two halves, editable as rows or as whole files.

Input is a log, verbatim. Each non-empty line is one event, processed independently, in the order the lines appear. A blank line is not run, but it keeps its slot so the expectations stay aligned to line numbers.

Expected is one entry per input line. The Files view shows it as the JSON document it is stored as; the Rows view pairs each input line with its expectation, which is where most editing happens.

Assert Modes

An expectation carries a claim about the line, recorded separately from the expectation itself:

Mode	The row says
Asserted	The pipeline must produce this record
Expects a drop	The pipeline must drop this event
Not asserted	Nothing is claimed; the line runs but its output is not checked

The distinction matters because an empty expectation is ambiguous on its own. An author writing {} may mean "I have not filled this in yet" or "this event is dropped" — opposite claims, and a merge gate must not confuse them. Storing the mode alongside removes the guess.

Expect a drop and Clear expectation on a row's action menu set the mode; Accept actual fills the expectation from the last run's output and asserts it, and Accept all actual does that for every row at once. Copy expected file takes the whole expectation document.

An expectation with no matching input line is never checked, and the section warns when there are any.

What Counts as a Difference

Comparison follows the same rules as the test harness that runs against the product's own vendor packs, so a test authored here reaches the same verdict as one run in the build:

• Records are compared field by field, addressed by dotted path — a.b.c. Nesting is not itself a difference, and an object with no fields is the same as no field at all.
• Fields under _vmetric are excluded. A pipeline writes to that namespace freely and no expectation has to know about it. Include them with Enable system fields.
• A top-level field whose value is an empty string reads as not set and is dropped from both sides before comparing. Nested empty strings are compared normally.
• Values are compared without regard to case. Field names are not — a field name that changed case is reported as a difference.
• An array is a single value: its order and its length are part of what is compared.

Running

Run runs the selected test; Run all runs every test in the tree, and Cancel run stops in place. Lines are executed one at a time through the same debug session the debugger uses, so progress is reported per line, and for Run all per test as well.

A test whose entry pipeline has no debug session cannot run, and the section says so. This is the case for a child pipeline that has never been committed — it has no session of its own until it exists in a merged pipeline.

Each test reports a status:

Status	Meaning
Passed	Every asserted line matched
Failed	At least one asserted line differed
Error	The run could not produce a verdict
Never run	No run has been recorded
Stale	The last result predates the pipeline's last save

A stale result is not a failure and not a pass — it was produced against content the pipeline has since moved past. Run the test again to turn it back into a verdict.

Per row, the result is reported as a pass, a count of differing fields, a drop that was asserted, or a line that carried no expectation. A row that differs expands into a field-by-field table of expected against actual, marking each difference as missing, unexpected, or changed.

Records produced by a route rather than by the pipeline itself are reported but not asserted.

Run History

The History view lists a test's runs, newest first, with the time taken. Opening one compares its stored output against the expectation as it stands now, which is what makes history useful for a bisect: an expectation edited since the run is applied to the old output.

Runs are kept for seven days. Older ones are removed, so history is a short-term working record rather than an audit trail.

Blocking a Merge on Tests

When an organization turns on the merge requirement, a commit can only be merged once every test in its pipeline tree has last passed. The merge dialogs show a Unit tests summary counting passed against total and listing what is blocking.

Only a pass clears a test. Failed, errored, and never run all block — a test nobody has run establishes nothing about the change.

Two properties are worth knowing:

• A tree with no tests merges as it always did. The requirement is that existing tests pass, not that tests exist.
• A stale pass does not clear the gate. A pass counts only when it was recorded against the version being merged; one recorded against content the pipeline has since moved past is listed as blocking, alongside the failures and the never-run.
• The merge screen re-runs what it cannot accept. Before deciding, it runs every test that is not already a fresh pass — the red ones, because they may have been fixed since, and the stale greens, because that is the case this exists for.

The requirement is set in Content Settings and enforced by the server on both merge paths. The server reads each test's last recorded status; the staleness check above is applied by the merge screen before it lets the request through.

Permissions

Reading and running tests requires PIPELINE_READ; creating, editing, and deleting them requires PIPELINE_EDIT. Running is deliberately a read: it changes no pipeline and no test definition, so someone who can look at a pipeline can also confirm what it does.

Without edit permission the section is read-only and says so — the tests are visible and runnable, and nothing can be changed.