Tool Reference

The server exposes thirteen tools in four groups. This page documents each one's arguments and what it returns. Two conventions apply to all of them and are described first, because most first-use problems come from missing them.

The Inline-Arguments Contract

Every tool takes pipeline YAML, sample logs, and expected fixtures as inline strings. None of them accepts a file path, and the server has no filesystem you can address. An agent reads a file on your machine if it has access to one, and posts its contents as the argument value.

The consequence for an agent's workflow is that the pipeline under test is whatever text was posted on that call. Nothing persists between calls: there is no session, no uploaded workspace, and no state to invalidate. Each call is complete on its own, which is why a conversation can iterate on a pipeline freely — every call re-posts the current version.

The Multi-Pipeline Pack Convention

A production pipeline often calls other pipelines. To test such a pack, put every pipeline into the single pipeline argument and set name to the entry pipeline.

The argument accepts three shapes:

One pipeline, as a mapping...

name: my_pipeline processors: - json: field: message

Several pipelines, as a top-level YAML list...

- name: entry processors: - pipeline: name: child - name: child processors: - set: field: handled value: true

Several pipelines, as separate YAML documents...

name: entry processors: - pipeline: name: child --- name: child processors: - set: field: handled value: true

The server decides which shape it received from the parsed YAML type, not from text matching: a document that parses to a sequence is expanded into one pipeline per element, and a document that parses to a mapping is taken as a single pipeline. Empty documents are skipped. A pipeline argument that yields no pipelines at all is rejected.

When several pipelines are posted, name selects which one receives the input. Omitting it leaves the entry point undetermined, so set it whenever the argument holds more than one pipeline.

Each pipeline is checked against the processor guard before any of them runs — see Limits and Safety.

Testing

test_pipeline

Runs a sample log through a pipeline and returns the normalized output as JSON.

Argument	Type	Required	Default	Description
pipeline	string	Y		Pipeline YAML: one pipeline, or several as a YAML list or ----separated documents
input	string	Y		Raw sample log — one event per line, or a JSON array
name	string	N		Entry pipeline name; set this when posting multiple pipelines
verbose	boolean	N	false	Include the full per-processor input/output trace

With verbose left at its default, the response reports only the steps that dropped or failed an event. Setting it to true returns the input and output of every processor in order, which is what you want when an event is being transformed but not in the way you expected.

validate_pipeline

Runs a sample log through a pipeline and reports whether the result matches an expected fixture.

Argument	Type	Required	Default	Description
pipeline	string	Y		Pipeline YAML: one pipeline, or several as a YAML list or ----separated documents
input	string	Y		Raw sample log — one event per line, or a JSON array
expected	string	Y		Expected output JSON — {"expected": [...]}, a bare array, or JSONL
name	string	N		Entry pipeline name; set this when posting multiple pipelines
verbose	boolean	N	false	Include the full per-processor input/output trace

The result is a pass or fail verdict. Use it as the regression check once a pipeline is known good; use diff_pipeline while you are still working out why it is not.

diff_pipeline

Runs a sample log through a pipeline and returns the expected output, the actual output, and a unified diff between them.

Argument	Type	Required	Default	Description
pipeline	string	Y		Pipeline YAML: one pipeline, or several as a YAML list or ----separated documents
input	string	Y		Raw sample log — one event per line, or a JSON array
expected	string	Y		Expected output JSON — {"expected": [...]}, a bare array, or JSONL
name	string	N		Entry pipeline name; set this when posting multiple pipelines
verbose	boolean	N	false	Include the full per-processor input/output trace

The arguments are identical to validate_pipeline. The difference is the response: a verdict against a diff of the two documents, which is the form that tells you which field is wrong.

validate_schema

Normalizes a sample log with a pipeline, then checks the resulting event against a target schema. This is the tool that answers whether a normalization pipeline actually produces the schema it claims to.

Argument	Type	Required	Default	Description
pipeline	string	Y		Pipeline YAML: one pipeline, or several as a YAML list or ----separated documents
input	string	Y		Raw sample log — one event per line, or a JSON array
schema	string	Y		Target schema reference
name	string	N		Entry pipeline name; set this when posting multiple pipelines
schema_type	string	N		Schema type hint
mode	string	N	both	Check mode — missing, extra, or both
recommended	boolean	N	false	Treat missing recommended fields as invalid
optional	boolean	N	false	Treat missing optional fields as invalid

The schema reference is schema-agnostic. It may name an ASIM table, an OCSF class written as ocsf: followed by the class name, a UDM reference written as udm: followed by the path, or a custom schema.

mode selects which direction is checked: missing reports fields the schema requires that the event lacks, extra reports fields the event carries that the schema does not define, and both reports each. A value outside these three is rejected before the pipeline runs, with the allowed set named in the error.

By default only required fields count towards a failure. Set recommended or optional to raise the bar to those tiers as well — useful when tightening a pipeline that already passes the required-field check.

The check runs against the routed, normalized event, so it reflects what a target would actually receive rather than the intermediate document. A pipeline that produces no output at all is reported as such rather than as a schema failure.

Conversion

The conversion tools translate a pipeline written for another platform into a native DataStream pipeline. They are the first step of a migration, not the whole of it: the returned YAML is an editable draft that you then verify with the testing tools above. See Migration for the surrounding guidance.

convert_cribl_pipeline

Converts a Cribl Stream pipeline export into a DataStream pipeline.

Argument	Type	Required	Default	Description
cribl	string	Y		The Cribl pipeline YAML to convert — an export or conf.yml with a top-level functions: list
name	string	N		Name for the resulting DataStream pipeline; Cribl bodies carry no name of their own

Disabled Cribl functions, and functions belonging to a disabled group, are converted into disabled processors rather than dropped, so the result is a faithful editable copy of the original rather than a subset of it.

convert_kql_query

Converts a KQL query or an ASIM parser into a DataStream pipeline.

Argument	Type	Required	Default	Description
query	string	Y		The KQL to convert — a raw query, or an ASIM parser YAML export
name	string	N		Name for the resulting DataStream pipeline

The input may be a bare KQL query or a full ASIM parser YAML export; when it is the latter, the parser query and name are extracted automatically, and name falls back to the parser's own name if you do not supply one.

convert_logstash_pipeline

Converts a Logstash pipeline configuration into a DataStream pipeline.

Argument	Type	Required	Default	Description
logstash	string	Y		The Logstash configuration to convert — a .conf file's contents with its input, filter, and output sections
name	string	N		Name for the resulting DataStream pipeline; Logstash configs carry no name of their own

Filter plugins become native processors and Logstash conditionals become native if: gates and groups. The input and output sections are summarized as comments rather than converted, because devices and targets are configured separately in DataStream and have no pipeline-level equivalent.

Plugins and options with no native equivalent degrade into comment processors, and a corresponding warning line is added to the pipeline's description. Read those before trusting the result — they mark exactly where the conversion could not carry the original's behavior across.

convert_pipeline_to_kql

Converts a DataStream pipeline back into KQL — the reverse of convert_kql_query.

Argument	Type	Required	Default	Description
pipeline	string	Y		The DataStream pipeline YAML to convert
envelope	boolean	N	false	Wrap the query in the full ASIM parser YAML envelope instead of emitting a bare parser query
parser_name	string	N		ASIM parser function name for the envelope; falls back to the pipeline's name

Processors with no faithful KQL lowering — reroute, stateful and external processors, and calls into another pipeline — are emitted as // UNSUPPORTED comments rather than silently omitted. Review every one of them: the emitted query is valid KQL, but it does less than the pipeline it came from wherever such a comment appears.

generate_schema_template

Converts a table schema into what Microsoft Sentinel needs to ingest the table through the Logs Ingestion API: the Data Collection Rule alone, or an ARM or Bicep template that also creates a custom _CL table. It is the generator behind the Library's Generate template drawer (see Library), and it reads nothing of a tenant.

Argument	Type	Required	Default	Description
schema	string	Y		The schema document inline, or the name of a built-in table (CommonSecurityLog, Syslog, ASimNetworkSessionLogs, an OCSF class…)
template	string	Y		dcr, arm or bicep
table_name	string	N	the built-in table's name	The table the rule sends to: a custom table when it ends in _CL, else an Azure table that accepts the Logs Ingestion API. Required for an inline schema
stream_name	string	N	Custom- and the table name less _CL	The stream the rule declares, which a sender posts to
transform_kql	string	N	source	The rule's transformation. A custom table's data without TimeGenerated gets the time of ingestion. Text in AdditionalFields is read as JSON with parse_json
dcr_name	string	N	vmetric-<table>-dcr	The rule's name
description	string	N		What the rule is for
data_collection_endpoint	string	N	a Direct rule	The data collection endpoint's name, in the rule's resource group, or its resource ID
workspace_name	string	N		The Log Analytics workspace (Microsoft Sentinel's)
subscription_id	string	N		The subscription of the workspace and the rule
resource_group	string	N		The resource group of the workspace and the rule
location	string	N		The workspace's region, which the rule must be in
table_plan	string	N	Analytics	A custom table's plan: Analytics, Basic or Auxiliary
retention_in_days	integer	N		An Analytics custom table's interactive retention, 4 to 730 days
total_retention_in_days	integer	N		A custom table's total retention, 4 to 4383 days

An inline schema can be any of:

• an Avro record;
• a VirtualMetric Parquet schema;
• a JSON Schema object;
• a Log Analytics table's JSON, as az monitor log-analytics workspace table show prints it;
• a JSON list of columns ([{"name": …, "type": …}, …]), as a table's columns or getschema's rows are exported;
• a Kusto column list, as .show table T cslschema prints it.

The result is the file name, the deploy command, the warnings, then the template. A DCR holds a placeholder for each Azure name left out. The warnings name the columns a custom table cannot have and say what to do before deploying.

Catalog

list_processors

Lists the available pipeline processors, each with its key and a one-line description.

Argument	Type	Required	Default	Description
search	string	N		Keyword filter; omit to list the full catalog

get_processor

Returns one processor's full option schema — each option's field name, type, and whether it is required — together with a link to its documentation page.

Argument	Type	Required	Default	Description
name	string	Y		Processor key, for example grok

Guides

list_skills

Lists the embedded authoring guides with their descriptions. Takes no arguments.

get_skill

Returns the full text of one authoring guide.

Argument	Type	Required	Default	Description
name	string	Y		Guide name, for example building-a-pipeline

The guides and what each covers are described in Authoring Guides.