MCP Server: Overview
VirtualMetric hosts a Model Context Protocol (MCP) server that exposes the DataStream pipeline engine to an AI coding agent. Once an agent is connected, it can run sample logs through a pipeline, compare the result against an expected fixture, check the output against a target schema, convert pipelines written for another platform, and read the authoring guides that describe DataStream's pipeline conventions.
The server is a managed remote service. There is nothing to install and nothing to run locally: you connect an agent to the hosted endpoint, and the agent drives the tools on your behalf.
The Same Engine as Production
The server runs the same service/pipeline engine that a director runs in --mode pipeline. A pipeline tested through the MCP server produces the same normalized output that a live director produces for the same input, so a result verified here does not need to be re-verified after deployment.
This is the property that makes the server useful for authoring rather than only for experimentation. The agent is not simulating DataStream's behavior from a description of it — it is executing the production code path against your configuration.
What an Agent Can Do
The tool surface divides into four groups, documented in full in Tool Reference:
| Group | Tools | Purpose |
|---|---|---|
| Testing | test_pipeline, validate_pipeline, diff_pipeline, validate_schema | Execute a pipeline against a sample log and check the result |
| Conversion | convert_cribl_pipeline, convert_kql_query, convert_logstash_pipeline, convert_pipeline_to_kql | Translate between DataStream pipelines and another platform's format |
| Catalog | list_processors, get_processor | Look up an available processor and its option schema |
| Guides | list_skills, get_skill | Retrieve the embedded pipeline-authoring guidance |
Every tool takes its pipeline YAML and sample logs inline, as string arguments — never as file paths. The agent composes a pipeline in the conversation and posts its text; the server holds nothing between calls. This is what allows an agent with no filesystem access to your environment to still author and verify a pipeline end to end.
What an Agent Cannot Do
The MCP server is a testing and authoring surface, not a control plane. It cannot create, modify, or deploy anything in your DataStream installation: there is no tool that writes a pipeline to the Pipeline Library, edits a route, or restarts a director. A pipeline an agent has produced and verified is text that you then install through the web interface or your configuration files.
Within pipeline execution itself, a further restriction applies. Processors that execute arbitrary code, reach the network, or mutate cluster-shared state are rejected before the pipeline runs. See Limits and Safety for the full set and for what to use instead when a pipeline you need to test contains one.
Where This Fits
DataStream offers three ways to exercise a pipeline against sample data, and they are complementary rather than alternatives:
| Surface | Use when |
|---|---|
| MCP Server | An agent is authoring or converting the pipeline and needs to iterate on it in text |
| Pipeline Debugger | You are working in the web interface and want to watch a pipeline execute processor by processor |
| Pipeline Management | You are importing a pipeline from another platform through the Create Pipeline wizard |
The MCP server and the Pipeline Debugger run the same engine, so a pipeline behaves identically in both.
Reading Order
- Connecting an Agent — the endpoint, the transport, and the client configuration.
- Tool Reference — every tool, its arguments, and what it returns.
- Limits and Safety — the processor guard, request limits, and transport behavior.
- Authoring Guides — the guidance documents the server ships to agents.
- Workflows — the end-to-end sequences these tools are designed for.