Skip to main content
Version: 1.2.0

Example: Forwarding Data

This section will help you get started with targets to write processed data to specific formats, walking you through common use cases.

info

For a detailed discussion of targets, see this section.

Scenarios​

Send data received from Syslog to:

  • Console: the data will be sent first raw, then normalized to ECS
  • Storage File: the data will be converted to JSON storage format

Console Setup and Trial​

Loading include…

The most basic target to which we can direct our output is a console. In the previous section, to have the syslog data we have ingested—using the device we have configured there—directed to the console, we created a route like so:

routes:
  - name: syslog_to_console
    devices:
      - name: from_syslog
    targets:
      - name: to_console

Configure the target​

Assume, however, that we want to customize the stream directed to the console. For that purpose, we will modify our target configuration as:

to-console.yml
targets:
  - name: to_console
    type: console
    status: true
    properties:
      format: "ecs"
Loading include…

Put this configuration in the indicated file.

Note that we added a format property. With this setting, we will normalize our data to the ECS format which provides a standardized, consistent structure to query Web data.

note

Mapping the data fields of disparate log formats to widely known ECS fields is one of the commonly used normalization techniques.

Run the target​

To see the results on the console, go to the terminal you are using for Director, and invoke it by entering:

.\vmetric-director -background

This will start Director as a background process which you can verify as indicated before. Now enter:

.\vmetric-director -console

The -console switch will print status messages to the terminal.

Forward messages to the console​

Open another terminal to using our message sending tool, and start sending our familiar message:

  • Using generator mode
.\vmetric-director -generator -mode syslog -now -count 1 -address "127.0.0.1:514" -message "Hello world"
Loading include…

After sending a limited number of messages—say 5 of them—press Ctrl+C to exit the process.

If you now switch back to the other terminal, this time you will see that our message is printed to the console in ECS format we have specified:

{"message":"<3> 2025-06-02T10:38:02+03:00 VirtualMetric Test[7916]: Hello world","@timestamp":"2025-06-02T10:38:02+03:00"}

Storage File Setup and Trial​

The next type of output we can use is a file. Various formats are available in DataStream, but for the sake of simplicity, we will pick the widely used JSON format which is the default for the file target type:

targets:
  - name: to_json
    type: file
    status: true
    properties:
      location: "<vm_root>/Director/config/Examples"
      name: "from_syslog-{{.Year}}_{{.Month}}_{{.Day}}.json"

Create a file named to-json.yml in our working directory and save this code in it.

Loading include…

The path we have specified for location is where the JSON file will be created. (Do not forget to replace <vm_root> with the actual path of your VirtualMetric installation.) The nested name parameter is for the file: it will have a from_syslog_ prefix to which the internal field values of Year, Month, and Day will be appended.

note

Under properties, we did not specify any format since JSON is the default for the file target type.

To be able to use this target, we have to configure another route. Create a file named syslog-to-json.yml and place this new route definition in it:

syslog-to-json.yml
routes:
  - name: syslog_to_json
    devices:
      - name: from_syslog
    targets:
      - name: to_json

Forward messages to the output file​

Once again, run vmetric-director in generator mode (or other equivalent tool) to send messages to Syslog:

  • Using generator mode
.\vmetric-director -generator -mode syslog -now -count 1 -address "127.0.0.1:514" -message "Hello world"

To have this Syslog message directed to the storage file we have configured, check Director's status messages from your other terminal:

Director will issue prompts like the following while it is writing the ingested data to our JSON file:

[2025-06-02 11:15:58] [Information] [vmetric-director] Completed processing of to_json target logs. Number of processed logs: 1000

Now we can check our output. In our working directory, you will find a file with a name like from_syslog-2025_06_02.json. Open it with a text editor. You will find that the ingested data has been stored in it in a format like this:

{"epoch":1748852145,"message":"<3> 2025-06-02T11:15:45+03:00 VirtualMetric Test[5832]: Hello world"}

Since we have only sent the same message multiple times, there will be multiple copies of it in the file.

Monitoring​

Check that data forwarding worked by verifying:

  • JSON output file was created at the specified location
  • File contains the forwarded syslog messages
  • Log file indicated earlier shows no errors

If the output file exists and contains your test messages, data forwarding is working correctly.

In the next section, we will learn how Director handles pipelines.