VictoriaMetrics

Databases Analytics

Synopsis

Creates a VictoriaMetrics target that writes each record as one point of InfluxDB line protocol, through the InfluxDB write API of VictoriaMetrics. It writes to a single node, or to a tenant of a cluster.

To send metrics to VictoriaMetrics with the Prometheus remote-write protocol instead, use the Prometheus remote-write target.

Schema

- name: <string>
  description: <string>
  type: victoriametrics
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    address: <string>
    port: <integer>
    database: <string>
    tenant: <string>
    token: <string>
    username: <string>
    password: <string>
    measurement_field: <string>
    measurement: <string>
    tags: <string[]>
    fields: <string[]>
    batch_size: <integer>
    max_bytes: <integer>
    timeout: <integer>
    field_format: <string>
    tls:
      status: <boolean>
      verify: <boolean>
      ca_name: <string>
      cert_name: <string>
      key_name: <string>
      passphrase: <string>
      server_name: <string>
      min_tls_version: <string>
      max_tls_version: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be victoriametrics
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Connection

Field	Required	Default	Description
address	Y	-	Host name or IP address of a single node, or of the vminsert service of a cluster
port	N	8428	Port of the server. The vminsert service of a cluster usually listens on 8480, so a cluster sets port
token	N	-	A bearer token, for example for vmauth in front of VictoriaMetrics
username	N	-	The user the target signs in as with HTTP Basic, when no token is set
password	N	-	Password for that user
timeout	N	30	Seconds one request may take. At least 1

Where the Points Land

Field	Required	Default	Description
database	N	-	VictoriaMetrics keeps this name as the db label of each series. When unset, no label is added
tenant	N	-	The tenant of a cluster, as accountID or accountID:projectID. Each one is a whole number. When unset, the target writes to a single node

Point Mapping

These four fields apply to every record that the InfluxDB device did not write. See How a Record Becomes a Point.

Field	Required	Default	Description
measurement_field	N	event.dataset	The record field whose text names the measurement
measurement	N	record	The measurement when that field is missing or empty
tags	N	-	The record fields that become tags. A list, or one text with the paths separated by commas
fields	N	-	The record fields that become fields. When empty, every number and boolean in the record

Delivery

Field	Required	Default	Description
batch_size	N	5000	The most lines in one request
max_bytes	N	10485760 (10 MB)	The most bytes of line protocol in one request, before compression
field_format	N	-	Converts each record to a format such as ECS or OCSF before it is mapped. See applicable Normalization section

note

max_bytes caps the size of a single request body or batch. It behaves differently from max_size on the file-writing targets, and the difference is easy to get wrong:

• An explicit max_bytes: 0 IS honoured and removes the ceiling on targets that have no ceiling of their own. The value is read with Int64, which returns what you wrote — unlike max_size, which is read with MustInt64 and silently substitutes its default for a zero. Where a target has a fixed ceiling of its own, its page says so and a zero is clamped to that ceiling instead.
• A negative value is rejected at configuration time with max_bytes must not be negative.

The ceiling is validated when the target is built and enforced again on every batch, so a value the receiving service will refuse fails early rather than per request.

TLS

Field	Required	Default	Description
tls.status	N	false	Use HTTPS. TLS stays off unless this is true

TLS material is resolved through the shared client builder, so these keys mean the same thing on every target that uses it. They are nested under a tls: block.

Field	Required	Default	Description
tls.verify	N	true	Verify the server certificate. On by default — set it to false only to accept an untrusted certificate, and only where you control the network path
tls.server_name	N	-	SNI hostname override for the handshake. Use it when the certificate's name does not match the address you connect to
tls.ca_name	N	-	CA bundle used to verify the server certificate. When unset the host trust store is used; when set it replaces the host trust store rather than adding to it
tls.cert_name	N*	-	Client certificate, for mutual TLS
tls.key_name	N*	-	Client private key, for mutual TLS
tls.passphrase	N	-	Passphrase for an encrypted client private key
tls.min_tls_version	N	tls1.2	Lowest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)
tls.max_tls_version	N	tls1.3	Highest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)

* cert_name and key_name are individually optional but must be supplied together — a certificate without its key, or a key without its certificate, is a configuration error.

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Details

The VictoriaMetrics target writes each record as one point of InfluxDB line protocol, and sends each batch in one HTTP request. The body is gzip compressed. The target sends no precision setting, because the write API reads nanoseconds by default.

It shares its sender with the InfluxDB target, so it maps records the same way.

Single Node or Cluster

• A single node, when tenant is unset: the target writes to /write on port 8428 by default.
• A cluster, when tenant is set: the target writes to /insert/<tenant>/influx/write on the vminsert service, which usually listens on port 8480. Set port for a cluster, because the default is the single node's.

database works in both: VictoriaMetrics keeps it as the db label of each series.

How a Record Becomes a Point

• A record that the InfluxDB device wrote keeps its measurement, its tags and its fields. Each field keeps its type: a float, an integer, an unsigned integer, a boolean or a string. The point mapping fields do not apply to it.
• Any other record is mapped. The measurement is the text at measurement_field, or measurement when that field is missing or empty. Each field that tags lists becomes a tag, and each field that fields lists becomes a field, named by its path. When fields is empty, every number in the record becomes a float field and every boolean becomes a boolean field. Numbers in arrays, the record's time and the fields used as tags are left out.
• The time of the point is the record's @timestamp, in nanoseconds. A record without one gets the server's time. VictoriaMetrics stores timestamps in milliseconds, so it keeps no precision finer than a millisecond.
• A record that gives no field is skipped and counted as dropped. The Director logs a warning when no record of a batch gives a field.
• VictoriaMetrics stores every field as a number. It stores true as 1 and false as 0. A string that holds a number, such as "42", is stored as that number, and any other string is stored as 0. To keep a text value, send it as a tag: VictoriaMetrics keeps each tag as a label of the series.

Prerequisites

1. A running VictoriaMetrics single node, or the vminsert service of a cluster, reachable from the Director host
2. The token or the user that vmauth, or the proxy in front of the server, expects, if it asks for one

Examples

Single Node

Writing to a single node, with a db label on each series:

targets:
  - name: victoriametrics_single
    type: victoriametrics
    properties:
      address: "victoriametrics.example.com"
      database: "datastream"

port is left out, so it defaults to 8428.

Cluster Tenant through vmauth

Writing to tenant 42:7 of a cluster, through vmauth over HTTPS with a bearer token:

targets:
  - name: victoriametrics_cluster
    type: victoriametrics
    properties:
      address: "vmauth.example.com"
      port: 8427
      tenant: "42:7"
      token: "your-vmauth-token"
      tls:
        status: true

The address and the port are those of vmauth, which listens on 8427 by default and passes the write on to vminsert. Write the tenant in quotes, so that YAML reads it as text. With tls.status: true the target sends the token over HTTPS rather than in cleartext. vmauth must accept HTTPS on its port.

Troubleshooting

Situation	Result
The server refuses the data (400, 413 or 422)	The batch is tried up to four times, then dropped
Sign-in fails (401 or 403), a 404 answer, the server is busy (429), a server error (5xx), or the connection fails	The batch is retried
The server answers with a redirect	Counted as a failure: the target does not follow redirects

Records are dropped although nothing fails

Cause: the records give no field. Either they hold no number or boolean, or none of the paths that fields lists has a value in them. When no record of a batch gives a field, the Director logs a warning that starts with Skipped.

Fix: list the record fields that carry the values in fields, or leave fields empty to write every number and boolean.

Configuration errors

These stop the target from starting:

Reason text	Fix
address is required for victoriametrics target	Set address
tenant must be accountID or accountID:projectID, each a whole number up to 4294967295, for victoriametrics target, got "..."	Write the tenant as one whole number, or two separated by a colon
timeout must be at least 1 second for victoriametrics target, got ...	Set timeout to 1 or more
batch_size must be a whole number greater than 0 for victoriametrics target, got ...	Use a whole number of at least 1, or leave batch_size out to use 5000
failed to build TLS config: ...	The rest of the line names the TLS setting at fault