QuestDB

Databases Analytics

Synopsis

Creates a QuestDB target that writes each record as one point of InfluxDB line protocol, over the HTTP transport of the QuestDB line protocol. Each measurement is a table, so there is no database or bucket to name.

Schema

- name: <string>
  description: <string>
  type: questdb
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    address: <string>
    port: <integer>
    token: <string>
    username: <string>
    password: <string>
    measurement_field: <string>
    measurement: <string>
    tags: <string[]>
    fields: <string[]>
    batch_size: <integer>
    max_bytes: <integer>
    timeout: <integer>
    field_format: <string>
    tls:
      status: <boolean>
      verify: <boolean>
      ca_name: <string>
      cert_name: <string>
      key_name: <string>
      passphrase: <string>
      server_name: <string>
      min_tls_version: <string>
      max_tls_version: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be questdb
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Connection

Field	Required	Default	Description
address	Y	-	Host name or IP address of the server
port	N	9000	The HTTP port of QuestDB
token	N	-	A token of QuestDB Enterprise, sent as a bearer token. It is used before username and password
username	N	-	The user the target signs in as with HTTP Basic, when no token is set
password	N	-	Password for that user
timeout	N	30	Seconds one request may take. At least 1

Point Mapping

These four fields apply to every record that the InfluxDB device did not write. See How a Record Becomes a Point.

Field	Required	Default	Description
measurement_field	N	event.dataset	The record field whose text names the measurement, which is the table
measurement	N	record	The measurement when that field is missing or empty
tags	N	-	The record fields that become tags. A list, or one text with the paths separated by commas
fields	N	-	The record fields that become fields. When empty, every number and boolean in the record

Delivery

Field	Required	Default	Description
batch_size	N	5000	The most lines in one request
max_bytes	N	10485760 (10 MB)	The most bytes of line protocol in one request, before compression
field_format	N	-	Converts each record to a format such as ECS or OCSF before it is mapped. See applicable Normalization section

note

max_bytes caps the size of a single request body or batch. It behaves differently from max_size on the file-writing targets, and the difference is easy to get wrong:

• An explicit max_bytes: 0 IS honoured and removes the ceiling on targets that have no ceiling of their own. The value is read with Int64, which returns what you wrote — unlike max_size, which is read with MustInt64 and silently substitutes its default for a zero. Where a target has a fixed ceiling of its own, its page says so and a zero is clamped to that ceiling instead.
• A negative value is rejected at configuration time with max_bytes must not be negative.

The ceiling is validated when the target is built and enforced again on every batch, so a value the receiving service will refuse fails early rather than per request.

TLS

Field	Required	Default	Description
tls.status	N	false	Use HTTPS. TLS stays off unless this is true

TLS material is resolved through the shared client builder, so these keys mean the same thing on every target that uses it. They are nested under a tls: block.

Field	Required	Default	Description
tls.verify	N	true	Verify the server certificate. On by default — set it to false only to accept an untrusted certificate, and only where you control the network path
tls.server_name	N	-	SNI hostname override for the handshake. Use it when the certificate's name does not match the address you connect to
tls.ca_name	N	-	CA bundle used to verify the server certificate. When unset the host trust store is used; when set it replaces the host trust store rather than adding to it
tls.cert_name	N*	-	Client certificate, for mutual TLS
tls.key_name	N*	-	Client private key, for mutual TLS
tls.passphrase	N	-	Passphrase for an encrypted client private key
tls.min_tls_version	N	tls1.2	Lowest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)
tls.max_tls_version	N	tls1.3	Highest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)

* cert_name and key_name are individually optional but must be supplied together — a certificate without its key, or a key without its certificate, is a configuration error.

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Details

The QuestDB target writes each record as one point of InfluxDB line protocol, and sends each batch in one HTTP request. The body is gzip compressed. The target sends no precision setting, because the write API reads nanoseconds by default.

QuestDB takes these writes only while its line.http.enabled setting is true.

QuestDB accepts a gzip-compressed body from version 9.1.0 on. The target always compresses the body and has no setting to turn compression off, so it cannot write to an older QuestDB server.

It shares its sender with the InfluxDB target, so it maps records the same way.

How a Record Becomes a Point

• A record that the InfluxDB device wrote keeps its measurement, its tags and its fields. Each field keeps its type: a float, an integer, an unsigned integer, a boolean or a string. The point mapping fields do not apply to it.
• Any other record is mapped. The measurement is the text at measurement_field, or measurement when that field is missing or empty. Each field that tags lists becomes a tag, and each field that fields lists becomes a field, named by its path. When fields is empty, every number in the record becomes a float field and every boolean becomes a boolean field. Numbers in arrays, the record's time and the fields used as tags are left out.
• The time of the point is the record's @timestamp, in nanoseconds. A record without one gets the server's time.
• A record that gives no field is skipped and counted as dropped. The Director logs a warning when no record of a batch gives a field.

Prerequisites

1. A running QuestDB server, version 9.1.0 or later, whose HTTP port is reachable from the Director host
2. line.http.enabled set to true on the server
3. For QuestDB Enterprise, a token with permission to write; for QuestDB open source with authentication, a user name and a password

Examples

Basic

Writing to a QuestDB server without authentication:

targets:
  - name: questdb_metrics
    type: questdb
    properties:
      address: "questdb.example.com"

port is left out, so it defaults to 9000.

QuestDB Enterprise

Signing in with a token of QuestDB Enterprise, over HTTPS:

targets:
  - name: questdb_enterprise
    type: questdb
    properties:
      address: "questdb.example.com"
      token: "your-questdb-token"
      tls:
        status: true

Troubleshooting

Situation	Result
The server refuses the data (400, 413 or 422)	The batch is tried up to four times, then dropped
Sign-in fails (401 or 403), a 404 answer, the server is busy (429), a server error (5xx), or the connection fails	The batch is retried
The server answers with a redirect	Counted as a failure: the target does not follow redirects

Records are dropped although nothing fails

Cause: the records give no field. Either they hold no number or boolean, or none of the paths that fields lists has a value in them. When no record of a batch gives a field, the Director logs a warning that starts with Skipped.

Fix: list the record fields that carry the values in fields, or leave fields empty to write every number and boolean.

Configuration errors

These stop the target from starting:

Reason text	Fix
address is required for questdb target	Set address
timeout must be at least 1 second for questdb target, got ...	Set timeout to 1 or more
batch_size must be a whole number greater than 0 for questdb target, got ...	Use a whole number of at least 1, or leave batch_size out to use 5000
failed to build TLS config: ...	The rest of the line names the TLS setting at fault