GreptimeDB

Databases Analytics

Synopsis

Creates a GreptimeDB target that writes each record as one point of InfluxDB line protocol, through the InfluxDB compatible write API of GreptimeDB. Each measurement becomes a table of the database.

Schema

- name: <string>
  description: <string>
  type: greptimedb
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    address: <string>
    port: <integer>
    database: <string>
    username: <string>
    password: <string>
    measurement_field: <string>
    measurement: <string>
    tags: <string[]>
    fields: <string[]>
    batch_size: <integer>
    max_bytes: <integer>
    timeout: <integer>
    field_format: <string>
    tls:
      status: <boolean>
      verify: <boolean>
      ca_name: <string>
      cert_name: <string>
      key_name: <string>
      passphrase: <string>
      server_name: <string>
      min_tls_version: <string>
      max_tls_version: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be greptimedb
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Connection

Field	Required	Default	Description
address	Y	-	Host name or IP address of the server
port	N	4000	The HTTP port of GreptimeDB
username	N	-	The user the target signs in as. Leave both username and password empty for a server without authentication
password	N	-	Password for that user
timeout	N	30	Seconds one request may take. At least 1

GreptimeDB signs in with a user name and a password only. The target has no token.

Where the Points Land

Field	Required	Default	Description
database	N	public	The database the points are written to

Point Mapping

These four fields apply to every record that the InfluxDB device did not write. See How a Record Becomes a Point.

Field	Required	Default	Description
measurement_field	N	event.dataset	The record field whose text names the measurement, which is the table
measurement	N	record	The measurement when that field is missing or empty
tags	N	-	The record fields that become tags. A list, or one text with the paths separated by commas
fields	N	-	The record fields that become fields. When empty, every number and boolean in the record

Delivery

Field	Required	Default	Description
batch_size	N	5000	The most lines in one request
max_bytes	N	10485760 (10 MB)	The most bytes of line protocol in one request
field_format	N	-	Converts each record to a format such as ECS or OCSF before it is mapped. See applicable Normalization section

note

max_bytes caps the size of a single request body or batch. It behaves differently from max_size on the file-writing targets, and the difference is easy to get wrong:

• An explicit max_bytes: 0 IS honoured and removes the ceiling on targets that have no ceiling of their own. The value is read with Int64, which returns what you wrote — unlike max_size, which is read with MustInt64 and silently substitutes its default for a zero. Where a target has a fixed ceiling of its own, its page says so and a zero is clamped to that ceiling instead.
• A negative value is rejected at configuration time with max_bytes must not be negative.

The ceiling is validated when the target is built and enforced again on every batch, so a value the receiving service will refuse fails early rather than per request.

TLS

Field	Required	Default	Description
tls.status	N	false	Use HTTPS. TLS stays off unless this is true

TLS material is resolved through the shared client builder, so these keys mean the same thing on every target that uses it. They are nested under a tls: block.

Field	Required	Default	Description
tls.verify	N	true	Verify the server certificate. On by default — set it to false only to accept an untrusted certificate, and only where you control the network path
tls.server_name	N	-	SNI hostname override for the handshake. Use it when the certificate's name does not match the address you connect to
tls.ca_name	N	-	CA bundle used to verify the server certificate. When unset the host trust store is used; when set it replaces the host trust store rather than adding to it
tls.cert_name	N*	-	Client certificate, for mutual TLS
tls.key_name	N*	-	Client private key, for mutual TLS
tls.passphrase	N	-	Passphrase for an encrypted client private key
tls.min_tls_version	N	tls1.2	Lowest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)
tls.max_tls_version	N	tls1.3	Highest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)

* cert_name and key_name are individually optional but must be supplied together — a certificate without its key, or a key without its certificate, is a configuration error.

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Details

The GreptimeDB target writes each record as one point of InfluxDB line protocol, and sends each batch in one HTTP request. Unlike the other line protocol targets, it does not compress the body. The target sends no precision setting, because the write API reads nanoseconds by default.

It shares its sender with the InfluxDB target, so it maps records the same way.

How a Record Becomes a Point

• A record that the InfluxDB device wrote keeps its measurement, its tags and its fields. Each field keeps its type: a float, an integer, an unsigned integer, a boolean or a string. The point mapping fields do not apply to it.
• Any other record is mapped. The measurement is the text at measurement_field, or measurement when that field is missing or empty. Each field that tags lists becomes a tag, and each field that fields lists becomes a field, named by its path. When fields is empty, every number in the record becomes a float field and every boolean becomes a boolean field. Numbers in arrays, the record's time and the fields used as tags are left out.
• The time of the point is the record's @timestamp, in nanoseconds. A record without one gets the server's time.
• A record that gives no field is skipped and counted as dropped. The Director logs a warning when no record of a batch gives a field.

Prerequisites

1. A running GreptimeDB server whose HTTP port is reachable from the Director host
2. The database, unless you write to public
3. A user with permission to write to the database, if the server has authentication

Examples

Basic

Writing to the public database of a server without authentication:

targets:
  - name: greptimedb_metrics
    type: greptimedb
    properties:
      address: "greptimedb.example.com"

port and database are left out, so they default to 4000 and public.

With a User and Tags

Writing to a database of its own over HTTPS, signing in as a user, with one record field as a tag:

targets:
  - name: greptimedb_observability
    type: greptimedb
    properties:
      address: "greptimedb.example.com"
      database: "observability"
      username: "datastream"
      password: "secret"
      tags:
        - host.name
      tls:
        status: true

With tls.status: true the target sends the user name and the password over HTTPS rather than in cleartext. The server must accept HTTPS on its port.

Troubleshooting

Situation	Result
The server refuses the data (400, 413 or 422)	The batch is tried up to four times, then dropped
Sign-in fails (401 or 403), the database does not exist yet (404), the server is busy (429), a server error (5xx), or the connection fails	The batch is retried
The server answers with a redirect	Counted as a failure: the target does not follow redirects

Records are dropped although nothing fails

Cause: the records give no field. Either they hold no number or boolean, or none of the paths that fields lists has a value in them. When no record of a batch gives a field, the Director logs a warning that starts with Skipped.

Fix: list the record fields that carry the values in fields, or leave fields empty to write every number and boolean.

Configuration errors

These stop the target from starting:

Reason text	Fix
address is required for greptimedb target	Set address
timeout must be at least 1 second for greptimedb target, got ...	Set timeout to 1 or more
batch_size must be a whole number greater than 0 for greptimedb target, got ...	Use a whole number of at least 1, or leave batch_size out to use 5000
failed to build TLS config: ...	The rest of the line names the TLS setting at fault