Graylog

SIEM Platform

Synopsis

The Graylog target forwards events, formatted as GELF, to a Graylog GELF input over TCP, TLS, UDP, or HTTP(S). It handles GELF wire framing for the chosen transport — the event payload is expected to already be a complete GELF JSON document when it reaches the target.

Schema

- name: <string>
  description: <string>
  type: graylog
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    address: <string>
    port: <numeric>
    protocol: <string>
    max_message_size: <numeric>
    url: <string>
    timeout: <numeric>
    tls:
      verify: <boolean>
      server_name: <string>
      ca_name: <string>
      cert_name: <string>
      key_name: <string>
      passphrase: <string>
      min_tls_version: <string>
      max_tls_version: <string>
    field_format: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be graylog
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Connection

Field	Required	Default	Description
address	Y*	-	Graylog GELF input host. Required for the tcp, tls, and udp transports
port	N	12201	GELF input port. Defaults to the TLS port 12202 when protocol is tls; https keeps 12201. Valid range 1–65535
protocol	N	tcp	Transport: tcp (null-delimited) | tls | udp (chunked) | http | https
max_message_size	N	8192	UDP only: payload bytes per chunk before GELF chunking. 0 sends a single datagram. Ignored for tcp/tls/http/https
url	N*	built as scheme://address:port/gelf	http/https only: explicit endpoint override
timeout	N	30	Connection/write timeout in seconds. Bounds the stream dial and write deadline for tcp/tls/udp, and the HTTP client timeout for http/https

* = Conditionally required. address is required for tcp, tls, and udp; for http/https, either address (with port) or an explicit url must be supplied.

TLS

Field	Required	Default	Description
tls.verify	N	true	Verify the server certificate
tls.server_name	N	-	SNI / expected server hostname
tls.ca_name	N	-	Custom CA to trust. Empty uses the OS trust store
tls.cert_name	N*	-	Client certificate for mutual TLS
tls.key_name	N*	-	Client key for mutual TLS
tls.passphrase	N	-	Passphrase for an encrypted client key
tls.min_tls_version	N	tls1.2	Minimum negotiated TLS version
tls.max_tls_version	N	tls1.3	Maximum negotiated TLS version

* = Conditionally optional. tls.cert_name and tls.key_name must be supplied together to enable mutual TLS; supplying only one fails configuration validation. There is no tls.status field — TLS is engaged by setting protocol: tls or protocol: https.

note

TLS material fields (cert_name, key_name, ca_name, client_ca_name) accept any of the following:

• File name — resolved relative to the service root directory. Nested paths such as certs/prod/server.pem are supported.
• Absolute path — honored only if it resolves inside the service root. Any path that escapes the root is refused.
• Inline PEM content — used verbatim when the value contains -----BEGIN.
• Environment variable — ${ENV_VAR}.
• Vault reference — $secret{id=...} or $secret{store=...,ref=...}.

Processing

Field	Required	Default	Description
field_format	N	-	Optional target-side normalization format applied before delivery. Typically left empty, since the payload already arrives as GELF from upstream processing. See applicable Normalization section

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Details

This target is transport-only: it does not render GELF itself. It expects each event to already be a complete GELF JSON document when it reaches the target, and applies wire framing and delivery on top of that payload. There is no batching — each event is a single, complete delivery to the Graylog GELF input.

Transports

• GELF-TCP (default): the document is written null-delimited over a persistent stream.
• GELF-TLS: the same null-delimited framing, over a TLS-wrapped stream.
• GELF-UDP: the document is sent as one datagram, or split into GELF-chunked datagrams (2-byte magic 0x1e 0x0f, 8-byte message ID, sequence number, sequence count) when it exceeds max_message_size. A document that would need more than 128 chunks fails — use tcp or tls for events that large.
• GELF-HTTP(S): the document is POSTed as the request body to the GELF HTTP input.

Endpoint Resolution (HTTP/HTTPS)

An explicit url always wins. Otherwise the endpoint is built as scheme://address:port/gelf, where scheme is http or https depending on protocol.

Default Transport

Graylog defaults to plain GELF-TCP on port 12201, matching the standard Graylog GELF TCP input. Configure the corresponding GELF input type (TCP, TLS, UDP, or HTTP) on the Graylog side to match the protocol selected here.

TLS

TLS is engaged only via protocol: tls or protocol: https — there is no independent tls.status toggle. A client certificate is optional — supplying tls.cert_name without tls.key_name (or vice versa) fails configuration validation. tls.verify defaults to true.

note

The target does not perform GELF rendering or field mapping — it only frames and delivers the payload it receives. Format the event as GELF before it reaches this target.

Examples

Basic Configuration

Sending GELF events to a Graylog GELF-TCP input...

targets: - name: graylog type: graylog properties: address: "10.0.0.5" port: 12201

Chunked UDP

Delivering over GELF-UDP with chunking for events larger than the datagram size...

targets: - name: graylog-udp type: graylog properties: address: "10.0.0.5" protocol: udp max_message_size: 8192

HTTP Endpoint

Posting GELF events to an explicit HTTP input URL...

targets: - name: graylog-http type: graylog properties: protocol: http url: "http://graylog:12201/gelf" timeout: 30

Troubleshooting

This section covers the errors you are most likely to see with the graylog target, what causes each one, and how to fix it. The same behavior applies to the OVHcloud Logs Data Platform target documented on its own page, because both types deliver GELF the same way. Only the defaults differ: graylog starts from protocol: tcp on port 12201, ovhcloud from protocol: tls on port 12202. Where OVHcloud differs beyond that, the entry says so.

Where to look:

• Director logs. Target errors are tagged with the target name and carry "Section":"SenderPool". The part after Reason: or after the last colon is the actual cause.
• The target's connection status in the web interface. It shows the same reason as the log line.

See Target Delivery Errors for how Director logs and retries target failures.

What the input needs

This target calls no cloud service and has no credential of its own, so every prerequisite sits on the receiving side or on the Director host. Nothing is opened while the target starts, so a wrong host or port is first reported a moment later, when the sender workers connect.

Prerequisite	Applies to	Error text when it is missing
A GELF input of the matching type, GELF TCP, GELF TCP with TLS, GELF UDP or GELF HTTP, listening on port	all transports	connect: connection refused at startup, or gelf http endpoint returned status 404
Name resolution for address on the Director host	all transports	lookup graylog.example.com: no such host
Egress from the Director host to address and port	all transports	i/o timeout, or context deadline exceeded over HTTP
The input certificate signed by an authority the host trusts, or named in tls.ca_name	tls and https	x509: certificate signed by unknown authority
A client certificate and key, when the input asks every client for one	tls and https with mutual TLS	remote error: tls: certificate required

An input of the wrong type on the right port is the one prerequisite that produces no error at all. It is covered under the healthy-but-empty entry below.

"failed to create per-worker gelf connection" at startup

[Error] [director] [target-<target id>] [graylog] Sender worker 0 ThreadSafeInit on reinit failed for "graylog": failed to create per-worker gelf connection: dial tcp graylog.example.com:12201: connect: connection refused

Cause: with tcp, tls and udp, each sender worker opens its own connection when the target starts, and this is where a connection problem first shows. The text after the address names the step that failed.

• connect: connection refused means nothing is listening on that port. On Windows the wording is connectex: No connection could be made because the target machine actively refused it. The usual reason is that the GELF input was never created, or that it listens on another port.
• lookup graylog.example.com: no such host means address does not resolve from the Director host.
• tls: failed to verify certificate: x509: certificate signed by unknown authority means the input presents a certificate from a private authority that the host does not trust.
• tls: failed to verify certificate: x509: certificate is valid for ..., not ... means address holds an IP address, or a name the certificate does not carry.
• tls: first record does not look like a TLS handshake means protocol: tls was pointed at a plain GELF TCP input.
• remote error: tls: certificate required or remote error: tls: bad certificate means the input asks for a client certificate that was not sent, or did not trust the one it got.

Fix: create the GELF input whose type matches protocol, confirm its port, and open that port from the Director host. For a private authority, set tls.ca_name to the issuing CA and connect by the name the certificate carries rather than by IP address, or set tls.server_name to that name. For mutual TLS, set both tls.cert_name and tls.key_name, plus tls.passphrase when the key is encrypted. Keep tls.verify at true outside a lab.

Nothing is sent while this lasts. The workers retry until the connection succeeds, incoming data waits in the Director queue, and no restart is needed once the input is reachable.

"Reason: invalid address" when a URL is configured

[Error] [director] [target-<target id>] [graylog] Failed to reinitialize target "graylog" (attempt 41). Reason: invalid address

Cause: url is set, address is empty, and protocol is still one of the stream transports, tcp, tls or udp. Those are the defaults for both types, so this happens whenever an endpoint URL is pasted into a target that was never switched to an HTTP transport. The configuration is accepted, because a url on its own passes validation. The target then cannot start, because url is read only by http and https, and the stream transports are left with no address to dial.

Fix: decide which transport you want.

• To post to a GELF HTTP input, set protocol: http or protocol: https, and keep url. It is then used as written, so include both the scheme and the input path, as in http://graylog.example.com:12201/gelf. A url without a scheme fails later with unsupported protocol scheme "", and one that cannot be parsed at all fails with failed to create request: parse ....
• To use a stream transport, set address, and port when it is not the default. The url is ignored in this mode and can be removed.

The message reads like a connection failure but is a configuration error, so waiting does not clear it. Nothing is sent, the reason repeats once a minute, and the queue grows meanwhile.

"gelf http endpoint returned status ..."

[Error] [director] [target-<target id>] [graylog] Sender worker 0 execute() failed for natsobj:/vm-pipeline-payloads/sender.graylog.<id>: target broken: gelf http endpoint returned status 404

Cause: over http and https the input answered with a status outside the 2xx range. The status is the input's own and its response body is not carried into the log line, so the number is all you get. These statuses typically mean:

• 404, the path is wrong. A url without the /gelf path is the usual cause, and so is a port that belongs to an input of another type.
• 400, the document was not accepted as GELF. Confirm that a complete GELF JSON document reaches the target and that field_format is empty.
• 413, the document is larger than the input accepts. Trim the event upstream.
• 401 or 403, a reverse proxy in front of the input demands authentication. This target sends no credentials, so terminate that authentication elsewhere.
• 429 or a 5xx, the receiver is overloaded or down. These clear on their own.

Fix: correct url for 404, and fix the event at its source for 400 and 413.

warning

Every one of these is retried, 400 and 413 included. A document the input will never accept is redelivered about every five seconds for as long as it exists, and each attempt reinitializes the whole sender pool, which interrupts healthy traffic. Treat a repeating 400 or 413 as urgent.

"gelf udp message needs ... chunks, exceeds the 128-chunk limit"

[Error] [director] [target-<target id>] [graylog] Sender worker 3 execute() failed for natsobj:/vm-pipeline-payloads/sender.graylog.<id>: target broken: gelf udp message needs 131 chunks, exceeds the 128-chunk limit (use tcp/tls for large events)

Cause: over UDP a document longer than max_message_size is split into GELF chunks, and at most 128 chunks may carry one document. At the default max_message_size of 8192 that caps a single event at 1,048,576 bytes.

A second size error belongs here. With max_message_size: 0 chunking is switched off and the document travels as one datagram, so anything above the operating system datagram limit, around 65,507 bytes, fails on the write instead. That wording comes from the operating system and varies, but it contains message too long.

Fix: use tcp or tls, which apply no size limit of their own, or raise max_message_size while keeping it below the path MTU. Trimming the event upstream, full_message in particular, works as well.

warning

Nothing is dropped here and nothing is delivered either. The same event fails identically on every attempt and is redelivered about every five seconds, holding up everything queued behind it. Change the transport rather than waiting it out.

"target broken: write tcp ..." with a broken pipe or a reset

[Error] [director] [target-<target id>] [graylog] Sender worker 2 execute() failed for natsobj:/vm-pipeline-payloads/sender.graylog.<id>: target broken: write tcp ...->graylog.example.com:12201: write: connection reset by peer

Cause: the connection was open and the write failed. The input was restarted, a firewall or load balancer dropped a connection it considered idle, or the receiver stopped reading and the write deadline of timeout seconds expired. write: broken pipe means the same thing. The target reconnects once by itself and writes again, so a single line after a restart is expected and delivery continues.

Fix: if the message repeats, check that the input is running and that nothing on the path closes idle connections. Persistent resets against OVHcloud almost always mean protocol: tcp where tls is required, since that platform expects an encrypted connection.

The payload is redelivered once the connection is back, so nothing is lost.

"i/o timeout" or "context deadline exceeded"

[Error] [director] [target-<target id>] [graylog] Sender worker 1 execute() failed for natsobj:/vm-pipeline-payloads/sender.graylog.<id>: target broken: failed to send request: Post "http://graylog.example.com:12201/gelf": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Cause: timeout, 30 seconds by default, bounds the connection attempt, each write on a stream transport, and the whole HTTP request. The wording tells you how far the exchange got. i/o timeout while connecting means the packets are dropped rather than refused, usually by a firewall or a security group on the path. context deadline exceeded (Client.Timeout exceeded while awaiting headers) means the input accepted the connection and never answered.

Fix: open the port from the Director host to the input, and check the receiver's own health. Raise timeout only when the latency really is that high. With http and https the request also follows HTTP_PROXY and HTTPS_PROXY from the Director service environment, so proxyconnect tcp in the message points at the proxy rather than at the input. The stream transports never use a proxy.

warning

timeout: 0 removes every bound instead of restoring the default. A receiver that accepts a connection and then never answers holds the worker indefinitely, with no error to search for. Keep a positive value.

The target is healthy but nothing arrives

Nothing fails here, so there is no error text to search for. Check the following in order.

1. debug.dont_send_logs is enabled. Events are processed by the pipeline and then discarded before delivery, and the events-out counter for the target stays flat. The only trace is one startup line, Log sending is disabled for this target (graylog). Logs will be processed by the pipeline but will not be sent to the target., and it is written only when debug.status is enabled as well.

2. The transport is UDP. UDP reports nothing back, so a wrong port, a missing input, a blocked packet and a lost chunk all look like a successful send, and the events-out counter climbs. Chunks are also IP-fragmented whenever max_message_size is above the path MTU, and one lost fragment discards the whole event. Prefer tcp or tls. Where UDP is required, set max_message_size below the path MTU minus the 12 bytes of chunk header, 1400 for example, and confirm arrival with a packet capture.

3. The input on that port is not a GELF input. A write over tcp or tls counts as done once the bytes are handed to the network, so a Raw or Syslog input on the same port accepts the null-delimited document and then misparses or discards it without telling you. Match the input type to protocol.

4. An encrypted endpoint is being written to in the clear. With protocol: tcp against a TLS-only endpoint, which is what OVHcloud expects, the first write on each fresh connection is buffered and the server closes, the next write fails, the target reconnects, and the retry appears to succeed. Every event is counted as sent and none arrive. Set protocol: tls.

5. The input rejects individual documents. Over tcp, tls and udp a document the receiver refuses, one missing short_message or host for example, is discarded on its side with nothing reported back. Check the input's own failed-message metrics, or the stream statistics in the OVHcloud platform.

6. field_format is set. A value such as ecs or cim rewrites the document into another schema after it was already rendered as GELF, and conversion failures are not reported. What arrives is no longer a GELF document. Leave field_format empty for this target.

7. The OVHcloud stream token is missing. OVHcloud authenticates each log stream with a token carried as a GELF field on the event itself, and this target neither adds it nor checks for it. An event that arrives without the field is typically accepted on the wire and then discarded by the platform, so the connection stays healthy and the stream stays empty. Set the field in a pipeline before the event reaches the target, and confirm that it survives GELF normalization.

A NUL byte inside a document is replaced with a space before the frame is written, because that byte marks the end of an event on the wire. The event arrives with that one character changed and nothing is logged.

The target never starts and the reason names a setting

These reach you through the same retry loop as a connection failure, so Director keeps retrying a configuration that cannot work until you change it. Nothing is sent while any of them lasts.

Reason text	Fix
invalid address	Set address, or switch protocol to http or https when a url was meant. See the entry above
port must be between 1 and 65535, got 0	Set port within the range, or remove it to use the default
invalid protocol "TCP"	The five accepted values are lower case, and the message lists them. Write tcp, not TCP
invalid tls configuration	Exactly one of tls.cert_name and tls.key_name is set. Supply both, or neither
min TLS version "tls1.3" is greater than max TLS version "tls1.2"	tls.min_tls_version is above tls.max_tls_version. A spelling neither field recognizes is read as the default without a warning, so write them as tls1.2 and tls1.3
ca_name: "..." could not be resolved (env/vault token, inline PEM, or a path under the service root)	Place the CA file under the service root, or supply it as inline PEM
ca_name "..." contains no valid PEM certificate(s)	The file is DER or PFX. Convert it to PEM
client certificate: certificate: "..." is not inline PEM and does not resolve to a path under the service root	Copy the certificate and the key under the service root, or inline them
client certificate: read key "...": ...	The service account cannot read the file. Fix the file permissions
client certificate: invalid private key PEM	The key file is not PEM
client certificate: decrypt PKCS#8 private key (wrong passphrase?): ...	Set tls.passphrase to the passphrase of that key
client certificate: tls: private key does not match public key	The certificate and the key are not a pair
credential: env variable "..." is not set	An environment variable referenced from a tls field is not set for the Director service. Export it, or inline the value