Datadog
Synopsis
The Datadog target forwards events to the Datadog Cloud SIEM logs intake API over HTTPS. It handles batching and delivery — the event payload is expected to already be a complete JSON document in Datadog's log schema when it reaches the target.
Schema
- name: <string>
description: <string>
type: datadog
pipelines: <pipeline[]>
status: <boolean>
properties:
api_key: <string>
site: <string>
url: <string>
batch_size: <integer>
timeout: <integer>
field_format: <string>
debug:
status: <boolean>
dont_send_logs: <boolean>
Configuration
The following fields are used to define the target:
| Field | Required | Default | Description |
|---|---|---|---|
name | Y | Target name | |
description | N | - | Optional description |
type | Y | Must be datadog | |
pipelines | N | - | Optional post-processor pipelines |
status | N | true | Enable/disable the target |
Connection
| Field | Required | Default | Description |
|---|---|---|---|
api_key | Y* | - | Datadog API key, sent as the DD-API-KEY header |
site | N | datadoghq.com | Datadog site: datadoghq.com (US1), us3.datadoghq.com, us5.datadoghq.com, datadoghq.eu, ap1.datadoghq.com, ddog-gov.com |
url | N* | built from site | Explicit logs intake URL. Overrides site. Default: `https://http-intake.logs.{site}/api/v2/logs` |
max_bytes | N | 5242880 (5 MB) | Maximum size of a single request body or batch, in bytes |
max_bytes caps the size of a single request body or batch. It behaves differently from max_size on the file-writing targets, and the difference is easy to get wrong:
- An explicit
max_bytes: 0IS honoured and removes the ceiling on targets that have no ceiling of their own. The value is read withInt64, which returns what you wrote — unlikemax_size, which is read withMustInt64and silently substitutes its default for a zero. Where a target has a fixed ceiling of its own, its page says so and a zero is clamped to that ceiling instead. - A negative value is rejected at configuration time with max_bytes must not be negative.
The ceiling is validated when the target is built and enforced again on every batch, so a value the receiving service will refuse fails early rather than per request.
* = Conditionally required/optional. Either api_key (endpoint built from site) or an explicit url is required. A url without an api_key passes validation, but no DD-API-KEY header is then sent and the intake API typically refuses the request, so set api_key as well unless the address you point at supplies the key itself.
Batch Configuration
| Field | Required | Default | Description |
|---|---|---|---|
batch_size | N | 1000 | Maximum events per POST, serialized as a JSON array |
timeout | N | 30 | HTTP client timeout, in seconds |
Processing
| Field | Required | Default | Description |
|---|---|---|---|
field_format | N | - | Optional target-side normalization format applied before delivery. Typically left empty, since the payload already arrives in Datadog's JSON schema from upstream processing. See applicable Normalization section |
Scheduling
See Scheduling and Pool Behavior for interval and cron fields shared by all targets.
Debug Options
| Field | Required | Default | Description |
|---|---|---|---|
debug.status | N | false | Enable debug logging |
debug.dont_send_logs | N | false | Process logs but don't send to target (testing) |
Details
This target is transport-only: it does not render Datadog's log schema itself. It expects each event to already be a complete JSON document when it reaches the target, buffers events in memory, and delivers them as one HTTP POST — a JSON array ([{...},{...}]) — once batch_size is reached or during finalization.
Endpoint and Authentication
An explicit url always wins. Otherwise the endpoint is built from site as `https://http-intake.logs.{site}/api/v2/logs`. When api_key is set, it is sent as the DD-API-KEY request header; configuration validation rejects a target that has neither api_key nor url.
Batching and Retry
Events buffer until batch_size is reached, then POST as a single request. A failed POST does not retry the batch internally: the buffer is cleared unconditionally after Finalize, and delivery failure is surfaced to the sender pool, which handles redelivery through the JetStream layer. Retaining the batch locally in addition to that redelivery would duplicate events at the destination. Error responses include up to the first 1 MB of the response body for troubleshooting.
The target does not render Datadog's JSON schema — it only batches and delivers the payload it receives. Format the event as Datadog-compatible JSON before it reaches this target.
Examples
Basic Configuration
Sending JSON events to Datadog Cloud SIEM logs intake on the default US1 site... | |
EU Site
Routing to the Datadog EU site instead of the US1 default... | |
Explicit Endpoint Override
Overriding the intake URL directly instead of deriving it from | |
Troubleshooting
This section covers the errors you are most likely to see with the datadog target, what causes each one, and how to fix it. The sumologic and rapid7 targets behave the same way and report the same errors, so these entries apply to them too. Their own pages list what differs.
Where to look:
- Director logs. Target errors are tagged with the target name. The part after
Reason:, or after the last colon, is the actual cause. See Target Delivery Errors for how Director logs and retries target failures. - The target's connection status in the web interface. It shows the same reason as the log line, prefixed with
connection failed for <target name>:.
A refusal from the intake API is logged as datadog api returned status <code>: followed by the response body, up to its first 1 MB. The status code is the quickest way to find the setting at fault.
Which setting is wrong?
| Status | Setting to check | What it means |
|---|---|---|
400 | The pipeline that feeds the target | The body is not in the schema the intake API expects |
401 | api_key | No usable key reached the intake API |
403 | api_key, site | The key was read and refused, or it belongs to another site |
404 | url | The path is wrong, or the source behind it no longer exists |
413 | max_bytes | The request body is larger than the intake API accepts |
429 | batch_size, and how much traffic you route here | You are over the ingestion quota |
5xx | Nothing on your side | The intake API is failing, and Director keeps retrying |
Which status a vendor picks for a given mistake is not guaranteed, so read the quoted response body as well as the code.
"datadog target requires api_key (or an explicit url)"
ValidateConfig failed for target "datadog": datadog target requires api_key (or an explicit url)
Cause: neither api_key nor url is set. An ${DATADOG_API_KEY} reference that resolves to an empty value counts as not set.
Fix: set api_key to a Datadog API key. An Application key is a different credential and the intake API does not accept it.
Nothing is sent and nothing is lost while this message repeats. The configuration is re-checked about every 30 seconds, so the target starts on its own once the value is there.
"batch_size must be greater than 0", "max_bytes must not be negative", "timeout must not be negative"
Cause: one of the three numeric settings is negative.
Fix: use a positive value, or remove the setting to take the default. Note that 0 is read as "not set" rather than as a value, so batch_size: 0 becomes 1000 and timeout: 0 becomes 30. The exception is max_bytes: 0, which is honored and removes the ceiling. Keep batch_size at or below 1000, which is the largest array the intake API typically accepts.
Nothing is sent while a value is invalid, and nothing is lost.
"api returned status 401" or "403"
[Error] [director] [target-<target id>] [datadog] Sender worker 1 execute() failed for mem://<payload id>: target broken: failed to finalize target cache: datadog api returned status 401: {"errors":["Missing API key"]}
Cause: 401 means no usable key reached the intake API. The common way to produce it here is to set url and leave api_key empty. Validation accepts that pair, but no DD-API-KEY header is then sent. 403 typically means the key was read and refused, because it was revoked, mistyped, or is an Application key rather than an API key.
Fix: set api_key even when you set url. The only case that does not need one is an address of your own that adds the key on the way. On sumologic and rapid7 the equivalent mistake is a collector URL that lost characters when it was pasted, because there the credential is part of the path.
No data is lost. Both statuses are retried until you fix them. The payload comes back about every five seconds, and the queue grows in the meantime.
"403" from a key that works elsewhere
Cause: the key belongs to a different Datadog site than the one this target sends to. site defaults to datadoghq.com, so a target that never sets site goes to US1 whatever region the key came from. A key from another site is typically refused with 403.
Fix: set site to the site of the organization that issued the key. The accepted values are listed under Connection above. If you set url instead, site is ignored and the host in url decides where the batch goes.
No data is lost, and the batch is retried until the key and the site agree.
"no such host", "i/o timeout", "connection refused", or "unsupported protocol scheme"
[Error] [director] [target-<target id>] [datadog] Sender worker 2 execute() failed for natsobj://vm-sender/<payload id>: target broken: failed to finalize target cache: failed to send request: Post "https://http-intake.logs.example.com/api/v2/logs": dial tcp: lookup http-intake.logs.example.com: no such host
Cause: the request never reached the intake API. The tail of the message says which step failed.
no such hostis a misspelledsite, or a typo in the host part ofurl. Dropping the last letter ofdatadoghq.comis the usual one.i/o timeoutorconnection refusedis egress. Outbound HTTPS on port 443 to the intake host is blocked, or the network requires a proxy that is not configured.proxyconnect tcpmeansHTTPS_PROXYpoints at a proxy that is down or wrong.unsupported protocol scheme ""meansurlwas pasted withouthttps://.invalid characterin a host name meansurlholds a stray space or quote. The address is not parsed when the configuration is validated, so a malformedurlonly fails here, at the first delivery.
Fix: correct the address, allow outbound 443 from the Director host to the intake host, or set HTTP_PROXY, HTTPS_PROXY and NO_PROXY for the Director service.
No data is lost. All of these are retried until fixed, so the queue grows until the address or the route is corrected.
"x509: certificate signed by unknown authority"
... failed to finalize target cache: failed to send request: Post "https://http-intake.logs.datadoghq.com/api/v2/logs": tls: failed to verify certificate: x509: certificate signed by unknown authority
Cause: something between Director and the intake API is terminating TLS with its own certificate, and the authority that signed it is not trusted on the Director host.
Fix: this target has no TLS settings. There is no property for a certificate authority file and no option to skip verification, so the trust has to be granted at the operating system level. Install the inspecting authority's root certificate in the Director host's trust store, or exclude the intake host from TLS inspection. Restart Director after changing the trust store.
No data is lost, and the batch is retried until the handshake succeeds.
The scheme is not checked. An http:// address is accepted and sends the batch, and any credential in that address, in clear text. Always use https://.
"context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Cause: the whole request, including connecting and sending the body, did not finish inside timeout seconds. The default is 30. A multi-megabyte body on a slow link, or a slow intake API, reaches it.
Fix: raise timeout, or make the batches smaller by lowering batch_size or max_bytes.
No data is lost, but duplicates are possible. The intake API may have stored the batch before Director stopped waiting for the response, and the payload is redelivered, so those events arrive twice.
"record rejected by target": the only errors that lose data
[Error] [director] [target-<target id>] [datadog] Sender worker 3 execute() failed for natsobj://vm-sender/<payload id>: record rejected by target: datadog api returned status 400: {"errors":["Invalid message"]}
[Error] [director] [target-<target id>] [datadog] Sender worker 3 deterministic failure for natsobj://vm-sender/<payload id> after 4 attempts — dropping (giving up): record rejected by target: datadog api returned status 400: {"errors":["Invalid message"]}
Cause: two different problems carry this prefix.
- A rejected body, logged with status
400,413or422.400means the payload is not what the intake API expects, because the matching content pack pipeline was not applied on the route, or because a raw non-JSON message reached the target.413means the body was larger than the intake API accepts, which happens whenmax_bytesis raised above the vendor's own limit or set to0.422means the document was understood and refused on its content. - An oversized single record, logged as
record rejected by target: record size <n> exceeds configured max_bytes <m>. One event on its own is larger than the ceiling. Windows events with very large message fields, and base64 blobs, are the usual source.
Fix: for a rejected body, route the events through the matching Datadog content pack pipeline and put max_bytes back to its default. For an oversized record, trim the event in the pipeline by dropping or truncating the offending field. Raising max_bytes past the vendor's own limit only turns the rejection into a 413.
These two are the only errors on this target that lose data. They are treated as permanent, so Director gives up after four deliveries. A rejected body loses the whole batch, up to batch_size events. An oversized record loses that record and everything after it in the same payload, while the records ahead of it were already delivered.
"429 Too Many Requests"
... target broken: failed to finalize target cache: datadog api returned status 429: {"errors":["Too Many Requests"]}
Cause: you are sending faster than the account's ingestion quota allows.
Fix: send fewer and larger requests by raising batch_size, reduce how many routes write to this target, or raise the quota with the vendor. The target does not back off on its own and does not read a Retry-After header, so it keeps offering the batch at the redelivery pace until the limit clears.
No data is lost, and the batch is retried until it is accepted.
The target looks healthy but nothing arrives
Check these in order.
debug.dont_send_logsis enabled. Events are processed and then discarded. The payload is marked delivered, the queue drains, and the target's event-out counters stay flat while the route's counters climb. Nothing is written to the log unlessdebug.statusis also enabled, in which case startup logsLog sending is disabled for this target. Remove the setting.field_formatis set and the mapping failed. A normalization failure here is ignored rather than reported, so the event is delivered without being normalized and nothing is logged. Leavefield_formatempty when the payload already arrives in the vendor's schema.- The events arrived but were not parsed. On
sumologicandrapid7the body is newline-delimited, so the receiver typically accepts arbitrary lines with a success status and stores them as unparsed text. Delivery counters climb and no error is logged. Check the vendor's own parse errors, and route the events through the matching content pack pipeline. intervalorcronis set. The target then flushes on that schedule instead of on every payload, so expect a delay before anything is sent. See Scheduling above.
Log lines and connection status can contain the full endpoint URL
When a request fails before a response arrives, for example on a DNS, connection, proxy, TLS or timeout error, the message includes the full URL the batch was posted to. The same text is shown as the target's connection status in the web interface.
... failed to send request: Post "https://<endpoint>.collection.sumologic.com/receiver/v1/http/<collector code>": tls: failed to verify certificate: x509: certificate signed by unknown authority
On datadog the credential travels in the DD-API-KEY header, so the URL holds no secret. On sumologic and rapid7 the credential is part of the URL path: the Sumo Logic collector code, and the Rapid7 ingestion token. For those two, a line like the one above carries a working ingestion credential, and anyone who can read the log or the connection status can write data into your collector.
Treat Director logs and connection status for sumologic and rapid7 targets as sensitive. Restrict who can read them, and remove the URL before attaching a log line to a ticket, an email or a chat message. If such a line has already been shared, rotate the credential: create a new HTTP Source or ingestion endpoint, point url at the new one, and delete the old one.