MongoDB

Databases Analytics

Synopsis

Creates a MongoDB target that inserts records as documents into the collections of a MongoDB server or a MongoDB Atlas cluster. It supports collection names taken from record fields, time-series collections, batch processing and field normalization.

Schema

- name: <string>
  description: <string>
  type: mongodb
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    uri: <string>
    database: <string>
    collection: <string>
    collection_missing_field: <string>
    collection_fallback: <string>
    collection_default_value: <string>
    username: <string>
    password: <string>
    timeseries:
      status: <boolean>
      time_field: <string>
      meta_field: <string>
      granularity: <string>
      expire_after_seconds: <integer>
    batch_size: <integer>
    max_bytes: <integer>
    timeout: <integer>
    max_retries: <integer>
    retry_delay: <integer>
    disable_retries: <boolean>
    field_format: <string>
    tls:
      status: <boolean>
      verify: <boolean>
      ca_name: <string>
      cert_name: <string>
      key_name: <string>
      passphrase: <string>
      server_name: <string>
      min_tls_version: <string>
      max_tls_version: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be mongodb
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Connection

Field	Required	Default	Description
uri	Y	-	The connection string: mongodb:// or, for MongoDB Atlas, mongodb+srv://. Its options, such as authSource or replicaSet, apply. Plain text, an environment variable (${VARIABLE}) or a secret ($secret{...}). In the console: Connection string
database	Y	-	The database the documents go to. Keep the name under 64 bytes: MongoDB refuses a longer one, and the target refuses one over 64 bytes before it starts. It cannot hold a space or any of / \ . " $ * < > : | ?. In the console: Database
collection	Y	-	The collection the documents go to. It can take part of its name from each record (see Collection per Record below). In the console: Collection
collection_missing_field	N	drop	What happens to a record that lacks a field the collection name uses: drop, fallback or default_value. In the console: the Missing collection field choice (Drop the record, Send it to a fallback collection, Use a default value)
collection_fallback	N	-	The collection for those records. Required when collection_missing_field is fallback. It is used as written, so it cannot hold braces, $ or a NUL character, and it cannot start with a dot or with system.. In the console: Fallback collection
collection_default_value	N	unknown	The text that takes the place of a missing value when collection_missing_field is default_value. It cannot hold $ or a NUL character. In the console: Default value
timeout	N	30	The seconds one database call may take. The lowest value is 1. In the console: Timeout

Authentication

Field	Required	Default	Description
username	N	-	Replaces the user name in the connection string. Plain text, an environment variable or a secret. In the console: Username
password	N	-	Replaces the password in the connection string. A password needs a user name, in username or in the connection string. Plain text, an environment variable or a secret. In the console: Password

MongoDB Settings

Field	Required	Default	Description
batch_size	N	1000	The most records in one batch. The target's own value comes first, then the service-level batch_size, then 1000. In the console: Batch size
max_bytes	N	16777216 (16 MB)	The most bytes of records in one batch. In the console: Maximum batch payload
max_retries	N	3	How often the target tries again after the server refused some documents or a whole insert failed (see Delivery and Retries below). 0 means the default. In the console: Maximum retries
retry_delay	N	1	The longest wait before the first of those tries, in seconds. The wait is random, and its limit doubles with each try, up to 30 seconds. In the console: Retry delay
disable_retries	N	false	true makes one try: documents the server refused are dropped at once, and a failed insert goes back to the director's own retry of the batch. In the console: Disable retries
field_format	N	-	The format each record is converted to before the target writes it, such as ecs or ocsf. See applicable Normalization section. In the console: Field format

Time-series Settings

These fields sit in the timeseries block.

Field	Required	Default	Description
timeseries.status	N	false	Creates each new collection as a time-series collection. In the console: the Time-series collections switch
timeseries.time_field	N	@timestamp	The top-level field that holds each document's time. In the console: Time field
timeseries.meta_field	N	-	The top-level field MongoDB groups records into series by, such as host. It must differ from the time field. In the console: Metadata field
timeseries.granularity	N	seconds	seconds, minutes or hours: the unit closest to the time between two records of the same series. In the console: Granularity
timeseries.expire_after_seconds	N	0	MongoDB deletes documents once they are this many seconds past their time. It deletes them in the background, a bucket of documents at a time and only once every document in the bucket has expired, so a document can stay well past this period: a bucket spans longer with a coarser granularity. 0 keeps documents. In the console: Retention period

The time and metadata fields cannot be _id, start with $, or hold a dot or a NUL character.

TLS Configuration

Field	Required	Default	Description
tls.status	N	false	Enable TLS with the settings below

TLS material is resolved through the shared client builder, so these keys mean the same thing on every target that uses it. They are nested under a tls: block.

Field	Required	Default	Description
tls.verify	N	true	Verify the server certificate. On by default — set it to false only to accept an untrusted certificate, and only where you control the network path
tls.server_name	N	-	SNI hostname override for the handshake. Use it when the certificate's name does not match the address you connect to
tls.ca_name	N	-	CA bundle used to verify the server certificate. When unset the host trust store is used; when set it replaces the host trust store rather than adding to it
tls.cert_name	N*	-	Client certificate, for mutual TLS
tls.key_name	N*	-	Client private key, for mutual TLS
tls.passphrase	N	-	Passphrase for an encrypted client private key
tls.min_tls_version	N	tls1.2	Lowest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)
tls.max_tls_version	N	tls1.3	Highest protocol version accepted (tls1.0, tls1.1, tls1.2, tls1.3)

* cert_name and key_name are individually optional but must be supplied together — a certificate without its key, or a key without its certificate, is a configuration error.

A mongodb+srv:// connection string turns TLS on by itself, and so does tls=true in the string. The tls block is needed only for a private CA or a client certificate. In the console these fields are the TLS settings step.

Scheduling

See Scheduling and Pool Behavior for interval and cron fields shared by all targets.

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to target (testing)

Details

Records and Documents

• Each record must be a JSON object. The target reads it as relaxed MongoDB Extended JSON. A value in Extended JSON form becomes the type it names: {"time": {"$date": "2026-10-23T10:00:00Z"}} becomes a date, and {"$oid": "…"} an ObjectId. A plain JSON value keeps its JSON type.
• The target skips a record that is not a JSON object, or that MongoDB cannot store, and counts it as dropped. MongoDB cannot store a document larger than 16 MB, or a field name that holds a NUL character.

Collection per Record

• The collection name can hold record fields in braces, such as logs_{event.dataset}. The target puts the value of each field in its place, and the record goes to the collection that this name gives.
• A text value is used as it is. A number or a boolean is used as its JSON text.
• A field is missing when it is absent, or holds null, an empty text, an object or an array. collection_missing_field then decides what happens.
• $ and NUL characters in a value become _.
• The target drops a record whose collection name would be too long for MongoDB, or would start with a dot or with system., whatever collection_missing_field says, because MongoDB would refuse it every time.

Time-series Collections

• With timeseries.status on, the target creates each collection as a time-series collection before it first writes to it. A collection that exists already is used as it is: the timeseries settings, expire_after_seconds included, apply only to a collection the target creates. To change an existing collection's retention, change it in MongoDB.
• The time field becomes a date. A date stays as it is, and RFC 3339 text becomes that time. Any other value, and a missing field, becomes the time of the write.

Delivery and Retries

• The target connects once, when it starts, and checks the connection first. A wrong address or sign-in therefore shows up when the target starts.
• It writes the documents of each collection in a batch with one unordered insert. The server writes every document it accepts and goes on after a document it refuses, such as one that breaks a unique index or a validation rule.
• The target never sends accepted documents again. It tries the refused documents again on their own, up to max_retries times. A document still refused after the last try is dropped and counted as dropped. With debug.status on, the log names its error code.
• A document refused for a duplicate key is not tried again, and is counted as dropped. When the duplicate is its _id, the collection already holds a document with that _id. That is the same record when it was delivered before. A different record that carries the same _id is not stored, so a dropped count can also mean records lost to an _id they share. When it breaks another unique index, the record is not stored at all, so a dropped count on a collection with such an index can mean records lost to it.
• If a whole insert fails, such as for a lost connection, a timeout or a write concern error, the target sends the same documents again, up to max_retries times. In a plain collection, a document that landed before the failure comes back as a duplicate key and counts as sent, not as a copy.
• Delivery is at least once for every document the server does not refuse. A document the server still refuses after the last try, or one it refuses for a duplicate key, is dropped, as described above. A time-series collection does not refuse a duplicate _id, so a retry there can write a copy. In a plain collection, a record that carries its own _id is never written twice.

Examples

Basic

Inserting records into one collection:

targets:
  - name: mongodb_logs
    type: mongodb
    properties:
      uri: "mongodb://mongo.example.com:27017"
      database: "telemetry"
      collection: "logs"

MongoDB Atlas

Connecting to a MongoDB Atlas cluster, with the password kept in a secret:

targets:
  - name: atlas_logs
    type: mongodb
    properties:
      uri: "mongodb+srv://cluster0.example.mongodb.net"
      username: "datastream"
      password: "$secret{atlas-password}"
      database: "telemetry"
      collection: "logs"

Collection per Dataset

A collection per dataset, with a fallback collection for records that have no dataset:

targets:
  - name: mongodb_per_dataset
    type: mongodb
    properties:
      uri: "mongodb://mongo.example.com:27017"
      database: "telemetry"
      collection: "logs_{event.dataset}"
      collection_missing_field: fallback
      collection_fallback: "logs_unknown"

Time-series Collections

Time-series collections for sensor readings, a collection per site, kept for 30 days:

targets:
  - name: mongodb_readings
    type: mongodb
    properties:
      uri: "mongodb://mongo.example.com:27017"
      database: "iot"
      collection: "readings_{site}"
      timeseries:
        status: true
        time_field: "@timestamp"
        meta_field: "device"
        granularity: minutes
        expire_after_seconds: 2592000

Troubleshooting

Configuration the target refuses

Cause: the director refuses a configuration it cannot use. The message names the setting:

• uri is required for mongodb target, and uri must be a MongoDB connection string, such as mongodb://host:27017 or mongodb+srv://cluster.example.net
• password needs a user name: set username, or put the user in the uri
• database is required for mongodb target
• database name "<name>" cannot hold a space, a NUL character or any of / \ . " $ * < > : | ?
• database name is <n> bytes long, and MongoDB allows <m> at most
• collection is required for mongodb target
• collection "<name>" has a "{" without a "}", and collection "<name>" has a "}" without a "{"
• collection "<name>" has an empty placeholder "{}": put a record field between the braces
• collection "<name>" cannot hold "$" or a NUL character outside a placeholder
• collection "<name>" cannot start with "." or "system.": MongoDB refuses the first and keeps the second for its own collections
• collection_missing_field must be "drop", "fallback" or "default_value", not "<value>"
• collection_fallback is required when collection_missing_field is "fallback"
• collection_fallback "<name>" is used as it is, so it cannot hold a placeholder, "$" or a NUL character, and collection_fallback "<name>" cannot start with "." or "system.": MongoDB refuses the first and keeps the second for its own collections
• timeseries.granularity must be "seconds", "minutes" or "hours", not "<value>"
• timeseries.meta_field cannot be the same field as timeseries.time_field
• timeseries.expire_after_seconds must be 0 or more, got <n>
• timeout must be at least 1 second for mongodb target, got <n>

Fix: correct the setting the message names. Put a record field between each pair of braces in collection, and keep collection_fallback free of braces, since it is used as written.

"could not connect to MongoDB database"

could not connect to MongoDB database <database>: …

Cause: the target's start-up check failed: the address, the sign-in, the network or TLS. The text after the colon says which.

Fix: check uri, username and password, the network path from the director to the server, and, for a private CA or a client certificate, the tls block.

"mongodb insert into … failed and will be retried"

mongodb insert into <database>.<collection> failed and will be retried: …

Cause: a whole insert failed, such as for a lost connection, a timeout or a write concern error.

Fix: none is needed when it passes: the target sends the same documents again, as described under Delivery and Retries. When the message repeats, the text after the colon names the cause.

"mongodb could not create the time-series collection"

mongodb could not create the time-series collection <database>.<collection> and will try again: …

Cause: usually, the user may not create collections in that database.

Fix: let the user create collections in that database. The target tries again.