Skip to main content

Redis

Synopsis​

Creates a Pub/Sub subscriber that connects to Redis servers and processes messages from specified channels. Supports authentication, TLS encryption, and multiple workers with automatic message handling.

For details, see Appendix.

Schema​

- id: <numeric>
name: <string>
description: <string>
type: redis
tags: <string[]>
pipelines: <pipeline[]>
status: <boolean>
properties:
address: <string>
port: <numeric>
username: <string>
password: <string>
channel: <string>
reuse: <boolean>
workers: <numeric>
tls:
status: <boolean>
cert_name: <string>
key_name: <string>
passphrase: <string>
ca_name: <string>
server_name: <string>
min_tls_version: <string>
max_tls_version: <string>
insecure_skip_verify: <boolean>

Configuration​

The following fields are used to define the device:

Device​

FieldRequiredDefaultDescription
idY-Unique numeric identifier
nameY-Device name
descriptionN-Optional description
typeY-Must be redis
tagsN-Optional tags
pipelinesN-Optional pre-processor pipelines
statusNtrueEnable/disable the device

Connection​

FieldRequiredDefaultDescription
addressN"0.0.0.0"Redis server address. The console requires it on a new device; a YAML that leaves it out dials 0.0.0.0
portY-Redis server port
usernameN-Authentication username
passwordN-Authentication password
channelY-Channel pattern to subscribe to (supports * wildcard, e.g. logs.*)

TLS​

FieldRequiredDefaultDescription
tls.statusNfalseEnable TLS encryption
tls.cert_nameY*cert.pemClient certificate for mutual TLS
tls.key_nameY*key.pemClient private key for mutual TLS
tls.passphraseN-Passphrase for an encrypted private key
tls.ca_nameN-CA bundle used to verify the server certificate. When unset, the host trust store is used; when set, it replaces the host trust store rather than adding to it.
tls.server_nameN-SNI hostname override for the TLS handshake
tls.min_tls_versionNtls1.2Minimum accepted TLS version (tls1.0, tls1.1, tls1.2, tls1.3)
tls.max_tls_versionN-Maximum accepted TLS version. When unset, the highest mutually supported version is negotiated.
tls.insecure_skip_verifyNfalseSkip server certificate verification. Use only for testing.

* = Required when tls.status is true.

note

TLS material fields (cert_name, key_name, ca_name, client_ca_name) accept any of the following:

  • File name — resolved relative to the service root directory. Nested paths such as certs/prod/server.pem are supported.
  • Absolute path — honored only if it resolves inside the service root. Any path that escapes the root is refused.
  • Inline PEM content — used verbatim when the value contains -----BEGIN.
  • Environment variable — ${ENV_VAR}.
  • Vault reference — $secret{id=...} or $secret{store=...,ref=...}.

Performance​

FieldRequiredDefaultDescription
reuseNfalseEnable multi-worker mode
workersN4Number of worker processes when reuse enabled (capped at the number of available CPU cores)
max_buffer_sizeN128MBIngest buffer admission cap. Also sizes this listener's ingest buffers: 3/8 of this value at start, growing to at most 3x
note

max_buffer_size is the ingest-buffer admission cap, and is distinct from any buffer_size field above, which sizes the network read buffer. Left unset it is 128MB, whose buffers take 48 MiB of heap per listener worker at start and grow to at most 384 MiB as bursts need the room — and with reuse: true a device runs one worker per listener. Size strings are binary, and 32MB and 32MiB are exact synonyms.

See Listener Memory Sizing for the arithmetic, the sizing table, and what happens when the cap is exhausted.

Examples​

Basic​

Creating a simple Redis Pub/Sub subscriber on a single channel...

- id: 1
name: basic_redis
type: redis
properties:
address: "redis.example.com"
port: 6379
channel: "logs"

Secure​

Connecting with authentication and TLS encryption...

- id: 2
name: secure_redis
type: redis
properties:
address: "redis.example.com"
port: 6379
username: "subscriber"
password: "secret"
channel: "secure.logs"
tls:
status: true
cert_name: "redis.crt"
key_name: "redis.key"

High-Volume​

Multi-worker mode for high-throughput consumption...

- id: 3
name: performant_redis
type: redis
properties:
address: "redis.example.com"
port: 6379
channel: "high-volume"
reuse: true
workers: 4

Pattern Subscription​

Pattern-based subscription using a wildcard channel...

- id: 4
name: pattern_redis
type: redis
properties:
address: "redis.example.com"
port: 6379
channel: "logs.*"
reuse: true
workers: 2
tip

Redis channel patterns support the * wildcard character for matching multiple channels.

Pipelines​

Applying custom processing to messages...

- id: 5
name: pipeline_redis
type: redis
pipelines:
- json_parser
- field_extractor
properties:
address: "redis.example.com"
port: 6379
channel: "raw.logs"
note

Pipelines are processed sequentially, and can modify or drop messages before ingestion.