Proofpoint On Demand

Pull

Synopsis

Creates a WebSocket consumer that connects to Proofpoint's On Demand (POD) log stream service and receives email security event data. Supports both message and maillog data types with secure token authentication.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: proofpoint
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    endpoint: <string>
    cluster: <string>
    token: <string>
    type: <string>
    secure: <boolean>
    workers: <numeric>
    reuse: <boolean>

Configuration

The following fields are used to define the device:

Device

Field	Required	Default	Description
id	Y		Unique numeric identifier
name	Y		Device name
description	N	-	Optional description
type	Y		Must be proofpoint
tags	N	-	Array of labels for categorization
pipelines	N	-	Array of preprocessing pipeline references
status	N	true	Enable/disable the device

Connection

Field	Required	Default	Description
endpoint	Y	"wss://logstream.proofpoint.com:443/v1/stream"	Proofpoint WebSocket endpoint URL
cluster	Y		Proofpoint cluster identifier
token	Y		Authentication token for Proofpoint API
type	Y	"message"	Data type to consume (message or maillog)
secure	N	false	Enable token encryption in configuration

Performance

Field	Required	Default	Description
workers	N	1	Number of worker processes
reuse	N	true	Enable multi-worker mode

Details

WebSocket Connection

The device establishes a persistent WebSocket connection to Proofpoint's On Demand log stream service. The connection URL includes query parameters for cluster ID and data type. Bearer token authentication is used in the Authorization header.

Data Types

Proofpoint supports two log data types:

• message: Email processing logs including connection metadata, envelope details, message headers, and filter module results
• maillog: Mail transfer agent logs including SMTP transactions and delivery status

Token Security

When secure is set to true, the token is encrypted in the YAML configuration using the service shared key. The token is decrypted at runtime before authentication. This prevents token exposure in configuration files.

Performance Tuning

The reuse field enables multi-worker mode for processing messages concurrently. When enabled, the specified number of workers processes consume data in parallel, improving throughput for high-volume log streams.

Connection Management

The device handles WebSocket connection lifecycle including automatic reconnection on failure. Query parameters are encoded in the connection URL including cluster ID (cid) and data type (type).

Examples

The following are commonly used configuration types.

Basic Configuration

The minimum required configuration creates the consumer:

Creating a basic Proofpoint On Demand consumer for email processing logs...	devices: - id: 1 name: proofpoint-pod type: proofpoint properties: cluster: "your-cluster-id" token: "your-api-token" type: "message"
Device receives Proofpoint email events in real-time...	{ "guid": "abc123-def456", "ts": "2025-01-02T10:30:00.000000", "connection": { "ip": "192.0.2.10", "host": "mail.example.com" }, "envelope": { "from": "[email protected]", "rcpts": ["[email protected]"] }, "msg": { "header": { "subject": "Quarterly Report", "from": ["[email protected]"] } }, "filter": { "modules": { "spam": { "verdict": "notspam" } } } }

Secure Token Storage

Token encryption prevents credential exposure:

Configuring encrypted token storage...	devices: - id: 2 name: proofpoint-secure type: proofpoint properties: cluster: "your-cluster-id" token: "encrypted-token-string" type: "message" secure: true
Token is decrypted at runtime using service shared key...

note

The token must be encrypted using the service shared key before setting secure: true.

Maillog Collection

Mail flow logs can be consumed:

Collecting mail routing and delivery logs...	devices: - id: 3 name: proofpoint-maillog type: proofpoint properties: cluster: "your-cluster-id" token: "your-api-token" type: "maillog"
Device receives mail transfer agent log events including delivery status...	{ "ts": "2025-01-02T10:30:00.000000", "data": "smtp;250 2.0.0 OK", "sm": { "from": "[email protected]", "to": ["[email protected]"] }, "pps": { "cid": "cluster01", "agent": "m0042.ppops.net" }, "id": "msg-20250102-001" }

High-Volume Processing

Performance can be optimized for high message rates:

Enabling multi-worker processing for throughput...	devices: - id: 4 name: proofpoint-highvolume type: proofpoint properties: cluster: "your-cluster-id" token: "your-api-token" type: "message" reuse: true workers: 4
Four worker processes consume messages in parallel...

Custom Endpoint

Alternative Proofpoint endpoints can be configured:

Connecting to a regional or custom endpoint...	devices: - id: 5 name: proofpoint-custom type: proofpoint properties: endpoint: "wss://logstream-eu.proofpoint.com:443/v1/stream" cluster: "eu-cluster-id" token: "your-api-token" type: "message"
Device connects to the specified regional endpoint...

Pipeline Processing

Email events can be preprocessed:

Applying custom processing to email security events...	devices: - id: 6 name: proofpoint-pipeline type: proofpoint pipelines: - email_enrichment - geo_lookup properties: cluster: "your-cluster-id" token: "your-api-token" type: "message"
Pipelines enrich email data before routing to targets...

note

Pipelines are processed sequentially and can modify or drop events before ingestion.