Microsoft Sentinel

Microsoft Azure SIEM

Synopsis

Creates a target that ingests log messages into Microsoft Sentinel workspace tables using Data Collection Rules (DCRs). Supports automatic table selection, field normalization, and filtering options.

tip

For more details, refer to our Microsoft Sentinel Overview and Microsoft Sentinel Integration chapters.

Schema

- name: <string>
  description: <string>
  type: sentinel
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    tenant_id: <string>
    client_id: <string>
    client_secret: <string>
    rule_id: <string>
    endpoint: <string>
    streams:
      - name: <string>
        rule_id: <string>
    stream: <string[]>
    buffer_size: <numeric>
    batch_size: <numeric>
    keep_phantom_fields: <boolean>
    drop_unknown_stream_events: <boolean>
    cache:
      timeout: <numeric>
    field_format: <string>
    debug:
      status: <boolean>
      dont_send_logs: <boolean>

Configuration

The following fields are used to define the target:

Core Settings

Field	Required	Default	Description
name	Y		Target name
description	N	-	Optional description
type	Y		Must be sentinel
pipelines	N	-	Optional post-processor pipelines
status	N	true	Enable/disable the target

Authentication

Field	Required	Default	Description
tenant_id	N*	-	Azure tenant ID (required unless using managed identity)
client_id	N*	-	Azure client ID (required unless using managed identity)
client_secret	N*	-	Client secret (required unless using managed identity)

* = Conditionally required (see authentication methods above)

Stream Configuration

Field	Required	Default	Description
endpoint	Y		DCR ingestion endpoint or Resource ID
rule_id	N		Default Data Collection Rule (DCR) ID
streams	N	-	Detailed stream configurations
stream	N	-	Legacy string array of stream names
buffer_size	N	1048576	Buffer size in bytes (1MB)
batch_size	N	1000	Maximum number of messages per batch
keep_phantom_fields	N	false	Keep fields not defined in DCR schema
drop_unknown_stream_events	N	true	Silently drop events for undefined streams
cache.timeout	N	300	Stream cache timeout in seconds
field_format	N	-	Data normalization format. See applicable Normalization section

Debug Options

Field	Required	Default	Description
debug.status	N	false	Enable debug logging
debug.dont_send_logs	N	false	Process logs but don't send to Sentinel (testing)

Automatic Table Selection

When streams is not specified, tables are automatically selected based on input type:

Input Type	Target Table
Windows Event Log	Custom-WindowsEvent
Windows Application Log	Custom-WindowsEvent
Windows System Log	Custom-WindowsEvent
Windows Security Log	Custom-SecurityEvent
Syslog	Custom-Syslog
Linux Audit Report	Custom-CommonSecurityLog
Windows Audit Report	Custom-CommonSecurityLog

Available Tables

Standard Tables (Prefix: Custom-)

WindowsEvent SecurityEvent CommonSecurityLog Syslog

ASim Tables (Prefix: Custom- or Microsoft-)

ASimAuditEventLogs ASimAuthenticationEventLogs ASimDhcpEventLogs ASimDnsActivityLogs ASimFileEventLogs ASimNetworkSessionLogs ASimProcessEventLogs ASimRegistryEventLogs ASimUserManagementActivityLogs ASimWebSessionLogs

Details

The Microsoft Sentinel target enables direct ingestion into Microsoft Sentinel tables with flexible configuration options. It supports using the SystemS3 field to route messages to specific stream tables, using the format Custom-TableName.

The target automatically detects table schemas and can clean messages to remove phantom fields that aren't defined in the schema when keep_phantom_fields is set to false.

warning

Disabling keep_phantom_fields may result in data loss for undefined fields.

Data is buffered until either the batch size limit is reached or an explicit flush is triggered. Each stream type has different limits based on the Log Analytics ingestion API.

warning

Enabling drop_unknown_stream_events silently discards unmatched events.

Preconfigured Schemas

The target includes built-in schema definitions for standard tables like:

• Syslog
• Common Security Log
• Security Event
• Windows Event
• ASim tables (various types)

These predefined schemas ensure proper column mapping and validation when sending data to Sentinel.

warning

Large buffer sizes or batch sizes increase memory usage.

Autodiscovery

Director provides an autodiscovery feature that automatically configures Data Collection Rules and their associated streams.

The required permissions are:

• For Data Collection Rules

  Role	Scope
  Monitoring Metrics Publisher	Each DCR with name starting with vmetric

• For Resource Groups

  Role	Scope
  Monitoring Reader	Resource Group containing your DCE

warning

Always assign the Monitoring Reader role at the Resource Group level, not at the Subscription level.

The feature uses Resource IDs to discover DCRs and their configurations. It then automatically detects table schemas and validates fields, and prevents phantom fields through schema validation.

Autodiscovery adapts to environment changes automatically.

Examples

Basic

Configuration using Resource ID-based autodiscovery:

targets:
  - name: auto_sentinel
    type: sentinel
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      endpoint: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myDCE"

Managed Identity

Using Managed Identity Authentication instead of App Registration:

targets:
  - name: managed_identity_sentinel
    type: sentinel
    properties:
      endpoint: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myDCE"

Filtered

Using specific stream filtering and custom cache timeout:

targets:
  - name: filtered_autodiscovery
    type: sentinel
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      endpoint: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myDCE"
      streams:
        - name: "Custom-WindowsEvent"
        - name: "Custom-SecurityEvent"
      cache:
        timeout: 300  # 5 minutes
      keep_phantom_fields: false
      drop_unknown_stream_events: true

High-Volume

Optimization for high-volume ingestion:

targets:
  - name: optimized_sentinel
    type: sentinel
    pipelines:
      - normalization
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      endpoint: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myDCE"
      buffer_size: 5242880  # 5MB
      batch_size: 5000
      field_format: "asim"
      streams:
        - name: "Custom-ASimProcessEventLogs"
        - name: "Custom-ASimNetworkSessionLogs"

With Debugging

Configuration with debug options:

targets:
  - name: debug_sentinel
    type: sentinel
    properties:
      tenant_id: "00000000-0000-0000-0000-000000000000"
      client_id: "00000000-0000-0000-0000-000000000000"
      client_secret: "your-client-secret"
      endpoint: "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionEndpoints/myDCE"
      debug:
        status: true
        dont_send_logs: true  # Test mode that doesn't actually upload