Microsoft Sentinel

Microsoft Azure SIEM Pull

Synopsis

Creates a collector that reads the security incidents of a Microsoft Sentinel workspace, and their alerts, bookmarks and comments from the datasets assigned to the device. Director polls the source system's API at configured intervals to retrieve log and telemetry data rather than receiving pushed events. Supports authentication, batch processing, and automatic incident tracking with incremental updates. See Datasets.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: sentinel
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    tenant_id: <string>
    client_id: <string>
    client_secret: <string>
    subscription_id: <string>
    resource_group_name: <string>
    workspace_name: <string>
    batch_size: <numeric>
    initial_lookback: <numeric>

Configuration

The following fields are used to define the device:

Device

Field	Required	Default	Description
id	Y		Unique identifier
name	Y		Device name
description	N	-	Optional description
type	Y		Must be sentinel
tags	N	-	Optional tags
pipelines	N	-	Optional pre-processor pipelines
status	N	true	Enable/disable the device

Azure Authentication

Field	Required	Default	Description
tenant_id	N*		Azure tenant ID
client_id	N*		Azure client ID
client_secret	N*		Azure client secret
subscription_id	Y		Azure subscription ID

* = Omit all three to authenticate with a Managed Identity. See below.

Managed Identity Authentication

Azure targets support Managed Identity authentication for credential-free access when Director is deployed on Azure infrastructure.

How it works: When tenant_id, client_id, and client_secret are omitted from the configuration, the target automatically uses Azure's DefaultAzureCredential, which attempts authentication in the following order:

1. Environment variables (AZURE_TENANT_ID, AZURE_CLIENT_ID, AZURE_CLIENT_SECRET)
2. Workload Identity (for Kubernetes deployments)
3. Managed Identity (system-assigned or user-assigned)
4. Azure CLI credentials
5. Azure PowerShell credentials

Deployment requirement: Director must run on Azure infrastructure that supports Managed Identity:

• Azure Virtual Machines
• Azure App Service
• Azure Container Instances
• Azure Kubernetes Service (AKS)
• Azure Functions

Required permissions: The Managed Identity must be granted the appropriate Azure RBAC roles documented in each target's IAM Permissions section.

note

Managed Identity eliminates credential management overhead and is the recommended authentication method for Azure-hosted Director deployments.

Workspaces

Field	Required	Default	Description
resource_group_name	Y		Azure resource group name
workspace_name	Y		Log Analytics workspace name

Collection

Field	Required	Default	Description
batch_size	N	1000	Number of incidents to fetch per page, from 1 to 1000. A larger value is read as 1000.
initial_lookback	N	86400	How far back, in seconds, the device's first poll looks when it has no checkpoint yet. 0 starts at the time of that first poll, with nothing earlier. The incidents and the device's Incident Alerts, Incident Bookmarks and Incident Comments datasets all start there; a dataset's own initial_lookback overrides the device's. A negative value falls back to the default. In the console this is Initial Lookback (Seconds) on Advanced settings; it affects only a reading that has no checkpoint yet, so changing it on a device that has already read changes nothing that was read.

Details

IAM Permissions

The service principal requires the following Azure RBAC role:

Azure Role	Scope	Purpose
Microsoft Sentinel Reader	Log Analytics Workspace	Read incidents from Microsoft Sentinel

The device is read-only. It lists incidents using Microsoft.SecurityInsights/incidents/read, and with datasets assigned it also lists the alerts, bookmarks and comments of each changed incident. The alert and bookmark lists are POST list actions that change nothing; Microsoft Sentinel Reader covers them through Microsoft.SecurityInsights/*/read, as it covers the comments (Microsoft.SecurityInsights/incidents/comments/read). No create, update, or delete operations are performed.

Assign the role at the Log Analytics Workspace level rather than the subscription or resource group level for least-privilege access.

Incidents

The collector captures comprehensive incident data including basic incident details (ID, title, description), severity and status, classification and labels, owner information, temporal data (create, modify, activity dates and times), and information on resources.

Incremental Updates

The collector tracks the last-modified time of the incidents it has written, fetches the incidents created or changed since then, oldest change first, and writes each change of an incident as a record of its own. A change it has already written is not written again.

A change of an incident's status, owner, severity, classification or labels is therefore a new record, as in Sentinel's own SecurityIncident table, which appends a row per change. Each record is dated when the change happened (the incident's last-modified time); event.created still holds the creation time.

A query that needs an incident's current state takes the latest record per incident, as with the SecurityIncident table: in KQL, arg_max(TimeGenerated, *) by IncidentNumber, or the equivalent on the record's event.sequence (the incident number).

note

After an upgrade from a version that wrote each incident only once, the first poll writes the current state of every incident changed since the device's last checkpoint once more. Nothing is lost; that one batch repeats.

Normalization

Incidents are automatically normalized with ECS field mapping in addition to using consistent timestamp formats, structured label handling, and owner information. (See Appendix for details of ECS.)

Datasets

The incidents are built in: the device reads them as described above whatever datasets it has. Three datasets can be assigned on top, to read what each incident holds. In the console they are the datasets and profiles assigned to the device in its Data collection, each a tile under Datasets › Microsoft Sentinel; see Datasets and Profiles. In a configuration file each dataset is a definition named after its type:

Dataset	Definition	event.dataset	What it reads
Sentinel Incident Alerts	sentinel_incident_alert_collector	sentinel.incident_alert	The alerts of every incident modified since the last reading, each alert once
Sentinel Incident Bookmarks	sentinel_incident_bookmark_collector	sentinel.incident_bookmark	The hunting bookmarks of each changed incident, and a bookmark again when it is edited
Sentinel Incident Comments	sentinel_incident_comment_collector	sentinel.incident_comment	The comments on each changed incident, and a comment again when it is edited

Each dataset is read on its own interval, set with interval: 300 seconds by default, at least 60.

• Joining to incidents. Every record carries its incident's ARM ID (cloud.resource_id), number (event.sequence) and title (event.name), the same paths the incident records use, so the two can be joined.
• Checkpoints. Each dataset resumes from where its last delivered reading stopped, across a restart or a failover. Its first reading goes back as far as the initial lookback: the dataset's own initial_lookback, else the device's, one day by default.
• Director version. The datasets need Director 2.2.3 or later. An older Director keeps reading the incidents and ignores the datasets assigned to the device; the console warns about this.

Examples

Basic

Creating a simple Sentinel collector...

- id: 1 name: basic_sentinel type: sentinel properties: tenant_id: "00000000-0000-0000-0000-000000000000" client_id: "11111111-1111-1111-1111-111111111111" client_secret: "your-client-secret" subscription_id: "22222222-2222-2222-2222-222222222222" resource_group_name: "your-resource-group" workspace_name: "your-workspace"

High-Volume

Optimizing for high incident volumes...

- id: 2 name: volume_sentinel type: sentinel properties: tenant_id: "00000000-0000-0000-0000-000000000000" client_id: "11111111-1111-1111-1111-111111111111" client_secret: "your-client-secret" subscription_id: "22222222-2222-2222-2222-222222222222" resource_group_name: "your-resource-group" workspace_name: "your-workspace" batch_size: 1000

Pipelines

Applying custom processing to incidents...

- id: 3 name: pipeline_sentinel type: sentinel pipelines: - incident_enricher - severity_classifier properties: tenant_id: "00000000-0000-0000-0000-000000000000" client_id: "11111111-1111-1111-1111-111111111111" client_secret: "your-client-secret" subscription_id: "22222222-2222-2222-2222-222222222222" resource_group_name: "your-resource-group" workspace_name: "your-workspace" batch_size: 1000

Multiple Workspaces

Configuring multiple workspace collectors...

- id: 4 name: prod_sentinel type: sentinel properties: tenant_id: "00000000-0000-0000-0000-000000000000" client_id: "11111111-1111-1111-1111-111111111111" client_secret: "your-client-secret" subscription_id: "22222222-2222-2222-2222-222222222222" resource_group_name: "prod-rg" workspace_name: "prod-workspace" - id: 5 name: dev_sentinel type: sentinel properties: tenant_id: "00000000-0000-0000-0000-000000000000" client_id: "11111111-1111-1111-1111-111111111111" client_secret: "your-client-secret" subscription_id: "22222222-2222-2222-2222-222222222222" resource_group_name: "dev-rg" workspace_name: "dev-workspace"

Incident Fields

The collector maps incident fields to standardized ECS fields:

Sentinel Field	ECS Field	Description
Title	event.name	Incident title
Description	event.description	Incident description
Severity	event.severity	Incident severity level
Status	event.outcome	Current incident status
IncidentNumber	event.sequence	Sequential incident number
IncidentURL	url.original	Link to the incident in the portal
Classification	event.classification	Incident classification
ClassificationReason	event.classification_reason	Why that classification was applied
ClassificationComment	event.reason	Free-text comment on the classification
CreatedTime	event.created	Incident creation time
LastModifiedTime	event.last_modified	Last update time
FirstActivityTime	event.start	First detected activity
LastActivityTime	event.end	Latest detected activity
OwnerAssignedTo	user.name	Assigned owner name
OwnerEmail	user.email	Assigned owner email
OwnerUserPrincipalName	user.principal_name	Assigned owner UPN
OwnerObjectID	user.id	Assigned owner object ID
ResourceID	cloud.resource_id	Azure resource ID
ResourceName	cloud.resource_name	Azure resource name
ResourceType	cloud.resource_type	Azure resource type
Etag	cloud.resource_version	Resource version (ETag)
Labels	labels	Incident labels

All timestamps are normalized at nanosecond level in UTC.