Skip to main content

Azure Monitor

Microsoft Azure Pull

Synopsis​

Creates an Azure Monitor collector that gathers alerts, logs, and metrics from Azure. Director polls the source system's API at configured intervals to retrieve log and telemetry data rather than receiving pushed events. Each collection type maintains its own checkpoint for incremental updates.

The collector is available as four device types that run the same module:

TypeCollects
azmonAlerts, Log Analytics tables and metrics, concurrently. Collection types that are not configured are skipped automatically
azlogsLog Analytics tables only
azalertsAlerts only
azmetricsMetrics only

azlogs, azalerts and azmetrics each run their one collection type alone, so the properties of the other two are not read.

In the web interface, the Azure Logs, Azure Alerts and Azure Metrics cards create azlogs, azalerts and azmetrics devices. azmon has no card, so it is configured in a Director configuration file. The last step of each of the three forms ends with the Simulation section; see Simulation.

Schema​

- id: <numeric>
name: <string>
description: <string>
type: <string>
tags: <string[]>
pipelines: <pipeline[]>
status: <boolean>
properties:
tenant_id: <string>
client_id: <string>
client_secret: <string>
workspace_id: <string>
stream: <string[]>
subscription_id: <string>
alert_rule: <string>
alert_state: <string>
monitor_condition: <string>
monitor_service: <string>
severity: <string>
smart_group_id: <string>
target_resource: <string>
target_resource_group: <string>
target_resource_type: <string>
resource_uri: <string>
namespace: <string>
metrics: <string[]>
aggregation: <string[]>
interval: <string>
order_by: <string>
filter: <string>
result_type: <string>
batch_size: <numeric>
event_frequency: <numeric>

Configuration​

The following fields are used to define the device.

Device​

FieldRequiredDefaultDescription
idYUnique identifier
nameYDevice name
descriptionN-Optional description
typeYazmon, azlogs, azalerts or azmetrics
tagsN-Optional tags
pipelinesN-Optional pre-processor pipelines
statusNtrueEnable/disable the device

Authentication​

All collection types share a single set of credentials. With all three fields set, the device signs in as that service principal. With any of them left empty, it uses the default credential chain (DefaultAzureCredential) instead, tried in order: environment variables, workload identity, managed identity, then developer sign-ins such as the Azure CLI; a tenant_id that is set is still passed to the chain.

FieldRequiredDefaultDescription
tenant_idN-Azure tenant ID. Required for a service principal
client_idN-Client ID of the service principal. Required for a service principal
client_secretN-Client secret of the service principal. Required for a service principal

Collection​

FieldRequiredDefaultDescription
batch_sizeNPer typeLogs: rows read from each table per round (10000). Alerts: alerts per page (250). Metrics: time series one query returns (10000)
event_frequencyN-Seconds between two collection rounds, 1 or more. When it is unset, not a number or below 1, the device keeps the cadence described under Collection Architecture

In the web interface, event_frequency is Event frequency (Optional) on all three forms, and a new device starts with 300. A cleared field is not sent, so the device falls back to the cadence described under Collection Architecture. The field is on these steps:

CardStepWhere
Azure LogsAdvanced settingsBeside Batch size (Optional)
Azure AlertsAlert filtersUnder the Polling heading after the filters, beside Batch size (Optional)
Azure MetricsQuery settingsOn its own row, the last field of the step

Logs​

Read by azmon and azlogs.

FieldRequiredDefaultDescription
workspace_idYLog Analytics workspace ID
streamYWorkspace tables to read, such as SecurityEvent or Syslog

Alerts​

Read by azmon and azalerts. The filters are optional; an empty one is not sent.

FieldRequiredDefaultDescription
subscription_idYAzure subscription ID whose alerts are read
alert_ruleN-Only alerts of this alert rule
alert_stateN-New, Acknowledged or Closed
monitor_conditionN-Fired or Resolved
monitor_serviceN-The service that raised the alert, such as Platform, Log Analytics, Application Insights or ActivityLog Administrative
severityN-Sev0 to Sev4
smart_group_idN-Only alerts of this smart group
target_resourceN-Only alerts on this resource
target_resource_groupN-Only alerts on resources in this resource group
target_resource_typeN-Only alerts on resources of this type

The first round reads the alerts of the last 30 days. Every later round reads from the checkpoint onward, oldest first.

Metrics​

Read by azmon and azmetrics. Metrics are read from resource_uri alone, so subscription_id is not used here.

FieldRequiredDefaultDescription
resource_uriYResource ID of the Azure resource whose metrics are read
namespaceN-Metric namespace
metricsN-Metric names, as a list or a comma-separated string. See Microsoft's metrics index for the names each resource type has
aggregationN-Average, Count, Maximum, Minimum, None or Total, as one value or a list
intervalNPT1MTime grain of the series, as an ISO 8601 duration
order_byNTimeStamp ascSort order of the results
filterN-OData filter on the metric dimensions
result_typeN-Data or Metadata. When unset, Azure returns Data. With Data, the device sends one record per data point. Metadata collects nothing: a Metadata query returns no data points, so the device sends no records. In the web interface, a new Azure Metrics device offers only Data under Result type (Optional), and a device stored with Metadata keeps it

Details​

Collection Architecture​

azmon runs alerts, logs, and metrics collection concurrently via separate goroutines within a single collection round. azlogs, azalerts and azmetrics run their one type in the same way. Each collection type maintains its own checkpoint keyed by device ID and type, so a failure in one type does not affect the others' progress.

After each round the device waits event_frequency seconds before the next one. Without event_frequency, the cadence is set service-wide through the FrequencySettings Definition on the platform (the event collector frequency), or 5 seconds when that sets none.

Every round reads the device as it is configured at that moment, so an edit to the tables, the filters, batch_size or event_frequency applies from the next round without a restart. A change to tenant_id, client_id, client_secret, subscription_id or workspace_id restarts the device.

On azmon, subscription_id is required to query alerts, resource_uri to query metrics, and workspace_id with stream to query Log Analytics tables. Any combination can be provided; collection types with missing configuration are skipped.

Replay​

The azlogsreplay device type re-reads Log Analytics workspace tables over a bounded historical window. The window, chunking and run controls are described in Replay Devices.

note

The replay covers logs only. Alerts and metrics have no replay: this device type reads workspace tables, so a replay device selects the tables it reads rather than collecting all three types the way the azmon device does.

Each selected table is queried once per chunk, bounded on TimeGenerated, and the rows are paged by batch_size. The checkpoints of the azmon and azlogs devices are never touched.

FieldRequiredDefaultDescription
workspace_idY-Log Analytics workspace to read from
streamY-Workspace tables to replay, such as SecurityEvent or Syslog
batch_sizeN10000Rows per page within a chunk
start_dateY-Lower bound of the replay window
end_dateN-Upper bound, exclusive. Unset means an open window that follows the present
timezoneNUTCZone used to read an offset-less bound
window_chunkN3600Seconds of the window covered by one query. Minimum 60
intervalN60Seconds between runs while the window still has work
cronN-Cron expression for the run, taking precedence over interval

A Microsoft Sentinel workspace is a Log Analytics workspace, so azlogsreplay covers Sentinel table history as well.

Examples​

Basic​

Collecting all three data types with one azmon device...

- id: 1
name: azmon
type: azmon
properties:
tenant_id: "00000000-0000-0000-0000-000000000000"
client_id: "11111111-1111-1111-1111-111111111111"
client_secret: "your-client-secret"
subscription_id: "22222222-2222-2222-2222-222222222222"
workspace_id: "33333333-3333-3333-3333-333333333333"
stream:
- SecurityEvent
- Syslog
resource_uri: "/subscriptions/22222222-2222-2222-2222-222222222222/resourceGroups/web/providers/Microsoft.Compute/virtualMachines/web-01"

Logs Only​

Reading two Log Analytics tables every five minutes...

- id: 2
name: azure_logs
type: azlogs
properties:
tenant_id: "00000000-0000-0000-0000-000000000000"
client_id: "11111111-1111-1111-1111-111111111111"
client_secret: "your-client-secret"
workspace_id: "33333333-3333-3333-3333-333333333333"
stream:
- SecurityEvent
- Syslog
batch_size: 1000
event_frequency: 300

Alerts Only​

Reading the fired alerts of severity 0 in one subscription...

- id: 3
name: azure_alerts
type: azalerts
properties:
tenant_id: "00000000-0000-0000-0000-000000000000"
client_id: "11111111-1111-1111-1111-111111111111"
client_secret: "your-client-secret"
subscription_id: "22222222-2222-2222-2222-222222222222"
monitor_condition: Fired
severity: Sev0

Metrics Only​

Reading the average CPU of one virtual machine per minute...

- id: 4
name: azure_metrics
type: azmetrics
properties:
tenant_id: "00000000-0000-0000-0000-000000000000"
client_id: "11111111-1111-1111-1111-111111111111"
client_secret: "your-client-secret"
resource_uri: "/subscriptions/22222222-2222-2222-2222-222222222222/resourceGroups/web/providers/Microsoft.Compute/virtualMachines/web-01"
metrics:
- Percentage CPU
aggregation:
- Average
interval: PT1M