Microsoft Graph API Datasets

Microsoft Graph API datasets define what a Microsoft Graph API device reads from a Microsoft 365 tenant through Microsoft Graph (dataset type graphapi). They attach to devices of type graphapi, and in Datasets & Profiles the category is labelled Microsoft Graph API. To create and assign datasets through the web interface, see Management. For the dataset and profile model, see Overview.

Each dataset reads one Graph resource, on its own interval, through its own pipeline, with a saved position of its own. The device holds the connection: the app registration, the API version, the timeout, the retries and TLS. Every dataset on the device uses it.

Director version

A Director must run 2.6.1 or later to read Graph datasets. A Director at 2.6.0 or older refuses a Microsoft Graph API device that has no resource of its own. The device's Data collection step warns about this as soon as you choose Datasets or Profiles.

Collector Types

There are 34 dataset types: 33 that each read one Graph resource, and a custom type that reads a resource you name. Each is a tile in the wizard's Microsoft Graph API category, and every title starts with "Graph API". Intervals and delays are in seconds. "Read" is how the dataset reads its resource (see How a Dataset Reads).

Dataset (console title)	Definition	Definition id	Resource	Read	Kind	Default interval	Minimum	Delay
Graph API Sign-ins	`graphapi_signin_collector`	36100	`auditLogs/signIns`	window, `createdDateTime`	log	60	10	300
Graph API Directory Audits	`graphapi_directory_audit_collector`	36101	`auditLogs/directoryAudits`	window, `activityDateTime`	log	60	10	300
Graph API Provisioning Logs	`graphapi_provisioning_collector`	36102	`auditLogs/provisioning`	window, `activityDateTime`	log	60	10	300
Graph API Risk Detections	`graphapi_risk_detection_collector`	36103	`identityProtection/riskDetections`	changed since, `lastUpdatedDateTime`	log	300	60	600
Graph API Risky Users	`graphapi_risky_user_collector`	36104	`identityProtection/riskyUsers`	changed since, `riskLastUpdatedDateTime`	log	300	60	600
Graph API Risky Service Principals	`graphapi_risky_service_principal_collector`	36105	`identityProtection/riskyServicePrincipals`	changed since, `riskLastUpdatedDateTime`	log	900	300	600
Graph API Service Health Issues	`graphapi_service_issue_collector`	36106	`admin/serviceAnnouncement/issues`	window, `lastModifiedDateTime`	log	300	60	60
Graph API Message Center Posts	`graphapi_service_message_collector`	36107	`admin/serviceAnnouncement/messages`	window, `lastModifiedDateTime`	log	900	300	60
Graph API Service Health Overview	`graphapi_service_health_collector`	36108	`admin/serviceAnnouncement/healthOverviews`	whole	inventory	300	60	-
Graph API Security Alerts	`graphapi_alert_collector`	36109	`security/alerts_v2`	window, `lastUpdateDateTime`	log	60	30	120
Graph API Security Incidents	`graphapi_incident_collector`	36110	`security/incidents`	window, `lastUpdateDateTime`	log	60	30	120
Graph API Secure Score	`graphapi_secure_score_collector`	36111	`security/secureScores`	window, `createdDateTime`	log	21600	3600	0
Graph API Secure Score Controls	`graphapi_secure_score_control_collector`	36112	`security/secureScoreControlProfiles`	whole	inventory	86400	3600	-
Graph API Attack Simulations	`graphapi_attack_simulation_collector`	36113	`security/attackSimulation/simulations`	whole	log	3600	900	-
Graph API eDiscovery Cases	`graphapi_ediscovery_case_collector`	36114	`security/cases/ediscoveryCases`	changed since, `lastModifiedDateTime`	log	3600	900	0
Graph API Threat Intelligence Articles	`graphapi_threat_article_collector`	36115	`security/threatIntelligence/articles`	changed since, `lastUpdatedDateTime`	log	3600	900	0
Graph API Email Activity per User	`graphapi_email_activity_user_collector`	36116	`reports/getEmailActivityUserDetail`	by day	log	86400	3600	-
Graph API Email Activity Counts	`graphapi_email_activity_count_collector`	36117	`reports/getEmailActivityCounts`	changed since, `reportDate`	log	86400	3600	4 days
Graph API Email App Usage per User	`graphapi_email_app_usage_user_collector`	36118	`reports/getEmailAppUsageUserDetail`	by day	log	86400	3600	-
Graph API Mailbox Usage	`graphapi_mailbox_usage_collector`	36119	`reports/getMailboxUsageDetail`	whole, for a period	inventory	86400	3600	-
Graph API Microsoft 365 Activations per User	`graphapi_office365_activation_user_collector`	36120	`reports/getOffice365ActivationsUserDetail`	whole	inventory	86400	3600	-
Graph API Microsoft 365 Active Users	`graphapi_office365_active_user_collector`	36121	`reports/getOffice365ActiveUserDetail`	by day	log	86400	3600	-
Graph API Microsoft 365 Group Activity	`graphapi_office365_group_activity_collector`	36122	`reports/getOffice365GroupsActivityDetail`	by day	log	86400	3600	-
Graph API OneDrive Activity per User	`graphapi_onedrive_activity_user_collector`	36123	`reports/getOneDriveActivityUserDetail`	by day	log	86400	3600	-
Graph API OneDrive Usage per Account	`graphapi_onedrive_usage_account_collector`	36124	`reports/getOneDriveUsageAccountDetail`	by day	log	86400	3600	-
Graph API SharePoint Activity per User	`graphapi_sharepoint_activity_user_collector`	36125	`reports/getSharePointActivityUserDetail`	by day	log	86400	3600	-
Graph API SharePoint Site Usage	`graphapi_sharepoint_site_usage_collector`	36126	`reports/getSharePointSiteUsageDetail`	by day	log	86400	3600	-
Graph API Teams Activity per User	`graphapi_teams_user_activity_collector`	36127	`reports/getTeamsUserActivityUserDetail`	by day	log	86400	3600	-
Graph API Teams Device Usage	`graphapi_teams_device_usage_collector`	36128	`reports/getTeamsDeviceUsageUserDetail`	by day	log	86400	3600	-
Graph API Viva Engage Activity per User	`graphapi_viva_engage_activity_user_collector`	36129	`reports/getYammerActivityUserDetail`	by day	log	86400	3600	-
Graph API Authentication Method Registration	`graphapi_user_registration_collector`	36130	`reports/authenticationMethods/userRegistrationDetails`	whole	inventory	86400	3600	-
Graph API Exchange Message Trace	`graphapi_message_trace_collector`	36131	`admin/exchange/tracing/messageTraces`	window, `receivedDateTime`	log	300	60	300
Graph API Service Principal Risk Detections	`graphapi_service_principal_risk_detection_collector`	36132	`identityProtection/servicePrincipalRiskDetections`	changed since, `lastUpdatedDateTime`	log	900	300	600
Graph API Custom Resource	`graphapi_custom_collector`	36199	the one you name (see The Custom Dataset)	window, or whole	log	60	10	300

The kind is how the console lists a type. Every record goes to the custom log table, whatever its kind.

Permissions and Licences

Each dataset needs an application permission on the device's app registration, with admin consent. Some also need a licence in the tenant. Without the licence, Graph refuses the reading whatever the app's permissions: a tenant with no Entra ID P1 or P2, for example, answers sign-ins and authentication method registration with 403 Authentication_RequestFromNonPremiumTenantOrB2CTenant.

Dataset	Application permission	Licence
Sign-ins	`AuditLog.Read.All`	Microsoft Entra ID P1 or P2
Directory audits	`AuditLog.Read.All`
Provisioning logs	`AuditLog.Read.All` and `Directory.Read.All`
Risk detections	`IdentityRiskEvent.Read.All`	Microsoft Entra ID P1 or P2
Risky users	`IdentityRiskyUser.Read.All`	Microsoft Entra ID P2
Risky service principals	`IdentityRiskyServicePrincipal.Read.All`	Microsoft Entra Workload ID Premium
Service principal risk detections	`IdentityRiskEvent.Read.All`	Microsoft Entra Workload ID Premium
Service health issues, service health overview	`ServiceHealth.Read.All`
Message center posts	`ServiceMessage.Read.All`
Security alerts	`SecurityAlert.Read.All`
Security incidents	`SecurityIncident.Read.All`
Secure score, secure score controls	`SecurityEvents.Read.All`
Attack simulations	`AttackSimulation.Read.All`	Microsoft Defender for Office 365 Plan 2
eDiscovery cases	`eDiscovery.Read.All`	An eDiscovery licence
Threat intelligence articles	`ThreatIntelligence.Read.All`	Microsoft Defender XDR or Microsoft Sentinel
Every Microsoft 365 usage report	`Reports.Read.All`
Authentication method registration	`AuditLog.Read.All`	Microsoft Entra ID P1 or P2
Exchange message trace	`ExchangeMessageTrace.Read.All`	A one-time setup step (see Message Trace)
Custom resource	The permission the resource asks for	The licence the resource asks for

How a Dataset Reads

Window. Each reading reads the records whose time field falls in one window: from where the last window ended to delay seconds before the reading. Microsoft adds records to Graph some minutes after they happen, and a window that ended at the reading would miss the ones on their way. A record at the end of one window belongs to the next, so it is written once. The first reading starts initial_lookback seconds back.
Changed since. For a resource that Graph cannot filter by its time field, the dataset reads the whole resource and keeps the items whose time field moved into the window.
Whole. The dataset reads the whole resource and writes every item, at every reading.
By day. A usage report is read one day at a time, from the day after the last one read up to the last day that is report_lag_days old, at most a month in one reading.

Pages. A reading reads at most max_pages pages. A reading that reaches the limit saves the link to the next page, and the next reading reads on from it, so no page is skipped. A saved link that Graph refuses (an expired link) is dropped, and the window is read again from its start. That can write some records twice, but never skips one.

Saved positions. A dataset saves its position only once the reading's records are written, on disk in crash-resistant mode. A reading that fails saves nothing, and the next one reads the same records again. A saved position lasts 90 days without a reading. After a longer stop, the dataset starts again from its initial_lookback.

Moving a resource from the device to a dataset. The first time a dataset reads, it takes over the device's own position when the dataset reads by time window and the device read the same resource by the same field. The dataset then starts where the device stopped: nothing is missed and nothing is read twice. Every other dataset starts at its initial_lookback, and can write again records that the device already wrote.

Dataset Properties

Every dataset takes these properties. The console labels them on the dataset's Dataset configuration step, where they apply to the type.

Field	Console label	Default	Description
`interval`	Interval (Optional)	per type	Seconds between readings. At least the type's minimum, and at most 30 days (2592000).
`pipeline_name`	Pre-processing pipeline (Optional)	the device's	The dataset's own pre-processing pipeline. Two datasets on one device can reach two pipelines.
`initial_lookback`	Initial lookback (Optional)	the device's, else `3600`	Seconds back that a dataset with no saved position starts. It is cut to how long Graph keeps the records.
`delay`	Delay (Optional)	per type	Seconds behind a reading that its window ends.
`page_size`	Page size (Optional)	per type	The number of records asked for in each page, cut to the most Graph returns. See Limits.
`max_pages`	Maximum pages (Optional)	`10`	Pages one reading reads. The next reading reads on from where this one stopped. `0` reads every page.

Some types take more:

Field	Console label	Types	Default	Description
`sign_in_types`	Sign-in types (Optional)	Sign-ins	`interactiveUser`	Any of `interactiveUser`, `nonInteractiveUser`, `servicePrincipal` and `managedIdentity`. See Sign-ins.
`expand_alerts`	Each incident's alerts	Security incidents	`false`	`true` reads each incident with its alerts.
`report_lag_days`	Report lag (Optional)	The usage reports read by day	`4`	How many days old a day must be before it is read, from 1 to 27. See Usage Reports.
`period`	Period	Email activity counts, mailbox usage	`D7`	`D7`, `D30`, `D90` or `D180`: Last 7 days, Last 30 days, Last 90 days or Last 180 days.

The Custom Dataset

The custom dataset (Graph API Custom Resource) reads a resource that you name. It takes the properties above, and these:

Field	Console label	Default	Description
`resource`	Resource path	-	Required. A resource that the device may read (see Microsoft Graph API), or a path under one, such as one entity. A resource outside that list is refused.
`version`	API version	the device's	`v1.0` or `beta`. In the console, Same as the device uses the device's API version.
`query_params`	Query parameters (Optional)	-	OData query options sent with each request. With a timestamp field, the dataset adds its own time filter and keeps your `$filter` beside it.
`timestamp_field`	Timestamp field (Optional)	-	The date and time field to read windows by. Empty reads the whole resource at every reading.
`timestamp_operator`	Timestamp operator	`ge`	How the start of a window is asked for: `ge` (ge (on or after)), or `gt` (gt (after)) for a resource that refuses `ge`.

The custom type is the only one that can be assigned to a device more than once. Each custom dataset keeps its own position.
Its page size defaults to Graph's own. 0 leaves the page size to Graph.
A custom dataset that names a resource Microsoft retired, or one Graph cannot list, still loads, and the device's log says why it cannot be read.

Notes on the Types

Sign-ins

Graph returns interactive user sign-ins only, unless sign_in_types asks for more. Non-interactive user, service principal and managed identity sign-ins are read from Graph's beta endpoint.
Until it is confirmed that one dataset can mix user and service principal sign-in types, use a separate sign-in dataset for user sign-ins and for service principal or managed identity sign-ins.
Microsoft publishes a sign-in again when it adds risk information, and adds MFA details to the first one up to two hours later. The version read is the one Graph held when the window was read.

Changed Items Are Written Again

For security alerts, security incidents, risk detections, service principal risk detections, risky users and service health issues, every change to an item moves its time field, and the item is written again as a new record. That is how a change of status or classification reaches the pipeline.

security/alerts_v2 leaves out the Microsoft Sentinel alerts of a workspace that is not onboarded to the Defender portal, and Intune alerts.

Usage Reports

The reports read by day are read one day at a time, report_lag_days behind today. The default is 4 days, and the most is 27, because Graph keeps no older days. Microsoft takes 24 to 72 hours to prepare a day, and a live tenant had its last two or three days still empty.
A reading stops at an empty day that is less than a week old, and the next reading reads it again. An empty day that is a week old had no activity, and is passed.
Microsoft 365 hides user names, group names and site URLs in its usage reports by default, and a record then holds a hash in their place. A tenant administrator turns this off in the Microsoft 365 admin center's report settings, with the setting that shows concealed user, group and site names in all reports.
Email activity counts are read for a period. A record is written for each day, once that day is four days old.

Message Trace

The message trace dataset reads admin/exchange/tracing/messageTraces, Exchange Online's message trace in Graph. It needs the ExchangeMessageTrace.Read.All permission and a one-time setup step: before the first reading, a tenant administrator creates the service principal of Microsoft's Transport Data Platform app in the tenant:

az ad sp create --id 8bd644d1-64a1-4d4b-ae52-2e0cbf64e373

Until that is done, Graph refuses every reading with 403 and Service principal-less Authentication failed: the service principal for App ID 8bd644d1-... was not found. Microsoft says the refusals can go on for some hours after the step. A reading that is refused saves nothing, and the next one reads the same records again. A dataset that is catching up reads at most ten days in one reading.

eDiscovery Cases and Threat Intelligence Articles

Graph allows five eDiscovery requests a minute per app, so the eDiscovery cases dataset can be read every 15 minutes at most. A tenant without an eDiscovery licence answers 401 with no message.
Graph cannot filter or sort threat intelligence articles by their last update, so the whole list is read at each reading: a few thousand articles, in pages of 1,000.

Limits

One of each type. A type other than the custom one is read once per device. A second dataset of the same type is left out, and the device's log says so.
Interval. A dataset's interval is at most 30 days. An interval under the type's minimum, or over 30 days, leaves the dataset out, with an error in the device's log that says why. The device's other datasets go on.
Page size. Some types read a list for which Graph documents no page size. Their page_size is not used, and the device logs that it is not. The types that take one are sign-ins and directory audits (1,000), risk detections and risky users (500), secure score controls and threat intelligence articles (1,000), message trace (1,000, at most 5,000), and the custom type.
Records too large to write. A record too large for the Director to write is logged as an error and left out, and the rest of the reading is written. A dataset that reads by time window, by day or by changed items moves past the record. A dataset that reads its whole resource meets it again, and logs it again, at every reading.
Throttling. Microsoft's limits are per app and per tenant, so the datasets on one device share them: 5 requests per 10 seconds for the audit logs, 150 per minute for security, and 100 per 5 minutes per tenant for message trace. Identity protection allows 1 request per second per tenant, shared by every app in the tenant. A throttled request is sent again, as the device's max_retries and retry_delay say.
Refusals. A dataset that Graph refuses for a missing permission, licence or setup step pauses before it reads again: first for a minute, then twice as long after each refusal, up to 15 minutes. The device's connection state names the dataset. A dataset's failure is its own: the device's other datasets and its resource go on.

Records

Each record is written as Graph wrote it, through the dataset's pipeline, and carries the type's definition id. Each record also carries event.dataset, graphapi.<key> (for example graphapi.signin, graphapi.message_trace or graphapi.custom), so a pipeline that takes the records of several datasets can tell them apart. A record that has an event object of its own gets the dataset added to it, unless it already names one.

To parse the records, install the Microsoft Graph API pack (ms_microsoft_graph_api) from Content Hub, with the Microsoft Entra ID pack it depends on:

The Entra ID datasets go to the ASIM tables.
Message traces go to EmailEvents. On Microsoft Sentinel, that means a custom table, such as EmailEvents_CL.
Every other dataset is passed on with Graph's field names and its event time.

Examples

Reading interactive and non-interactive user sign-ins, security alerts, each user's Microsoft 365 activity a day at a time, and the directory audits through a custom dataset...

- id: 588104372
  name: m365-tenant
  type: graphapi
  properties:
    tenant_id: "00000000-0000-0000-0000-000000000000"
    client_id: "11111111-1111-1111-1111-111111111111"
    client_secret: "$secret{store=corp-vault,ref=graph/client-secret}"
  definitions:
    - name: graphapi_signin_collector
      properties:
        interval: 60
        sign_in_types: [interactiveUser, nonInteractiveUser]
    - name: graphapi_alert_collector
    - name: graphapi_office365_active_user_collector
      properties:
        report_lag_days: 4
    - name: graphapi_custom_collector
      properties:
        resource: auditLogs/directoryAudits
        query_params: "$select=id,activityDateTime,activityDisplayName"
        timestamp_field: activityDateTime
        timestamp_operator: ge

Collector Types​

Permissions and Licences​

How a Dataset Reads​

Dataset Properties​

The Custom Dataset​

Notes on the Types​

Sign-ins​

Changed Items Are Written Again​

Usage Reports​

Message Trace​

eDiscovery Cases and Threat Intelligence Articles​

Limits​

Records​

Examples​