Codeless Connector Framework (CCF)

Synopsis

Creates a REST API poller driven by a Microsoft Sentinel Codeless Connector Framework (CCF) PollingConfig.json file. Secrets and environment-specific values referenced as {{placeholder}} tokens are supplied as separate device properties, keeping credentials out of the polling config.

Schema

- id: <numeric>
  name: <string>
  description: <string>
  type: ccf
  tags: <string[]>
  pipelines: <pipeline[]>
  status: <boolean>
  properties:
    ccf_config: <string>
    <placeholder_key>: <string>
    tls:
      insecure_skip_verify: <boolean>
    poll_interval: <numeric>
    workers: <numeric>
    reuse: <boolean>

Configuration

Device

Field	Required	Default	Description
id	Y		Unique numeric identifier
name	Y		Device name
description	N	-	Optional description
type	Y		Must be ccf
tags	N	-	Optional tags
pipelines	N	-	Optional preprocessing pipelines
status	N	true	Enable/disable the device

CCF Config

Field	Required	Description
ccf_config	Y	Contents of the connector's PollingConfig.json. Any {{placeholder}} token is resolved from a sibling device property with the same name.
<placeholder_key>	N*	Any property whose name matches a {{placeholder}} token in ccf_config. Typically used for secrets such as API keys and passwords.

* Required for each {{placeholder}} token present in ccf_config.

note

A CCF connector package contains multiple files. Only the PollingConfig.json is used here — the connector definition, DCR, and workbook files are not required.

The PollingConfig.json top-level fields connectorDefinitionName, dataType, and dcrConfig are ignored. Only auth, request, paging, and response are read.

[[concat(...)]] ARM template expressions found in some polling configs are not evaluated. Replace them with {{placeholder}} tokens and supply the resolved value as a sibling property.

TLS

Field	Required	Default	Description
tls.insecure_skip_verify	N	false	Skip TLS certificate verification (useful for dev/test environments or APIs with self-signed certificates)

Performance

These fields are device-level controls and are not part of the connector JSON.

Field	Required	Default	Description
poll_interval	N	10	Polling interval in seconds
workers	N	1	Number of concurrent queue processing workers
reuse	N	true	Enable multi-worker mode

Details

Placeholder Resolution

Placeholder resolution runs on the raw JSON string before parsing. If any placeholder cannot be resolved the device fails to start and reports all missing keys in the error message.

Defaults

When optional numeric fields such as retryCount, timeoutInSeconds, queryWindowInMin, and pageSize are omitted from the connector JSON, the following defaults are applied:

Field	Default
queryWindowInMin	5
retryCount	3
timeoutInSeconds	20
pageSize	100

If these fields are present in the connector JSON they are used as-is. An explicit value of 0 or negative causes the device to silently fall back to the default value listed above.

rateLimitQPS and maxPages are exceptions: 0 is a valid value for both. A rateLimitQPS of 0 disables rate limiting; a maxPages of 0 allows unlimited pages.

Examples

Basic Auth (Confluence Audit Logs)

Polling Confluence audit logs using Basic authentication with offset-based pagination...

devices: - id: 1 name: confluence_audit type: ccf properties: userid: "[email protected]" apikey: "my-secret-key" ccf_config: | { "kind": "RestApiPoller", "properties": { "auth": { "type": "Basic", "UserName": "{{userid}}", "Password": "{{apikey}}" }, "request": { "apiEndpoint": "https://myorg.atlassian.net/wiki/rest/api/audit", "httpMethod": "GET", "retryCount": 3, "timeoutInSeconds": 60, "queryTimeFormat": "UnixTimestampInMills", "startTimeAttributeName": "startDate", "endTimeAttributeName": "endDate", "headers": { "Accept": "application/json" } }, "paging": { "pagingType": "Offset", "offsetParaName": "start", "pageSizeParaName": "limit", "pageSize": 1000 }, "response": { "eventsJsonPaths": ["$.results"], "format": "json" } } }

OAuth2 (Auth0)

Polling Auth0 logs using OAuth2 client credentials with persistent token pagination across poll cycles...

devices: - id: 2 name: auth0_logs type: ccf properties: ClientId: "my-client-id" ClientSecret: "my-client-secret" Domain: "https://myorg.auth0.com" ccf_config: | { "kind": "RestApiPoller", "properties": { "auth": { "type": "OAuth2", "ClientId": "{{ClientId}}", "ClientSecret": "{{ClientSecret}}", "GrantType": "client_credentials", "TokenEndpoint": "{{Domain}}/oauth/token", "TokenEndpointQueryParameters": { "audience": "{{Domain}}/api/v2/" } }, "request": { "apiEndpoint": "{{Domain}}/api/v2/logs", "httpMethod": "GET", "headers": { "Accept": "application/json" } }, "Paging": { "pagingType": "PersistentToken", "nextPageParaName": "from", "nextPageTokenJsonPath": "$.[-1:].log_id", "PageSizeParameterName": "take", "PageSize": 100 }, "response": { "eventsJsonPaths": ["$"], "format": "json" } } }

API Key (Illumio)

Polling Illumio Insights using an API key header with a rate limit and extended query window...

devices: - id: 3 name: illumio_insights type: ccf properties: apiToken: "my-api-token" ccf_config: | { "kind": "RestApiPoller", "properties": { "auth": { "type": "APIKey", "ApiKey": "{{apiToken}}", "ApiKeyName": "X-api-secret" }, "request": { "apiEndpoint": "https://gw.console.illum.io/api/v1/insights-summary", "httpMethod": "GET", "rateLimitQPS": 10, "queryWindowInMin": 360, "retryCount": 3, "timeoutInSeconds": 60, "headers": { "Accept": "application/json" }, "queryParameters": { "persona": "threat_hunter", "report_interval": "daily" } }, "response": { "eventsJsonPaths": ["$"], "format": "json" } } }

With Pipeline Processing

Attaching preprocessing pipelines to normalize timestamps and enrich fields before events reach the route...

devices: - id: 4 name: ccf_with_pipeline type: ccf pipelines: - timestamp_normalizer - field_enrichment properties: apiKey: "my-key" ccf_config: | { "kind": "RestApiPoller", "properties": { "auth": { "type": "APIKey", "ApiKey": "{{apiKey}}", "ApiKeyName": "Authorization", "ApiKeyIdentifier": "Bearer" }, "request": { "apiEndpoint": "https://api.example.com/v1/events", "httpMethod": "GET" }, "response": { "eventsJsonPaths": ["$.events"], "format": "json" } } } poll_interval: 30 workers: 4