Azure Service Bus
Synopsis
Creates a target that sends processed messages to Azure Service Bus queues or topics with support for multiple authentication methods, batch processing, and advanced message properties. Provides reliable message delivery to Azure Service Bus for decoupled application architectures and event-driven systems.
To receive from a Service Bus queue or topic subscription, use the Azure Service Bus device. It signs in with the same fields as this target.
Schema
- name: <string>
description: <string>
type: azservicebus
pipelines: <pipeline[]>
status: <boolean>
properties:
client_connection_string: <string>
tenant_id: <string>
client_id: <string>
client_secret: <string>
namespace: <string>
transport: <string>
queue: <string>
topic: <string>
session_id: <string>
correlation_id: <string>
content_type: <string>
time_to_live: <numeric>
max_events: <numeric>
timeout: <numeric>
field_format: <string>
debug:
status: <boolean>
dont_send_logs: <boolean>
Configuration
The following fields are used to define the target:
| Field | Required | Default | Description |
|---|---|---|---|
name | Y | Target name | |
description | N | - | Optional description |
type | Y | Must be azservicebus | |
pipelines | N | - | Optional post-processor pipelines |
status | N | true | Enable/disable the target |
Connection
The sign-in method is chosen by which fields are set, and the first match wins. Fields are compared after ${ENV} and $secret{...} references are resolved, so a reference that resolves to an empty value counts as unset.
| Fields set | Sign-in | Needs namespace |
|---|---|---|
client_connection_string | Connection string (shared access key). The other sign-in fields are ignored | N |
tenant_id, client_id, and client_secret | Service principal | Y |
client_id only | User-assigned managed identity. On a pod with workload identity federation, it signs in as the workload identity | Y |
| None of the three | Default credential chain: system-assigned managed identity, workload identity, and the rest of the chain | Y |
Any other mix, such as tenant_id alone, client_secret alone, or two of the three | Rejected at validation | - |
The namespace field accepts either form:
namespace value | Host dialled |
|---|---|
A bare name without a dot, such as mynamespace | mynamespace.servicebus.windows.net |
A host name containing a dot, such as mynamespace.servicebus.windows.net or mynamespace.servicebus.chinacloudapi.cn | Exactly as given |
Transport
| Field | Required | Default | Description |
|---|---|---|---|
transport | N | amqp | amqp connects with AMQP over TLS on TCP port 5671. amqp_websockets connects with AMQP over WebSockets on TCP port 443, and its WebSocket dial honours HTTPS_PROXY. Use amqp_websockets on networks that only allow HTTPS egress, or only through a proxy. Values are matched case-insensitively. Applies to every sign-in method, the connection string included |
Method 1: Connection String Authentication
| Field | Required | Default | Description |
|---|---|---|---|
client_connection_string | Y* | Service Bus connection string. When set, the other sign-in fields are ignored | |
queue | N** | Queue name to send messages to | |
topic | N** | Topic name to send messages to |
Method 2: Service Principal Authentication
| Field | Required | Default | Description |
|---|---|---|---|
tenant_id | Y* | Azure tenant ID | |
client_id | Y* | Azure service principal client ID | |
client_secret | Y* | Azure service principal client secret | |
namespace | Y* | Service Bus namespace, as a bare name or a full host name (see above). Required for every sign-in except a connection string | |
queue | N** | Queue name to send messages to | |
topic | N** | Topic name to send messages to |
Method 3: Managed Identity
Set client_id alone to sign in as a user-assigned managed identity, or leave tenant_id, client_id, and client_secret all empty to use the default credential chain. Both need namespace.
| Field | Required | Default | Description |
|---|---|---|---|
client_id | N | Client ID of a user-assigned managed identity attached to the resource Director runs on, or, on a pod with workload identity federation, of the identity the pod federates as. Set it without tenant_id and client_secret | |
namespace | Y* | Service Bus namespace, as a bare name or a full host name | |
queue | N** | Queue name to send messages to | |
topic | N** | Topic name to send messages to |
* = Conditionally required (see the sign-in table above)
** = Either queue or topic must be specified, but not both
Managed Identity Authentication
Azure targets support Managed Identity authentication for credential-free access when Director is deployed on Azure infrastructure.
How it works: When tenant_id, client_id, and client_secret are omitted from the configuration, the target automatically uses Azure's DefaultAzureCredential, which attempts authentication in the following order:
- Environment variables (
AZURE_TENANT_ID,AZURE_CLIENT_ID,AZURE_CLIENT_SECRET) - Workload Identity (for Kubernetes deployments)
- Managed Identity (system-assigned or user-assigned)
- Azure CLI credentials
- Azure PowerShell credentials
Deployment requirement: Director must run on Azure infrastructure that supports Managed Identity:
- Azure Virtual Machines
- Azure App Service
- Azure Container Instances
- Azure Kubernetes Service (AKS)
- Azure Functions
Required permissions: The Managed Identity must be granted the appropriate Azure RBAC roles documented in each target's IAM Permissions section.
Managed Identity eliminates credential management overhead and is the recommended authentication method for Azure-hosted Director deployments.
Message Properties
| Field | Required | Default | Description |
|---|---|---|---|
session_id | N | - | Session ID for session-enabled queues/topics |
correlation_id | N | - | Correlation ID for request-response patterns |
content_type | N | application/json | Message content type |
time_to_live | N | - | Message time-to-live in seconds |
Performance
| Field | Required | Default | Description |
|---|---|---|---|
max_events | N | 1000 | Maximum number of messages per batch |
field_format | N | - | Data normalization format. See applicable Normalization section |
timeout | N | 30 | Seconds bounding a single batch send |
Scheduling
See Scheduling and Pool Behavior for interval and cron fields shared by all targets.
Debug Options
| Field | Required | Default | Description |
|---|---|---|---|
debug.status | N | false | Enable debug logging |
debug.dont_send_logs | N | false | Process logs but don't send to target (testing) |
Details
The Azure Service Bus target sends processed messages to Azure Service Bus queues or topics for reliable message delivery in distributed systems. It supports automatic batching for optimal performance, advanced message properties for complex workflows, and multiple authentication methods for flexible deployment scenarios.
Messages are sent in batches to improve throughput and reduce network overhead. The target handles connection pooling and automatic reconnection on network failures.
IAM Permissions
When using Service Principal or Managed Identity authentication, including a user-assigned managed identity, the following Azure role is required:
| Azure Role | Scope | Purpose |
|---|---|---|
Azure Service Bus Data Sender | Service Bus Namespace, Queue, or Topic | Send message batches to queue or topic |
When using connection string authentication, Azure RBAC roles are not applicable -- the connection string must include Send permission.
Queues vs Topics
Queues
- Point-to-point messaging
- Single consumer per message
- FIFO ordering (optional)
- Ideal for task distribution and load leveling
Topics
- Publish-subscribe messaging
- Multiple subscribers per message
- Message filtering with subscriptions
- Ideal for event broadcasting and fan-out scenarios
Session Support
Session-enabled queues and topics provide FIFO guarantees for messages with the same session ID. Use the session_id property to enable session-based processing for ordered message delivery.
Message Properties
The target supports setting custom message properties:
- Correlation ID: Links related messages in request-response patterns
- Content type: MIME type of message body
- Time to live: Automatic message expiration
The target does not set a message ID. Service Bus assigns each message its own, so duplicate detection on the queue or topic does not drop records as repeats of one another. A message_id saved with a target is ignored, and the target logs a Warning when it starts. The console no longer offers a
Batch Processing
Messages are accumulated in memory and sent in batches when the batch size limit is reached or during finalization. The maximum batch size is configurable via the max_events parameter.
Dead Letter Queue
Azure Service Bus automatically moves messages to the dead letter queue when they exceed max delivery count or expire. Configure these settings in the Azure portal or via infrastructure as code.
At-Least-Once Delivery
Service Bus guarantees at-least-once delivery. Messages may be delivered more than once in case of network failures or processing errors. Design your message handlers to be idempotent.
Examples
The following are commonly used configuration types.
Basic with Connection String
Creating a basic Service Bus queue target with connection string... | |
Target sends JSON messages to Service Bus queue in batches... | |
Service Principal Authentication
Using service principal authentication for secure access... | |
Managed Identity
Using Managed Identity for credential-free authentication on Azure infrastructure... | |
Signing in as a user-assigned managed identity by setting | |
Session-Enabled Queue
Using sessions for ordered message processing... | |
Request-Response Pattern
Using correlation ID for request-response messaging... | |
Message Expiration
Setting time-to-live for automatic message expiration... | |
Topic with Subscriptions
Publishing messages to a topic for multiple subscribers... | |
Pipeline Processing
Applying post-processing pipelines before sending... | |
Field Normalization
Using field normalization for standard format... | |
High-Volume Configuration
Optimizing for high-volume message sending... | |
Troubleshooting
This section covers the errors you are most likely to see with the azservicebus target, what causes each one, and how to fix it.
Where to look:
- Director logs. Target errors are tagged with the target name and carry
"Section":"SenderPool". The part afterReason:or after the last colon is the actual cause. - The target's connection status in the web interface. For this target the status only reflects startup. Configuration and credential-resolution problems appear there as
connection failed for <target name>: <reason>. Anything that needs the network, which includes every authentication and permission problem, does not. The status stays atconnection ready for <target name>while the log fills with errors, so for this target read the log rather than the status.
See Target Delivery Errors for how Director logs and retries target failures.
Which permission is missing?
Match the error against this table first. Which right is missing depends on the authentication method.
| Error text | Missing role or right | Scope |
|---|---|---|
failed to create message batch: (unauthorized): rpc: failed, status code 401 ... | A current SharedAccessKey for the policy named in SharedAccessKeyName. The key is wrong, was rotated, or the signature has expired | The |
failed to create message batch: (unauthorized): *Error{Condition: amqp:unauthorized-access ...}, connection string mode | A shared access policy carrying the Send claim for the configured queue or topic | The namespace, or that same queue or topic. A string containing EntityPath= grants rights on that one entity only |
The same amqp:unauthorized-access error with tenant_id, client_id and client_secret set, or with Managed Identity | Azure Service Bus Data Sender | The Service Bus namespace, or the specific queue or topic |
ClientSecretCredential authentication failed. ... AADSTS7000215 (or AADSTS7000222, AADSTS700016) | A valid client secret value for an app registered in tenant_id. No Service Bus right is reached yet | The app registration in Microsoft Entra ID |
DefaultAzureCredential: failed to acquire a token. or ManagedIdentityCredential authentication failed. | A managed identity assigned to the resource Director runs on | The Azure virtual machine, container or cluster hosting Director |
No management rights (Manage or Listen) are needed. The queue or topic must already exist and be active. The target never creates it.
The status says connected but "ThreadSafeInit on reinit failed" repeats every few seconds
[Error] [director] [target-<target id>] [basic_servicebus_queue] Sender worker 1 ThreadSafeInit on reinit failed for "basic_servicebus_queue": failed to create message batch: <reason>
[Information] [director] [target-<target id>] [basic_servicebus_queue] Target "basic_servicebus_queue" reinitialized successfully (generation 7).
Cause: the target contacts Azure for the first time when it sends, not when it starts. Startup only builds the client from the configuration, so a wrong key, a missing role, a mistyped namespace and a missing queue all pass it, and the status shows connected. The first send is the first real test. It opens the connection, and every authentication, DNS and entity error surfaces there as failed to create message batch: followed by the cause. Each failure reinitializes the target, which logs reinitialized successfully with a higher generation number, and the next delivery fails the same way.
Fix: read the reason after failed to create message batch: and match it against the entries below. A climbing generation counter next to a connected status is always this pattern.
Data waits in the Director queue and is redelivered about every 5 seconds. It is kept for queue.limit seconds, 48 hours by default, and with the In-memory storage tier it does not survive a Director restart.
"failed to create message batch: (unauthorized)"
Sender worker 2 ThreadSafeInit on reinit failed for "basic_servicebus_queue": failed to create message batch: (unauthorized): rpc: failed, status code 401 and description: ...
Sender worker 1 ThreadSafeInit on reinit failed for "basic_servicebus_queue": failed to create message batch: (unauthorized): *Error{Condition: amqp:unauthorized-access, Description: ..., Info: map[]}
Cause: two forms, and the second half of the line tells them apart.
rpc: failed, status code 401means the token itself was refused. In connection string mode theSharedAccessKeyis wrong, was rotated, or the shared access signature has expired. The description typically mentions an invalid signature.amqp:unauthorized-accessmeans the token was accepted but the identity may not send to this entity. The description typically states that aSendclaim is required. In connection string mode the policy lacks Send, or the string is scoped throughEntityPath=to a different entity than the configuredqueueortopic. With a service principal or Managed Identity theAzure Service Bus Data Senderrole is missing.
Fix: for a connection string, open the namespace in the Azure Portal, then Endpoint=sb://. For a service principal or Managed Identity, open the namespace or the queue or topic, then Azure Service Bus Data Sender to the identity. Director retries on its own, so no restart is needed once the right is in place.
Data waits and is redelivered about every 5 seconds. The connection status stays green throughout.
"ClientSecretCredential authentication failed" with an AADSTS code
Sender worker 1 ThreadSafeInit on reinit failed for "basic_servicebus_queue": failed to create message batch: ClientSecretCredential authentication failed. POST https://login.microsoftonline.com/00000000-0000-0000-0000-000000000000/oauth2/v2.0/token ... RESPONSE 401: 401 Unauthorized ... AADSTS7000215: Invalid client secret provided. ...
Cause: the service principal credentials are wrong, so the token request fails before any Service Bus permission is checked. The code names the field:
| Code | Meaning | Check |
|---|---|---|
AADSTS7000215 | Invalid client secret | The secret ID was pasted instead of the secret value, or the secret was rotated |
AADSTS7000222 | Client secret expired | Create a new secret on the app registration and update client_secret |
AADSTS700016 | Application not found in the directory | client_id is wrong, or tenant_id is a different tenant than the one the app registration lives in |
AADSTS90002 | Tenant not found | tenant_id is not a valid tenant ID |
If the reason is failed to create credentials: invalid tenantID instead, the target failed at startup and the status does show connection failed. tenant_id then holds a character other than letters, digits, dots and hyphens, typically a stray quote or brace. Use the tenant GUID.
Data waits and is redelivered about every 5 seconds until the credentials are corrected.
"DefaultAzureCredential: failed to acquire a token" or "ManagedIdentityCredential authentication failed"
Cause: client_connection_string is empty and Director authenticates as a Managed Identity: through the default credential chain when tenant_id, client_id and client_secret are all empty, or as a user-assigned managed identity when client_id alone is set. The token request fails when no credential obtains a token. For the default chain the reason lists every credential it attempted, typically ending in a managed identity timeout when Director does not run on an Azure resource that has a managed identity. A reason containing the requested identity isn't assigned to this resource means the user-assigned identity named in client_id is not attached to the resource.
client_id alone is the only partial set of service principal fields that selects a managed identity. Any other mix, such as tenant_id or client_secret without the others, or two of the three, does not fall back to Managed Identity. That configuration is rejected at validation with tenant_id, client_id, and client_secret must all be provided together for service principal authentication; set client_id alone for a user-assigned managed identity, and nothing is sent until it is corrected. An empty ${VAR} behind one of the fields is a common trigger.
Fix: fill in all three service principal fields, switch to a connection string, or run Director on an Azure resource with a managed identity and assign Azure Service Bus Data Sender to that identity. For a user-assigned identity, attach it to the resource and set client_id to its client ID. On a pod with workload identity federation, give the identity a federated identity credential for the pod's service account instead of attaching it.
Data waits and is redelivered about every 5 seconds.
"failed to resolve connection string" or "failed to create service bus client from connection string"
Failed to reinitialize target "basic_servicebus_queue" (attempt 3). Reason: failed to resolve connection string: credential: env variable "SERVICEBUS_CONNECTION_STRING" is not set
Failed to reinitialize target "basic_servicebus_queue" (attempt 1). Reason: failed to create service bus client from connection string: key "SharedAccessKeyName" must not be empty
Cause: both fail at startup, so here the status does show connection failed. failed to resolve connection string means a ${VAR} or $secret{...} reference could not be read. The variable is not set in the environment of the Director service, or the secret store is misnamed, which reads credential: store "vault" not found in configuration. The same applies to failed to resolve tenant ID, failed to resolve client ID and failed to resolve client secret. failed to create service bus client from connection string means the string was read but is not complete: a missing prefix gives key "Endpoint" must not be empty, and a truncated paste gives failed parsing connection string due to unmatched key value separated by '='.
Fix: set the variable for the account the Director service runs under, or correct the secret store name, then paste the full string in the form Endpoint=sb://mynamespace.servicebus.windows.net/;SharedAccessKeyName=...;SharedAccessKey=.... If the variable exists but resolves to an empty value, the reason becomes namespace is required when not using connection string.
Startup is retried with a growing backoff and the attempt counter climbs until the cause is fixed. Data waits in the queue.
"no such host" while connecting
Cause: the namespace host cannot be resolved. The reason starts with (connlost): and typically ends in lookup mynamespace.servicebus.windows.net: no such host. The usual reasons are a typo in namespace, or DNS on the Director host. A bare name gets .servicebus.windows.net appended, and a value containing a dot is dialled exactly as given, so a mistyped suffix in a host name is not corrected.
Fix: compare namespace with the namespace's host name in the Azure Portal. Give either the bare name, mynamespace, or the full host name. A namespace in a sovereign cloud, on a .chinacloudapi.cn or .usgovcloudapi.net host, is set by its full host name, such as mynamespace.servicebus.chinacloudapi.cn.
Data waits and is redelivered about every 5 seconds.
"amqp:not-found" or "com.microsoft:entity-disabled" while creating the message batch
Sender worker 1 ThreadSafeInit on reinit failed for "basic_servicebus_queue": failed to create message batch: *Error{Condition: amqp:not-found, Description: ..., Info: map[]}
Cause: the namespace was reached and the credentials were accepted, but the queue or topic in the configuration does not exist there. The description typically names the entity that could not be found. The usual reasons are a misspelled name, a deleted entity, an entity that lives in another namespace, and a topic name placed in queue or the reverse. com.microsoft:entity-disabled means the entity exists but is not active.
Fix: compare queue or topic against the entity list of the namespace in the Azure Portal, and correct the name or create the entity. For a disabled entity set its
Data waits and is redelivered about every 5 seconds.
"the message is too large"
Sender worker 3 execute() failed for basic_servicebus_queue: record rejected by target: the message is too large
Target "basic_servicebus_queue" rejected record 7 of <payload> on its last delivery attempt — dropping that record and delivering the rest of the payload: record rejected by target: the message is too large
Cause: a single record exceeds the largest message the namespace accepts. See your tier's message size limit in the Azure documentation. Director fills a batch until it holds max_events messages or the link's size limit is reached. A record that does not fit into a batch already holding messages is normal: that batch is sent and the record opens the next one, with no error. Only a record that does not fit into an empty batch is rejected. A few bytes of framing per message count toward the limit.
Fix: shrink the record in a pipeline before it reaches the target, or move the namespace to a tier with a larger message size. Lowering max_events does not help, because the limit applies to one record. Note that max_events: 0 is read as unset and becomes 1000.
The rejected record is dropped alone on its fourth delivery, as the second line shows, and counted on the target's dropped counter. The other records in the payload continue to be delivered.
"failed to send message batch: context deadline exceeded" or "com.microsoft:server-busy"
Sender worker 4 execute() failed for basic_servicebus_queue: target broken: failed to finalize target cache: failed to send message batch: context deadline exceeded
Cause: the batch was built but the namespace did not acknowledge it within timeout seconds, 30 by default. The usual reasons are a congested or half-open connection, or a Standard tier namespace that is throttling. Throttling also surfaces directly as *Error{Condition: com.microsoft:server-busy, Description: ..., Info: map[]}, whose description typically says the entity is being throttled. amqp:resource-limit-exceeded in the same position means the queue or topic is full because nothing is consuming it.
Fix: raise timeout when sends are slow but succeed. For throttling, check the namespace's throttled requests metric, lower the send rate, spread the load across entities, or move to Premium. For a full entity, drain or purge it, or raise its maximum size.
The batch is redelivered about every 5 seconds until it is accepted. Nothing is dropped.
"amqp:not-allowed" when sending to a session-enabled entity
Cause: the queue or subscription requires sessions but session_id is not configured, so the service refuses the batch. The description typically states that the SessionId was not set. The reverse case is harmless: a session_id sent to an entity without sessions is ignored, though the ordering it implies is then not guaranteed.
Fix: set session_id to a fixed string, as in the Session-Enabled Queue example, or turn sessions off on the entity.
The verdict is deterministic, but the batch is not dropped. It is redelivered about every 5 seconds and the target stalls until the configuration or the entity changes.
"i/o timeout" or "tls: failed to verify certificate" while connecting
Cause: Director cannot open the AMQP connection. With the default transport: amqp, this target speaks AMQP over TLS on TCP port 5671, and the connection is dialled directly: HTTPS_PROXY applies to the Microsoft Entra token request, not to the namespace connection. A (connlost): reason that typically ends in i/o timeout means port 5671 is blocked, or the namespace firewall excludes the Director host. On a network that only allows HTTPS egress, or only through a proxy, every send fails this way while the status stays green. A reason that typically contains x509: certificate signed by unknown authority means a TLS-intercepting proxy, or missing root certificates on the host.
Fix: allow outbound TCP 5671 from the Director host to the namespace host, or set transport: amqp_websockets to connect with AMQP over WebSockets on TCP port 443, whose dial honours HTTPS_PROXY. Add the host's egress IP address or virtual network to the namespace's network rules. The host is <namespace>.servicebus.windows.net when namespace is a bare name, the namespace value itself when it is a host name, and the host of Endpoint=sb://... when a connection string is used. For TLS interception, exempt that same host or install the intercepting CA in the operating system trust store. This target has no custom CA option.
An isolated (connlost): failure during a send, such as a forced link detach after a quiet period, is redelivered on the next attempt and usually heals by itself. Otherwise data waits and is redelivered about every 5 seconds.
"ValidateConfig failed" for this target
| Reason | Fix |
|---|---|
either queue or topic must be specified | Set queue or topic |
cannot specify both queue and topic | Remove one of them |
tenant_id, client_id, and client_secret must all be provided together for service principal authentication; set client_id alone for a user-assigned managed identity | Fill in all three, set client_id alone for a user-assigned managed identity, or clear all three to use the default credential chain |
namespace is required when not using connection string authentication | Set namespace when no connection string is configured |
transport must be "amqp" or "amqp_websockets", got "<value>" | Set transport to amqp or amqp_websockets, or remove it |
max_events must be greater than 0, got -5 | Use a positive integer |
Nothing is sent while the configuration is rejected. The check repeats about every 30 seconds until the configuration changes.
The target is healthy but nothing arrives in the queue or topic
Check these in order.
-
The connection was never tested. A connected status does not mean a send succeeded. Search the log for
ThreadSafeInit on reinit failedand follow the first entry above. -
debug.dont_send_logsis enabled. Events are processed but never buffered or transmitted, and the sent counter does not move. The only notice is a debug-level line at startup, and only whendebug.statusis enabled as well. Remove the flag after testing. -
A
message_idis saved with the target. It no longer causes this. Earlier releases stamped that one ID on every message, so on an entity with duplicate detection enabled Service Bus kept the first message of each detection window and discarded the rest, while Director counted every record as sent. The target now ignores the key, and Service Bus assigns each message its own ID. A target that still carries it starts normally and logs one Warning:message_id is ignored for target <target name>: one fixed ID on every message makes a queue or topic with duplicate detection drop every message after the first, so Service Bus assigns each message its own ID.Remove the key to stop the Warning. The console no longer offers a
Message ID field, so saving the target from the console removes it. -
time_to_liveis too short. Messages are accepted, then expire before a consumer reads them, and are dead-lettered or discarded according to the entity settings. A value above the entity's own default is adjusted down without notice. Raise or removetime_to_live, and check the dead-letter queue. -
The topic has no subscription. Messages published to a topic without subscriptions are accepted and discarded by the service. Create the subscription before pointing the target at the topic.