Vault
Synopsis
Manages credential stores that securely resolve secrets at runtime, so sensitive values never appear in configuration files. DataStream supports five providers: VirtualMetric Vault (built-in encrypted store), Azure Key Vault, CyberArk Central Credential Provider, HashiCorp Vault, and AWS Secrets Manager.
The four external providers — Azure Key Vault, CyberArk, HashiCorp Vault, and AWS Secrets Manager — are gated. Below Enterprise their cards are locked when you choose a provider, and only the built-in VirtualMetric Vault can be selected.
The Vault is administered through two surfaces:
- The Vault page in the web interface — covered in the sections below. Secrets created here are synchronized to each Director and Agent and become resolvable in pipeline, device, and target configurations through
$secret{id=<numeric-id>}tokens. - The YAML configuration reference — for infrastructure-as-code deployments (see YAML Configuration Reference).
Any string-valued field in a device, target, or other component configuration can reference a secret using a $secret{...} token; DataStream resolves the token at runtime.
VirtualMetric Vault encrypts secret values at rest using a 32-byte AES-256 key loaded from the credentialstore.key platform configuration value. The service does not start when this key is missing. Set the key once per deployment before enabling the Vault feature.
Accessing the Vault
To open the Vault management interface:
- Click the hamburger menu in the top left corner.
- Select
Content Management >Vault .
The page opens with two tabs:
Vault requires Director 1.10 or later. Both tabs show an alert when any connected Director or cluster is below that version — secrets resolve normally on the Directors that are current, and not at all on the ones that are behind.
Secrets
A secret is a single credential entry — a password, API key, certificate, or other sensitive value — that devices and targets reference through a $secret{...} token. Each secret is stored by one of the supported providers. Secrets backed by VirtualMetric Vault store the value directly, encrypted at rest using AES-256-GCM with a service-level key. Secrets backed by any other provider store only a reference that DataStream resolves against the external provider at runtime.
Secrets Table
The central table displays essential information for each secret:
- Friendly Name - Human-readable identifier. Click to open the secret detail drawer.
- Provider - VirtualMetric Vault, CyberArk, Azure Key Vault, HashiCorp Vault, or AWS Secrets Manager.
- Configuration - The provider configuration the secret uses. Blank for VirtualMetric Vault secrets.
- Value Format - Single-line or Multi-line.
- Assigned Resources - Count of devices, targets, and other components currently referencing this secret.
Controls
Above the table, the
At the bottom of the table, pagination controls let you adjust
Actions Menu
On the right side of each row, a vertical ellipsis (⋮) opens the
Manage secret - Opens the secret's detail page.Clone secret - Duplicates the secret. The new friendly name is prefixed withclone-.Delete secret - Prompts for confirmation; see Edit, Clone, and Delete Secrets.
Create a Secret
To add a new secret:
- On the
Secrets tab, clickAdd new secret . - Complete the three-step wizard described below.
- Click
Complete setup on the review step.
General Settings
Friendly Name - Unique, human-readable identifier. Required.Description - Optional free-text note.Provider - Select one of: VirtualMetric Vault, CyberArk, Azure Key Vault, HashiCorp Vault, AWS Secrets Manager. Required.Configuration - Required when the provider is not VirtualMetric Vault. Select one of the provider configurations defined on the Provider Configurations tab.
If you select a provider other than VirtualMetric Vault and no provider configurations exist for that provider, an alert appears and the
Secret Configuration
Value Format - Single-line (compact input, for short credentials like tokens or passwords) or Multi-line (expanded text area, for PEM blocks, JSON payloads, and other long values).Secret Value (VirtualMetric Vault) - The credential value itself. Stored encrypted.Reference (CyberArk, Azure Key Vault, HashiCorp Vault, AWS Secrets Manager) - The provider-specific lookup string that identifies the value inside the external vault. The helper text adjusts to the selected provider; for AWS Secrets Manager it reads: "The secret's name or ARN. Add #key to read one field of a JSON secret, e.g. prod/amq#password." See YAML Configuration Reference for the exact reference grammar of each provider.
Review and Complete Setup
Review the summary of your entries from the preceding steps, then click
VirtualMetric Vault stores the secret value itself, encrypted at rest with AES-256-GCM. The encryption key is loaded from the credentialstore.key platform configuration value. All other providers store only the reference; the actual value remains in the external vault and is fetched each time DataStream resolves the token.
Secret Details
Click a secret's Friendly Name to open the detail drawer on the right side of the page. The drawer summarises the secret's friendly name, description, provider, configuration, and current assigned-resource count. Click
The detail page has three tabs:
- General Settings - Friendly Name, Description, and Provider. Click
Manage secret details to modify the Friendly Name and Description. - Secret Configuration - Value Format and the stored value or reference. Click
Manage secret configuration to modify. - Activity Logs - Audit trail of all actions performed on this secret.
Provider and Configuration cannot be changed after a secret is created. To move a secret to a different provider or configuration, delete the secret and create a new one. This constraint is enforced in both the UI and the backend API.
Edit, Clone, and Delete Secrets
Edit - Open the detail page (click the Friendly Name, then
Clone - In the Secrets table, open the row action menu and select clone- and must be changed to a unique value before the new secret can be saved.
Delete - From the table row action menu or the detail page actions, select
If a secret is currently assigned to any device, target, or other resource, deletion is blocked. DataStream displays an error modal listing each resource that still references the secret. Remove the secret from every listed resource, then retry the delete operation.
Provider Configurations
A provider configuration is a reusable connection definition for an external vault — the URL, credentials, and options DataStream uses to talk to Azure Key Vault, CyberArk, HashiCorp Vault, or AWS Secrets Manager. One configuration can back many secrets. VirtualMetric Vault does not appear here because it is built-in and has no external connection to configure.
Provider Configurations Table
The central table displays essential information for each configuration:
- Configuration Name - Unique identifier. Click to open the configuration detail drawer.
- Provider - CyberArk, Azure Key Vault, HashiCorp Vault, or AWS Secrets Manager.
- Description - Free text, or
-when empty. - Assigned Secrets - Count of secrets that reference this configuration.
Controls
Above the table, the
Pagination matches the Secrets tab: 10, 25, or 50 items per page (default 25).
The
Actions Menu
On the right side of each row, the
Manage configuration - Opens the configuration's detail page.Delete configuration - Prompts for confirmation; see Edit and Delete Provider Configurations.
Unlike the Secrets tab, this tab has no Clone action.
Create a Provider Configuration
To add a new provider configuration:
- On the
Provider Configurations tab, clickAdd new provider configuration . - Complete the three-step wizard described below.
- Click
Complete setup on the review step.
General Settings
Configuration Name - Unique identifier. Required.Description - Optional free text.Provider - CyberArk, Azure Key Vault, HashiCorp Vault, or AWS Secrets Manager. Required. VirtualMetric Vault is not available here.
Provider Configuration
The fields shown depend on the provider selected under General Settings. For the exact field semantics, defaults, and reference-format rules, see YAML Configuration Reference.
-
CyberArk - URL, Application ID, default Safe, default Folder, timeout, optional mutual TLS (enable toggle plus certificate and key file names), and optional CA certificate file name.
-
Azure Key Vault - Vault URL, a
Managed Identity toggle, and — when it is off — Tenant ID, Client ID and Client Secret. Turning the toggle on is what selects managed-identity authentication in the interface; it clears the three credential fields. In YAML there is no toggle: omitting all three fields is what selects managed identity. -
HashiCorp Vault - URL, Token, Mount path (default
secret), KV version (1or2, default2), Namespace (Enterprise only), timeout, optional mutual TLS, and optional CA certificate file name. -
AWS Secrets Manager - Three sections:
- Connection - Region (required), Connection Timeout (default 30 seconds), and Cache TTL (default 300 seconds).
- Authentication - A
Credential source choice, then the optional Role ARN (the IAM role to assume through STS) and External ID (sent with the role ARN when the role's trust policy requires one). Default chain (instance profile, IRSA) takes no credentials, and shows a warning that it resolves on Directors only, because rule actions read secrets on the platform and need an access key — see the platform-reads note under AWS Secrets Manager. Access key shows Access key ID, Secret access key, and Session token (optional, for temporary credentials). - Advanced - Endpoint, for a VPC endpoint. Must be an
httpsURL; leave it empty to use the region's public endpoint.
The credential source is a choice in the interface only and is not saved. Default chain writes no
keyorsecret; Access key writeskey,secret, andsession. A saved store with akeyorsecretreopens with Access key selected.
TLS certificate and CA material given as a file name must exist in the service root directory before the configuration can authenticate successfully.
Review and Complete Setup
Review the summary of your entries from the preceding steps, then click
Provider Configuration Details
Click a configuration's Configuration Name to open the detail drawer, or use the
The detail page has three tabs:
- General Settings - Configuration Name, Description, and Provider. Click
Manage configuration details to modify the name and description. - Provider Configuration - The provider-specific fields entered during creation. Click
Manage provider configuration to modify. - Activity Logs - Audit trail of all actions performed on this configuration.
Provider cannot be changed after a configuration is created. To migrate to a different provider, create a new configuration, reassign dependent secrets, and delete the old configuration. This constraint is enforced in both the UI and the backend API.
Edit and Delete Provider Configurations
Edit - Open the detail page and click
Delete - From the table row action menu or the detail page actions, select
If any secret currently references the configuration, deletion is blocked. DataStream displays an error modal listing each secret that must be deleted or reassigned before the configuration can be removed.
Using Vault Secrets in Device and Target Configurations
Once a secret is created, any string-valued configuration field in a device, target, or other component can reference it using a $secret{id=<numeric-id>} token. Because the token resolves at runtime, the underlying credential can be rotated in the Vault page without touching any device or target definition.
A value fetched from an external provider is held for that store's cache_ttl — five minutes by default. Rotating the secret in the backing vault without changing the store definition therefore takes effect only once the cached value expires. Set the store's cache_ttl to 0 where a rotation has to take effect immediately.
The following field categories most commonly consume vault secrets:
| Category | Example Fields | Where Used |
|---|---|---|
| Username / Password | username, password | SSH devices, Windows agentless listeners, database targets, HTTP basic-auth targets |
| Client ID / Client Secret | client_id, client_secret | Azure targets, OAuth-protected HTTP targets, Entra ID-authenticated devices |
| API Tokens and Keys | token, api_key, bearer_token | HTTP targets, SaaS connectors, threat-intel feeds |
| SSH Key Material | private_key, passphrase | Linux agentless devices, SFTP targets |
| TLS Material | certificate, key, and CA contents | Any device or target using mutual TLS when certificates are stored as values |
Token grammar and the mapping from tokens to credential-store references is documented in Secret Token Syntax below. Per-field usage is documented on each component's own reference page — this section is a map of where secrets apply, not a per-field catalog.
Prefer the $secret{id=<id>} form over $secret{store=...,ref=...} for production configurations. Registry-based IDs let you change the underlying store or reference without editing every component that consumes the secret.
YAML Configuration Reference
The sections below describe how to declare credential stores and secrets directly in YAML. YAML-driven management is the source-of-truth format for the configuration files that DataStream loads at startup; the GUI surfaces the same concepts for interactive administration.
Secret Token Syntax
Any configuration field that accepts a string value can use a secret token instead of a plain-text value. Three token formats are supported:
| Format | Description |
|---|---|
$secret{store=<name>,ref=<value>} | Resolve a secret from a named credential store using the given reference |
$secret{id=<id>} | Resolve a secret by its ID from the secrets registry |
$env{VAR_NAME} | Read a value from an environment variable |
When DataStream encounters a token, it resolves the value before using it. Plain-text values are passed through unchanged.
Direct store reference — specifies the store name and reference explicitly:
$secret{store=azure-prod,ref=linux-root-password}
ID-based lookup — resolves the store name and reference from the secrets registry:
$secret{id=2142151252151}
The secrets registry entry must contain credential_store and credential_ref properties that map to a configured credential store.
Schema
Credential stores are declared under the credentials: section. The optional secrets: section defines a registry of secret entries that map stable IDs to store/ref pairs for use with $secret{id=...} tokens.
credentials:
- name: <string>
type: local | azurevault | cyberark | hashicorpvault | awssecretsmanager
description: <string>
status: <boolean>
properties:
# provider-specific fields — see sections below
secrets:
- name: <string>
id: <numeric>
description: <string>
properties:
credential_store: <string>
credential_ref: <string>
Credentials
Credential Store
Common fields shared by all provider types:
| Field | Required | Default | Description |
|---|---|---|---|
name | Y | Unique credential store identifier | |
type | Y | Provider type: local, azurevault, cyberark, hashicorpvault, or awssecretsmanager | |
description | N | - | Optional description |
cache_ttl | N | 300 | How long a fetched secret is held in memory, in seconds. 0 disables caching for this store |
VirtualMetric Vault
Stores credentials directly in the configuration file as key-value pairs under properties. The ref value in a secret token maps to a property key name.
Type: local
| Field | Required | Default | Description |
|---|---|---|---|
properties.<key> | N | - | Arbitrary key-value pairs storing secret values directly |
Ref format:
| Ref | Resolves to |
|---|---|
<property_key> | Value of the matching property |
Azure Key Vault
Fetches secrets from Azure Key Vault using the REST API. Authenticates with client credentials when tenant_id, client_id, and client_secret are provided. Falls back to managed identity when these fields are omitted.
Type: azurevault
| Field | Required | Default | Description |
|---|---|---|---|
url | Y | Azure Key Vault URL (must use https) | |
tenant_id | N* | - | Azure AD tenant ID |
client_id | N* | - | Azure AD application (client) ID |
client_secret | N* | - | Azure AD client secret |
timeout | N | 30 | Request timeout in seconds |
* = Optional. When omitted, authentication falls back to managed identity.
Ref format:
| Ref | Resolves to |
|---|---|
<secret_name> | Value of the named secret in Key Vault |
CyberArk
Fetches credentials from CyberArk Central Credential Provider (CCP) using the REST API. Authenticates with an Application ID and supports optional mutual TLS for client certificate authentication. A custom CA certificate can be configured independently of mutual TLS for server certificate verification.
Type: cyberark
| Field | Required | Default | Description |
|---|---|---|---|
url | Y | CyberArk PVWA base URL (must use https) | |
app_id | Y | Application ID registered in CyberArk | |
safe | N | - | Default safe name, used when ref omits it |
folder | N | - | Default folder name, used when ref omits it |
timeout | N | 30 | Request timeout in seconds |
tls.status | N | false | Enable mutual TLS client certificate authentication |
tls.cert_name | N* | "cert.pem" | Client certificate file name |
tls.key_name | N* | "key.pem" | Client private key file name |
ca_name | N | - | Custom CA certificate file name for server verification |
* = Required when tls.status is true.
Ref format:
| Ref | Safe | Folder | Object |
|---|---|---|---|
Object | From store config | From store config | Object |
Safe/Object | Safe | From store config | Object |
Safe/Folder/Object | Safe | Folder | Object |
HashiCorp Vault
Fetches secrets from HashiCorp Vault using the KV secrets engine (v1 or v2). Authenticates with a static Vault token. Supports Vault Enterprise namespaces.
Type: hashicorpvault
| Field | Required | Default | Description |
|---|---|---|---|
url | Y | HashiCorp Vault server URL | |
token | Y | Vault authentication token | |
mount | N | "secret" | KV secrets engine mount path |
kv_version | N | 2 | KV engine version: 1 or 2 |
namespace | N | - | Vault Enterprise namespace |
timeout | N | 30 | Request timeout in seconds |
tls.status | N | false | Enable mutual TLS client certificate authentication |
tls.cert_name | N* | "cert.pem" | Client certificate file name |
tls.key_name | N* | "key.pem" | Client private key file name |
ca_name | N | - | Custom CA certificate file name for server verification |
* = Required when tls.status is true.
Ref format:
| Ref | Behavior |
|---|---|
path/to/secret | Returns the value when the secret holds exactly one field. A secret with more than one field is an error asking you to name the field with #field |
path/to/secret#field | Returns the value of the specified field |
When using KV v2 (default), the API path automatically includes /data/ between the mount path and the secret path. No manual adjustment is needed.
AWS Secrets Manager
Fetches a secret's current value (the AWSCURRENT version stage) from AWS Secrets Manager. The store only reads: it never writes or rotates secrets, and it does not read Systems Manager Parameter Store.
Type: awssecretsmanager
| Field | Required | Default | Description |
|---|---|---|---|
region | Y | AWS region of the secret, e.g. eu-west-1 | |
key | N* | - | Access key ID. Accepts plain text, ${ENV} or $secret{...} |
secret | N* | - | Secret access key. Same forms as key |
session | N | - | Session token, for temporary credentials. Used only with key and secret |
role_arn | N | - | IAM role assumed through STS before reading |
external_id | N | - | External ID sent when assuming role_arn |
endpoint | N | - | Secrets Manager endpoint override, e.g. a VPC interface endpoint. Must use https. Applies to Secrets Manager only; STS keeps its regional endpoint |
timeout | N | 30 | Request timeout in seconds, for the whole lookup including STS |
* = Optional. See the credential resolution order below.
Credential resolution, in precedence order:
| Config | Credentials used (Director) |
|---|---|
key and secret set | That static key, plus session if set |
| Either one missing | The AWS default chain: environment, shared config, IRSA / web identity, then the instance profile |
role_arn set | The identity above assumes role_arn (with external_id), and the secret is read as that role |
The platform's own reads of a store — for example, secrets used by rule actions — never use the AWS default chain. They read only with the store's own key and secret, optionally assuming role_arn with them. A store without key and secret works on Directors (instance profile, IRSA) but is refused with a configuration error for platform-side reads, because the platform's ambient AWS identity belongs to the platform, not to the tenant.
Ref format:
| Ref | Resolves to |
|---|---|
<secret-name> or <full secret ARN> | The whole SecretString |
<name or ARN>#<json-key> | The SecretString parsed as a JSON object, and that key's value |
- A
#json-keyvalue that is a string is returned as-is. A number or a boolean is returned as its JSON text, e.g.5432ortrue. An object, an array ornullis refused. - A missing key is an error that names the key, never the value. The same applies when
#json-keyis given and theSecretStringis not a JSON object. - A secret that holds only
SecretBinaryis refused: binary secrets are not supported. - An empty value is an error.
Failures: a secret that is not found, access denied on secretsmanager:GetSecretValue, a KMS decrypt failure, and STS refusing role_arn (trust policy or external ID) are permanent — they fail again until fixed. Throttling and network errors are transient. No error or log line carries the secret's value.
Required IAM permissions, for the identity the secret is read as:
secretsmanager:GetSecretValueon the secret. Usearn:aws:secretsmanager:REGION:ACCOUNT_ID:secret:NAME-*as the resource, because AWS appends six random characters to a secret's ARN.kms:Decrypton the key, only when the secret is encrypted with a customer-managed KMS key. The defaultaws/secretsmanagerkey needs no grant.sts:AssumeRoleonrole_arn, only when it is set.
Secrets Registry
Defines named secret entries under the top-level secrets: section. Each entry maps a stable ID (the entry name) to a target credential store and reference, enabling indirection so that components can use $secret{id=<name>} tokens without hard-coding the store and ref.
| Field | Required | Default | Description |
|---|---|---|---|
name | Y | Human-readable secret entry identifier | |
id | Y | Numeric identifier used in $secret{id=<value>} tokens | |
description | N | - | Optional description |
properties.credential_store | Y | Name of a credential store defined in the credentials: section | |
properties.credential_ref | Y | Reference passed to the credential store to fetch the value |
Resolution Details
Resolution Flow
When DataStream encounters a $secret{...} token in a configuration field, it follows this sequence:
- Parse the token to extract either
store/refparameters or anidparameter - If
idis provided, look up thecredential_storeandcredential_reffrom the secrets registry - Find the named credential store in the
credentialsconfiguration - Look up the registered provider for the store's
type - Invoke the provider with the
refvalue to retrieve the secret
Supported Fields
Any string-valued configuration field can use secret tokens. Common use cases include authentication fields in devices, targets, and other components:
usernameandpassword— login credentialsprivate_keyandpassphrase— SSH key-based authenticationclient_secret— OAuth and API credentialstoken— API tokens and bearer credentials
TLS Certificate Location
When tls.cert_name, tls.key_name, or ca_name is given as a file name, the file must be placed in the service root directory. These fields also accept an in-root absolute path or inline PEM content.
URL Requirements
Azure Key Vault and CyberArk providers require https URLs. Connections using http are rejected. An AWS Secrets Manager endpoint override must also use https.
Examples
VirtualMetric Vault
Storing SSH credentials locally in the configuration... | |
Referencing a value from this store... | |
Azure Key Vault
Connecting with client credentials... | |
Using managed identity (omit tenant_id, client_id, client_secret)... | |
CyberArk
Basic configuration with default safe and folder... | |
With mutual TLS and custom CA... | |
HashiCorp Vault
KV v2 engine (default)... | |
KV v1 engine with Enterprise namespace... | |
AWS Secrets Manager
Assuming a role with the Director's default credential chain... | |
Reading the password key of a JSON secret... | |
Secrets Registry
Defining a registry of named secret entries that map to credential stores... | |
Using Secret Tokens in Configuration
Direct reference — specifies the store and ref inline... | |
ID-based lookup — references a registered secret entry by its numeric id... | |
Mixing secret tokens with environment variables... | |