Skip to main content

Reference

This reference covers the configuration files and fields available for self-managed Director deployments. Managed Directors receive their configuration from the DataStream platform automatically; the fields documented here are relevant when you manage Director infrastructure directly.

For Director concepts and GUI-based setup, see Overview. For installation procedures, see Deployment.

Configuration Files​

A Director reads two categories of configuration at startup:

FileLocationPurpose
Service configurationvmetric.yml in the Director working directoryCore service settings (identity, network, queue, processing defaults)
Environment configurationconfig/environments/*.ymlTopology, clustering, ACL, and TLS

Supported formats: YAML (.yml / .yaml), TOML (.toml), and VMF (.vmf). Maximum file size: 1 MB.

Config file lookup order (first match wins):

  1. vmetric.toml
  2. vmetric.yml
  3. vmetric.yaml
Debug Mode

When the virtualmetric=debug environment variable is set, the service loads vmetric-local.yml (or .toml / .yaml) instead of the standard file. This is intended for local development and troubleshooting only -- do not use in production.

Service Configuration (vmetric.yml)​

Director Identity​

FieldYAML PathTypeDefaultDescription
IDdirector.idint64--Unique Director instance identifier
Statusdirector.statusbooltrueEnable or disable the Director service
Streamdirector.streamstring--NATS stream name for Director replies

Network​

FieldYAML PathTypeDefaultDescription
Addresslisten.addressstring0.0.0.0Bind address for all listeners
External Addresslisten.external_addressstring--Advertised address for remote connections (comma-separated for multiple)
Portlisten.portint8890Main collector listening port
WebSocket Portlisten.ws_portint9080VMMQ WebSocket port
HTTP Config Portlisten.httpconfigportint80 / 443HTTP configuration server port

Queue​

FieldYAML PathTypeDefaultDescription
Limitqueue.limitint172800 (48 h)Queue size limit in seconds
Parallelismqueue.parallelismintCPU countQueue processing parallelism
Non-Existing Target Limitqueue.non_existing_target_limitint86400 (24 h)TTL for queued data to non-existing targets (seconds)
Cleanup Intervalqueue.cleanup_intervalint900 (15 min)Cleanup interval for non-existing target queues (seconds)
DeQueue Intervalqueue.dequeue_intervalint1DeQueue processing interval in seconds

Debug and Logging​

FieldYAML PathTypeDefaultDescription
Leveldebug.levelint0Debug verbosity: 0 = off, 1--3 = increasing detail
Console Statusdebug.console.statusboolfalseEnable console debug output
Log Statusdebug.log.statusboolfalseEnable file-based debug logging
Remotedebug.remoteboolfalseEnable remote debug mode (disables device tasks when true)

Fleet​

Fleet settings control how a Director connects to the DataStream cloud platform. Self-managed Directors that operate independently (no fleet connection) set self_managed: true and omit the fleet block.

FieldYAML PathTypeDefaultDescription
Self-Managedself_managedbool--If true, Director runs independently without fleet connection
Fleet Typefleet.typestring--Connection type (e.g., httpclient)
Fleet Instancefleet.instancestring--Fleet environment instance name
Fleet Addressfleet.addressstring--Fleet server WebSocket URL
Fleet Tokenfleet.tokenstring--Authentication token for fleet

Data Processing Defaults​

These defaults apply to all data processing unless overridden at the device or target level.

FieldYAML PathTypeDefaultDescription
CompressioncompressionbooltrueEnable data compression
Compression Levelcompression_levelstring--Compression level (algorithm-specific)
ReusereusebooltrueEnable connection reuse
NormalizationnormalizationbooltrueEnable data normalization
Smart Data Enginesmart_data_enginebooltrueEnable SDxL processing
No Bufferno_bufferboolfalseDisable buffering
Batch Sizebatch_sizeint6410000Read batch size
Write Batch Sizewrite_batch_sizeint641000Write batch size
WorkersworkersintCPU countWorker count
Chunk Sizechunk_sizeint4194304 (4 MB)Data chunk size in bytes
Min Chunk Countmin_chunk_countint10Minimum chunk count

Collector Resource Management​

Controls resource allocation and scheduling for device collectors.

FieldYAML PathTypeDefaultDescription
Retry Intervalretry_intervalint30Device retry delay in seconds
Start Delaystart_delayint1Collector startup delay in seconds
Windows Startwindows_startint524288000 (500 MB)Memory threshold for Windows collectors (bytes)
Windows Start Bufferwindows_start_bufferbool--Buffer mode for Windows collectors
Monitor Intervalmonitor_intervalint30Monitor interval in seconds
Flush Intervalflush_intervalint1Log flush interval in seconds
ModulesmodulesbooltrueEnable or disable module loading

Profiler​

FieldYAML PathTypeDefaultDescription
Profile Portprofiler.director_profile_portint6061pprof profiler port
Timeoutprofiler.timeoutint60Profiler and metrics collection timeout in seconds

Load Balancer​

FieldYAML PathTypeDefaultDescription
Modeloadbalancer.modestring--Set to dynamic for dynamic load balancing across cluster nodes

Environment Configuration​

Environment files define Director topology, clustering, and access control. Located in config/environments/. Files containing .example. or .sample. in their name are templates only and are not loaded by the system.

Standalone Director​

Minimal configuration for a single Director instance. The node name must match the director.id from vmetric.yml.

environments:
  - name: "1"
    status: true
    nodes:
      - name: "1"
        status: true

Clustered Director​

Clustered configurations define a cluster with multiple nodes for high availability. See Clusters for operational details.

cluster: "{cluster-id}"
environments:
  - name: "{cluster-id}"
    description: "Production Cluster"
    status: true
    clusters:
      - name: "{cluster-id}"
        description: "Primary Cluster"
        status: true
        type: director
        properties:
          address: "10.0.0.1"
          port: 6222
          authentication:
            username: ""
            password: ""
            token: ""
          tls:
            status: false
            cert_name: "cert.pem"
            key_name: "key.pem"
            ca_name: ""
            verify: false
        nodes:
          - name: "{node-1-id}"
            status: true
            properties:
              address: "10.0.0.1"
              port: 4222
              cluster:
                address: "10.0.0.1"
                port: 6222
                authentication:
                  token: ""
                tls:
                  status: false

ACL​

ACL rules can be applied at the cluster or node level within environment files.

FieldYAML PathTypeDefaultDescription
Allowed IPsacl.allowed_ips[]string--IP allowlist (individual IPs, ranges "ip1 - ip2", or CIDR notation)
Access Tokensacl.allowed_tokens[]string--Valid access tokens for API authentication
Rate Limit Maxacl.rate_limit_max_requestsint100Maximum requests per rate limit window
Rate Limit Windowacl.rate_limit_windowint60Rate limit window in seconds
Update Intervalacl.update_intervalint20ACL configuration refresh interval in seconds

TLS​

TLS settings for inter-node and cluster communication.

FieldYAML PathTypeDefaultDescription
Statustls.statusboolfalseEnable TLS
Cert Nametls.cert_namestringcert.pemTLS certificate filename
Key Nametls.key_namestringkey.pemTLS key filename
CA Nametls.ca_namestring--CA certificate filename
Verifytls.verifyboolfalseEnable certificate verification

Node Properties​

FieldYAML PathTypeDefaultDescription
Addressproperties.addressstring--Node bind address
Portproperties.portint4222Node NATS port
Cluster Addressproperties.cluster.addressstring--Cluster communication address
Cluster Portproperties.cluster.portint6222Cluster communication port

Health Check Configuration​

Health checks monitor device collector resources. A resource must exceed its threshold for the configured number of consecutive iterations before action is taken.

Check Toggles​

FieldYAML PathTypeDefaultDescription
Check CPU Usagehealth.check_cpu_usageboolfalseMonitor CPU usage
Check Memory Usagehealth.check_memory_usageboolfalseMonitor memory usage
Check Log File Usagehealth.check_log_file_usagebooltrueMonitor log file sizes
Check TMP Usagehealth.check_tmp_usageboolfalseMonitor temp directory
Check Report Usagehealth.check_report_usagebooltrueMonitor report files
Check Simulator Usagehealth.check_simulator_usagebooltrueMonitor simulator
Check Hanged Processeshealth.check_hanged_processesbooltrueDetect hung processes
Check Stats Usagehealth.check_stats_usagebooltrueMonitor stats collection

Thresholds​

FieldDefaultDescription
CPU Limit80%CPU usage threshold percentage
Memory Limit1024 MBMemory usage threshold
Max Log Size10 MBMaximum log file size
Stats Retention24 hoursStats retention period

Threshold Iterations​

All threshold iteration defaults are 10. A resource must exceed its threshold for this many consecutive check iterations before the health system takes action.

VIP (Virtual IP) Configuration​

VIP enables high-availability failover for clustered Directors. Configured within cluster properties in the environment file.

FieldYAML PathTypeDefaultDescription
Use VIPvip.use_vipboolfalseEnable VIP management
Bind VIPvip.bind_vipboolfalseBind VIP on startup
VIP Addressvip.vip_addressstring--Virtual IP address
VIP Netmaskvip.vip_netmaskstring--VIP subnet mask
VIP Gatewayvip.vip_gatewaystring--VIP gateway address
VIP Interfacevip.vip_interfacestring--Network interface for VIP
VIP Monitor Intervalvip.vip_monitor_intervalint--Health check interval in seconds
Is NATS Clustervip.is_nats_clusterboolfalseWhether this is a NATS cluster (affects VIP binding logic)

Environment Variables​

VariableValueEffect
virtualmetricdebugLoads vmetric-local.yml instead of vmetric.yml

Debug config file lookup order:

  1. {serviceName}-local.toml
  2. {serviceName}-local.yml
  3. {serviceName}-local.yaml

Internal Timing Constants​

These values are hardcoded and not user-configurable. They are documented here for operational awareness.

Manager Intervals​

ConstantValuePurpose
Heartbeat Update Interval5 sDevice heartbeat publishing frequency
Leader Check Interval1 sLeadership election check frequency
KV Sync Interval5 sJetStream KV synchronization
Config Sync Interval1 sConfiguration change polling
Heartbeat Check Interval15 sDevice health check and node offline threshold

Cluster Update Intervals​

ConstantValuePurpose
Cluster Update Check Interval10 sPoll for pending updates
Node Timeout30 minMaximum wait for a single node update
Health Check Wait30 sPost-update health check delay
Leader Stability30 sRequired leader stability before proceeding
Command Retry Interval30 sBetween retry attempts
Max Retries3Maximum update retries per node
Max Offline Retries5Maximum retries for offline nodes

Configuration Limits​

ConstantValuePurpose
Config File Max Size1 MBMaximum configuration file size
Retention Interval~3 monthsDefault data retention period

Examples​

Minimal Standalone Director​

vmetric.yml:

director:
  id: 1
listen:
  address: "0.0.0.0"
  port: 8080
debug:
  level: 1
  console:
    status: true

config/environments/env.yml:

environments:
  - name: "1"
    status: true
    nodes:
      - name: "1"
        status: true

Production Standalone with ACL​

vmetric.yml:

director:
  id: 1
listen:
  address: "0.0.0.0"
  external_address: "10.0.1.100"
  port: 8890
queue:
  limit: 900000000
debug:
  level: 1
  console:
    status: false
  log:
    status: true

config/environments/env.yml:

environments:
  - name: "1"
    status: true
    nodes:
      - name: "1"
        status: true
        properties:
          acl:
            allowed_ips:
              - "10.0.0.0/24"
              - "192.168.1.50"
              - "172.16.0.1 - 172.16.0.254"
            access_tokens:
              - "prod-token-abc123"
            rate_limit_max_requests: 200
            rate_limit_window: 60

Three-Node HA Cluster with VIP​

vmetric.yml (same on all nodes, only director.id differs):

director:
  id: 1  # Change to 2, 3 on other nodes
listen:
  address: "0.0.0.0"
  external_address: "10.0.1.101"
  port: 8890
queue:
  limit: 900000000
debug:
  level: 2
  console:
    status: true
  log:
    status: true
loadbalancer:
  mode: dynamic

config/environments/cluster.yml:

cluster: "prod-cluster"
environments:
  - name: "prod-cluster"
    description: "Production HA Cluster"
    status: true
    clusters:
      - name: "prod-cluster"
        description: "3-node director cluster"
        status: true
        type: director
        properties:
          address: "10.0.1.101"
          port: 6222
          authentication:
            token: "cluster-secret-token"
          tls:
            status: true
            cert_name: "cluster-cert.pem"
            key_name: "cluster-key.pem"
            ca_name: "ca.pem"
            verify: true
          acl:
            allowed_ips:
              - "10.0.0.0/16"
            allowed_tokens:
              - "api-token-xyz"
            rate_limit_max_requests: 500
            rate_limit_window: 60
          vip:
            use_vip: true
            bind_vip: true
            vip_address: "10.0.1.100"
            vip_netmask: "255.255.255.0"
            vip_gateway: "10.0.1.1"
            vip_interface: "eth0"
            vip_monitor_interval: 5
            is_nats_cluster: true
          update:
            mode: auto
            method: cron
            cron: "0 3 * * 0"
            version: "latest"
        nodes:
          - name: "1"
            description: "Node 1 - 10.0.1.101"
            status: true
            properties:
              address: "10.0.1.101"
              port: 4222
              cluster:
                address: "10.0.1.101"
                port: 6222
                authentication:
                  token: "cluster-secret-token"
                tls:
                  status: true
                  cert_name: "cluster-cert.pem"
                  key_name: "cluster-key.pem"
                  ca_name: "ca.pem"
                  verify: true
          - name: "2"
            description: "Node 2 - 10.0.1.102"
            status: true
            properties:
              address: "10.0.1.102"
              port: 4222
              cluster:
                address: "10.0.1.102"
                port: 6222
                authentication:
                  token: "cluster-secret-token"
                tls:
                  status: true
                  cert_name: "cluster-cert.pem"
                  key_name: "cluster-key.pem"
                  ca_name: "ca.pem"
                  verify: true
          - name: "3"
            description: "Node 3 - 10.0.1.103"
            status: true
            properties:
              address: "10.0.1.103"
              port: 4222
              cluster:
                address: "10.0.1.103"
                port: 6222
                authentication:
                  token: "cluster-secret-token"
                tls:
                  status: true
                  cert_name: "cluster-cert.pem"
                  key_name: "cluster-key.pem"
                  ca_name: "ca.pem"
                  verify: true

Fleet-Managed Director​

vmetric.yml:

self_managed: false
fleet:
  type: httpclient
  instance: production
  address: wss://fleet.virtualmetric.io/ws
  token: "fleet-auth-token-abc123"
director:
  id: 2028574672450228224
listen:
  address: "0.0.0.0"
  port: 8890
debug:
  level: 1
  console:
    status: true