Administration: Overview
The following chapters document the administration features of Director.
Key Components
VirtualMetric Director™ is a tool used primarily for telemetry purposes, and its administration involves the following key components:
Sources of log data, e.g. syslog or estreamer | |
Processing workflows for data transformation | |
Individual data manipulation functions | |
Traffic control for directing data flows | |
Destinations for processed data, e.g. Sentinel or a storage system |
The following graphic illustrates the various stages where these components are used and how they connect to each other:
For the PREPROCESSING, ROUTES, and POSTPROCESSING stages, Director uses Pipelines, which are composed of Processors.
To ingest data from the SOURCES and to communicate with them, Director uses Devices.
To forward processed data to TARGETS and to communicate with them, Director uses Targets.
Directories
To help design the logic of the components that will define, combine, and run the above stages, Director uses the configuration files that can be found in the folders under vm_root:
All administration tasks are carried out with these files.
The YML configuration files for each type of component have their own directory. These files contain predefined fields that these components recognize, and Director uses the settings defined in them to spawn and run its processes.
Definition Files
Two directories are of key significance: package and user.
The package directory contains templates and ready-to-use definitions. These definitions are updated with newer versions of Director.
Never modify the definition files under package directly. To create a configuration using one of these as a template, copy the relevant file to the corresponding location under user first, and then edit it to suit your needs.
The user directory contains custom configurations. These definitions take precedence over those under the package directory.
The definitions under user override the definitions under package, and they are preserved between updates.
Basic Tasks
The system administrator's responsibilities cover the following.
Maintenance
This entails maintaining an up-and-running system by attending to the following:
-
Settings - Creating and modifying the setting files, validating and applying them, and keeping backups of them.
-
Devices - Creating devices to listen new data sources, configuring their input parameters, monitoring their health, and troubleshooting connectivity issues.
-
Pipelines - Designing workflows, configuring processors, test the transformations, and optimizing performance
-
Routes - Defining traffic patterns, setting up filtering rules, configuring destinations, and monitoring data flows
-
System monitoring - Tracking performance metrics, monitoring resource usage, reviewing error logs, and generating reports
Security
A crucial aspect of system administration is security. This requires attending to the following:
-
Authentication - Configuring access controls, managing credentials, setting up encryption, and monitoring access logs
-
Networks - Configuring TLS/SSL, setting up firewalls, managing certificates, and controlling port access
-
Data protection - Securing sensitive data, configuring encryption, managing data retention, and supervising compliance
Troubleshooting
System administration frequently involves dealing with errors and failures.
The most common issues are related to misconfiguration, connectivity, unhandled disruption of pipelines, and violated resource constraints.
The resolution generally involves checking the configurations and reviewing the logs, verifying connectivity and resource availability, and testing the solution in its final state.
Best Practices
In order to maintaing the integrity, robustness, and health of a system, some guidelines have to observed:
For managing configurations, use version control, document the changes, and always keep backups. Also, never forget to test before deploying.
For performance optimization, first monitor the resource usage and balance the loads if necessary. Optimizing the configurations is the next thing to check. And do not forget to schedule maintenance. Error tracking and health checks must also be always kept in mind.
Finally, for security reasons, updates are essential. Official security audits must be conducted regularly, particularly on access control and encryption.