Search Results for: error 0

The “meta configuration” is provides a simple-to-use way for rsyslog server customization, without the need to manage rsyslog.conf and it’s helpers directly.

The configuration is stored inside the text file “rsyslog-features.env” in “/opr/rsyslog/config“. Each line consists of a keyword, an equal sign, and an associated value. The AMI file contains this file with default settings. The CloudFormation script generates an initial file based on data provided by the user during the CloudFormation process.

This is an example of a meta-config file:

# Internal Statistic Counters
RSYSLOG_PSTATS_ENABLED="on"
RSYSLOG_PSTATS_FILE_ENABLED="on"
RSYSLOG_PSTATS_CLOUDWATCH_ENABLED="off"

# Logstore Parameters
RSYSLOG_LOGSTORE_PATH="/var/log/rsyslog.store/"
RSYSLOG_LOGSTORE_CLOUDWATCH_ENABLED="off"
RSYSLOG_LOGSTORE_CLOUDWATCH_RETENTION="7"

# Logrotation Parameters
LOGROTATE_COMPRESS="zstd"
RSYSLOG_LOGROTATION_DAYS="7"

# S3 Storage Parameters
RSYSLOG_S3_ID="rsyslogfiles"
RSYSLOG_S3_LOGSTORE="off"
RSYSLOG_S3_LOGSTORE_PATH="/rsyslog.store/"
RSYSLOG_S3_CONFIG_PATH="/rsyslog.config/"

Editing the Meta-Config

Currently no specific feature for editing the meta-config exists. We suggest to use the text editor of your choice to modify the text file directly. Note: the file currently is writeable for the “root” user, only.

Configuration Parameters

Note: Parameter values contain either a string or number or the special values “on” and “off” to turn specific features on or off.

Storage Options

The storage options permit to assign where persistent storage is located.

• RSYSLOG_LOGSTORE_PATH
  Full path name where logs incoming from the network should be stored. On this store, a two-level directory structure is automatically be created: first level is the hostname, second level is the syslog tag from the incoming message. For security reasons, slashes inside hostname or tag are replaced by underscores. The log store is automatically rotated when log rotation is turned on.
• RSYSLOG_LOGSTORE_CLOUDWATCH_ENABLED
  Enables or disables sending all syslog data to a CloudWatch LogGroup.
• RSYSLOG_LOGSTORE_CLOUDWATCH_RETENTION
  Retention time in days for CloudWatch LogGroup.

Please note that sending syslog data to CloudWatch is a priced item in AWS and can cause additional costs of being used. Therefor it is disabled by default.

Internal Statistic Counters

• RSYSLOG_PSTATS_ENABLED
  Turns the rsyslog-internal statistics system (impstats) completely on or off. If any other pstats-related settings are to be used, this must be set to “on”.
• RSYSLOG_PSTATS_FILE_ENABLED
  Turns on or off writing statistics records to a local log file.
• RSYSLOG_PSTATS_CLOUDWATCH_ENABLED
  Turns on or off pushing select statistics counters to the AWS CloudWatch Dashboard.

Please note that sending statistics to CloudWatch is a priced item in AWS. For that very reason, we also permit to write statistics to local files, which involves no cost but storage fees. File writing and CloudWatch push can be turned on or off independently. Both can be used together.

Log Rotation Parameters

Our application rotates log files daily shortly after midnight using advanced rsyslog capabilities and our own custom logic. We do not utilize the logrotate tool. As part of the log rotation process, the previous day’s log files are compressed (if the compression feature is enabled) and then synced to an S3 bucket (if the RSYSLOG_S3_ID configuration variable is set). After the log files have been successfully synced, our local retention policy is applied to manage disk space usage.

It’s important to note that our application does not enforce any retention policy for S3 buckets. Therefore, it’s crucial to set an appropriate retention policy for your S3 bucket using AWS tooling to avoid exceeding your desired storage limits or incurring unnecessary costs. Please ensure that you set a retention policy that meets your specific business requirements.

The configuration can be customized via these settings:

• LOGROTATE_COMPRESS
  Specifies the compression method to be used during rotation. Currently, the available options are “no” and “zstd“. If set to “Parameters”, no compression will be applied.
• RSYSLOG_LOGROTATION_DAYS
  Specifies the number of days that log files should be kept locally. After the specified period, they will be deleted. To delete local log files immediately after they have been synced to S3, set the value to zero.

S3 Storage Parameters

• RSYSLOG_S3_ID
  The id (name) of the S3 store to be used for persistent storage. Leave empty to disable automatic synchronizing to S3. The S3 store’s structure is described here.
  Note: automatic data sync to S3 happens after log rotation, which is daily run shortly after midnight.
• RSYSLOG_S3_LOGSTORE
  Enables or disables storing rotated logfiles into S3 Bucket.
• RSYSLOG_S3_LOGSTORE_PATH
  Like the local LOGSTORE_PATH, this is the “path” (prefix) used inside the S3 store for log files.
  Note: if RSYSLOG_S3_ID is empty, this parameter is ignored.
• RSYSLOG_S3_CONFIG_PATH
  The “home path” for persistent storage of the rsyslog user configuration.
  Note: if RSYSLOG_S3_ID is empty, this parameter is ignored.

Important: The beta version’s default log store is not persistent and will not survive instance termination. To ensure data is not lost, users should configure an external EBS disk for persistent storage. Currently, mount settings must be created manually as there is no automatic setting. However, this feature is planned for future beta versions.

Applying Configuration Changes

Important: after changing the meta configuration, be sure to check it against any typos or other problems. This can be done via the command sudo rsyslogctl check-config. If this is error-free, you can apply the new config via sudo rsyslogctl reload-config.

For more information on how to apply configuration changes to rsyslog on AWS configuration changes, please refer to this dedicated page.

Back to aws rsyslog overview.