rsyslog on Amazon Web Services (AWS Marketplace App)

Welcome to this guide on updating an existing CloudFormation stack for the rsyslog server on AWS. In this tutorial, we will walk you through the steps necessary to ensure your rsyslog server is running the latest version with all the benefits of updated features and performance improvements. We will provide detailed instructions and screenshots to make the update process straightforward, ensuring minimal disruption to your logging setup. Whether you’re a seasoned AWS user or new to CloudFormation, this guide will help you achieve a smooth and efficient update.

Prerequisites

If changes were made to the rsyslog configuration, use the guide in this article to back up and restore configuration: AWS rsyslog Sync Configuration with S3.

Step 1: Select the CloudFormation Stack

To begin the update process for your rsyslog server on AWS, first, navigate to the AWS Management Console and go to the CloudFormation section. Here, locate the stack you wish to update.

1. Visit AWS CloudFormation: Log in to your AWS Management Console and go to the CloudFormation service.
2. Select Your Stack: Identify and select the CloudFormation stack for your rsyslog server. In this example, the stack is named “rsyslogtest”.
3. Initiate Update: Click on the Update button, as highlighted in the screenshot above.

This will start the process to update your existing CloudFormation stack.
Click Update to proceed.

Step 2: Prepare the Template

After selecting the stack to update, the next step involves preparing the template for the update. Follow these instructions:

1. Choose Template Option: In the “Prepare template” section, select the Replace existing template option.
2. Specify Template Source: Under “Template source”, choose Amazon S3 URL.
3. Enter S3 URL: Enter the following URL in the provided field:

   https://rsyslogpublic.s3.amazonaws.com/office_rsyslog_server.yaml

Alternatively, you can use the template URL provided on the AWS Marketplace product page for the rsyslog server.

This will prepare the new template to be applied to your existing stack.
Click Next to proceed.

Step 3: Specify Stack Details

After preparing the template, proceed to specify the stack details:

1. Review Parameters: Ensure all parameters are correct. Adjust as necessary.
2. Instance Settings:

• Identifier Name: Change if necessary.
• EC2 Instance Type: Change if necessary, as a new instance will be deployed.

Review all options carefully in case new features have been added.
Once all configurations are reviewed and adjusted, click Next to proceed.

Step 4: Configure Stack Options and Review

Review the stack options and make any necessary adjustments.

1. Review Changes: Carefully review the list of changes in the “Change set preview”. Ensure all modifications align with your expectations.
2. Submit: Once everything is reviewed and confirmed, click the Submit button to start the update process.

After clicking Submit, AWS will begin updating your CloudFormation stack. Monitor the progress to ensure the update completes successfully. If any issues arise, refer to the stack events for troubleshooting.

Step 5: Monitor the Update Process

1. Monitor Progress: Check the events tab to monitor the progress of the update. The status should show “UPDATE_IN_PROGRESS” and various components being modified.
2. Confirm Completion: When the update completes, ensure the status changes to “UPDATE_COMPLETE”.

Once the process is complete, verify that the CloudFormation stack was updated successfully by checking the final status and confirming that all intended changes were applied correctly.

Confirm EC2 Instance Running rsyslog Server

1. Access the EC2 Instance: Use SSH to log in to your EC2 instance running the rsyslog server.
2. Verify rsyslog: Once logged in, confirm that the rsyslog server is running properly. You should see the rsyslog welcome message, indicating that the application is installed and operational.

Check the rsyslog meta configuration located in /opt/rsyslog/config to ensure all settings are correct and the service is functioning as expected. This final verification confirms the successful update of your CloudFormation stack and the deployment of the new rsyslog server instance.

Back to aws rsyslog overview.

S2: v13 rsyslog

We are excited to announce the second public release of Rsyslog Server on AWS Marketplace. This version includes efficient logging, noise event filtering, and a streamlined web interface for system management. New features: Cloudwatch LogGroups, logfile compression, S3 log/config storage, enhanced CloudFormation support, and improved AWS region handling. Experience enhanced logging capabilities and simplified management with Rsyslog Server.

Key Features:

• Cloudwatch LogGroups Integration: Now you can leverage AWS Cloudwatch LogGroups for better log management and analysis directly through our CloudFormation templates.
• Logfile Compression: Enabled logfile compression to optimize storage and improve performance.
• S3 Bucket Support: Added S3 support for both log storage and configuration backup, ensuring your data is safely stored and easily accessible.
• Improved AWS Region Handling: Fixed issues with AWS_REGION in rsyslogctl to automatically use the correct region configuration.
• Configuration Sync to S3: Fixed the sync_config_history_to_s3 feature, ensuring your configuration history is consistently backed up.
• Enhanced CloudFormation Features: Added new features, including S3 support, access policies, and additional InstanceTypes, to our CloudFormation file for easier and more flexible deployments.
• EFS Resource Management: Added Delete and Retain policies to EFS resources to ensure they survive a Stack Update.

S2: v12 rsyslog

We are excited to announce the inaugural public release of the official Rsyslog Server product on AWS Marketplace. This release introduces an efficient logging solution right out of the box. Additionally, it empowers users with seamless configuration options to filter out noise events and includes a streamlined web interface for system operation management. Get ready for enhanced logging capabilities and simplified management with the Rsyslog Server.

Back to aws rsyslog overview.

Ensuring the integrity and availability of your rsyslog configuration is crucial for maintaining a robust logging system. By syncing your rsyslog configuration to AWS S3, you create a reliable backup that can be easily restored when needed. This guide will walk you through the process of syncing your rsyslog configuration to S3 and restoring it when necessary.

Prerequisites

Before you begin, ensure you have the following:

• rsyslog server from AWS Marketplace with at least Version S2: v13 rsyslog
• S3Bucket (Is created by default since S2: v13 rsyslog or higher).

Syncing Configuration History to S3

This command synchronizes the current rsyslog configuration history to a specified S3 bucket. It ensures all configuration changes are backed up, providing a reliable recovery solution.

sudo rsyslogctl sync-config-history-to-s3

When executed, this command uploads your rsyslog configuration history to the S3 bucket configured in your settings. Regular execution of this command, especially after making significant changes, ensures your backups are always current.

Restoring Configuration from S3

This command downloads the rsyslog configuration history from an S3 bucket to the local machine, facilitating configuration restoration.

sudo rsyslogctl sync-s3-to-config-history

Executing this command retrieves the configuration history from S3 and applies it to your local rsyslog setup.

Back to aws rsyslog overview.

Once you’ve updated the configuration of the AWS rsyslog application, it’s important to manually apply the new settings as rsyslog doesn’t do this automatically. This is to prevent partial changes from being loaded and potentially causing issues.

The AWS rsyslog AWS application provides a dedicated tool, rsyslogctl, which can be used to check and reload the configuration. During the reload process, rsyslogctl determines the most efficient way to apply the changes. For example, some changes like drop rules can be applied without interrupting message processing, while others require a full restart, causing a brief interruption.

The EBS disk included in the product is only used for day-to-day storage of logs. Persistent log storage is kept on an S3 store. This store also contains some other data items which should persist over upgrades of the rsyslog on AWS application.

The following prefixes/folders are used by rsyslog:

• /rsyslog.logstore/ – the actual logstore
  This is synced with data from the local EBS disk once a day for the past day (in default settings).
• /rsyslog.config/ – config data items.
  This contains the user-based config which can be restored from here during an upgrade or on misconfiguration.

The users should select proper S3 policies based on her or his needs. Most importantly, Versioning and Retention Period should be set accordingly.

The S3 store to use can be configured during the cloud formation process and manually via the meta config.

Back to aws rsyslog overview.

Our team at Adiscon offers a comprehensive paid full-service rsyslog product, available on the AWS Marketplace. As the same team that develops and supports the rsyslog open source project, we’re dedicated to providing exceptional service and ongoing innovation.

By purchasing our AWS Marketplace product, you’re also supporting the continued development of rsyslog. This ensures that the open source project remains robust, reliable, and up-to-date.

Our full-service rsyslog offering is designed specifically for organizations seeking a seamless and hassle-free way to collect syslog data on the cloud. We provide ongoing support and maintenance, along with regular updates to ensure the highest level of performance and security.

In summary, our AWS Marketplace product is the perfect solution for organizations that value simplicity, efficiency, and reliability when it comes to collecting syslog data in the cloud.

Documentation is available (and constantly being improved). Please follow these links:

• Technical Overview
• Configuration
  • Meta-Configuration settings
  • Message-Drop Filter
  • Applying Configuration Changes
• S3 file structure
  • Sync Configuration with S3
• Updating with CloudFormation stack
• Version History