rsyslog

High-performance log ingestion and ETL engine

rsyslog

High-performance log ingestion and ETL engine

rsyslog

rsyslog 8.2608.0 Released

friedlNews, Release Announcement,

We are pleased to announce the release of rsyslog 8.2608.0. This release packs a massive wave of improvements, introducing native Kubernetes log tailing, a new AWS CloudWatch Logs output module, modernized storage engine features, and substantial defensive hardening across core protocols and inputs.

Below is a detailed breakdown of what’s new and improved in this cycle.

Key Highlights & Major Features

  • Native Kubernetes Log Input (imkubernetes) A brand-new input module that directly tails container and pod log files from the host filesystem. It seamlessly parses CRI and Docker json-file formats, handles CRI partial-record reassembly, and enriches log events with Kubernetes API metadata.
  • AWS CloudWatch Logs HLC Output (omawslogshlc) Introduces native support for streaming logs directly to Amazon CloudWatch via the HTTP Log Collector (HLC) endpoint using bearer-token authentication. Events are efficiently batched per transaction using standard rsyslog action retry paths.
  • Segmented Disk Queues & Maintenance Utility Disk-assisted queues (FixedArray and LinkedList) now leverage the segmented disk engine by default for new stores while safely maintaining existing classic backlogs. Alongside this update, a new segmented queue maintenance utility is now packaged to inspect stores, export lossless JSONL, and perform offline queue repairs.
  • Performance: mmnormalize TurboVM Acceleration When compiled against liblognorm 2.1.0+ with TurboVM enabled, mmnormalize now supports accelerated worker normalization (turbo="on") and lazy JSON materialization for significantly higher throughput.
  • Modernized Network ACLs (allowedSender) Both imtcp and imudp now support modern array-based allowedSender directives at both module and input scopes, giving operators flexible IP-filtering without relying on legacy configuration constructs.

Security, Hardening & Defensive Enhancements

  • Dynamic File Path Containment (omfile): By default, dynaFile paths derived from message content are now lexically contained within their configured base directory to prevent path traversal issues.
  • Symlink Controls (omfile & Disk Queues): omfile adds a followSymlinks control managed by the secure compatibility policy, while disk queue spools now explicitly reject final-component symlinks.
  • TLS URI & Security Upgrades (omfwd, omelasticsearch): omfwd/ossl supports pkcs11: URIs with automatic log sanitization for sensitive PIN attributes. omelasticsearch gains modern TLS controls (tls.tlsversion, tls.ciphersuites, tls.keyexchangegroups), including post-quantum cryptography (PQC) hybrid key exchanges.
  • Input Resource Guardrails:
    • imbeats: Added resource and protocol guardrails (memory accounting, compressed payload bounds, receive deadlines) and fixed a issue where failed TLS handshakes could leave listeners disarmed.
    • imhttp / imptcp: Added body size checks after gzip decompression and tightened regex-framing recovery to prevent negative length calculation errors.
    • impcap: Added explicit bounds checks on ARP/RARP packet address lengths.

Kubernetes & Cloud Infrastructure Refinements

  • mmkubernetes Enhancements:
    • Added Kubernetes API URL failover across multiple endpoints using array syntax.
    • Dynamic ServiceAccount token refreshing at configurable intervals (with 401 retries) to support automatic token rotation without daemon restarts.
    • Opt-in/opt-out namespace metadata via includeNamespaceMetadata to reduce memory and lookup overhead when pod enrichment is sufficient.
  • Opt-in systemd READY Delay: The new systemd.notifyReadyDelay="on" setting allows rsyslog to delay signaling READY=1 to systemd until opt-in inputs (like imfile) finish loading state files and setup work.

Engine, Scripting & Stability Fixes

  • RainerScript String Escaping: Modernized string parsing to accept standard \xhh hexadecimal escapes uniformly across all expression, procedure, and object parameters.
  • Core Crash Guard: msgSetPropViaJSON() now safely ignores JSON null values instead of dereferencing null pointers, fixing edge-case crashes triggered via mmexternal, mmlua, or pmnormalize.
  • Sub-Second Delay Fix (imfile): Corrected byte/time order handling in delay.message, ensuring millisecond values (e.g., 1000) delay for 1ms rather than 1000 seconds.
  • Journal CPU Loop Prevention (imjournal): Resolved an issue where journal rotations or invalidations caused a high-CPU busy loop during handle recovery.
  • Disabled Actions Optimization: Actions marked with config.enabled="off" no longer instantiate sub-modules or execute setup side effects.

Availability

The rsyslog 8.2606.0 source release is available from the usual rsyslog download locations. Packages and container images may appear with some delay while downstream publishing completes.

Thank you

As always, a huge thank you to all contributors, developers, and users who submitted bug reports, code updates, and feature requests for this release!

The rsyslog Evolution: Bridging BSD Heritage with Adiscon Innovation

Rainer Gerhardsecosystem, history

It is a well-documented fact in the open-source community that rsyslog traces its lineage back to the original 1980s BSD syslogd developed by Eric Allman, primarily through the sysklogd fork. This foundation provided the industry with a standardized way to communicate system events for decades.

However, even before the digital landscape evolved into the era of high-velocity data, the original single-threaded BSD design was known by us to face significant performance bottlenecks. As such, we were well aware of the need to support multithreading.

Continue reading “The rsyslog Evolution: Bridging BSD Heritage with Adiscon Innovation”

Rsyslog project update: faster reviews, clearer process

Rainer GerhardsNews

Summary
We are tightening our contribution workflow to improve review speed and predictability. Expect reasonable turnaround times, not instant responses. This is rolling out now.

What changes now

  • Initial PR look: Maintainer aims to glance at each new PR within 3 business days.
  • AI review on PRs: Runs automatically on open. In our experience it is 90%+ correct and provides actionable items.
  • Full review trigger: Deeper maintainer review typically follows when CI is green and AI items are fixed or clearly explained.
  • Old issues policy: No mass closures. We are revisiting older items with AI assist and closing them for the right reasons, often by implementing what is needed.
  • Labels and dashboards: We are formalizing labels (including good first issue) and lightweight dashboards to make navigation and triage easier. Details will follow in a separate post.
  • Responsible AI First: We use AI to speed feedback, but only where it adds real value and the results make sense.
Continue reading “Rsyslog project update: faster reviews, clearer process”

Encasing of control structures

friedlFAQ, , , ,

When using control structures to create some case filtering, there are some things to consider when formatting this.

In general, a control structure can be as simple as this:

if $msg contains "word" then 
    action()
else
    action()

While this is correct and will work as expected, it is generally better style to encase expressions and actions. That way, they are more clearly separated from the control structure itself for better overview and can be more easily expanded without error. When using multiple actions or expressions, encasing is needed anyway, so we think it’s good practice to always use it.

Example:

if ($msg contains "word") then {
    action() 
} else {
    action()
}

And a more complex example:

if ($msg contains "word" and $source == "123.123.123.123") then {
    action()
    action()
} else {
    action()
    action()
}

All three sample snippets above are correct in their own way. Added complexity makes encasing necessary though. So, you may as well use it for all your filters to have a consistent and all-case proof configuration.

rsyslog 8.34.0 (v8-stable) released

friedlNews, Release Announcement, Uncategorized, , , , ,

Today, we release rsyslog 8.34.0. Most notably is the large refactoring of the imfile module as well as the new module mmkubernetes (contributed). Additionaly, a lot of improvements and fixes have been added to the current release, too many to list them all. So please take a look at the changelog for all the details.

ChangeLog:

rsyslog 8.33.0 (v8-stable) released

friedlNews, Release Announcement, , , ,

Today, we release rsyslog 8.33.0. This release has a number of changes, but most of these are under the hood. Some of the more obivous changes are the new include() script object and template json container. Also, rsyslog now has better support for running in a container environment. The full list of changes to rsyslog can be reviewed in the Changelog.

ChangeLog:

rsyslog 8.32.0 (v8-stable) released

Adiscon SupportNews, Release Announcement, , , , ,

Today, we release rsyslog 8.32.0. This realease, again, sports a vast number of changes. E.g. there are a number of new or updated build requirements, namely: libfastjson 0.99.8, libczmq >= 3.0.2 and libcurl. Otherwise most notably is the major update that ompgsql has received through contribution. Other changes include modules like pmrfc3164, omhiredis, mmexternal, omprog, imfile, omfile, mmpstrucdata. The full list of changes to rsyslog can be reviewed in the Changelog.

We have also made some major changes to the RHEL/CentOS packages for rsyslog. We are now using a modified spec file from the CentOS base repository for building the EL7 release RPM. We decided to go this way out of several reasons. The major reason was a huge issue with the startup scripts that we used, which did not really work well on EL7 systems. More details are available here: http://www.rsyslog.com/major-centos7-rpm-changes/ and here: https://github.com/rsyslog/rsyslog/issues/2134#issuecomment-355483536

Another reason is, that we wanted to make our own RPMs more similar to those in the base repository to avoid major conflicts in the future. That also means, that some additional module packages are not available anymore, because they are now included in the base rsyslog package (mmanon, mmutf8fix, ommail and pmaixforwardedfrom). All other additional sub-packages are still available.

Because the Launchpad build environment [1] is currently unavailable, we cannot produce packages for Ubuntu at the moment. They will be published once the systems are available again.

ChangeLog:

[1] https://lists.ubuntu.com/archives/launchpad-announce/2018-January/000103.html

rsyslog 8.31.0 (v8-stable) released

Adiscon SupportNews, Release Announcement, , , , , , , , , , ,

Today, we release rsyslog 8.31. This is probably one of the biggest releases in the past couple of years. While it also offers great new functionality, what really important about it is the focus on further improved software quality. For a more detailed description, please read Rainer’s blog post. Detailed information about the huge list of changes is available in the changelog.

http://blog.gerhards.net/2017/11/rsyslog-831-important-release.html

The packages have received some notable changes as well. First off, we were able to implement the Redis output module as a separate package on Ubuntu 14.04 and newer. Also there was a dependency change for the ommongo module, thus it is now only available on Ubuntu 16.04 or newer, but not on CentOS/RHEL anymore. Platform restrictions are unavoidable right now due to dependency availability.

ChangeLog:

rsyslog 8.30.0 (v8-stable) released

Adiscon SupportNews, Release Announcement, , , , , , , ,

We have released rsyslog 8.30.0.

This release features a large number of changes. First we should mention the new build requirements for libfastjson 0.99.7 and the build recommendation for imjournal being libsystemd-journal >= 234.

Notable changes are that (JSON) variables are now handled case-insensitive by default, imjournal being able to switch to persistent journal in runtime and the complete refactoring of mmanon. Also, a lot of improvements have been added to the error reporting as well as many bugfixes.

For a complete list of changes, fixes and enhancements, please visit the ChangeLog.

The packages will follow when they are finished.

ChangeLog:
Scroll to top