Slightly Changed rsyslog Stable Release Cycle

For the past couple of years, rsyslog made scheduled releases every 6 weeks. We now changed this slightly to make version numbers easier to understand.

Remember, rsyslog versions are called 8.<yy><mm>.0, so the April 2020 release is 8.2004.0. When we release very six weeks, we get odd and even month numbers and, even more confusing, we sometimes seem to “skip” a month while at other times it looks like we craft a scheduled stable “every month”. To avoid this type of confusion, we have now decided to release every two month, and do that on even month.

We will usually try to release in the second half of the given month. However, we will no longer tell the exact target date. We need some flexibility here to avoid targeting “bad release periods”. As a concrete example, we will probably never do a December release during the holiday period. As such, December releases are more likely to happen in the first half of the month, which should give admins also some time to do all of their internal testing work ahead of the holidays.

We originally used the six week schedule to provide a balance between frequent bug fixes and not too frequent releases. With the appearance of daily stable releases a longer release cycle is no more a real concern. Everybody in need of a fix not yet present in the scheduled stable can just switch to the daily stable as needed. Remember that both are stable versions. The daily stable is often more stable as it contains the latest fixes.

RSyslog Windows Agent 6.1 Released

Release Date: 2020-01-31

Build-IDs: Service 6.1.0.205, Client 6.1.0.280

Features

  • Property engine: Added new static property %localhostname% which contains the local computer name.
  • Syslog Action: Fixed Syslog Version in RFC5424 Header to 1.

Bugfixes

  • EventLog Monitor V2: Fixed an issue losing the first record LastRecord was resetted.
  • EventLog Monitor V2: Fixed minor issues in new caching code.
  • Queue Engine: Fixed an issue in the Action retry logic which caused the same information to be reprocessed again.
  • Property Engine: Fixed an issue initializing the socket subsystem if no network action/service was used related to the toipv4address/toipv6address options.
  • Engine: Fixed multiple memory leaks when a name was resolvedto an IP address.

You can download Free Trial Version of RSyslog Windows Agent.

Encasing of control structures

When using control structures to create some case filtering, there are some things to consider when formatting this.

In general, a control structure can be as simple as this:

if $msg contains "word" then 
    action()
else
    action()

While this is correct and will work as expected, it is generally better style to encase expressions and actions. That way, they are more clearly separated from the control structure itself for better overview and can be more easily expanded without error. When using multiple actions or expressions, encasing is needed anyway, so we think it’s good practice to always use it.

Example:

if ($msg contains "word") then {
    action() 
} else {
    action()
}

And a more complex example:

if ($msg contains "word" and $source == "123.123.123.123") then {
    action()
    action()
} else {
    action()
    action()
}

All three sample snippets above are correct in their own way. Added complexity makes encasing necessary though. So, you may as well use it for all your filters to have a consistent and all-case proof configuration.

RSyslog Windows Agent 6.0 Released

Release Date: 2019-07-01

Build-IDs: Service 6.0.0.200, Client 6.0.0.277

Features

  • Configuration Client using latest .Net Framework now.
  • Windows 2019 Support.
  • Configuration reload: Added support to update configuration from https url.
  • Property engine: Added new options
    • date-rfc3164strict: Same as date-rfc3164 but with two spaces
      after month if day is below 10.
    • crlftovbar: Replaces all CRLF or LF to vertical bar character.
    • replacechar: Replaces a single character with another single
      character, sample: %msg:$x:$y:replacechar%. More information’s can be
      found in the manual.
    • removecc: Removes all control characters from 0x00 to 0x1F
  • EventLog Monitor V2: Added JSON Output format Option to MessageFormat
    selection.
  • EventLog Monitor V2: Refactored code for performance improvements.
    Implemented Event Caching options for Publisher handles, Levels, Tasks and
    Keywords.
  • TLS Support: Enhanced error reporting when OpenSSL errors are being
    processed internally.
  • HTTP Request Action: Added support for https urls.

Bugfixes

  • Fixed configuration reload freeze when debug logging was enabled.
  • Syslog Action: Added missing RealSource option when using RFC3164 syslog
    header.

You can download Free Trial Version of RSyslog Windows Agent.

RSyslog Windows Agent 5.2 Released

Release Date: 2019-04-25

Build-IDs: Service 5.2.0.195, Client 5.2.0.274

Features

  • Hardened Code using new utilities like cppcheck and LLVM.

Bugfixes

  • Fixed processing issues with date property replacer options.
  • Fixed a bug with UTF8 encoding detection (String content before the encoded string (BOM) was dropped).

You can download Free Trial Version of RSyslog Windows Agent.

Avoid overly-large in memory queues

Rsyslog provides the “queue.size” parameter to set a limit on the number of messages a queue can keep in memory. This is primarily meant to support peak traffic.

Note that this counter is given in number of messages, not bytes. A frequent mistake is to think in bytes and select very large values (e.g. 7 million frequently seen, maybe due to a web tutorial somewhere). If queues are that large there is a chance the rsyslog will be aborted by out of memory condition when the queue gets fuller and fuller.

An example. You send data to a remote syslog server. You define a very large queue on it. Usually, the queue keeps very slow. But when the system goes offline, the queue fills up. This will lead to sharply increasing memory usage. Depending on all circumstances this may not be a problem – or it may be! The likelihood of becoming problematic, and harder to reproduce, increases with the number of queues defined.

To avoid such misunderstandings, rsyslog starting at 8.1905.0 emits a warning message. It has probably lead you to this page. If the queue size is correct, you can ignore the warning message. You can also filter it out via regular rules, if you like. But if you did not intend to define such a large queue, please reconsider the value.

Note: rsyslog considers queues larger than 500,000 messages to be overly large – there seldom is a good reason to use sizes in excess of that.

solving rsyslog write errors

When rsyslog reports a write error, it includes the operating-system generated error message. It should hopefully give you a clue what the problem cause was. Unfortunately from time to time to root cause is not obvious.

In this case please check the following potential causes:

  • Was OS/rsyslog config change applied but rsyslog not restarted? Rsyslog configuration changes are only applied when rsyslog is restarted. Similarly, many operating system process limitations (like file size and several permission settings) are only applied if process is restarted. If in doubt, do a restart of rsyslog. Doing so can potentially save you a lot of time.
  • Is rsyslog configured to drop privileges? If so, the user or group dropped to may simply not have the right permission. Try to comment out the privilege drop to see if this is the root cause.
  • Does SELinux prevent rsyslog to access the file? This is often the case if you write to non-standard locations. To check if this is the cause, you can disable SELinux on the system. If it then works, you know the root cause. But please do not run with SELinux disabled. Instead, configure it correctly.
  • Are you using something similar to SELinux? For example AppArmor on Ubuntu? Investigate and check if it causes the trouble.
  • Do you run rsyslog via systemd? Are there any limits specified in the service file? Most modern Linuxes use systemd, so this is for sure a place to check.
  • Are there any global limits specified in the system configuration? Note: systemd ignores them, so if you use systemd, your really need to check the systemd configuration and rsyslog’s unit file!
  • Are there any file system limitations?
  • Did the system (temporarily) run out of space? This could especially be the case for intermittent problems.

This list probably is not conclusive but should give you a good idea of known trouble spots.

For a quick but rough check to find the culprit, you can run rsyslog in an interactive terminal window. Use the root account and do not drop privileges. If it works there, chances are pretty good that some other operating system component is causing the trouble.

rsyslog version numbering change

Rsyslog used a version number scheme of

8.<real-version>.0

where we incremented <real-version> every 6 weeks with each release. The 8 and 0 are constant (well, the 0 could change to 1 with a very important patch, but in practice we have only done this once).

While this scheme has worked pretty well since we introduced it, we often see people not understanding that there is really a big difference between 8.24 and e.g. 8.40. Followind recent trends in software versioning, we  will make more clear how old a version really is. Begining with today’s release, we change the version number slightly  to

8.yymm.0

where yy is the two-digit year and mm the two-digit month of the release date. We release every 6 weeks, so we will never have two releases within the same month.

So while you expected 8.41.0, you will now get 8.1901.0. To make things even more clear, rsyslog visible version output will be even more up to the point: rsyslog -v will now report “8.1901.0 (aka 2019.01)“.

Rainer Gerhards’ blog has more details on why we did this change and how we came to the new system.

Scroll to top