rsyslog

The rocket-fast system for log processing pipelines

rsyslog

The rocket-fast system for log processing pipelines

Search Results for: queue

rsyslog 6.4.1 (v6-stable) released

Adiscon SupportNews, Release Announcement, , , , ,

We have just released 6.4.1 of the v6-stable branch. This is primarily a maintenance release. It includes several bugfixes for using JSON, if you are using multiple main queues and many more. More details can be found in the ChangeLog.

ChangeLog:

http://www.rsyslog.com/changelog-for-6-4-1-v6-stable/

Download:

http://www.rsyslog.com/rsyslog-6-4-1-v6-stable/

As always, feedback is appreciated.

Best regards,
Florian Riedl

Changelog for 5.10.0 (v5-stable)

Adiscon SupportChangelog, , , , ,

Version 5.10.0  [V5-STABLE], 2012-08-23

NOTE: this is the new rsyslog v5-stable, incorporating all changes from the 5.9.x series. In addition to that, it contains the fixes and enhancements listed below in this entry.

  • bugfix: delayble source could block action queue, even if there was a disk queue associated with it. The root cause of this problem was that it makes no sense to delay messages once they arrive in the action queue – the “input” that is being held in that case is the main queue worker, what makes no sense.
    Thanks to Marcin for alerting us on this problem and providing instructions to reproduce it.
  • bugfix: disk queue was not persisted on shutdown, regression of fix to
    http://bugzilla.adiscon.com/show_bug.cgi?id=299
    The new code also handles the case of shutdown of blocking light and full delayable sources somewhat smarter and permits, assuming sufficient timouts, to persist message up to the max queue capacity. Also some nits in debug instrumentation have been fixed.
  • add small delay (50ms) after sending shutdown message
    There seem to be cases where the shutdown message is otherwise not processed, not even on an idle system.
    Thanks to Marcin for bringing this problem up.
  • support for resolving huge groups
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=310
    Thanks to Alec Warner for the patch
  • bugfix: potential hang due to mutex deadlock
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=316
    Thanks to Andreas Piesk for reporting&analyzing this bug as well as providing patches and other help in resolving it.
  • bugfix: property PROCID empty instead of proper nilvalue if not present
    If it is not present, it must have the nilvalue “-” as of RFC5424
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=332
    Thanks to John N for reporting this issue.
  • bugfix: “last message repeated n times” message was missing hostname
    Thanks to Zdenek Salvet for finding this bug and to Bodik for reporting
  • bugfix: multiple main queues with same queue file name was not detected
    This lead to queue file corruption. While the root cause is a config error, it is a bug that this important and hard to find config error was not detected by rsyslog.

rsyslog 5.8.13 (v5-stable) released

Adiscon SupportNews, Release Announcement, , , , ,

We just released 5.8.13 of the v5-stable branch. This is primarily a maintenance release. It includes several bugfixes for potential aborts or DA queues as well as other fixes. For more details, please review the ChangeLog.

This is the last release of 5.8.x to get the latest patches into this stable version. Soon we will release 5.10.1 which should be the version to be used if you are following the stable releases.

ChangeLog:

http://www.rsyslog.com/changelog-for-5-8-13-v5-stable/

Download:

http://www.rsyslog.com/rsyslog-5-8-13-v5-stable/

As always, feedback is appreciated.

Best regards,

Florian Riedl

How to check if config variables are used?

Adiscon SupportFAQ, ,

Sometimes you might wonder, if the configuration you created is really used. At least parts of it. This could really happen in a lot of situations. Currently, the config format is changed a lot. This forces users who want to use the new format to use a mixed mode of old and new config style. And this is where a lot of confusion can occur, which results in not properly set config variables. In fact, you just need to create some consistency in the configuration. Most output modules are already updated. If you want to use the new format, you cannot use some old config directives and use some new config directives at the same time. The old ones will be simply ignored then. Instead the default values will be used. A very common case is with queues and this is what I will use for my example. I will show, how you can identify if configuration directives are used correctly. We will use the debug log for this. The rsyslog version I will use is the 6.3.12 beta. In this version, the main message queue still needs to be configured with the old config directives, whereas action queues already support the new config directives. You can enable debug mode for rsyslog with the following commands in a terminal:

export RSYSLOG_DEBUGLOG="/path/to/debuglog" export RSYSLOG_DEBUG="Debug"

You can now start rsyslog on the same command line with:

rsyslogd -c6

You will usually see the debug output in that same terminal, as rsyslog runs in the foreground. That should now just serve as a indicator, that the debug output works. Now here is the config snippet I used:

$ActionQueueType LinkedList $ActionQueueSize 2000000 $ActionQueueTimeoutEnqueue 0 $ActionQueueDequeueBatchSize 400 *.* action(type="omfwd" target="10.10.10.12" port="514" protocol="udp")

As you can see, we configured the action queue with some custom variables in the old fashion, basically the action queue and the action itself with the new style. If you create a new debug file now and review it, search for “action 1” in this case. You should see the following. This screenshot is a excerpt of the debug log. I marked several positions. The green circles show, that the action parameters have been used correctly. The red circles show two of the action queues parameters, where the defaults have been used. This is just a example on how to identify, if your configuration was loaded successfully.

Changelog for 6.2.2 (v6-stable)

Adiscon SupportChangelog, , , , ,

Version 6.2.2  [v6-stable], 2012-06-13

  • build system improvements and spec file templates
    Thanks to Abby Edwards for providing these enhancements
  • bugfix: disk queue was not persisted on shutdown, regression of fix to
    http://bugzilla.adiscon.com/show_bug.cgi?id=299
    The new code also handles the case of shutdown of blocking light and full delayable sources somewhat smarter and permits, assuming sufficient timouts, to persist message up to the max queue capacity. Also some nits in debug instrumentation have been fixed.
  • bugfix: –enable-smcustbindcdr configure directive did not work
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=330
    Thanks to Ultrabug for the patch.
  • add small delay (50ms) after sending shutdown message. There seem to be cases where the shutdown message is otherwise not processed, not even on an idle system.
    Thanks to Marcin for bringing this problem up.
  • support for resolving huge groups
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=310
    Thanks to Alec Warner for the patch
  • bugfix: potential hang due to mutex deadlock
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=316
    Thanks to Andreas Piesk for reporting&analyzing this bug as well as providing patches and other help in resolving it.
  • bugfix: property PROCID empty instead of proper nilvalue if not present
    If it is not present, it must have the nilvalue “-” as of RFC5424
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=332
    Thanks to John N for reporting this issue.
  • bugfix: did not compile under solaris due to $uptime property code
    For the time being, $uptime is not supported on Solaris
  • bugfix: “last message repeated n times” message was missing hostname
    Thanks to Zdenek Salvet for finding this bug and to Bodik for reporting

Changelog for 5.8.12 (v5-stable)

Adiscon SupportChangelog, , , , ,

Version 5.8.12  [V5-stable] 2012-06-06

  • add small delay (50ms) after sending shutdown message
    There seem to be cases where the shutdown message is otherwise not processed, not even on an idle system.
    Thanks to Marcin for bringing this problem up.
  • support for resolving huge groups
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=310
    Thanks to Alec Warner for the patch
  • bugfix: delayble source could block action queue, even if there was a disk queue associated with it. The root cause of this problem was that it makes no sense to delay messages once they arrive in the action queue – the “input” that is being held in that case is the main queue worker, what makes no sense.
    Thanks to Marcin for alerting us on this problem and providing instructions to reproduce it.
  • bugfix: disk queue was not persisted on shutdown, regression of fix to
    http://bugzilla.adiscon.com/show_bug.cgi?id=299
    The new code also handles the case of shutdown of blocking light and full delayable sources somewhat smarter and permits, assuming sufficient timeouts, to persist message up to the max queue capacity. Also some nits in debug instrumentation have been fixed.
  • bugfix/omudpspoof: problems, including abort, happend when run on multiple threads. Root cause is that libnet is not thread-safe. omudpspoof now guards libnet calls with their own mutex.
  • bugfix: if debug message could end up in log file when forking if rsyslog was set to auto-background (thus fork, the default) and debug mode to stdout was enabled, debug messages ended up in the first log file opened. Currently, stdout logging is completely disabled in forking mode (but writing to the debug log file is still possible). This is a change in behaviour, which is under review. If it causes problems to you, please let us know.
    Thanks to Tomas Heinrich for the patch.
  • bugfix/tcpflood: sending small test files did not work correctly
  • bugfix: potential hang due to mutex deadlock
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=316
    Thanks to Andreas Piesk for reporting&analyzing this bug as well as providing patches and other help in resolving it.
  • bugfix: property PROCID empty instead of proper nilvalue if not present
    If it is not present, it must have the nilvalue “-” as of RFC5424
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=332
    Thanks to John N for reporting this issue.

Changelog for 6.3.10 (v6-beta)

Adiscon SupportChangelog, , , ,

Version 6.3.10 [BETA] 2012-06-04

  • bugfix: delayble source could block action queue, even if there was disk queue associated with it. The root cause of this problem was that it makes no sense to delay messages once they arrive in the action queue – the “input” that is being held in that case is the main queue worker, what makes no sense.
    Thanks to Marcin for alerting us on this problem and providing instructions to reproduce it.
  • bugfix: invalid free in imptcp could lead to abort during startup
  • bugfix: if debug message could end up in log file when forking if rsyslog was set to auto-background (thus fork, the default) and debug mode to stdout was enabled, debug messages ended up in the first log file opened. Currently, stdout logging is completely disabled in forking mode (but writing to the debug log file is still possible). This is a change in behaviour, which is under review. If it causes problems to you, please let us know.
    Thanks to Tomas Heinrich for the patch.
  • bugfix: –enable-smcustbindcdr configure directive did not work
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=330
    Thanks to Ultrabug for the patch.
  • bugfix: made rsyslog compile when libestr ist not installed in /usr
    Thanks to Miloslav Trmač for providing patches and suggestions

Changelog for 6.3.9 (v6-beta)

Adiscon SupportChangelog, , , , ,

Version 6.3.9  [BETA] 2012-05-22

  • bugfix: imtcp could cause hang during reception this also applied to other users of core file tcpsrv.c, but imtcp was by far the most prominent and widely-used, the rest rather exotic (like imdiag)
  • added capability to specify substrings for field extraction mode
  • added the “jsonf” property replacer option (and fieldname)
  • bugfix: omudpspoof did not work correctly if no spoof hostname was configured
  • bugfix: property replacer option “json” could lead to content loss message was truncated if escaping was necessary
  • bugfix: assigned ruleset was lost when using disk queues
    This looked quite hard to diagnose for disk-assisted queues, as the pure memory part worked well, but ruleset info was lost for messages stored inside the disk queue.
  • bugfix/imuxsock: solving abort if hostname was not set; configured hostname was not used (both merge regressions)
  • bugfix/omfile: template action parameter was not accepted (and template name set to “??” if the parameter was used)
    Thanks to Brian Knox for alerting us on this bug.
  • bugfix: ommysql did not properly init/exit the mysql runtime library this could lead to segfaults. Triggering condition: multiple action instances using ommysql.  Thanks to Tomas Heinrich for reporting this problem and providing an initial patch (which my solution is based on, I need to add more code to clean the mess up).
  • bugfix: rsyslog did not terminate when delayable inputs were blocked due to unvailable sources. Fixes:
    http://bugzilla.adiscon.com/show_bug.cgi?id=299
    Thanks to Marcin M for bringing up this problem and Andre Lorbach for helping to reproduce and fix it.
  • added capability to specify substrings for field extraction mode
  • bugfix: disk queue was not persisted on shutdown, regression of fix to
    http://bugzilla.adiscon.com/show_bug.cgi?id=299
    The new code also handles the case of shutdown of blocking light and full delayable sources somewhat smarter and permits, assuming sufficient timouts, to persist message up to the max queue capacity. Also some nits in debug instrumentation have been fixed.

Changelog for 6.2.1 (v6-stable)

adisconteamChangelog, , , , , , , , , ,

Version 6.2.1 [v6-stable], 2012-05-10

  • change plugin config interface to be compatible with pre-v6.2 system
    The functionality was already removed (because it is superseeded by the
    v6.3+ config language), but code was still present. I have now removed
    those parts that affect interface. Full removal will happen in v6.3, in
    order to limit potential regressions. However, it was considered useful
    enough to do the interface change in v6-stable; this also eases merging
    branches!
  • re-licensed larger parts of the codebase under the Apache license 2.0
  • bugfix: omprog made rsyslog abort on startup if not binary to
    execute was configured
  • bugfix: imklog invalidly computed facility and severity
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=313
  • bugfix: stopped DA queue was never processed after a restart due to a
    regression from statistics module
  • bugfix: memory leak in array passing output module mode
  • bugfix: ommysql did not properly init/exit the mysql runtime library
    this could lead to segfaults. Triggering condition: multiple action
    instances using ommysql. Thanks to Tomas Heinrich for reporting this
    problem and providing an initial patch (which my solution is based on,
    I need to add more code to clean the mess up).
  • bugfix: rsyslog did not terminate when delayable inputs were blocked
    due to unvailable sources. Fixes:
    http://bugzilla.adiscon.com/show_bug.cgi?id=299
    Thanks to Marcin M for bringing up this problem and Andre Lorbach
    for helping to reproduce and fix it.
  • bugfix/tcpflood: sending small test files did not work correctly
Scroll to top