imudp

rsyslog 8.24.0 (v8-stable) released

We have released rsyslog 8.24.0.

This first release for 2017 brings a lot of changes. Most are detail enhancements for different modules. The biggest change is probably, that rsyslog now builds on the AIX platform.  Also 8.24.0 has two new message modification modules, which might be worth checking out. And a very useful addition is the ability to bind imudp and omfwd to a specific device.

For a complete list of changes, fixes and enhancements, please visit the ChangeLog.

ChangeLog:

rsyslog statistic counter plugin imudp

Plugin – imudp

This plugin maintains statistics for each listener and for each worker thread.

The listener statistic is named starting with “imudp”, followed followed by the listener IP, a colon and port in parenthesis. For example, the counter for a listener on port 514 (on all IPs) with no set name is called “imudp(*:514)”.

If an “inputname” is defined for a listener, that inputname is used instead of “imudp” as statistic name. For example, if the inputname is set to “myudpinut”, that corresponding statistic name in above case would be “myudpinput(*:514)”. This has been introduced in 7.5.3.

The following properties are maintained for each listener:

  • submitted – total number of messages submitted for processing since startup

The worker thread (in short: worker) statistic is named “imudp(wX)” where “X” is the worker thread ID, which is an monotonically increasing integer starting at 0. This means the first worker will have the name “imudp(w0”), the second “imudp(w1)” and so on. Note that workers are all equal. It doesn’t really matter which worker processes which messages, so the actual worker ID is not of much concern. More interesting is to check how the load is spread between the worker. Also note that there is no fixed worker-to-listener relationship: all workers process messages from all listeners.

Note: worker thread statistics are available starting with rsyslog 7.5.5.

The following properties are maintained for each worker thread:

  • called.recvmmsg – number of recvmmsg() OS calls done
  • called.recvmsg – number of recvmsg() OS calls done
  • msgs.received – number of actual messages received

Note: usually either one of “called.recvmmsg” or “called.recvmsg” is non-zero. This is because these are alternative kernel interfaces and the one to be used is selected based on kernel capabilities. However, there are two counters, as the system call type used has important performance implications, and so we thought this information needs to be exposed.

On systems supporting recvmmsg, the quotient msgs.received / caled.recvmmsg tells the average number of messages that could be pulled from the kernel buffers with a single syscall.

Back to statistics counter overview

rsyslog 7.5.5 (v7-devel) released

This release offers some important bug fixes and exciting new features. Most importantly, imudp can now use multiple input threads for even higher UDP input throughput. And imfile now longer has a hardcoded limit on the number of files that can be monitored. You can now monitor as many files as you like. Support for setting default timezones has been added to imptcp and imudp, but is still in experimental state (but feel free to use it if it suits you!). More detailed information is available in the changelog.

ChangeLog: http://www.rsyslog.com/changelog-for-7-5-5-v7-devel/

Download: http://www.rsyslog.com/rsyslog-7-5-5-v7-devel/

As always, feedback is appreciated.

Best regards, Florian Riedl

Changelog for 7.5.5 (v7-devel)

Version 7.5.5 [devel] 2013-10-16

  • imfile: permit to monitor an unlimited number of files
  • imptcp: add “defaultTZ” input parameter
  • imudp: support for multiple receiver threads added
  • imudp: add “dfltTZ” input config parameter
  • bugfix: memory leak in mmnormalize
  • bugfix: mmutf8fix did not properly handle invalid UTF-8 at END of message if the very last character sequence was too long, this was not detected
    Thanks to Risto Vaarandi for reporting this problem.
  • mmanon: removed the check for specific “terminator characters” after  last octet. As it turned out, this didn’t work in practice as there was an enormous set of potential terminator chars — so removing them was the best thing to do. Note that this may change behaviour of existing installations. Yet, we still consider this an important bugfix, that should be applied to the stable branch.
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=477
    Thanks to Muri Cicanor for initiating the discussion
  • now requires libestr 0.1.7 as early versions had a nasty bug in string comparisons
  • bugfix: mmanon did not detect all IP addresses in rewrite mode
    The problem occured if two IPs were close to each other and the first one was shrunk.
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=485
    Thanks to micah-at-riseup.net for reporting this bug
  • bugfix: mmanon sometimes used invalid replacement char in simple mode depending on configuration sequence, the replacement character was set to ‘s’ instead of the correct value. Most importantly, it was set to ‘s’ if simple mode was selected and no replacement char set.
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=484
    Thanks to micah-at-riseup.net for reporting this bug
  • bugfix: memory leak in mmnormalize
  • bugfix: array-based ==/!= comparisions lead to invalid results
    This was a regression introduced in 7.3.5 bei the PRI optimizer

rsyslog 7.5.3 (v7-devel) released

This release offers important new features like support for global and local variables, improvements in imfile multi-line handling and enhancements in the statistics subsystem. It also include bug fixes, including those imported from 7.4.4.

Note that the new imfile multi-line handling may cause a change of behaviour if you use it together with the input() config statement. For details, please see

http://blog.gerhards.net/2013/09/imfile-multi-line-messages.html

Users of the rsyslog development branch are strongly encouraged to upgrade to the new version.

ChangeLog:

http://www.rsyslog.com/changelog-for-7-5-3-v7-devel/

Download:

http://www.rsyslog.com/rsyslog-7-5-3-v7-devel/

As always, feedback is appreciated.

Best regards,
Florian Riedl

Changelog for 7.5.3 (v7-devel)

Version 7.5.3 [devel] 2013-09-11

  • imfile: support for escaping LF characters added embedded LF in syslog messages cause a lot of trouble. imfile now has the capability to escape them to “#012″ (just like the regular control character escape option). This requires new-style input statements to be used. If legacy configuration statements are used, LF escaping is always turned off to preserve compatibility.
    NOTE: if input() statements were already used, there is a CHANGE OF BEHAVIOUR: starting with this version, escaping is enabled by default. So if you do not want it, you need to add escapeLF=”off” to the input statement. Given the trouble LFs cause and the fact that the majority of installations still use legacy config, we considered this behaviour change acceptable and useful.
    see also: http://blog.gerhards.net/2013/09/imfile-multi-line-messages.html
  • add support for global and local variables
  • bugfix: queue file size was not correctly processed
    this could lead to using one queue file per message for sizes >2GiB
    Thanks to Tomas Heinrich for the patch.
  • add main_queue() configuration object to configure main message queue
  • bugfix: stream compression in imptcp caused timestamp to be corrupted
  • imudp: add ability to specify SO_RCVBUF size (rcvbufSize parameter)
  • imudp: use inputname for statistics, if configured
  • impstats: add process resource usage counters [via getrusage()]
  • impstats: add paramter “resetCounters” to report delta values possible for most, but not all, counters. See doc for details.
  • librelp 1.2.0 is now required
  • make use of new librelp generic error reporting facility
    This leads to more error messages being passed to the user and thus simplified troubleshooting.
  • bugfix: very small memory leak in imrelp
    more or less cosmetic, a single memory block was not freed, but this only happens immediately before termination (when the OS automatically frees all memory). Still an annoyance e.g. in valgrind.
  • fix compile problem in debug build
  • imported fixes from 7.4.4

Changelog for 7.4.2 (v7-stable)

Version 7.4.2 [v7.4-stable] 2013-07-04

  • bugfix: in RFC5425 TLS, multiple wildcards in auth could cause segfault
  • bugfix: RainerScript object required parameters were not properly checked – this clould result to segfaults on startup if parameters were missing.
  • bugfix: double-free in omelasticsearch closes: http://bugzilla.adiscon.com/show_bug.cgi?id=461 a security advisory for this bug is available at: http://www.lsexperts.de/advisories/lse-2013-07-03.txt PLEASE NOTE: This issue only existed if omelasticsearch was used in a non-default configuration, where the “errorfile” parameter was specified. Without that parameter set, the bug could not be triggered. Thanks to Markus Vervier and Marius Ionescu for providing a detailled bug report. Special thanks to Markus for coordinating his security advisory with us.
  • bugfix: omrelp potential segfault at startup on invalid config parameters
  • bugfix: small memory leak when $uptime property was used
  • bugfix: potential segfault on rsyslog termination in imudp closes: http://bugzilla.adiscon.com/show_bug.cgi?id=456
  • bugfix: lmsig_gt abort on invalid configuration parameters closes: http://bugzilla.adiscon.com/show_bug.cgi?id=448 Thanks to Risto Laanoja for the patch.
  • imtcp: fix typo in “listner” parameter, which is “listener” Currently, both names are accepted.
  • solved build problems on FreeBSD closes: http://bugzilla.adiscon.com/show_bug.cgi?id=457 closes: http://bugzilla.adiscon.com/show_bug.cgi?id=458 Thanks to Christiano for reproting and suggesting patches
  • solved build problems on CENTOS5

rsyslog 7.4.2 (v7-stable) released

This is a maintenance release, consisting primarily of bug fixes. It also provides a fix for a potential security issue in omelasticsearch. Please note that the security issue only exists in non-default configuration if the “errorfile” parameter was specified.

ChangeLog:

http://www.rsyslog.com/changelog-for-7-4-2-v7-stable/

Download:

http://www.rsyslog.com/rsyslog-7-4-2-v7-stable/

As always, feedback is appreciated.

Best regards,
Florian Riedl

rsyslog 7.3.2 (v7-devel) released

This release introduced a totally new rate-limiting system for all modules as well as much better “last message repeated n times” handling. For details, please see Rainer’s blog at

http://blog.gerhards.net/2012/10/rate-limiting-in-rsyslog-732.html

Besides this feature, the version also includes enhancements for omelasticsearch as well as input batching support for imudp (which should provide some performance boost on very busy systems). Also, action-like statements (call, stop, …) are now supported in action lists. Of course, there are also some bugfixes, many imported from v6 and v7.2.

ChangeLog:

http://www.rsyslog.com/changelog-for-7-3-2-v7-devel/

Download:

http://www.rsyslog.com/rsyslog-7-3-2-v7-devel/

As always, feedback is appreciated.

Best regards,
Tim Eifler

rsyslog statistics counter

Rsyslog supports statistic counters via the impstats module. It is important to know that impstats and friends only provides an infrastructure where core components and plugins can register statistics counter. This FAQ entry tries to describe all counters available, but please keep in mind that there may exist that we  do not know about.

When interpreting rsyslog statistics, please keep in mind that statistics records are processed as regular syslog messages. As such, the statistics messages themselves increment counters when they are emitted via the regular syslog stream, which is the default (and so counters keep slowly increasing even if there is absolutely no other traffic). Also keep in mind that a busy rsyslog system is very dynamic. Most importantly, this means that the counters may not be 100% consistent, but some slight differences may exist. Avoiding such inconsistencies would be possible only at the price of a very tight locking discipline, which would cause serious performance bottlenecks. Thus, this is not done. Finally, though extremely unlikely, some counters may experience an overflow and restart at 0 for that reasons. However, most counters are 64-bit, so this is extremely unlikely. Those which are not 64 bit are typically taken from some internal data structure that uses lower bits for performance reasons and guards against overflow.

The listing starts with the core component or plugin that creates the counters and than specifies various counters that exist for the sub-entities. The listing below is extended as new counters are added. Some counters probably do not exist in older releases of rsyslog.

Below you can find all available core components and plugins. Please note that every core component or plugin are linked to a information site.

Queue
Actions

PLUGINS

Scroll to top