rsyslog

The rocket-fast system for log processing

impstats

Changelog for 7.5.3 (v7-devel)

By Adiscon SupportPosted on Posted in ChangelogTagged , , , , , , , ,

Version 7.5.3 [devel] 2013-09-11

  • imfile: support for escaping LF characters added embedded LF in syslog messages cause a lot of trouble. imfile now has the capability to escape them to “#012″ (just like the regular control character escape option). This requires new-style input statements to be used. If legacy configuration statements are used, LF escaping is always turned off to preserve compatibility.
    NOTE: if input() statements were already used, there is a CHANGE OF BEHAVIOUR: starting with this version, escaping is enabled by default. So if you do not want it, you need to add escapeLF=”off” to the input statement. Given the trouble LFs cause and the fact that the majority of installations still use legacy config, we considered this behaviour change acceptable and useful.
    see also: http://blog.gerhards.net/2013/09/imfile-multi-line-messages.html
  • add support for global and local variables
  • bugfix: queue file size was not correctly processed
    this could lead to using one queue file per message for sizes >2GiB
    Thanks to Tomas Heinrich for the patch.
  • add main_queue() configuration object to configure main message queue
  • bugfix: stream compression in imptcp caused timestamp to be corrupted
  • imudp: add ability to specify SO_RCVBUF size (rcvbufSize parameter)
  • imudp: use inputname for statistics, if configured
  • impstats: add process resource usage counters [via getrusage()]
  • impstats: add paramter “resetCounters” to report delta values possible for most, but not all, counters. See doc for details.
  • librelp 1.2.0 is now required
  • make use of new librelp generic error reporting facility
    This leads to more error messages being passed to the user and thus simplified troubleshooting.
  • bugfix: very small memory leak in imrelp
    more or less cosmetic, a single memory block was not freed, but this only happens immediately before termination (when the OS automatically frees all memory). Still an annoyance e.g. in valgrind.
  • fix compile problem in debug build
  • imported fixes from 7.4.4

rsyslog 6.3.7 (v6-devel) released

By Adiscon SupportPosted on Posted in News, Release AnnouncementTagged , , , , , , ,

With this release, all builtin actions support the new v6 config format. Also, the release contains much enhanced statistics counters and various bug fixes. Recommended for all users of the v6-devel branch.

ChangeLog:

http://www.rsyslog.com/changelog-for-6-3-7-v6-devel/

Download:

http://www.rsyslog.com/rsyslog-6-3-7-v6-devel/

As always, feedback is appreciated.

Best regards,
Florian Riedl

rsyslog statistics counter

By Rainer GerhardsPosted on Posted in FAQTagged , , , , , , , , , , , , , , ,

Rsyslog supports statistic counters via the impstats module. It is important to know that impstats and friends only provides an infrastructure where core components and plugins can register statistics counter. This FAQ entry tries to describe all counters available, but please keep in mind that there may exist that we  do not know about.

When interpreting rsyslog statistics, please keep in mind that statistics records are processed as regular syslog messages. As such, the statistics messages themselves increment counters when they are emitted via the regular syslog stream, which is the default (and so counters keep slowly increasing even if there is absolutely no other traffic). Also keep in mind that a busy rsyslog system is very dynamic. Most importantly, this means that the counters may not be 100% consistent, but some slight differences may exist. Avoiding such inconsistencies would be possible only at the price of a very tight locking discipline, which would cause serious performance bottlenecks. Thus, this is not done. Finally, though extremely unlikely, some counters may experience an overflow and restart at 0 for that reasons. However, most counters are 64-bit, so this is extremely unlikely. Those which are not 64 bit are typically taken from some internal data structure that uses lower bits for performance reasons and guards against overflow.

The listing starts with the core component or plugin that creates the counters and than specifies various counters that exist for the sub-entities. The listing below is extended as new counters are added. Some counters probably do not exist in older releases of rsyslog.

Below you can find all available core components and plugins. Please note that every core component or plugin are linked to a information site.

Queue
Actions

PLUGINS

Periodic statistics on rsyslog counters

By adisconteamPosted on Posted in FAQTagged , , ,

Since rsyslog 5.7.0 (V5-Devel) there is a new module called impstats.

This module provides periodic output of rsyslog internal counters. Note that the whole statistics system is currently under development. So availabilty and format of counters may change and is not yet stable (so be prepared to change your trending scripts when you upgrade to a newer rsyslog version).

The set of available counters will be output as a set of syslog messages. This output is periodic, with the interval being configurable (default is 5 minutes). Be sure that your configuration records the counter messages (default is syslog.info).

Note that loading this module has impact on rsyslog performance. Depending on settings, this impact may be severe (for high-load environments).

Please find below some simple steps how to use that module.

Of course you have to name it additionally in the configure. The configure should look like this (please note that the parameters can be different to your configure, the important thing is ––enable-impstats; this example is for Fedora 13 )

./configure --enable-impstats --sbindir=/sbin --libdir=/lib

The next steps are make and make install

make
make install

Furthermore we have to add that module in the rsyslog config. Below you will find a short excerpt of a sample config.

#### Modules ####
$ModLoad imuxsock
$ModLoad imklog
#$ModLoad immark

$ModLoad impstats
$PStatsInterval 600
$PStatsSeverity 7

syslog.debug  /var/log/rsyslog-stats

$ModLoad impstats – tells rsyslog to load the module impstats

$PStatsInterval 600 – is a configuration directive of impstats

$PStatsSeverity 7 – is also a configuration directive of impstats

Here you will find all information about the impstats module and its configuration directives.

If we have a look at the mentioned output file rsyslog-stats we can see the results of the static module.

The content of that file should look like this

Sep 17 11:43:49 localhost rsyslogd-pstats: umxsock: submitted=16
Sep 17 11:43:49 localhost rsyslogd-pstats: main Q: size=1 enqueued=2403 full=0 maxqsize=2

At the actual point all objects are shown in the results. Every Object has its own counter like

umxsock: (= object) submitted=16 (=counter; 16 logs received by the object umxsock). The main queue of rsyslog is also shown -> main Q with the parameters size (messages in the queue), enqueued (all received messages), full (how often was the queue full) and maxqsize (the maximal amount of messages in the queue).

Please be sure that the features and options of that module will be develeloped soon.

All information about the periodic statistics module you will find in the documentation. There is also a list of rsyslog impstats counters available on the web site.

Scroll to top