rsyslog 7.5.7 (v7-devel) released

This is primarily a bug-fixing release, but offers some improvements in worker thread handling (thanks to Pavel Levshin!) as well as usability improvements when working changing queue sizes.

More detailed information is available in the changelog.

ChangeLog:

http://www.rsyslog.com/changelog-for-7-5-7-v7-devel/

Download:

http://www.rsyslog.com/rsyslog-7-5-7-v7-devel/

As always, feedback is appreciated.

Best regards,
Tim Eifler

rsyslog 8.1.1 (v8-devel) released

This is the first bugfixing release for v8. It enhances overall stability and also re-enables some output modules. It is
highly suggested that v8 users upgrade to this release. Please note that it is still experimental code.

 

ChangeLog:

http://www.rsyslog.com/changelog-for-8-1-1-v8-devel/

Download:

http://www.rsyslog.com/rsyslog-8-1-1-v8-devel/

Feedback is *very much* appreciated.

Best regards,
Florian Riedl

Changelog for 8.1.1 (v8-devel)

Version 8.1.1 [devel] 2013-11-19

  • bugfix: STOP/discard(~) was mostly NOT honored
    This lead to execution of config code that was not meant to be executed.
  • bugfix: memory leak on worker thread termination
  • bugfix: potential segfault in omfile under heavy load
    Thanks to Pavel Levshin for alerting us.
  • bugfix: mmsequence: instance mode did not work
    Thanks to Pavel Levshin for the patch
  • bugfix: segfault on startup when certain script constructs are used
    e.g. “if not $msg …”
  • omhiredis: now supports v8 output module interface and works again
    Thanks to Pavel Levshin for the patch
  • mmaudit: now supports v8 output module interface and work again
  • bugfix: potential abort on startup in debug mode
    This depends on template type being used. The root cause was a non-necessary debug output, which were at the wrong spot (leftover from  initial testing).
    Thanks to Pavel Levshin for alerting us and providing a patch proposal.

How to use impstats

This summary was contributed by David Caplinger through the mailing list.

First, enable the module with something like:

module(load="impstats" interval="660" severity="7")

This will start generating logs tagged with “rsyslogd-pstats” every 600 seconds. If you like, you can use that tag to filter them into their own file:

if $syslogtag contains 'rsyslogd-pstats' then { 
     action(type="omfile" queue.type="linkedlist" queue.discardmark="980" 
            name="pstats" file="/var/log/pstats") 
     stop 
}

You’ll wind up with several log lines at each interval, all showing current counters (since rsyslog restart). So to determine inter-interval deltas, you’d have to import these into a spreadsheet. (Newer rsyslog can emit just the deltas in the log lines, but that’s in v7.5.x I believe.)

For example, if you want to filter based on some property (such as source IP address) and send the matching logs to both a local file and on to a remote destination, you might use something like:

if $fromhost-ip ==
     [ "1.1.1.1", 
       "2.2.2.2" ] 
then {
     action (type="omfwd" queue.type="linkedlist" queue.discardmark="980" 
             action.resumeretrycount="-1" name="NET.forward" target="10.10.10.10" 
             port="514" protocol="tcp")
     action (type="omfile" queue.type="linkedlist" queue.discardmark="980" 
             name="NET.local" file="/var/log/messages")
     stop
}

Which is a log flow like:

source -> imudp -> main Q -> NET.local (to local files) & NET.forward (to remote)

Here’s an example of a batch of pstats output (re-ordered slightly) from the above config:

Nov 13 14:31:35 loghost rsyslogd-pstats: imudp(*:514): submitted=23035
Nov 13 14:31:35 loghost rsyslogd-pstats: main Q: size=15 enqueued=89624087 full=0 discarded.full=0 discarded.nf=0 maxqsize=444
Nov 13 14:31:35 loghost rsyslogd-pstats: NET.local: size=0 enqueued=11541 full=0 discarded.full=0 discarded.nf=0 maxqsize=7
Nov 13 14:31:35 loghost rsyslogd-pstats: NET.local: processed=11541 failed=0
Nov 13 14:31:35 loghost rsyslogd-pstats: NET.forward: size=0 enqueued=11541 full=0 discarded.full=0 discarded.nf=0 maxqsize=7
Nov 13 14:31:35 loghost rsyslogd-pstats: NET.forward: processed=11541 failed=0
Nov 13 14:31:35 loghost rsyslogd-pstats: pstats: size=0 enqueued=65508 full=0 discarded.full=0 discarded.nf=0 maxqsize=25
Nov 13 14:31:35 loghost rsyslogd-pstats: pstats: processed=65500 failed=0

In this case we have:

1) A UDP input (imudp)

This logs message counts “submitted” to rsyslog via UDP port 514.

2) A main queue (main Q)

This shows messages entering the queue (enqueued), as well as any dropped messages (discarded.full=0, discarded.nf=0). It also shows how many times the queue has become completely full (full=0) and it keeps a running total of the maximum size the queue has ever hit (maxqsize=444). (All these counters are since rsyslog startup.)

3) Two output/action queues (NET.local, NET.forward)

These logs queue stats like above, as well as successfully “processed” (via omfile and omfwd in this case), indicating successful delivery to their final destination (local file or remote TCP receiver, in this case).

4) Another queue to handle pstats output itself (as I described above)

This example doesn’t happen to include DA-mode, which adds another pstats log line for the DA portion of the associated action queue.

If you don’t give your action queues names, you’ll wind up with pstats logs referring to things like “action 2”, and have a hard time figuring out what is going on.

A well-behaved queue will have zero discarded.full and discarded.nf, and a low maxqsize, meaning that everything entering the queue is leaving promptly. In a backlog situation, you’ll see size and maxqsize for an action/output queue increase over time, until maxqsize hits your configured queue.size parameter. Then the main Q will start increasing in size (and maxqsize) until it approaches and exceeds full. Then the discarded.nf and discarded.full counters will start climbing.

rsyslog 8.1.0 (v8-devel) released

This is the initial and EXPERIMENTAL release of rsyslog v8. This release contains a rewritten core engine and is expected to perform much better in high performance environments with complex configurations (namely log normalization, Elasticsearch, Databases and remote forwarding).

We release this version in the hopes that interested parties have a look at it and provide feedback (including bug reports). However, it is NOT suitable for production use — for the time being please use rsyslog v7.5 for this.

More details can be found at:

http://blog.gerhards.net/2013/11/on-to-lucky-number.html

Please be also sure to review the compatibility document:

http://www.rsyslog.com/doc/v8compatibility.html

Feedback is *very much* appreciated.

 

ChangeLog:

http://www.rsyslog.com/changelog-for-8-1-0-v8-devel/

Download:

http://www.rsyslog.com/rsyslog-8-1-0-v8-devel/

 

Best regards,
Florian Riedl

Changelog for 8.1.0 (v8-devel)

Version 8.1.0 [devel] 2013-11-15

  • rewritten core engine for higher performance and new features In detail:
    • completely rewritten rule execution engine
    • completely changed output module interface
    • remodelled output module interface
    • enabled important output modules to support full concurrent operation

    The core engine has been considerably changed and must be considered experimental at this stage. Note that it does not yet include all features planned for v8, but is close to this goal. In theory, the engine should perform much better, especially on complex configurations and busy servers. Most importantly, actions instances can now be called concurrently from worker threads and many important output modules support multiple concurrent action instances natively.

  • module omruleset is no longer enabled by default.
    Note that it has been deprecated in v7 and been replaced by the “call” statement. Also, it can still be build without problems, the option must just explicitely be given.

 

Scroll to top