rsyslog

The rocket-fast system for log processing pipelines

rsyslog 5.9.0 (devel) released

By adisconteamPosted on Posted in News, Release AnnouncementTagged ,

This version starts a new development branch for version v5 which brings some new, not-so-intrusive features. This release includes enhancements for imfile, support for TCP KEEP ALIVE and a way to name actions (primarily for impstats).

Note that the main development activity is still focused at version 6.
So this release is only for those interested in the minor enhancements to v5.

ChangeLog:

http://www.rsyslog.com/changelog-for-5-9-0-v5-devel/

Download:

http://www.rsyslog.com/rsyslog-5-9-0-devel/

As always, feedback is appreciated.

Best regards,
Tom Bergfeld

Changelog for 5.9.0 (v5-devel)

By adisconteamPosted on Posted in ChangelogTagged , , , ,

Version 5.9.0 [V5-DEVEL] (rgerhards), 2011-06-08

  • imfile: added $InputFileMaxLinesAtOnce directive
  • enhanced imfile to support input batching
  • added capability for imtcp and imptcp to activate keep-alive packets
    at the socket layer. This has not been added to imttcp, as the latter is
    only an experimental module, and one which did not prove to be useful.
    reference: http://kb.monitorware.com/post20791.html

  • added support to control KEEPALIVE settings in imptcp
    this has not yet been added to imtcp, but could be done on request.

  • $ActionName is now also used for naming of queues in impstats
    as well as in the debug output

  • bugfix: do not open files with full privileges, if privs will be dropped
    This make the privilege drop code more bulletproof, but breaks Ubuntu’s
    work-around for log files created by external programs with the wrong
    user and/or group. Note that it was long said that this “functionality”
    would break once we go for serious privilege drop code, so hopefully
    nobody still depends on it (and, if so, they lost…).

  • bugfix: pipes not opened in full priv mode when privs are to be dropped
  • this begins a new devel branch for v5
  • better handling of queue i/o errors in disk queues. This is kind of a
    bugfix, but a very intrusive one, this it goes into the devel version
    first. Right now, “file not found” is handled and leads to the new
    emergency mode, in which disk action is stopped and the queue run
    in direct mode. An error message is emited if this happens.

  • added support for user-level PRI provided via systemd
  • added new config directive $InputTCPFlowControl to select if tcp
    received messages shall be flagged as light delayable or not.

  • enhanced omhdfs to support batching mode. This permits to increase
    performance, as we now call the HDFS API with much larger message
    sizes and far more infrequently

  • bugfix: failover did not work correctly if repeated msg reduction was on
    affected directive was: $ActionExecOnlyWhenPreviousIsSuspended on
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=236

rsyslog 6.3.1 (devel) released

By Adiscon SupportPosted on Posted in News, Release AnnouncementTagged , , , ,

This is a first implementation of a full-blown DNS cache. While there were some optimizations for DNS queries in older releases, especially UDP sometimes suffered under slow DNS resolution performance.

This is solved with the new dnscache module. Note that the module will undergo some more enhancements in the next couple of weeks. Feedback on its effect would be deeply appreciated.

ChangeLog:

http://www.rsyslog.com/changelog-for-6-3-1-devel/

Download:

http://www.rsyslog.com/rsyslog-6-3-1-devel/

As always, feedback is appreciated.

Best regards,
Florian Riedl

Log normalization and the leading space

By Adiscon SupportPosted on Posted in FAQTagged , , ,

Log normalization is simple, but has its quirks. A common pitfall is syslog message format as induced by RFC3164. Let’s look at a common case:  A log message has been sent to rsyslog. The message itself had no irregular characters. But, the message that should have been parsed by mmnormalize now has a leading space character. Basically, the message that should be parsed looks like this:

This is a test

Usually, one would think, that a simple parser can be used here. You might be correct, but there is a small caveat about this. The rulebase entry we currently have looks something like this:

rule=:%word1:word% %word2:word% %word3:word% %word4%

But strangely, rsyslog responds the following:

mmnormalize generated: {“originalmsg”: ” This is a test″, “unparsed-data”: ” This is a test″}

How comes, that rsyslog cannot parse the message? Why is there a leading space character in from of the message? The answer is, that messages are processed as RFC3164. In this RFC it is defined, that everything after the “:” of the syslog header is to be considered as the message. Thus, the message has a leading space now.

How is this to be solved? Simply insert the space to your rules in the rulebase. This will lead to a rule like this:

rule=: %word1:word% %word2:word% %word3:word% %word4%

Please note, that there has just the space character been added. Further, this is really only a example. The rule will fit to all messages that are 4 words long, so it is really not very suitable to be adopted to your configuration.

rsyslog 6.3.0 (devel) released

By Adiscon SupportPosted on Posted in News, Release AnnouncementTagged , , , , ,

The release of rsyslog 6.3.0 introduces a new major feature. With this release we introduce a new config system. You can find some information about the new config system in Rainer’s blog:

http://blog.gerhards.net/2011/06/new-rsyslog-config-system-materializes.html

ChangeLog:

http://www.rsyslog.com/changelog-for-6-3-0-devel/

Download:

http://www.rsyslog.com/rsyslog-6-3-0-devel/

As always, feedback is appreciated.

Best regards,
Florian Riedl

Scroll to top