rsyslog

The rocket-fast system for log processing

rsyslog

Changelog for 6.1.3 (v6-devel)

By adisconteamPosted on Posted in ChangelogTagged , , ,

Version 6.1.3 [DEVEL] (rgerhards), 2011-02-01

  • experimental support for monogodb added
  • added $IMUDPSchedulingPolicy and $IMUDPSchedulingPriority config settings
  • added $LocalHostName config directive
  • improved tcpsrv performance by enabling multiple-entry epoll
    so far, we always pulled a single event from the epoll interface.
    Now 128, what should result in performance improvement (less API
    calls) on busy systems. Most importantly affects imtcp.

  • imptcp now supports non-cancel termination mode, a plus in stability
  • imptcp speedup: multiple worker threads can now be used to read data
  • new directive $InputIMPTcpHelperThreads added
  • bugfix: fixed build problems on some platforms
    namely those that have 32bit atomic operations but not 64 bit ones

  • bugfix: local hostname was pulled too-early, so that some config
    directives (namely FQDN settings) did not have any effect

  • enhanced tcpflood to support multiple sender threads
    this is required for some high-throughput scenarios (and necessary to
    run some performance tests, because otherwise the sender is too slow).

  • added some new custom parsers (snare, aix, some Cisco “specialities”)
    thanks to David Lang

Changelog for 6.1.2 (devel)

By Adiscon SupportPosted on Posted in ChangelogTagged , , , ,

Version 6.1.2  [DEVEL] (rgerhards), 2010-12-16

  • added experimental support for log normalizaton (via liblognorm) support for normalizing log messages has been added in the form of mmnormalize. The core engine (property replacer, filter engine) has been enhanced to support properties from normalized events.

Note: this is EXPERIMENTAL code. It is currently known that there are issues if the functionality is used with

  • disk-based queues
  • asynchronous action queues

You can not use the new functionality together with these features.

This limitation will be removed in later releases. However, we preferred to release early, so that one can experiment with the new feature set and accepted the price that this means the full set of functionality is not yet available. If not used together with these features, log normalizing should be pretty stable.

  • enhanced testing tool tcpflood now supports sending via UDP and the capability to run multiple iterations and generate statistics data records
  • bugfix: potential abort when output modules with different parameter passing modes were used in configured output modules

normalizer – first steps for mmnormalize

By adisconteamPosted on Posted in FAQTagged , , , , , ,

Since rsyslog 6.1.2 there is a new module called mmnormalize. This module provides the capability to normalize log messages via liblognorm. (This example has been done on a Fedora 13 workstation. Please note that there can be some differences in the file paths.)

For using this module you need

libestr
libee
liblognorm
rsyslog 6.1.2

After downloading the mentioned items we have to install them.
You can find a guide for installing liblestr, libee and liblognorm at
http://www.liblognorm.com/help/first-steps-using-liblognorm/.

The next step would be to extract the downloaded rsyslog folder and install it. Please note that we habe to mention the mmnormalize module in the configure
(after extracting:)

$ ./configure --libdir=/usr/lib --sbindir=/sbin --enable-mmnormalize
$ make
$ make install

Now we have to load the mmnormalize module in the rsyslog.conf.

$ModLoad mmnormalize # loads the mmnormalize module
$mmnormalizeRuleBase rulebase.rb # specifies which rulebase is to use
*.* :mmnormalize: # applies normalization to all messages

Finally we need an action that uses the normalizer. Below you will find a simple sample for an action using normalizer

$template cee,"msg is '%msg%', %$!all-json%\n"
*.* /home/test/logfile;cee

This example will write everything to logfile and shows you the original message and what parts have been parsed and which not.

A further sample how to use mmnormalize to filter for IP in messages

$template cee, "ip is '%$!ip%', %$!all-json%\n"
if $!ip != '' then /home/test/logfile-cee;cee

With this example you can extract messages which have an IP in the ‘field’ ip. The available fields are applying to your rulebase, there they are specified. Fields are used in a template like properties but they use the additional signs ‘$!’ in their names, e.g. %$!name% instead of %name%.  Below you will find a link which tells you more about creating a rule base http://www.liblognorm.com/help/creating-a-rulebase/.

Scroll to top