rsyslog

The rocket-fast system for log processing

rsyslog 8.1.0 (v8-devel) released

By Adiscon SupportPosted on Posted in News, Release AnnouncementTagged , , , , ,

This is the initial and EXPERIMENTAL release of rsyslog v8. This release contains a rewritten core engine and is expected to perform much better in high performance environments with complex configurations (namely log normalization, Elasticsearch, Databases and remote forwarding).

We release this version in the hopes that interested parties have a look at it and provide feedback (including bug reports). However, it is NOT suitable for production use — for the time being please use rsyslog v7.5 for this.

More details can be found at:

http://blog.gerhards.net/2013/11/on-to-lucky-number.html

Please be also sure to review the compatibility document:

http://www.rsyslog.com/doc/v8compatibility.html

Feedback is *very much* appreciated.

 

ChangeLog:

http://www.rsyslog.com/changelog-for-8-1-0-v8-devel/

Download:

http://www.rsyslog.com/rsyslog-8-1-0-v8-devel/

 

Best regards,
Florian Riedl

Changelog for 8.1.0 (v8-devel)

By Adiscon SupportPosted on Posted in ChangelogTagged , , , , , ,

Version 8.1.0 [devel] 2013-11-15

  • rewritten core engine for higher performance and new features In detail:
    • completely rewritten rule execution engine
    • completely changed output module interface
    • remodelled output module interface
    • enabled important output modules to support full concurrent operation

    The core engine has been considerably changed and must be considered experimental at this stage. Note that it does not yet include all features planned for v8, but is close to this goal. In theory, the engine should perform much better, especially on complex configurations and busy servers. Most importantly, actions instances can now be called concurrently from worker threads and many important output modules support multiple concurrent action instances natively.

  • module omruleset is no longer enabled by default.
    Note that it has been deprecated in v7 and been replaced by the “call” statement. Also, it can still be build without problems, the option must just explicitely be given.

 

action.execOnlyWhenPreviousIsSuspended preciseness

By Rainer GerhardsPosted on Posted in UncategorizedTagged ,

The “action.execOnlyWhenPreviousIsSuspended” parameter permits to conditionally execute an action only if the previous one is suspended. When an action actually gets suspended depends on the output module implementing the action, but the general rule is that an action suspends itself if there is a problem doing it’s job (e.g. a target server is down). This setting is generally used to implement failover destinations. So let’s call actions which have this attribute set “failover actions”.

Note that many action implement transactions to improve performance. If so, usually the success or error state of the action is only available after the transaction has been comitted. At this point, the messages have already been submitted to the action and the action could not set suspension state — simply because at that point everything looked fine. So when the failover action was about to be executed, the suspended flag was not set and the action not executed. Depending on system configuration, this can lead to loss of some messages during the failover process. If this seems totally unacceptable to you, you need to make sure that the transaction is committed after each message — this permits the output to obtain the actual success state (note that with some technologies, even this is not totally possible – a prime example is plain TCP syslog protocol).

To ensure commit of each message, the queue’s maximum batch size needs to be reduced to one. This can be done via the “queue.dequeuebatchsize” parameter. Note that reducing the batch size can have severe performance impact. Unfortunately, it is impossible to to have both high performance and precise failover actions. The simple reason is that for precise failover actions we need to know the result of the commit operation, which makes it necessary to do immediate commits.

Lower Bound for Queue Sizes

By Rainer GerhardsPosted on Posted in FAQTagged ,

The queue.size parameter permits to specify the maximum queue size in number of messages. While not technically enforced, there is a lower limit on this parameter. Setting it to very low values (roughly below 100 messages) is not supported and can lead to unpredictable results. Also, future version my automatically adjust to a safe lower bound and/or decide to fail queue startup in those cases. So if you use very low values, do so at your own risk.

Changelog for 7.4.6 (v7-stable)

By adisconteamPosted on Posted in ChangelogTagged , , , , ,

Version 7.4.6 [v7.4-stable] 2013-10-31

  • bugfix: potential abort during HUP
    This could happen when one of imklog, imzmq3, imkmsg, impstats,
    imjournal, or imuxsock were under heavy load during a HUP.
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=489
    Thanks to Guy Rozendorn for reporting the problem and Peval Levhshin for
    his analysis.
  • bugfix: imtcp flowControl parameter incorrectly defaulted to “off”
    This could cause message loss on systems under heavy load and was
    a change-of-behaviour to previous version. This is a regression
    most probably introduced in 5.9.0 (but did not try hard to find the
    exact point of its introduction).
  • now requires libestr 0.1.9 as earlier versions lead to problems with
    number handling in RainerScript
  • bugfix: memory leak in strlen() RainerScript function
    Thanks to Gregoire Seux for reportig this bug.
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=486
  • bugfix: buffer overrun if re_extract function was called for submatch 50
    Thanks to Pavel Levshin for reporting the problem and its location.
  • bugfix: memleak in re_extract() function
    Thanks to Pavel Levshin for reporting this problem.
  • bugfix: potential abort in RainerScript optimizer
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=488
    Thanks to Thomas Doll for reporting the problem and Pavel Levshin for
    fixing it.
  • bugfix: memory leak in omhiredis
    Thanks to Pavel Levshin for the fix
  • bugfix: segfault if variable was assigned to non-container subtree
    Thanks to Pavel Levshin for the fix

 

Changelog for 7.5.6 (v7-devel)

By adisconteamPosted on Posted in ChangelogTagged , , , , ,

Version 7.5.6 [devel] 2013-10-29

  • improved performance of RainerScript variable access
    by refactoring the whole body of variable handling code. This also
    solves some of the anomalies experienced in some versions of rsyslog.
    All variable types are now handled in unified code, including
    access via templates.
  • RainerScript: make use of 64 bit for numbers where available
    Thanks to Pavel Levshin for enhancement.
  • slight performance optimization if GCC is used
    We give branch prediction hints for the frequent RETiRet macro which is
    used for error handling. Some slight performance gain is to be expected
    from that.
  • removed global variable support
    The original idea was not well thought out and global variables, as
    implemented, worked far different from what anybody would expect. As
    such, we consider the current approach as an experiment that did not
    work out and opt to removing it, clearing the way for a better future
    solution. Note: global vars were introduced in 7.5.3 on Sept, 11th 2013.
  • new module mmsequence, primarily used for action load balancing
    Thanks to Pavel Levshin for contributing this module.
  • bugfix: unset statement always worked on message var, even if local
    var was given
  • bugfix: segfault if variable was assigned to non-container subtree
    Thanks to Pavel Levshin for the fix
  • bugfix: imuxsock did not suport addtl sockets if syssock was disabled
    Thanks to Pavel Levshin for the fix
  • bugfix: running imupd on multiple threads lead to segfault if recvmmsg
    is available
  • bugfix: segfault if re_extract() function was used and no match found
  • bugfix: omelasticsearch did not compile on platforms without atomic
    instructions
  • bugfix: potential misadressing on startup if property-filter was used
    This could happen if the property name was longer than 127 chars, a case
    that would not happen in practice.
  • bugfix: invalid property filter was not properly disabled in ruleset
    Note: the cosmetic memory leak introduced with that patch in 7.4.5 is
    now also fixed.
Scroll to top