rsyslog

High-performance log ingestion and ETL engine

rsyslog

High-performance log ingestion and ETL engine

Contribution Policy

Rainer GerhardsContributions

Rsyslog is very open to contributions and they are highly appreciated. Basically, we accept any contribution that can potentially be of use for the rsyslog community. Documentation and code contributions are especially welcome. A successful contribution should

  • come under the Apache Software License (ASL) 2.0; much of rsyslog’s code already has been converted to this license, but not everything yet. So please make sure you feel OK with ASL 2.0. We generally do not like GPL contributions any longer, as this causes trouble with the long-term goal of changing to ASL.
  • provide a bugfix or feature that is -at least potentially- of general interest. If it’s really just for you and so specific that nobody else will ever want it, chances are a bit lower we’ll merge (except, of course, if it is an interesting PoC).
  • a bit of documentation for new features or modules would be immensely useful. While we usually merge even without that, this actually means nobody will ever know about that particular new feature (we are way behind with doing our own doc, don’t expect us to document any other work ;)).
  • do not break upward compatibility; if you really need to, discuss this on the rsyslog mailing list first
  • do not unnecessarily duplicate a module; while we tend to accept such contributions, the process can take much longer as we will try to merge the functionality into the alread existing module
  • if you contribute a new module with some non-standard tools, do not expect that we actually try the module out. We’ll usually merge it, but will flag it as non-project-supported and name the original author as contact points. Omoracle is one example. Don’t be put off contributing such a module — they can be immensly useful to the community, we just can’t work on all of them. If things become popular, we may involve ourselfs into it (ommongodb is an example for that).
  • doc contributions should not be primarily promotional for some third party. If you contribute something you have originally written for another party, it’s usualy fine to include a “orignaly appeared on” backlink, but that should be sufficient.

As usual, these are just some general rules, and most of them common knowledge in any case. In simple words it’s very hard to find a reason NOT to merge a serious contribtion.

Where to find the rsyslog source code

Rainer Gerhardsfaq-devel

Rsyslog’s source is kept in git repositories. Git is a system designed to support flexible de-centralized collaboration. The rsyslog project currently provides two official repositories. These are equal peers, pick the one that you prefer to work with.

By popular request, rsyslog is available on github. We do not provide further instructions as github users already know how to work with that site (and if you don’t, be told that they have great tutorials).

We also keep a repository on Adiscon’s git server. This may also contain some arcane branches not of popular interest (like custom builds and such). You can clone this repository either via http or git protocol (with the later being much faster). URLS are:

  • git://git.adiscon.com/git/rsyslog.git
  • http://git.adiscon.com/git/rsyslog.git

There is also a browsable version (gitweb) available at http://git.adiscon.com/?p=rsyslog.git;a=summary. This version also offers snapshots of each commit for easy download. You can use these if you do not have git present on your system.

Changelog for 7.4.7 (v7-stable)

Adiscon SupportChangelog, , , , , , , ,

Version 7.4.7  [v7.4-stable] 2013-12-10

  • bugfix: limiting queue disk space did not work properly
    •   queue.maxdiskspace actually initializes queue.maxfilesize
    •   total size of queue files was not checked against queue.maxdiskspace for disk assisted queues.

    Thanks to Karol Jurak for the patch.

  • bugfix: linux kernel-like ratelimiter did not work properly with all inputs (for example, it did not work with imdup). The reason was that the PRI value was used, but that needed parsing of the message, which was done too late.
  • bugfix: disk queues created files in wrong working directory if the $WorkDirectory was changed multiple times, all queues only used the last value set.
  • bugfix: legacy directive $ActionQueueWorkerThreads was not honored
  • bugfix: segfault on startup when certain script constructs are used
    e.g. “if not $msg …”
  • bugfix: imuxsock: UseSysTimeStamp config parameter did not work correctly
    Thanks to Tomas Heinrich for alerting us and provinding a solution suggestion.
  • bugfix: $SystemLogUseSysTimeStamp/$SystemLogUsePIDFromSystem did not work
    Thanks to Tomas Heinrich for the patch.
  • improved checking of queue config parameters on startup
  • bugfix: call to ruleset with async queue did not use the queue
    closes: http://bugzilla.adiscon.com/show_bug.cgi?id=443
  • bugfix: if imtcp is loaded and no listeners are configured (which is uncommon), rsyslog crashes during shutdown.

rsyslog 8.1.3 (v8-devel) released

Adiscon SupportNews, Release Announcement, , , , , ,

We have just released 8.1.3 of the v8-devel branch.

This release contains further performance improvements. Note that both the output module interface and strgen interface have been changed and (some) modules may need to be modified. For the v8 supported rsyslog provided modules this has already be done (see v8 compatibility document for a list of supported modules). This release also detects failed actions more precisely. Finally, there are a couple of bug fixes in this release.

Note that this release can be considered a “normal” devel version, with moderate risk associated to it. Experience in the past weeks suggest so. Special thanks to Pavel Levshin for all his work and tests. Still note that there is a considerably higher risk running the devel version than the stable one.

ChangeLog:

http://www.rsyslog.com/changelog-for-8-1-3-v8-devel/

Download:

http://www.rsyslog.com/rsyslog-8-1-3-v8-devel/

Feedback is *very much* appreciated.

Best regards,
Florian Riedl

Changelog for 8.1.3 (v8-devel)

Adiscon SupportChangelog, , , , , , ,

Version 8.1.3 [devel] 2013-12-06

THIS VERSION CAN BE CONSIDERED A “NORMAL” DEVEL RELEASE. It’s no longer
highly experimental. This assertion is based on real-world feedback.

  • changes to the strgen module interface
  • new output module interface for transactional modules
  • performance improvements
    • reduced number of malloc/frees due to further changes to the output module interface
    • reduced number of malloc/frees during string template processing
      We now re-use once allocated string template memory for as long as the worker thread exists. This saves us from doing new memory allocs (and their free counterpart) when the next message is processed. The drawback is that the cache always is the size of the so-far largest message processed. This is not considered a problem, as in any case a single messages’ memory footprint should be far lower than that of a whole set of messages (especially on busy servers).
    • used variable qualifiers (const, __restrict__) to hopefully help the compiler generate somewhat faster code
  • failed action detection more precisely for a number of actions
    If an action  uses string parameter passing but is non-transactional it can be executed immediately, giving a quicker indicatio of action failure.
  • bugfix: limiting queue disk space did not work properly
    • queue.maxdiskspace actually initializes queue.maxfilesize
    • total size of queue files was not checked against queue.maxdiskspace for disk assisted queues.

    Thanks to Karol Jurak for the patch.

rsyslog statistic counter Queues

adisconteamsub-faq, , , ,

Queue

For each queue inside the system its own set of statistics counters is created. If there are multiple action (or main) queues, this can become a rather lengthy list. The stats record begins with the queue name (e.g. “main Q” for the main queue; ruleset queues have the name of the ruleset they are associated to, action queues the name of the action).

  • size – currently active messages in queue
  • enqueued – total number of messages enqueued into this queue since startup
  • maxsize – maximum number of active messages the queue ever held
  • full – number of times the queue was actually full and could not accept additional messages
  • discarded.full – number of messages discarded because the queue was full
  • discarded.nf – number of messages discarded because the queue was nearly full. Starting at this point, messages of lower-than-configured severity are discarded to save space for higher severity ones.

Back to statistics counter overview

rsyslog statistic counter plugin imuxsocks

adisconteamfaq-windows, , , ,

Plugin – imuxsock

This plugin maintains a global statistics with the following properties:

  • submitted – total number of messages submitted for processing since startup
  • ratelimit.discarded – number of messages discarded due to rate limiting
  • ratelimit.numratelimiters – number of currently active rate limiters (small data structures used for the rate limiting logic)

Back to statistics counter overview

Scroll to top