Changelog for 8.1.0 (v8-devel)
Version 8.1.0 [devel] 2013-11-15
- rewritten core engine for higher performance and new features In detail:
- completely rewritten rule execution engine
- completely changed output module interface
- remodelled output module interface
- enabled important output modules to support full concurrent operation
The core engine has been considerably changed and must be considered experimental at this stage. Note that it does not yet include all features planned for v8, but is close to this goal. In theory, the engine should perform much better, especially on complex configurations and busy servers. Most importantly, actions instances can now be called concurrently from worker threads and many important output modules support multiple concurrent action instances natively.
- module omruleset is no longer enabled by default.
Note that it has been deprecated in v7 and been replaced by the “call” statement. Also, it can still be build without problems, the option must just explicitely be given.
rsyslog 8.1.0 (v8-devel)
Download file name: rsyslog 8.1.0 (devel)
rsyslog 8.1.0 (devel)
sha256 hash: 09699b52fbabb2a515f47f036950ec
Author: Rainer Gerhards (rgerhards@adiscon.com)
Version: 8.1.0 File size: 2.601 MB
rsyslog 7.5.5 (v7-devel) released
ChangeLog: http://www.rsyslog.com/changelog-for-7-5-5-v7-devel/
Download: http://www.rsyslog.com/rsyslog-7-5-5-v7-devel/
As always, feedback is appreciated.
Best regards, Florian Riedl
Changelog for 7.5.5 (v7-devel)
Version 7.5.5 [devel] 2013-10-16
- imfile: permit to monitor an unlimited number of files
- imptcp: add “defaultTZ” input parameter
- imudp: support for multiple receiver threads added
- imudp: add “dfltTZ” input config parameter
- bugfix: memory leak in mmnormalize
- bugfix: mmutf8fix did not properly handle invalid UTF-8 at END of message if the very last character sequence was too long, this was not detected
Thanks to Risto Vaarandi for reporting this problem. - mmanon: removed the check for specific “terminator characters” after last octet. As it turned out, this didn’t work in practice as there was an enormous set of potential terminator chars — so removing them was the best thing to do. Note that this may change behaviour of existing installations. Yet, we still consider this an important bugfix, that should be applied to the stable branch.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=477
Thanks to Muri Cicanor for initiating the discussion - now requires libestr 0.1.7 as early versions had a nasty bug in string comparisons
- bugfix: mmanon did not detect all IP addresses in rewrite mode
The problem occured if two IPs were close to each other and the first one was shrunk.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=485
Thanks to micah-at-riseup.net for reporting this bug - bugfix: mmanon sometimes used invalid replacement char in simple mode depending on configuration sequence, the replacement character was set to ‘s’ instead of the correct value. Most importantly, it was set to ‘s’ if simple mode was selected and no replacement char set.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=484
Thanks to micah-at-riseup.net for reporting this bug - bugfix: memory leak in mmnormalize
- bugfix: array-based ==/!= comparisions lead to invalid results
This was a regression introduced in 7.3.5 bei the PRI optimizer
impstats delayed or lost? – cause and cure
Some users report that they do not receive all impstats log records or that these log records are delayed. The common thing about these questions is that those user tend to have very large main message queues.
By default, impstats is run in-band, which means that it’s messages are submitted to the main message queue just like any other messages are. So if the main queue takes very long to processs, impstats messages get delayed as well. They may be discarded, too, based on queue settings and queue full status. In this scenarios, impstats obviously has problems reporting what is going on.
A simple solution is to run impstats off-band. This is done by simply instructing it to directly write to file. In that mode, the rsyslog engine is not used at all, and output will always be available and happen right on the interval. There is a drawback, though: as the rsyslog core engine is not involved, things like dynafiles, different templates or forwarding to another host are not possible.
Note that impstats can submit messages both to a file and the regular message stream. This may be an interesting alternative if the main queue causes trouble but usually logs shall be gathered at some central place.
Example for writing to a local file:
module(load="impstats" interval="600" severity="7" log.file="/var/log/impstats")
rsyslog 7.5.4 (v7-devel) released
This release offers some interesting features. It provides a new module called mmpstrucdata to parse RFC5424 structured data into json message properties. Also the default queue.size values have been altered to more suitable values. Omfwd and omfile received new parameters and we changed a bigger portion of the documentation to improve usability by linking relevant web ressources to quickly find additional information. Finally, there have been a few other changes and bugfixes.
More detailed information is available in the changelog.
ChangeLog:
http://www.rsyslog.com/changelog-for-7-5-4-v7-devel/
Download:
http://www.rsyslog.com/rsyslog-7-5-4-v7-devel/
As always, feedback is appreciated.
Best regards,
Florian Riedl
Changelog for 7.5.4 (v7-devel)
Version 7.5.4 [devel] 2013-10-07
- mmpstrucdata: new module to parse RFC5424 structured data into json message properties
- change main/ruleset queue defaults to be more enterprise-like
new defaults are queue.size 100,000 max workers 2, worker activation after 40,000 msgs are queued, batch size 256. These settings are much more useful for enterprises and will not hurt low-end systems that much. This is part of our re-focus on enterprise needs. - omfwd: new action parameter “maxErrorMessages” added
- omfile: new module parameters to set action defaults added
* dirCreateMode
* fileCreateMode - mmutf8fix: new module to fix invalid UTF-8 sequences
- imuxsock: handle unlimited number of additional listen sockets
- doc: improve usability by linking to relevant web ressources
The idea is to enable users to quickly find additional information, samples, HOWTOs and the like on the main site. At the same time, (very) slightly remove memory footprint when few listeners are monitored. - bugfix: omfwd parameter streamdrivermmode was not properly handled
It was always overwritten by whatever value was set via the legacy directive $ActionSendStreamDriverMode - imtcp: add streamdriver.name module parameter
permits overriding the system default stream driver (gtls, ptcp) - bugfix: build system: libgcrypt.h needed even if libgrcypt was disabled
Thanks to Jonny Törnbom for reporting this problem - imported bugfixes from 7.4.4
rsyslog 7.5.4 (v7-devel)
Download file name: rsyslog 7.5.4 (devel)
rsyslog 7.5.4 (devel)
sha256 hash: 9e307bfcfe294de435de6acda58feb
Author: Rainer Gerhards (rgerhards@adiscon.com)
Version: 7.5.4 File size: 2.991 MB
How to add a HMAC to RFC5424 structured data messages
rsyslog features a new message modification module, that will check for the SD ID in RFC5424 messages and append a HMAC hash to the structured data part of the message. Please note, that even if the module works on all messages, only RFC5424 messages will be processed.
Before starting, you need a private Enterprise Number from IANA so you can use the module to add the HMAC hash to the message.
For the functionality you need the module “mmrfc5424addhmac”. This is currently available only in the git master branch and will be first released in the next devel release 7.5.4 and for stable in 7.6.0.
When doing the configure, please do not forget to enable this module:
./configure --prefix=/usr --enable-imtcp --enable-mmrfc5424addhmac
Now for the configuration:
module(load="imtcp") module(load="mmrfc5424addhmac")
input(type="imtcp" port="514")
action(type="mmrfc5424addhmac" key="yourenterprisekey" hashFunction="sha256" sd_id="id@32473")
template(name="addhmac" type="string" string="<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n")
action(type="omfile" file="/var/log/logfile" template="addhmac")
This is a relatively simple configuration. We load the modules imtcp and mmrfc5424addhmac. We will receive all RFC5424 messages through our tcp input. The really interesting part are the actions. Since we need to modify the messages, we need an action with the message modification module. As parameters for the action we define a key that will be used to create the hash, choose a hash function (basically all hash functions from openssl work) and define our SD ID which consists of a name, “@” and the ID received from IANA.
The message will be parsed for the ID, if it exists, a hash will be generated and appended to the structured data of the message.
Now we need to do something with these messages. The template above is a RFC5424 representation and gives out accordingly formatted messages. As follow-up action we will write all messages into a file. Alternatively, you could also forward them to another host or write them into a database.
rsyslog 7.5.3 (v7-devel) released
Note that the new imfile multi-line handling may cause a change of behaviour if you use it together with the input() config statement. For details, please see
http://blog.gerhards.net/2013/
Users of the rsyslog development branch are strongly encouraged to upgrade to the new version.
ChangeLog:
http://www.rsyslog.com/changelog-for-7-5-3-v7-devel/
Download:
http://www.rsyslog.com/rsyslog-7-5-3-v7-devel/
As always, feedback is appreciated.
Best regards,
Florian Riedl