rsyslog 7.6.5 (v7-stable) released
This released provides an important regression fix, which rendered 7.6.4 unusable, as selector line evaluation was incorrect. Users of 7.6.4 are highly advised to upgrade to this version.
http://www.rsyslog.com/changelog-for-7-6-5-v7-stable/
Download:
http://www.rsyslog.com/downloads/download-v7-stable/
As always, feedback is appreciated.
Best regards,
Florian Riedl
rsyslog 7.6.4 (v7-stable) released
This is a new release for the v7-stable branch. It contains a lot of bug fixes and patches. Several issues have been fixed, thus ensuring better stability and reliability. This is a recommended update for all v7 users.
http://www.rsyslog.com/changelog-for-7-6-4-v7-stable/
Download:
http://www.rsyslog.com/downloads/download-v7-stable/
As always, feedback is appreciated.
Best regards,
Florian Riedl
Changelog for 7.6.4 (v7-stable)
Version 7.6.4 [v7.6-stable] 2014-09-12
- add –enable-generate-man-pages configure switch (default: enabled)
This forces generation of man pages, even if cached ones exists. This “fixes” a typical release tarball nit. While it is hackish, the benefit is clear given the history of failed tarball releases since we changed the cached man page handling. It was just too easy to get that wrong. - removed obsolete –disable-fsstnd configure option
Thanks to Thomas D. for alerting us.
Closes: https://github.com/rsyslog/rsyslog/issues/72 - permits to build against json-c 0.12
Unfortunately, json-c had an ABI breakage, so this is necessary. Note that versions prior to 0.12 had security issues (CVE-2013-6370, CVE-2013-6371) and so it is desirable to link against the new version.
Thanks to Thomas D. for the patch. Note that at least some distros have fixed the security issue in older versions of json-c, so this seems to apply mostly when building from sources. - new omfile default module parameters
- filecreatemode
- fileowner
- fileownernum
- filegroup
- filegroupnum
- dirowner
- dirownernum
- dirgroup
- dirgroupnum
Thanks to Karol Jurak for the patch.
- bugfix: memory leak in TCP TLS mode
- bugfix: imfile: if a state file for a different file name was set, that different file (name) was monitored instead of the configured one. Now, the state file is deleted and the correct file monitored.
closes: https://github.com/rsyslog/rsyslog/issues/103 - bugfix: using UUID property could cause segfault
- bugfix: mmutf8fix did not detect two invalid sequences
Thanks to Axel Rau for the patch. - bugfix: file descriptor leak with Guardtime signatures
When a .gtstate file is opened it is never closed. This is especially bad when dynafiles frequently get evicted from dynafile cache and be re-opened again. - bugfix: busy loop in tcp listener when running out of file descriptors
Thanks to Susant Sahani for the patch. - bugfix: mishandling of input modules not supporting new input instances
If they did not support this, accidently the output module part of the module union was written, leading to unpredictable results. Note: all core modules do support this interface, but some contributed or very old ones do not. - bugfix: double-free when ruleset() parser parameters were used
While unlikely, this could cause stability issues even after the config phase. - bugfix: output modules with parameters with multiple passing modes could caused strange behaviour including aborts
This was due to the fact that the action module only preserved and processed the last set passing mode. Note that this was not a problem for the plugins provided by the rsyslog git: none of them uses different passing modes.
Thanks to Tomas Heinrich for providing a very detailled bug report. - various fixes after coverty scan
These do not address issues seen in practice but those seen by the tool. Some of them may affect practical deployments.
Thanks to Tomas Heinrich for the patches. - bugfix imuxsock: “Last message repeated…” was not emitted at shutdown
The “Last message repeated…” notice didn’t get printed if rsyslog was shut down before the repetition was broken.
Thanks to Tomas Heinrich for the patch. - bugfix: make dist failed when GUARDTIME or LIBGCRYPT feature was disabled
- bugfix: mmjsonparse did not build with json-c < 0.10
This was a regression introduced some time in the past in order to support API changes in json-c. Now we check for the version and use proper code. - bugfix: mmanon did not properly anonymize IP addresses starting with ‘9’
Thanks to defa-at-so36.net for reporting this problem.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=529
rsyslog v8 improvements and how to write plugins in any language
In the first part, we will explain the new RSYSLOG v8 engine, its motivation and its benefits. Learn, for example, why writing to Elasticsearch is much faster with the new engine. We will describe the tuning parameters vital for making best use of the new features.
In the second part we will explain how to write RSYSLOG plugins in any language. Traditionally, writing rsyslog plugins has been considered quite hard, with at least C knowledge necessary. In v8, we have introduced new interfaces which make it possible to write plugins in any language – be it Python, Perl or Java. Even bash will do. In essence, this is a great tool for any admin to add special needs with just a bit of scripting. We will proivde concrete instructions on how to write a plugin, point to read-to-copy samples and tell how to integrate this into rsyslog.
NOTE: This is Rainers LinuxTag Berlin 2014 talk.
New 8.4 stable is ready
A new rsyslog v8-stable has been released. It is not just the next iteration of 8.2, instead it will be a new feature release based on the latest 8.3 devel. So please welcome 8.4.
Frequent followers may wonder why 8.4 is ready. Originally, we planned to release it after the summer break. The reason is simple: its ready to come up, albeit with a little less functionality than originally anticipated. Since we were primarily doing maintenance and bug fixing on v8-devel the past couple of weeks, just as it normally happens before a new stable branch comes up. So the code has matured and we decided it was ready to be released as stable. We released 8.4.0 a week ago, and it inherits all the enhancements and fixes of rsyslog 8.3. We hope rsyslog 8.4 fulfils your expectations and provides a flawless logging experience.
rsyslog 8.4.0 (v8-stable) released
We have just released 8.4.0 of the v8-stable branch.
This release introduces the new stable version that inherits all the enhancements and improvements of rsyslog 8.3.
Additionaly, the separated documentation is available as a tarball download on the download page.
http://www.rsyslog.com/changelog-for-8-4-0-v8-stable/
Download:
http://www.rsyslog.com/downloads/download-v8-stable/
As always, feedback is appreciated.
Best regards,
Florian Riedl
rsyslog 8.3.5 (v8-devel) released
We have just released 8.3.5 of the v8-devel branch.
This is basically a maintenance release. It adds improved error output to omprog and a couple of patches were imported from the latest v7-stable (7.6.4).
ChangeLog:
http://www.rsyslog.com/changelog-for-8-3-5-v8-devel/
Download:
http://www.rsyslog.com/download-v8-devel/
As always, feedback is appreciated.
Best regards,
Florian Riedl
Changelog for 8.3.5 (v8-devel)
Version 8.3.5 [v8-devel] 2014-08-05omprog:
- emit error message via syslog() if loading binary fails
This happens after forking, so omprog has no longer access to rsyslog’s regular error reporting functions. Previously, this meant any error message was lost. Now it is emitted via regular syslog (which may end up in a different instance, if multiple instances run…) - couple of patches imported from v7-stable (7.6.4)
Masking data in logs and RSYSLOG
As a mobile payments company, we at SumUp are obligated to follow many industry regulations, one of them being PCI DSS. Restricting access to credit card numbers is a clear need and this implies ensuring they are not part of the logs which are used for various purposes and have bigger audience, not restricted to the authorized list of employees who have access to sensitive data.
PAN or primary account number is used by card issuers as card number which is unique and brings information about the issuer and also in majority of the cases can be validated with Luhn algorithm. This is the number on you credit or debit card and it should be kept secret by us.
One widely used approach is to have quality assurance of the logs all over the development and deployment cycle. This is a needed and valuable attitude however first it takes a lot of human resources and second it is kind of reactive approach in terms of dealing with production systems. So we want something better, something mandatory which can leave us on the safe side if we got human error somewhere in the chain. This is very important in our case where we need to put logging management system out of PCI scope. From four ways which are offered by PCI DSS, Requirement 3.4:
3.4 Render PAN unreadable anywhere it is stored (including on portable digital media, backup media, and in logs)
by using any of the following approaches:
- One-way hashes based on strong cryptography, (hash must be of the entire PAN).
- Truncation (hashing cannot be used to replace the truncated segment of PAN).
- Index tokens and pads (pads must be securely stored).
- Strong cryptography with associated key-management processes and procedures.
our natural choice for log messages is truncation. We want to truncate PAN data if it’s present in the logs for some reason in example in situation when temporary the log level is increased for investigation. While we have centralized log storage which is in PCI scope we want to transfer the logs in real-time in some external location, accessible for developers and BI where they can find and use the information they need.
Since we are using rsyslog for logging daemon our next step was to get in touch with Adiscon – the company behind this brilliant piece of software. They were very interested when I explained the idea and the work started. A little bit later we got new message modification module called mmexternal. It sends the message to some external binary and expects an input. More on the implementation here.
Let me give you an example with a code snippet from rsyslog config and an example of python script which is doing a regular expression to catch and replace i.e. VISA, MasterCard and AMEX cards. You may find a lot of useful regular expressions here:
rsyslog.conf
module(load=”mmexternal”)
action(type=”mmexternal”
binary=”/usr/local/bin/external_python_cards_replace.py”
interface.input=”msg” )
external_python_cards_replace.py
Please note that the above snippets are only examples. With using regular expressions you are going to have many false positives but in general this won’t be an issue. Also note that you can modify completely different parts of the logs and also you are not limited to any language or technique for doing so.
With the following example we have negligible resource consumption on the server where log modification is done. Synthetic test which not claim for accuracy shows around 5% CPU usage on single core 2.5GHz virtual CPU for 100 messages/s.
This is how we are doing it. All comments and suggestions are welcome!
rsyslog 8.3.4 (v8-devel) released
We have just released 8.3.4 of the v8-devel branch.
This is a somewhat unexpected new 8.3 devel version: thanks to some sponsor, we could work on providing better parsers for Cisco IOS devices as well as some improvements to the general date parser. As we want to integrate this into 8.4, we have decided to release another 8.3 version. Please note that 8.4 stable is still immanent. This version also includes a number of bug fixes.
ChangeLog:
http://www.rsyslog.com/changelog-for-8-3-4-v8-devel/
Download:
http://www.rsyslog.com/download-v8-devel/
As always, feedback is appreciated.
Best regards,
Florian Riedl
