Rsyslog Windows Agent 2.0 Released
Adiscon is proud to announce the 2.0 release of RSyslog Windows Agent.
This new major release adds full Continue reading “Rsyslog Windows Agent 2.0 Released”
How can I check the config?
We have often seen the case, that someone has rsyslog running and makes changes to the configuration. And usually, after making the changes, rsyslog gets restarted, but the changed config is invalid. rsyslog has a function to check the configuration for validity. This can be done very easily by invoking this command:
rsyslogd -N1
(Note that rsyslogd may not be in your search path – then it usually is found in /sbin/rsyslogd)
This tells rsyslog to do a config check. It does NOT run in regular mode, but just check configuration file correctness. This option is meant to verify a config file. To do so, run rsyslogd interactively in foreground, specifying -f <config-file> and -N level. The level argument modifies behaviour. Currently, 0 is the same as not specifying the -N option at all (so this makes limited sense) and 1 actually activates the code.
This configuration check will only check the configuration for integrity like syntax. Additionaly, the modules will be loaded to make sure that they work properly. On the downside, since the engine will not be loaded, errors with permissions or alike cannot be checked. These will occur only when running rsyslog normally.
The verdict for this option is, that it is quite useful for a first check if the changes were correct, without running the configuration in live mode. This might help to prevent that rsyslog gets restarted with a basically wrong configuration and thus rendering rsyslog useless, because it might not work or not work properly.
Modern syslogd Design (rsyslog)
In this (~5 minutes) Video Rainer Gerhards describes the core elements of a modern syslogd (like rsyslog) and how they enable the capability to evolve logging without loosing backwards compatibility. Continue reading “Modern syslogd Design (rsyslog)”
rsyslog 7.3.6 (v7-devel)
Download file name: rsyslog 7.3.6 (devel)
rsyslog 7.3.6 (devel)
sha256 hash: 1dd904459de0aaacf83d7d164caf419ba41fa6936a193e03ba57410b844d422b
Author: Rainer Gerhards (rgerhards@adiscon.com)
Version: 7.3.6 File size: 2.680 MB
rsyslog 7.3.6 (v7-devel) released
The 7.3.6 release contains a number of performance enhancements as well as some functional enhancements and bug fixes.
ChangeLog:
http://www.rsyslog.com/changelog-for-7-3-6-v7-devel/
Download:
http://www.rsyslog.com/rsyslog-7-3-6-v7-devel/
As always, feedback is appreciated.
Best regards,
Tim Eifler
Changelog for 7.3.6 (v7-devel)
Version 7.3.6 [devel] 2013-01-28
- greatly improved speed of large-array [N]EQ RainerScript comparisons
Thanks to David Lang for a related discussion that inspired the idea
to do this with a much simpler (yet sufficient) approach than orignally
planned for. - greatly improved speed of DNS cache for large cache sizes
- general performance improvements
- omfile: added stats counters for dynafile caches
- omfile: improved async writing, finally enabled full async write
also fixed a couple of smaller issues along that way - impstats: added ability to write stats records to local file
and avoid going through the syslog log stream. syslog logging can now
also be turned off (see doc for details). - bugfix: imklog issued wrong facility in error messages
…what could lead to problems in other parts of the code - fix compile problem in imklog
- added capability to output thread-id-to-function debug info
This is a useful debug aid, but nothing of concern for regular users.
How to use the Ubuntu repository
The Adiscon Ubuntu Repository has been setup to provide support for the latest rsyslog versions on Ubuntu 12 including support for necessary third party packages. Please note that the Ubuntu Repository is open for testing at the moment, and contains only the latest version for v7-devel. The new packages are based in the original and latest Ubuntu 12 rsyslog packages, so in most cases an simple sudo apt-get update && sudo apt-get upgrade will be enough to update rsyslog. Please note that these packages are currently experimental. Use at your own risk.
To use our repository, follow these steps:
Install our PGP Key into your apt system
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com AEF0CF8E
Edit your /etc/apt/sources.list and add these lines to the end
# Adiscon repository
deb http://ubuntu.adiscon.com/v7-devel precise/ deb-src http://ubuntu.adiscon.com/v7-devel precise/
Once done perform these commands to update your apt cache and install the latest rsyslog version
sudo apt-get update && sudo apt-get upgrade
If you receive a message like this while upgrading follow these steps below:
The following packages have been kept back:
rsyslog
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.sudo apt-get install rsyslog
We highly appriciate any feedback or bug reports.
The newest information will be available on the Ubuntu Repository project page.
Filter optimization with arrays
If you are using a lot of filters and templates in rsyslog, this can not only be affecting the performance drastically, but it is also a hassle to set up all the different actions and templates. It is always worthy to check, if there isn’t a shortcut somewhere, which might not only save you time for creating the configuration, but also make it much simpler in the end to keep track of all the actions.
In our example, we have several programnames. The log messages should be sorted by programname and then be stored in a specific file and be sorted by host. After storing the log messages, the message should be discarded, so it won’t be processed by the following filters, thus saving otherwise wasted processing time. This example is applicable to rsyslog v7.2.x and above.
Here are some sample config lines.
## apache_access
template(name="DailyPerHost_apache_access" type="string" string="/syslog/%FROMHOST%/apache_access.log")
:programname, isequal, "apache_access" { action(type="omfile" DynaFile="DailyPerHost_apache_access") stop }
## apache_error
template(name="DailyPerHost_apache_error" type="string" string="/syslog/%FROMHOST%/apache_error.log")
:programname, isequal, "apache_error" { action(type="omfile" DynaFile="DailyPerHost_apache_error") stop }## mysql
template(name="DailyPerHost_mysql" type="string" string="/syslog/%FROMHOST%/mysql.log")
:programname, isequal, "mysql" { action(type="omfile" DynaFile="DailyPerHost_mysql") stop }
## php
template(name="DailyPerHost_php" type="string" string="/syslog/%FROMHOST%/php.log")
:programname, isequal, "php" { action(type="omfile" DynaFile="DailyPerHost_php") stop }These are some basic services, which are often run together. Please note, that these are just a few examples. As you can see here, the template is created first. It is given a name, type and format. Templates of type string are usually used for file names. Here the log messages get stored in the folder /syslog a subfolder for the host where the message occured and then a filename which reflects the type of message that occured.
The second line holds the actions. First you see the property based filter (programname) and the condition. After that the actions get chained with the curly braces. The log messages where the filter evaluates to true get stored in a file. The filename and path is generated dynamically with the DynaFile parameter. Through this, the above written template will be used to generate the path and filename. The second action is represented by stop. Please note that this is case sensitive. Basically, stop means to stop the message processing. No further processing of the message will take place.
If we look closely at the sample config lines, we see, that the filter condition is basically always the same. It will always filter the programname property for a certain value. This is a predestinated case for using an array for simplification. We can use the property programname in the file template as well and filter an array of values. This will greatly save the overhead for all the seperate filter, not only in the configuration, but also in processing the messages.
template(name="DailyPerHost_app" type="string" string="/syslog/%FROMHOST%/%programname%.log")
if $programname == ["apache_access",
"apache_error",
"mysql",
"php"]
then {
action(type="omfile" DynaFile="DailyPerHost_app")
stop
}Again, we first create the template. Please note the difference in the filename where the hardcoded text has been replaced by the property programname. In the next lines, we see the filter and the array of values. This is just to reflect the example. Virtually, the array can have near-infinite values. The filter is also a common if/then construct. After the then we see our chain of commands. First the action which writes the log messages into a file where the filename is created by the above template and then a stop as second action.
This case is applicable in many forms. It is also most useful if you are filtering and the discarding a lot of messages with very common filter settings. You could use it to filter for an array of property values and even chain comparison operations.
rsyslog 7.2.5 (v7-stable) released
We have just released 7.2.5 of the v7 stable branch. This is a pure bug-fixing release. More information on the changes can be found in the ChangeLog.
ChangeLog: http://www.rsyslog.com/changelog-for-7-2-5-v7-stable/
Download: http://www.rsyslog.com/rsyslog-7-2-5-v7-stable/
As always, feedback is appreciated.
Best regards,
Florian Riedl
Changelog for 7.2.5 (v7-stable)
Version 7.2.5 [v7-stable] 2013-01-08
- build system cleanup (thanks to Michael Biebl for this!)
- bugfix: omelasticsearch did not properly compile on some platforms due to missing libmath. Thanks to Michael Biebl for the fix
- bugfix: invalid DST handling under Solaris
Thanks to Scott Severtson for the patch. - bugfix: on termination, actions were incorrectly called
The problem was that incomplete fiter evaluation was done *during the shutdown phase*. This affected only the LAST batches being processed. No problem existed during the regular run. Could usually only happen on very busy systems, which were still busy during shutdown. - bugfix: very large memory consumption (and probably out of memory) when FromPos was specified in template, but ToPos not.
Thanks to Radu Gheorghe for alerting us of this bug. - bugfix: timeval2syslogTime cause problems on some platforms due to invalid assumption on structure data types.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=394
Thanks to David Hill for the patch [under ASL2.0 as per email conversation 2013-01-03]. - bugfix: compile errors in im3195
Thanks to Martin Körper for the patch - bugfix: doGetFileCreateMode() had invalid validity check ;)
Thanks to Chandler Latour for the patch. - bugfix: mmjsonparse errornously returned action error when no CEE cookie was present.
