How to use the Ubuntu repository
The Adiscon Ubuntu Repository has been setup to provide support for the latest rsyslog versions on Ubuntu 12 including support for necessary third party packages. Please note that the Ubuntu Repository is open for testing at the moment, and contains only the latest version for v7-devel. The new packages are based in the original and latest Ubuntu 12 rsyslog packages, so in most cases an simple sudo apt-get update && sudo apt-get upgrade will be enough to update rsyslog. Please note that these packages are currently experimental. Use at your own risk.
To use our repository, follow these steps:
Install our PGP Key into your apt system
sudo apt-key adv --recv-keys --keyserver keyserver.ubuntu.com AEF0CF8E
Edit your /etc/apt/sources.list and add these lines to the end
# Adiscon repository
deb http://ubuntu.adiscon.com/v7-devel precise/ deb-src http://ubuntu.adiscon.com/v7-devel precise/
Once done perform these commands to update your apt cache and install the latest rsyslog version
sudo apt-get update && sudo apt-get upgrade
If you receive a message like this while upgrading follow these steps below:
The following packages have been kept back:
rsyslog
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.sudo apt-get install rsyslog
We highly appriciate any feedback or bug reports.
The newest information will be available on the Ubuntu Repository project page.
Filter optimization with arrays
If you are using a lot of filters and templates in rsyslog, this can not only be affecting the performance drastically, but it is also a hassle to set up all the different actions and templates. It is always worthy to check, if there isn’t a shortcut somewhere, which might not only save you time for creating the configuration, but also make it much simpler in the end to keep track of all the actions.
In our example, we have several programnames. The log messages should be sorted by programname and then be stored in a specific file and be sorted by host. After storing the log messages, the message should be discarded, so it won’t be processed by the following filters, thus saving otherwise wasted processing time. This example is applicable to rsyslog v7.2.x and above.
Here are some sample config lines.
## apache_access template(name="DailyPerHost_apache_access" type="string" string="/syslog/%FROMHOST%/apache_access.log") :programname, isequal, "apache_access" { action(type="omfile" DynaFile="DailyPerHost_apache_access") stop } ## apache_error template(name="DailyPerHost_apache_error" type="string" string="/syslog/%FROMHOST%/apache_error.log") :programname, isequal, "apache_error" { action(type="omfile" DynaFile="DailyPerHost_apache_error") stop }
## mysql template(name="DailyPerHost_mysql" type="string" string="/syslog/%FROMHOST%/mysql.log") :programname, isequal, "mysql" { action(type="omfile" DynaFile="DailyPerHost_mysql") stop } ## php template(name="DailyPerHost_php" type="string" string="/syslog/%FROMHOST%/php.log") :programname, isequal, "php" { action(type="omfile" DynaFile="DailyPerHost_php") stop }
These are some basic services, which are often run together. Please note, that these are just a few examples. As you can see here, the template is created first. It is given a name, type and format. Templates of type string are usually used for file names. Here the log messages get stored in the folder /syslog a subfolder for the host where the message occured and then a filename which reflects the type of message that occured.
The second line holds the actions. First you see the property based filter (programname) and the condition. After that the actions get chained with the curly braces. The log messages where the filter evaluates to true get stored in a file. The filename and path is generated dynamically with the DynaFile parameter. Through this, the above written template will be used to generate the path and filename. The second action is represented by stop. Please note that this is case sensitive. Basically, stop means to stop the message processing. No further processing of the message will take place.
If we look closely at the sample config lines, we see, that the filter condition is basically always the same. It will always filter the programname property for a certain value. This is a predestinated case for using an array for simplification. We can use the property programname in the file template as well and filter an array of values. This will greatly save the overhead for all the seperate filter, not only in the configuration, but also in processing the messages.
template(name="DailyPerHost_app" type="string" string="/syslog/%FROMHOST%/%programname%.log")
if $programname == ["apache_access", "apache_error", "mysql", "php"] then { action(type="omfile" DynaFile="DailyPerHost_app") stop }
Again, we first create the template. Please note the difference in the filename where the hardcoded text has been replaced by the property programname. In the next lines, we see the filter and the array of values. This is just to reflect the example. Virtually, the array can have near-infinite values. The filter is also a common if/then construct. After the then we see our chain of commands. First the action which writes the log messages into a file where the filename is created by the above template and then a stop as second action.
This case is applicable in many forms. It is also most useful if you are filtering and the discarding a lot of messages with very common filter settings. You could use it to filter for an array of property values and even chain comparison operations.
rsyslog 7.2.5 (v7-stable) released
We have just released 7.2.5 of the v7 stable branch. This is a pure bug-fixing release. More information on the changes can be found in the ChangeLog.
ChangeLog: http://www.rsyslog.com/changelog-for-7-2-5-v7-stable/
Download: http://www.rsyslog.com/rsyslog-7-2-5-v7-stable/
As always, feedback is appreciated.
Best regards,
Florian Riedl
Changelog for 7.2.5 (v7-stable)
Version 7.2.5 [v7-stable] 2013-01-08
- build system cleanup (thanks to Michael Biebl for this!)
- bugfix: omelasticsearch did not properly compile on some platforms due to missing libmath. Thanks to Michael Biebl for the fix
- bugfix: invalid DST handling under Solaris
Thanks to Scott Severtson for the patch. - bugfix: on termination, actions were incorrectly called
The problem was that incomplete fiter evaluation was done *during the shutdown phase*. This affected only the LAST batches being processed. No problem existed during the regular run. Could usually only happen on very busy systems, which were still busy during shutdown. - bugfix: very large memory consumption (and probably out of memory) when FromPos was specified in template, but ToPos not.
Thanks to Radu Gheorghe for alerting us of this bug. - bugfix: timeval2syslogTime cause problems on some platforms due to invalid assumption on structure data types.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=394
Thanks to David Hill for the patch [under ASL2.0 as per email conversation 2013-01-03]. - bugfix: compile errors in im3195
Thanks to Martin Körper for the patch - bugfix: doGetFileCreateMode() had invalid validity check ;)
Thanks to Chandler Latour for the patch. - bugfix: mmjsonparse errornously returned action error when no CEE cookie was present.
Changelog for 7.3.5 (v7-devel)
Version 7.3.5 [devel] 2012-12-19
- ommysql: addded batching/transaction support
- enhanced script optimizer to optimize common PRI-based comparisons
These constructs are especially used in SUSE default config files,
but also by many users (as they are more readable than the equivalent
PRI-based filter). - omudpspoof: add support for new config system
- omudpspoof: add support for packets larger than 1472 bytes
On Ethernet, they need to be transmitted in multiple fragments. While
it is known that fragmentation can cause issues, it is the best choice
to be made in that case. Also improved debug output. - bugfix: omudpspoof failed depending on the execution environment
The v7 engine closes fds, and closed some of libnet’s fds as well, what
lead to problems (unfortunately, at least some libnet versions do not
report a proper error state but still “success”…). The order of libnet
calls has been adjusted to by in sync with what the core engine does. - bugfix: segfault on imuxsock startup if system log socket is used
and no ratelimiting supported. Happens only during initial config
read phase, once this is over, everything works stable. - bugfix: mmnormalize build problems
- bugfix: mmnormalize could abort rsyslog if config parameter was in error
- bugfix: no error message for invalid string template parameters
rather a malformed template was generated, and error information emitted
at runtime. However, this could be quite confusing. Note that with this
“bugfix” user experience changes: formerly, rsyslog and the affected
actions properly started up, but the actions did not produce proper
data. Now, there are startup error messages and the actions are NOT
executed (due to missing template due to template error). - bugfix[minor]: invalid error code when mmnormalize could not access rulebase
- bugfix(kind of): script optimizer did not work for complex boolean expressions
- doc bugfix: corrections and improvements in mmnormalize html doc page
- bugfix: some message properties could be garbled due to race condition
This happened only on very high volume systems, if the same message was
being processed by two different actions. This was a regression caused
by the new config processor, which did no longer properly enable msg
locking in multithreaded cases. The bugfix is actually a refactoring of
the msg locking code – we no longer do unlocked operations, as the use
case for it has mostly gone away. It is potentially possible only at
very low-end systems, and there the small additional overhead of doing
the locking does not really hurt. Instead, the removal of that
capability can actually slightly improve performance in common cases,
as the code path is smaller and requires slightly less memory writes.
That probably outperforms the extra locking overhead (which in the
low-end case always happens in user space, without need for kernel
support as we can always directly aquire the lock – there is no
contention at all). - build system cleanup (thanks to Michael Biebl for this!)
- bugfix: omelasticsearch did not properly compile on some platforms
due to missing libmath. Thanks to Michael Biebl for the fix.
rsyslog 7.3.5 (v7-devel) released
Version 7.3.5 is an update to the development branch. It contains new features: now, ommysql supports batching, which results in overall better system performance. Also, omudpspoof has been enhanced to support fragmentation for large messages. Omudpspoof now also supports the native v7 config format. Lastly, the script optimizer has been enhanced and detects and considerably optimizes some frequently-used constructs (like if $syslogfacility-text == “kernel” and such). Of course, there are also a couple of bug fixes, the most important one addresses a potential segfault that can happen in imuxock.
All users of the 7.3 branch are advised to update to this version.
ChangeLog:
http://www.rsyslog.com/changelog-for-7-3-5-v7-devel/
Download:
http://www.rsyslog.com/rsyslog-7-3-5-v7-devel/
As always, feedback is appreciated.
Best regards,
Tim Eifler
rsyslog 7.3.5 (v7-devel)
Download file name: rsyslog cialis 7.3.5 (devel)
rsyslog 7.3.5 (devel) generic cialis online
sha256 hash: 8135fa1dbcda4b5026879eb0e39556fa90cf4b3af28a80d3400e2d707571f43a
Author: Rainer Gerhards (rgerhards@adiscon.com)
Version: 7.3.5 File size: 2.607 MB
rsyslog 7.2.4 (v7-stable)
Download file name: rsyslog 7.2.4 (stable)
rsyslog 7.2.4 (stable)
sha256 hash: 90af55f696db3209a9ef28d8e8052c48917479dd994838f5ef678b1f701c4540
Author: Rainer Gerhards (rgerhards@adiscon.com)
Version: 7.2.4 File size: 2.577 MB
rsyslog 7.2.4 (v7-stable) released
We have just released 7.2.4 of the v7 stable branch. This is a pure bug-fixing release. More information on the changes can be found in the ChangeLog.
ChangeLog:
http://www.rsyslog.com/changelog-for-7-2-4-v7-stable/
Download:
http://www.rsyslog.com/rsyslog-7-2-4-v7-stable/
As always, feedback is appreciated.
Best regards,
Tim Eifler
What is the difference between timereported and timegenerated?
Each message that is received by rsyslog is usually available with two timestamps. They can be accessed by using the properties “timereported” and “timegenerated”.
“timegenerated” is always the time when rsyslog generated the message object on the local machine. That actually means it is the time when the message was received (either via the oscall layer or on some inputs based on information the OS provides).
“timereported” is what the sending device reports as time. This is taken from the appropriate syslog header field. If and only if the syslog date header cannot properly be parsed, “timereported” is populated with the same value as “timegenerated”.
Assuming that all systems in a relay chain use valid syslog format, “timereported” will be the same on all relay machines, whereas “timegenerated” reflects the local time of message reception and thus is different on each relay machine.
Please keep in mind the mentioned difference between both properties. It is well known, that the property name for “timegenerated” might be a bit confusing. As such, “timereceived” would probably be a better name, but changing it is not possible without breaking existing deplyoments.