Integration with “standard” syslogd
Many people call sysklogd that “standard” syslogd because it comes by default with many distributions. Well, more precisely we should say “it came by default”. Over time, rsyslog has replaced sysklogd in most Linux distributions (for example, Fedora, Debian and Ubuntu). So it may be worth checking if rsyslog is already the standard syslogd on a system in question.
Things to think about
In any case, rsyslogd can work well with sysklogd. However, there are a number of limitations, based in missing capabilities of sysklogd. Most importantly, sysklogd does not support tcp syslog or any other protocol but UDP. Note that UDP syslogd does not offer reliability. So some syslog messages will probably be lost, especially on a busy system and/or network. If this is not acceptable, sysklogd must be replaced.
Also, various implementations of legacy syslogd’s use somewhat malformed message formats (based on current standards). As such, rsyslog may not correctly interpret the messages. There are solutions for this problem, but this currently is out of scope for the cookbook-type approach of this book. If you need to deal with these issues, please look at the official reference documentation or ask on the rsyslog forum or mailing list.
Config Statements
You need to configure the legacy syslogd (sysklogd, for example) to send messages to the machine running rsyslogd. The syntax is somewhat similar to rsyslogd’s, but offers limited options. Only basic priority filters can be used, and ports, zip compression or templates can not be specified.
Let us assume that the rsyslogd runing on the machine 192.0.0.1 shall receive a copy of all messages and the rsyslgod running on machine maillog.example.net shall receive copy of all mail-related logs.
In legacy syslog you configure this as follows:
*.* @192.0.0.1
mail.* @maillog.example.net
On the rsyslog side, you need to set up UDP reception. For the machine that gathers all logs, rsyslog.conf may look like below. Note that the listening port must be 514 as legacy syslogd (usually) does not support any other.
$ModLoad imudp
$UDPServerRun 514
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* /var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log
Note that this configuration will store both local and remote messages into the same files. This often is not desirable. Please refer to recipe 2.2.1 to see how to split local and remote logs to different files.
About this document …
Rsyslog Cookbook
This document was generated using the LaTeX2HTML translator Version 2008 (1.71)
Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.
The command line arguments were:
latex2html -t ‘rsyslog cookbook’ -local_icons rsyslog_cookbook.tex
The translation was initiated by Rainer Gerhards on 2010-02-23
ChangeLog for 4.5.8 (v4-beta)
Version 4.5.8 [v4-beta] (rgerhards), 2010-02-10
- enhanced doc for using PostgreSQL
Thanks to Marc Schiffbauer for the new/updated doc - bugfix: property replacer returned invalid parameters under some (unusual) conditions. In extreme cases, this could lead to garbled logs and/or a system failure.
- bugfix: invalid length returned (often) when using regular expressions inside the property replacer
- bugfix: submatch regex in property replacer did not honor “return 0 on no match” config case
- bugfix: imuxsock incorrectly stated inputname “imudp”
Thanks to Ryan Lynch for reporting this. - (slightly) enhanced support for FreeBSD by setting _PATH_MODDIR to the correct value on FreeBSD.
Thanks to Cristiano for the patch. - bugfix: -d did not enable display of debug messages regression from introduction of “debug on demand” mode
Thanks to Michael Biebl for reporting this bug - bugfix: blanks inside file names did not terminate file name parsing.
This could reslult in the whole rest of a line (including comments) to be treated as file name in “write to file” actions.
Thanks to Jack for reporting this issue. - bugfix: rsyslog hang when writing to a named pipe which nobody was reading. Thanks to Michael Biebl for reporting this bug.
- bugfix: memory leak when sending messages in zip-compressed format
Thanks to Naoya Nakazawa for analyzing this issue and providing a patch. - bugfix: potential segfaults during queue shutdown (bugs require certain non-standard settings to appear)
Thanks to varmojfekoj for the patch
ChangeLog for 5.5.2 (devel)
Version 5.5.2 [DEVEL] (rgerhards), 2010-02-05
- applied patches that make rsyslog compile under Apple OS X.
Thanks to trey for providing these. - replaced data type “bool” by “sbool” because this created some portability issues.
- added $Escape8BitCharactersOnReceive directive
Thanks to David Lang for suggesting it. - worked around an issue where omfile failed to compile on32 bit platforms under some circumstances (this smells like a gcc problem, but a simple solution was available). Thanks to Kenneth Marshall for some advice.
- extended testbench
- bugfix: queues in direct mode could case a segfault, especially if an action failed for action queues. The issue was an invalid increment of a stack-based pointer which lead to destruction of the stack frame and thus a segfault on function return.
Thanks to Michael Biebl for alerting us on this problem. - bugfix: hostname accidently set to IP address for some message sources, for example imudp. Thanks to Anton for reporting this bug. [imported v4]
- bugfix: ompgsql had problems with transaction support, what actually rendered it unsuable. Thanks to forum user “horhe” for alerting me on this bug and helping to debug/fix it! [imported from 5.3.6]
- bugfix: $CreateDirs variable not properly initialized, default thus was random (but most often “on”) [imported from v3]
ChangeLog for 5.3.7 (v5-beta)
Version 5.3.7 [BETA] (rgerhards), 2010-01-27
- bugfix: queues in direct mode could case a segfault, especially if an action failed for action queues. The issue was an invalid increment of a stack-based pointer which lead to destruction of the stack frame and thus a segfault on function return.
Thanks to Michael Biebl for alerting us on this problem. [backport from 5.5.2] - bugfix: wrong memory assignment for a config variable (probably without causing any harm) [backport from 5.2.2]
- bugfix: rsyslog hangs when writing to a named pipe which nobody was reading. Thanks to Michael Biebl for reporting this bug.
Bugzilla entry: http://bugzilla.adiscon.com/show_bug.cgi?id=169 [imported from 4.5.8]
ChangeLog for 5.3.6 (v5-beta)
Version 5.3.6 [BETA] (rgerhards), 2010-01-13
- bugfix: ompgsql did not properly check the server connection in
tryResume(), which could lead to rsyslog running in a thight loop - bugfix: suspension during beginTransaction() was not properly handled
by rsyslog core - bugfix: omfile output was only written when buffer was full, not at
end of transaction - bugfix: commit transaction was not properly conveyed to message layer,
potentially resulting in non-message destruction and thus hangs - bugfix: enabling GSSServer crashes rsyslog startup
Thanks to Tomas Kubina for the patch [imgssapi] - bugfix (kind of): check if TCP connection is still alive if using TLS
Thanks to Jonathan Bond-Caron for the patch. - bugfix: $CreateDirs variable not properly initialized, default thus
was random (but most often “on”) [imported from v3] - bugfix: ompgsql had problems with transaction support, what actually
rendered it unsuable. Thanks to forum user “horhe” for alerting me
on this bug and helping to debug/fix it! - bugfix: memory leak when sending messages in zip-compressed format
Thanks to Naoya Nakazawa for analyzing this issue and providing a patch. - worked around an issue where omfile failed to compile on 32 bit platforms
under some circumstances (this smells like a gcc problem, but a simple
solution was available). Thanks to Kenneth Marshall for some advice.
[backported from 5.5.x branch]
ChangeLog for 5.5.1 (devel)
Version 5.5.1 [DEVEL] (rgerhards), 2009-11-27
- introduced the ablity for netstream drivers to utilize an epoll interface
This offers increased performance and removes the select() FDSET size limit from imtcp. Note that we fall back to select() if there is no epoll netstream drivers. So far, an epoll driver has only been implemented for plain tcp syslog, the rest will follow once the code proves well in practice AND there is demand. - re-implemented $EscapeControlCharacterTab config directive
Based on Jonathan Bond-Caron’s patch for v4. This now also includes some automatted tests. - bugfix: enabling GSSServer crashes rsyslog startup
Thanks to Tomas Kubina for the patch [imgssapi] - bugfix (kind of): check if TCP connection is still alive if using TLS
Thanks to Jonathan Bond-Caron for the patch.
ChangeLog for 4.5.7 (v4-beta)ChangeLog for 4.5.7 (v4-beta)
Version 4.5.7 [v4-beta] (rgerhards), 2009-11-18
- added a so-called “On Demand Debug” mode, in which debug output can be generated only after the process has started, but not right from the beginning. This is assumed to be useful for hard-to-find bugs. Also improved the doc on the debug system.
- bugfix [imported from 4.4.3]: $ActionExecOnlyOnceEveryInterval did not work.
ChangeLog for 5.5.0 (devel)
Version 5.5.0 [DEVEL] (rgerhards), 2009-11-18
- moved DNS resolution code out of imudp and into the backend processing
Most importantly, DNS resolution now never happens if the resolved name is not required. Note that this applies to imudp – for the other inputs, DNS resolution almost comes for free, so we do not do it there. However, the new method has been implemented in a generic way and as such may also be used by other modules in the future. - added option to use unlimited-size select() calls
Thanks to varmjofekoj for the patch
This is not done in imudp, as it natively supports epoll(). - doc: improved description of what loadable modules can do
ChangeLog for 5.3.5 (beta)
Version 5.3.5 [BETA] (rgerhards), 2009-11-13
- some light performance enhancement by replacing time() call with much faster (at least under linux) gettimeofday() calls.
- some improvement of omfile performance with dynafiles saved costly time() calls by employing a logical clock, which is sufficient for the use case
- bugfix: omudpspoof miscalculated source and destination ports while this was probably not noticed for source ports, it resulted in almost all destination ports being wrong, except for the default port of 514, which by virtue of its binary representation was calculated correct (and probably thus the bug not earlier detected).
- bugfixes imported from earlier releases
- bugfix: named pipes did no longer work (they always got an open error)
this was a regression from the omfile rewrite in 4.5.0 - bugfix(testbench): sequence check was not always performed correctly, that could result in tests reporting success when they actually failed
- bugfix: named pipes did no longer work (they always got an open error)
- improved testbench: added tests for UDP forwarding and omudpspoof
- doc bugfix: omudpspoof had wrong config command names (“om” missing)
- bugfix [imported from 4.4.3]: $ActionExecOnlyOnceEveryInterval did not work.
- [inport v4] improved testbench, contains now tcp and gzip test cases
- [import v4] added a so-called “On Demand Debug” mode, in which debug output can be generated only after the process has started, but not right from the beginning. This is assumed to be useful for hard-to-find bugs. Also improved the doc on the debug system.
- bugfix: segfault on startup when -q or -Q option was given [imported from v3-stable]