Guides for rsyslog

Assumed Standard rsyslog.conf

When I initially started to write this book, I provided only excerpts of rsyslog.conf that showed which lines you had to add. Quickly, I received feedback that this is inadequate to make the recipies easy to use – because everyone still needed to guess where to place the excerpts. Or, even worse, place them at the wrong spots, not knowing that sequence of statements is important in rsyslog.conf.

To solve this issue, I now use a “standard” syslog config, just like can be found on many systems. Your’s may not be exactly the same, but I guess the standard config helps you find where the configuration samples need to go into. As a reference point, this is the “standard” config that all samples build on:

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none      /var/log/messages
# The authpriv file has restricted access.
authpriv.*                                    /var/log/secure
# Log all the mail messages in one place.
mail.*                                        /var/log/maillog
# Log cron stuff
cron.*                                        /var/log/cron
# Everybody gets emergency messages
*.emerg                                       *
# Save news errors of level crit and higher in a special file.
uucp,news.crit                                /var/log/spooler
# Save boot messages also to boot.log
local7.*                                      /var/log/boot.log

I was hesitant at first to use a real-life sample – repeating it in any recipe requires much space. However, I have convinced myself that this space is well spent, facilitating the adoption of the samples. Only occasionally, when I give some counter-examples, I spare myself from reproducing the full standard rsyslog.conf, hoping that the context makes clear what is meant.

In each recipe, a full rsyslog.conf, based on above example, is specified. The recipe-specific alterations to the config file are given in a different typeface, so that they can be easily identified.

Writing specific messages to a file and discarding them

Messages with the text “error” inside the text part of the message shall be written to a specific file. They shall not be written to any other file or be processed in any other way.

Things to think about

The configuration given here should be placed on top of the rsyslog.conf file.

Config Statements

:msg, contains, "error" /var/log/error.log
& ~
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* /var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log

How it works

The configuration uses a property-based filter to see if the string “error is contained” inside the MSG part of the syslog message. If so, the message is written to /var/log/error.log. The next line then discards all messages that have been written. Thus, no additional rules will be applied to the message. As such, it will not be written to /var/log/other.log.

Note the difference to this invalid sequence:

*.* /var/log/other.log
:msg, contains, "error" /var/log/error.log
& ~

Here everything is first written to /var/log/other.log and only then the message content is checked. In the later case, the message with “error” in them will be written to both files.

Important

Sequence of configuration statements is very important. Invalid sequence of otherwise perfectly legal configuration statements can lead to totally wrong results.

Sending Messages to a Remote Syslog Server

In this recipe, we forward messages from one system to another one. Typical use cases are:

  • the local system does not store any messages (e.g. has not sufficient space to do so)
  • there is a (e.g. legal) requirement to consolidate all logs on a single system
  • the server may run some advanced alerting rules, and needs to have a full picture or network activity to work well
  • you want to get the logs to a different system in a different security domain (to prevent attackers from hiding their tracks)
  • and many more …

In our case, we forward all messages to the remote system. Note that by applying different filters, you may only forward select entries to the remote system. Also note that you can include as many forwarding actions as you like. For example, if you need to have a backup central server, you can simply forward to both of them, using two different forwarding actions.

To learn how to configure the remote server, see recipe Receiving Messages from a Remote System.

Config Statements

# this is the simplest forwarding action:
*.* action(type="omfwd" target="192.0.2.1" port="10514" protocol="tcp")
# it is equivalent to the following obsolete legacy format line:
*.* @@192.0.2.1:10514 # do NOT use this any longer!
# Note: if the remote system is unreachable, processing will
# block here and discard messages after a while

# so a better use is
*.*  action(type="omfwd" target="192.0.2.2" port="10514" protocol="tcp"
            action.resumeRetryCount="100"
            queue.type="linkedList" queue.size="10000")
# this will de-couple the sending from the other logging actions,
# and prevent delays when the remote system is not reachable. Also,
# it will try to connect 100 times before it discards messages as
# undeliverable.
# the rest below is more or less a plain vanilla rsyslog.conf as 
# many distros ship it - it's more for your reference...
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none      /var/log/messages
# The authpriv file has restricted access.
authpriv.*                                    /var/log/secure
# Log all the mail messages in one place.
mail.*                                        /var/log/maillog
# Log cron stuff
cron.*                                        /var/log/cron
# Everybody gets emergency messages
*.emerg                                       :omusrmsg:*
# Save news errors of level crit and higher in a special file.
uucp,news.crit                                /var/log/spooler
# Save boot messages also to boot.log
local7.*                                      /var/log/boot.log

Things to think about

You need to select the protocol best suitable for your use case. If in doubt, TCP is a decent choice. This recipe uses TCP for that reason.

TCP forwarding is a build-in capability and always present. As such, no plugin needs to be loaded. The target can be specified by DNS name or IP address. Use IP addresses for most robust operations. If you use a DNS name and name resolution fails, forwarding may be disabled for some time. DNS resolution typically fails on the DNS server itself during system startup.

In this example, we forward to port 10514. We could as well remove the port=”…” parameter from the configuration, which would result in the default port being used. However, you need to specify the port address on the server in any case. So it is strongly advised to use an explicit port number to make sure that client and server configuration match each other (if they used different ports, the message transfer would not work.

Receiving Messages from a Remote System

This is a log-consolidation scenario. There exist at least two systems, a server and at least one client. The server is meant to gather log data from all the clients. Clients may (or may not) process and store messages locally. If they do doesn’t matter here. See recipe Sending Messages to a Remote Syslog Server for how to configure the clients.

Note that in this scenario, we just receive messages from remote machines but do not process them in any special way. Thus, messages from both the local and all remote systems show up in all log files that are written (as well, of course, in all other actions). While the log files contain the source, messages from all systems are intermixed. If you would like to record messages from remote systems to files different from the local system, please see recipe Storing Messages from a Remote System into a specific File for a potential solution.

This scenario provides samples for both UDP and TCP reception. There exist other choices (like RELP), but these are less frequently used. If in question what to use, check the rsyslog module reference and protocol documentation. Note that most devices send UDP messages by default. UDP is an unreliable transmission protocol, thus messages may get lost. TCP supports much more reliability, so if you can not accept message loss, you need to use TCP. Not all devices support TCP-based transports.

Things to think about

TCP and UDP recpetion are not build-in capabilities. You need to load the imtcp and/or imudp plugin in order to enable it. This needs to be done only once in rsyslog.conf. Do it right at the top. Also note that some distributions may package imtcp and/or imudp in separate packages. If so, you need to install them first.

Rsyslog versions prior to v3 had a command-line switch (-r/-t) to activate remote listening. This switch is still available by default and loads the required plugins and configures them with default parameters. However, that still requires the plugins are present on the system. It is recommended not to rely on compatibility mode but rather use proper configuration.

Note that the server port address specified in $InputTCPServerRun must match the port address that the clients send messages to.

Config Statements

# for TCP use:
module(load="imtcp") # needs to be done just once 
input(type="imtcp" port="514")
# for UDP use:
module(load="imudp") # needs to be done just once 
input(type="imudp" port="514")
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none      /var/log/messages
# The authpriv file has restricted access.
authpriv.*                                    /var/log/secure
# Log all the mail messages in one place.
mail.*                                        /var/log/maillog
# Log cron stuff
cron.*                                        /var/log/cron
# Everybody gets emergency messages
*.emerg                                       *
# Save news errors of level crit and higher in a special file.
uucp,news.crit                                /var/log/spooler
# Save boot messages also to boot.log
local7.*                                      /var/log/boot.log

How it works

Note that loading the plugins is not sufficient. You also need to activate the listeners. Note the subtle difference between the two startup commands. If you need to have listeners for multiple ports, you can define the startup commands more than once. If you need only TCP or only UDP, you can comment out the other part.

Using a different log Format for all Files

Rsyslog comes with a limited set of log file formats. These resemble the default format that people (and log analyzers) usually expect. However, for some reason or another, it may be required to change the log format. In this recipe, we define a new format and use it as the default format for all log files.

Config Statements

$template myFormat,"%rawmsg%\n"
$ActionFileDefaultTemplate myFormat
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* /var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log

Things to think about

The template and ActionFileDefaultTemplate statements must be made at the top of the configuration file, before any of the files are specified.

How it works

The Template-statement defines the new format. It consist of fields to be written, potential modifications as well as literal text. In the sample config statement, “rawmsg” ist a property that contains the syslog message as it was received by rsyslogd (“received” from any source, for example a remote system or the local log socket). The string “\n” is a line feed (ASCII LF), a constant being added to the string. Usually, log line templates need to end with “\n”, because without that, all log records would be written into a single line. Note that there are many fields and options for these fields that you can specify. The system is very flexible. But getting into the detail of all of that is beyond the scope of this cookbook-style book. Please see the “property replacer” official documentation for more details.

The $ActionFileDefaultTemplate then makes the newly defined template the default for all file actions. This saves you from specifying it with any single action line. But otherwise, it is equivalent to

$template myFormat,"%rawmsg%\n"
# The authpriv file has restricted access.
authpriv.*      /var/log/secure;myFormat
# Log all the mail messages in one place.
mail.*          /var/log/maillog;myFormat
# Log cron stuff
cron.*          /var/log/cron;myFormat
# Everybody gets emergency messages
*.emerg                                       *
# Save news errors of level crit and higher in a special file.
uucp,news.crit  /var/log/spooler;myFormat
# Save boot messages also to boot.log
local7.*        /var/log/boot.log;myFormat

Discarding unwanted messages

Often, there are some messages that you know you will never store in any log file. Even worse, these messages are sometimes very frequently emitted. There are various ways to get rid of those unwanted messages.

First of all, you need to identify them. Then look carfully and see what is special with these messages. A common case may be that they contain a specific text inside the message itself. If so, you can filter on that text and discard anything that matches. You need to be careful, though: if there are other messages matching this text, these other messages will also be discarded. So it is vital to make sure the text you use is actually unique.

In the sample below, let’s assume that you want to discard messages that contain either the text “user nagios” or “module-alsa-sink.c: ALSA woke us up to write new data to the device, but there was actually nothing to write”. The later is an actual sample from pulseaudio, which is known to spam syslog with an enourmous volume of these messages.

Config Statements

:msg, contains, "user nagios" ~
:msg, contains, "module-alsa-sink.c: ALSA woke us up to
write new data to the device, but there was actually
nothing to write" ~
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none      /var/log/messages
# The authpriv file has restricted access.
authpriv.*                                    /var/log/secure
# Log all the mail messages in one place.
mail.*                                        /var/log/maillog
# Log cron stuff
cron.*                                        /var/log/cron
# Everybody gets emergency messages
*.emerg                                       *
# Save news errors of level crit and higher in a special file.
uucp,news.crit                                /var/log/spooler
# Save boot messages also to boot.log
local7.*                                      /var/log/boot.log

Note that these are just two lines. The second to forth line are just broken for printing purposes. These two must be on a single line in an actual rsyslog.conf.

How it works

Note that the statements are placed on top of rsyslog.conf. This makes them being executed before any other action statement. So each message received will be checked against the two string and be discarded, if a match is found. Note that you can move the discard action to another place inside rsyslog.conf if you would like to write the messages to some files, but not to others. For example, this configuration:

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none      /var/log/messages
# do not log the following to other files

:msg, contains, "user nagios" ~

:msg, contains, "module-alsa-sink.c: ALSA woke us up to

write new data to the device, but there was actually

nothing to write" ~
# The authpriv file has restricted access.
authpriv.*                                    /var/log/secure
# Log all the mail messages in one place.
mail.*                                        /var/log/maillog

logs all messages to /var/log/messages, even those that then shall be discarded.

Storing Messages from a Remote System into a specific File

This is a log-consolidation scenario. There exist at least two systems, a server and at least one client. The server is meant to gather log data from all the clients. Clients may (or may not) process and store messages locally. If they do, doesn’t matter here. See recipe Sending Messages to a Remote Syslog Server for how to configure the clients.

Messages from remote hosts in the 192.0.1.x network shall be written to one file and messages from remote hosts in the 192.0.2.x network shallbe written to another file.

Things to think about

TCP recpetion is not a build-in capability. You need to load the imtcp plugin in order to enable it. This needs to be done only once in rsyslog.conf. Do it right at the top.

Note that the server port address specified in $InputTCPServerRun must match the port address that the clients send messages to.

Config Statements

$ModLoad imtcp
$InputTCPServerRun 10514
# do this in FRONT of the local/regular rules
if $fromhost-ip startswith '192.0.1.' then /var/log/network1.log
& ~
if $fromhost-ip startswith '192.0.2.' then /var/log/network2.log
& ~
# local/regular rules, like
*.* /var/log/syslog.log

How it works

It is important that the rules processing the remote messages come before any rules to process local messages. The if’s above check if a message originates on the network in question and, if so, writes them to the appropriate log. The next line (“& ~”) is important: it tells rsyslog to stop processing the message after it was written to the log. As such, these messages will not reach the local part. Without that “& ~”, messages would also be written to the local files.

Also note that in the filter there is a dot after the last number in the IP address. This is important to get reliable filters. For example, both of the addresses “192.0.1.1” and “192.0.10.1” start with “192.0.1” but only one actually starts with “192.0.1.”!

Integration with “standard” syslogd

Many people call sysklogd that “standard” syslogd because it comes by default with many distributions. Well, more precisely we should say “it came by default”. Over time, rsyslog has replaced sysklogd in most Linux distributions (for example, Fedora, Debian and Ubuntu). So it may be worth checking if rsyslog is already the standard syslogd on a system in question.

Things to think about

In any case, rsyslogd can work well with sysklogd. However, there are a number of limitations, based in missing capabilities of sysklogd. Most importantly, sysklogd does not support tcp syslog or any other protocol but UDP. Note that UDP syslogd does not offer reliability. So some syslog messages will probably be lost, especially on a busy system and/or network. If this is not acceptable, sysklogd must be replaced.

Also, various implementations of legacy syslogd’s use somewhat malformed message formats (based on current standards). As such, rsyslog may not correctly interpret the messages. There are solutions for this problem, but this currently is out of scope for the cookbook-type approach of this book. If you need to deal with these issues, please look at the official reference documentation or ask on the rsyslog forum or mailing list.

Config Statements

You need to configure the legacy syslogd (sysklogd, for example) to send messages to the machine running rsyslogd. The syntax is somewhat similar to rsyslogd’s, but offers limited options. Only basic priority filters can be used, and ports, zip compression or templates can not be specified.

Let us assume that the rsyslogd runing on the machine 192.0.0.1 shall receive a copy of all messages and the rsyslgod running on machine maillog.example.net shall receive copy of all mail-related logs.

In legacy syslog you configure this as follows:

*.* @192.0.0.1
mail.* @maillog.example.net

On the rsyslog side, you need to set up UDP reception. For the machine that gathers all logs, rsyslog.conf may look like below. Note that the listening port must be 514 as legacy syslogd (usually) does not support any other.

$ModLoad imudp
$UDPServerRun 514
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* /var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log

Note that this configuration will store both local and remote messages into the same files. This often is not desirable. Please refer to recipe 2.2.1 to see how to split local and remote logs to different files.

About this document …

Rsyslog Cookbook

This document was generated using the LaTeX2HTML translator Version 2008 (1.71)

Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.

The command line arguments were:
latex2html -t ‘rsyslog cookbook’ -local_icons rsyslog_cookbook.tex

The translation was initiated by Rainer Gerhards on 2010-02-23

Scroll to top