Storing and forwarding remote messages
In this scenario, we want to store remote sent messages into a specific local file and forward the received messages to another syslog server. Local messages should still be locally stored.
Things to think about
How should this work out? Basically, we need a syslog listener for TCP and one for UDP, the local logging service and two rulesets, one for the local logging and one for the remote logging.
TCP recpetion is not a build-in capability. You need to load the imtcp plugin in order to enable it. This needs to be done only once in rsyslog.conf. Do it right at the top.
Note that the server port address specified in $InputTCPServerRun must match the port address that the clients send messages to.
Config Statements
# Modules
$ModLoad imtcp $ModLoad imudp $ModLoad imuxsock $ModLoad imklog
# Templates
# log every host in its own directory $template RemoteHost,"/var/syslog/hosts/%HOSTNAME%/%$YEAR%/%$MONTH%/%$DAY%/syslog.log"
### Rulesets
# Local Logging $RuleSet local kern.* /var/log/messages *.info;mail.none;authpriv.none;cron.none /var/log/messages authpriv.* /var/log/secure mail.* -/var/log/maillog cron.* /var/log/cron *.emerg * uucp,news.crit /var/log/spooler local7.* /var/log/boot.log # use the local RuleSet as default if not specified otherwise $DefaultRuleset local # Remote Logging $RuleSet remote *.* ?RemoteHost # Send messages we receive to Gremlin *.* @@W.X.Y.Z:514
### Listeners
# bind ruleset to tcp listener $InputTCPServerBindRuleset remote # and activate it: $InputTCPServerRun 10514 $InputUDPServerBindRuleset remote $UDPServerRun 514
How it works
The configuration basically works in 4 parts. First, we load all the modules (imtcp, imudp, imuxsock, imklog). Then we specify the templates for creating files. The we create the rulesets which we can use for the different receivers. And last we set the listeners.
The rulesets are somewhat interesting to look at. The ruleset “local” will be set as the default ruleset. That means, that it will be used by any listener if it is not specified otherwise. Further, this ruleset uses the default log paths vor various facilities and severities.
The ruleset “remote” on the other hand takes care of the local logging and forwarding of all log messages that are received either via UDP or TCP. First, all the messages will be stored in a local file. The filename will be generated with the help of the template at the beginning of our configuration (in our example a rather complex folder structure will be used). After logging into the file, all the messages will be forwarded to another syslog server via TCP.
In the last part of the configuration we set the syslog listeners. We first bind the listener to the ruleset “remote”, then we give it the directive to run the listener with the port to use. In our case we use 10514 for TCP and 514 for UDP.
Important
There are some tricks in this configuration. Since we are actively using the rulesets, we must specify those rulesets before being able to bind them to a listener. That means, the order in the configuration is somewhat different than usual. Usually we would put the listener commands on top of the configuration right after the modules. Now we need to specify the rulesets first, then set the listeners (including the bind command). This is due to the current configuration design of rsyslog. To bind a listener to a ruleset, the ruleset object must at least be present before the listener is created. And that is why we need this kind of order for our configuration.
Using the syslog receiver module
We want to use rsyslog in its general purpose. We want to receive syslog. In rsyslog, we have two possibilities to achieve that.
Things to think about
First of all, we will determine, which way of syslog reception we want to use. We can receive syslog via UDP or TCP. The config statements are each a bit different in both cases.
In most cases, UDP syslog should be fully sufficient and performing well. But, you should be aware, that on large message bursts messages can be dropped. That is not the case with TCP syslog, since the sender and receiver communicate about the arrival of network packets. That makes TCP syslog more suitable for environments where log messages may not be lost or that must ensure PCI compliance (like banks).
Config Statements
module(load="imudp") # needs to be done just once input(type="imudp" port="514")
and
module(load="imtcp") # needs to be done just once input(type="imtcp" port="514")
How it works
The configuration part for the syslog server is pretty simple basically. Though, there are some parameters that can be set for both modules. But this is not necessary in most cases.
In general, first the module needs to be loaded. This is done via the directive module().
module(load="imudp / imtcp")
The module must be loaded, because the directives and functions rely on it. Just by using the right commands, rsyslog will not know where to get the functionality code from.
And next is the command that runs the syslog server itself is called input().
input(type="imudp" port="514") input(type="imtcp" port="514")
Basically, the command says to run a syslog server on a specific port. Depending on the command, you can easily determine the UDP and the TCP server.
You can of course use both types of syslog server at the same time, too. You just need to load both modules for that and configure the server command to listen to specific ports. Then you can receive both UDP and TCP syslog.
Important
In general, we suggest to use TCP syslog. It is way more reliable than UDP syslog and still pretty fast. The main reason is, that UDP might suffer of message loss. This happens when the syslog server must receive large bursts of messages. If the system buffer for UDP is full, all other messages will be dropped. With TCP, this will not happen. But sometimes it might be good to have a UDP server configured as well. That is, because some devices (like routers) are not able to send TCP syslog by design. In that case, you would need both syslog server types to have everything covered. If you need both syslog server types configured, please make sure they run on proper ports. By default UDP syslog is received on port 514. TCP syslog needs a different port because often the RPC service is using this port as well.