rsyslog

The rocket-fast system for log processing

Receiving Messages from a Remote System

This is a log-consolidation scenario. There exist at least two systems, a server and at least one client. The server is meant to gather log data from all the clients. Clients may (or may not) process and store messages locally. If they do doesn’t matter here. See recipe Sending Messages to a Remote Syslog Server for how to configure the clients.

Note that in this scenario, we just receive messages from remote machines but do not process them in any special way. Thus, messages from both the local and all remote systems show up in all log files that are written (as well, of course, in all other actions). While the log files contain the source, messages from all systems are intermixed. If you would like to record messages from remote systems to files different from the local system, please see recipe Storing Messages from a Remote System into a specific File for a potential solution.

This scenario provides samples for both UDP and TCP reception. There exist other choices (like RELP), but these are less frequently used. If in question what to use, check the rsyslog module reference and protocol documentation. Note that most devices send UDP messages by default. UDP is an unreliable transmission protocol, thus messages may get lost. TCP supports much more reliability, so if you can not accept message loss, you need to use TCP. Not all devices support TCP-based transports.

Things to think about

TCP and UDP recpetion are not build-in capabilities. You need to load the imtcp and/or imudp plugin in order to enable it. This needs to be done only once in rsyslog.conf. Do it right at the top. Also note that some distributions may package imtcp and/or imudp in separate packages. If so, you need to install them first.

Rsyslog versions prior to v3 had a command-line switch (-r/-t) to activate remote listening. This switch is still available by default and loads the required plugins and configures them with default parameters. However, that still requires the plugins are present on the system. It is recommended not to rely on compatibility mode but rather use proper configuration.

Note that the server port address specified in $InputTCPServerRun must match the port address that the clients send messages to.

Config Statements

# for TCP use:
module(load="imtcp") # needs to be done just once 
input(type="imtcp" port="514")
# for UDP use:
module(load="imudp") # needs to be done just once 
input(type="imudp" port="514")
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none      /var/log/messages
# The authpriv file has restricted access.
authpriv.*                                    /var/log/secure
# Log all the mail messages in one place.
mail.*                                        /var/log/maillog
# Log cron stuff
cron.*                                        /var/log/cron
# Everybody gets emergency messages
*.emerg                                       *
# Save news errors of level crit and higher in a special file.
uucp,news.crit                                /var/log/spooler
# Save boot messages also to boot.log
local7.*                                      /var/log/boot.log

How it works

Note that loading the plugins is not sufficient. You also need to activate the listeners. Note the subtle difference between the two startup commands. If you need to have listeners for multiple ports, you can define the startup commands more than once. If you need only TCP or only UDP, you can comment out the other part.

12 thoughts on “Receiving Messages from a Remote System

  1. Pingback: Sending Messages to a Remote Syslog Server rsyslog

  2. Hi,

    Could you point me to the right direction.
    I want to sort out the log messages from remote servers into separate files as they do it locally, I am using rsyslog 4.2.0 on Ubuntu 10.04 LTS. Seems i cannot find a proper howto or documentation what directivas should i use. Sorry if i am missing something.

    THx
    Peter

  3. I have two devices, sending logs to two destination ip addresses on the same server, and i would like to split them.
    Devices IP’s are changing, server IP,s are 10.7.1.1 and 10.7.1.2

    I have something like this on my mind:
    $template FilenameTemplateOne,"/var/log/dev01/%$year%/%$month%/%$day%/%$year%%$month%%$day%-%fromhost-ip%-%hostname%.log"
    $template FilenameTemplateTwo,"/var/log/dev02/%$year%/%$month%/%$day%/%$year%%$month%%$day%-%fromhost-ip%-%hostname%.log"
    if $fromhost-ip == ‘10.7.1.1’ then -?FilenameTemplateOne
    & ~
    if $fromhost-ip == ‘10.7.1.2’ then -?FilenameTemplateTwo
    & ~

    Only fromhost-ip will not help, since i want to split them by destination ip, not device ip.

  4. I am trying to get log from multiple systems on port 514 (general) and another set on port 525 (firewall) they are stored /usr/logs/general and /usr/logs/firewall so they need to be broken out. I am new to rsyslog. I am moving from syslog-ng. I am having no luck so far.

  5. Hi,
    There is a typo in section "Things to think about". "Them" is misspelled in following line:
    "If so, you need to install thenm first"

    Cheers,
    Salim Pathan

  6. It is amazing to see how incomplete rsyslog documentation is, it seems without any direction on what message to deliver. For the same task i’ve seen at least three solutions for which i find no documenation, incomplete syntax documentation etc. In terms of vendor lock-in or rather lock-out this seems a prime example.

  7. We know, that the documentation is far from being optimal, but since the beginning of this year a big community-driven effort has been pulled to make it better. Helping hands are always welcome. So if you are interested in helping, jump over to github and take a look: https://github.com/rsyslog/rsyslog-doc

  8. Where the log files will be stored in the remote syslog server while I am sending the log files from another machine to the remote server

  9. On my Debian Squeeze boxes, incoming log files are stored in /var/log/syslog unless directed elsewhere.

Comments are closed.