Search Results for: install

Periodic statistics on rsyslog counters

Since rsyslog 5.7.0 (V5-Devel) there is a new module called impstats.

This module provides periodic output of rsyslog internal counters. Note that the whole statistics system is currently under development. So availabilty and format of counters may change and is not yet stable (so be prepared to change your trending scripts when you upgrade to a newer rsyslog version).

The set of available counters will be output as a set of syslog messages. This output is periodic, with the interval being configurable (default is 5 minutes). Be sure that your configuration records the counter messages (default is syslog.info).

Note that loading this module has impact on rsyslog performance. Depending on settings, this impact may be severe (for high-load environments).

Please find below some simple steps how to use that module.

Of course you have to name it additionally in the configure. The configure should look like this (please note that the parameters can be different to your configure, the important thing is ––enable-impstats; this example is for Fedora 13 )

./configure --enable-impstats --sbindir=/sbin --libdir=/lib

The next steps are make and make install

make
make install

Furthermore we have to add that module in the rsyslog config. Below you will find a short excerpt of a sample config.

#### Modules ####
$ModLoad imuxsock
$ModLoad imklog
#$ModLoad immark

$ModLoad impstats
$PStatsInterval 600
$PStatsSeverity 7

syslog.debug  /var/log/rsyslog-stats

$ModLoad impstats – tells rsyslog to load the module impstats

$PStatsInterval 600 – is a configuration directive of impstats

$PStatsSeverity 7 – is also a configuration directive of impstats

Here you will find all information about the impstats module and its configuration directives.

If we have a look at the mentioned output file rsyslog-stats we can see the results of the static module.

The content of that file should look like this

Sep 17 11:43:49 localhost rsyslogd-pstats: umxsock: submitted=16
Sep 17 11:43:49 localhost rsyslogd-pstats: main Q: size=1 enqueued=2403 full=0 maxqsize=2

At the actual point all objects are shown in the results. Every Object has its own counter like

umxsock: (= object) submitted=16 (=counter; 16 logs received by the object umxsock). The main queue of rsyslog is also shown -> main Q with the parameters size (messages in the queue), enqueued (all received messages), full (how often was the queue full) and maxqsize (the maximal amount of messages in the queue).

Please be sure that the features and options of that module will be develeloped soon.

All information about the periodic statistics module you will find in the documentation. There is also a list of rsyslog impstats counters available on the web site.

How to write to a local socket?

One member of the rsyslog comunity wrote:

I’d like to forward via a local UNIX domain socket, instead. I think  I understand how to configure the ‘imuxsock’ module so my unprivileged instance reads from a non-standard socket location. But I can’t figure out how to tell my root instance to forward via a local domain socket.

I didn’t figure out a completely RSyslog-native method, but another poster’s message pointed me toward ‘socat’ and ‘omprog’, which I have working, now. (It would be really nice if RSyslog could support this natively, though.)

In case anyone else wants to set this up, maybe this will save you some effort. I’m also interested in any comments/criticisms about this method, I’d love to hear suggestions for better ways to make this work.

Also, I rolled it all up into a Fedora/EL RPM spec, and I’ll send it on to anyone who’s interested–just ask.

Setup steps:

  • Install the ‘socat’ utility.
  • Build RSyslog with the ‘–enable-omprog’ ./configure flag.
  • Create two separate RSyslog config files, one for the ‘root’ instance (writes to the socket) and a second for the ‘unprivileged’ instance (reads from the socket).
  • Rewrite your RSyslog init script to start two separate daemon instances, one using each config file (and separate PID files, too).
  • Create the user ‘rsyslogd’ and the group ‘rsyslogd’.
  • Set permissions/ownerships as needed to allow the user ‘rsyslogd’ to write to the file ‘/var/log/rsyslog.log’
  • Create an executable script called
    '/usr/libexec/rsyslogd/omprog_socat' that contains the lines:
    #!/bin/bash
    /usr/bin/socat -t0 -T0 -lydaemon -d - UNIX-SENDTO:/dev/log
  • The ‘root’ instance config file should contain (modifying the output actions to taste):

    $ModLoad imklog
    $ModLoad omprog
    $Template FwdViaUNIXSocket,"<%pri%>%syslogtag%%msg%"
    $ActionOMProgBinary /usr/libexec/rsyslogd/omprog_socat
    *.* :omprog:;FwdViaUNIXSocket

  • The ‘unprivileged’ instance config file should contain (modifying the output actions to taste):

    $ModLoad imuxsock
    $PrivDropToUser rsyslogd
    $PrivDropToGroup rsyslogd
    *.* /var/log/rsyslog.log

    The ‘root’ daemon can only accept input from the kernel message buffer, and nothing else (especially not the syslog socket (/dev/log) or any network sockets). The unprivileged user will handle all of local and network log messages. To merge the kernel logs into the same data channel as everything else, here’s what happens:

    [During the RSyslog daemons’ startup]

A) At startup, the ‘root’ daemon’s ‘imklog’ module starts listening for kernel messages (via ‘/prog/kmsg’), and its ‘omprog’ module starts an instance of ‘socat’ (called via the ‘omprog_socat’ wrapper), establishing a persistent one-way IO connection where ‘omprog’ pipes its output to the STDIN of ‘socat’.

  • (Note that this same ‘socat’ instance remains running throughout the life of the RSyslog daemon, handling everything ‘omprog’ outputs. Contrast this, efficiency-wise, against the built-in ‘subshell’ module [the ‘^/path/to/program’ action], which runs a separate instance instance of the child program for each message.)

B) At startup, the ‘unprivileged’ daemon’s ‘imuxsock’ module opens the system logging socket (‘/dev/log’) and starts listening for incoming log messages from other programs.

  • [During normal operation]1) The kernel buffer produces a message string on ‘/proc/kmsg’.2) The ‘root’ RSyslog daemon reads the message from ‘/proc/kmsg’, assigning it the priority number of ‘kern.info’ and the string tag ‘kernel’.3) The ‘root’ daemon prepends the priority number and tag as a header to the message string, and then passes it to the ‘omprog’ module for output (via persistent pipe) to the running ‘socat’ instance.4) The ‘socat’ instance receives the header-framed message and sends it to the system logging socket (‘/dev/log’).

    5) The ‘unprivileged’ RSyslog daemon reads the message from ‘/dev/log’, assigning it the priority and tag given in the message header, plus all of the other properties (timestamp, hostname, etc.) a message object should have.

    6) The ‘unprivileged’ daemon formats the message and writes it to the output file.

    The only real difference I can see in the forwarded messages is that the ‘source’ property is set to ‘imuxsock’ instead of ‘imklog’. I don’t think that’s a real problem, though, since the priority and tag are still distinct.

Build problems with 0.9.6

Bennett Todd has just alerted me of some problems with the new release. In short: the build process seems not to work at all (nor does the install do).

This is actually a documentation issue. The way of compiling rsyslog has changed slightly but importantly. You need to CD into an distribution-specific subdirectory (use linux of in doubt) and then call make. Do NOT do this in the root directory of the rsyslog project. It’s documented here

http://www.rsyslog.com/Documentation-/install.html.phtml

Looks like I must find some other place to document it… Or find a
different solution (read my blog on why I ended up with this:

http://rgerhards.blogspot.com/2005/08/make-syntax-differences.html

Sorry for any hassle. Suggestions are welcome!
Rainer

rsyslog now available on Sun Solaris

Rsyslog has become the de-facto standard on modern Linux operating systems. It’s high-performance log processing, database integration, modularity and support for multiple logging protocols make it the sysadmin’s logging daemon of choice. The project was started in 2004 and has since then evolved rapidly.

Starting with today, rsyslog is not only available on Linux and BSD, but also on Sun Solaris. Both Intel and Sparc machines are fully supported under Solaris. Depending on operator need, rsyslog can replace stock Solaris syslogd or be used in conjunction with it. The later case provides enhanced rsyslog functionality without the need to change the system infrastructure. Continue reading “rsyslog now available on Sun Solaris”

Receiving Messages from a Remote System

This is a log-consolidation scenario. There exist at least two systems, a server and at least one client. The server is meant to gather log data from all the clients. Clients may (or may not) process and store messages locally. If they do doesn’t matter here. See recipe Sending Messages to a Remote Syslog Server for how to configure the clients.

Note that in this scenario, we just receive messages from remote machines but do not process them in any special way. Thus, messages from both the local and all remote systems show up in all log files that are written (as well, of course, in all other actions). While the log files contain the source, messages from all systems are intermixed. If you would like to record messages from remote systems to files different from the local system, please see recipe Storing Messages from a Remote System into a specific File for a potential solution.

This scenario provides samples for both UDP and TCP reception. There exist other choices (like RELP), but these are less frequently used. If in question what to use, check the rsyslog module reference and protocol documentation. Note that most devices send UDP messages by default. UDP is an unreliable transmission protocol, thus messages may get lost. TCP supports much more reliability, so if you can not accept message loss, you need to use TCP. Not all devices support TCP-based transports.

Things to think about

TCP and UDP recpetion are not build-in capabilities. You need to load the imtcp and/or imudp plugin in order to enable it. This needs to be done only once in rsyslog.conf. Do it right at the top. Also note that some distributions may package imtcp and/or imudp in separate packages. If so, you need to install them first.

Rsyslog versions prior to v3 had a command-line switch (-r/-t) to activate remote listening. This switch is still available by default and loads the required plugins and configures them with default parameters. However, that still requires the plugins are present on the system. It is recommended not to rely on compatibility mode but rather use proper configuration.

Note that the server port address specified in $InputTCPServerRun must match the port address that the clients send messages to.

Config Statements

# for TCP use:
module(load="imtcp") # needs to be done just once 
input(type="imtcp" port="514")
# for UDP use:
module(load="imudp") # needs to be done just once 
input(type="imudp" port="514")
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none      /var/log/messages
# The authpriv file has restricted access.
authpriv.*                                    /var/log/secure
# Log all the mail messages in one place.
mail.*                                        /var/log/maillog
# Log cron stuff
cron.*                                        /var/log/cron
# Everybody gets emergency messages
*.emerg                                       *
# Save news errors of level crit and higher in a special file.
uucp,news.crit                                /var/log/spooler
# Save boot messages also to boot.log
local7.*                                      /var/log/boot.log

How it works

Note that loading the plugins is not sufficient. You also need to activate the listeners. Note the subtle difference between the two startup commands. If you need to have listeners for multiple ports, you can define the startup commands more than once. If you need only TCP or only UDP, you can comment out the other part.

ChangeLog for 4.5.2 (beta)

Version 4.5.2 [v4-beta] (rgerhards), 2009-08-21

  • legacy syslog parser changed so that it now accepts date stamps in wrong case. Some devices seem to create them and I do not see any harm in supporting that.
  • added $InputTCPMaxListeners directive – permits to specify how many TCP servers shall be possible (default is 20).
  • bugfix: memory leak with some input modules. Those inputs that use parseAndSubmitMsg() leak two small memory blocks with every message. Typically, those process only relatively few messages, so the issue does most probably not have any effect in practice.
  • bugfix: if tcp listen port could not be created, no error message was emitted
  • bugfix: potential segfault in output file writer (omfile) In async write mode, we use modular arithmetic to index the output buffer array. However, the counter variables accidently were signed, thus resulting in negative indizes after integer overflow. That in turn could lead to segfaults, but was depending on the memory layout of the instance in question (which in turn depended on a number of variables, like compile settings but also configuration). The counters are now unsigned (as they always should have been) and so the dangling mis-indexing does no longer happen. This bug potentially affected all installations, even if only some may actually have seen a segfault.
  • bugfix: hostnames with dashes in them were incorrectly treated as malformed, thus causing them to be treated as TAG (this was a regression introduced from the “rfc3164 strict” change in 4.5.0).

ChangeLog for 5.1.4 (devel)

Version 5.1.4 [DEVEL] (rgerhards), 2009-08-20

  • legacy syslog parser changed so that it now accepts date stamps in wrong case. Some devices seem to create them and I do not see any harm in supporting that.
  • added $InputTCPMaxListeners directive – permits to specify how many TCP servers shall be possible (default is 20).
  • bugfix: memory leak with some input modules. Those inputs that use parseAndSubmitMsg() leak two small memory blocks with every message. Typically, those process only relatively few messages, so the issue does most probably not have any effect in practice.
  • bugfix: if tcp listen port could not be created, no error message was emitted
  • bugfix: discard action did not work (did not discard messages)
  • bugfix: discard action caused segfault
  • bugfix: potential segfault in output file writer (omfile) In async write mode, we use modular arithmetic to index the output buffer array. However, the counter variables accidently were signed, thus resulting in negative indizes after integer overflow. That in turn could lead to segfaults, but was depending on the memory layout of the instance in question (which in turn depended on a number of variables, like compile settings but also configuration). The counters are now unsigned (as they always should have been) and so the dangling mis-indexing does no longer happen. This bug potentially affected all installations, even if only some may actually have seen a segfault.
Scroll to top