Search Results for: error 0

Installing RSyslog 5 on RHEL 4 / 5

To have rsyslog working correctly on RHEL 4 or 5, some conditions have to be met. The method described has been tested with rsyslog 5.7.1.

First of all compile and install the dependencies.

  • gnutls-2.8.6.tar.bz2
  • libgcrypt-1.4.6.tar.gz
  • libgpg-error-1.9.tar.gz
  • libtasn1-2.2.tar.gz

After that, you can install rsyslog using the following commands:

./configure PKG_CONFIG_PATH=/usr/local/lib/pkgconfig --enable-gnutls
make
make install

It could happen, that the install might complain about gnutls.pc. Simply comment out the URL found near the start of the file /usr/local/lib/pkgconfig/gnutls.pc.

Credit for this find goes to Forum member Johann Reinhard (johannreinhard).

Changelog for 5.5.7 (v5-beta)

Version 5.5.7  [V5-BETA] (rgerhards), 2010-08-09

  • changed omudpspoof default spoof address to simplify typical use casethanks to David Lang for suggesting this
  • doc bugfix: pmlastmsg doc samples had errors
  • bugfix[minor]: pmrfc3164sd had invalid name (resided in rsyslog namespace, what should not be the case for a contributed module)
  • added omuxsock, which permits to write message to local Unix socketsthis is the counterpart to imuxsock, enabling fast local forwarding

Changelog for 4.6.3 (v4-stable)

Version 4.6.3 [v4-stable] (rgerhards), 2010-07-07

  • improvded testbench
  • added test with truly random data received via syslog to test robustness
  • added new configure option that permits to disable and enable an extended testbench
  • bugfix: segfault on HUP when “HUPIsRestart” was set to “on”
    thanks varmojfekoj for the patch
  • bugfix: default for $OMFileFlushOnTXEnd was wrong (“off”).
    This, in default mode, caused buffered writing to be used, what means that it looked like no output were written or partial lines. Thanks to Michael Biebl for pointing out this bug.
  • bugfix: testbench failed when not executed in UTC+1 timezone accidently, the time zone information was kept inside some to-be-checked-for responses
  • temporary bugfix replaced by permanent one for message-induced off-by-one error (potential segfault) (see 4.6.2)
    The analysis has been completed and a better fix been crafted and integrated.
  • some doc fixes; incorrect config samples could cause confusion
    thanks to Anthony Edwards for pointing the problems out

Regex

This is a regular expression checker especially programmed for rsyslog. Rsyslog uses POSIX ERE (and optionally BRE) expressions. This checker works with the php POSIX ERE
functions. So in theory, there can be a difference between what the engine included in rsyslog (clib) and this web app does. However, in practice the results should be fairly the
same.

Note: you need at least rsyslog version 3.19.5 (for v3) or 4.1.5 (for v4 and above) to use ERE expressions.

To use this tool, paste a sample of the field in question into “Sample Log Line”, write your regular expression and press the button ;) You will then see what the regular expression engine extracts. Also, the result fields contain a property replacer
field definition in theory suitable for copying and pasting into your rsyslog.conf.

rsyslog property
Regular Expression
Submatch to Use
Nomatch Mode
Sample Log Line
(use a single line,
wraps for readability)


‘;

writeTopicBoxTail();

if(0 == 1) echo ‘
‘;

// now check if we got any data to process. If so, do it
if(strlen($regex) == 0 && strlen($sampleData) == 0)
; // no data, nothing to do ;)
else {
writeTopicBoxHdr(‘Regular Expression Result’);
echo ‘

‘;
if(strlen($regex) == 0 && strlen($sampleData) > 0)
echo ‘error, regex must be specified

‘;
else if(strlen($regex) > 0 && strlen($sampleData) == 0)
echo ‘

error, sample log line must be specified

‘;
else if(strlen($regex) > 0 && strlen($sampleData) > 0) {
// ok, we actually have something to process
$regRes = ereg($regex, $sampleData, $matches);
if($regRes == FALSE) {
echo ‘

No matches found or invalid search expression.
Note that the lazy quantifier (question mark) is not part of POISX regular
expressions, this may be the trouble source (among many others).

‘;
if($notfound == 0)
$res = ‘**NO MATCH**’;
else if($notfound == 1)
$res = “”;
else if($notfound == 2)
$res = “0”;
else
$res = $sampleData;
displaySrchResult($res, $regex, $prop, $submatch, $notfound);
} else {
displaySrchResult($matches[$submatch], $regex, $prop, $submatch, $notfound);
displayExtraInfo($regRes, $matches, $regex, $prop, $submatch);
}
echo ‘

‘;
writeSupportAd();
echo ‘

‘;
writeTopicBoxTail();
}
}

//echo ‘


';
//print_r($_REQUEST);
//echo '

‘;

Changelog for 4.6.2 (v4-stable)

Version 4.6.2 [v4-stable] (rgerhards), 2010-03-26

  • new feature: “.” action type added to support writing files to relative pathes (this is primarily meant as a debug aid)
  • new feature: $OMFileAsyncWriting directive added it permits to specifiy if asynchronous writing should be done or not
  • bugfix(temporary): message-induced off-by-one error (potential segfault)
    Some types of malformed messages could trigger an off-by-one error (for example, \0 or \n as the last character, and generally control character escaption is questionable). This is due to not strictly following a the \0 or string counted string paradigm (during the last optimization on the cstring class). As a temporary fix, we have introduced a proper recalculation of the size. However, a final patch is expected in the future. See bug tracker for further details and when the final patch will be available: http://bugzilla.adiscon.com/show_bug.cgi?id=184 Note that the current patch is considered sufficient to solve the situation, but it requires a bit more runtime than desirable.
  • bugfix: potential segfault in dynafile cache
    This bug was triggered by an open failure. The the cache was full and a new entry needed to be placed inside it, a victim for eviction was selected. That victim was freed, then the open of the new file tried. If the open failed, the victim entry was still freed, and the function exited. However, on next invocation and cache search, the victim entry was used as if it were populated, most probably resulting in a segfault.
  • bugfix: race condition during directory creation
    If multiple files try to create a directory at (almost) the same time, some of them may fail. This is a data race and also exists with other processes that may create the same directory. We do now check for this condition and gracefully handle it.
  • bugfix: potential re-use of free()ed file stream object in omfile
    when dynaCache is enabled, the cache is full, a new entry needs to be allocated, thus the LRU discarded, then a new entry is opend and that fails. In that case, it looks like the discarded stream may be reused improperly (based on code analysis, test case and confirmation pending)
  • added new property replacer option “date-rfc3164-buggyday” primarily to ease migration from syslog-ng. See property replacer doc for details. [backport from 5.5.3 because urgently needed by some]
  • improved testbench
  • bugfix: invalid buffer write in (file) stream class
    currently being accessed buffer could be overwritten with new data. While this probably did not cause access violations, it could case loss and/or duplication of some data (definitely a race with no deterministic outcome)
  • bugfix: potential hang condition during filestream close predicate was not properly checked when waiting for the background file writer
  • bugfix: improper synchronization when “$OMFileFlushOnTXEnd on” was used
    Internal data structures were not properly protected due to missing mutex calls.
  • bugfix: potential data loss during file stream shutdown
  • bugfix: potential problems during file stream shutdown
    The shutdown/close sequence was not clean, what potentially (but unlikely) could lead to some issues. We have not been able to describe any fatal cases, but there was some bug potential. Sequence has now been straighted out.
  • bugfix: potential problem (loop, abort) when file write error occured
    When a write error occured in stream.c, variable iWritten had the error code but this was handled as if it were the actual number of bytes written. That was used in pointer arithmetic later on, and thus could lead to all sorts of problems. However, this could only happen if the error was EINTR or the file in question was a tty. All other cases were handled properly. Now, iWritten is reset to zero in such cases, resulting in proper retries.
  • bugfix: $omfileFlushOnTXEnd was turned on when set to off and vice versa due to an invalid check
  • bugfix: recent patch to fix small memory leak could cause invalid free.
    This could only happen during config file parsing.
  • bugfix(minor): handling of extremely large strings in dbgprintf() fixed
    Previously, it could lead to garbagge output and, in extreme cases, also to segfaults. Note: this was a problem only when debug output was actually enabled, so it caused no problem in production use.
  • bugfix(minor): BSD_SO_COMPAT query function had some global vars not properly initialized. However, in practice the loader initializes them with zero, the desired value, so there were no actual issue in almost all cases.

ChangeLog for 4.6.1 (v4-stable)

Version 4.6.1 [v4-stable] (rgerhards), 2010-03-04

  • re-enabled old pipe output (using new module ompipe, built-in) after some problems with pipes (and especially in regard to xconsole) were discovered.
    Thanks to Michael Biebl for reporting the issues.

  • bugfix: potential problems with large file support could cause segfault … and other weird problems. This seemed to affect 32bit-platforms only, but I can not totally outrule there were issues on other platforms as well. The previous code could cause system data types to be defined inconsistently, and that could lead to various troubles.
    Special thanks go to the Mandriva team for identifying an initial problem, help discussing it and ultimately a fix they contributed.

  • bugfix: fixed problem that caused compilation on FreeBSD 9.0 to fail.
    bugtracker: http://bugzilla.adiscon.com/show_bug.cgi?id=181
    Thanks to Christiano for reporting.

  • bugfix: potential segfault in omfile when a dynafile open failed
    In that case, a partial cache entry was written, and some internal pointers (iCurrElt) not correctly updated. In the next iteration, that could lead to a segfault, especially if iCurrElt then points to the then-partial record. Not very likely, but could happen in practice.

  • bugfix (theoretical): potential segfault in omfile under low memory condition. This is only a theoretical bug, because it would only happen when strdup() fails to allocate memory – which is highly unlikely and will probably lead to all other sorts of errors.
  • bugfix: comment char (‘#’) in literal terminated script parsing and thus could not be used. bugtracker: http://bugzilla.adiscon.com/show_bug.cgi?id=119
    [merged in from v3.22.2]

On the Use of English

I ventured to write this book in English because …
it will be more easily read in poor English,
than in good German by 90% of my intended readers.
— HANS J. STETTER, Analysis of Discretization Methods for
Ordinary Differential Equations (1973)

There is not much I could add to Mr. Stetter’s thought, except, maybe, that the number to quote probably tends more to 99% in this case than to the 90% Mr. Stetter notes. So please pardon those errors in language use that I have not yet been able to fix or even see. Suggestions for corrections and improvements are always welcome.

Assumed Standard rsyslog.conf

When I initially started to write this book, I provided only excerpts of rsyslog.conf that showed which lines you had to add. Quickly, I received feedback that this is inadequate to make the recipies easy to use – because everyone still needed to guess where to place the excerpts. Or, even worse, place them at the wrong spots, not knowing that sequence of statements is important in rsyslog.conf.

To solve this issue, I now use a “standard” syslog config, just like can be found on many systems. Your’s may not be exactly the same, but I guess the standard config helps you find where the configuration samples need to go into. As a reference point, this is the “standard” config that all samples build on:

# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none      /var/log/messages
# The authpriv file has restricted access.
authpriv.*                                    /var/log/secure
# Log all the mail messages in one place.
mail.*                                        /var/log/maillog
# Log cron stuff
cron.*                                        /var/log/cron
# Everybody gets emergency messages
*.emerg                                       *
# Save news errors of level crit and higher in a special file.
uucp,news.crit                                /var/log/spooler
# Save boot messages also to boot.log
local7.*                                      /var/log/boot.log

I was hesitant at first to use a real-life sample – repeating it in any recipe requires much space. However, I have convinced myself that this space is well spent, facilitating the adoption of the samples. Only occasionally, when I give some counter-examples, I spare myself from reproducing the full standard rsyslog.conf, hoping that the context makes clear what is meant.

In each recipe, a full rsyslog.conf, based on above example, is specified. The recipe-specific alterations to the config file are given in a different typeface, so that they can be easily identified.

Writing specific messages to a file and discarding them

Messages with the text “error” inside the text part of the message shall be written to a specific file. They shall not be written to any other file or be processed in any other way.

Things to think about

The configuration given here should be placed on top of the rsyslog.conf file.

Config Statements

:msg, contains, "error" /var/log/error.log
& ~
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none /var/log/messages
# The authpriv file has restricted access.
authpriv.* /var/log/secure
# Log all the mail messages in one place.
mail.* /var/log/maillog
# Log cron stuff
cron.* /var/log/cron
# Everybody gets emergency messages
*.emerg *
# Save news errors of level crit and higher in a special file.
uucp,news.crit /var/log/spooler
# Save boot messages also to boot.log
local7.* /var/log/boot.log

How it works

The configuration uses a property-based filter to see if the string “error is contained” inside the MSG part of the syslog message. If so, the message is written to /var/log/error.log. The next line then discards all messages that have been written. Thus, no additional rules will be applied to the message. As such, it will not be written to /var/log/other.log.

Note the difference to this invalid sequence:

*.* /var/log/other.log
:msg, contains, "error" /var/log/error.log
& ~

Here everything is first written to /var/log/other.log and only then the message content is checked. In the later case, the message with “error” in them will be written to both files.

Important

Sequence of configuration statements is very important. Invalid sequence of otherwise perfectly legal configuration statements can lead to totally wrong results.

Sending Messages to a Remote Syslog Server

In this recipe, we forward messages from one system to another one. Typical use cases are:

  • the local system does not store any messages (e.g. has not sufficient space to do so)
  • there is a (e.g. legal) requirement to consolidate all logs on a single system
  • the server may run some advanced alerting rules, and needs to have a full picture or network activity to work well
  • you want to get the logs to a different system in a different security domain (to prevent attackers from hiding their tracks)
  • and many more …

In our case, we forward all messages to the remote system. Note that by applying different filters, you may only forward select entries to the remote system. Also note that you can include as many forwarding actions as you like. For example, if you need to have a backup central server, you can simply forward to both of them, using two different forwarding actions.

To learn how to configure the remote server, see recipe Receiving Messages from a Remote System.

Config Statements

# this is the simplest forwarding action:
*.* action(type="omfwd" target="192.0.2.1" port="10514" protocol="tcp")
# it is equivalent to the following obsolete legacy format line:
*.* @@192.0.2.1:10514 # do NOT use this any longer!
# Note: if the remote system is unreachable, processing will
# block here and discard messages after a while

# so a better use is
*.*  action(type="omfwd" target="192.0.2.2" port="10514" protocol="tcp"
            action.resumeRetryCount="100"
            queue.type="linkedList" queue.size="10000")
# this will de-couple the sending from the other logging actions,
# and prevent delays when the remote system is not reachable. Also,
# it will try to connect 100 times before it discards messages as
# undeliverable.
# the rest below is more or less a plain vanilla rsyslog.conf as 
# many distros ship it - it's more for your reference...
# Log anything (except mail) of level info or higher.
# Don't log private authentication messages!
*.info;mail.none;authpriv.none;cron.none      /var/log/messages
# The authpriv file has restricted access.
authpriv.*                                    /var/log/secure
# Log all the mail messages in one place.
mail.*                                        /var/log/maillog
# Log cron stuff
cron.*                                        /var/log/cron
# Everybody gets emergency messages
*.emerg                                       :omusrmsg:*
# Save news errors of level crit and higher in a special file.
uucp,news.crit                                /var/log/spooler
# Save boot messages also to boot.log
local7.*                                      /var/log/boot.log

Things to think about

You need to select the protocol best suitable for your use case. If in doubt, TCP is a decent choice. This recipe uses TCP for that reason.

TCP forwarding is a build-in capability and always present. As such, no plugin needs to be loaded. The target can be specified by DNS name or IP address. Use IP addresses for most robust operations. If you use a DNS name and name resolution fails, forwarding may be disabled for some time. DNS resolution typically fails on the DNS server itself during system startup.

In this example, we forward to port 10514. We could as well remove the port=”…” parameter from the configuration, which would result in the default port being used. However, you need to specify the port address on the server in any case. So it is strongly advised to use an explicit port number to make sure that client and server configuration match each other (if they used different ports, the message transfer would not work.

Scroll to top