librelp 1.0.6
librelp 1.0.6 [download]: enhanced API to permit binding a client to a specific IP address [sha256 hash: 9b4304ff3302c60be03bb9bed82d15420bce51b72f836877e47622cd5417f6ab]
librelp 1.0.5
librelp 1.0.5 [download]: bugfix for compile problem on Solaris (Thanks to John Lyman for the patch) [sha256 hash: 5e98bf47d81cbe1ec0ea99fc42595943904c0cac11d5966408e4afcbca34d972]
librelp 1.0.3
librelp 1.0.3 [download]: added support for user-settable shutdown indicator, improved timeout handling [sha256 hash: bc052f061d79a0254b29a38b15d32757561c6e83a8fa91598943098b18aa9bff]
librelp 1.0.2
librelp 1.0.2 [download]: added capability to support only IPv4/v6 instead of both [sha256 hash:d8ef77cb00f7d4db9ff96157242b3e4fb0eb0d68d7e846fcf12dc5fed92e4d4a]
BSD-Style blocks will go away in rsyslog v7
Rsyslog supports BSD-style blocks since ages. They were a pretty handy tool to group actions together that should act only on remote hosts or log messages from specific programs. However, the v7 config system with its full nesting capabilities provides a much better – and easy to use – way to specify this. If both systems are mixed, the problem is that BSD-style blocks can be used to violate the nesting structure (examples below). Also, support for them adds a lot to rule engine code complexity. And finally, they are also very seldom used, few users even know they exist.
As a result, I have decided to drop support for BSD-style blocks in rsyslog v7 and above. A poll on the mailing list a week ago did not make anybody speak up against that change. So I assume none is hurt. This is especially the case as the conversion of BSD-style blocks to nested config is a very easy one.
Let’s look at this example:
!prog1 *.* /var/log/prog1.log *.* /var/log/prog1again.log !prog2 *.* /var/log/prog2.log *.* /var/log/prog2again.log
if $programname == 'prog1' then {
/var/log/prog1.log
/var/log/prog1again.log
}
if $programname == 'prog2' then {
/var/log/prog2.log
/var/log/prog2again.log
}And if you prefer the more powerful action statments (probably not so much benefit for this use case…), you can write:
if $programname == 'prog1' then {
action(type="omfile" file="/var/log/prog1.log")
action(type="omfile" file="/var/log/prog1again.log")
}
if $programname == 'prog2' then {
action(type="omfile" file="/var/log/prog2.log")
action(type="omfile" file="/var/log/prog2again.log")
}I expect that usually these easy cases happen. HOWEVER, if I had kept support for BSD-style blocks, one could configure things like this:
!prog1
if $msg contains 'test' then {
action(type="omfile" file="/var/log/somefile")
!prog2
mail.* :mmjsonparse:
& action(type="omfile" file="/var/log/somefile2")
!prog3
& ~
!prog4
if $msg contains 'test2' then
/var/log/logfile
else
/var/log/logfile2
}Can you make out the actual nesting structure of this config? When, for example, programname needs to be “prog3” and what happens then? IMHO this combination can cause considerable user confusion and frustration. As such, I made a sharp cut and removed it.
librelp 1.0.1
librelp 1.0.1 [download]: capability to stop server without canceling its thread, bugfix: interrupt of select() not properly handled.
Rsyslog Windows Agent Released
The rsyslog Windows agent enables customers to integrate Windows Event Logs into their central rsyslog-based logging system. Even more, it supports acting as a syslog relay as well. This enables small branch offices only runnig Windows to provide local syslog sources to the central server as well.
We have released the first full-featured version of this product today. It is based on Adiscon’s mature suite of Windows-to-syslog tools. Most importantly, it provides the ability to fully extract Windows Event Log data, including local resolution of user IDs, SIDs, GUIDs and similiar objects. It not only supports the current Windows Event Log format but also the legacy (pre Windows-2008/Vista) event log system. The technology behind this tool is already in use at thousands of customer sites for many years (in the form of, for example, EventReporter and other Adiscon products).
The rsyslog Agent for Windows has been highly optimized for use with rsyslog at the back end. For example, it supports mutually-authenticated, TLS encrypted syslog as well as the RELP protocol for reliable delivery. Also, it provides an initial implementation of cee-enhanced syslog, and is the world’s first-ever solution to do so (details can be found in the configuration guide for cee-enhanced syslog).
By purchasing rsyslog Agent for Windows customers not only get first class event log integration, they also help fund further improving the rsyslog solution as whole. The package even contains limited rsyslog professional services.
A trial version can be downloaded from the rsyslog Windows agent page.
TCP syslog / RFC5424 log4j appender
As part of a training project, we created a small log4j appender supporting TCP syslog and RFC5424. Most importantly, it is capable of formatting stack traces as a single syslog messages (NOT the usual bunch of multiple malformed messages). The work is based on the syslog4j implementation, which did not work for us (our fault? ;)) and so we extended this framework. Note that the code has never been used in practice by us, so there may be some risk associated – but that should be fairly easy to asses, given that the source is pretty small.
The download of the RFC5424 syslog tcp appender contains it all: doc, jar files as well as the source.
Feedback and usage reports are appreciated, please send mail to ugerhards@adiscon.com (if you run into troubles, you can also use the same email address to discuss the problem!).
rsyslog statistics counter
Rsyslog supports statistic counters via the impstats module. It is important to know that impstats and friends only provides an infrastructure where core components and plugins can register statistics counter. This FAQ entry tries to describe all counters available, but please keep in mind that there may exist that we do not know about.
When interpreting rsyslog statistics, please keep in mind that statistics records are processed as regular syslog messages. As such, the statistics messages themselves increment counters when they are emitted via the regular syslog stream, which is the default (and so counters keep slowly increasing even if there is absolutely no other traffic). Also keep in mind that a busy rsyslog system is very dynamic. Most importantly, this means that the counters may not be 100% consistent, but some slight differences may exist. Avoiding such inconsistencies would be possible only at the price of a very tight locking discipline, which would cause serious performance bottlenecks. Thus, this is not done. Finally, though extremely unlikely, some counters may experience an overflow and restart at 0 for that reasons. However, most counters are 64-bit, so this is extremely unlikely. Those which are not 64 bit are typically taken from some internal data structure that uses lower bits for performance reasons and guards against overflow.
The listing starts with the core component or plugin that creates the counters and than specifies various counters that exist for the sub-entities. The listing below is extended as new counters are added. Some counters probably do not exist in older releases of rsyslog.
Below you can find all available core components and plugins. Please note that every core component or plugin are linked to a information site.
Queue
Actions
PLUGINS
LogTools 0.1.0 Released
We are happy to announce the initial public release of LogTools. This package aims at providing useful tools for log file manipulation and processing. The initial release focuses on a core ingredient, the tamper-proof log store. Besides that, the well-documented format is designed for long-term archiving. Both features together make it very useful for storing both valuable audit information as well as log files potentially meant as evidence in court.
Both the source tarball and an experimental Debian package are available via the LogTools download page.
The log store is tamper-proof by using a chain of hashes. This provides tamper protection if the last hash of the hash chain is regularly saved. In upcoming releases, the hash chain will become protected by digital signatures using strong cryptography. With that, it is no longer necessary (but still beneficial) to keep hashes in a separate location (that would guard even against a compromised key).
The log store is long-term-archival-friendly because its documented format is designed to remain easy to read and interpret even in many years: it is text-based, clearly documented (in the man page or, for example, here) and easily extensible. To support extensibility, it supports different record types and a cryptographic field that can be extended by type specifiers as well (this is not used initially because it is not needed – but the capability is already present). It is our pledge that this format will remain readable while it is being extended in the future.
So LogStore format is a perfect solution for tamper-protected texttual data that must last. And note that we say “textual data”, not “syslog messages” or even “log messages”. No matter what it is, if it is in text format, LogStore can protect it!
LogTools are also a good Linux citizen, following the Unix tradition of small tools that do one job and do it well. With the initial release, two filter-type programs (logreader and logwriter) have been released. They permit to create and read log stores (including a functionality to extract the last used hash for easy transmission). These tools can perfectly be used inside processing pipes. That, among others, means that LogStore data is perfectly “greppable”. Integration with other tools is simple. For example, to make rsyslog write LogStore format, rsyslog’s omprog output module can be utilized. All tools that can emit data to stdout can also be used to write LogStore format. So immediately there is a myriad of options available.
LogTools is an active project that will be further extend with new tools with the same philosophy. One of the next steps planned is to integrate liblognorm techology for easy normalization of incoming semi-structured data. We also appreciate suggestions, so please make yourself heard.