Changelog for 5.8.2. (v5-stable)
Version 5.8.2 [V5-stable] (rgerhards), 2011-06-21
-  bugfix: problems in failover action handling
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=270
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=254 -  bugfix: mutex was invalidly left unlocked during action processing
At least one case where this can occur is during thread shutdown, which
may be initiated by lower activity. In most cases, this is quite
unlikely to happen. However, if it does, data structures may be
corrupted which could lead to fatal failure and segfault. I detected
this via a testbench test, not a user report. But I assume that some
users may have had unreproducable aborts that were cause by this bug. -  bugfix: memory leak in imtcp & subsystems under some circumstances
This leak is tied to error conditions which lead to incorrect cleanup
of some data structures. [backport from v6] -  bugfix/improvement:$WorkDirectory now gracefully handles trailing slashes
 
Changelog for 6.1.9
Version 6.1.9 [BETA] (rgerhards), 2011-06-14
-  bugfix: memory leak in imtcp & subsystems under some circumstances
This leak is tied to error conditions which lead to incorrect cleanup
of some data structures. [backport from v6.3] -  bugfix: $ActionFileDefaultTemplate did not work
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=262
 
Changelog for 5.9.0 (v5-devel)
Version 5.9.0 [V5-DEVEL] (rgerhards), 2011-06-08
- imfile: added $InputFileMaxLinesAtOnce directive
 - enhanced imfile to support input batching
 -  added capability for imtcp and imptcp to activate keep-alive packets
at the socket layer. This has not been added to imttcp, as the latter is
only an experimental module, and one which did not prove to be useful.
reference: http://kb.monitorware.com/post20791.html -  added support to control KEEPALIVE settings in imptcp
this has not yet been added to imtcp, but could be done on request. -  $ActionName is now also used for naming of queues in impstats
as well as in the debug output -  bugfix: do not open files with full privileges, if privs will be dropped
This make the privilege drop code more bulletproof, but breaks Ubuntu’s
work-around for log files created by external programs with the wrong
user and/or group. Note that it was long said that this “functionality”
would break once we go for serious privilege drop code, so hopefully
nobody still depends on it (and, if so, they lost…). - bugfix: pipes not opened in full priv mode when privs are to be dropped
 - this begins a new devel branch for v5
 -  better handling of queue i/o errors in disk queues. This is kind of a
bugfix, but a very intrusive one, this it goes into the devel version
first. Right now, “file not found” is handled and leads to the new
emergency mode, in which disk action is stopped and the queue run
in direct mode. An error message is emited if this happens. - added support for user-level PRI provided via systemd
 -  added new config directive $InputTCPFlowControl to select if tcp
received messages shall be flagged as light delayable or not. -  enhanced omhdfs to support batching mode. This permits to increase
performance, as we now call the HDFS API with much larger message
sizes and far more infrequently -  bugfix: failover did not work correctly if repeated msg reduction was on
affected directive was: $ActionExecOnlyWhenPreviousIsSuspended on
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=236 
Changelog for 5.8.1 (v5-stable)
Version 5.8.1 [V5-stable] (rgerhards), 2011-05-19
-  bugfix: invalid processing in QUEUE_FULL condition
If the the multi-submit interface was used and a QUEUE_FULL condition
occured, the failed message was properly destructed. However, the
rest of the input batch, if it existed, was not processed. So this
lead to potential loss of messages and a memory leak. The potential
loss of messages was IMHO minor, because they would have been dropped
in most cases due to the queue remaining full, but very few lucky ones
from the batch may have made it. Anyhow, this has now been changed so
that the rest of the batch is properly tried to be enqueued and, if
not possible, destructed. -  new module mmsnmptrapd, a sample message modification module
This can be useful to reformat snmptrapd messages and also serves as
a sample for how to write message modification modules using the
output module interface. Note that we introduced this new
functionality directly into the stable release, as it does not
modify the core and as such cannot have any side-effects if it is
not used (and thus the risk is solely on users requiring that
functionality). -  bugfix: rate-limiting inside imuxsock did not work 100% correct
reason was that a global config variable was invalidly accessed where a
listener variable should have been used.
Also performance-improved the case when rate limiting is turned off (this
is a very unintrusive change, thus done directly to the stable version). -  bugfix: $myhostname not available in RainerScript (and no error message)
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=233 -  bugfix: memory and file descriptor leak in stream processing
Leaks could occur under some circumstances if the file stream handler
errored out during the open call. Among others, this could cause very
big memory leaks if there were a problem with unreadable disk queue
files. In regard to the memory leak, this
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=256 -  bugfix: doc for impstats had wrong config statements
also, config statements were named a bit inconsistent, resolved that
problem by introducing an alias and only documenting the consistent
statements
Thanks to Marcin for bringing up this problem. -  bugfix: IPv6-address could not be specified in omrelp
this was due to improper parsing of “:”
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=250 -  bugfix: TCP connection invalidly aborted when messages needed to be
discarded (due to QUEUE_FULL or similar problem) -  bugfix: $LocalHostName was not honored under all circumstances
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=258 -  bugfix(minor): improper template function call in syslogd.c
 
Changelog for 5.7.10 (v5-beta)
Version 5.7.10 [V5-BETA] (rgerhards), 2011-03-29
-  bugfix: ompgsql did not work properly with ANSI SQL strings
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=229 -  bugfix: rsyslog did not build with –disable-regexp configure option
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=243 - bugfix: PRI was invalid on Solaris for message from local log socket
 - enhance: added $BOM system property to ease writing byte order masks
 -  bugfix: RFC5424 parser confused by empty structured data
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=237 -  bugfix: error return from strgen caused abort, now causes action to be
ignored (just like a failed filter) -  new sample plugin for a strgen to generate sql statement consumable
by a database plugin -  bugfix: strgen could not be used together with database outputs
because the sql/stdsql option could not be specified. This has been
solved by permitting the strgen to include the opton inside its name.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=195 
Using the Text File Input Module
Log files should be processed by rsyslog. Here is some information on how the file monitor works. This will only describe setting up the Text File Input Module. Further configuration like processing rules or output methods will not be described.
Things to think about
The configuration given here should be placed on top of the rsyslog.conf file.
Config Statements
module(load="imfile" PollingInterval="10") # needs to be done just once. PollingInterval is a module directive and is only set once when loading the module
# File 1 input(type="imfile" File="/path/to/file1" Tag="tag1" StateFile="/var/spool/rsyslog/statefile1" Severity="error" Facility="local7") # File 2 input(type="imfile" File="/path/to/file2" Tag="tag2" StateFile="/var/spool/rsyslog/statefile2") # ... and so on ... #
How it works
The configuration for using the Text File Input Module is very extensive. At the beginning of your rsyslog configuration file, you always load the modules. There you need to load the module for Text File Input as well. Like all other modules, this has to be made just once. Please note that the directive PollingInterval is a module directive which needs to be set when loading the module.
module(load="imfile" PollingInterval="10")
Next up comes the input and its parameters. We configure a input of a certain type and then set the parameters to be used by this input. This is basically the same principle for all inputs:
# File 1 input(type="imfile" File="/path/to/file1" Tag="tag1" StateFile="/var/spool/rsyslog/statefile1" Severity="error" Facility="local7")
File specifies, the path and name of the text file that should be monitored. The file name must be absolute.
Tag will set a tag in front of each message pulled from the file. If you want a colon after the tag you must set it as well, it will not be added automatically.
StateFile will create a file where rsyslog keeps track of the position it currently is in a file. You only need to set the filename. This file always is created in the rsyslog working directory (configurable via $WorkDirectory). This file is important so rsyslog will not pull messages from the beginning of the file when being restarted.
Severity will give all log messages of a file the same severity. This is optional. By default all mesages will be set to “notice”.
Facility gives alle log messages of a file the same facility. Again, this is optional. By default all messages will be set to “local0”.
These statements are needed for monitoring a file. There are other statements described in the doc, which you might want to use. If you want to monitor another file the statements must be repeated.
Since the files cannot be monitored in genuine real time (which generates too much processing effort) you need to set a polling interval:
PollingInterval 10
This is a module setting and it defines the interval in which the log files will be polled. By default this value is set to 10 seconds. If you want this to get more near realtime, you can decrease the value, though this is not suggested due to increasing processing load. Setting this to 0 is supported, but not suggested. Rsyslog will continue reading the file as long as there are unprocessed messages in it. The interval only takes effect once rsyslog reaches the end of the file.
Important
The StateFile needs to be unique for every file that is monitored. If not, strange things could happen.
Changelog for 6.1.5 (v6 devel)
Version 6.1.5 [DEVEL] (rgerhards), 2011-03-04
- improved testbench
 -  enhanced imtcp to use a pool of worker threads to process incoming
messages. This enables higher processing rates, especially in the TLS
case (where more CPU is needed for the crypto functions) - added support for TLS (in anon mode) to tcpflood
 - improved TLS error reporting
 -  improved TLS startup (Diffie-Hellman bits do not need to be generated,
as we do not support full anon key exchange — we always need certs) -  bugfix: fixed a memory leak and potential abort condition
this could happen if multiple rulesets were used and some output batches
contained messages belonging to more than one ruleset.
fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=226
fixes: http://bugzilla.adiscon.com/show_bug.cgi?id=218 -  bugfix: memory leak when $RepeatedMsgReduction on was used
bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=225 -  bugfix: potential abort condition when $RepeatedMsgReduction set to on
as well as potentially in a number of other places where MsgDup() was
used. This only happened when the imudp input module was used and it
depended on name resolution not yet had taken place. In other words,
this was a strange problem that could lead to hard to diagnose
instability. So if you experience instability, chances are good that
this fix will help. 
Changelog for 6.1.4 (devel)
Version 6.1.4 [DEVEL] (rgerhards), 2011-02-18
-  bugfix/omhdfs: directive $OMHDFSFileName rendered unusable
due to a search and replace-induced bug ;) -  bugfix: minor race condition in action.c – considered cosmetic
This is considered cosmetic as multiple threads tried to write exactly
the same value into the same memory location without sync. The method
has been changed so this can no longer happen. - added pmsnare parser module (written by David Lang)
 - enhanced imfile to support non-cancel input termination
 - improved systemd socket activation thanks to Marius Tomaschweski
 -  improved error reporting for $WorkDirectory
non-existance and other detectable problems are now reported,
and the work directory is NOT set in this case - bugfix: pmsnare causded abort under some conditions
 -  bugfix: abort if imfile reads file line of more than 64KiB
Thanks to Peter Eisentraut for reporting and analysing this problem.
bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=221 -  bugfix: queue engine did not properly slow down inputs in FULL_DELAY mode
when in disk-assisted mode. This especially affected imfile, which
created unnecessarily queue files if a large set of input file data was
to process. -  bugfix: very long running actions could prevent shutdown under some
circumstances. This has now been solved, at least for common
situations. -  bugfix: fixed compile problem due to empty structs
this occured only on some platforms/compilers. thanks to Dražen Kačar
for the fix 
Changelog for 5.7.4 (v5-beta)
Version 5.7.4 [V5-BETA] (rgerhards), 2011-02-17
- added pmsnare parser module (written by David Lang)
 - enhanced imfile to support non-cancel input termination
 - improved systemd socket activation thanks to Marius Tomaschweski
 -  improved error reporting for $WorkDirectory
non-existance and other detectable problems are now reported,
and the work directory is NOT set in this case - bugfix: pmsnare causded abort under some conditions
 -  bugfix: abort if imfile reads file line of more than 64KiB
Thanks to Peter Eisentraut for reporting and analysing this problem.
bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=221 -  bugfix: queue engine did not properly slow down inputs in FULL_DELAY mode
when in disk-assisted mode. This especially affected imfile, which
created unnecessarily queue files if a large set of input file data was
to process. -  bugfix: very long running actions could prevent shutdown under some
circumstances. This has now been solved, at least for common
situations. -  bugfix: fixed compile problem due to empty structs
this occured only on some platforms/compilers. thanks to Dražen Kačar
for the fix 
Changelog for 5.6.3 (v5-stable)
Version 5.6.3 [V5-STABLE] (rgerhards), 2011-01-26
- bugfix: action processor released memory too early, resulting inpotential issue in retry cases (but very unlikely due to anotherbug, which I also fixed — only after the fix this problem here became actually visible).
 - bugfix: batch processing flagged invalid message as “bad” under somecircumstances
 - bugfix: unitialized variable could cause issues under extreme conditionsplus some minor nits. This was found after a clang static code analyzeranalysis (great tool, and special thanks to Marcin for telling me about it!)
 - bugfix: batches which had actions in error were not properly retried inall cases
 - bugfix: imfile did duplicate messages under some circumstances
 
