devel

Version 5.3.2 [DEVEL] (rgerhards), 2009-10-21

• enhanced omfile to support transactional interface. This will increase performance in many cases.
• added multi-ruleset support to imudp
• re-enabled input thread termination handling that does avoid thread cancellation where possible. This provides a more reliable mode of rsyslogd termination (canceling threads my result in not properly freed resouces and potential later hangs, even though we perform proper cancel handling in our code). This is part of an effort to reduce thread cancellation as much as possible in rsyslog.
  NOTE: the code previously written code for this functionality had a subtle race condition. The new code solves that.

• enhanced immark to support non-cancel input module termination
• improved imudp so that epoll can be used in more environments, fixed potential compile time problem if EPOLL_CLOEXEC is not available.
• some cleanup/slight improvement:
  • changed imuxsock to no longer use deprecated submitAndParseMsg() IF
  • changed submitAndParseMsg() interface to be a wrapper around the new way of message creation/submission. This enables older plugins to be used together with the new interface. The removal also enables us to drop a lot of duplicate code, reducing complexity and increasing maintainability.
• bugfix: segfault when starting up with an invalid .qi file for a disk queue
  Failed for both pure disk as well as DA queues. Now, we emit an error message and disable disk queueing facility.

• bugfix: potential segfault on messages with empty MSG part. This was a recently introduced regression.
• bugfix: debug string larger than 1K were improperly displayed. Max size is now 32K, and if a string is even longer it is meaningfully truncated.
  

Version 5.3.0 [DEVEL] (rgerhards), 2009-09-14

• begun to add simple GUI programs to gain insight into running rsyslogd instances and help setup and troubleshooting (active via the –enable-gui ./configure switch)
• changed imudp to utilize epoll(), where available. This shall provide slightly better performance (just slightly because we called select() rather infrequently on a busy system)
• added new config option $ActionWriteAllMarkMessages this option permites to process mark messages under all circumstances, even if an action was recently called. This can be useful to use mark messages as a kind of heartbeat.
• added new config option $InputUnixListenSocketCreatePath to permit the auto-creation of pathes to additional log sockets. This turns out to be useful if they reside on temporary file systems and rsyslogd starts up before the daemons that create these sockets (rsyslogd always creates the socket itself if it does not exist).
• added $LogRSyslogStatusMessages configuration directive permitting to turn off rsyslog start/stop/HUP messages. See Debian ticket http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=463793
• bugfix: hostnames with dashes in them were incorrectly treated as malformed, thus causing them to be treated as TAG (this was a regression introduced from the “rfc3164 strict” change in 4.5.0). Testbench has been updated to include a smaple message with a hostname containing a dash.
• bugfix: strings improperly reused, resulting in some message properties be populated with strings from previous messages. This was caused by an improper predicate check.
• added new config directive $omfileForceChown [import from 4.7.0]
  

Version 5.3.1 [DEVEL] (rgerhards), 2009-09-05

• added $AbortOnUncleanConfig directive – permits to prevent startup when there are problems with the configuration file. See it’s doc for details.
• included some important fixes from v4-stable:
  • bugfix: invalid handling of zero-sized messages
  • bugfix: zero-sized UDP messages are no longer processed
  • bugfix: random data could be appended to message
  • bugfix: reverse lookup reduction logic in imudp do DNS queries too often
• bugfixes imported from 4.5.4:
  • bugfix: potential segfault in stream writer on destruction
  • bugfix: potential race in object loader (obj.c) during use/release
  • bugfixes: potential problems in out file zip writer
    

Version 5.1.4 [DEVEL] (rgerhards), 2009-08-20

• legacy syslog parser changed so that it now accepts date stamps in wrong case. Some devices seem to create them and I do not see any harm in supporting that.
• added $InputTCPMaxListeners directive – permits to specify how many TCP servers shall be possible (default is 20).
• bugfix: memory leak with some input modules. Those inputs that use parseAndSubmitMsg() leak two small memory blocks with every message. Typically, those process only relatively few messages, so the issue does most probably not have any effect in practice.
• bugfix: if tcp listen port could not be created, no error message was emitted
• bugfix: discard action did not work (did not discard messages)
• bugfix: discard action caused segfault
• bugfix: potential segfault in output file writer (omfile) In async write mode, we use modular arithmetic to index the output buffer array. However, the counter variables accidently were signed, thus resulting in negative indizes after integer overflow. That in turn could lead to segfaults, but was depending on the memory layout of the instance in question (which in turn depended on a number of variables, like compile settings but also configuration). The counters are now unsigned (as they always should have been) and so the dangling mis-indexing does no longer happen. This bug potentially affected all installations, even if only some may actually have seen a segfault.
  

Version 5.1.3 [DEVEL] (rgerhards), 2009-07-28

• architecture change: queue now always has at least one worker thread if not running in direct mode. Previous versions could run without any active workers. This simplifies the code at a very small expense. See v5 compatibility note document for more in-depth discussion.
• enhance: UDP spoofing supported via new output module omudpspoof. See the omudpspoof documentation for details and samples
• bugfix: message could be truncated after TAG, often when forwarding. This was a result of an internal processing error if maximum field sizes had been specified in the property replacer.
• bugfix: minor static memory leak while reading configuration did NOT leak based on message volume
• internal: added ability to terminate input modules not via pthread_cancel but an alternate approach via pthread_kill. This is somewhat safer as we do not need to think about the cancel-safeness of all libraries we use. However, not all inputs can easily supported, so this now is a feature that can be requested by the input module (the most important ones request it).
  

Version 4.5.1 [DEVEL] (rgerhards), 2009-07-15

• CONFIG CHANGE: $HUPisRestart default is now “off”. We are doing this to support removal of restart-type HUP in v5.
• bugfix: fromhost-ip was sometimes truncated
• bugfix: potential segfault when zip-compressed syslog records were received (double free)
• bugfix: properties inputname, fromhost, fromhost-ip, msg were lost when working with disk queues
• performance enhancement: much faster, up to twice as fast (depending on configuration)
• bugfix: abort condition when RecvFrom was not set and message reduction was on. Happend e.g. with imuxsock.
• added $klogConsoleLogLevel directive which permits to set a new console log level while rsyslog is active
• bugfix: message could be truncated after TAG, often when forwarding This was a result of an internal processing error if maximum field sizes had been specified in the property replacer.
• added ability for the TCP output action to “rebind” its send socket after sending n messages (actually, it re-opens the connection, the name is used because this is a concept very similiar to $ActionUDPRebindInterval). New config directive actionSendTCPRebindInterval added for the purpose. By default, rebinding is disabled. This is considered useful for load balancers.
• testbench improvements

Version 5.1.2 [DEVEL] (rgerhards), 2009-07-08

• bugfix: properties inputname, fromhost, fromhost-ip, msg were lost when working with disk queues
• some performance enhancements
• bugfix: abort condition when RecvFrom was not set and message reduction was on. Happend e.g. with imuxsock.
• added $klogConsoleLogLevel directive which permits to set a new console log level while rsyslog is active
• some internal code cleanup

Version 5.1.1 [DEVEL] (rgerhards), 2009-07-03

• bugfix: huge memory leak in queue engine (made rsyslogd unusable in production). Occured if at least one queue was in direct mode (the default for action queues)
• imported many performance optimizations from v4-devel (4.5.0)
• bugfix: subtle (and usually irrelevant) issue in timout processing timeout could be one second too early if nanoseconds wrapped
• set a more sensible timeout for shutdow, now 1.5 seconds to complete processing (this also removes those cases where the shutdown message was not written because the termination happened before it)
• numerous enhancements and fixes imported from the v4 branch
  

Version 4.5.0 [DEVEL] (rgerhards), 2009-07-02

• activation order of inputs changed, they are now activated only after privileges are dropped. Thanks to Michael Terry for the patch.
• greatly improved performance
• greatly reduced memory requirements of msg object to around half of the previous demand. This means that more messages can be stored in core! Due to fewer cache misses, this also means some performance improvement.
• improved config error messages: now contain a copy of the config line that (most likely) caused the error
• reduced max value for $DynaFileCacheSize to 1,000 (the former maximum of 10,000 really made no sense, even 1,000 is very high, but we like to keep the user in control ;)).
• added capability to fsync() queue disk files for enhanced reliability (also add’s speed, because you do no longer need to run the whole file system in sync mode)
• more strict parsing of the hostname in rfc3164 mode, hopefully removes false positives (but may cause some trouble with hostname parsing). For details, see this bug tracker: http://bugzilla.adiscon.com/show_bug.cgi?id=126
• added configuration commands (see doc for explanations)
  * $OMFileZipLevel
  * $OMFileIOBufferSize
  * $OMFileFlushOnTXEnd
  * $MainMsgQueueSyncQueueFiles
  * $ActionQueueSyncQueueFiles

• done some memory accesses explicitely atomic
• bugfix: subtle (and usually irrelevant) issue in timout processing timeout could be one second too early if nanoseconds wrapped
• set a more sensible timeout for shutdow, now 1.5 seconds to complete processing (this also removes those cases where the shutdown message was not written because the termination happened before it)
• internal bugfix: object pointer was only reset to NULL when an object was actually destructed. This most likely had no effect to existing code, but it may also have caused trouble in remote cases. Similarly, the fix may also cause trouble…

Version 5.1.0 [DEVEL] (rgerhards), 2009-05-29

• redesigned queue engine so that it supports ultra-reliable operations. This resulted in a rewrite of large parts. The new capability can be used to build audit-grade systems on the basis of rsyslog.
• added $MainMsgQueueDequeueBatchSize and $ActionQueueDequeueBatchSize configuration directives
• implemented a new transactional output module interface which provides superior performance (for databases potentially far superior performance)
• increased ompgsql performance by adapting to new transactional output module interface