Changelog for 8.1.6 (v8-devel)
Version 8.1.6 [devel] 2014-02-20
- omfile: permit to set global defaults for action parameters
Thanks to Nathan Brown for the patch.
See also: https://github.com/rsyslog/rsyslog/pull/23 - add capability to escape control characters in the C way of doing it
adds new global parameter “parser.escapeControlCharactersCStyle”
Thanks to Nathan Brown for the patch.
See also: https://github.com/rsyslog/rsyslog/pull/13 - parser global parameters can now be set using RainerScript global()
Thanks to Nathan Brown for the patch.
See also: https://github.com/rsyslog/rsyslog/pull/23 - omprog: guard program-to-be-executed against CTL-C
This can frequently happen in debug mode, where rsyslog is terminated
by ctl-c. In any case, SIGINT is not meant to control the child process,
so it should be blocked. - omprog bugfix: parameter “forceSingleInstance” is NOT mandatory
- add new jsonr property replacer option
Thanks to Nathan Brown for the patch. - added external plugin interface
- ommongodb: add authentication support (untested)
Thanks to JT for the patch.
See also: https://github.com/rsyslog/rsyslog/pull/17 - bugfix: json templates are improperly created
Strings miss the terminating NUL character, which obviously can lead
to all sorts of problems.
See also: https://github.com/rsyslog/rsyslog/issues/27
Thanks to Alain for the analysis and the patch. - ompgsql bugfix: improper handling of auto-backgrounding mode
If rsyslog was set to auto-background itself (default code behaviour, but
many distros now turn it off for good reason), ompgsql could not
properly connect. This could even lead to a segfault. The core reason
was that a PG session handle was kept open over a fork, something that
is explicitely forbidden in the PG API.
Thanks to Alain for the analysis and the patch. - bugfix: ommongodb’s template parameter was mandatory but should have
been optional
Thanks to Alain for the analysis and the patch. - bugfix: end of batch processing was not 100% correct. Could lead to
outputs not properly wirting messages. At least omelasticsearch did not
write anything to the database due to this bug.
See: https://github.com/rsyslog/rsyslog/issues/10
Thanks to Radu Gheorghe for reporting the issue.
RSYSLOG 8.1.6 (v8-devel)
Download file name:RSYSLOG 8.1.6 (devel)
RSYSLOG 8.1.6 (devel)
sha256 hash: 7505609b461813e5fea1168d386e4d07e3678c923c2b449684e95d07b6b71849
Author: Rainer Gerhards (rgerhards@adiscon.com)
Version: 8.1.6 File size: 2.511 MB
rsyslog 8.1.6 (v8-devel) released
We have just released 8.1.6 of the v8-devel branch.
This is the release candidate for v8.2.0 stable. It contains a number if smaller enhancements as well as bug fixes. We plan to release 8.2.0 soon. So we would appreciate if you could report any bugs you find in that version.
ChangeLog:
http://www.rsyslog.com/changelog-for-8-1-6-v8-devel/
Download:
http://www.rsyslog.com/rsyslog-8-1-6-v8-devel/
Feedback is *very much* appreciated.
Best regards,
Tim Eifler
rsyslog statistic counter Queues
Queue
For each queue inside the system its own set of statistics counters is created. If there are multiple action (or main) queues, this can become a rather lengthy list. The stats record begins with the queue name (e.g. “main Q” for the main queue; ruleset queues have the name of the ruleset they are associated to, action queues the name of the action).
- size – currently active messages in queue
- enqueued – total number of messages enqueued into this queue since startup
- maxsize – maximum number of active messages the queue ever held
- full – number of times the queue was actually full and could not accept additional messages
- discarded.full – number of messages discarded because the queue was full
- discarded.nf – number of messages discarded because the queue was nearly full. Starting at this point, messages of lower-than-configured severity are discarded to save space for higher severity ones.
rsyslog statistic counter plugin imuxsocks
Plugin – imuxsock
This plugin maintains a global statistics with the following properties:
- submitted – total number of messages submitted for processing since startup
- ratelimit.discarded – number of messages discarded due to rate limiting
- ratelimit.numratelimiters – number of currently active rate limiters (small data structures used for the rate limiting logic)
rsyslog statistic counter plugin imudp
Plugin – imudp
This plugin maintains statistics for each listener and for each worker thread.
The listener statistic is named starting with “imudp”, followed followed by the listener IP, a colon and port in parenthesis. For example, the counter for a listener on port 514 (on all IPs) with no set name is called “imudp(*:514)”.
If an “inputname” is defined for a listener, that inputname is used instead of “imudp” as statistic name. For example, if the inputname is set to “myudpinut”, that corresponding statistic name in above case would be “myudpinput(*:514)”. This has been introduced in 7.5.3.
The following properties are maintained for each listener:
- submitted – total number of messages submitted for processing since startup
The worker thread (in short: worker) statistic is named “imudp(wX)” where “X” is the worker thread ID, which is an monotonically increasing integer starting at 0. This means the first worker will have the name “imudp(w0”), the second “imudp(w1)” and so on. Note that workers are all equal. It doesn’t really matter which worker processes which messages, so the actual worker ID is not of much concern. More interesting is to check how the load is spread between the worker. Also note that there is no fixed worker-to-listener relationship: all workers process messages from all listeners.
Note: worker thread statistics are available starting with rsyslog 7.5.5.
The following properties are maintained for each worker thread:
- called.recvmmsg – number of recvmmsg() OS calls done
- called.recvmsg – number of recvmsg() OS calls done
- msgs.received – number of actual messages received
Note: usually either one of “called.recvmmsg” or “called.recvmsg” is non-zero. This is because these are alternative kernel interfaces and the one to be used is selected based on kernel capabilities. However, there are two counters, as the system call type used has important performance implications, and so we thought this information needs to be exposed.
On systems supporting recvmmsg, the quotient msgs.received / caled.recvmmsg tells the average number of messages that could be pulled from the kernel buffers with a single syscall.
rsyslog statistic counter plugin imtcp
Plugin – imtcp
This plugin maintains statistics for each listener. The statistic is named after the given input name (or “imtcp” if none is configured), followed by the listener port in parenthesis. For example, the counter for a listener on port 514 with no set name is called “imtcp(514)”.
The following properties are maintained for each listener:
- submitted – total number of messages submitted for processing since startup
rsyslog statistic counter plugin imptcp
Plugin – imptcp
This plugin maintains statistics for each listener. The statistic is named “imtcp” , followed by the bound address, listener port and IP version in parenthesis. For example, the counter for a listener on port 514, bound to all interfaces and listening on IPv6 is called “imptcp(*/514/IPv6)”.
The following properties are maintained for each listener:
- submitted – total number of messages submitted for processing since startup
rsyslog statistic counter plugin impstats
Plugin – impstats (rsyslog 7.5.3+) The impstats plugin gathers some internal statistics. They have different names depending on the actual statistics. Obviously, they do not relate to the plugin itself but rather to a broader object – most notably the rsyslog process itself. The “resource-usage” counter maintains process statistics. They base on the getrusage() system call. The counters are named like getrusage returned data memebers. So for details, looking them up in “man getrusage” is highly recommended, especially as value may be different depending on the platform. A getrusage() call is done immediately before the counter is emitted. The follwowing individual counters are maintained:
- utime – this is the user time in microseconds (thus the timeval structure combined)
- stime – again, time given in microseconds
- maxrss
- minflt
- majflt
- inblock
- oublock
- nvcsw
- nivcsw
rsyslog statistic counter plugin omfile
Plugin – omfile (rsyslog 7.3.6+)
This plugin maintains statistics for each dynafile cache. Dynafile cache performance is critical for overall system performance, so reviewing these counters on a busy system (especially one experiencing performance problems) is advisable. The statistic is named “dynafile cache”, followed by the template name used for this dynafile action.
The following properties are maintained for each dynafile:
- requests – total number of requests made to obtain a dynafile
- level0 – requests for the current active file, so no real cache lookup needed to be done. These are extremely good.
- missed – cache misses, where the required file did not reside in cache. Even with a perfect cache, there will be at least one miss per file. That happens when the file is being accessed for the first time and brought into cache. So “missed” will always be at least as large as the number of different files processed.
- evicted – the number of times a file needed to be evicted from the cache as it ran out of space. These can simply happen when date-based files are used, and the previous date files are being removed from the cache as time progresses. It is better, though, to set an appropriate “closeTimeout” (counter described below), so that files are removed from the cache after they become no longer accessed. It is bad if active files need to be evicted from the cache. This is a very costly operation as an evict requires to close the file (thus a full flush, no matter of its buffer state) and a later access requires a re-open – and the eviction of another file, as the cache obviously has run out of free entries. If this happens frequently, it can severely affect performance. So a high eviction rate is a sign that the dynafile cache size should be increased. If it is already very high, it is recommended to re-think about the design of the file store, at least if the eviction process causes real performance problems.
- maxused – the maximum number of cache entries ever used. This can be used to trim the cache down to a value that’s actually useful but does not waste resources. Note that when date-based files are used and rsyslog is run for an extended period of time, the cache gradually fills up to the max configured value as older files are migrated out of it. This will make “maxused” questionable after some time. Frequently enough purging the cache can prevent this (usually, once a day is sufficient).
- closetimeouts – available since 8.3.3 – tells how often a file was closed due to timeout settings (“closeTimeout” action parameter). These are cases where dynafiles or static files have been closed by rsyslog due to inactivity. Note that if no “closeTimeout” is specified for the action, this counter always is zero. A high or low number in itself doesn’t mean anything good or bad. It totally depends on the use case, so no general advise can be given.
Note that the dynafile caches are purged when a HUP is sent.