The rocket-fast system for log processing
We have released rsyslog 8.26.0. This release has liblognorm 2.0.3 as a build requirement. Also, there were quite some changes like internal error messages are enabled at all times and many more fixes and additions to modules like imrelp, imptcp, omfwd and many many more. For a complete list of changes, fixes and enhancements, please […]
Original post: Monitoring rsyslog with Kibana and SPM by @Sematext A while ago we published this post where we explained how you can get stats about rsyslog, such as the number of messages enqueued, the number of output errors and so on. The point was to send them to Elasticsearch (or Logsene, our logging SaaS, […]
Original post: Recipe: Apache Logs + rsyslog (parsing) + Elasticsearch by @Sematext This recipe is about tailing Apache HTTPD logs with rsyslog, parsing them into structured JSON documents, and forwarding them to Elasticsearch (or a log analytics SaaS, like Logsene, which exposes the Elasticsearch API). Having them indexed in a structured way will allow you […]
We have just released 8.1.2 of the v8-devel branch. This is primarily a maintenance release, but offers some improvements. Thanks to Pavel Levshin, mmnormalize now offers higher performance and we have added the ability to link directly to jemalloc. Also, usability of queue settings has been improved by using better (and dynamic) defaults for a […]
Version 8.1.2 [devel] 2013-11-28 support for liblognorm1 added – results in performance improvements Thanks to Pavel Levshin for his work in this regard. support for jemalloc added via –enable-jemalloc Thanks to Pavel Levshin for suggesting jemalloc queue defaults have changed high water mark is now dynamically 90% of queue size low water makr […]
This time, we want to parse log messages received from a Cisco ASA for specific parts. The log messages will be parsed by liblognorm and then they will be written into a specific file resembling the sent code. This guide has been tested with v7.3.4 of rsyslog. Things to think about We basically need a […]
When trying to normalize log messages via liblognorm and mmnormalize, you need to create a rulebase first. The rulebase is usually a representation of message formats. Due to the format of these rules, it is necessary to be cautious. Some messages and rule necessities could possibly cause confusion to the configuration interpreter. This mainly applies […]
Log normalization is simple, but has its quirks. A common pitfall is syslog message format as induced by RFC3164. Let’s look at a common case: A log message has been sent to rsyslog. The message itself had no irregular characters. But, the message that should have been parsed by mmnormalize now has a leading space […]
Using the mmnormalize module in rsyslog is a bit complicated at first. We want to describe in this article how to set up the basic components for using log normalization. In addition to that we will show how to configure these components so messages will be split into pieces of information. These pieces of information […]
Hi all, We have just released rsyslog 6.1.2, the new v6-devel. This release has as first release the experimental support for log