Tutorial: Sending impstats Metrics to Elasticsearch Using Rulesets and Queues

Originally posted on the Sematext blog: Monitoring rsyslog’s Performance with impstats and Elasticsearch If you’re using rsyslog for processing lots of logs (and, as we’ve shown before, rsyslog is good at processing lots of logs), you’re probably interested in monitoring it. To do that, you can use impstats, which comes from input module for process […]

Using rsyslog and Elasticsearch to Handle Different Types of JSON Logs

Originally posted on the Sematext blog: Using Elasticsearch Mapping Types to Handle Different JSON Logs By default, Elasticsearch does a good job of figuring the type of data in each field of your logs. But if you like your logs structured like we do, you probably want more control over how they’re indexed: is time_elapsed […]

rsyslog 8.2.0 (v8-stable) released

This is the first release of the greatly improved version 8 of rsyslog. Large parts of the core engine have been rewritten in order to support even greater performance and newly things like global variable support in RainerScript. The new engine is the foundation for the next couple of years of rsyslog technology. As another […]

Changelog for 8.2.0 (v8-stable)

Version 8.2.0 [v8-stable] 2014-04-02 This starts a new stable branch based on 8.1.6 plus the following changes: we now use doc from the rsyslog-doc project As such, the ./doc subtree has been removed. Instead, a cache of the rsyslog-doc project’s files has been included in ./rsyslog-doc.tar.gz. Note that the exact distribution mode for the doc […]

Performance Tuning&Tests for the Elasticsearch Output

Original post: Rsyslog 8.1 Elasticsearch Output Performance by @Sematext Version 8 brings major changes in rsyslog’s core – see Rainer’s presentation about it for more details. Those changes should give outputs better performance, and the Elasticsearch one should benefit a lot. Since we’re using rsyslog and Elasticsearch in Sematext‘s own log analytics product, Logsene, we […]

Output to Elasticsearch in Logstash format (Kibana-friendly)

Original post: Recipe rsyslog+Elasticsearch+Kibana by @Sematext In this post you’ll see how you can take your logs with rsyslog and ship them directly to Elasticsearch (running on your own servers, or the one behind Logsene’s Elasticsearch API) in a format that plays nicely with Logstash. So you can use Kibana to search, analyze and make […]

rsyslog and ElasticSearch

by Micah Yoder, originally published on rackspace. Minor changes through Adiscon. There is a clear benefit to being able to aggregate logs from various servers and services into one place and be able to search them for any sort of arbitrary event. Traditional syslog can aggregate logs, but aggregating events from them sometimes involves grep and […]

Parsing JSON (CEE) Logs and Sending them to Elasticsearch

Original post: Structured Logging with rsyslog and Elasticsearch via @sematext When your applications generate a lot of logs, you’d probably want to make some sense of them through searches and statistics. Here’s when structured logging comes in handy, and I would like to share some thoughts and configuration examples of how you could use a […]

rsyslog statistic counter plugin omelasticsearch

Plugin – omelasticsearch This plugin maintains global statistics, which accumulate all action instances. The statistic is named “omelasticsearch”. Parameters are: submitted – number of messages submitted for processing (with both success and error result) fail.httprequests – the number of times a http request failed. Note that a single http request may be used to submit […]

Changelog for 7.4.5 (v7-stable)

Version 7.4.5 [v7.4-stable] 2013-10-22 mmanon: removed the check for specific “terminator characters” after last octet. As it turned out, this didn’t work in practice as there was an enormous set of potential terminator chars — so removing them was the best thing to do. Note that this may change behaviour of existing installations. Yet, we […]

Scroll to top