Coupling with Logstash via Redis
Original post: Recipe: rsyslog + Redis + Logstash by @Sematext
OK, so you want to hook up rsyslog with Logstash. If you don’t remember why you want that, let me give you a few hints:
- Logstash can do lots of things, it’s easy to set up but tends to be too heavy to put on every server
- you have Redis already installed so you can use it as a centralized queue. If you don’t have it yet, it’s worth a try because it’s very light for this kind of workload.
- you have rsyslog on pretty much all your Linux boxes. It’s light and surprisingly capable, so why not make it push to Redis in order to hook it up with Logstash?
In this post, you’ll see how to install and configure the needed components so you can send your local syslog (or tail files with rsyslog) to be buffered in Redis so you can use Logstash to ship them to Elasticsearch, a logging SaaS like Logsene (which exposes the Elasticsearch API for both indexing and searching) so you can search and analyze them with Kibana:
rsyslog 8.13.0 (v8-stable) released
We have released rsyslog 8.13.0.
This release sports a big number of changes. While most are bugfixes, there are also some additions to existing functionality, most notably the enhancements for ZeroMQ and Redis modules.
http://www.rsyslog.com/changelog-for-8-13-0-v8-stable/
Download:
http://www.rsyslog.com/downloads/download-v8-stable/
As always, feedback is appreciated.
Best regards,
Florian Riedl
Changelog for 7.3.11 (v7-devel)
Version 7.3.11 [devel] 2013-04-23
- added support for encrypting log files
- omhiredis: added support for redis pipeline support
Thanks to Brian Knox for the patch. - bugfix: $PreserveFQDN is not properly working
Thanks to Louis Bouchard for the patch
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=426 - bugfix: imuxsock aborted due to problem in ratelimiting code
Thanks to Tomas Heinrich for the patch. - bugfix: imuxsock aborted under some conditions
regression from ratelimiting enhancements – this was a different one to the one Tomas Heinrich patched. - bugfix: timestamp problems in imkmsg