rsyslog

The rocket-fast system for log processing

Changelog for 5.8.9 (v5-stable)

By adisconteamPosted on Posted in ChangelogTagged , , , , , ,

Version 5.8.9 [V5-stable] 2012-03-15

  • added tool to recover disk queue if .qi file is missing (recover_qi.pl)
    Thanks to Kaiwang Chen for contributing this tool
  • bugfix: stopped DA queue was never processed after a restart due to a
    regression from statistics module
  • added better doc for statsobj interface
    Thanks to Kaiwang Chen for his suggestions and analysis in regard to the
    stats subsystem.

rsyslog 5.8.9 (v5-stable) released

By adisconteamPosted on Posted in News, Release AnnouncementTagged , , , , , ,

This is primarily a maintenance release. It also provides a new tool that permits the recovery of disk queues, when the queue information file (.qi) has been lost. Thanks to Kaiwang Chen for contributing this tool.

There was an important flaw in 5.8.8 that caused disk queue content to be lost after a shutdown and restart of rsyslogd. So if you run this version, an upgrade to 5.8.9 is highly suggested.

ChangeLog:

http://www.rsyslog.com/changelog-for-5-8-9-v5-stable/

Download:

http://www.rsyslog.com/rsyslog-5-8-9-v5-stable/

As always, feedback is appreciated.

Best regards,

Tim Eifler

Receiving CEE enhanced syslog in rsyslog

By Adiscon SupportPosted on Posted in More complex scenarios

We want to receive syslog messages from another system that contain Windows Events. The messages are sent in the CEE enhanced format. The syslog messages should be stored in a database to be reviewed with Adiscon LogAnalyzer.

Things to think about

For the basics, we need at least a listener for UDP or TCP, depending on what you are using. We will configure both for now. As additional modules, we will need mmjsonparse, which has been introduced in v6. And in addition the output module for a MySQL database.

To enable the modules, we need some templates.

Note, that the receiver ports you specify must match the ports on the sending machines.

Config Statements

# modules
$ModLoad imuxsock # provides support for local system logging
$ModLoad imklog   # provides kernel logging support (previously done by rklogd)
$ModLoad immark  # provides --MARK-- message capability
$ModLoad mmjsonparse
$ModLoad ommysql
$ModLoad imudp
$ModLoad imtcp
# provides UDP syslog reception
$UDPServerRun 514
# provides TCP syslog reception
$InputTCPServerRun 10514
# templates
$template database,"insert into systemevents (DeviceReportedTime, FromHost, EventLogType, EventSource, EventID, 
EventCategory, EventUser, Facility, Priority, Message) values ('%timereported:::date-rfc3339%', '%$!source%', 
'%$!nteventlogtype%', '%$!sourceproc%', '%$!id%', '%$!categoryid%', '%$!user%', '%syslogfacility%', 
'%syslogpriority%', '%$!msg:::space-cc%')",SQL
# actions
*.* :mmjsonparse:
& :ommysql:mysqlserver,database,user,password;database

How it works

First of all, we have the modules being loaded. In addition to the modules needed for local logging, we have loaded the input modules for UDP and TCP syslog.

The next part will enable the listeners. For UDP we use the default port 514. For TCP we use port 10514. Please note, that the senders MUST match the respective port.

The third step is to create a template. You can see it at as 4 lines here, but it really is a single line. mmjsonparse will take messages and process them. It will automatically detect the CEE enhanced json format an create custom properties with the name fields in the message and fill it the available content. But, to use it for a database insert, we need a custom insert statement. As you can see, the most important fields are present. These fields should already be available in the database that is used by Adiscon LogAnalyzer. We now only need to tell rsyslog, which database field will be filled with which field from the json message.

The format differences are important. Some fields will be filled with information from the syslog header, like the timestamp, facility and priority. Thus we can use the regular properties. The properties from the json message always need to start with “$!”. And for the msg property, we will additionaly use the space-cc option. This will strip line feeds from multiline messages.

Finally we have the actions. The first action “*.* :mmjsonparse:” will be applied to all messages here, though it will not be able to parse messages from the local log socket. This calls the mmjsonparse module to parse the message and fill the fields as mentioned before. The second line holds the output to mysql. We extended the chain of actions by “&” instead of a filter.

Important notes

The most crucial part is the template. If the template is not correct, the whole action chain will not work. Or it will work at least not correctly. Though, the above template should work fine with most Windows Events, it might be necessary to alter the template for other message types and database structures.

Rsyslog Windows Agent Released

By Rainer GerhardsPosted on Posted in News, Top NewsTagged , ,

The rsyslog Windows agent enables customers to integrate Windows Event Logs into their central rsyslog-based logging system. Even more, it supports acting as a syslog relay as well. This enables small branch offices only runnig Windows to provide local syslog sources to the central server as well.

We have released the first full-featured version of this product today. It is based on Adiscon’s mature suite of Windows-to-syslog tools.  Most importantly, it provides the ability to fully extract Windows Event Log data, including local resolution of user IDs, SIDs, GUIDs and similiar objects. It not only supports the current Windows Event Log format but  also the legacy (pre Windows-2008/Vista) event log system. The technology behind this tool is already in use at thousands of customer sites for many years (in the form of, for example, EventReporter and other Adiscon products).

The rsyslog Agent for Windows has been highly optimized for use with rsyslog at the back end. For example, it supports mutually-authenticated, TLS encrypted syslog as well as the RELP protocol for reliable delivery. Also, it provides an initial implementation of cee-enhanced syslog, and is the world’s first-ever solution to do so (details can be found in the configuration guide for cee-enhanced syslog).

By purchasing rsyslog Agent for Windows customers not only get first class event log integration, they also help fund further improving the rsyslog solution as whole. The package even contains limited rsyslog professional services.

A trial version can be downloaded from the rsyslog Windows agent page.

rsyslog 5.8.8 (v5-stable) released

By Adiscon SupportPosted on Posted in News, Release AnnouncementTagged , , , , , , ,

This is a bug-fixing release. It fixes the wrong computed facility and severity in imklog and a problem with rsyslog aborting on startup if no binary to execute was configured for omprog.

ChangeLog:

http://www.rsyslog.com/changelog-for-5-8-8-v5-stable/

Download:

http://www.rsyslog.com/rsyslog-5-8-8-v5-stable/

As always, feedback is appreciated.

Best regards,

Florian Riedl

RSyslog Windows Agent and CEE

By Adiscon SupportPosted on Posted in Guides for Windows AgentTagged , , , ,

The Rsyslog Windows Agent comes with support for the new CEE enhanced format out of the box. It is designed to work flawlessly with all components from the Adiscon product lines and other CEE enhanced-enabled products. And it is one of the first products to support the Project Lumberjack at all. If you do not know what CEE enhanced is good for, it might be wise to watch our introduction into CEE.

In this guide, we will show the necessary steps to create a configuration for the RSyslog Windows Agent to output CEE enhanced conform log messages. The setup itself is very simple and does not differ a lot from other basic setups. In the end we will have a configuration, that will poll Windows EventLogs and forward them via syslog in CEE enhanced format to another syslog server.

Step 1: Setting up the ruleset and action.

1. First we define a new rule set. Right-click “Rulesets”. A pop up menu will appear. Select “Add Rule Set” from this menu.

2. Then, a wizard starts. Change the name of the ruleset to whatever name you like. We will use “Forward syslog” in this example. The screen looks as follow:

Click “Next” to go on with the next step.

3. Select only Forward via Syslog. Do not select any other options for this sample. Also, leave the “Create a Rule for each of the following actions” setting selected. Click “Next”. You will see a confirmation page. Click “Finish” to create the rule set.
null

4. As you can see, the new Rule Set “Forward syslog” is present. Please expand it in the tree view until the action level of the “Forward syslog” Rule and select the “Forward syslog” action to configure.

5. Configure the “Forward via Syslog” Action
Insert the IP of your syslog server into the field “Syslog Server”. You can change the port if needed as well. We will keep it on the default port 514. You could also change to protocol type to TCP for example. Attention RSyslog Windows Agent and your syslog server must use the same port and the same protocol.

But you need to change the “Used Message Format”. Click on the dropdown menu to see the options and choose “Use CEE enhanced Syslog Format”.
null

The configuration for syslog forwarding should now look like this:
null

6. Finally, make sure you press the “Save” button – otherwise your changes will not be applied.

Step 2: Setting up the EventLog Monitor V2.

Note: This guide explains how to set up the EventLog Monitor V2 Service for Windows Vista/z/2008. These steps are not applicable if you are using Windows XP/2000/2003. In that case, please use the regular EventLog Monitor.

1. First, right click on “Services”, then select “Add Service” and then “Event Log Monitor V2”:

Again, you can use either the default name or any one you like. We will use the default name in this example. Leave the “Use default settings” selected and press “Finish”, as we are not changing any other settings right now.

2. Now, you will see the newly created service beneath the “Services” as part of the tree view. To check its parameters, select it:

As you can see, the service automatically checks for all present EventLogs. You can now select or disable certain logs or change some of their properties.

Note: The ruleset “Forward Syslog” has been automatically assigned as the ruleset to use. By default, the wizard will always assign the first rule set visible in the tree view to new services.

Step 3: Starting the Service.

5. The last step is to save the changes and start the service. This procedure completes the configuration of the syslog server.

The Service cannot dynamically read changed configurations. As such, it needs to be restarted after such changes. In our sample, the service was not yet started, so we simply need to start it. If it already runs, you need to restart it.

We are now finished. You should now receive the Eventlog messages on your syslog server in CEE enhanced format.

Scroll to top