Using RSyslog Windows Agent to forward log files
in this article we describe how to use the RSyslog Windows Agent to forward log messages that are stored in plain text files. The article itself will be made of two larger steps. Both steps contain some substeps which will be shown in detail in one of the smaller articles. We assume, that no basic configuration is currently available.
This time, we want to use textfiles as log sources. Many programs for Windows do not use the EventLog system. They use simple and plain text files to store their log messages. Though, the information that is logged there could be as important as EventLogs.
Basically, the configuration of RSyslog Windows Agent consists of 3 parts.
1. A so-called service which generates the log data to be processed by, for example, a file monitor.
2. Rules with Filters. Filters give you the power to decide which log messages are important enough to be kept or not.
3. The action that has to be taken. In our case, forwarding the syslog messages.
Step 1: Setting up the ruleset and action.
Usually we start by creating the ruleset, rule and action. The reason lies in the configuration structure. So we will first create the mentioned items. In the end, we will have a basic rule with no particular filter and a forward via syslog action. That means, that all messages will be forwarded to a central host.
Step 2: Setting up the service.
Now we will set up the service. We need to create a File Monitor Service. The File Monitor Service is able to monitor a file or a directory with files. It will check the specified file(s) periodically for new lines (which would be new log messages) and use them for further processing.
That’s it, you are already finished. Easy wasn’t it? Now you should be able to poll log files and forward the log messages to your central syslog server.
rsyslog 6.2.0 (v6-stable) released
This is the initial stable release of rsyslog v6. It is basically the last beta version plus some more fixes. This version provides functional and performance enhancements, for example the Hadoop (HDFS) support has been considerably speeded up by supporting batched insert mode. Also, TCP transmission overhead for TLS has been dramatically improved. TCP now also supports input worker thread pools. Most importantly, rsyslog now supports log normalization via liblognorm rule bases. This permits very high performance normalization of semantically equal messages from different devices (and thus in different syntaxes).
Note that config scoping, available in the beta versions, is NOT supported by v6-stable. This was decided because it would have been functionality equivalent to the new config language upcoming in v6.3 (already available as part of the devel version). As scoping was not available in any earlier versions, introducing it in v6.2 would have added, in the long term, just another method of doing some identicaly thing via the ugly old config language. This would have lead to user confusion and more complex than necessary code. If you are interested in the cleaner config language, we strongly encourage you to have a look at rsyslog 6.3.
With the arrival of the stable v6 version, version 4 will be retired and is no longer officially supported (but support is provided under maintenance contracts, of course).
ChangeLog:
http://www.rsyslog.com/changelog-for-6-2-0-v6-stable/
Download:
http://www.rsyslog.com/rsyslog-6-2-0-v6-stable/
As always, feedback is appreciated.
Best regards,
Florian Riedl
Changelog for 6.2.0 (v6-stable)
ChangeLog (from 6.1.12 to 6.2.0):
- bugfix (kind of): removed numerical part from pri-text see v6 compatibility document for reasons
- bugfix: race condition when extracting program name, APPNAME, structured data and PROCID (RFC5424 fields) could lead to invalid characters e.g. in dynamic file names or during forwarding (general malfunction of these fields in templates, mostly under heavy load)
- bugfix: imuxsock did no longer ignore message-provided timestamp, if so configured (the *default*). Lead to no longer sub-second timestamps.
closes: http://bugzilla.adiscon.com/show_bug.cgi?id=281 - bugfix: omfile returns fatal error code for things that go really wrong previously, RS_RET_RESUME was returned, which lead to a loop inside the rule engine as omfile could not really recover.
- bugfix: rsyslogd -v always said 64 atomics were not present
thanks to mono_matsuko for the patch - bugfix: potential abort after reading invalid X.509 certificate closes:
http://bugzilla.adiscon.com/show_bug.cgi?id=290
Thanks to Tomas Heinrich for the patch - $Begin, $End, $StrictScoping directives have been removed as v6.4 will provide the same functionality in a far better way. So we do not want to clutter the code.
rsyslog 6.2.0 (v6-stable)
Download file name: rsyslog 6.2.0 (stable)
rsyslog 6.2.0 (stable)
md5sum: 03e237abaa5d47f92c6e655f92f22532
Author: Rainer Gerhards (rgerhards@adiscon.com)
Version: 6.2.0 File size: 2.41 MB
Using RSyslog Windows Agent as a syslog relay
This time we want to use the RSyslog Windows Agent as a syslog relay. The article itself will be described in two larger steps. Both steps contain some substeps which will be shown in detail in one of the smaller articles. We assume, that no basic configuration is currently available.
This configuration can be used to attach several sites to a larger log network. Imagine you have a central log server at your company in your central facility. You have some branch offices around the country and their log messages should be stored in the central location as well. Now it wouldn’t be very clever to have every computer or device at these sites forward their log messages directly. It would be better to have a central machine at the site, that works as a relay. It will receive all the log messages via syslog and then again forward the messages to the central server. Cascading setups like this ensure a part of the reliability, stability and security of your infrastructure, by keeping the connection count low and lowering the amount of machines using the network.
Basically, the configuration of RSyslog Windows Agent consists of 3 parts.
1. A so-called service which generates the log data to be processed by, for example, a syslog server.
2. Rules with Filters. Filters give you the power to decide which log messages are important enough to be kept or not.
3. The action that has to be taken. In our case, forwarding the syslog messages.
Step 1: Setting up the ruleset and action.
Usually we start by creating the ruleset, rule and action. The reason lies in the configuration structure. So we will first create the mentioned items. In the end, we will have a basic rule with no particular filter and a forward via syslog action. That means, that all messages will be forwarded to a central host.
Step 2: Setting up the service.
Now we will set up the service. As we already know, we want to create a syslog server. Therefore we need the syslog server service. It will create a listener, that is listening to syslog traffic on a specific port. You can choose the protocol, like TCP or UDP as well.
That’s it, you are already finished. Easy wasn’t it? Now you should be able to receive syslog messages from different devices and forward them from only one machine to your central syslog server.
Forward Windows Eventlogs with RSyslog Windows Agent
This article will describe, how to use the RSyslog Windows Agent to forward the local Windows EventLog messages. This article will show the different steps. For this we take you to several smaller guides, that show you, how to setup each part. We assume, that no basic configuration is currently available.
A configuration like this is needed very often and basically on any Windows machine that should forward it’s logs. Therefore, this reflects the default configuration after installing the RSyslog Windows Agent. It can be used on machines in your local network or on a site to forward from the single machines to a central relay server, which then forwards all messages to your company’s central log server.
Basically, the configuration of RSyslog Windows Agent consists of 3 parts.
1. A so-called service which generates the log data to be processed by, for example, polling the Windows EventLog.
2. Rules with Filters. Filters give you the power to decide which log messages are important enough to be kept or not.
3. The action that has to be taken. In our case, forwarding the syslog messages.
Step 1: Setting up the ruleset and action.
Usually we start by creating the ruleset, rule and action. The reason lies in the configuration structure. So we will first create the mentioned items. In the end, we will have a basic rule with no particular filter and a forward via syslog action. That means, that all messages will be forwarded to a central host.
Step 2: Setting up the service.
Now we will set up the service. There is one thing to mention first. You need to know choose one of the latter links according to your operating system. This is important, or the setup might not work properly. We have 2 different versions of the EventLog Monitor. Here is a small list in which you can see, which service fits which operating systems.
1. EventLog Monitor: 2000, XP, 2003
2. EventLog Monitor V2: Vista, 2008, 7, 10
This is important. EventLog Monitor V2 will NOT work on the older operating systems. In contrary, the older EventLog Monitor will work on the newer systems, but might not work correctly, so it is advised to used the optimized EventLog Monitor V2. This is due to the massive changes that Microsoft introduced to the EventLog system with Vista.
That’s it, you are already finished. Easy wasn’t it? Now you should receive your EventLog messages on your central syslog server.
How To setup EventLogMonitor V2 Service
Note: This guide explains how to set up the EventLog Monitor Service for Windows Vista. These steps are not applicable if you are using Windows XP.
1. First, right click on “Services”, then select “Add Service” and then “Event Log Monitor V2”:
Again, you can use either the default name or any one you like. We will use the default name in this sample. Leave the “Use default settings” selected and press “Next”.
2. As we have used the default, the wizard will immediately proceed with step 3, the confirmation page. Press “Finish” to create the service. The wizard completes and returns to the configuration client.
3. Now, you will see the newly created service beneath the “Services” as part of the tree view. To check its parameters, select it:
As you can see, the service has been created with the default parameters.
Note: The “Default RuleSet” has been automatically assigned as the rule set to use. By default, the wizard will always assign the first rule set visible in the tree view to new services.
4. Finally we, bind a ruleset to this service. If you already have a ruleset, simply choose one. If not, then you will have to create one, or insert the actions you want to take in the default ruleset.
Remember, this is only an example. You can do it in any way you want.
5. The last step is to save the changes and start the service. This procedure completes the configuration of the syslog server.
The NT Service cannot dynamically read changed configurations. As such, it needs to be restarted after such changes. In our sample, the service was not yet started, so we simply need to start it. If it already runs, you need to restart it.
That’s it. This is how you create a simple Event Log Monitor V2 for Vista.
How To setup EventLogMonitor V1 Service
Attention: This Guide is for Windows XP or 2003 if you use Vista or Win7 then use EventLogMonitor V2.
1. First, right click on “Services”, then select “Add Service” and then “Event Log Monitor”:
2. Once you have done so, a new wizard starts.
If the following Popup appears, please select “Create Service”:
Again, you can use either the default name or any one you like. We will use “My Event Log Monitor” in this sample. Leave the “Use default settings” selected and press “Next”.
3. As we have used the default, the wizard will immediately proceed with step 3, the confirmation page. Press “Finish” to create the service. The wizard completes and returns to the configuration client.
4. Now, you will see the newly created service beneath the “Services” part of the tree view. To check its parameters, select it:
As you can see, the service has been created with the default parameters.
Note: The “Default RuleSet” has been automatically assigned as the rule set to use. By default, the wizard will always assign the first rule set visible in the tree view to new services. In our case, this is not correct and will be corrected soon.
5. Finally, save the change and start RSyslog Windows Agent.
That was it.
How To create a Syslog Server Service
Create a Syslog Server Service
Now we need to define a Syslog server service. A Syslog server is also sometimes called a “Syslog daemon”, “Syslogd” or “Syslog listener”. It is the process that receives incoming messages.
To define it, right click on “Services”, then select “Add Service” and the “Syslog Server”:
Once you have done so, a new wizard starts:
Again, you can use either the default name or any one you like. We will use “My Syslog Server” in this example. Leave the “Use default settings” selected and press “Next”:
As we have used the default, the wizard will immediately proceed with step 3, the confirmation page. Press “Finish” to create the service. The wizard completes and returns to the configuration client. There, you will see the newly created service beneath the “Services” part of the tree view:
Attention: All clients must use the same protocol. In my example I use TCP.
As you can see, the service has been created with the default parameters. As such, it operates as a RFC compliant standard Syslog server.
Please note that the “RuleSet 1” has been automatically assigned as the rule set to use. This is the case because i already created it and it is the only rule set. By default, the wizard will always assign the first rule set visible in the tree view to new services. If another one is to be used, you need to change it to the correct one here in the service definition.
Also, note that the wizard uses the default properties from the “Service Defaults”. Obviously, if these are changed, the default properties for new services will differ.
This procedure completes the configuration of the Syslog server.
At least Save and restart the service.
How To setup File Monitor Service
How To setup File Monitor Service
Article created 2011-12-29 by Florian Riedl.
1. First, right click on “Services”, then select “Add Service” and the “File Monitor”.
Once you have done so, a new wizard starts.
2. Again, you can use either the default name or any one you like. We will use “My FileMonitor” in this sample. Leave the “Use default settings” selected and press “Next”.
3. As we have used the default, the wizard will immediately proceed with step 3, the confirmation page. Press “Finish” to create the service. The wizard completes and returns to the configuration client.
4. Now, you will see the newly created service beneath the “Services” part of the tree view. To check its parameters, select it:
As you can see, the service has been created with the default parameters.
5. To make this Service work, we need to select a text file as source. To achieve this, click on the “Browse” button as you can see it marked in the screen above. A browsing window will open up. Move through your Files and choose one that you would like to monitor. For this example I chose a text file created by MonitorWare Agent.
6. Now we still need to set a ruleset for this service to work with. Since we have no configured ruleset available at the moment, simply use the Default Ruleset, if it’s not being used automatically.
7. Last, save the changes and then restart the application. This procedure completes the configuration of the FileMonitor Service.
The Application cannot dynamically read changed configurations. As such, it needs to be restarted after such changes.