TLS support for librelp

If you followed librelp’s git, you have probably already noticed that there is increased activity. This is due to the fact that TLS support is finally being added! Thanks to some unnamed sponsor, we could invest “a bit” of time to make this happen.

We have decided to base TLS support on GnuTLS, which has matured very much, is preferred by Debian and fully supported by Red Hat and has no licensing issues with GPL like openssl has (plus the sponsor also preferred it). We build TLS support directly into librelp, as we assume it will get very popular, so an abstraction layer would not make that much sense, especially given the fact the GnuTLS nowadays is almost already installed by default. And remember that an abstraction layer always adds code complexity and an (albeit limited) runtime overhead.

Librelp 1.1.0 will be the first version with basic TLS support. With “basic”, we mean that this is a full TLS implementation, but there are some useful additional features not yet present. Most importantly, this version will not support certifiates but rather work with anonymous Diffie-Hellmann key exchange. This means that while the integrity and privacy of the session can be guaranteed as far as the network is concerned, this version does not guard against man-in-the-middle attacks. The reason simply is that there is no way to mutually authenticate peers without certificates. We still think it makes a lot of sense to release that version, as it greatly improves the situation.

Obviously, we have plans to add certificate support in the very near future. And this also means we will add ways for mutual authentication, much like in rsyslog’s RFC 5425 implementation. It’s not finally decided if we will support all authentication options RFC 5425 offers (some may not be very relevant in practice), but that’s so far undecided. We currently strongly consider to start with fingerprint-based authentication, as this permits the ability to do mutual authentication without the need to setup a full-blown PKI. Also, most folks know fingerprint authentication: this is what ssh does when it connects to a remote machine.

So stay tuned to librelp development, many more exciting things are coming up. Please note that rsyslog 7.5.0 will be the first version to utilize the new librelp features – but that’s something for a different blog posting.

[This is a cross-post from Rainer Gerhards’ blog (main librelp author)]

Newbie guide to rsyslog

Written by Jan Gerhards

Here are some different guides for people, who never worked with rsyslog. First I’ll explain, how to install rsyslog. Besides, I will explain how to install some packages, which you will need to install rsyslog. There are two important ones and some minor ones. After that, I will show how to do some easy configurations. Questions like how can I configure a module? Or How can I configure the input? Will be answered. In the example I will configure rsyslog to receive messages, filter them for one specific IP and store only the messages in a file. In the end I’ll explain something about easy rulesets. In my example I will try the same like I did with the configuration, but I will work with rulesets.

Before you begin

Please note, there are many ways to distribute rsyslog. But, make sure that the platform you build rsyslog on is the same platform as where it should be used. You cannot build rsyslog on CentOS and use it on Ubuntu. The differences between the platforms are just to big. The same applies to different versions of the same platform. When building on a older platform and using it on a newer version, this may work, but with restrictions. Whereas building on a newer version and using it on a older version will probably not work at all. So, if you build rsyslog from Source and want to use it on another machine, make sure the platform is the same.

Preliminary actions

There are generally two very often needed packages, that you have to install before rsyslog: libestr and libee. These can be downloaded here:

Libestr:       http://libestr.adiscon.com/download/

Libee:         http://www.libee.org/download/

Both are being installed the same, so here is only one step-by-step instruction.

  1. Download the file
    Just download libestr or libee and “cd” into the folder, where you want them.
  2. “tar xzf” the file
    after you “cd” to the folder, where the file is, type “tar xzf -name of file”. Your command line should look like this:

     tar xzf -name of file-
  3. “Cd” into the new folder
    “cd” into the made directory. It will have the same name like the downloaded file. Your command line should look like this:

     cd -name of file-
  4. Run “./configure configure –libdir=/usr/lib –includedir=/usr/include”
    After you “cd” into that file, just run ” ./configure –libdir=/usr/lib –includedir=/usr/include”. Your command line should look like this:

     ./configure --libdir=/usr/lib --includedir=/usr/include
  5. Type “sudo make”
    Just type this and let the Computer work. Your command line should look like this:

     sudo make
  6. Type “sudo make install”
    This is the last step. Like with “sudo make”, just let the computer work. Your command line should look like this

     sudo make install
  7. The package should work now
    Congratulation! You have installed the package! If it doesn’t work properly, you should check if you followed every step right and if you have, contact the support.

There might also be some other packages like libjson0-dev, uuid-dev, bison, flex, libz-dev or python-docutils. These you’ll have to install before installing rsyslog, too. This guide was made on Ubuntu, so if you use any other system, they might be named differently. You can install them with the command

sudo apt-get install -package name-

After you got all the packages, you can install rsyslog. Here is a How to for installing rsyslog.

 

How to install rsyslog

In this guide is explained how to install rsyslog with the download from the homepage.

 

  1. Download rsyslog
    You can download rsyslog from the rsyslog webpage at http://www.rsyslog.com/download/
  2. “tar xzf” the file
    Open a terminal.  Then you “cd” to where you want the file and then type “tar xzf -name of downloaded file”. Your command line should look like this:

    tar xzf -name of downloaded file-
  3. “cd” into the new folder
    Then “cd” into the made directory. The new directory will have the same name like the downloaded file. Your command line should look like this:

     cd -name of file-
  4. Type “./configure –prefix=/usr”
    You just need to run “./configure –prefix=/usr”. There might be an Error that you need to install some packages to precede with installing. For two specific packages I’ve explained how to install them, because you will need them often. Your command line should look like this:

    ./configure --prefix=/usr
  5. Run “sudo make”
    it’s easy, just run “sudo make” and let the computer work. Your command line should look like this:

     sudo make
  6. Run “sudo make install”
    Just like “sudo make”. Type it, press enter and let the Computer work. Again, there might be some missing packages. Just install them. Your command line should look like this:

     sudo make install
  7. Rsyslog should now be installed
    Congratulation! You have installed rsyslog! You can check the version of rsyslog by typing: “rsyslogd -v”.
    If it doesn’t work properly, you should check if you followed every step right and if you have, contact the support.

 

Now after you installed rsyslog, you have to configure it. How to do this is explained here.

 

Configure rsyslog

In this part I’ll explain some basic configuration steps for rsyslog. We configure rsyslog to recive UDP messages, to filter them depending on the IP of the host and to store them in a file.

  1. How to configure the module
    The module has to be configured first. The general line for this configuration is: “module (load=”im-type of protocol-”). So in our example, where we want UDP, it will look like this:

    Module (load=”imudp”)
  2. How to configure the input for rsyslog
    For the input, you have to give two different information to rsyslog. The first information needed is the protocol type of the input; in my example again UDP. Like in the first line there is an “im-” in front of the protocol-type. The other information is to configure a port for rsyslog, in my example 514. These two information are together in only one line. The line is: “Input (type=”-protocol of input-“port=”-number of port-“). This means for my example, the line has to be

    Input (type=”imudp” port=”514”)
  3. How to configure a filter for fromhost-IPs and store them in a file
    A filter always has, like a normal conditional sentence, an “if…then” part. If you want to configure it to do something with all notes from a specific IP, between “if” and “then” will be the property “$fromhost-ip ==”-IP, you want to filter-”. After this stays a “then” and after the “then” follows an action in brackets, which I will explain later. In my example I want only the notes from the host with the IP 172.19.1.135. So the line will be

    If $fromhost-ip == “172.19.1.135” then {

    After this we have to tell the computer, what to do if that case is given. In this example we want him to store these messages in the file “/var/log/network1.log”. This is an action with the type “omfile”. To configure the file where to store the messages, the action is “action (type=”omfile” File=”-filename-“). So in this example, it will look like this:

    Action (type=”omfile” file=”/var/log/network1.log”)
    }

 

All the lines together now are

Module (load=“imudp“)

Input (type=”imudp” port=”514”)
If $fromhost-ip == “172.19.1.135“ then {
Action (type=”omfile” File=”/var/log/network1.log”)
}

All in all it means: The input for rsyslog will listen to syslog via UDP on port 514. If the IP from the Computer, which sends the messages, is 172.19.1.135, then the action in the brackets will get activated for these. In the action the messages will be stored in the file /var/log/network1.log.

 

Rsyslog and rulesets

Rulesets are a bit more complicated. A ruleset is a set of rules, as the name implies. These are bound to an input. This works by adding an option to the input, namely “ruleset=”-rulesetname-“”. For example, if I want to bind a ruleset “rs1” to a input the line will look like this:

Input (type=”imudp” port=”514” ruleset=”rs1”)

But you still have to define, what the ruleset should do. In this guide I will limit myself to explain, how to create a ruleset, which has one action: to store all the messages in a file. In my example I want to store the messages in the file /var/log/network1.log”.

You define a ruleset like the normal configuration. To define it, you first name it with ruleset (name=”-rulesetname-“). After this you write what it does, in my example the action action (type=”omfile” file=”/var/log/network1.log”). This action you write in these curly brackets: {}.

So my full example looks like this

Module (load=”imudp”)

Input (type=”imudp” port=”514” ruleset=”rs1”)

Ruleset (name=”rs1”) {
Action (type=”omfile” file=”/var/log/network1.log”)
}

In that second example for configurations you can see, how to store all messages from the input into a file by using a ruleset. A rulesset can consist of multiple rules, but without binding it to the input it is useless. It can be bound to an input multiple times or even other rulesets can be called.

 

Final Conclusion

In this guide I explained how to install rsyslog, how to configure it and how to use rulesets. After you read this guide you are able to do exactly this: you can install rsyslog, configure it and have basic knowlege about rulesets. If you want to learn more about rsyslog, how to configure it or about rulesets, you can find information in the other guides or in the documentation.

TLS secured syslog via RELP

This article will show you, how to use simple tls encryption with the RELP protocol for sending and receiving syslog messages.

We basically need two machines, both running at least rsyslog 7.3.16. In addition to rsyslog, we also need the most current version of librelp.

General information

When installing rsyslog, make sure to enable the RELP functionality by issuing the correct commands for the configure. The configure command should look like this:

./configure --prefix=/usr --enable-relp

This is the most basic command for our example. Please note, that you might need to enable other modules as well if you plan to use them.

Before you start to configure rsyslog on either machine, make sure you have librelp already installed. You might need to additionaly install the gnutls package.

Client Config

The configuration for the client is relatively simple.  Basically, we can use as inputs whatever we like and simply use RELP with TLS encryption for forwarding the messages. The configuration could look like this:

module(load="imuxsock")
module(load="imudp")
module(load="omrelp")

input(type="imudp" port="514")

action(type="omrelp" target="192.168.233.144" port="2514" tls="on")

As you can see, we first load our modules. That is a generic step. We also load the output module “omrelp” which enables us later to forward messages via RELP.

In the second stage we configure our input. This example has the ability to receive syslog via imudp on port 514.

And the final step is our action. We use omrelp to forward all log messages to our central server via port 2514. Please note the option tls=”on” which directs the module to encrypt all messages via TLS.

Server Config

The server configuration looks a bit different and is one step more complicated.

module(load="imuxsock")
module(load="imrelp" ruleset="relp")

input(type="imrelp" port="2514" tls="on")

ruleset(name="relp") {
action(type="omfile" file="/var/log/relptls")
}

Again, we first configure the modules. Contrary to the Client configuration, we load “imrelp” and create the input with it in the second step.

The input with imrelp must listen to the same port, that the client sends its messages to. Also we must enable the TLS option as well. The reason might seem obvious, because without the option enabled, imrelp will push only garbage messages into the processing system. So we need TLS enabled to decrypt the messages. Please note, that I also bound the input to a ruleset.

The ruleset and action are again very basic. The ruleset (which is bound to the input) ensures, that only the messages that are received via RELP are processend by the enclosed actions. This is much easier, than creating filters to determine the source of the message (not only from a setup point of view, but also in regards of processing speed). The action in the ruleset will then write all messages that run into the ruleset into a single file. Please note: for imrelp, you can only bind the module to a ruleset. In consequence, all created listeners of this type are bound to this single ruleset.

 

How to setup RSyslog Windows Agent to monitor NetApp devices using backup *.evt files

This article describes how to use RSyslog Windows Agent to monitor NetApp devices using the backup .evt files. In this guide we describe how to setup the service. For creating the actions, please refer to the our other guides.

There are basically two methods to monitor logs of NetApp devices. The first, described here, is to monitor the .evt files that the NetApp device generates. The second method is to monitor the device via the Eventlog API. Instructions can be found here.

The NetApp device basically offers to access the .evt files via a network share. Thus the files are easily accessible through our products.

Basically, we need to create the Event Log Monitor service in RSyslog Windows Agent. Simply right-click on services and from the popup list, choose “Add Service” and the “Event Log Monitor”.

Now disable all the currently available logs except for one. Double click on the one that is still checked. A new window opens.

In this new windows, enable the option “Read Eventlog from File”. The parameters belonging to this option are now available. Insert the file and path name into the field. Alternatively, you can use the browse button to navigate to the remote location of the NetApp and choose the file like that. You could now also change the “Type of Eventlog” if necessary.

Please note, that this method is also fit to monitor multiple files. You only need to change the file name accordingly and insert wildcards to replace name values like dates. This is good for cases, when a new log file is created every day and the filename reflects the date when the file was created, like below for file like adtlog.20130206110000.evt or adtlog.20130206121314.evt.

So thats it basically. You can now choose to forward the log messages via syslog to a central log host, write them into a database or use one of the many other options that are available in RSyslog Windows Agent.

How to setup RSyslog Windows Agent to monitor NetApp devices using Eventlog API

This article describes how to use RSyslog Windows Agent to monitor NetApp devices using the Eventlog API. In this guide we describe how to setup the service. For creating the actions, please refer to the our other guides.

There are basically two methods to monitor logs of NetApp devices. The first, described here, is to monitor the device via the Eventlog API. The second method is to monitor the device via the .evt files the device generates. Instructions can be found here.

The NetApp device basically offers to access the log storage via the Eventlog API. That makes it very easy to use our products to monitor NetApp devices.

Basically, we need to create the Event Log Monitor service in RSyslog Windows Agent. Simply right-click on services and from the popup list, choose “Add Service” and the “Event Log Monitor”.

In the next step, enable “remote EventLog monitoring”. Insert the hostname or IP of the NetApp device into the field. Verify the connection with the “Verify” button. You might need to run the MonitorWare Agent service with a account, that has both local administrative rights as well as rights to read the Eventlog of the NetApp device.

Now disable all the currently available logs except for Application, Security and System. Double click on the one that is still checked. A new window opens.

In this new windows, enable the option “Use Checksum to verify the last processed event”. The parameters belonging to this option are now available. Also enable “Always search for the last processed Event using the Checksum”. If these options are not enabled, polling the log messages will not work properly, because the NetApp logging system does not use a record number to identify single log messages. Repeat this step for the remaining two log types.

So thats it basically. You can now choose to forward the log messages via syslog to a central log host, write them into a database or use one of the many other options that are available in RSyslog Windows Agent.

librelp 1.0.7 released

librelp 1.0.7 [download]: removed relpCltConnect2() API which was against librelp API philosophy. This was only introduced in 1.0.6 and been in the code for a very short time. So we decided that the best thing to do is actually remove it (there is NO known released user, this change was for yet unreleased rsyslog 7.5.0). [sha256 hash: 615df51cbbd6e62ed3900e1934e79c2ac02b533c01de20d28009968e54e93779]

Changelog for 7.3.14 (v7-beta)

Version 7.3.14 [beta] 2013-05-06

  • bugfix: some man pages were not properly installed either rscryutil or rsgtutil man was installed, but not both Thanks to Marius Tomaschewski for the patch.
  • bugfix: potential segfault on startup when builtin module was specified in module() statement. Thanks to Marius Tomaschewski for reporting the bug.
  • bugfix: segfault due to invalid dynafile cache handling Accidently, the old-style cache size parameter was used when the dynafile cache was created in a RainerScript action. If the old-style size was lower than the one actually set, this lead to misadressing when the size was overrun, and that could lead to all kinds of “interesting things”, often in segfaults. closes: http://bugzilla.adiscon.com/show_bug.cgi?id=440
Scroll to top