TwitterFacebookGoogle

Tag Archives: filter

Encasing of control structures

Posted on by

When using control structures to create some case filtering, there are some things to consider when formatting this.

In general, a control structure can be as simple as this:

if $msg contains "word" then 
    action()
else
    action()

While this is correct and will work as expected, it is generally better style to encase expressions and actions. That way, they are more clearly separated from the control structure itself for better overview and can be more easily expanded without error. When using multiple actions or expressions, encasing is needed anyway, so we think it’s good practice to always use it.

Example:

if ($msg contains "word") then {
    action() 
} else {
    action()
}

And a more complex example:

if ($msg contains "word" and $source == "123.123.123.123") then {
    action()
    action()
} else {
    action()
    action()
}

All three sample snippets above are correct in their own way. Added complexity makes encasing necessary though. So, you may as well use it for all your filters to have a consistent and all-case proof configuration.

RSyslog Windows Agent 4.2 Released

Posted on by

Adiscon is proud to announce the 4.2 release of MonitorWare Agent.

Besides some bugfixes (See Version History for details) a few new features have been added to this minor release. Most important is the ability to use regular expressions as compare operation when filtering properties. Properties can also be converted into IPv4 or IPv6 Addresses now, and the Syslog Priority/Facility can be overwritten in the Syslog Action.

Detailed information can be found in the version history below.

Build-IDs: Service 4.2.0.170, Client 4.2.0.250

Features

  • Syslog Action: Added support to overwrite Syslog Priority/Facility
  • Property Engine: Added two new property replacer options “toipv4address” and “toipv6address” to resolve a property into a valid IPv4 or IPv6 Address.
  • Filter Engine: Implemented a new regular expressions compare operation.
    More details on how to use REGEX can be found in the new documentation.
  • Configuration Reload: Added new options to add a random delay between configuration checks. The delay is limited to 60 seconds as it will also delay the service control manager communication.

Bugfixes

  • Syslog Action: Fixed bug in Syslog Cache processing when saved messages were larger than 4096 bytes.
  • Filter Engine: Fixed Extended IP Filtering when using lower or greater compare operation.
  • File Configuration: Fixed reading Filter values containing backslashes.
    They weren’t removed properly in filter values.

Version 4.2 is a free download. Customers with existing 3.x keys can contact our Sales department for upgrade prices. If you have a valid Upgrade Insurance ID, you can request a free new key by sending your Upgrade Insurance ID to sales@adiscon.com. Please note that the download enables the free 30-day trial version if used without a key – so you can right now go ahead and evaluate it.

Filter optimization with arrays

Posted on by

If you are using a lot of filters and templates in rsyslog, this can not only be affecting the performance drastically, but it is also a hassle to set up all the different actions and templates. It is always worthy to check, if there isn’t a shortcut somewhere, which might not only save you time for creating the configuration, but also make it much simpler in the end to keep track of all the actions.

In our example, we have several programnames. The log messages should be sorted by programname and then be stored in a specific file and be sorted by host. After storing the log messages, the message should be discarded, so it won’t be processed by the following filters, thus saving otherwise wasted processing time. This example is applicable to rsyslog v7.2.x and above.

Here are some sample config lines.

## apache_access
template(name="DailyPerHost_apache_access" type="string" string="/syslog/%FROMHOST%/apache_access.log")
:programname, isequal, "apache_access" { action(type="omfile" DynaFile="DailyPerHost_apache_access") stop }

## apache_error
template(name="DailyPerHost_apache_error" type="string" string="/syslog/%FROMHOST%/apache_error.log")
:programname, isequal, "apache_error" { action(type="omfile" DynaFile="DailyPerHost_apache_error") stop }
## mysql
template(name="DailyPerHost_mysql" type="string" string="/syslog/%FROMHOST%/mysql.log")
:programname, isequal, "mysql" { action(type="omfile" DynaFile="DailyPerHost_mysql") stop }

## php
template(name="DailyPerHost_php" type="string" string="/syslog/%FROMHOST%/php.log")
:programname, isequal, "php" { action(type="omfile" DynaFile="DailyPerHost_php") stop }

These are some basic services, which are often run together. Please note, that these are just a few examples. As you can see here, the template is created first. It is given a name, type and format. Templates of type string are usually used for file names. Here the log messages get stored in the folder /syslog a subfolder for the host where the message occured and then a filename which reflects the type of message that occured.

The second line holds the actions. First you see the property based filter (programname) and the condition. After that the actions get chained with the curly braces. The log messages where the filter evaluates to true get stored in a file. The filename and path is generated dynamically with the DynaFile parameter. Through this, the above written template will be used to generate the path and filename. The second action is represented by stop. Please note that this is case sensitive. Basically, stop means to stop the message processing. No further processing of the message will take place.

If we look closely at the sample config lines, we see, that the filter condition is basically always the same. It will always filter the programname property for a certain value. This is a predestinated case for using an array for simplification. We can use the property programname in the file template as well and filter an array of values. This will greatly save the overhead for all the seperate filter, not only in the configuration, but also in processing the messages.

template(name="DailyPerHost_app" type="string" string="/syslog/%FROMHOST%/%programname%.log")
if $programname == ["apache_access",
                    "apache_error",
                    "mysql",
                    "php"]
                then {
                action(type="omfile" DynaFile="DailyPerHost_app")
                stop
                }

Again, we first create the template. Please note the difference in the filename where the hardcoded text has been replaced by the property programname. In the next lines, we see the filter and the array of values. This is just to reflect the example. Virtually, the array can have near-infinite values. The filter is also a common if/then construct. After the then we see our chain of commands. First the action which writes the log messages into a file where the filename is created by the above template and then a stop as second action.

This case is applicable in many forms. It is also most useful if you are filtering and the discarding a lot of messages with very common filter settings. You could use it to filter for an array of property values and even chain comparison operations.