rsyslog

The rocket-fast system for log processing

omfwd: syslog Forwarding Output Module

Module Name: omfwd
Author: Rainer Gerhards <rgerhards@adiscon.com>

Purpose

The omfwd plug-in provides the core functionality of traditional message forwarding via UDP and plain TCP. It is a built-in module that does not need to be loaded.

Notable Features

Configuration Parameters

Note

Parameter names are case-insensitive.

Module Parameters

Template

type default mandatory obsolete legacy directive
word RSYSLOG_TraditionalForwardFormat no $ActionForwardDefaultTemplateName

Sets a non-standard default template for this module.

Action Parameters

Target

type default mandatory obsolete legacy directive
word none no none

Name or IP-Address of the system that shall receive messages. Any resolvable name is fine.

Port

type default mandatory obsolete legacy directive
word 514 no none

Name or numerical value of port to use when connecting to target.

Protocol

type default mandatory obsolete legacy directive
word udp no none

Type of protocol to use for forwarding. Note that ``tcp’’ means both legacy plain tcp syslog as well as RFC5425-based TLS-encrypted syslog. Which one is selected depends on the StreamDriver parameter. If StreamDriver is set to “gtls” it will use TLS-encrypted syslog.

NetworkNamespace

type default mandatory obsolete legacy directive
word none no none

Name of a network namespace as in /var/run/netns/ to use for forwarding.

If the setns() system call is not available on the system (e.g. BSD kernel, linux kernel before v2.6.24) the given namespace will be ignored.

Device

type default mandatory obsolete legacy directive
word none no none

Bind socket to given device (e.g., eth0)

For Linux with VRF support, the Device option can be used to specify the VRF for the Target address.

TCP_Framing

type default mandatory obsolete legacy directive
word traditional no none

Framing-Mode to be for forwarding. This affects only TCP-based protocols. It is ignored for UDP. In protocol engineering, “framing” means how multiple messages over the same connection are separated. Usually, this is transparent to users. Unfortunately, the early syslog protocol evolved, and so there are cases where users need to specify the framing. The traditional framing is nontransparent. With it, messages are end when a LF (aka “line break”, “return”) is encountered, and the next message starts immediately after the LF. If multi-line messages are received, these are essentially broken up into multiple message, usually with all but the first message segment being incorrectly formatted. The octet-counting framing solves this issue. With it, each message is prefixed with the actual message length, so that a receivers knows exactly where the message ends. Multi-line messages cause no problem here. This mode is very close to the method described in RFC5425 for TLS-enabled syslog. Unfortunately, only few syslogd implementations support octet-counted framing. As such, the traditional framing is set as default, even though it has defects. If it is known that the receiver supports octet-counted framing, it is suggested to use that framing mode.

TCP_FrameDelimiter

type default mandatory obsolete legacy directive
integer 10 no none

Sets a custom frame delimiter for TCP transmission when running TCP_Framing in “traditional” mode. The delimiter has to be a number between 0 and 255 (representing the ASCII-code of said character). The default value for this parameter is 10, representing a ‘\n’. When using Graylog, the parameter must be set to 0.

ZipLevel

type default mandatory obsolete legacy directive
integer 0 no none

Compression level for messages.

Up until rsyslog 7.5.1, this was the only compression setting that rsyslog understood. Starting with 7.5.1, we have different compression modes. All of them are affected by the ziplevel. If, however, no mode is explicitely set, setting ziplevel also turns on “single” compression mode, so pre 7.5.1 configuration will continue to work as expected.

The compression level is specified via the usual factor of 0 to 9, with 9 being the strongest compression (taking up most processing time) and 0 being no compression at all (taking up no extra processing time).

compression.Mode

type default mandatory obsolete legacy directive
word none no none

mode is one of “none”, “single”, or “stream:always”. The default is “none”, in which no compression happens at all. In “single” compression mode, Rsyslog implements a proprietary capability to zip transmitted messages. That compression happens on a message-per-message basis. As such, there is a performance gain only for larger messages. Before compressing a message, rsyslog checks if there is some gain by compression. If so, the message is sent compressed. If not, it is sent uncompressed. As such, it is totally valid that compressed and uncompressed messages are intermixed within a conversation.

In “stream:always” compression mode the full stream is being compressed. This also uses non-standard protocol and is compatible only with receives that have the same abilities. This mode offers potentially very high compression ratios. With typical syslog messages, it can be as high as 95+% compression (so only one twentieth of data is actually transmitted!). Note that this mode introduces extra latency, as data is only sent when the compressor emits new compressed data. For typical syslog messages, this can mean that some hundered messages may be held in local buffers before they are actually sent. This mode has been introduced in 7.5.1.

Note: currently only imptcp supports receiving stream-compressed data.

compression.stream.flushOnTXEnd

type default mandatory obsolete legacy directive
binary on no none

New in version 7.5.3.

This setting affects stream compression mode, only. If enabled (the default), the compression buffer will by emptied at the end of a rsyslog batch. If set to “off”, end of batch will not affect compression at all.

While setting it to “off” can potentially greatly improve compression ratio, it will also introduce severe delay between when a message is being processed by rsyslog and actually sent out to the network. We have seen cases where for several thousand message not a single byte was sent. This is good in the sense that it can happen only if we have a great compression ratio. This is most probably a very good mode for busy machines which will process several thousand messages per second and te resulting short delay will not pose any problems. However, the default is more conservative, while it works more “naturally” with even low message traffic. Even in flush mode, notable compression should be achivable (but we do not yet have practice reports on actual compression ratios).

RebindInterval

type default mandatory obsolete legacy directive
integer 0 no $ActionSendTCPRebindInterval or $ActionSendUDPRebindInterval

Permits to specify an interval at which the current connection is broken and re-established. This setting is primarily an aid to load balancers. After the configured number of messages has been transmitted, the current connection is terminated and a new one started. Note that this setting applies to both TCP and UDP traffic. For UDP, the new ``connection’’ uses a different source port (ports are cycled and not reused too frequently). This usually is perceived as a ``new connection’’ by load balancers, which in turn forward messages to another physical target system.

KeepAlive

type default mandatory obsolete legacy directive
binary off no none

Enable or disable keep-alive packets at the tcp socket layer. The default is to disable them.

KeepAlive.Probes

type default mandatory obsolete legacy directive
integer 0 no none

The number of unacknowledged probes to send before considering the connection dead and notifying the application layer. The default, 0, means that the operating system defaults are used. This has only effect if keep-alive is enabled. The functionality may not be available on all platforms.

KeepAlive.Interval

type default mandatory obsolete legacy directive
integer 0 no none

The interval between subsequential keepalive probes, regardless of what the connection has exchanged in the meantime. The default, 0, means that the operating system defaults are used. This has only effect if keep-alive is enabled. The functionality may not be available on all platforms.

KeepAlive.Time

type default mandatory obsolete legacy directive
integer 0 no none

The interval between the last data packet sent (simple ACKs are not considered data) and the first keepalive probe; after the connection is marked to need keepalive, this counter is not used any further. The default, 0, means that the operating system defaults are used. This has only effect if keep-alive is enabled. The functionality may not be available on all platforms.

StreamDriver

type default mandatory obsolete legacy directive
word none no $ActionSendStreamDriver

Choose the stream driver to be used. Default is plain tcp, but you can also choose gtls for TLS encryption.

StreamDriverMode

type default mandatory obsolete legacy directive
integer 0 no $ActionSendStreamDriverMode

Mode to use with the stream driver (driver-specific)

StreamDriverAuthMode

type default mandatory obsolete legacy directive
string none no $ActionSendStreamDriverAuthMode

Authentication mode to use with the stream driver. Note that this parameter requires TLS netstream drivers. For all others, it will be ignored. (driver-specific).

StreamDriverPermittedPeers

type default mandatory obsolete legacy directive
word none no $ActionSendStreamDriverPermittedPeers

Accepted fingerprint (SHA1) or name of remote peer. Note that this parameter requires TLS netstream drivers. For all others, it will be ignored. (driver-specific)

ResendLastMSGOnReconnect

type default mandatory obsolete legacy directive
binary off no $ActionSendResendLastMsgOnReconnect

Permits to resend the last message when a connection is reconnected. This setting affects TCP-based syslog, only. It is most useful for traditional, plain TCP syslog. Using this protocol, it is not always possible to know which messages were successfully transmitted to the receiver when a connection breaks. In many cases, the last message sent is lost. By switching this setting to “yes”, rsyslog will always retransmit the last message when a connection is reestablished. This reduces potential message loss, but comes at the price that some messages may be duplicated (what usually is more acceptable).

Please note that busy systems probably loose more than a single message in such cases. This is caused by an inherant unreliability in plain tcp syslog and there is no way rsyslog could prevent this from happening (if you read the detail description, be sure to follow the link to the follow-up posting). In order to prevent these problems, we recommend the use of omrelp.

udp.SendToAll

type default mandatory obsolete legacy directive
binary off no none

When sending UDP messages, there are potentially multiple paths to the target destination. By default, rsyslogd only sends to the first target it can successfully send to. If this option is set to “on”, messages are sent to all targets. This may improve reliability, but may also cause message duplication. This option should be enabled only if it is fully understood.

Note: this option replaces the former -A command line option. In contrast to the -A option, this option must be set once per input() definition.

udp.SendDelay

type default mandatory obsolete legacy directive
integer 0 no none

New in version 8.7.0.

This is an expert option, do only use it if you know very well why you are using it!

This options permits to introduce a small delay after each send operation. The integer specifies the delay in microseconds. This option can be used in cases where too-quick sending of UDP messages causes message loss (UDP is permitted to drop packets if e.g. a device runs out of buffers). Usually, you do not want this delay. The parameter was introduced in order to support some testbench tests. Be sure to think twice before you use it in producetion.

gnutlsPriorityString

type default mandatory obsolete legacy directive
string none no none

New in version 8.29.0.

The GnuTLS priority strings specify the TLS session’s handshake algorithms and options. These strings are intended as a user-specified override of the library defaults. If this parameter is NULL, the default settings are used. More information about priority Strings here.

Examples

Example 1

The following command sends all syslog messages to a remote server via TCP port 10514.

action(type="omfwd" Target="192.168.2.11" Port="10514" Protocol="tcp" Device="eth0")

Example 2

In case the system in use has multiple (maybe virtual) network interfaces network namespaces come in handy, each with its own routing table. To be able to distribute syslogs to remote servers in different namespaces specify them as separate actions.

action(type="omfwd" Target="192.168.1.13" Port="10514" Protocol="tcp" NetworkNamespace="ns_eth0.0")
action(type="omfwd" Target="192.168.2.24" Port="10514" Protocol="tcp" NetworkNamespace="ns_eth0.1")
action(type="omfwd" Target="192.168.3.38" Port="10514" Protocol="tcp" NetworkNamespace="ns_eth0.2")

See also

Help with configuring/using Rsyslog:

  • Mailing list - best route for general questions
  • GitHub: rsyslog source project - detailed questions, reporting issues that are believed to be bugs with Rsyslog
  • Stack Exchange (View, Ask) - experimental support from rsyslog community

See also

Contributing to Rsyslog:

© 2008-2017, Rainer Gerhards and Others. This site uses the “better” theme for Sphinx.