imtcp: TCP Syslog Input Module

Provides the ability to receive syslog messages via TCP. Encryption is natively provided by selecting the approprioate network stream driver and can also be provided by using stunnel (an alternative is the use the imgssapi module).

Author:Rainer Gerhards <rgerhards@adiscon.com>

Configuration Parameters

Module Parameters

AddtlFrameDelimiter <Delimiter>

This directive permits to specify an additional frame delimiter for Multiple receivers may be configured by specifying $InputTCPServerRun multiple times. This is available since version 4.3.1, earlier versions do NOT support it.

DisableLFDelimiter on/off

Default is off

Industry-strandard plain text tcp syslog uses the LF to delimit syslog frames. However, some users brought up the case that it may be useful to define a different delimiter and totally disable LF as a delimiter (the use case named were multi-line messages). This mode is non-standard and will probably come with a lot of problems. However, as there is need for it and it is relatively easy to support, we do so. Be sure to turn this setting to “on” only if you exactly know what you are doing. You may run into all sorts of troubles, so be prepared to wrangle with that!

maxFrameSize <int>

Default: 200000; Max: 200000000

When in octet counted mode, the frame size is given at the beginning of the message. With this parameter the max size this frame can have is specified and when the frame gets to large the mode is switched to octet stuffing. The max value this parameter can have was specified because otherwise the integer could become negative and this would result in a Segmentation Fault.

NotifyOnConnectionClose on/off

Default is off

Instructs imtcp to emit a message if the remote peer closes a connection.

Important: This directive is global to all listeners and must be given right after loading imtcp, otherwise it may have no effect.

KeepAlive on/off

Default is off

enable of disable keep-alive packets at the tcp socket layer. The default is to disable them.

KeepAlive.Probes <number>

The number of unacknowledged probes to send before considering the connection dead and notifying the application layer. The default, 0, means that the operating system defaults are used. This has only effect if keep-alive is enabled. The functionality may not be available on all platforms.

KeepAlive.Interval <number>

The interval between subsequential keepalive probes, regardless of what the connection has exchanged in the meantime. The default, 0, means that the operating system defaults are used. This has only effect if keep-alive is enabled. The functionality may not be available on all platforms.

KeepAlive.Time <number>

The interval between the last data packet sent (simple ACKs are not considered data) and the first keepalive probe; after the connection is marked to need keepalive, this counter is not used any further. The default, 0, means that the operating system defaults are used. This has only effect if keep-alive is enabled. The functionality may not be available on all platforms.

FlowControl on/off

Default is on

This setting specifies whether some message flow control shall be exercised on the related TCP input. If set to on, messages are handled as “light delayable”, which means the sender is throttled a bit when the queue becomes near-full. This is done in order to preserve some queue space for inputs that can not throttle (like UDP), but it may have some undesired effect in some configurations. Still, we consider this as a useful setting and thus it is the default. To turn the handling off, simply configure that explicitely.

MaxListeners <number>

Default is 20

Sets the maximum number of listeners (server ports) supported. This must be set before the first $InputTCPServerRun directive.

MaxSessions <number>

Default is 200

Sets the maximum number of sessions supported. This must be set before the first $InputTCPServerRun directive

StreamDriver.Name <string>

Selects network stream driver for all inputs using this module.

StreamDriver.Mode <number>

Sets the driver mode for the currently selected network stream driver. <number> is driver specific.

StreamDriver.AuthMode <mode-string>

Sets permitted peer IDs. Only these peers are able to connect to the listener. <id-string> semantics depend on the currently selected AuthMode and network stream driver. PermittedPeers may not be set in anonymous modes.

PermittedPeer <id-string>

Sets permitted peer IDs. Only these peers are able to connect to the listener. <id-string> semantics depend on the currently selected AuthMode and network stream driver. PermittedPeer may not be set in anonymous modes. PermittedPeer may be set either to a single peer or an array of peers either of type IP or name, depending on the tls certificate.

Single peer: PermittedPeer=”127.0.0.1”

Array of peers: PermittedPeer=[“test1.example.net”,”10.1.2.3”,”test2.example.net”,”...”]

discardTruncatedMsg on/off

Default is off

Normally when a message is truncated in octet stuffing mode the part that is cut off is processed as the next message. When this parameter is activated, the part that is cut off after a truncation is discarded and not processed.

gnutlsPriorityString <string>

Default is NULL

Available since 8.29.0

The GnuTLS priority strings specify the TLS session’s handshake algorithms and options. These strings are intended as a user-specified override of the library defaults. If this parameter is NULL, the default settings are used. More information about priority Strings here.

Input Parameters

Port <port>

Starts a TCP server on selected port

address <name>

Default: all interfaces

On multi-homed machines, specifies to which local address the listerner should be bound.

Name <name>

Sets a name for the inputname property. If no name is set “imtcp” is used by default. Setting a name is not strictly necessary, but can be useful to apply filtering based on which input the message was received from.

Ruleset <ruleset>

Binds the listener to a specific ruleset.

SupportOctetCountedFraming on/off

Default is on

If set to “on”, the legacy octed-counted framing (similar to RFC5425 framing) is activated. This should be left unchanged until you know very well what you do. It may be useful to turn it off, if you know this framing is not used and some senders emit multi-line messages into the message stream.

RateLimit.Interval [number]

Specifies the rate-limiting interval in seconds. Default value is 0, which turns off rate limiting. Set it to a number of seconds (5 recommended) to activate rate-limiting.

RateLimit.Burst [number]

Specifies the rate-limiting burst in number of messages. Default is 10,000.

Statistic Counter

This plugin maintains statistics for each listener. The statistic is named after the given input name (or “imtcp” if none is configured), followed by the listener port in parenthesis. For example, the counter for a listener on port 514 with no set name is called “imtcp(514)”.

The following properties are maintained for each listener:

  • submitted - total number of messages submitted for processing since startup

Caveats/Known Bugs

  • module always binds to all interfaces
  • can not be loaded together with imgssapi (which includes the functionality of imtcp)

See also

Example

This sets up a TCP server on port 514 and permits it to accept up to 500 connections:

module(load="imtcp" MaxSessions="500")
input(type="imtcp" port="514")

Note that the global parameters (here: max sessions) need to be set when the module is loaded. Otherwise, the parameters will not apply.

Legacy Configuration Directives

$InputTCPServerAddtlFrameDelimiter <Delimiter>

equivalent to: AddtlFrameDelimiter

$InputTCPServerDisableLFDelimiter on/off

equivalent to: DisableLFDelimiter

$InputTCPServerNotifyOnConnectionClose on/off

equivalent to: NotifyOnConnectionClose

$InputTCPServerKeepAlive** <on/**off**>

equivalent to: KeepAlive

$InputTCPServerKeepAlive_probes <number>

Equivalent to: KeepAlive.Probes

$InputTCPServerKeepAlive_intvl <number>

Equivalent to: KeepAlive.Interval

$InputTCPServerKeepAlive_time <number>

Equivalent to: KeepAlive.Time

$InputTCPServerRun <port>

equivalent to: Port

$InputTCPFlowControl on/off

equivalent to: FlowControl

$InputTCPMaxListeners <number>

equivalent to: MaxListeners

$InputTCPMaxSessions <number>

equivalent to: MaxSessions

$InputTCPServerStreamDriverMode <number>

equivalent to: StreamDriver.Mode

$InputTCPServerInputName <name>

equivalent to: Name

$InputTCPServerStreamDriverAuthMode <mode-string>

equivalent to: StreamDriver.AuthMode

$InputTCPServerStreamDriverPermittedPeer <id-string>

equivalent to: PermittedPeer.

$InputTCPServerBindRuleset <ruleset>

equivalent to: Ruleset.

$InputTCPSupportOctetCountedFraming on/off

equivalent to: SupportOctetCountedFraming

Caveats/Known Bugs

  • module always binds to all interfaces
  • can not be loaded together with imgssapi (which includes the functionality of imtcp)
  • increasing MaxSessions and MaxListeners doesn’t change MaxOpenFiles, consider increasing this global configuration parameter too (on Linux check the actual value for running process by in /proc/PID/limits; default limit on Linux is 1024)

Example

This sets up a TCP server on port 514 and permits it to accept up to 500 connections:

$ModLoad imtcp # needs to be done just once
$InputTCPMaxSessions 500
$InputTCPServerRun 514

Note that the parameters (here: max sessions) need to be set before the listener is activated. Otherwise, the parameters will not apply.