imkafka: read from Apache Kafka¶

Module Name:	imkafka
Author:	Andre Lorbach <alorbach@adiscon.com>
Available since:	8.27.0

Purpose¶

The imkafka plug-in implements an Apache Kafka consumer, permitting rsyslog to receive data from Kafka.

Configuration Parameters¶

Note that imkafka supports some Array-type parameters. While the parameter name can only be set once, it is possible to set multiple values with that single parameter.

For example, to select a broker, you can use

input(type="imkafka" topic="mytopic" broker="localhost:9092" consumergroup="default")

which is equivalent to

input(type="imkafka" topic="mytopic" broker=["localhost:9092"] consumergroup="default")

To specify multiple values, just use the bracket notation and create a comma-delimited list of values as shown here:

input(type="imkafka" topic="mytopic"
       broker=["localhost:9092",
               "localhost:9093",
               "localhost:9094"]
      )

Note

Parameter names are case-insensitive; camelCase is recommended for readability.

Module Parameters¶

Currently none.

Input Parameters¶

Parameter	Summary
broker	Selects the Kafka broker(s) imkafka connects to when consuming messages.
confparam	Passes arbitrary librdkafka configuration parameters to the imkafka consumer.
consumergroup	Sets the Kafka consumer group identifier (group.id) used by imkafka.
parseHostname	Controls whether imkafka parses the hostname from each received message.
ruleset	Assigns the rsyslog ruleset that processes messages received via imkafka.
topic	Identifies the Kafka topic from which imkafka consumes messages.

Caveats/Known Bugs¶

currently none

Examples¶

Example 1¶

In this sample a consumer for the topic static is created and will forward the messages to the omfile action.

module(load="imkafka")
input(type="imkafka" topic="static" broker="localhost:9092"
                     consumergroup="default" ruleset="pRuleset")

ruleset(name="pRuleset") {
     action(type="omfile" file="path/to/file")
}

Support: rsyslog Assistant | GitHub Discussions | GitHub Issues: rsyslog source project

Contributing: Source & docs: rsyslog source project