.. index:: ! omsendertrack .. _omsendertrack: omsendertrack: Sender Tracking Output Module ============================================= .. rst-class:: AdisconInfo :Module Name: **omsendertrack** :Author: `Rainer Gerhards `_ :Available since: 8.2506.0 (Proof-of-Concept) **Status:** Proof-of-concept implementation. This module is currently in an experimental stage. Further details and discussion regarding its development and progress can be found in `issue #5599 `_. Purpose ------- The ``omsendertrack`` output module is designed to collect and maintain real-time statistics about message senders across all configured rsyslog inputs. Its primary goal is to provide a flexible and persistent mechanism for tracking message flow from various sources. Key uses for ``omsendertrack`` include: * **Identifying Top Senders:** Quickly pinpoint which hosts or applications are generating the most log traffic. * **Monitoring Sender Behavior:** Detect changes in message rates or patterns from specific senders, which can indicate issues or unusual activity. * **Understanding Message Distribution:** Gain insights into the overall distribution of messages within your logging infrastructure. * **Persistency:** Message counts and last event times persist across rsyslog daemon restarts, ensuring continuous tracking. The module achieves this by periodically writing these statistics to a JSON :ref:`statefile `. Functionality ------------- The ``omsendertrack`` module operates through several key stages and mechanisms to ensure accurate and persistent sender tracking. Initialization ^^^^^^^^^^^^^^ Upon rsyslog startup, the ``omsendertrack`` module attempts to load its previously saved state from the configured :ref:`statefile `. This data, which includes sender identifiers, message counts, and last event times, is loaded into an in-memory hash table. This ensures that message statistics are restored and tracking continues seamlessly across daemon restarts. A background task is then spawned to handle periodic state persistence. OnAction Call (Message Processing) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When a message is routed to an ``omsendertrack`` action: 1. **Sender Identification:** The module uses the configured :ref:`senderid ` template to derive a unique identifier for the message sender. 2. **Statistic Update:** It then updates or inserts an entry for this sender in its internal hash table. 3. **Timestamp Recording:** The `last-event-time` for the sender is updated with the current UTC timestamp of the received message. 4. **Message Counting:** The `message-count` for that sender is incremented. 5. **Average Rate (Optional):** If configured, the `avg-message-count` (average message rate) may also be recalculated. HUP Signal Handling ^^^^^^^^^^^^^^^^^^^ When rsyslog receives a HUP signal (typically used for configuration reloads), the ``omsendertrack`` module is designed to check for the existence of a :ref:`cmdfile `. If a `cmdfile` is specified and found, it would be read and its commands processed. After processing, the `cmdfile` would be deleted to prevent re-execution on subsequent HUP signals. **Note:** Command file support is currently **not implemented** in this proof-of-concept version of the module. Background Task ^^^^^^^^^^^^^^^ A dedicated background task is responsible for persisting the module's current state to the configured :ref:`statefile `. This task wakes up at the `interval` specified in the configuration. It performs atomic writes to the `statefile` to prevent data corruption, even if rsyslog unexpectedly terminates during a write operation. Shutdown ^^^^^^^^ During rsyslog shutdown, the ``omsendertrack`` module ensures that the most current sender statistics are saved to the :ref:`statefile `. This critical step guarantees data persistence and allows for an accurate resumption of tracking when rsyslog restarts. Configuration ------------- The ``omsendertrack`` module supports the following action parameters. .. note:: Parameter names are case-insensitive. Action Parameters ----------------- senderid ^^^^^^^^ .. _omsendertrack_senderid: .. csv-table:: :header: "Type", "Default", "Mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "string", "RSYSLOG_FileFormat", "no", "none" This parameter defines the **template used to determine the sender's unique identifier**. The value produced by this template will be used as the key for tracking individual senders within the module's internal statistics. For instance: * A simple template like ``"%hostname%"`` will track each unique host that submits messages to rsyslog. * Using ``"%fromhost-ip%"`` will track senders based on their IP address. * A more granular template such as ``"%hostname%-%app-name%"`` can differentiate between applications on the same host. **Note:** The processing of this template for every incoming message can impact overall throughput, especially if complex templates are used. Choose your template wisely based on your tracking needs and performance considerations. .. important:: The current Proof-of-Concept implementation of the ``omsendertrack`` module might still refer to this parameter as ``template`` instead of ``senderid``. Please use ``template`` if ``senderid`` is not recognized by your rsyslog version, and be aware that this will be harmonized in future releases. interval ^^^^^^^^ .. _omsendertrack_interval: .. csv-table:: :header: "Type", "Default", "Mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "integer", "60", "no", "none" This parameter defines the **interval in seconds** after which the module writes the current sender statistics to the configured :ref:`statefile `. A smaller `interval` value results in more frequent updates to the state file, reducing potential data loss in case of an unexpected system crash, but it also increases disk I/O. A larger `interval` reduces I/O but means less up-to-date statistics on disk. statefile ^^^^^^^^^ .. _omsendertrack_statefile: .. csv-table:: :header: "Type", "Default", "Mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "string", "none", "yes", "none" This mandatory parameter specifies the **absolute path to the JSON file** where sender information will be stored. The module updates this file periodically based on the :ref:`interval ` and also upon rsyslog shutdown to preserve the latest statistics. **Important:** Ensure that the rsyslog user has appropriate write permissions to the directory where this `statefile` is located. Failure to do so will prevent the module from saving its state. cmdfile ^^^^^^^ .. _omsendertrack_cmdfile: .. csv-table:: :header: "Type", "Default", "Mandatory", "|FmtObsoleteName| directive" :widths: auto :class: parameter-table "string", "none", "no", "none" This optional parameter allows you to specify the **absolute path to a command file**. This file *is designed to be processed when rsyslog receives a HUP signal* (e.g., via `systemctl reload rsyslog`). **Note:** Command file support is currently **not implemented** in this proof-of-concept version of the module. When implemented, this feature is intended to allow dynamic control over the module's behavior, such as resetting statistics for specific senders, without requiring an rsyslog restart. Statistic Counter ----------------- The ``omsendertrack`` module is designed to maintain a set of statistics for each unique sender identifier it tracks. These statistics are intended to be periodically serialized and written to the configured :ref:`statefile ` in JSON format. **Important:** This module **does not offer statistics counters in the typical sense** that are consumable by other rsyslog modules like `impstats`. The collected data is primarily intended for direct consumption from the generated state file. **Note:** There are currently **no statistics counters available** in this proof-of-concept version of the module. The JSON structure for each sender entry is envisioned to look like this: .. code-block:: json { "senderid": "value_from_template", "last-event-time": "YYYY-MM-DDTHH:MM:SS.sssZ", "message-count": "N_VALUE", "avg-message-count": "M_POINT_M_VALUE" } Where: * ``senderid``: The unique identifier for the sender, as determined by the :ref:`senderid ` template. * ``last-event-time``: A UTC timestamp (ISO 8601 format) indicating when the last message from this sender was received. * ``message-count``: The total number of messages received from this sender since tracking began (or since the last reset). * ``avg-message-count``: (Optional) The average message rate from this sender since tracking began, calculated over the total elapsed time. This field's presence depends on future module configuration and implementation details. Usage within Rsyslog Configuration ---------------------------------- The ``omsendertrack`` module functions as an output module (OM), meaning you integrate its action where you want sender statistics to be collected within your rsyslog configuration. Each instance of ``omsendertrack`` counts its senders independently. **Queue Considerations:** It's technically possible to place the ``omsendertrack`` action within a dedicated ruleset that has a queue, or to add a queue directly to the action itself. However, ``omsendertrack`` processing is **extremely fast**, with the overhead of a queue often being multiple times greater than the actual call to the module. For this reason, adding a queue generally **does not make sense** for ``omsendertrack`` and is **not recommended** as it would introduce unnecessary complexity and potential latency without significant benefit. For optimal performance, always consider calling ``omsendertrack`` actions synchronously. This can be done within an existing ruleset, or by a synchronous ``call`` statement to a dedicated ruleset that has **no queue**. Best Practices -------------- To ensure efficient and correct operation of the ``omsendertrack`` module, adhere to the following best practices: * **Prioritize Synchronous Calls:** Always call ``omsendertrack`` actions synchronously. The module is highly optimized for quick processing, and asynchronous calls with queues are generally unnecessary and can introduce overhead without benefit. * **Avoid Queues on Dedicated Rulesets:** If you use a dedicated ruleset to house the ``omsendertrack`` action (as shown in Example 2), ensure that this specific ruleset **does not have a queue configured**. The module's fast execution makes queues redundant here. * **Efficient Sender Identification:** Choose your :ref:`senderid ` template carefully. Simpler templates (e.g., ``"%hostname%"``, ``"%fromhost-ip%"``) result in better performance, as template processing occurs for every message. * **Appropriate `interval` for State File Writes:** Balance your need for up-to-date statistics against disk I/O. A very small `interval` can lead to increased disk writes, while a larger one might mean slightly older data on disk in case of an unexpected shutdown. * **Ensure State File Write Permissions:** Verify that the rsyslog user has proper write permissions to the directory specified in the :ref:`statefile ` parameter. Without this, statistics cannot be persisted. * **Dedicated Ruleset for Unified Stats:** Use a dedicated ruleset that is called from multiple input-bound rulesets (Example 2) **only when** you need to collect statistics from those diverse inputs into a **single, unified sender statistics file**. * **Multiple Instances for Separate Stats:** Deploy multiple ``omsendertrack`` action instances (Example 3) **only when** you explicitly desire to generate **separate sender statistics files** based on different filtering criteria or input sources. Do not create multiple instances if a single, aggregated statistic file is your goal. Examples -------- Let's look at some examples of how to configure the ``omsendertrack`` module. Example 1: Basic Configuration ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This is the simplest way to use ``omsendertrack``. It loads the module and configures it to track senders based on their hostname, updating statistics every 60 seconds and storing them in a state file. This approach is suitable when all messages you wish to track are processed within a single ruleset or when the overall volume is low. .. code-block:: rsyslog module(load="omsendertrack") action(type="omsendertrack" senderid="%hostname%" interval="60" statefile="/var/lib/rsyslog/senderstats.json") Example 2: Usage with Dedicated Ruleset ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A dedicated ruleset for ``omsendertrack`` is suggested **specifically when** you need to count senders where the incoming messages are bound to **different rulesets**, **and** you want all those messages to contribute to a **single, unified sender statistics file**. This example shows how to set up ``omsendertrack`` within a dedicated ruleset, which is then called synchronously from multiple input-bound rulesets. This allows you to centralize sender tracking while maintaining separate message processing flows for other actions. .. code-block:: rsyslog # Define the template for senderid in omsendertrack template(name="id-template" type="list") { property(name="hostname") } # Ruleset omsendertrack-ruleset: Must only contain the omsendertrack action # This ruleset should NOT have a queue. ruleset(name="omsendertrack-ruleset") { action( type="omsendertrack" senderid="id-template" interval="60" statefile="/var/lib/rsyslog/senderstats.json" cmdfile="/var/lib/rsyslog/sendercommands.txt" ) } # Ruleset a: Calls omsendertrack-ruleset synchronously, then forwards messages ruleset(name="a" queue.type="LinkedList" queue.spoolDirectory="/var/lib/rsyslog/queue_a" queue.fileName="q_a" queue.maxDiskSpace="1g" queue.saveOnShutdown="on" queue.discardSeverity="8" queue.discardMark="1" ) { call omsendertrack-ruleset action( type="omfwd" target="192.0.2.1" port="10514" protocol="udp" ) action( type="omfwd" target="192.0.2.2" port="10514" protocol="tcp" ) } # Ruleset b: Calls omsendertrack-ruleset synchronously, then forwards messages ruleset(name="b" queue.type="LinkedList" queue.spoolDirectory="/var/lib/rsyslog/queue_b" queue.fileName="q_b" queue.maxDiskSpace="1g" queue.saveOnShutdown="on" queue.discardSeverity="8" queue.discardMark="1" ) { call omsendertrack-ruleset action( type="omfwd" target="192.0.2.3" port="514" protocol="udp" ) action( type="omfwd" target="192.0.2.4" port="514" protocol="tcp" ) } # Input for ruleset a (example: UDP input) input(type="imudp" port="5140" ruleset="a") # Input for ruleset b (example: TCP input) input(type="imtcp" port="5141" ruleset="b") # Default ruleset (if messages don't match other inputs) # This is here for completeness, you can remove or modify it as needed. ruleset(name="RSYSLOG_DefaultRuleset") { stop } Example 3: Multiple Instances for Separate Statistics ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ It is possible to use multiple instances of ``omsendertrack`` if it is desired to create **separate sender statistics files** based on different criteria. For example, you might want to track UDP senders and TCP senders in distinct state files. .. code-block:: rsyslog # Track UDP senders in a separate state file ruleset(name="udp-sender-tracking") { action( type="omsendertrack" senderid="%fromhost-ip%" interval="300" statefile="/var/lib/rsyslog/udp_sender_stats.json" ) # Add other actions for UDP messages here (e.g., forwarding, writing to file) } # Track TCP senders in another state file ruleset(name="tcp-sender-tracking") { action( type="omsendertrack" senderid="%fromhost-ip%" interval="300" statefile="/var/lib/rsyslog/tcp_sender_stats.json" ) # Add other actions for TCP messages here } # Bind inputs to the respective sender tracking rulesets input(type="imudp" port="514" ruleset="udp-sender-tracking") input(type="imtcp" port="514" ruleset="tcp-sender-tracking") # Further processing for all messages (e.g., default ruleset) ruleset(name="RSYSLOG_DefaultRuleset") { stop }