Phase 1 of the omotlp output plugin streams OpenTelemetry logs over OTLP/HTTP JSON with configurable batching, gzip compression, retry/backoff controls, TLS/mTLS support for secure HTTPS connections, and HTTP proxy support for corporate networks and firewalled environments.

omotlp: OpenTelemetry output module (preview)

Warning

The OTLP/HTTP JSON transport is ready for preview deployments. The optional OTLP/gRPC façade and HTTP/protobuf fast-path remain under development, so keep action queues enabled and plan future upgrades accordingly.

Overview

omotlp prepares rsyslog for native OTLP exports. Phase 1 focuses on the OTLP/HTTP JSON transport path: the module maps rsyslog metadata into the canonical OTLP JSON structure, joins the configured endpoint and path, and posts batches of rendered payloads via libcurl using application/json semantics. Batching thresholds, gzip compression, retry and backoff policies, custom headers, TLS/mTLS authentication, and HTTP proxy support are all configurable. Subsequent phases will extend the module with the gRPC façade and, optionally, HTTP/protobuf support.

Availability

The module is built only when ./configure is invoked with --enable-omotlp=yes and both libcurl and libfastjson are present. The default --enable-omotlp setting is no, so you must opt in explicitly. The HTTP transport depends on libcurl at runtime.

Configuration

The action parameters listed below mirror the current transport design. All parameters are optional and fall back to sensible defaults inspired by the OTLP specification.

omotlp action parameters

Parameter	Type	Default	Description
endpoint	string	http://127.0.0.1:4318	Base OTLP collector URL
path	string	/v1/logs	Request path appended to the endpoint
protocol	word	http/json	Transport variant
template	word	RSYSLOG_FileFormat	Message template used for the log body
timeout.ms	integer	10000	HTTP request timeout in milliseconds
compression	word	none	Request payload compression (none or gzip)
batch.max_items	integer	512	Flush the batch after this many records (0 disables the limit)
batch.max_bytes	integer	524288	Approximate uncompressed payload threshold in bytes (0 disables the limit)
batch.timeout.ms	integer	5000	Flush the oldest batch after this idle period in milliseconds (0 disables the user-configured timer, but a short safety flush still runs periodically to avoid indefinite buffering)
retry.initial.ms	integer	1000	Initial backoff before retrying retryable HTTP responses
retry.max.ms	integer	30000	Maximum backoff applied between retries
retry.max_retries	integer	5	Attempts made before returning the batch to the action queue
retry.jitter.percent	integer	20	Jitter applied to retry delays (0–100)
headers	string (JSON object)	—	Additional HTTP headers expressed as a JSON object
bearer_token	string	—	Convenience token that expands to Authorization: Bearer <token>
resource	string (JSON)	—	Full resource attributes as JSON object. Merged with automatic attributes (service.name, telemetry.sdk.*). Supports string, integer, and boolean values (boolean exported as OTLP boolValue).
resource.service_instance_id	string	—	Legacy: service.instance.id attribute (deprecated, use resource JSON)
resource.deployment.environment	string	—	Legacy: deployment.environment attribute (deprecated, use resource JSON)
trace_id.property	string	trace_id	Property name containing trace ID (32 hex characters = 128 bits)
span_id.property	string	span_id	Property name containing span ID (16 hex characters = 64 bits)
trace_flags.property	string	trace_flags	Property name containing trace flags (hex string, 0-255)
attributeMap	string (JSON object)	—	Custom mapping of rsyslog properties to OTLP attribute names. JSON object with rsyslog property names as keys and OTLP attribute names as values. Supported properties: hostname, appname, procid, msgid, facility.
severity.map	string (JSON object)	—	Custom mapping of syslog priorities to OTLP severity. JSON object with priority numbers (0-7) as keys and objects with number (OTLP severity number) and text (OTLP severity text) as values.
tls.cacert	string	—	Path to CA certificate bundle file (PEM format)
tls.cadir	string	—	Path to directory containing CA certificates
tls.cert	string	—	Path to client certificate file (PEM format) for mTLS
tls.key	string	—	Path to client private key file (PEM format) for mTLS
tls.verify_hostname	boolean	on	Enable/disable hostname verification in certificate
tls.verify_peer	boolean	on	Enable/disable peer certificate verification
proxy	string	—	Proxy server URL. Must include scheme: http://, https://, socks4://, or socks5://. Example: http://proxy.example.com:8080. The proxy type is automatically detected from the URL scheme. If omitted, direct connections are used.
proxy.user	string	—	Proxy username for authentication. Required when proxy requires authentication. Used with proxy.password to enable basic or digest authentication. If proxy.user is provided without proxy.password, an empty password is used.
proxy.password	string	—	Proxy password for authentication. Required when proxy requires authentication. Used together with proxy.user. The authentication method (basic or digest) is automatically negotiated with the proxy server.

Batch sizes are estimated from the body length plus per-record overhead so the module can limit payloads without rendering JSON for each candidate message. When a threshold is reached the batch is flushed immediately; otherwise it is sent when either the timeout elapses or rsyslog finishes the current queue transaction.

Environment variables from the OpenTelemetry specification are consulted when a configuration omits explicit values. endpoint falls back to OTEL_EXPORTER_OTLP_LOGS_ENDPOINT and then OTEL_EXPORTER_OTLP_ENDPOINT; protocol and compression honour the matching *_PROTOCOL and *_COMPRESSION variables; timeout.ms defaults to OTEL_EXPORTER_OTLP_LOGS_TIMEOUT or OTEL_EXPORTER_OTLP_TIMEOUT; and headers consumes OTEL_EXPORTER_OTLP_LOGS_HEADERS or the generic OTEL_EXPORTER_OTLP_HEADERS when no explicit headers are configured. Action parameters always take precedence, giving operators an easy override when reusing existing collector deployments.

When the endpoint string includes an explicit path (for example, https://otel:4318/v1/logs), the module automatically splits the final path segment into the path parameter so that future HTTP transport code can join the pieces without duplicating /v1/logs.

Example

The example below batches up to 200 records or 256 KiB every two seconds, uses gzip compression, and adds a custom tenant header alongside an Authorization bearer token read from the environment. Retry/backoff settings align with the default action queue behaviour: HTTP status codes 429 and 5xx trigger RS_RET_SUSPENDED so queued records are retried with jittered backoff, while other non-success responses discard the message and log an error.

module(load="omotlp")
action(
  type="omotlp"
  endpoint="https://otel-collector:4318"
  path="/v1/logs"
  protocol="http/json"
  template="RSYSLOG_SyslogProtocol23Format"
  batch.max_items="200"
  batch.max_bytes="262144"
  batch.timeout.ms="2000"
  compression="gzip"
  retry.max_retries="7"
  retry.max.ms="45000"
  headers='{ "X-OTel-Tenant": "blue" }'
  bearer_token="${env:OTEL_TOKEN}"
)

Note

When using an https:// endpoint, TLS configuration is recommended for proper certificate verification. If the system’s default CA certificates are sufficient, TLS parameters may be omitted. For production deployments or when using custom CA certificates or mTLS, configure the appropriate tls.* parameters as shown in the TLS Configuration Example. For environments requiring proxy support (corporate networks, firewalls), see the Proxy Configuration Example.

Trace Correlation Example

The following example demonstrates how to extract trace context from JSON messages and populate OTLP trace correlation fields. The trace context is extracted from the message using mmjsonparse and then exported via omotlp with trace correlation enabled.

module(load="mmjsonparse")
module(load="omotlp")

# Parse JSON messages to extract trace properties
action(type="mmjsonparse" mode="find-json")

# Export with trace correlation
action(
  type="omotlp"
  endpoint="https://otel-collector:4318"
  path="/v1/logs"
  trace_id.property="trace_id"
  span_id.property="span_id"
  trace_flags.property="trace_flags"
)

In this example, if a JSON message contains fields like trace_id, span_id, or trace_flags, they will be extracted and included in the OTLP export. The property names can be customized using the trace_id.property, span_id.property, and trace_flags.property parameters.

Resource Configuration Example

The following example demonstrates how to configure full resource attributes using the resource parameter with a JSON object. This enables full OpenTelemetry resource semantic conventions compliance.

module(load="omotlp")
action(
  type="omotlp"
  endpoint="https://otel-collector:4318"
  path="/v1/logs"
  resource='{
    "service.name": "my-application",
    "service.instance.id": "${hostname}",
    "service.version": "1.2.3",
    "deployment.environment": "production",
    "cloud.provider": "aws",
    "cloud.region": "us-east-1",
    "k8s.namespace.name": "default",
    "k8s.pod.name": "${hostname}"
  }'
)

The resource attributes are merged with automatic attributes (service.name, telemetry.sdk.*) that are always added by the module. The resource JSON supports string, integer, and boolean values. Boolean values are exported as OTLP boolean attributes using the boolValue field.

TLS Configuration Example

The following example demonstrates how to configure TLS/mTLS for secure HTTPS connections. This enables server certificate verification using a CA certificate bundle and optional mutual TLS authentication with client certificates.

module(load="omotlp")
action(
  type="omotlp"
  endpoint="https://otel-collector:4318"
  path="/v1/logs"
  tls.cacert="/etc/ssl/certs/ca-bundle.pem"
  tls.cert="/etc/rsyslog/client.pem"
  tls.key="/etc/rsyslog/client-key.pem"
  tls.verify_hostname="on"
  tls.verify_peer="on"
)

The TLS parameters support:

• tls.cacert: Path to a CA certificate bundle file (PEM format) for server certificate verification

• tls.cadir: Path to a directory containing CA certificates (alternative to tls.cacert)

• tls.cert: Path to a client certificate file (PEM format) for mutual TLS authentication

• tls.key: Path to a client private key file (PEM format) for mutual TLS authentication

• tls.verify_hostname: Enable/disable hostname verification in the server certificate (default: on)

• tls.verify_peer: Enable/disable peer certificate verification (default: on)

File paths are validated at configuration time. Invalid or inaccessible certificate files will cause configuration to fail with an error message indicating the specific file and reason (e.g., “tls.cacert file ‘/path/to/ca.pem’ cannot be accessed: No such file or directory”). If both tls.cacert and tls.cadir are specified, both are used (libcurl behavior). Client certificate and key must be provided together for mTLS. The default behavior verifies both hostname and peer certificate for secure connections.

When TLS verification fails at runtime (e.g., certificate mismatch, expired certificate, or hostname mismatch), the HTTP request fails and the batch is retried according to the configured retry policy. TLS verification errors are logged with details from libcurl (e.g., “SSL: certificate subject name does not match target host name”). For test environments or self-signed certificates, you may need to set tls.verify_hostname="off" if certificates don’t have proper Subject Alternative Name (SAN) entries matching the endpoint hostname.

Note that tls.verify_hostname requires tls.verify_peer to be enabled (on) for meaningful verification. If peer verification is disabled, hostname verification is effectively ignored.

Proxy Configuration Example

The following example demonstrates how to configure HTTP proxy support for corporate networks or environments that require traffic to pass through a proxy server. Proxy support enables omotlp to work through firewalls, network gateways, and corporate proxies.

module(load="omotlp")
action(
  type="omotlp"
  endpoint="https://otel-collector:4318"
  path="/v1/logs"
  proxy="http://proxy.example.com:8080"
)

For proxies that require authentication, provide both username and password:

module(load="omotlp")
action(
  type="omotlp"
  endpoint="https://otel-collector:4318"
  path="/v1/logs"
  proxy="http://proxy.example.com:8080"
  proxy.user="myuser"
  proxy.password="mypassword"
)

The proxy parameters support:

• proxy: Full proxy server URL including scheme and port. Supported schemes:

  • http:// - HTTP proxy (most common for corporate proxies)

  • https:// - HTTPS proxy (secure proxy connection)

  • socks4:// - SOCKS4 proxy

  • socks5:// - SOCKS5 proxy (supports authentication and UDP)

  The proxy URL format is validated at configuration time. Invalid schemes or malformed URLs will cause configuration to fail with an error message.

• proxy.user: Username for proxy authentication. When provided, the module automatically sends proxy authentication credentials with each request. The authentication method (basic or digest) is automatically negotiated with the proxy server based on the proxy’s requirements.

• proxy.password: Password for proxy authentication. Must be provided together with proxy.user when the proxy requires authentication. If proxy.user is specified without proxy.password, an empty password is used (which may work for some proxy configurations).

Proxy authentication uses HTTP Basic authentication by default. If the proxy server requires Digest authentication, libcurl automatically upgrades to Digest when the proxy responds with a 407 (Proxy Authentication Required) status code. The authentication credentials are sent in the Proxy-Authorization header for each HTTP request.

When a proxy is configured, all HTTP requests to the OTLP collector endpoint are routed through the proxy server. The proxy acts as an intermediary, forwarding requests to the target collector and returning responses. This enables omotlp to work in environments where:

• Direct connections to the collector are blocked by firewall rules

• Network traffic must pass through a corporate proxy gateway

• Network policies require all outbound HTTP/HTTPS traffic to use a proxy

• The collector is only accessible through a proxy server

Proxy configuration works with both HTTP and HTTPS endpoints. When using an HTTPS endpoint through an HTTP proxy, the proxy performs HTTP CONNECT tunneling to establish the secure connection. SOCKS proxies (SOCKS4 and SOCKS5) can also handle both HTTP and HTTPS traffic transparently.

If proxy authentication fails (invalid credentials or unsupported authentication method), the HTTP request fails and the batch is retried according to the configured retry policy. Authentication errors are logged with details from libcurl (e.g., “Proxy authentication required” or “Invalid proxy credentials”).

Note that proxy configuration is independent of TLS configuration. You can use both proxy and TLS parameters together. For example, you might route HTTPS traffic through an HTTP proxy, or use an HTTPS proxy with TLS verification enabled for the collector endpoint.

Legacy single-attribute parameters (resource.service_instance_id and resource.deployment.environment) are still supported for backward compatibility but may be deprecated in future versions.

Trace IDs must be exactly 32 hexadecimal characters (128 bits), span IDs must be exactly 16 hexadecimal characters (64 bits), and trace flags are parsed as hexadecimal values (0-255). Invalid values are logged but do not cause message processing to fail.

OTLP Payload Structure

The module generates OTLP/HTTP JSON payloads conforming to the OpenTelemetry log data model. Each batch wraps log records in the following hierarchy:

Resource attributes (per-batch, automatically populated):

Attribute	Value
service.name	rsyslog
telemetry.sdk.name	rsyslog-omotlp
telemetry.sdk.language	C
telemetry.sdk.version	rsyslog version string
host.name	hostname (only if uniform across all records in batch)

Scope metadata (per-batch):

Field	Value
scope.name	rsyslog.omotlp
scope.version	rsyslog version string

Per-record attributes (derived from syslog metadata):

Attribute	Source
log.syslog.appname	syslog APP-NAME field (customizable via attributeMap)
log.syslog.procid	syslog PROCID field (customizable via attributeMap)
log.syslog.msgid	syslog MSGID field (customizable via attributeMap)
log.syslog.facility	syslog facility code (integer, customizable via attributeMap)
log.syslog.hostname	syslog HOSTNAME field (customizable via attributeMap)

Per-record fields:

• body.stringValue: rendered template output

• timeUnixNano: message timestamp in nanoseconds

• observedTimeUnixNano: reception timestamp in nanoseconds

• severityNumber: mapped OTLP severity (see table below)

• severityText: severity name string

• traceId: trace ID string (32 hex characters, if present in message properties)

• spanId: span ID string (16 hex characters, if present in message properties)

• flags: trace flags integer (0-255, if present in message properties)

All of the attributes and fields above are emitted for every record; values are only omitted when the originating syslog message does not carry the associated property (for example, when APP-NAME is empty). Trace correlation fields (traceId, spanId, flags) are populated from message JSON variables when present and valid.

Severity Mapping

Syslog priority values are mapped to OTLP severity numbers following the OpenTelemetry semantic conventions. The default mapping can be overridden using the severity.map parameter.

Default severity mapping:

Syslog Priority	OTLP SeverityNumber	OTLP SeverityText
0 (Emergency)	24	EMERGENCY
1 (Alert)	23	ALERT
2 (Critical)	22	CRITICAL
3 (Error)	17	ERROR
4 (Warning)	13	WARNING
5 (Notice)	11	NOTICE
6 (Info)	9	INFO
7 (Debug)	5	DEBUG

Custom Attribute and Severity Mapping Example

The following example demonstrates how to customize attribute names and severity mapping to match collector expectations or custom severity schemes:

module(load="omotlp")
action(
  type="omotlp"
  endpoint="https://otel-collector:4318"
  path="/v1/logs"
  attributeMap='{
    "hostname": "host.name",
    "appname": "service.name",
    "procid": "process.pid",
    "msgid": "message.id",
    "facility": "log.syslog.facility.code"
  }'
  severity.map='{
    "0": { "number": 24, "text": "FATAL" },
    "1": { "number": 23, "text": "ALERT" },
    "2": { "number": 22, "text": "CRITICAL" },
    "3": { "number": 17, "text": "ERROR" },
    "4": { "number": 13, "text": "WARN" },
    "5": { "number": 11, "text": "NOTICE" },
    "6": { "number": 9, "text": "INFO" },
    "7": { "number": 5, "text": "DEBUG" }
  }'
)

The attributeMap parameter allows you to remap standard syslog properties to different OTLP attribute names. This is useful when integrating with collectors that expect different attribute naming conventions.

The severity.map parameter allows you to customize the mapping of syslog priorities (0-7) to OTLP severity numbers and text. All 8 priorities must be specified in the mapping, or defaults will be used for missing priorities. Custom severity numbers should follow OTLP severity conventions (0-24 range recommended).

Statistic Counter

This plugin maintains statistics for each worker instance. The statistic origin is named “omotlp” with the instance URL appended (e.g., “omotlp-http://127.0.0.1:4318/v1/logs”). Statistics are visible via the impstats module when enabled.

The following counters are tracked per worker instance:

• batches.submitted - Total number of batches submitted for transmission. Each batch may contain multiple log records.

• batches.success - Number of batches successfully sent (HTTP 2xx response).

• batches.retried - Number of batches that triggered retry logic due to retryable HTTP errors (5xx or 429 status codes).

• batches.dropped - Number of batches dropped due to non-retryable errors (HTTP 4xx status codes).

• http.status.4xx - Total number of HTTP 4xx responses received from the collector.

• http.status.5xx - Total number of HTTP 5xx responses received from the collector.

• records.sent - Total number of log records successfully sent to the collector (only incremented for successful batches).

• http.request.latency.ms - Cumulative request latency in milliseconds across all HTTP requests. This value represents the total time spent waiting for HTTP responses and can be used to calculate average latency by dividing by the number of requests.

Counters are thread-safe and use atomic operations for updates. Each worker instance maintains its own statistics object, allowing operators to monitor performance per action instance when multiple omotlp actions are configured.

---

Support: rsyslog Assistant | GitHub Discussions | GitHub Issues: rsyslog source project

Contributing: Source & docs: rsyslog source project

© 2008–2025 Rainer Gerhards and others. Licensed under the Apache License 2.0.