batch.format¶

Chooses how omhttp combines multiple messages when batching is enabled.

This parameter applies to omhttp: HTTP Output Module.

Name:: batch.format
Scope:: input
Type:: word
Default:: input=newline
Required?:: no
Introduced:: Not specified

Description¶

This parameter specifies how to combine multiple messages into a single batch. Valid options are newline (default), jsonarray, kafkarest, and lokirest.

Each message on the “Inputs” line is the templated log line that is fed into the omhttp action, and the “Output” line describes the resulting payload sent to the configured HTTP server.

newline - Concatenates each message into a single string joined by newline ("\\n") characters. This mode is default and places no restrictions on the structure of the input messages.

Inputs: "message 1" "message 2" "message 3"
Output: "message 1\nmessage 2\nmessage 3"

jsonarray - Builds a JSON array containing all messages in the batch. This mode requires that each message is parsable JSON, since the plugin parses each message as JSON while building the array.

Inputs: {"msg": "message 1"} {"msg": "message 2"} {"msg": "message 3"}
Output: [{"msg": "message 1"}, {"msg": "message 2"}, {"msg": "message 3"}]

kafkarest - Builds a JSON object that conforms to the Kafka Rest Proxy specification. This mode requires that each message is parsable JSON, since the plugin parses each message as JSON while building the batch object.

Inputs: {"msg": "message 1"} {"msg": "message 2"} {"msg": "message 3"}
Output: {"records": [{"value": {"msg": "message 1"}}, {"value": {"msg": "message 2"}}, {"value": {"msg": "message 3"}}]}

lokirest - Builds a JSON object that conforms to the Loki Rest specification. This mode requires that each message is parsable JSON, since the plugin parses each message as JSON while building the batch object. Additionally, the operator is responsible for providing index keys, and message values.

Inputs: {"stream": {"tag1":"value1"}, values:[[ "%timestamp%", "message 1" ]]} {"stream": {"tag2":"value2"}, values:[[ "%timestamp%", "message 2" ]]}
Output: {"streams": [{"stream": {"tag1":"value1"}, values:[[ "%timestamp%", "message 1" ]]},{"stream": {"tag2":"value2"}, values:[[ "%timestamp%", "message 2" ]]}]}

Input usage¶

module(load="omhttp")

action(
    type="omhttp"
    batch="on"
    batchFormat="jsonarray"
)

batch.format¶

Description¶

Input usage¶

See also¶