Decoders

Original message generated by the application:

Then the Docker log message become encapsulated as follows:

as you can see the original message is handled as an escaped string. Ideally in Fluent Bit we would like to keep having the original structured message and not a string.

Decode_Field: if the content can be decoded in a structured message, append that structure message (keys and values) to the original log message.

Our pre-defined Docker Parser have the following definition:

Each line in the parser with a key Decode_Field instruct the parser to apply a specific decoder on a given field, optionally it offer the option to take an extra action if the decoder cannot succeed.

By default if a decoder fails to decode the field or want to try a next decoder, is possible to define an optional action. Available actions are:

Name	Description
try_next	if the decoder failed, apply the next Decoder in the list for the same field.
do_next	if the decoder succeeded or failed, apply the next Decoder in the list for the same field.

on Decode_Field_As, if succeeded, another decoder of the same type in the same field can be applied only if the data continues being an unstructured message (raw text).
on Decode_Field, if succeeded, can only be applied once for the same field. By nature Decode_Field aims to decode a structured message.

Example input (from /path/to/log.log in configuration below)

{"log":"\u0009Checking indexes...\n","stream":"stdout","time":"2018-02-19T23:25:29.1845444Z"}
{"log":"\u0009\u0009Validated: _audit _internal _introspection _telemetry _thefishbucket history main snmp_data summary\n","stream":"stdout","time":"2018-02-19T23:25:29.1845536Z"}
{"log":"\u0009Done\n","stream":"stdout","time":"2018-02-19T23:25:29.1845622Z"}

Example output

Configuration file


    Parsers_File fluent-bit-parsers.conf
[INPUT]
    Name        tail
    Path        /path/to/log.log
[OUTPUT]
    Name   stdout