Throttle

    The plugin supports the following configuration parameters:

    Functional description

    Lets imagine we have configured:

    we received 1 message first second, 3 messages 2nd, and 5 3rd. As you can see, disregard that Window is actually 5, we use “slow” start to prevent overflooding during the startup.

    2. |1|3|5| | | | | 
    3. +-------+-+-+-+ 
    4. |  3  |         average = 3, and not 1.8 if you calculate 0 for last 2 panes. 
    5. +-----+

    But as soon as we reached Window size * Interval, we will have true sliding window with aggregation over complete window.

    1. +-------------+ 
    2. |1|3|5|7|3|4| | 
    3. +-------------+ 
    4.   |  4.4    |   
    5.   ----------+

    will become:

    1. |1|3|5|7|3|4|6|
    2. +-------------+
    3.     |   5     |

    As you can see, last pane of the window was overwritten and 1 message was dropped.

    You might noticed possibility to configure Interval of the Window shift. It is counter intuitive, but there is a difference between two examples above:

    1. Rate 60
    2. Window 5
    3. Interval 1m

    and

    1. XX        XX        XX
    2. XX        XX        XX
    3. XX        XX        XX
    4. XX        XX        XX
    5. XX        XX        XX
    6. XX        XX        XX
    7. +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

    While the second example will not allow more than 1 message per second every second, making output rate more smooth:

    1.   X    X     X    X    X    X
    2. +-+-+-+-+-+--+-+-+-+-+-+-+-+-+-+

    It may drop some data if the rate is ragged. I would recommend to use bigger interval and rate for streams of rare but important events, while keep Window bigger and Interval small for constantly intensive inputs.

    The following command will load the tail plugin and read the content of lines.txt file. Then the throttle filter will apply a rate limit and only pass the records which are read below the certain rate:

    2.     Name   tail
    3.     Path   lines.txt
    5. [FILTER]
    6.     Name     throttle
    7.     Match    *
    8.     Rate     1000
    9.     Window   300
    10.     Interval 1s
    12. [OUTPUT]
    13.     Name   stdout
    14.     Match  *