Fault Injection

    The scope of failures is restricted to those that are observable by an application communicating over the network. CPU and disk failures on the local host cannot be emulated.

    Currently, the fault injection filter has the following limitations:

    • Delays are restricted to fixed duration.

    Future versions will include support for restricting faults to specific routes, injecting gRPC and HTTP/2 specific error codes and delay durations based on distributions.

    Note

    The fault injection filter must be inserted before any other filter, including the router filter.

    fault.http.abort.abort_percent

    % of requests that will be aborted if the headers match. Defaults to the abort_percent specified in config. If the config does not contain an abort block, then abort_percent defaults to 0.

    fault.http.abort.http_status

    HTTP status code that will be used as the of requests that will be aborted if the headers match. Defaults to the HTTP status code specified in the config. If the config does not contain an abort block, then http_status defaults to 0.

    fault.http.delay.fixed_delay_percent

    fault.http.delay.fixed_duration_ms

    The delay duration in milliseconds. If not specified, the fixed_duration_ms specified in the config will be used. If this field is missing from both the runtime and the config, no delays will be injected.

    Note, fault filter runtime settings for the specific downstream cluster override the default ones if present. The following are downstream specific runtime keys:

    • fault.http..abort.http_status
    • fault.http..delay.fixed_delay_percent
    • fault.http..delay.fixed_duration_ms

    Downstream cluster name is taken from header. If the following settings are not found in the runtime it defaults to the global runtime settings which defaults to the config settings.

    The fault filter outputs statistics in the http..fault. namespace. The stat prefix comes from the owning HTTP connection manager.