Performance and Scalability

    The Istio data plane components, the Envoy proxies, handle data flowing throughthe system. The Istio control plane components, Pilot, Galley and Citadel, configurethe data plane. The data plane and control plane have distinct performance concerns.

    The Istio load tests mesh consistsof 1000 services and 2000 sidecars with 70,000 mesh-wide requests per second.After running the tests using Istio 1.4.2, we get the following results:

    • The Envoy proxy uses 0.5 vCPU and 50 MB memory per 1000 requests per second going through the proxy.
    • The service uses 0.6 vCPU per 1000 mesh-wide requests per second.
    • The Envoy proxy adds 6.3 ms to the 90th percentile latency.

    Pilot configures sidecar proxies based on user authored configuration files and the currentstate of the system. In a Kubernetes environment, Custom Resource Definitions (CRDs) and deploymentsconstitute the configuration and state of the system. The Istio configuration objects like gateways and virtualservices, provide the user-authored configuration.To produce the configuration for the proxies, Pilot processes the combined configuration and system statefrom the Kubernetes environment and the user-authored configuration.

    The control plane supports thousands of services, spread across thousands of pods with asimilar number of user authored virtual services and other configuration objects.Pilot’s CPU and memory requirements scale with the amount of configurations and possible system states.The CPU consumption scales with the following factors:

    • The rate of deployment changes.
    • The rate of configuration changes.
    • The number of proxies connecting to Pilot.

    however this part is inherently horizontally scalable.

    When is enabled,a single Pilot instance can support 1000 services, 2000 sidecars with 1 vCPU and 1.5 GB of memory.You can increase the number of Pilot instances to reduce the amount of time it takes for the configurationto reach all proxies.

    Data plane performance depends on many factors, for example:

    • Number of client connections
    • Target request rate
    • Request size and Response size
    • Number of proxy worker threads
    • Protocol
    • CPU cores

    The latency, throughput, and the proxies’ CPU and memory consumption are measured as a function of said factors.

    Since the sidecar proxy performs additional work on the data path, it consumes CPUand memory. As of Istio 1.1, a proxy consumes about 0.6 vCPU per 1000requests per second.

    Since the proxy normally doesn’t buffer the data passing through,request rate doesn’t affect the memory consumption.

    Since Istio injects a sidecar proxy on the data path, latency is an importantconsideration. Istio adds an authentication and a Mixer filter to the proxy. Everyadditional filter adds to the path length inside the proxy and affects latency.

    The Envoy proxy collects raw telemetry data after a response is sent to theclient. The time spent collecting raw telemetry for a request does not contributeto the total time taken to complete that request. However, since the workeris busy handling the request, the worker won’t start handling the next requestimmediately. This process adds to the queue wait time of the next request and affectsaverage and tail latencies. The actual tail latency depends on the traffic pattern.

    Inside the mesh, a request traverses the client-side proxy and then the server-sideproxy. This two proxies on the data path add about 6.3 ms to the 90th percentile latency at 1000 requests per second.The server-side proxy alone adds 1.7 ms to the 90th percentile latency.

    The default configuration of Istio 1.4.2 adds 6.3 ms to the 90th percentile latency of the data plane over the baseline.We obtained these results using the Istio benchmarksfor the http/1.1 protocol, with a 1 kB payload at 1000 requests per second using 16 client connections, 2 proxy workers and mutual TLS enabled.

    In upcoming Istio releases we are moving istio-policy and istio-telemetry functionality into the proxy as TelemetryV2.This will decrease the amount data flowing through the system, which will in turn reduce the CPU usage and latency.

    P90 latency vs client connections

    • Client pod directly calls the server pod, no sidecars are present.
    • server-sidecar Only server sidecar is present.
    • both-sidecars Client and server sidecars are present. This is a default case inside the mesh.
    • nomixer-both Same as both-sidecars without Mixer.
    • nomixer-server Same as server-sidecar without Mixer.
    • Same as both-sidecars but with telemetry v2. This is targeted to perform the same as “No Mixer” in the future.
    • telemetryv2-nullvm_serveronly Same as server-sidecar but with telemetry v2. This is targeted to perform the same as “No Mixer” in the future.

    Istio uses the following tools for benchmarking

    • fortio.org - a constant throughput load testing tool.
    • - a realistic cloud native application.
    • isotope - a synthetic application with configurable topology.

    Tools and guidance for evaluating Istio's data plane performance.

    An overview of Istio 1.1 performance.

    Egress Gateway Performance Investigation

    Verifies the performance impact of adding an egress gateway.

    Addressing application startup ordering and startup latency using AppSwitch.

    Delayering Istio with AppSwitch

    Automatic application onboarding and latency optimizations using AppSwitch.