Traffic Log

    The logs can be then forwarded to a collector that can further transmit them into systems like Splunk, ELK and Datadog.

    Configuring access logs in Kuma is a 2-step process:

    1. First, you need to configure logging backends that will be available for use in a given Mesh.

      A logging backend is essentially a sink for access logs.

      In the current release of Kuma, a logging backend can be either a file or a TCP log collector, such as Logstash.

    2. Second, you need to create a TrafficLog policy to select a subset of traffic and forward its access logs into one of the logging backends configured for that Mesh.

    1. type: TrafficLog
    2. name: all-traffic
    3. mesh: default
    4. # This TrafficLog policy applies to all traffic in the Mesh.
    5. sources:
    6.   - match:
    7.       service: '*'
    8. destinations:
    9.   - match:
    10.       service: '*'
    11. # When `backend ` field is omitted, the logs will be forwarded into the `defaultBackend` of that Mesh.
    1. type: TrafficLog
    2. name: backend-to-database-traffic
    3. # this TrafficLog policy applies only to traffic from service `backend` to service `database`.
    4. sources:
    5.   - match:
    6.       service: backend
    7. destinations:
    8.   - match:
    9.       service: database
    10. conf:
    11.   # Forward the logs into the logging backend named `logstash`.
    12.   backend: logstash

    On Kubernetes

    1. apiVersion: kuma.io/v1alpha1
    2. kind: TrafficLog
    3. metadata:
    4.   namespace: kuma-example
    6. mesh: default
    7. spec:
    8.   # This TrafficLog policy applies all traffic in that Mesh.
    9.   sources:
    10.     - match:
    11.         service: '*'
    12.     - match:
    13.         service: '*'
    14.   # When `backend ` field is omitted, the logs will be forwarded into the `defaultBackend` of that Mesh.
    1. apiVersion: kuma.io/v1alpha1
    2. kind: TrafficLog
    3. metadata:
    4.   namespace: kuma-example
    5.   name: backend-to-database-traffic
    6. spec:
    7.   # This TrafficLog policy applies only to traffic from service `backend` to service `database`.
    8.   sources:
    9.     - match:
    10.         service: backend.kuma-example.svc:8080
    11.   destinations:
    12.     - match:
    13.         service: database.kuma-example.svc:5432
    14.   conf:
    15.     # Forward the logs into the logging backend named `logstash`.
    16.     backend: logstash

    Kuma gives you full control over the format of access logs.

    The shape of a single log record is defined by a template string that uses to extract and format data about a TCP connection or an HTTP request.

    E.g.,

    where %START_TIME% and are examples of available command operators.

    A complete set of supported command operators consists of:

    1. All command operators supported by EnvoyTraffic Log - 图2
    2. Command operators unique to Kuma

    The latter include:

    Access Logs for TCP and HTTP traffic

    If a command operator is specific to HTTP traffic, such as %REQ(X?Y):Z% or %RESP(X?Y):Z%, it will be replaced by a symbol “-“ in case of TCP traffic.

    Internally, Kuma determines traffic protocol based on the value of protocol tag on the inbound interface of a destination Dataplane.

    The default format string for TCP traffic is:

      The default format string for HTTP traffic is:

      1. [%START_TIME%] %KUMA_MESH% "%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% "%REQ(X-FORWARDED-FOR)%" "%REQ(USER-AGENT)%" "%REQ(X-REQUEST-ID)%" "%REQ(:AUTHORITY)%" "%KUMA_SOURCE_SERVICE%" "%KUMA_DESTINATION_SERVICE%" "%KUMA_SOURCE_ADDRESS_WITHOUT_PORT%" "%UPSTREAM_HOST%"

      To provide different format for TCP and HTTP logging you can define two separate logging backends with the same address and different format. Then define two TrafficLog entity, one for TCP and one for HTTP with protocol: http selector.

      If you need an access log with entries in JSON format, you have to provide a template string that is a valid JSON object, e.g.

      Logging external services

      When running Kuma on Kubernetes you can also log the traffic to external services. To do it, the matched TrafficPermission destination section has to have wildcard * value. In such case %KUMA_DESTINATION_SERVICE% will have value and %UPSTREAM_HOST% will have an IP of the service.