Traefik & Kubernetes

    In early versions, Traefik supported Kubernetes only through the Kubernetes Ingress provider, which is a Kubernetes Ingress controller in the strict sense of the term.

    However, as the community expressed the need to benefit from Traefik features without resorting to (lots of) annotations, the Traefik engineering team developed a (CRD) for an IngressRoute type, defined below, in order to provide a better way to configure access to a Kubernetes cluster.

    All Steps for a Successful Deployment

    • Add/update all the Traefik resources definitions
    • Add/update the for the Traefik custom resources
    • Use Helm Chart or use a custom Traefik Deployment
      • Enable the kubernetesCRD provider
      • Apply the needed kubernetesCRD provider
    • Add all necessary Traefik custom resources

    Deprecated apiextensions.k8s.io/v1beta1 CRD

    The apiextensions.k8s.io/v1beta1 CustomResourceDefinition is deprecated in Kubernetes v1.16+ and will be removed in v1.22+.

    For Kubernetes v1.16+, please use the Traefik apiextensions.k8s.io/v1 CRDs instead.

    Initializing Resource Definition and RBAC

    Traefik Resource Definition

    RBAC for Traefik CRD

    1. apiVersion: rbac.authorization.k8s.io/v1
    2. kind: ClusterRole
    3. metadata:
    4.   name: traefik-ingress-controller
    6. rules:
    7.   - apiGroups:
    8.       - ""
    9.     resources:
    10.       - services
    11.       - endpoints
    12.       - secrets
    13.     verbs:
    14.       - get
    15.       - list
    16.       - watch
    17.   - apiGroups:
    18.       - extensions
    19.       - networking.k8s.io
    20.     resources:
    21.       - ingresses
    22.       - ingressclasses
    23.     verbs:
    24.       - get
    25.       - list
    26.       - watch
    27.   - apiGroups:
    28.       - extensions
    29.     resources:
    30.       - ingresses/status
    31.       - update
    32.   - apiGroups:
    33.       - traefik.containo.us
    34.     resources:
    35.       - middlewares
    36.       - middlewaretcps
    37.       - ingressroutes
    38.       - traefikservices
    39.       - ingressroutetcps
    40.       - ingressrouteudps
    41.       - tlsoptions
    43.       - serverstransports
    44.     verbs:
    45.       - get
    46.       - list
    47.       - watch
    49. ---
    50. apiVersion: rbac.authorization.k8s.io/v1
    51. kind: ClusterRoleBinding
    52. metadata:
    53.   name: traefik-ingress-controller
    55. roleRef:
    56.   apiGroup: rbac.authorization.k8s.io
    57.   kind: ClusterRole
    58.   name: traefik-ingress-controller
    59. subjects:
    60.   - kind: ServiceAccount
    61.     name: traefik-ingress-controller
    62.     namespace: default

    When using KubernetesCRD as a provider, Traefik uses to retrieve its routing configuration. Traefik Custom Resource Definitions are a Kubernetes implementation of the Traefik concepts. The main particularities are:

    • The usage of name and namespace to refer to another Kubernetes resource.
    • The usage of secret for sensitive data (TLS certificates and credentials).
    • The structure of the configuration.
    • The requirement to declare all the .

    The Traefik CRDs are building blocks that you can assemble according to your needs. See the list of CRDs in the dedicated routing section.

    By design, Traefik is a stateless application, meaning that it only derives its configuration from the environment it runs in, without additional configuration. For this reason, users can run multiple instances of Traefik at the same time to achieve HA, as is a common pattern in the kubernetes ecosystem.

    When using a single instance of Traefik with Let’s Encrypt, you should encounter no issues. However, this could be a single point of failure. Unfortunately, it is not possible to run multiple instances of Traefik Proxy 2.0 with Let’s Encrypt enabled, because there is no way to ensure that the correct instance of Traefik will receive the challenge request and subsequent responses. Previous versions of Traefik used a to attempt to achieve this, but due to sub-optimal performance that feature was dropped in 2.0.

    If you need Let’s Encrypt with HA in a Kubernetes environment, we recommend using Traefik Enterprise, which includes distributed Let’s Encrypt as a supported feature.

    If you want to keep using Traefik Proxy, high availability for Let’s Encrypt can be achieved by using a Certificate Controller such as . When using Cert-Manager to manage certificates, it creates secrets in your namespaces that can be referenced as TLS secrets in your ingress objects. When using the Traefik Kubernetes CRD Provider, unfortunately Cert-Manager cannot yet interface directly with the CRDs. A workaround is to enable the to allow Cert-Manager to create ingress objects to complete the challenges. Please note that this still requires manual intervention to create the certificates through Cert-Manager, but once the certificates are created, Cert-Manager keeps them renewed.

    Optional, Default=””

    The Kubernetes server endpoint URL.

    When deployed into Kubernetes, Traefik reads the environment variables KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT or KUBECONFIG to construct the endpoint.

    The access token is looked up in /var/run/secrets/kubernetes.io/serviceaccount/token and the SSL CA certificate in /var/run/secrets/kubernetes.io/serviceaccount/ca.crt. Both are mounted automatically when deployed inside Kubernetes.

    The endpoint may be specified to override the environment variable values inside a cluster.

    When the environment variables are not found, Traefik tries to connect to the Kubernetes API server with an external-cluster client. In this case, the endpoint is required. Specifically, it may be set to the URL used by kubectl proxy to connect to a Kubernetes cluster using the granted authentication and authorization of the associated kubeconfig.

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     endpoint: "http://localhost:8080"
    4.     # ...

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   endpoint = "http://localhost:8080"
    3.   # ...
    1. --providers.kubernetescrd.endpoint=http://localhost:8080

    token

    Optional, Default=””

    Bearer token used for the Kubernetes client configuration.

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     token: "mytoken"
    4.     # ...

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   token = "mytoken"

    CLI

    1. --providers.kubernetescrd.token=mytoken

    certAuthFilePath

    Optional, Default=””

    Path to the certificate authority file. Used for the Kubernetes client configuration.

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     certAuthFilePath: "/my/ca.crt"
    4.     # ...

    File (TOML)

    CLI

    1. --providers.kubernetescrd.certauthfilepath=/my/ca.crt

    Optional, Default: []

    Array of namespaces to watch. If left empty, watches all namespaces if the value of namespaces.

    File (YAML)

    1. providers:
    3.     namespaces:
    4.     - "default"
    5.     - "production"
    6.     # ...

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   namespaces = ["default", "production"]
    3.   # ...

    CLI

    1. --providers.kubernetescrd.namespaces=default,production

    labelselector

    Optional, Default: “”

    A label selector can be defined to filter on specific resource objects only, this applies only to Traefik and has no effect on Kubernetes Secrets, Endpoints and Services. If left empty, Traefik processes all resource objects in the configured namespaces.

    See label-selectors for details.

    Warning

    Because the label selector is applied to all Traefik Custom Resources, they all must match the filter.

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     labelSelector: "app=traefik"
    4.     # ...

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   labelSelector = "app=traefik"
    3.   # ...

    CLI

    1. --providers.kubernetescrd.labelselector="app=traefik"

    ingressClass

    Value of kubernetes.io/ingress.class annotation that identifies resource objects to be processed.

    If the parameter is set, only resources containing an annotation with the same value are processed. Otherwise, resources missing the annotation, having an empty value, or the value traefik are processed.

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     ingressClass: "traefik-internal"
    4.     # ...

    File (TOML)

    CLI

    1. --providers.kubernetescrd.ingressclass=traefik-internal

    Optional, Default: 0

    The throttleDuration option defines how often the provider is allowed to handle events from Kubernetes. This prevents a Kubernetes cluster that updates many times per second from continuously changing your Traefik configuration.

    If left empty, the provider does not apply any throttling and does not drop any Kubernetes events.

    The value of throttleDuration should be provided in seconds or as a valid duration format, see time.ParseDuration.

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     throttleDuration: "10s"
    4.     # ...

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   throttleDuration = "10s"
    3.   # ...

    CLI

    1. --providers.kubernetescrd.throttleDuration=10s

    allowCrossNamespace

    Optional, Default: false

    If the parameter is set to true, IngressRoutes are able to reference resources in other namespaces than theirs.

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     allowCrossNamespace: true
    4.     # ...

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   allowCrossNamespace = true
    3.   # ...

    CLI

    1. --providers.kubernetescrd.allowCrossNamespace=true

    allowExternalNameServices

    Optional, Default: false

    If the parameter is set to true, IngressRoutes are able to reference ExternalName services.

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     allowExternalNameServices: true
    4.     # ...

    File (TOML)

    CLI

    For additional information, refer to the with Let’s Encrypt.