Traefik & Kubernetes

    Traefik used to support Kubernetes only through the Kubernetes Ingress provider, which is a Kubernetes Ingress controller in the strict sense of the term.

    However, as the community expressed the need to benefit from Traefik features without resorting to (lots of) annotations, we ended up writing a (alias CRD in the following) for an IngressRoute type, defined below, in order to provide a better way to configure access to a Kubernetes cluster.

    All Steps for a Successful Deployment

    • Add/update all the Traefik resources definitions
    • Add/update the for the Traefik custom resources
    • Use Helm Chart or use a custom Traefik Deployment
      • Enable the kubernetesCRD provider
      • Apply the needed kubernetesCRD provider
    • Add all needed traefik custom resources

    Initializing Resource Definition and RBAC

    Traefik Resource Definition

    RBAC for Traefik CRD

    1. kind: ClusterRole
    2. apiVersion: rbac.authorization.k8s.io/v1beta1
    3. metadata:
    4.   name: traefik-ingress-controller
    6. rules:
    7.   - apiGroups:
    8.       - ""
    9.     resources:
    10.       - services
    11.       - endpoints
    12.       - secrets
    13.     verbs:
    14.       - get
    15.       - list
    16.       - watch
    17.   - apiGroups:
    18.       - extensions
    19.       - networking.k8s.io
    20.     resources:
    21.       - ingresses
    22.       - ingressclasses
    23.     verbs:
    24.       - get
    25.       - list
    26.       - watch
    27.   - apiGroups:
    28.       - extensions
    29.       - ingresses/status
    30.     verbs:
    31.       - update
    32.   - apiGroups:
    33.       - traefik.containo.us
    34.     resources:
    35.       - middlewares
    36.       - ingressroutes
    37.       - traefikservices
    38.       - ingressroutetcps
    39.       - ingressrouteudps
    40.       - tlsoptions
    41.       - tlsstores
    43.     verbs:
    44.       - get
    45.       - list
    46.       - watch
    48. ---
    49. kind: ClusterRoleBinding
    50. apiVersion: rbac.authorization.k8s.io/v1beta1
    51. metadata:
    52.   name: traefik-ingress-controller
    54. roleRef:
    55.   apiGroup: rbac.authorization.k8s.io
    56.   kind: ClusterRole
    57.   name: traefik-ingress-controller
    58. subjects:
    59.   - kind: ServiceAccount
    60.     name: traefik-ingress-controller
    61.     namespace: default

    When using KubernetesCRD as a provider, Traefik uses to retrieve its routing configuration. Traefik Custom Resource Definitions are a Kubernetes implementation of the Traefik concepts. The main particularities are:

    • The usage of name and namespace to refer to another Kubernetes resource.
    • The usage of secret for sensible data like:
      • TLS certificate.
      • Authentication data.
    • The structure of the configuration.
    • The obligation to declare all the .

    The Traefik CRD are building blocks which you can assemble according to your needs. See the list of CRDs in the dedicated routing section.

    By design, Traefik is a stateless application, meaning that it only derives its configuration from the environment it runs in, without additional configuration. For this reason, users can run multiple instances of Traefik at the same time to achieve HA, as is a common pattern in the kubernetes ecosystem.

    When using a single instance of Traefik with LetsEncrypt, no issues should be encountered, however this could be a single point of failure. Unfortunately, it is not possible to run multiple instances of Traefik 2.0 with LetsEncrypt enabled, because there is no way to ensure that the correct instance of Traefik will receive the challenge request, and subsequent responses. Previous versions of Traefik used a to attempt to achieve this, but due to sub-optimal performance was dropped as a feature in 2.0.

    If you require LetsEncrypt with HA in a kubernetes environment, we recommend using Traefik Enterprise where distributed LetsEncrypt is a supported feature.

    If you want to continue to run Traefik Community Edition, LetsEncrypt HA can be achieved by using a Certificate Controller such as . When using Cert-Manager to manage certificates, it will create secrets in your namespaces that can be referenced as TLS secrets in your ingress objects. When using the Traefik Kubernetes CRD Provider, unfortunately Cert-Manager cannot interface directly with the CRDs yet, but this is being worked on by our team. A workaround is to enable the to allow Cert-Manager to create ingress objects to complete the challenges. Please note that this still requires manual intervention to create the certificates through Cert-Manager, but once created, Cert-Manager will keep the certificate renewed.

    Optional, Default=empty

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   endpoint = "http://localhost:8080"
    3.   # ...

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     endpoint: "http://localhost:8080"
    4.     # ...

    CLI

    1. --providers.kubernetescrd.endpoint=http://localhost:8080

    The Kubernetes server endpoint as URL.

    When deployed into Kubernetes, Traefik will read the environment variables KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT or KUBECONFIG to construct the endpoint.

    The access token will be looked up in /var/run/secrets/kubernetes.io/serviceaccount/token and the SSL CA certificate in /var/run/secrets/kubernetes.io/serviceaccount/ca.crt. Both are provided mounted automatically when deployed inside Kubernetes.

    The endpoint may be specified to override the environment variable values inside a cluster.

    token

    Optional, Default=empty

    File (TOML)

    1.   token = "mytoken"
    2.   # ...

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     token: "mytoken"
    4.     # ...

    CLI

    1. --providers.kubernetescrd.token=mytoken

    Bearer token used for the Kubernetes client configuration.

    Optional, Default=empty

    File (TOML)

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     certAuthFilePath: "/my/ca.crt"

    CLI

    1. --providers.kubernetescrd.certauthfilepath=/my/ca.crt

    Path to the certificate authority file. Used for the Kubernetes client configuration.

    namespaces

    Optional, Default: all namespaces (empty array)

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   namespaces = ["default", "production"]
    3.   # ...

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     namespaces:
    4.     - "default"
    5.     - "production"
    6.     # ...

    CLI

    1. --providers.kubernetescrd.namespaces=default,production

    Array of namespaces to watch.

    Optional,Default: empty (process all resources)

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   labelselector = "app=traefik"
    3.   # ...

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     labelselector: "app=traefik"
    4.     # ...

    CLI

    By default, Traefik processes all resource objects in the configured namespaces. A label selector can be defined to filter on specific resource objects only, this will apply only on Traefik Custom Resources and has no effect on Kubernetes Secrets, Endpoints and Services.

    Warning

    As the LabelSelector is applied to all Traefik Custom Resources, they all must match the filter.

    ingressClass

    Optional, Default: empty

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   ingressClass = "traefik-internal"
    3.   # ...

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     ingressClass: "traefik-internal"
    4.     # ...

    CLI

    1. --providers.kubernetescrd.ingressclass=traefik-internal

    Value of kubernetes.io/ingress.class annotation that identifies resource objects to be processed.

    If the parameter is non-empty, only resources containing an annotation with the same value are processed. Otherwise, resources missing the annotation, having an empty value, or the value traefik are processed.

    Optional, Default: 0 (no throttling)

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   throttleDuration = "10s"
    3.   # ...

    File (YAML)

    1. providers:
    2.   kubernetesCRD:
    3.     throttleDuration: "10s"
    4.     # ...

    CLI

    1. --providers.kubernetescrd.throttleDuration=10s

    allowCrossNamespace

    Optional, Default: true

    File (TOML)

    1. [providers.kubernetesCRD]
    2.   allowCrossNamespace = false

    File (YAML)

    CLI

    1. --providers.kubernetescrd.allowCrossNamespace=false

    If the parameter is set to false, an IngressRoute will not be able to reference any resources in another namespace than the IngressRoute namespace.

    Deprecation

    Please notice that the default value for this option will be set to in a future version.

    Also see the with Let’s Encrypt.