Install Istio with an External Control Plane

    This feature is currently considered alpha.

    External control plane cluster and remote cluster

    Requirements

    This guide requires that you have two Kubernetes clusters with any of the supported Kubernetes versions: 1.17, 1.18, 1.19, 1.20.

    The first cluster contains the external control plane installed in the namespace. An ingress gateway is also installed in the istio-system namespace to provide mesh sidecars access to the external control plane.

    The second cluster is a remote cluster running the mesh workloads. Its Kubernetes API server also provides the configuration for the control plane (istiod) running in the external cluster.

    The Kubernetes API server in the remote cluster must be accessible to the external control plane cluster. Many cloud providers make API servers publicly accessible via network load balancers (NLBs). If the API server is not directly accessible, you will have to modify the installation procedure to enable access. For example, the gateway used in the multi-network and primary-remote configurations could also be used to enable access to the API server.

    The following environment variables will be used throughout to simplify the instructions:

    Set the CTX_EXTERNAL_CLUSTER, CTX_REMOTE_CLUSTER, and REMOTE_CLUSTER_NAME now. You will set the others later.

    Cluster configuration

    Create the Istio install configuration for the ingress gateway that exposes the external control plane ports to other clusters:

    1. $ cat <<EOF > controlplane-gateway.yaml
    2. apiVersion: install.istio.io/v1alpha1
    3. kind: IstioOperator
    4. metadata:
    5.   namespace: istio-system
    6. spec:
    7.   components:
    8.     ingressGateways:
    9.       - name: istio-ingressgateway
    10.         enabled: true
    11.         k8s:
    12.           service:
    13.             ports:
    14.               - port: 15021
    15.                 targetPort: 15021
    16.                 name: status-port
    17.               - port: 15012
    18.                 targetPort: 15012
    19.                 name: tls-xds
    20.               - port: 15017
    21.                 targetPort: 15017
    22.                 name: tls-webhook
    23. EOF
    1. $ istioctl install -f controlplane-gateway.yaml --context="${CTX_EXTERNAL_CLUSTER}"

    You may notice an istiod deployment created in the istio-system namespace. This is used only to configure the ingress gateway and is NOT the control plane used by remote clusters. This ingress gateway could, in fact, be configured to host multiple external control planes, in different namespaces on the cluster, even though in this example you will only deploy a single external istiod in the external-istiod namespace.

    Configure your environment to expose the Istio ingress gateway service using a public hostname with TLS. Set the EXTERNAL_ISTIOD_ADDR environment variable to the hostname and SSL_SECRET_NAME environment variable to the secret that holds the TLS certs:

    1. $ export EXTERNAL_ISTIOD_ADDR=<your external istiod host>
    2. $ export SSL_SECRET_NAME=<your external istiod secret>

    Create the Istio Gateway, VirtualService, and DestinationRule configuration for the yet to be installed external control plane:

    1. $ cat <<EOF > external-istiod-gw.yaml
    2. apiVersion: networking.istio.io/v1beta1
    3. kind: Gateway
    4. metadata:
    5.   name: external-istiod-gw
    6.   namespace: external-istiod
    7. spec:
    8.   selector:
    9.     istio: ingressgateway
    10.   servers:
    11.     - port:
    12.         number: 15012
    13.         protocol: https
    14.         name: https-XDS
    15.       tls:
    16.         mode: SIMPLE
    17.         credentialName: $SSL_SECRET_NAME
    18.       hosts:
    19.       - $EXTERNAL_ISTIOD_ADDR
    20.     - port:
    21.         number: 15017
    22.         protocol: https
    23.         name: https-WEBHOOK
    24.       tls:
    26.         credentialName: $SSL_SECRET_NAME
    27.       - $EXTERNAL_ISTIOD_ADDR
    28. ---
    29. apiVersion: networking.istio.io/v1beta1
    30. kind: VirtualService
    31. metadata:
    32.    name: external-istiod-vs
    33.    namespace: external-istiod
    34. spec:
    35.     hosts:
    36.     - $EXTERNAL_ISTIOD_ADDR
    37.     gateways:
    38.     - external-istiod-gw
    39.     http:
    40.     - match:
    41.       - port: 15012
    42.       route:
    43.       - destination:
    44.           host: istiod.external-istiod.svc.cluster.local
    45.           port:
    46.             number: 15012
    47.     - match:
    48.       - port: 15017
    49.       route:
    50.       - destination:
    51.           host: istiod.external-istiod.svc.cluster.local
    52.           port:
    53.             number: 443
    54. ---
    55. apiVersion: networking.istio.io/v1alpha3
    56. kind: DestinationRule
    57. metadata:
    58.   name: external-istiod-dr
    59.   namespace: external-istiod
    60. spec:
    61.   host: istiod.external-istiod.svc.cluster.local
    62.   trafficPolicy:
    63.     portLevelSettings:
    64.     - port:
    65.         number: 15012
    66.       tls:
    67.         mode: SIMPLE
    68.       connectionPool:
    69.         http:
    70.           h2UpgradePolicy: UPGRADE
    71.     - port:
    72.         number: 443
    73.       tls:
    74.         mode: SIMPLE
    75. EOF

    Create the external-istiod namespace and apply the configuration:

    Create the remote Istio install configuration:

    1. $ cat <<EOF > remote-config-cluster.yaml
    2. apiVersion: install.istio.io/v1alpha1
    3. kind: IstioOperator
    4. metadata:
    5.  namespace: external-istiod
    6. spec:
    7.   profile: remote
    8.   meshConfig:
    9.     rootNamespace: external-istiod
    10.     defaultConfig:
    11.       discoveryAddress: $EXTERNAL_ISTIOD_ADDR:15012
    12.       proxyMetadata:
    13.         XDS_ROOT_CA: /etc/ssl/certs/ca-certificates.crt
    14.         CA_ROOT_CA: /etc/ssl/certs/ca-certificates.crt
    16.       enabled: false
    17.     istiodRemote:
    18.       enabled: true
    20.   values:
    21.     global:
    22.       caAddress: $EXTERNAL_ISTIOD_ADDR:15012
    23.       istioNamespace: external-istiod
    24.       meshID: mesh1
    25.       multiCluster:
    26.         clusterName: $REMOTE_CLUSTER_NAME
    27.     istiodRemote:
    28.       injectionURL: https://$EXTERNAL_ISTIOD_ADDR:15017/inject
    29.     base:
    30.       validationURL: https://$EXTERNAL_ISTIOD_ADDR:15017/validate
    31. EOF

    Install the configuration on the remote cluster:

    1. $ kubectl create namespace external-istiod --context="${CTX_REMOTE_CLUSTER}"
    2. $ istioctl manifest generate -f remote-config-cluster.yaml  | kubectl apply --context="${CTX_REMOTE_CLUSTER}" -f -

    NOTE: An ingress gateway, for accessing services in the remote cluster mesh, is included in the above installation. However it will not start working until you install the external control plane in the next section.

    The control plane in the external cluster needs access to the remote cluster to discover services, endpoints, and pod attributes. Create a secret with credentials to access the remote cluster’s kube-apiserver and install it in the external cluster.

    1. $ kubectl create sa istiod-service-account -n external-istiod --context="${CTX_EXTERNAL_CLUSTER}"
    2. $ istioctl x create-remote-secret \
    3.   --context="${CTX_REMOTE_CLUSTER}" \
    4.   --type=config \
    5.   --namespace=external-istiod | \
    6.   kubectl apply -f - --context="${CTX_EXTERNAL_CLUSTER}"

    Create the Istio install configuration to create the control plane in the external-istiod namespace of the external cluster:

    1. $ cat <<EOF > external-istiod.yaml
    2. apiVersion: install.istio.io/v1alpha1
    3. kind: IstioOperator
    4. metadata:
    5.   namespace: external-istiod
    6. spec:
    7.   meshConfig:
    8.     rootNamespace: external-istiod
    9.     defaultConfig:
    10.       discoveryAddress: $EXTERNAL_ISTIOD_ADDR:15012
    11.       proxyMetadata:
    12.         XDS_ROOT_CA: /etc/ssl/certs/ca-certificates.crt
    13.         CA_ROOT_CA: /etc/ssl/certs/ca-certificates.crt
    14.   components:
    15.     base:
    16.       enabled: false
    17.     ingressGateways:
    18.     - name: istio-ingressgateway
    19.       enabled: false
    20.   values:
    21.     global:
    22.       caAddress: $EXTERNAL_ISTIOD_ADDR:15012
    23.       istioNamespace: external-istiod
    24.       operatorManageWebhooks: true
    25.       meshID: mesh1
    26.       multiCluster:
    27.         clusterName: $REMOTE_CLUSTER_NAME
    28.     pilot:
    29.       env:
    30.         INJECTION_WEBHOOK_CONFIG_NAME: ""
    31.         VALIDATION_WEBHOOK_CONFIG_NAME: ""
    32. EOF

    Apply the Istio configuration on the external cluster:

    1. $ kubectl get pod -l app=istio-ingressgateway -n external-istiod --context="${CTX_REMOTE_CLUSTER}"

    Deploy the helloworld sample to the remote cluster. Wait a few seconds for the helloworld pods to be running with sidecars injected.

    1. $ kubectl label namespace default istio-injection=enabled --context="${CTX_REMOTE_CLUSTER}"
    2. $ kubectl apply -f samples/helloworld/helloworld.yaml --context="${CTX_REMOTE_CLUSTER}"
    3. $ kubectl get pod -l app=helloworld --context="${CTX_REMOTE_CLUSTER}"

    Expose the helloworld application on the ingress gateway:

    1. $ kubectl apply -f samples/helloworld/helloworld-gateway.yaml --context="${CTX_REMOTE_CLUSTER}"

    Follow to set GATEWAY_URL and then confirm you can access the application:

    1. $ curl -s "http://${GATEWAY_URL}/hello" | grep -o "Hello"

    Congratulations! You successfully installed an external control plane and used it to manage services running in a remote cluster!

    See also

    Deploy multiple Istio egress gateways independently to have fine-grained control of egress communication from the mesh.

    Deploying Istio Control Planes Outside the Mesh

    A new deployment model for Istio.

    Istiod consolidates the Istio control plane components into a single binary.

    Describes the telemetry and monitoring features provided by Istio.