Configuring Kubeflow with kfctl and kustomize

    Kfctl is the Kubeflow command-line interface (CLI) that you can use toinstall and configure Kubeflow.

    Kubeflow makes use of kustomize to help customize YAMLconfigurations. With kustomize, you can traverse a Kubernetes manifest to add,remove, or update configuration options without forking the manifest. Amanifest is a YAML file containing a description of the applications that youwant to include in your Kubeflow deployment.

    This section describes how kfctl works with kustomize to set up yourKubeflow deployment.

    Kfctl is the Kubeflow CLI that you can use to set up a Kubernetes cluster withKubeflow installed, or to deploy Kubeflow to an existing Kubernetes cluster.See the forinstallation instructions based on your deployment scenario.

    The kfctl deployment process includes the following commands:

    • - (Optional) Creates configuration files defining the variousresources in your deployment but does not deploy Kubeflow.You only need to run kfctl build if you wantto edit the resources before running kfctl apply.
    • kfctl apply - Creates or updates the resources.
    • kfctl delete - Deletes the resources.

    Specifying a configuration file when initializing your deployment

    When you install Kubeflow, the deployment process uses one of a few possibleYAML configuration files to bootstrap the configuration. You can see all the.

    As an example, this guide uses thekfctl_k8s_istio.yamlconfiguration. For more details about this configuration, see the.

    Typically, you specify the configuration file with a -f <config-file>parameter when you run kfctl build or kfctl apply. The following exampleuses kfctl build:

    Kfctl has now built the configuration files in your Kubeflow applicationdirectory (see below) but has not yet deployed Kubeflow.To complete the deployment, run kfctl apply. See the next section on.

    When you first run kfctl build or kfctl apply, kfctl createsa local version of the YAML configuration file,which you can further customize if necessary.

    Follow these steps to apply the configurations to your Kubeflow cluster:

    • Set an environment variable pointing to your local configuration file.For example, this guide uses the kfctl_k8s_istio.0.7.1.yamlconfiguration. If you chose a different configuration in the previous step,you must change the file name to reflect your configuration:
    1. export CONFIG_FILE=${KF_DIR}/kfctl_k8s_istio.0.7.1.yaml
    • Apply the configurations:
    1. kfctl apply -V -f ${CONFIG_FILE}

    Your Kubeflow directory layout

    Your Kubeflow application directory is the directory where you choose to storeyour Kubeflow configurations during deployment. This guide refers to thedirectory as ${KF_DIR}. The directory contains the following files anddirectories:

    • ${CONFIG_FILE} is a YAML file that stores your primary Kubeflowconfiguration in the form of a KfDef Kubernetes object.

      • This file is a copy of the GitHub-based configuration YAMLfile thatyou used when deploying Kubeflow.
      • When you first run kfctl build or kfctl apply, kfctl createsa local version of the configuration file at ${CONFIG_FILE},which you can further customize if necessary.
      • The YAML defines each Kubeflow application as a kustomize package.

    • kustomize is a directory that contains Kubeflow application manifests.That is, the directory contains the kustomize packages for the Kubeflowapplications that are included in your deployment.

      • The directory is created when you run kfctl build or kfctl apply.
      • To customize these configurations, you can modify parametersin your ${CONFIG_FILE}, and then run kfctl apply to applythe configuration to your Kubeflow cluster.

    The content of your ${CONFIG_FILE} is the result of running kustomizeon the base and overlay kustomization.yaml files in the.The overlays reflect the configuration file that you specify when runningkfctl build or kfctl apply.

    Below are some examples of configuration files:

    • kfctl_k8s_istio.yamlto install Kubeflow on an existing Kubernetes cluster.
    • to install Kubeflow on an existing Kubernetes cluster with Dex and Istio forauthentication.
    • kfctl_gcp_iap.yamlto create a Google Kubernetes Engine (GKE) cluster with Kubeflow usingCloud Identity-Aware Proxy (Cloud IAP) for access control.

    The kustomize package manager in kfctl uses the information in your${CONFIG_FILE} to traverse the directories under the and tocreate kustomize build targets based on the manifests.

    Make sure that you have the minimum required version of kustomize:2.0.3 or later.

    • Follow the kustomize installationguide,choosing the relevant options for your operating system. For example, ifyou’re on Linux:

      • Set some variables for the operating system:
    • Download the kustomize binary:
    1. curl -s https://api.github.com/repos/kubernetes-sigs/kustomize/releases |\
    2. grep browser_download |\
    3. grep download/kustomize |\
    4. grep -m 1 $opsys |\
    5. cut -d '"' -f 4 |\
    6. xargs curl -O -L
    • Move the binary:
    1. mkdir -p ${HOME}/bin
    2. mv kustomize_*_${opsys}_amd64 ${HOME}/bin/kustomize
    3. chmod u+x ${HOME}/bin/kustomize
    • Include the command in your path:

    Kustomize lets you customize raw, template-free YAML files for multiplepurposes, leaving the original YAML untouched and usable as is.

    You can use the following command to build and apply kustomize directories:

      The containskustomize build targets, each with a base directory. You can use kustomize togenerate YAML output and pass it to kfctl. You can also makechanges to the kustomize targets in the manifests repo as needed.

      For example, to modify settings for the Spartakus usage reporting tool withinyour Kubeflow deployment:

      • Edit the configuration file at ${CONFIG_FILE}.

      • Find and replace the parameter values for spartakus to suit yourrequirements:

      1. - kustomizeConfig:
      2.     parameters:
      3.     - initRequired: true
      4.         name: usageId
      5.         value: <randomly-generated-id>
      6.     - initRequired: true
      7.         name: reportUsage
      8.         value: "true"
      9.     repoRef:
      10.         name: manifests
      11.         path: common/spartakus
      12.     name: spartakus
      • Regenerate and deploy your Kubeflow resources:

      More examples

      For examples of customizing your deployment, see the guide to .

      For information about how Kubeflow uses Spartakus, see the guide tousage reporting.

      Below are some useful kustomize terms, from the:

      • base: A combination of a kustomization and resource(s). Bases can bereferred to by other kustomizations.

      • kustomization: Refers to a kustomization.yaml file, or more generally toa directory containing the kustomization.yaml file and all the relative filepaths that the YAML file references.

      • overlay: A combination of a kustomization that refers to a base, and apatch. An overlay may have multiple bases.

      • patch: General instructions to modify a resource.

      • resource: Any valid YAML file that defines an object with a kind and ametadata/name field.

      • target: The argument to kustomize build. For example,. A target must be a path or a URL to akustomization. A target can be a base or an overlay.

      Read more about kustomize in thekustomize documentation.