Upgrading and Reinstalling

    Starting from Kubeflow v0.5, Kubeflow Pipelines persists thepipeline data in permanent storage volumes. Kubeflow Pipelines thereforesupports the following capabilities:

    • Reinstall: You can delete a cluster and create a new cluster, specifyingthe existing storage volumes to retrieve the original data in the new cluster.This guide tells you how to reinstall Kubeflow Pipelines as part of afull Kubeflow deployment.

    The full Kubeflow deployment currently supports upgrading in Alphastatus with limited support. Check the following sources for progressupdates:

    This guide tells you how to reinstall Kubeflow Pipelines as part of afull Kubeflow deployment on Google Kubernetes Engine (GKE). See theKubeflow deployment guide.

    Instead of the full Kubeflow deployment, you can use Kubeflow PipelinesStandalone or GCP Hosted ML Pipelines (Alpha), which support different optionsfor upgrading and reinstalling. See the .

    Kubeflow Pipelines creates and manages the following data related to yourmachine learning pipeline:

    • Metadata: Experiments, jobs, runs, etc. Kubeflow Pipelinesstores the pipeline metadata in a MySQL database.
    • Artifacts: Pipeline packages, metrics, views, etc. Kubeflow Pipelinesstores the artifacts in a Minio server.

    After , you can see two entries inthe GCP Deployment Manager,one for the cluster deployment and one for the storage deployment:

    The entry with the suffix creates one PD for the metadata store andone for the artifact store:

    Deployment Manager showing details of the storage deployment entry

    You can delete a Kubeflow cluster and create a new one, specifyingyour existing storage to retrieve the original data in the new cluster.

    Notes:

    • You must use command-line deployment.You cannot reinstall Kubeflow Pipelines using the web interface.

    To reinstall Kubeflow Pipelines:

    • Set a different name from your existing ${KF_NAME}.

    • Before running the kfctl apply command:

      • Edit and set the followingflag to skip creating new storage:
    • Edit ${KF_DIR}/kustomize/minio/overlays/minioPd/params.env and specifythe PD that your existing deployment uses for the Minio server:
    • Run the command to deploy Kubeflow as usual:

    You should now have a new Kubeflow deployment that uses the same pipelines datastorage as your previous deployment. Follow the steps in the deployment guideto .

    Was this page helpful?

    Glad to hear it! Please tell us how we can improve.

    Sorry to hear that. Please .

    Last modified 04.01.2020: Created GCP Pipelines section and moved GCP-specific pipelines upgrade instructions (#1494) (b78d8c21)