Backing up etcd

A Kubernetes cluster deployed with kops stores the etcd state in two differentAWS EBS volumes per master node. One volume is used to store the Kubernetesmain data, the other one for events. For a HA master with three nodes this willresult in six volumes for etcd data (one in each AZ). An EBS volume is designedto have a failure rateof 0.1%-0.2% per year.

Kubernetes does currently not provide any option to do regular backups of etcdout of the box.

Note: this is one of many examples on how to do scheduled snapshots.

In case the Kubernetes cluster fails in a way that too many master nodes can’taccess their etcd volumes it is impossible to get a etcd quorum.

Kubernetes uses protokube to identify the right volumes for etcd. Therefore itis important to tag the EBS volumes with the correct tags after restoring themfrom a EBS snapshot.

protokube will look for the following tags:

• containing the volume name (e.g. eu-central-1a.etcd-main.k8s.mycompany.tld)
• with the value 1