Install ETCD on Kubernetes with High Availability
In this multi-part series, we will cover how to install, maintain and monitor an Etcd cluster on Kubernetes with High Availability. We will be installing Etcd v3.5.4 on Amazon Elastic Kubernetes Service (EKS), but similar setup should work for other Kubernetes clusters as well.
Prerequisites
- Kubernetes 1.19+
- Helm 3.2.0+
- PV provisioner support in the underlying infrastructure
- Clone and create a feature branch in `etcd` repository. Update the etcd cluster using helm to v3.5.4 from here – https://github.com/bitnami/charts/tree/master/bitnami/etcd
-
Install Etcd on Kubernetes using Helm
We will be using Etcd helm package to install Etcd on Kubernetes. We can directly execute helm commands or for simplicity, we can create a Makefile which acts as a wrapper. First, create a Makefile as shown below.
NAMESPACE=coderiseio-etcd
#install
.PHONY: install-etcd
install-etcd: create-etcd-ns create-etcd-secret setup-etcd upgrade-etcd
.PHONY: create-etcd-ns
create-etcd-namespace:
kubectl create namespace $(NAMESPACE)
.PHONY: create-etcd-secret
install-etcd-secret:
@kubectl create secret generic etcd-root-password --from-literal=etcd-root-password=$(ETCD_ROOT_PASSWORD) -n $(NAMESPACE)
.PHONY: setup-etcd
setup-etcd:
helm install etcd charts/etcd/ -n $(NAMESPACE)
.PHONY: upgrade-etcd
upgrade-etcd:
helm upgrade etcd charts/etcd/ -n $(NAMESPACE)
.PHONY: init-etcd
init-etcd:
@kubectl exec etcd-0 -n $(NAMESPACE) -- bash -c "etcdctl role add root && etcdctl user add root:$(ETCD_ROOT_PASSWORD) && etcdctl user grant-role root root && etcdctl auth enable"
#uninstall
.PHONY: uninstall-etcd
uninstall-etcd:
@echo -n "Uninstall etcd and delete $(NAMESPACE) namespace. Are you sure? [y/N] " && read ans && [ ${ans:-N} = y ]
helm uninstall etcd -n $(NAMESPACE)
kubectl delete namespace $(NAMESPACE)
Installation
To install Etcd, run the following command:
make install-etcd ETCD_ROOT_PASSWORD=<password>
The command will create a namespace, secret, run helm to install etcd and finally upgrade etcd. An upgrade step is required to switch the initial cluster state attribute value from new to existing as described below.
In this paragraph, we will discuss about the initial cluster state attribute that plays a key role in Etcd installation. When Etcd cluster is provisioned, it sets the –initial-cluster-state attribute to new. This configuration works well for new clusters but pods won’t be able to join existing clusters upon restart. To fix this issue, we will need to update the flag from “new to existing” using the make install-etcd command.
The –initial-cluster-state attribute is defined in the statefulset i.e.
InitialClusterState Initial cluster state. Allowed values: ‘new’ or ‘existing’. If this values is not set, the default values below are set:
~ ‘new’: when installing the chart (‘helm install …’)
~ ‘existing’: when upgrading the chart (‘helm upgrade …’)
Configuration
To configure Etcd, run the following command:
make init-etcd ETCD_ROOT_PASSWORD=<password>
The command will add a role, user and assign role to that user. It will also enable authentication.
Uninstallation
To uninstall Etcd, run the following command:
make uninstall-etcd
The command will uninstall Etcd and delete the namespace.
Conclusion
In conclusion, we covered how to install Etcd on Kubernetes in high availability mode. Since, we have an Etcd cluster running now, in the next blog, we will cover how to assign Etcd pods to specific nodes using node affinities.
