This article is about Federation v2 in Kubernetes, how it works currently with some use case description. At first, you need to know that federation v1 is deprecated, taking almost all Kubernetes APIs back. But a new version v2 is currently distributed, in alpha version.
Targeted audience: Kubernetes administrator
By Maud Laurent, System administrator @Objectif Libre
Federation v2 in Kubernetes
K8s Federation: What is it?
Federation allow you to join different Kubernetes clusters and to control them with a single point of entry.
- Federation ensures the same deployment exists in multiple specified clusters
- It enables high-availability if data are federated and replicated.
- It offers hybrid cloud possibility: clusters can come from multiple providers; and/or be multi-region: clusters are hosted in distant sites.
- Federation provides an auto-configured DNS across the cluster (with coreDNS or an external DNS).
However, running services needs to have a
LoadBalancer type to be discovered across clusters.
Warning: the federation integration reduces cluster isolation. It increases the network bandwidth cost, depending on the clusters placement on the same cloud provider, the same region or not.
Federation v2 deployment
A user guide is available on Github and explains all installation process: https://github.com/kubernetes-sigs/kubefed/blob/master/docs/userguide.md.
Federation deployment is made with Helm chart on one cluster that is used as a host (the current kubectl context is used by default).
The host cluster runs a deployment
federation-controller-manager in a
federation-system namespace. Its uses by default a namespace
kube-multicluster-public to save clusters on a ‘cluster-registry’.
When Federation is deployed, all the management is done with
Kubernetes resources are added, in federated resources, with
kubefed2 enable and disabled with
kubefed2 disable. Federated resources are written in the same things that basic resources, only
kind field change to be
Resources are added in the API group
types.federation.k8s.io. These specific created resources are visible with
kubectl api-resources | grep federation command.
Adding or deleting clusters in federation is easily performed with
kubefed2 and keywords
unjoin. For example, the addition of a cluster to the federation would be done as follows :
kubefed2 join clusterName --cluster-context clusterContext \ --host-cluster-context clusterHost --add-to-registry --v=2
Contexts are based on local kubeconfig file (~/.kube/config).
You have to be careful of the name given. It must not contain ‘@’ symbol to not interfere with DNS domain and sub-domain format. The list of all federated clusters are available with command
kubectl get clusters.clusterregistry.k8s.io -n kube-multicluster-public.
All federated deployment need to have a federated namespace of type
federatednamespaces and a k8s basic namespace on the host cluster. These two namespaces should have the same name.
At the end of each federated resource, you can add a
placement dictionary is used to specify cluster name where the resource is deployed. Clusters can be filtered with labels.
overrides dictionary is optional and offers the possibility to overwrite some values like replicas number.
placement: clusterNames: - cluster2-fede - cluster1-fede # you can also filter cluster by labels overrides: # optional, you can override value by cluster - clusterName: cluster2-fede clusterOverrides: - path: spec.replicas value: 5
All federated resources created with yaml files, are applied from the host cluster and are automatically dispatched according to the
placement section. Without placement block, all federated clusters are used.
It is possible to convert host cluster resource in a federated resource with
kubefed2 federate resourceType resourceName -n namespace if the namespace is also federated.
Federation can be used to instantiate replication mechanism. Regarding the infrastructure, this is more or less necessary. For example, an infrastructure with 3 clusters: one dev, one staging, and one prod will not need to deploy an app on several clusters at the same time. Even though, in the case of two or more clusters at the same level (dev, staging or prod) it becomes interesting to deploy simultaneously and to manage common resources inside these two clusters.
Federation can also be used for applications migration (for example to migrate from private cloud to public cloud).
A federated cluster can be removed from federation whenever you want and keep all applications and resources deployed on it.
What about storage?
On top of that, there is a storage issue regarding the applications deployed. If they use a connection from an external database, the different clusters will be able to join the database; in this case, there isn’t any issue.
But if the application uses volumes, the data replication between clusters can be done either using federated persistent volume and persistent volume claim or using an appropriate back-end managing this replication.
Federation is a powerful tool which answers a special requirement that will satisfy important multi-cluster infrastructures.
Federation v2 is currently in alpha state and should be in beta with Kubernetes version 1.16 or 1.17. We will surely follow the project’s evolution.