Kubernetes offers many features dedicated to scalability to simplify infrastructure management for companies of any size. Those features present the expected, straightforward behaviors when used in a single-cloud Kubernetes cluster, but one question remains: how will they react when a single Kubernetes cluster regroups servers from multiple cloud providers?

Working in a multi-cloud environment presents multiple perks in terms of redundancy, reliability, and customer coverage. However, it also raises questions that we will address here about the particularity of the management and the implementation of Multi-Cloud.

How are Horizontal Node Auto-Scaling (HNA) and Auto-Healing managed?

As of today (october 2021), HNA and Auto-Healing are not covered by Kubernetes Kosmos for external servers. However, they are available for managed Scaleway node pools in any Availability Zone, regardless of the region of the Kubernetes Kosmos Control-Plane.

The reasons behind this choice are highlighted in our previous articles, which you can find below:

How can traffic be load-balanced between multiple Cloud providers?

Scaleway's Kubernetes Kosmos is fully integrated within the Scaleway ecosystem, which means that it benefits from the conversion of a Kubernetes service of type LoadBalancer into a Scaleway Multi-Cloud Load Balancer.

How can persistent volumes be managed in a Multi-Cloud Kubernetes cluster?

Persistent Volumes are created through the Container Storage Interface (CSI) of each cloud provider. In a Kubernetes Kosmos cluster, Scaleway's CSI is only deployed on the cluster's managed Scaleway Instances. Nonetheless, almost every Cloud provider's CSI is open-sourced and can be deployed on the corresponding nodes to benefit from Persistent Volumes from any cloud provider, within the same cluster.

How is communication managed between the nodes of a Kubernetes Kosmos cluster?

By design, it is not possible to have a private network regrouping servers from different cloud providers, but it is possible to have them communicate with each other.

Kubernetes Kosmos uses Kilo, a Container Network Interface (CNI) based on Wireguard, managing a Virtual Private Network (VPN).

Can the CNI be changed in a Kubernetes Kosmos cluster?

Kilo - the CNI used by Kubernetes Kosmos - is the solution allowing a multi-cloud Kubernetes. Indeed, if we were to change the CNI, every unmanaged node would become unreachable from the API-Server of the cluster Control-Plane. Furthermore, the cluster would be highly impacted and mostly unavailable.

How can application workloads be distributed between Cloud providers across a Kubernetes Kosmos cluster?

Kubernetes has configuration options designed specifically for this kind of use case, such as selectors, taints, and tolerations.

How does a Multi-Cloud environment impact the services availability and latency?

Choosing between High Availability and Low Latency needs to be taken into account, whether we are working in a Multi-Cloud environment or not. On a Kubernetes Kosmos cluster, pods will be scheduled by default on the nodes with the least latency from the cluster's Control-Plane.

Does Multi-Cloud make infrastructure management more complicated?

Of course, managing multiple cloud providers’ accounts and credentials is more restrictive than keeping a single provider. However, studies have shown that over 80% of IT companies already use multiple cloud providers. That way, they can benefit from a larger range of services.