Introduction

Kubernetes is designed to be cloud-agnostic, yet it needs to interact with cloud providers to manage compute, networking, and storage resources. This is where Cloud Controllers come in.

A Cloud Controller is a Kubernetes component that acts as a bridge between Kubernetes and cloud provider APIs. It automates cloud resource management, ensuring that Kubernetes workloads can seamlessly integrate with external cloud services.

In this post, we’ll explore:

• What Cloud Controllers are and why they are needed
• How Cloud Controllers work in Kubernetes
• The shift from in-tree to out-of-tree cloud providers
• Best practices for managing Cloud Controllers effectively

What is a Cloud Controller?

A Cloud Controller in Kubernetes is a component responsible for integrating cloud provider resources (such as networking, storage, and compute) with a Kubernetes cluster. It ensures that Kubernetes can provision and manage cloud resources automatically.

Why Cloud Controllers are Needed

Without Cloud Controllers, Kubernetes would not be able to provision, manage, or update cloud-based resources dynamically. Instead, administrators would have to manually configure resources like:

• Load Balancers for external traffic routing
• Persistent Storage for databases and stateful applications
• Network Routes and Firewall Rules for Pod-to-Pod communication across nodes

Cloud Controllers automate these processes, enabling seamless cloud integration and efficient scaling.

How Cloud Controllers Work

Cloud Controllers operate by interacting with the cloud provider’s API to provision and manage resources. They continuously monitor Kubernetes resources and reconcile the desired state with the actual cloud environment.

Cloud Controller Responsibilities:

1. Node Management

   • Ensures that Kubernetes Nodes (VMs/instances) match their cloud provider counterparts.
   • Removes Nodes from the cluster when instances are deleted in the cloud.

2. Load Balancer Provisioning

   • Creates cloud-based load balancers for Service type=LoadBalancer.
   • Updates DNS records or firewall rules as needed.

3. Persistent Storage Management

   • Handles cloud-based persistent volumes (e.g., AWS EBS, Azure Disks, GCP PD).
   • Ensures correct attachment/detachment of volumes to nodes.

4. Networking and Routes

   • Configures cloud-based network routes and firewall rules.
   • Ensures cross-node communication functions correctly.

The Cloud Controller Manager (CCM)

The Cloud Controller Manager (CCM) is the Kubernetes component that runs cloud-specific controllers separately from the Kubernetes core. It allows each cloud provider to develop and maintain its own controllers independently.

The Shift from In-Tree to Out-of-Tree Cloud Providers

Originally, Kubernetes embedded cloud provider code directly into its core—a model known as in-tree cloud providers. While convenient at first, this approach introduced security, maintenance, and scalability issues.

To solve these challenges, Kubernetes introduced the out-of-tree model, where Cloud Controllers run separately from the Kubernetes core via the Cloud Controller Manager.

Why In-Tree Cloud Providers Were Deprecated

1. Security Risks – Cloud provider credentials were embedded in Kubernetes core.
2. Slower Updates – Cloud provider updates required Kubernetes releases.
3. Scalability Challenges – Managing multiple cloud providers in Kubernetes core became unsustainable.

Out-of-Tree Cloud Providers: The Solution

With out-of-tree cloud providers:

• The CCM runs as a Deployment inside the cluster.
• Cloud providers maintain their own controllers independently.
• Kubernetes no longer ships with built-in cloud integrations.

This modular architecture improves security, flexibility, and ease of maintenance.

Migration from In-Tree to Out-of-Tree Cloud Providers

If your cluster is still using an in-tree cloud provider, you must migrate to an external Cloud Controller Manager.

1. Check If Your Cluster Uses an In-Tree Provider

Run:

kubectl get nodes -o wide

If your EXTERNAL-IP is automatically assigned by your cloud provider, you are likely using an in-tree provider.

2. Disable the In-Tree Provider

Modify your Kubernetes components:

• Add --cloud-provider=external to kubelet and kube-controller-manager configurations.

3. Deploy the Out-of-Tree Cloud Controller

Each cloud provider offers its own CCM:

• AWS CCM: aws-cloud-controller-manager
• GCP CCM: gcp-cloud-controller-manager
• Azure CCM: azure-cloud-controller-manager
• OpenStack CCM: openstack-cloud-controller-manager
• vSphere CCM: vsphere-cloud-controller-manager

Example (AWS CCM Deployment with Helm):

helm repo add aws-cloud-controller https://kubernetes.github.io/cloud-provider-aws
helm install aws-ccm aws-cloud-controller/aws-cloud-controller-manager -n kube-system

4. Validate Migration

Check CCM logs to confirm successful integration:

kubectl logs -n kube-system deployment/cloud-controller-manager

Once validated, remove all references to the old in-tree provider.

Troubleshooting Cloud Controller Issues

Best Practices for Managing Cloud Controllers

1. Always use the latest CCM version

   • Cloud providers frequently update their CCM to add new features and security patches.

2. Monitor CCM logs and metrics

   • Use Prometheus, Grafana, and Kubernetes Events to track CCM performance.

3. Follow cloud provider-specific documentation

   • Each provider has unique requirements and configurations.

4. Test migrations in a non-production environment first

   • Ensure a smooth transition by validating CCM deployment before migrating production workloads.

Conclusion

Cloud Controllers are critical components in Kubernetes, enabling seamless cloud integration while keeping Kubernetes modular, flexible, and scalable.

The transition from in-tree to out-of-tree Cloud Controllers improves security, update cycles, and cloud provider flexibility. If your cluster still relies on in-tree cloud providers, migrating to an external CCM is essential for long-term compatibility and stability.

Got questions about Kubernetes Cloud Controllers? Drop them in the comments or explore more Kubernetes Deep Dive posts at support.tools!

Want to learn more about Kubernetes? Browse the Kubernetes Deep Dive series for more expert insights!