Cilium on Support Tools

Cilium MTU Black Hole: How a Linux Bridge and KubeVirt VMs Silently Broke VXLAN and Kept Longhorn Degraded

Sun, 25 Oct 2026 09:00:00 -0500

For about ten hours, roughly 45 Longhorn volumes on one of our production RKE2 clusters sat in a degraded state. Replicas flapped to ERR, rebuilds started and failed, and the Longhorn event stream logged more than 1,400 replica errors, nearly 800 of them in a single hour on one node. Small API calls worked. DNS worked. Health checks were green. Only bulk traffic between certain pairs of nodes was disappearing, and which pairs changed over the course of the day.

The cause turned out to be three ordinary behaviors stacked on top of each other. A Linux bridge takes the MTU of its smallest port. KubeVirt VMs attach 1500-byte veths to that bridge. And Cilium, with no MTU configured, does not just read the device MTU at startup: it follows it while running. Pods keep the MTU they were created with. So every time a VM moved onto or off a node, that node’s tunnel MTU changed underneath pods that still believed the old value, and their large VXLAN packets went into a black hole with no error anywhere.

This post walks through the mechanism, a reproducible demonstration of the bridge behavior, the commands that prove it on a live cluster, the fix (pin the MTU, flatten the underlay), and the planned full-cluster outage it took to apply that fix safely.

Kubernetes Network Policies for Zero-Trust Architecture: Enterprise Implementation Guide

Thu, 03 Sep 2026 00:00:00 -0500

Zero-trust network architecture represents a fundamental shift in cloud-native security strategy, requiring explicit authorization for every network connection rather than implicit trust based on network location. Kubernetes Network Policies provide the foundational building blocks for implementing zero-trust networking at the pod level. This comprehensive guide explores enterprise-grade network policy implementations using native Kubernetes Network Policies and advanced CNI-specific features from Calico and Cilium.

Cilium eBPF Networking Deep Dive: Advanced Kubernetes CNI Implementation

Tue, 12 May 2026 00:00:00 -0500

Cilium represents a paradigm shift in Kubernetes networking by leveraging eBPF (extended Berkeley Packet Filter) technology to provide high-performance, secure, and observable networking capabilities. This comprehensive guide explores advanced Cilium implementation patterns, performance optimization techniques, and enterprise-grade configurations for production Kubernetes clusters.

Cilium BGP Control Plane for Kubernetes: Complete Enterprise Implementation Guide

Tue, 10 Feb 2026 09:00:00 -0500

Bare-metal and on-premises Kubernetes clusters face a fundamental networking challenge that cloud providers solve transparently: making Services and Pods reachable from outside the cluster. In cloud environments, a LoadBalancer Service automatically provisions an external IP and programs the cloud provider’s network fabric. On bare metal, that integration does not exist. Border Gateway Protocol (BGP) fills this gap by enabling Kubernetes nodes to advertise routes for Pod CIDRs, LoadBalancer IPs, and custom prefixes directly to the upstream network infrastructure.

Cilium’s BGP Control Plane integrates this capability directly into the CNI layer. Built on the GoBGP routing library, the BGP Control Plane runs a BGP speaker on each selected node, advertising routes to configured peers without requiring an external component like MetalLB. Combined with Cilium’s LB-IPAM (LoadBalancer IP Address Management), this provides a complete, single-stack solution for bare-metal service exposure — from IP allocation through route advertisement to eBPF-accelerated packet forwarding.

Running K3s in an IPv6-Only Environment with Cilium

Sun, 01 Jun 2025 00:00:00 -0500

Most guides for Kubernetes focus on IPv4 by default, but IPv6 is no longer a second-class citizen. In this post, I walk through configuring a K3s cluster using only IPv6, with Cilium as the CNI, and BGP to announce addresses to my router. This setup runs entirely on a public IPv6 prefix—no dual-stack, no IPv4 fallback.

RKE2 the Hard Way: Part 8 – Installing Cilium CNI

Tue, 01 Apr 2025 00:00:00 +0000

Part 8 – Installing Cilium CNI

In this part of the “RKE2 the Hard Way” training series, we will install and configure the Cilium CNI (Container Network Interface). Cilium is a powerful, open-source CNI solution that provides advanced networking, security, and observability features for Kubernetes clusters.

At this point in our tutorial series, we have:

Set up all the core Kubernetes components (etcd, kube-apiserver, kube-controller-manager, kube-scheduler)
Configured kubelet and kube-proxy on our nodes

However, our cluster network is not yet functional for pod-to-pod communication. The CNI plugin is essential for implementing the Kubernetes networking model, which requires that:

Cilium Troubleshooting Guide: Commands That Work in 2026

Fri, 14 Mar 2025 14:00:00 -0500

Troubleshooting Cilium is mostly about running the right command in the right place. This guide is a working reference for current Cilium releases (1.16 through 1.20, with 1.20.3 the current stable at the time of this update): the cilium CLI on your workstation, cilium-dbg inside the agent, and Hubble for flows. It builds on our earlier Cilium Troubleshooting Guide.

Fixing Argo CD Out-of-Sync Due to Cilium Identity on RKE2

Tue, 15 Oct 2024 11:30:00 -0500

Fixing Argo CD Out-of-Sync Due to Cilium Identity on RKE2

If you’re running a Kubernetes cluster with RKE2 and Cilium as your networking solution, you might encounter a situation where Argo CD shows an out-of-sync status due to the presence of CiliumIdentity resources. Although the actual application is in sync and functioning normally, this false out-of-sync status can be frustrating to see in your CI/CD workflows.

Cilium Troubleshooting Guide

Wed, 27 Mar 2024 10:00:00 -0500

Troubleshooting Cilium in Kubernetes can be daunting. This guide provides steps and tools to help diagnose and resolve common issues. Utilize upstream guides and tools for in-depth analysis and health verification of your cluster’s Cilium deployment.

Navigating the Depths of Kubernetes Networking with Cilium: A Comprehensive Guide

Wed, 27 Mar 2024 10:00:00 -0500

This guide embarks on a journey to unravel the complexities of Kubernetes networking and spotlights Cilium’s transformative role in the container orchestration landscape. Through an in-depth exploration, we’ll navigate the challenges and opportunities presented by integrating advanced Container Network Interface (CNI) solutions, focusing on Cilium’s capabilities and symbiotic relationship with eBPF technology.

Kubernetes Networking with Cilium: Understanding the Basics

Thu, 02 Feb 2023 23:51:00 -0600

Kubernetes is an open-source platform for automating the deployment, scaling, and management of containerized applications. One of the key components of a Kubernetes deployment is its network infrastructure, and there are various solutions available to address this requirement. One such solution is Cilium, an open-source tool that provides advanced networking capabilities for Kubernetes.