Troubleshooting Cilium in Kubernetes can be daunting. This guide provides steps and tools to help diagnose and resolve common issues. Utilize upstream guides and tools for in-depth analysis and health verification of your cluster’s Cilium deployment.

Upstream Guides

Refer to the official Cilium documentation for troubleshooting and performance tuning:

Cluster Health

Ensure that all nodes have a Cilium agent pod in the Running state:

kubectl -n kube-system get pods -l k8s-app=cilium

Cilium agent pods include a CLI for useful commands. To verify the state of an agent:

kubectl -n kube-system exec -it <agent-pod> -- cilium status --verbose

Inspect agent logs for more insights:

kubectl -n kube-system logs --timestamps <agent-pod>

Connectivity Health

Perform upstream validation to ensure proper operation:

kubectl create ns cilium-test
kubectl apply -n cilium-test -f<cilium_version>/examples/kubernetes/connectivity-check/connectivity-check.yaml

This creates pods to verify connectivity, network policies, etc. All pods must be Running to conclude successful setup. It’s crucial to replace <cilium_version> with your Cilium version.

Cilium Connectivity Check


Cilium translates the frontend cluster IP address into a backend pod IP. To list and inspect services:

cilium status --verbose
cilium service list

Cilium Status Cilium Service List

Endpoint Health & Monitoring

Identify and monitor the health and status of endpoints. To list endpoints:

kubectl -n kube-system exec -it <agent-pod> -- cilium endpoint list

Check the health and status of a specific endpoint by its ID:

kubectl -n kube-system exec -it <agent-pod> -- cilium endpoint health <id>
kubectl -n kube-system exec -it <agent-pod> -- cilium endpoint get <id>

Monitor events for a specific endpoint to troubleshoot connectivity or policy enforcement issues:

kubectl -n kube-system exec -it <agent-pod> -- cilium endpoint monitor --related-to <id>


Details for managing services in a kube-proxyless environment using Cilium. For IP masquerading and viewing NAT entries:

kubectl exec -it <agent-pod> -n kube-system -- cilium bpf nat list

For eBPF Conntrack table listings, which help in understanding how connections are being tracked:

kubectl exec -it <agent-pod> -n kube-system -- cilium bpf ct list global

List load-balanced services managed by Cilium:

kubectl exec -it <agent-pod> -n kube-system -- cilium bpf lb list


For diagnosing common issues, finding the lxc interface of a pod can be crucial for debugging. To find the lxc interface:

kubectl -n kube-system exec -it <agent-pod> -- cilium endpoint list
kubectl -n kube-system exec -it <agent-pod> -- cilium endpoint get <id>

This provides detailed information about the endpoint, including the lxc interface.

To inspect packets dropped by network policies, which is useful for diagnosing connectivity issues caused by policy misconfigurations:

kubectl -n kube-system exec -it <agent-pod> -- cilium monitor --type drop


For managing and diagnosing Cilium within your cluster, these tools can be very helpful. To retrieve a Cilium pod managing a particular Kubernetes pod:

curl -sLO
./ <pod> <namespace>

Execute a command in all Kubernetes Cilium pods to help in widespread troubleshooting:

curl -sLO
./ <command>

List unmanaged Kubernetes pods, which might not be under Cilium’s control or observation:

curl -sLO

Data Collection

Automate log and state collection for deep analysis. This can be crucial for troubleshooting complex issues or for providing information to support cases:

curl -sLO
python --nodes <nodes,> --since <duration> --size-limit <size>

Remember to replace <nodes,>, <duration>, and <size> with your specific parameters for targeted data collection.

For more detailed instructions and advanced troubleshooting, refer to the Cilium documentation.