The recent discovery of CVE-2025-1974, a critical vulnerability in the ingress-nginx controller with a CVSS score of 9.8, has sent Kubernetes administrators scrambling to patch their clusters. While my previous post covered general mitigation strategies, RKE2 users face unique challenges due to its specialized deployment methodology. As someone who’s helped numerous RKE2 users tackle this vulnerability, I’ve compiled this guide to address the specific nuances of updating ingress-nginx in RKE2 environments.

Understanding the RKE2 Difference

RKE2 (Rancher Kubernetes Engine 2) handles ingress-nginx differently than standard Kubernetes distributions. There are two key scenarios to consider:

  1. Default RKE2 ingress-nginx - RKE2 ships with ingress-nginx as a built-in component
  2. Custom ingress-nginx deployments - Separately deployed via Helm or manifests

Each scenario requires a different mitigation approach, so let’s dive into the specific strategies for each.

Scenario 1: Updating the Default RKE2 ingress-nginx Controller

If you’re using the default ingress-nginx that ships with RKE2, your update path depends on how your cluster was deployed - either via Rancher’s UI or using the standalone RKE2 CLI.

For Rancher-Managed RKE2 Clusters

If your RKE2 cluster is managed through Rancher, follow these steps:

  1. Check Current RKE2 Version

    Navigate to your cluster in the Rancher UI and note the RKE2 version.

  2. Verify Vulnerability Status

    Check if your RKE2 version includes a vulnerable ingress-nginx version:

    # SSH into a control plane node
ssh user@rke2-server

# Check the running ingress-nginx version
kubectl get deployment -n kube-system rke2-ingress-nginx-controller -o jsonpath='{.spec.template.spec.containers[0].image}'

    If the version is older than v1.11.5 or v1.12.1, your cluster is vulnerable.

  3. Update the Cluster

    In the Rancher UI:

    • Navigate to the cluster
    • Click “⋮” > “Edit Config”
    • Under “Kubernetes Version,” select a version that includes the patched ingress-nginx
    • Recent RKE2 releases (v1.28.6+, v1.27.10+, v1.26.14+) include the patched versions
    • Click “Save” to begin the upgrade process

    Rancher will orchestrate a rolling upgrade of your cluster nodes.

  4. Verify the Update

    After the upgrade completes, verify the new ingress-nginx version:

    kubectl get deployment -n kube-system rke2-ingress-nginx-controller -o jsonpath='{.spec.template.spec.containers[0].image}'

For Standalone RKE2 Clusters

If you’re running RKE2 without Rancher management, you’ll need to update using the RKE2 CLI:

  1. Check Which Version to Upgrade To

    Review the RKE2 release notes to identify a version with the patched ingress-nginx.

  2. Update RKE2 on the First Server Node

    # Update the RKE2 package
curl -sfL https://get.rke2.io | INSTALL_RKE2_CHANNEL=v1.28 sh -

# Restart the RKE2 service
systemctl restart rke2-server.service

# Verify the ingress-nginx version after restart
/var/lib/rancher/rke2/bin/kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get deployment -n kube-system rke2-ingress-nginx-controller -o jsonpath='{.spec.template.spec.containers[0].image}'

  3. Update Remaining Control Plane Nodes

    Repeat the same steps on each control plane node, one at a time. Wait for each node to become ready before proceeding to the next:

    # Check node status
/var/lib/rancher/rke2/bin/kubectl --kubeconfig /etc/rancher/rke2/rke2.yaml get nodes

  4. Update Worker Nodes

    Finally, update each worker node:

    # On worker nodes:
curl -sfL https://get.rke2.io | INSTALL_RKE2_CHANNEL=v1.28 sh -
systemctl restart rke2-agent.service

Disable the Default ingress-nginx (Alternative Approach)

If you can’t immediately upgrade RKE2 but need to mitigate the vulnerability, you can disable the default ingress-nginx and deploy a patched version separately:

  1. Create a Custom Config to Disable Default ingress-nginx

    # On server nodes
mkdir -p /etc/rancher/rke2

# Create or edit the config file
cat << EOF > /etc/rancher/rke2/config.yaml
disable: rke2-ingress-nginx
EOF

# Restart RKE2
systemctl restart rke2-server.service

    For worker nodes, restart the rke2-agent service instead.

  2. Deploy a Patched Version (Continue to the next section)

Scenario 2: Deploying a Custom Patched ingress-nginx

Whether you’ve disabled the default controller or are already using a custom deployment, here’s how to deploy a patched version of ingress-nginx on RKE2:

  1. Add the Helm Repository

    helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

  2. Install or Upgrade ingress-nginx

    # For new installations
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace \
  --set controller.image.tag=v1.12.1 \
  --set controller.admissionWebhooks.enabled=true \
  --set controller.service.type=LoadBalancer

# For upgrades of existing Helm installations
helm upgrade ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --set controller.image.tag=v1.12.1 \
  --reuse-values

  3. Verify the Deployment

    kubectl get deployment -n ingress-nginx ingress-nginx-controller -o jsonpath='{.spec.template.spec.containers[0].image}'

RKE2-Specific Configuration Considerations

When deploying ingress-nginx on RKE2, consider these important configuration adjustments:

  1. Set the Publish Service Correctly

    RKE2 uses a specific networking setup. Ensure the publish-service flag is correctly set:

    controller:
  publishService:
    enabled: true
    pathOverride: ingress-nginx/ingress-nginx-controller

  2. Configure IngressClass

    If migrating from the default RKE2 controller, ensure you set up the IngressClass properly:

    # For Helm values
controller:
  ingressClassResource:
    name: nginx
    default: true  # Only if you want this to be the default

    Create a matching IngressClass:

    apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: nginx
  annotations:
    ingressclass.kubernetes.io/is-default-class: "true"  # Only if you want this to be the default
spec:
  controller: k8s.io/ingress-nginx

  3. Update Existing Ingress Resources

    Check if your existing Ingress resources need to be updated:

    # Find Ingress resources using the rke2-nginx class
kubectl get ingress --all-namespaces -o json | jq '.items[] | select(.spec.ingressClassName == "rke2-nginx") | .metadata.namespace + "/" + .metadata.name'

    Update them to use your new IngressClass:

    kubectl patch ingress INGRESS_NAME -n NAMESPACE --type=json -p='[{"op": "replace", "path": "/spec/ingressClassName", "value": "nginx"}]'

Temporary Mitigation: Disable the Admission Webhook

If you can’t immediately update ingress-nginx but need to mitigate the vulnerability, you can disable the admission webhook component:

For Default RKE2 ingress-nginx

# Patch the deployment to disable the admission webhook
kubectl patch deployment rke2-ingress-nginx-controller -n kube-system --type=json \
  -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/args", "value": ["--election-id=ingress-controller-leader", "--controller-class=k8s.io/ingress-nginx", "--configmap=kube-system/rke2-ingress-nginx-controller", "--validating-webhook=false", "--validating-webhook-certificate=", "--validating-webhook-key="]}]'

# Delete the ValidatingWebhookConfiguration
kubectl delete validatingwebhookconfiguration rke2-ingress-nginx-admission -n kube-system

For Custom Helm-Deployed ingress-nginx

# Update with admission webhooks disabled
helm upgrade ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --set controller.admissionWebhooks.enabled=false \
  --reuse-values

Advanced Mitigation: Network-Level Protection

If you need a network-level mitigation for RKE2, you can use the same DaemonSet approach detailed in my previous post, but with RKE2-specific adjustments:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: webhook-protector-rke2
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: webhook-protector-rke2
  template:
    metadata:
      labels:
        app: webhook-protector-rke2
    spec:
      hostNetwork: true
      containers:
      - name: iptables-manager
        image: rancher/mirrored-alpine:3.19
        command:
        - /bin/sh
        - -c
        - |
          apk add --no-cache iptables curl
          
          # Allow API server access (RKE2-specific approach)
          # RKE2 typically uses the API server on localhost:6443
          iptables -N WEBHOOK-PROTECTION || true
          iptables -F WEBHOOK-PROTECTION
          
          # Allow localhost access - needed for RKE2's local kube-apiserver
          iptables -A WEBHOOK-PROTECTION -s 127.0.0.1 -p tcp --dport 8443 -j ACCEPT
          
          # Allow pod CIDR ranges (specific to your RKE2 setup)
          POD_CIDR=$(ip route | grep -E "pod|cluster" | awk '{print $1}')
          if [ -n "$POD_CIDR" ]; then
            echo "Detected Pod CIDR: $POD_CIDR"
            # Allow only kube-apiserver from the pod network
            iptables -A WEBHOOK-PROTECTION -s $POD_CIDR -p tcp --dport 8443 -j ACCEPT
          fi
          
          # Block all other access
          iptables -A WEBHOOK-PROTECTION -p tcp --dport 8443 -j DROP
          
          # Insert the chain into the INPUT chain
          iptables -D INPUT -j WEBHOOK-PROTECTION 2>/dev/null || true
          iptables -I INPUT -j WEBHOOK-PROTECTION
          
          echo "Protection applied, sleeping..."
          sleep infinity
        securityContext:
          privileged: true
      tolerations:
      - operator: "Exists"

Apply this with:

kubectl apply -f webhook-protector-rke2.yaml

Special Considerations for Air-Gapped RKE2 Environments

Air-gapped RKE2 installations require additional steps:

  1. Pre-Download Updated Container Images

    # On a connected system, pull and save the updated images
docker pull registry.k8s.io/ingress-nginx/controller:v1.12.1
docker pull registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0
docker save registry.k8s.io/ingress-nginx/controller:v1.12.1 > ingress-controller.tar
docker save registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0 > webhook-certgen.tar

  2. Transfer Images to Air-Gapped Environment

    Transfer the image tarballs to your air-gapped environment.

  3. Import Images to Private Registry

    # Load images
docker load < ingress-controller.tar
docker load < webhook-certgen.tar

# Tag for your private registry
docker tag registry.k8s.io/ingress-nginx/controller:v1.12.1 your.private.registry/ingress-nginx/controller:v1.12.1
docker tag registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.0 your.private.registry/ingress-nginx/kube-webhook-certgen:v1.4.0

# Push to private registry
docker push your.private.registry/ingress-nginx/controller:v1.12.1
docker push your.private.registry/ingress-nginx/kube-webhook-certgen:v1.4.0

  4. Deploy Using Private Registry

    helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx \
  --create-namespace \
  --set controller.image.registry=your.private.registry \
  --set controller.image.image=ingress-nginx/controller \
  --set controller.image.tag=v1.12.1 \
  --set controller.admissionWebhooks.patch.image.registry=your.private.registry \
  --set controller.admissionWebhooks.patch.image.image=ingress-nginx/kube-webhook-certgen \
  --set controller.admissionWebhooks.patch.image.tag=v1.4.0

Monitoring and Verification

After applying mitigations, verify that your RKE2 cluster is protected:

  1. Check Running Version

    For default RKE2 controller:

    kubectl get deployment -n kube-system rke2-ingress-nginx-controller -o jsonpath='{.spec.template.spec.containers[0].image}'

    For custom deployments:

    kubectl get deployment -n ingress-nginx ingress-nginx-controller -o jsonpath='{.spec.template.spec.containers[0].image}'

  2. Verify Webhook Status

    If you’ve disabled the webhook:

    # Check if the webhook configuration exists
kubectl get validatingwebhookconfiguration | grep nginx

# Check if the controller has the webhook disabled in its args
kubectl get deployment -n kube-system rke2-ingress-nginx-controller -o jsonpath='{.spec.template.spec.containers[0].args}' | grep validating-webhook

  3. Test Ingress Functionality

    Deploy a test application to verify your ingress continues to work:

    # Create a test deployment
kubectl create deployment nginx --image=nginx
kubectl expose deployment nginx --port=80

# Create a test ingress
cat << EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-test
spec:
  ingressClassName: nginx  # Or "rke2-nginx" for default controller
  rules:
  - host: "test.example.com"
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx
            port:
              number: 80
EOF

# Test access
curl -H "Host: test.example.com" http://<INGRESS_IP>/

Understanding the Risk in RKE2 Environments

RKE2 clusters face the same risks as any Kubernetes cluster with this vulnerability, but there are additional considerations:

  1. Default Controller is System-Level - The default ingress-nginx runs in the kube-system namespace with elevated privileges

  2. Fleet/Rancher Integration - If you’re using Rancher Fleet for GitOps, the vulnerability could potentially be exploited to gain access across multiple clusters

  3. CNI Considerations - RKE2’s default CNI (Canal/Calico) may influence how you can isolate the webhook endpoint

Long-Term Recommendations for RKE2 Environments

Beyond immediate mitigation, consider these RKE2-specific recommendations:

  1. Implement Systematic Updates

    Create a regular process to update RKE2, particularly when security patches are released:

    #!/bin/bash
# Script to systematically update RKE2 clusters

# Define update order - control plane first, then workers
CONTROL_PLANE_NODES="server1 server2 server3"
WORKER_NODES="worker1 worker2 worker3"

# Target version
TARGET_VERSION="v1.28.6+rke2r1"

# Update control plane nodes one by one
for node in $CONTROL_PLANE_NODES; do
  echo "Updating control plane node $node..."
  ssh $node "curl -sfL https://get.rke2.io | INSTALL_RKE2_VERSION=$TARGET_VERSION sh -"
  ssh $node "systemctl restart rke2-server.service"

  # Wait for node to become ready
  echo "Waiting for node $node to become ready..."
  sleep 60
done

# Update worker nodes (can be done in parallel if desired)
for node in $WORKER_NODES; do
  echo "Updating worker node $node..."
  ssh $node "curl -sfL https://get.rke2.io | INSTALL_RKE2_VERSION=$TARGET_VERSION sh -"
  ssh $node "systemctl restart rke2-agent.service"
done

  2. Consider Custom Ingress Controller Deployment

    Instead of relying on the built-in controller, consider deploying ingress-nginx separately via Helm. This gives you more control over updates:

    # Disable the default controller in RKE2 config
cat << EOF > /etc/rancher/rke2/config.yaml
disable: rke2-ingress-nginx
EOF

# Deploy your own via Helm or operator

  3. Implement Network Policies

    Use NetworkPolicies to isolate the ingress-nginx controller:

    apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: protect-ingress-nginx
  namespace: kube-system  # or ingress-nginx for custom deployments
spec:
  podSelector:
    matchLabels:
      app: rke2-ingress-nginx  # adjust selector as needed
  policyTypes:
  - Ingress
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          kubernetes.io/metadata.name: kube-system
  - ports:
    - port: 80
      protocol: TCP
    - port: 443
      protocol: TCP

Conclusion

Mitigating CVE-2025-1974 in RKE2 environments requires understanding the unique aspects of how RKE2 deploys and manages the ingress-nginx controller. Whether you’re running the default controller or a custom deployment, the strategies outlined in this guide provide multiple paths to securing your clusters.

For RKE2 users, the most straightforward approach is typically to update to a newer RKE2 version that includes the patched ingress-nginx. If that’s not immediately feasible, disabling the admission webhook component provides effective interim protection without significant operational impact.

Remember that the security of your Kubernetes infrastructure is an ongoing responsibility. Use this vulnerability as an opportunity to implement more robust update procedures and security measures for your RKE2 clusters.

Have you encountered any RKE2-specific challenges when mitigating this vulnerability? Share your experiences in the comments to help the community navigate these security waters.