https://support.tools/tags/rancher/Recent content in Rancher on Support ToolsHugoen-USWed, 04 Feb 2026 00:00:00 -0500https://support.tools/ingress-nginx-retirement-rke2-migrating-traefik-without-downtime/Wed, 04 Feb 2026 00:00:00 -0500https://support.tools/ingress-nginx-retirement-rke2-migrating-traefik-without-downtime/<p>The Kubernetes community announced in November 2025 that Ingress-NGINX is being retired. Maintenance halts completely in March 2026 — no more releases, no more bug fixes, no more security patches. If you&rsquo;re running RKE2, which ships ingress-nginx by default, this directly affects you.</p> <p>This post walks through the practical side of migrating RKE2 clusters from ingress-nginx to Traefik. The emphasis is on running both controllers side-by-side so you can migrate gradually without taking production offline.</p>https://support.tools/kubernetes-practical-tips/Tue, 01 Jul 2025 00:00:00 -0500https://support.tools/kubernetes-practical-tips/<p>Kubernetes is a powerful tool for managing containerized workloads—but with great power comes great complexity. Whether you&rsquo;re just starting with Kubernetes or looking to tighten up your production clusters, these <strong>10 practical tips (plus a bonus)</strong> will help you reduce pain, improve resilience, and simplify day-to-day operations.</p>https://support.tools/mitigating-cve-2025-1974-ingress-nginx-rke2/Thu, 29 May 2025 09:00:00 -0500https://support.tools/mitigating-cve-2025-1974-ingress-nginx-rke2/<p>The recent discovery of CVE-2025-1974, a critical vulnerability in the ingress-nginx controller with a CVSS score of 9.8, has sent Kubernetes administrators scrambling to patch their clusters. While my <a href="https://support.tools/ingress-nginx-cve-2025-1974-critical-vulnerability/">previous post</a> covered general mitigation strategies, RKE2 users face unique challenges due to its specialized deployment methodology. As someone who&rsquo;s helped numerous RKE2 users tackle this vulnerability, I&rsquo;ve compiled this guide to address the specific nuances of updating ingress-nginx in RKE2 environments.</p>https://support.tools/known-good-designs/downstream-rke2-cluster/rke2-aws/Tue, 17 Dec 2024 00:00:00 -0500https://support.tools/known-good-designs/downstream-rke2-cluster/rke2-aws/<p>This guide demonstrates how to add AWS cloud credentials in Rancher and use them to provision an RKE2 cluster in AWS.</p>https://support.tools/known-good-designs/downstream-rke2-cluster/rke2-do/Tue, 17 Dec 2024 00:00:00 -0500https://support.tools/known-good-designs/downstream-rke2-cluster/rke2-do/<p>This guide demonstrates how to add DigitalOcean cloud credentials in Rancher and use them to provision an RKE2 cluster in DigitalOcean (DO).</p>https://support.tools/known-good-designs/downstream-rke2-cluster/rke2-custom-vms/Tue, 17 Dec 2024 00:00:00 -0500https://support.tools/known-good-designs/downstream-rke2-cluster/rke2-custom-vms/<p>This guide demonstrates how to provision an RKE2 cluster on custom VMs that are created and configured outside Rancher, and then import the cluster into Rancher for centralized management.</p>https://support.tools/known-good-designs/upstream-rancher-cluster/rancher-doks-small/Tue, 17 Dec 2024 00:00:00 -0500https://support.tools/known-good-designs/upstream-rancher-cluster/rancher-doks-small/<p>This guide demonstrates how to deploy a small DOKS (DigitalOcean Kubernetes Service) cluster with Rancher installed, using Terraform for infrastructure provisioning and Helm for application deployment.</p>https://support.tools/known-good-designs/upstream-rancher-cluster/rancher-eks-small/Tue, 17 Dec 2024 00:00:00 -0500https://support.tools/known-good-designs/upstream-rancher-cluster/rancher-eks-small/<p>This guide demonstrates how to deploy a small EKS cluster with Rancher installed, using Terraform for infrastructure provisioning and Helm for application deployment.</p>https://support.tools/known-good-designs/upstream-rancher-cluster/rancher-rke2-small/Tue, 17 Dec 2024 00:00:00 -0500https://support.tools/known-good-designs/upstream-rancher-cluster/rancher-rke2-small/<p>This guide demonstrates how to deploy a small RKE2 cluster with Rancher installed, designed for on-premises environments with three pre-configured servers.</p>https://support.tools/rancher-projects-namespaces/Tue, 10 Dec 2024 13:30:00 -0500https://support.tools/rancher-projects-namespaces/<p>When managing Kubernetes clusters with Rancher, Projects and Namespaces play a crucial role in organizing workloads and setting access controls. This post explores how Rancher handles Projects and Namespaces, their capabilities, limitations, and integration with downstream clusters.</p> <h1 id="overview-of-rancher-projects-and-namespaces"><a href="#overview-of-rancher-projects-and-namespaces">Overview of Rancher Projects and Namespaces</a></h1> <h2 id="section-1-what-are-rancher-projects-and-namespaces">Section 1: What Are Rancher Projects and Namespaces?</h2> <p>Rancher introduces the concept of Projects as an abstraction layer on top of Kubernetes Namespaces. While Kubernetes natively uses Namespaces to segment workloads, Rancher Projects allow you to group multiple Namespaces under a single entity for easier management and access control.</p>https://support.tools/rancher-capi-finalizers/Tue, 03 Dec 2024 21:00:00 -0500https://support.tools/rancher-capi-finalizers/<p>Rancher and Cluster API (CAPI) are powerful tools for managing Kubernetes clusters, from provisioning to lifecycle management. Both frameworks rely heavily on <strong>finalizers</strong> to coordinate resource cleanup and ensure that no dangling dependencies or orphaned infrastructure remain after a resource is deleted. In this post, we’ll explore how Rancher and CAPI use finalizers, what happens during cluster deletion, common pitfalls, and best practices for handling stuck finalizers.</p>https://support.tools/finalizers-kubernetes-rancher/Tue, 03 Dec 2024 20:00:00 -0500https://support.tools/finalizers-kubernetes-rancher/<p>Finalizers in Kubernetes are powerful tools for resource lifecycle management. They ensure that cleanup tasks and dependencies are handled properly before a resource is deleted. However, finalizers can become problematic, especially when mismanaged or when resources get stuck in a <code>Terminating</code> state. This post provides a deep dive into finalizers, including their role in Kubernetes, namespace management, controllers, and how to handle them safely.</p>https://support.tools/what-is-ranchers-steve-and-why-does-it-matter/Tue, 19 Nov 2024 19:26:00 -0500https://support.tools/what-is-ranchers-steve-and-why-does-it-matter/<p>Steve is a critical component in Rancher’s architecture, acting as a caching proxy between Rancher and downstream Kubernetes clusters. By introducing an efficient caching layer, Steve optimizes how Rancher&rsquo;s UI interacts with large-scale Kubernetes environments.</p> <p><a href="https://github.com/rancher/steve">Steve GitHub Repository</a></p>https://support.tools/who-is-norman-and-why-does-he-do-so-much-work/Tue, 19 Nov 2024 19:26:00 -0500https://support.tools/who-is-norman-and-why-does-he-do-so-much-work/<p>Rancher’s Norman is a key component of the Rancher ecosystem. It not only facilitates API interactions between Rancher and Kubernetes but also powers the internal operations of Rancher server pods, ensuring consistency and reliability across multiple deployments.</p> <p><a href="https://github.com/rancher/norman">Norman GitHub Repository</a></p>https://support.tools/rancher-system-agent-rke2-config/Tue, 12 Nov 2024 20:00:00 -0500https://support.tools/rancher-system-agent-rke2-config/<p>Rancher System Agent plays a pivotal role in automating the installation and management of RKE2 clusters. This lightweight agent simplifies operations by handling the setup and maintenance of RKE2 configuration files.</p>https://support.tools/cluster-autoscaling-rancher/Sun, 27 Oct 2024 09:30:00 -0500https://support.tools/cluster-autoscaling-rancher/<p>Autoscaling Kubernetes clusters is essential to <strong>optimizing performance and reducing costs</strong>. <strong>Cluster Autoscaler (CA)</strong> helps you dynamically scale nodes to meet changing workloads, reducing resource waste while keeping applications reliable. In this post, we’ll explore <strong>how CA works</strong>, how it compares with <strong>Horizontal Pod Autoscaler (HPA)</strong> and <strong>Vertical Pod Autoscaler (VPA)</strong>, and walk through implementing CA in Kubernetes using <strong>Rancher</strong>.</p> <hr> <h2 id="understanding-the-different-types-of-autoscaling-in-kubernetes">Understanding the Different Types of Autoscaling in Kubernetes</h2> <p>Kubernetes supports three key types of autoscaling:</p>https://support.tools/minio-backup-target-longhorn/Sat, 26 Oct 2024 11:45:00 -0500https://support.tools/minio-backup-target-longhorn/<p><strong>Longhorn</strong>, an open-source container-native storage system developed by Rancher, simplifies <strong>persistent volume backups</strong> by supporting <strong>S3 and NFS</strong> storage systems as backup targets. In this guide, we’ll walk through how to <strong>configure MinIO</strong> as an S3-compatible backup target for Longhorn, enabling reliable volume backups.</p> <p>This post assumes that <strong>Longhorn and MinIO are already installed</strong>—for installation guides, refer to the official <a href="https://longhorn.io/">Longhorn documentation</a> and <a href="https://min.io/docs/">MinIO Quickstart Guide</a>.</p> <hr> <h2 id="environment-overview">Environment Overview</h2> <ul> <li><strong>MinIO version:</strong> RELEASE.2021-08-17T20-53-08Z (deployed with Podman on SUSE SLES 15 SP2)</li> <li><strong>Longhorn version:</strong> 1.1.1 (installed via Rancher’s Application Catalog on an RKE cluster running Kubernetes 1.20.9-rancher1-1)</li> </ul> <hr> <h2 id="step-1-configure-minio-for-longhorn-backups">Step 1: Configure MinIO for Longhorn Backups</h2> <p>First, we’ll set up <strong>MinIO storage</strong> by creating a dedicated bucket, folder, user, and policy for Longhorn backups. Use MinIO’s command-line tool, <code>mc</code>, to manage these objects.</p>https://support.tools/training/rancher/Fri, 25 Oct 2024 00:00:00 +0000https://support.tools/training/rancher/<p>Rancher provides powerful tools to simplify multi-cluster management. In this section, explore guides, troubleshooting tips, and strategies for getting the most out of Rancher.</p> <h2 id="rancher-training-resources">Rancher Training Resources</h2> <ul> <li><a href="https://support.tools/training/rancher/rancher-agents/">Rancher Agents</a></li> <li><a href="https://support.tools/training/rancher/rancher-certs/">Rancher Certificates</a></li> </ul>https://support.tools/rancher-rke2-backups/Tue, 22 Oct 2024 19:45:00 -0500https://support.tools/rancher-rke2-backups/<h1 id="no-one-has-ever-gotten-fired-for-having-too-many-backups">“No One Has Ever Gotten Fired for Having Too Many Backups”</h1> <p>When running <strong>Rancher</strong> or <strong>RKE2</strong>, there’s one mantra to live by: <strong>“Always back it up before making a change.”</strong> Backups are not just about recovering from disasters—they’re about giving you the confidence to experiment, upgrade, and troubleshoot without fear. This post covers essential strategies for <strong>Kubernetes backups</strong>, with practical advice and links to tools and methods to make backup processes effortless.</p>https://support.tools/kubernetes-master-class-upgrades/Mon, 14 Oct 2024 17:30:00 -0500https://support.tools/kubernetes-master-class-upgrades/<h2 id="kubernetes-master-class-a-seamless-approach-to-rancher--kubernetes-upgrades">Kubernetes Master Class: A Seamless Approach to Rancher &amp; Kubernetes Upgrades</h2> <p>Upgrading your <strong>Rancher</strong> and <strong>Kubernetes clusters</strong> requires a strategic approach to minimize downtime and ensure seamless operation. This guide covers <strong>rules</strong>, <strong>backup operations</strong>, <strong>planning steps</strong>, and <strong>upgrade procedures</strong> for Rancher and Kubernetes environments using <strong>RKE1</strong> and <strong>RKE2</strong>.</p> <hr> <h2 id="high-level-rules-for-upgrades">High-Level Rules for Upgrades</h2> <ol start="0"> <li> <p><strong>Create an upgrade plan</strong></p> <ul> <li>Using the rule listed below, create a plan for all of your upgrades and follow it.</li> <li>You might need to do multiple upgrades to get to the latest version.</li> <li>Please use Rancher Upgrade Tool to help you pick the right version(s) to upgrade to. <a href="https://rancher.com/docs/rancher/v2.x/en/upgrades/tools/">Rancher Upgrade Tool</a></li> </ul> </li> <li> <p><strong>Don’t Rush Upgrades:</strong></p> <ul> <li>Allow at least <strong>24 hours</strong> between upgrades to ensure the stability of each component.</li> <li>Give yourself plenty of time for backups, testing, monitoring, and rollback. I recommend at least 4hr change window for each upgrade.</li> </ul> </li> <li> <p><strong>Don’t Stack Upgrades:</strong></p> <ul> <li>Avoid upgrading Rancher, Kubernetes, and Docker/Containerd in one session to reduce risk. Perform them <strong>sequentially</strong>.</li> </ul> </li> <li> <p><strong>Backups Are Mandatory:</strong></p> <ul> <li>Take <strong>ETCD snapshots</strong> and use the <strong>Rancher Backup Operator</strong> to ensure quick recovery in case of failure.</li> </ul> </li> <li> <p><strong>Upgrade Order:</strong></p> <ul> <li>Follow the sequence: <strong>Rancher → Kubernetes → Docker/Containerd → Operating System</strong>.</li> </ul> </li> <li> <p><strong>Pause CI/CD Pipelines:</strong></p> <ul> <li>Halt pipelines using the Rancher API to prevent conflicts during upgrades.</li> </ul> </li> <li> <p><strong>Test in Non-Production Environments:</strong></p> <ul> <li>Always validate upgrades in a <strong>lab environment</strong> before deploying to production.</li> </ul> </li> <li> <p><strong>Review Release Notes and Support Matrix:</strong></p> <ul> <li>Check the <strong>Rancher release notes</strong> and <strong>Kubernetes support matrix</strong> to avoid issues with version incompatibilities.</li> </ul> </li> <li> <p><strong>Monitor and Verify:</strong></p> <ul> <li>Continuously monitor the health of nodes and pods after each upgrade to ensure everything is running smoothly.</li> </ul> </li> </ol>https://support.tools/fleet-argocd-audit-logs-troubleshooting/Wed, 11 Sep 2024 02:26:00 -0500https://support.tools/fleet-argocd-audit-logs-troubleshooting/<p>In environments where GitOps tools like <strong>Fleet</strong> or <strong>ArgoCD</strong> are used to manage Kubernetes clusters, changes can be rapid and automated. However, when things go wrong—such as deployments failing, or configurations being overwritten—it&rsquo;s important to know <strong>who</strong> made the change, <strong>what</strong> was changed, and <strong>when</strong> the change occurred.</p> <p>In this post, we’ll focus on how you can use audit logs from Rancher, Fleet, and ArgoCD to track down the root cause of issues. We&rsquo;ll walk through practical examples, and show you how to trace changes back to the <strong>IP address</strong> or <strong>user</strong> responsible.</p>https://support.tools/post/rancher-codebase-guide/Wed, 21 Aug 2024 01:00:00 -0500https://support.tools/post/rancher-codebase-guide/<p>As a support engineer working with Rancher, understanding the codebase on GitHub is essential for troubleshooting, debugging, and providing effective support. The Rancher codebase is extensive and involves multiple components that work together to deliver the full functionality of the platform. This guide will help you navigate the codebase, understand its structure, and identify where to look when addressing common support issues. Additionally, we’ll dive into the Go programming language basics, which is used extensively in Rancher, and introduce tools like Sourcegraph and Visual Studio Code (VS Code) to help you navigate the code more effectively.</p>https://support.tools/coredns-and-rancher/Wed, 14 Aug 2024 05:15:00 -0500https://support.tools/coredns-and-rancher/<p>In the evolving landscape of Kubernetes, DNS plays a crucial role in ensuring smooth service discovery and communication within and outside the cluster. CoreDNS, a flexible and scalable DNS server, has been the default choice for Kubernetes since version 1.14. In this post, we&rsquo;ll dive into CoreDNS&rsquo;s role within Rancher-managed Kubernetes environments, particularly focusing on RKE1 and RKE2. We&rsquo;ll explore common issues, troubleshooting techniques, and best practices to optimize your CoreDNS setup.</p>https://support.tools/post/rancher-backups-and-disaster-recovery/Sun, 30 Jun 2024 23:18:00 -0500https://support.tools/post/rancher-backups-and-disaster-recovery/<p>Eventually, something will happen that requires restoring a Kubernetes cluster. That&rsquo;s not a question of if but when.</p> <p>In the case of Kubernetes, this means regularly backing up the etcd datastore.</p> <p>How regular? The choice is yours. It is up to you how much data you are willing to lose.</p> <p>For example, a highly active cluster may take snapshots every fifteen minutes or even every five minutes. In the case of a cluster that does not receive a lot of activity, a snapshot of the cluster might be taken daily.</p> <p>You can never be fired for having too many backups.</p> <p>As a default, RKE takes a snapshot every six hours and keeps it for one day, but you can change the settings in the <code>cluster.yaml</code>.</p> <p>Any time you wish, you can run a snapshot by executing the command <code>rke etcd snapshot-save</code> and passing it the name of the backup and the location of <code>cluster.yml</code>. This command will write a snapshot of all the etcd hosts to <code>/opt/rke/etcd-snapshots</code>. It is possible to mount an NFS volume there, or you can configure RKE to copy the snapshot to S3 to restore it.</p>https://support.tools/post/hardening-rke1-rke2/Wed, 08 May 2024 04:30:00 -0500https://support.tools/post/hardening-rke1-rke2/<p>With this expert-led training class, learn the essentials of securing RKE1 and RKE2 in your Rancher environments. From basic principles to advanced strategies, discover how to protect your Kubernetes clusters effectively.</p>https://support.tools/post/harvester-101/Thu, 01 Feb 2024 10:00:00 -0500https://support.tools/post/harvester-101/<p>After transitioning my homelab from Proxmox to Harvester-v1.1.2, I&rsquo;ve gained a unique perspective on Rancher Harvester&rsquo;s role in modern virtualization. This post delves deeper into Harvester, answering key questions and sharing my hands-on experience.</p>https://support.tools/post/rancher-monitoring-v2/Wed, 10 Jan 2024 09:00:00 -0500https://support.tools/post/rancher-monitoring-v2/<h2 id="rancher-monitoring-v2-workshop">Rancher Monitoring v2 Workshop</h2> <p>Welcome to the Rancher Monitoring v2 Workshop! This workshop is designed to provide a comprehensive guide to deploying, managing, and troubleshooting Rancher Monitoring v2 in Kubernetes environments.</p> <h2 id="overview">Overview</h2> <p>Rancher Monitoring v2 offers advanced monitoring capabilities for Kubernetes clusters managed by Rancher. It is a powerful tool that integrates Prometheus, Grafana, and other monitoring components to provide deep insights into the health and performance of clusters.</p> <h3 id="goals-of-the-workshop">Goals of the Workshop</h3> <ul> <li>Understanding the architecture of Rancher Monitoring v2</li> <li>Learning how to deploy and configure Rancher Monitoring v2</li> <li>Best practices for monitoring Kubernetes clusters</li> <li>Troubleshooting common issues in Rancher Monitoring v2</li> <li>Hands-on experience with Rancher Monitoring v2</li> <li>Setting up alerts and notifications</li> <li>Setting up ingress for Grafana and Prometheus</li> <li>Setting up Grafana dashboards</li> </ul>https://support.tools/post/backup-rancher-and-its-clusters/Wed, 03 Jan 2024 10:00:00 +0000https://support.tools/post/backup-rancher-and-its-clusters/<h2 id="comprehensive-guide-to-backing-up-rancher-and-its-clusters">Comprehensive Guide to Backing Up Rancher and its Clusters</h2> <p>Ensuring the safety and continuity of data in Rancher-managed Kubernetes clusters is paramount. This workshop aims to provide you with a thorough understanding and practical approach to back up Rancher and its clusters, safeguarding your data against potential losses.</p> <h2 id="understanding-rancher-backups">Understanding Rancher Backups</h2> <p>Rancher, a popular Kubernetes management platform, simplifies container orchestration. However, like any system, it&rsquo;s prone to data loss due to hardware failures, software bugs, or human errors. Regular backups are crucial.</p>https://support.tools/post/k8s-ingress-troubleshooting/Wed, 13 Dec 2023 06:00:00 -0500https://support.tools/post/k8s-ingress-troubleshooting/<p>In the world of Kubernetes (k8s), setting up and maintaining Ingress controllers like Ingress-Nginx and Traefik is crucial for managing external access to your cluster&rsquo;s services. However, troubleshooting these can be a daunting task. This post aims to simplify that process, offering practical solutions for common issues faced with k8s Ingress, especially in a Rancher-managed environment.</p>https://support.tools/post/when-rancher-upgrades-go-sideways/Wed, 06 Dec 2023 00:00:00 +0000https://support.tools/post/when-rancher-upgrades-go-sideways/<p>Upgrading Rancher, the popular Kubernetes management platform, can be an essential task to ensure your cluster stays up-to-date with the latest features and security patches. However, sometimes Rancher upgrades can hit a roadblock, causing frustration for cluster administrators. In this post, we&rsquo;ll explore a common Rancher upgrade failure and provide a solution to get your upgrade back on track.</p> <h2 id="common-rancher-upgrade-failures">Common Rancher Upgrade Failures</h2> <ul> <li>Unsupported Kubernetes version</li> <li>Incompatible upgrade path</li> <li>Broken rancher-webhook</li> <li>SSL certificate issues</li> <li>Downstream cluster failing to reconnect</li> </ul> <h2 id="unsupported-kubernetes-version">Unsupported Kubernetes Version</h2> <p>It&rsquo;s important to note that Rancher is not compatible with every version of Kubernetes. For example, Rancher v2.7.9 only supports Kubernetes v1.23 through v1.26. So if you&rsquo;re running Kubernetes v1.22, you&rsquo;ll need to upgrade your cluster to v1.23 or higher before upgrading Rancher. There of course overlap between Rancher and Kubernetes versions, so you&rsquo;ll review the official <a href="https://www.suse.com/suse-rancher/support-matrix/all-supported-versions/">Support Matrix</a> to determine which versions are compatible.</p>https://support.tools/post/longhorn-deepdive/Wed, 01 Nov 2023 14:30:00 -0500https://support.tools/post/longhorn-deepdive/<p>In the world of Kubernetes, managing stateful applications can be a daunting task. That&rsquo;s where Rancher Longhorn comes into play. This blog post will cover what Rancher Longhorn is, its core features, and how it&rsquo;s making life easier for Kubernetes administrators and developers.</p>https://support.tools/post/rke2-with-rancher-disaster-recovery/Wed, 18 Oct 2023 01:15:00 -0600https://support.tools/post/rke2-with-rancher-disaster-recovery/<h2 id="rke2-with-rancher---disaster-recovery-guide">RKE2 with Rancher - Disaster Recovery Guide</h2> <h3 id="introduction">Introduction</h3> <p>The purpose of this guide is to provide a comprehensive guide for disaster recovery of RKE2 clusters managed by Rancher. RKE2, a CNCF-certified Kubernetes distribution, is a fully conformant Kubernetes distribution focusing on security and simplicity. Rancher is a CNCF-certified Kubernetes management platform that provides a single pane of glass for managing multiple Kubernetes clusters. Rancher can manage RKE2 clusters and other Kubernetes distributions such as RKE, EKS, AKS, GKE, and more.</p> <p>Please note that this guide is intended to be something other than a replacement for the official documentation. This guide is designed to supplement the official documentation and provide a comprehensive guide for disaster recovery of RKE2 clusters that Rancher manages. This guide also assumes a basic understanding of Kubernetes, RKE2, and Rancher.</p>https://support.tools/post/rke2-standalone-disaster-recovery/Wed, 11 Oct 2023 00:00:00 -0600https://support.tools/post/rke2-standalone-disaster-recovery/<h2 id="rke2-standalone---disaster-recovery-guide">RKE2 Standalone - Disaster Recovery Guide</h2> <h3 id="introduction">Introduction</h3> <p>The purpose of this guide is to provide comprehensive instructions for recovering a standalone RKE2 cluster that Rancher does not manage. RKE2, a lightweight Kubernetes distribution, is a robust solution, but disasters can happen. In this guide, we&rsquo;ll address common scenarios like recovering after etcd quorum loss, restoring a cluster from a backup, and troubleshooting issues when nodes cannot join the cluster.</p> <p>Please note that this guide assumes a basic familiarity with RKE2 and Kubernetes concepts. If you encounter any challenges or have questions, please ask for assistance.</p>https://support.tools/post/rancher-deep-dive/Mon, 18 Sep 2023 00:00:00 -0600https://support.tools/post/rancher-deep-dive/<p>In this blog post, we will explore Rancher, a powerful platform designed to simplify the management of Kubernetes clusters at scale. We will delve into Rancher&rsquo;s architecture, covering controllers, agents, WebSockets, the Rancher API, and the use of Custom Resource Definitions (CRDs).</p>https://support.tools/post/rke2-cni-deepdive/Tue, 08 Nov 2022 01:30:00 -0500https://support.tools/post/rke2-cni-deepdive/<h2 id="table-of-contents">Table of Contents</h2> <ul> <li><a href="#rke2s-cni-options-canal-cilium-calico-and-multus">RKE2&rsquo;s CNI Options: Canal, Cilium, Calico, and Multus</a> <ul> <li><a href="#canal-the-best-of-flannel-and-calico">Canal: The Best of Flannel and Calico</a> <ul> <li><a href="#canal-key-features">Canal Key Features</a></li> </ul> </li> <li><a href="#cilium-advanced-security-with-ebpf">Cilium: Advanced Security with eBPF</a> <ul> <li><a href="#cilium-key-features">Cilium Key Features</a></li> </ul> </li> <li><a href="#calico-fine-grained-network-policies">Calico: Fine-Grained Network Policies</a> <ul> <li><a href="#calico-key-features">Calico Key Features</a></li> </ul> </li> <li><a href="#multus-the-multi-network-swiss-army-knife">Multus: The Multi-Network Swiss Army Knife</a> <ul> <li><a href="#multus-key-features">Multus Key Features</a></li> </ul> </li> </ul> </li> <li><a href="#conclusion">Conclusion</a></li> </ul>https://support.tools/post/rke2-deepdive/Mon, 24 Oct 2022 21:01:00 -0500https://support.tools/post/rke2-deepdive/https://support.tools/post/rancher-monitoring-operator/Thu, 29 Sep 2022 00:51:00 -0500https://support.tools/post/rancher-monitoring-operator/<p>Error:</p> <div class="code-block" data-copy-label="Copy code, block 1"> <div class="code-block-header" data-pagefind-ignore></div> <pre tabindex="0"><code>Error: Internal error occurred: failed calling webhook &#34;prometheusrulemutate.monitoring.coreos.com&#34;: failed to call webhook: Post &#34;https://rancher-monitoring-operator.cattle-monitoring-system.svc:443/admission-prometheusrules/mutate?timeout=10s&#34;: service &#34;rancher-monitoring-operator&#34; not found</code></pre> </div> <p>This error can be caused by a number of different issues. But first we need to understand what this error means.</p> <p>The TLDR is that the webhook is not available. The webhook is a service that is used to validate and mutate resources. In this case, the webhook is used to validate and mutate PrometheusRule resources.</p>https://support.tools/training/rancher/rancher-agents/Fri, 13 May 2022 04:04:00 +0000https://support.tools/training/rancher/rancher-agents/<h1 id="what-do-the-ranchercattle-agents-do">What do the Rancher/cattle agents do?</h1> <p>The Rancher agents are how Rancher communicates with downstream clusters. It&rsquo;s important to understand that Rancher does not create outbound connections to downstream clusters for gaining access to the Kubernetes API endpoint. This includes Rancher deployed clusters and imported clusters.</p> <p><img src="../images/rancher-architecture-rancher-api-server.svg" alt="rancher-architecture"></p> <p>There are two different agent resources deployed on Rancher managed clusters:</p> <ul> <li>cattle-cluster-agent</li> <li>cattle-node-agent</li> </ul> <h2 id="cattle-cluster-agents">cattle-cluster-agents</h2> <p>In Rancher v2.5, the cattle-cluster-agent Deployment has a single replica; in Rancher v2.6, it runs as at least two replicas for HA. When this pod starts, it creates a WebSocket connection to Ranchers API.</p>https://support.tools/training/rancher/rancher-certs/Fri, 29 Apr 2022 02:03:30 -0500https://support.tools/training/rancher/rancher-certs/<h2 id="what-are-ssl-certs">What are SSL Certs?</h2> <p>Generally referred to as a cert or certificate, an SSL certificate authenticates a website&rsquo;s identity and enables an encrypted connection. A web browser and a server create an encrypted link using Secure Sockets Layer or SSL.</p> <p>In short: SSL ensures the data exchanged between two systems is kept private. When you see a padlock icon next to the URL in the address bar, SSL protects the connection between you and the website you are visiting.</p>https://support.tools/post/how-to-change-rancher-2.x-server-cert/Mon, 01 Mar 2021 11:10:00 +0000https://support.tools/post/how-to-change-rancher-2.x-server-cert/<p>It&rsquo;s often necessary to migrate from a self-signed or LetsEncrypt certificate to an externally created certificate like DigiCert or Comodo in Rancher v2.x.</p>https://support.tools/post/how-to-resolve-a-service-stuck-in-removing/Sun, 28 Feb 2021 23:26:00 +0000https://support.tools/post/how-to-resolve-a-service-stuck-in-removing/<p>In Rancher v1.6, sometimes a service can be stuck in a <code>removing</code> state.</p> <p><img src="https://support.tools/images/how-to-resolve-a-service-stuck-in-removing/service-stuck-removing_small.jpg" alt=""></p> <p>All containers of this service were already deleted in the user interface. I verified this on the Docker hosts using &ldquo;docker ps -a,&rdquo; and yes, all container instances were correctly removed. But the service in Rancher was still stuck in removing.</p> <p>Furthermore, in Admin -&gt; Processes the service.remove processes (which seem to because of being stuck in that service removing in progress) never disappeared and were re-started every 2 minutes:</p> <p><img src="https://support.tools/images/how-to-resolve-a-service-stuck-in-removing/rancher-service-remove-running_small.jpg" alt=""> <img src="https://support.tools/images/how-to-resolve-a-service-stuck-in-removing/rancher-service-remove-restarted_small.jpg" alt=""></p> <p>Although I&rsquo;m not sure what caused this, the reason <em>might</em> be several actions happening on that particular service almost at the same time:</p> <p><img src="https://support.tools/images/how-to-resolve-a-service-stuck-in-removing/rancher-audit-logs_small.jpg" alt=""></p>https://support.tools/post/how-to-change-etcd-ciphers/Sun, 28 Feb 2021 23:22:00 +0000https://support.tools/post/how-to-change-etcd-ciphers/<p>This article will walk Rancher administrators through hardening the cluster communication between etcd nodes. We&rsquo;ll go over configuring etcd to use specific ciphers which enable stronger encryption for securing intra-cluster etcd traffic.</p> <p>The cipher suites defined in the example could trade off speed for stronger encryption. Consider the level of ciphers in use and how they could impact the performance of an etcd cluster. Testing should be done to factor the spec of your hosts (cpu, memory, disk, network, etc&hellip;) and the typical types of interacting with kubernetes as well as the amount of resources under management within the k8s cluster.</p>https://support.tools/post/how-to-add-additional-scrape-configs-to-a-rancher-cluster-or-project-monitoring-prometheus/Sun, 28 Feb 2021 23:11:00 +0000https://support.tools/post/how-to-add-additional-scrape-configs-to-a-rancher-cluster-or-project-monitoring-prometheus/<p><a href="https://rancher.com/docs/rancher/v2.x/en/cluster-admin/tools/monitoring/#monitoring-scope">The Rancher cluster and project monitoring tools</a>, allow you to monitor cluster components and nodes, as well as workloads and <a href="https://rancher.com/docs/rancher/v2.x/en/project-admin/tools/monitoring/#project-metrics">custom metrics from any HTTP or TCP/UDP metrics endpoint</a> that these workloads expose.</p> <p>This article will detail how to manually define additional scrape configs for either the cluster or project monitoring Prometheus instance, where you want to scrape other metrics.</p> <p>Whether to define the additional scrape config at the cluster or project level would depend on the desired scope for the metrics and possible alerts. If you wish to scope the metrics scraped and likely alerts configured for these metrics, you could configure the additional scrape config at the project monitoring level to a project. If you wish to scope the metrics at the cluster level, only those with cluster-admin access could see the metrics or configure alerts. You could configure the additional scrape config at the cluster monitoring level.</p>https://support.tools/post/how-to-increase-the-log-level-of-kubernetes-components-in-an-rke-cli-or-rancher-provisioned-kubernetes-cluster/Sun, 28 Feb 2021 22:53:00 +0000https://support.tools/post/how-to-increase-the-log-level-of-kubernetes-components-in-an-rke-cli-or-rancher-provisioned-kubernetes-cluster/<p>When troubleshooting an issue with an RKE CLI or Rancher provisioned Kubernetes cluster, it may help to increase the verbosity of logging on one or more of the Kubernetes components above the default level. This article details the process of increasing logging on both those components that use the Kubernetes hyperkube image (kubelet, kube-apiserver, kube-controller-manager, kube-scheduler, kube-proxy) as well as the etcd component.</p>https://support.tools/post/how-to-block-useragents-ingress-nginx/Sun, 28 Feb 2021 16:49:00 +0000https://support.tools/post/how-to-block-useragents-ingress-nginx/<p>At times it&rsquo;s necessary to block specific user agents from connecting to workloads within your cluster. Whether it&rsquo;s bad actors or for compliance reasons, we&rsquo;ll go through how to get it done with Rancher/RKE created clusters.</p>https://support.tools/post/how-to-enable-envoy-access-logging/Sun, 28 Feb 2021 16:26:00 +0000https://support.tools/post/how-to-enable-envoy-access-logging/<p>This article details how to enable <a href="https://istio.io/v1.5/docs/tasks/observability/logs/access-log/">Envoy&rsquo;s access logging</a>, for <a href="https://rancher.com/docs/rancher/v2.x/en/istio/v2.3.x-v2.4.x/">Rancher deployed Istio</a>, in Rancher v2.3+</p>https://support.tools/post/how-to-retrieve-kubeconfig-from-custom-cluster/Sun, 28 Feb 2021 16:26:00 +0000https://support.tools/post/how-to-retrieve-kubeconfig-from-custom-cluster/<p>During a Rancher outage or other disaster event, you may lose access to a downstream cluster via Rancher and be unable to manage your applications. This process allows to bypass Rancher and connects directly to the downstream cluster.</p>https://support.tools/post/how-to-change-rancher-2-x-server-url/Wed, 17 Feb 2021 19:21:00 +0000https://support.tools/post/how-to-change-rancher-2-x-server-url/<p>This article details how to change the server URL for the Rancher v2.x cluster.</p>https://support.tools/post/how-can-i-enable-debugging-for-alertmanager/Sat, 24 Oct 2020 07:00:00 +0000https://support.tools/post/how-can-i-enable-debugging-for-alertmanager/<p>This article details how to enable debug level logging on the Alertmanager instance in a Rancher v2.x managed Kubernetes cluster, which may assist when troubleshooting <a href="https://rancher.com/docs/rancher/v2.x/en/cluster-admin/tools/alerts/">cluster</a> or <a href="https://rancher.com/docs/rancher/v2.x/en/project-admin/tools/alerts/">project alerting</a>.</p>https://support.tools/post/how-to-check-default-coredns-configmap-of-rke-k8s-version/Fri, 23 Oct 2020 14:28:00 +0000https://support.tools/post/how-to-check-default-coredns-configmap-of-rke-k8s-version/<p>This article details how to enable TLS 1.1 on the ingress-nginx controller in Rancher Kubernetes Engine (RKE) CLI or Rancher v2.x provisioned Kubernetes clusters.</p>https://support.tools/post/how-to-enable-legacy-tls-versions/Fri, 23 Oct 2020 14:28:00 +0000https://support.tools/post/how-to-enable-legacy-tls-versions/<p>This article details how to enable TLS 1.1 on the ingress-nginx controller in Rancher Kubernetes Engine (RKE) CLI or Rancher v2.x provisioned Kubernetes clusters.</p>https://support.tools/post/how-to-use-external-tls-termination-with-aws/Fri, 23 Oct 2020 14:28:00 +0000https://support.tools/post/how-to-use-external-tls-termination-with-aws/<p>This document covers setting up Rancher using an AWS SSL certificate and an ALB (Application Load Balancer).</p>https://support.tools/post/docker-bridge-for-rancher-clusters/Wed, 20 Nov 2019 13:18:00 +0000https://support.tools/post/docker-bridge-for-rancher-clusters/<p>The docker0 bridge network has a default IP range of 172.17.0.0/16 (with an additional docker-sys bridge for system-docker using 172.18.0.0/16 by default on RancherOS). These ranges will be routed to these interfaces, per the below example of the route output. If the range(s) overlap with the internal IP space usage in your own network, the host will not be able to route packets to other hosts in your network that lie within these ranges. As a result you may wish to change the bridge range(s) to enable successful routing to hosts within these.</p>https://support.tools/post/rancher-nginx-ssl-passthrough/Wed, 20 Nov 2019 00:03:00 +0000https://support.tools/post/rancher-nginx-ssl-passthrough/<p>Recently while setting up Vault inside Rancher. I ran into an issue with the NGINX ingress terminating TLS and forwarding traffic unencrypted to Vault.</p>https://support.tools/post/persisting-rancheros-state/Tue, 19 Nov 2019 21:05:00 +0000https://support.tools/post/persisting-rancheros-state/<p><img src="https://support.tools/images/rancher/RancherOS-1088X725-678x381.png" alt=""></p> <p>Recently I set about installing RancherOS. This was just to have a look and see what use cases it might help with. I chose to install RancherOS to a VM. In my case, VMware workstation. A roadblock I hit was providing an SSH key to the cloud-config.yml file.</p> <p>The roadblock specifically, how can I send a file to a system I don’t have a password or SSH key for?</p> <p>This article covers, generating an SSH key, SSH access to live CD and installation to hard disk.</p>