Karpenter for EKS: A Comprehensive Guide to Efficient Kubernetes Autoscaling

Kubernetes autoscaling has traditionally been a balance between maintaining enough capacity for workloads and controlling cloud costs. Karpenter introduces a more efficient approach to this challenge, providing just-in-time compute resources that perfectly match your workload requirements. This guide explores Karpenter’s architecture, implementation strategies, and advanced configurations for optimal EKS cluster management.

What is Karpenter?

Karpenter is a flexible, high-performance Kubernetes cluster autoscaler designed specifically to optimize compute resource allocation. Unlike traditional autoscalers, Karpenter works directly with your cloud provider to provision the exact infrastructure needed for your workloads.

Key Differentiators from Traditional Autoscaling

Feature	Karpenter	Cluster Autoscaler + Node Groups
Scaling Trigger	Pod scheduling events	Node group utilization
Instance Selection	Just-in-time, diverse instance types	Pre-defined node groups
Scaling Speed	Seconds (direct EC2 API)	Minutes (via ASG)
Empty Node Handling	Automatic consolidation	Manual or limited cleanup
Infrastructure Representation	Kubernetes-native resources	Cloud provider resources
Instance Type Selection	Dynamic, workload-specific	Fixed per node group

How Karpenter Works: The Basic Flow

Pod Scheduling: A new pod is created but can’t be scheduled due to insufficient resources
Karpenter Detection: Karpenter observes the pending pod through its controller
Resource Evaluation: Karpenter analyzes the pod’s requirements (CPU, memory, GPU, etc.)
Instance Selection: Karpenter selects the optimal EC2 instance type based on requirements
Provisioning: Karpenter directly calls EC2 APIs to launch the instance(s)
Node Registration: The new node joins the cluster through the Bootstrap process
Pod Placement: The previously pending pod is scheduled onto the new node
Deprovisioning: When nodes become empty or underutilized, Karpenter removes them

Karpenter Architecture Components

Karpenter introduces two custom resource definitions (CRDs) that control its behavior:

1. NodePool

A NodePool defines the blueprint for node creation, including constraints, requirements, and lifecycle policies. It answers questions like:

What types of nodes can be created?
What workloads can run on these nodes?
How many nodes can be provisioned?
When should nodes be terminated?

2. EC2NodeClass

The EC2NodeClass provides AWS-specific configuration for node provisioning, including:

AMI selection
Instance types
Security groups
Subnets
IAM roles
Block device mappings

Let’s explore each component in detail.

NodePool: Defining Node Creation Rules

The NodePool is a Kubernetes resource that defines when and how Karpenter should provision nodes.

Basic NodePool Example

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: default
spec:
  template:
    spec:
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["on-demand"]
        - key: "kubernetes.io/arch"
          operator: In
          values: ["amd64"]
        - key: "node.kubernetes.io/instance-type"
          operator: In
          values: ["m5.large", "m5.xlarge", "m5.2xlarge"]
      nodeClassRef:
        apiVersion: karpenter.k8s.aws/v1
        kind: EC2NodeClass
        name: default
  limits:
    cpu: "1000"
    memory: 1000Gi
  disruption:
    consolidationPolicy: WhenEmpty
    consolidateAfter: 30s

Key NodePool Fields Explained

1. Requirements

Requirements define constraints for node selection. They function similarly to node selectors in Kubernetes, using the well-known label key format.

Common requirement keys include:

requirements:
  # Capacity type (on-demand vs spot)
  - key: "karpenter.sh/capacity-type"
    operator: In
    values: ["on-demand", "spot"]
  
  # Instance types
  - key: "node.kubernetes.io/instance-type"
    operator: In
    values: ["m5.large", "c5.large", "r5.large"]
  
  # Availability Zone
  - key: "topology.kubernetes.io/zone"
    operator: In
    values: ["us-east-1a", "us-east-1b", "us-east-1c"]
  
  # CPU architecture
  - key: "kubernetes.io/arch"
    operator: In
    values: ["amd64", "arm64"]

Karpenter also supports the NotIn operator and extended requirements like:

# Exclude specific instance types
- key: "node.kubernetes.io/instance-type"
  operator: NotIn
  values: ["t3.nano", "t3.micro"]

# CPU requirements
- key: "karpenter.k8s.aws/instance-cpu"
  operator: In
  values: ["4", "8", "16"]

# Memory requirements
- key: "karpenter.k8s.aws/instance-memory"
  operator: In
  values: ["8192", "16384", "32768"]

2. Limits

Limits specify the maximum amount of resources Karpenter can provision for this NodePool:

limits:
  cpu: "1000"      # 1000 vCPUs total
  memory: 4000Gi   # 4 TB of RAM total
  pods: 1000       # 1000 pods maximum

These limits help prevent runaway costs and enforce resource governance.

3. Disruption Settings

Disruption settings control how Karpenter consolidates and terminates nodes:

disruption:
  # When to consolidate nodes
  consolidationPolicy: WhenEmpty  # or WhenUnderutilized or WhenEmptyOrUnderutilized
  
  # How long to wait before consolidating
  consolidateAfter: 30s
  
  # Optional: Expire nodes after this duration regardless of utilization
  expireAfter: 720h  # 30 days

The consolidationPolicy values determine when Karpenter can remove nodes:

WhenEmpty: Only remove nodes with zero non-daemonset pods
WhenUnderutilized: Remove nodes even if they have pods when those pods can fit elsewhere
WhenEmptyOrUnderutilized: Either condition can trigger node removal

4. TTL-Based Expirations

To maintain a fresh fleet and allow for node rotation, you can configure nodes to expire after a certain time:

spec:
  template:
    spec:
      expireAfter: 168h  # Expire nodes after 7 days

This helps ensure security patches are applied and prevents node configuration drift.

EC2NodeClass: Configuring AWS-Specific Settings

The EC2NodeClass is an AWS-specific resource that defines how EC2 instances should be configured when provisioned by Karpenter.

Basic EC2NodeClass Example

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: default
spec:
  # AMI selection
  amiFamily: AL2023
  
  # Instance specific details  
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: ${CLUSTER_NAME}
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: ${CLUSTER_NAME}
  
  # IAM instance profile
  role: "KarpenterNodeRole-${CLUSTER_NAME}"
  
  # Block device configuration  
  blockDeviceMappings:
    - deviceName: /dev/xvda
      ebs:
        volumeSize: 100Gi
        volumeType: gp3
        iops: 3000
        throughput: 125
        encrypted: true
  
  # Optional metadata configurations
  metadataOptions:
    httpEndpoint: enabled
    httpProtocolIPv6: disabled
    httpPutResponseHopLimit: 2
    httpTokens: required
  
  # Optional detailed AMI specification
  detailedMonitoring: true
  
  # Tags applied to EC2 instances
  tags:
    managed-by: karpenter

Key EC2NodeClass Fields Explained

1. AMI Selection

Karpenter supports several AMI selection methods:

# Using AMI family (simplest approach)
amiFamily: AL2023  # or Bottlerocket, Ubuntu, Custom

# Using AMI selectors for more control
amiSelectorTerms:
  - tags:
      Name: "eks-al2023-*"
      kubernetes.io/cluster/${CLUSTER_NAME}: owned

Supported AMI families include:

AL2 (Amazon Linux 2)
AL2023 (Amazon Linux 2023)
Bottlerocket
Ubuntu
Custom (requires additional configuration)

2. Network Configuration

Subnets and security groups define the network environment:

# Select subnets by tag
subnetSelectorTerms:
  - tags:
      karpenter.sh/discovery: ${CLUSTER_NAME}
      Type: private

# Select security groups by tag
securityGroupSelectorTerms:
  - tags:
      karpenter.sh/discovery: ${CLUSTER_NAME}
      
# Or directly by ID
securityGroupSelectorTerms:
  - ids:
      - sg-0123456789abcdef0

3. Instance Configuration

Additional EC2 configurations include:

# Block device mappings for storage
blockDeviceMappings:
  - deviceName: /dev/xvda
    ebs:
      volumeSize: 100Gi
      volumeType: gp3
      deleteOnTermination: true
      encrypted: true
      
# User data (base64 encoded)
userData: |
    MIME-Version: 1.0
    Content-Type: multipart/mixed; boundary="BOUNDARY"
    
    --BOUNDARY
    Content-Type: text/x-shellscript; charset="us-ascii"
    
    #!/bin/bash
    echo "Hello World" > /etc/myapp/config.yaml
    
    --BOUNDARY--

# EC2 tags
tags:
  Environment: production
  Team: platform

Advanced Karpenter Configuration Patterns

Beyond the basics, Karpenter offers advanced configurations for specialized workloads.

1. Multi-NodePool Strategy for Workload Segregation

Using multiple NodePools allows you to segregate workloads based on requirements:

# General purpose workloads
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: general-purpose
spec:
  template:
    spec:
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["on-demand"]
        - key: "node.kubernetes.io/instance-type"
          operator: In
          values: ["m5.large", "m5.xlarge"]
      nodeClassRef:
        name: general-purpose
---
# Compute intensive workloads
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: compute-optimized
spec:
  template:
    spec:
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["on-demand"]
        - key: "node.kubernetes.io/instance-type"
          operator: In
          values: ["c5.large", "c5.xlarge", "c5.2xlarge"]
      nodeClassRef:
        name: compute-optimized
      taints:
        - key: workload-type
          value: compute
          effect: NoSchedule

Workloads can then target specific NodePools using node selectors and tolerations:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: compute-intensive-app
spec:
  template:
    spec:
      nodeSelector:
        workload-type: compute
      tolerations:
      - key: workload-type
        value: compute
        effect: NoSchedule

2. Cost Optimization with Spot Instances

Karpenter excels at managing Spot instances efficiently:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: spot-pool
spec:
  template:
    spec:
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["spot"]
        - key: "node.kubernetes.io/instance-type"
          operator: In
          values: [
            "m5.large", "m5.xlarge", "m5.2xlarge",
            "c5.large", "c5.xlarge", "c5.2xlarge",
            "r5.large", "r5.xlarge", "r5.2xlarge"
          ]
      taints:
        - key: spot
          value: "true"
          effect: NoSchedule
  disruption:
    consolidationPolicy: WhenUnderutilized
    consolidateAfter: 30s

Applications can use pod disruption budgets and tolerations to handle Spot interruptions:

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: spot-app-pdb
spec:
  minAvailable: 1
  selector:
    matchLabels:
      app: spot-app
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: spot-app
spec:
  template:
    spec:
      tolerations:
      - key: spot
        value: "true"
        effect: NoSchedule
      terminationGracePeriodSeconds: 60

3. GPU Workload Support

For machine learning and GPU workloads:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: gpu-pool
spec:
  template:
    spec:
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["on-demand"]
        - key: "node.kubernetes.io/instance-type"
          operator: In
          values: ["g4dn.xlarge", "g5.xlarge", "p3.2xlarge"]
        - key: "nvidia.com/gpu"
          operator: Exists
      taints:
        - key: nvidia.com/gpu
          value: "true"
          effect: NoSchedule

Practical Implementation: Step-by-Step

Let’s walk through a complete implementation of Karpenter for a production EKS cluster.

1. Prerequisites

Before deploying Karpenter, ensure you have:

An EKS cluster (1.23+)
IAM permissions for Karpenter controller
Worker node security groups and subnets

2. Install Karpenter with Helm

# Add the Karpenter Helm repository
helm repo add karpenter https://charts.karpenter.sh
helm repo update

# Install Karpenter
helm upgrade --install karpenter karpenter/karpenter \
  --namespace karpenter \
  --create-namespace \
  --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"=${KARPENTER_IAM_ROLE_ARN} \
  --set settings.aws.clusterName=${CLUSTER_NAME} \
  --set settings.aws.defaultInstanceProfile=KarpenterNodeInstanceProfile-${CLUSTER_NAME} \
  --set settings.aws.interruptionQueueName=${CLUSTER_NAME} \
  --set controller.resources.requests.cpu=1 \
  --set controller.resources.requests.memory=1Gi \
  --set controller.resources.limits.cpu=1 \
  --set controller.resources.limits.memory=1Gi \
  --version 0.32.3

3. Create Core NodePool and EC2NodeClass

# EC2NodeClass for general workloads
apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: default
spec:
  amiFamily: AL2023
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: ${CLUSTER_NAME}
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: ${CLUSTER_NAME}
  role: "KarpenterNodeRole-${CLUSTER_NAME}"
  blockDeviceMappings:
    - deviceName: /dev/xvda
      ebs:
        volumeSize: 100Gi
        volumeType: gp3
        encrypted: true
  tags:
    karpenter.sh/discovery: ${CLUSTER_NAME}
---
# Default NodePool
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: default
spec:
  template:
    spec:
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["on-demand"]
        - key: "kubernetes.io/arch"
          operator: In
          values: ["amd64"]
        - key: "node.kubernetes.io/instance-type"
          operator: In
          values: [
            "m5.large", "m5.xlarge", "m5.2xlarge",
            "c5.large", "c5.xlarge", "c5.2xlarge",
            "r5.large", "r5.xlarge", "r5.2xlarge"
          ]
      nodeClassRef:
        apiVersion: karpenter.k8s.aws/v1
        kind: EC2NodeClass
        name: default
  limits:
    cpu: "100"
    memory: 400Gi
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 30s

4. Testing Karpenter

Deploy a test workload to verify Karpenter’s functionality:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: inflate
spec:
  replicas: 0
  selector:
    matchLabels:
      app: inflate
  template:
    metadata:
      labels:
        app: inflate
    spec:
      containers:
      - name: inflate
        image: public.ecr.aws/eks-distro/kubernetes/pause:3.7
        resources:
          requests:
            cpu: 1
            memory: 1Gi

Scale the deployment to trigger node provisioning:

kubectl scale deployment inflate --replicas=10

Observe Karpenter’s actions:

kubectl logs -f -n karpenter -l app.kubernetes.io/name=karpenter -c controller

Monitoring and Observability

Karpenter exposes Prometheus metrics that provide insights into its operations:

# Install Prometheus and Grafana using Helm
helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm install prometheus prometheus-community/kube-prometheus-stack \
  --namespace monitoring \
  --create-namespace

Key metrics to monitor include:

karpenter_nodes_allocatable: Allocatable resources on Karpenter-provisioned nodes
karpenter_nodes_total: Total number of nodes managed by Karpenter
karpenter_provisioner_scheduling_duration_seconds: Time taken to provision nodes
karpenter_deprovisioning_actions: Count of deprovisioning actions by result

Best Practices and Performance Tuning

1. Diverse Instance Type Selection

Always provide multiple instance type options to increase availability and optimize costs:

requirements:
  - key: "node.kubernetes.io/instance-type"
    operator: In
    values: [
      "m5.large", "m5.xlarge", "m5.2xlarge", 
      "m5a.large", "m5a.xlarge", "m5a.2xlarge",
      "m6i.large", "m6i.xlarge", "m6i.2xlarge",
      "m6a.large", "m6a.xlarge", "m6a.2xlarge"
    ]

2. Optimize Disruption Settings

Balance cost savings with stability by tuning consolidation settings:

disruption:
  consolidationPolicy: WhenEmptyOrUnderutilized
  consolidateAfter: 300s  # 5 minutes (higher in production)

3. Resource Request Accuracy

Ensure pod resource requests accurately reflect actual usage to allow Karpenter to make optimal decisions.

4. Use Startup Taints for Node Initialization

Prevent pods from scheduling on nodes before they’re fully ready:

template:
  spec:
    startupTaints:
      - key: node.kubernetes.io/not-ready
        effect: NoSchedule

5. Node Termination Handling

Implement proper termination handling for graceful node removal:

maxPodGracePeriod: 300

Common Pitfalls and Troubleshooting

Symptom: Karpenter Not Provisioning Nodes

Potential Causes:

Insufficient IAM permissions
NodePool constraints too restrictive
Resource limits reached

Solution:

Check Karpenter logs:

kubectl logs -n karpenter -l app.kubernetes.io/name=karpenter -c controller

Verify that the pending pods match the NodePool requirements

Check if resource limits are reached:

kubectl get nodepool default -o jsonpath='{.status.resources}'

Symptom: Node Provisioning Errors

Potential Causes:

AMI not found
Security group or subnet issues
Instance type availability

Solution:

Verify AMI availability:

aws ec2 describe-images --image-ids ami-12345678

Check EC2 service quotas
Expand instance type selection

Symptom: Nodes Not Consolidating

Potential Causes:

Restrictive disruption settings
PodDisruptionBudgets blocking evictions
Pods with local storage

Solution:

Check disruption settings
Examine PDBs:
```
kubectl get pdb -A
```
Look for pods using local storage or hostPath volumes

Comparing Karpenter to Cluster Autoscaler: When to Use Which

Consideration	Karpenter	Cluster Autoscaler
Maturity	Newer (2021+)	Mature (2016+)
Speed	Fast (seconds)	Slower (minutes)
Flexibility	High (diverse instance types)	Lower (fixed node groups)
Integration	Direct EC2 API	ASG-based
Management Overhead	Lower	Higher
Infrastructure as Code	Kubernetes-native	ASG templates
Cost Optimization	Better	Good
Bin-packing	Excellent	Limited

Use Karpenter when:

Speed of scaling is critical
Cost optimization is a priority
You want to minimize infrastructure management
You need workload-aware instance type selection

Use Cluster Autoscaler when:

You have existing ASG-based infrastructure
You need the mature, battle-tested approach
You have complex integrations with ASGs
You need to maintain compatibility with existing tools

Advanced Scenario: Hybrid Deployments with Karpenter and Managed Node Groups

You can use both Karpenter and managed node groups in the same cluster:

# Core system nodes using managed node groups
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: system
spec:
  template:
    spec:
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["on-demand"]
        - key: "node.kubernetes.io/instance-type"
          operator: In
          values: ["m5.xlarge"]
      nodeClassRef:
        name: system
      taints:
        - key: node-type
          value: system
          effect: NoSchedule
  disruption:
    consolidationPolicy: WhenEmpty
    expireAfter: 720h  # 30 days
---
# Workload nodes using Karpenter
apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: workload
spec:
  template:
    spec:
      requirements:
        - key: "karpenter.sh/capacity-type"
          operator: In
          values: ["spot", "on-demand"]
        - key: "node.kubernetes.io/instance-type"
          operator: In
          values: ["m5.large", "m5.xlarge", "c5.large", "c5.xlarge", "r5.large", "r5.xlarge"]
      nodeClassRef:
        name: workload
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 30s

Deploy system components to the system node pool:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: system-component
spec:
  template:
    spec:
      nodeSelector:
        node-type: system
      tolerations:
      - key: node-type
        value: system
        effect: NoSchedule

Future of Karpenter

The Karpenter project continues to evolve with new features:

Drift Detection: Automatically replace nodes that drift from desired state
Weight-Based Instance Selection: Prefer specific instance types while maintaining flexibility
Enhanced Spot Integration: Better handling of spot interruptions and rebalancing recommendations
Cross-Provider Support: Expanding beyond AWS to other cloud providers

Conclusion

Karpenter represents a significant evolution in Kubernetes autoscaling, providing faster provisioning, better cost optimization, and simplified management. By understanding and implementing the concepts covered in this guide, you can build a highly efficient and responsive EKS infrastructure that scales precisely with your application needs.

The core advantages of Karpenter - just-in-time provisioning, workload-aware instance selection, and automatic node consolidation - enable a new approach to cluster scaling that minimizes both cost and operational overhead.

For most modern EKS deployments, Karpenter offers a compelling solution that bridges the gap between infrastructure flexibility and application-centric orchestration.

Karpenter for EKS: A Comprehensive Guide to Efficient Kubernetes Autoscaling

What is Karpenter?

Key Differentiators from Traditional Autoscaling

How Karpenter Works: The Basic Flow

Karpenter Architecture Components

1. NodePool

2. EC2NodeClass

NodePool: Defining Node Creation Rules

Basic NodePool Example

Key NodePool Fields Explained

1. Requirements

2. Limits

3. Disruption Settings

4. TTL-Based Expirations

EC2NodeClass: Configuring AWS-Specific Settings

Basic EC2NodeClass Example

Key EC2NodeClass Fields Explained

1. AMI Selection

2. Network Configuration

3. Instance Configuration

Advanced Karpenter Configuration Patterns

1. Multi-NodePool Strategy for Workload Segregation

2. Cost Optimization with Spot Instances

3. GPU Workload Support

Practical Implementation: Step-by-Step

1. Prerequisites

2. Install Karpenter with Helm

3. Create Core NodePool and EC2NodeClass

4. Testing Karpenter

Monitoring and Observability

Best Practices and Performance Tuning

1. Diverse Instance Type Selection

2. Optimize Disruption Settings

3. Resource Request Accuracy

4. Use Startup Taints for Node Initialization

5. Node Termination Handling

Common Pitfalls and Troubleshooting

Symptom: Karpenter Not Provisioning Nodes

Symptom: Node Provisioning Errors

Symptom: Nodes Not Consolidating

Comparing Karpenter to Cluster Autoscaler: When to Use Which

Advanced Scenario: Hybrid Deployments with Karpenter and Managed Node Groups

Future of Karpenter

Conclusion

Additional Resources