Jaeger is the industry-standard distributed tracing system for microservices, but achieving high performance at enterprise scale requires careful optimization. This guide covers production deployment strategies, storage backend optimization, adaptive sampling, and query performance tuning for processing millions of spans per second.

Jaeger Performance Optimization

Executive Summary

Jaeger provides distributed tracing for microservices architectures, helping teams troubleshoot complex distributed systems. At enterprise scale, optimizing Jaeger for performance requires proper architecture, storage backend configuration, intelligent sampling, and query optimization. This guide demonstrates production-grade deployment handling millions of spans per second.

Jaeger Architecture

Applications → Jaeger Agent → Jaeger Collector → Storage (ES/Cassandra) → Jaeger Query

High-Performance Collector

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jaeger-collector
  namespace: tracing
spec:
  replicas: 10
  selector:
    matchLabels:
      app: jaeger-collector
  template:
    metadata:
      labels:
        app: jaeger-collector
    spec:
      containers:
      - name: collector
        image: jaegertracing/jaeger-collector:1.51
        args:
          - --es.server-urls=http://elasticsearch:9200
          - --es.num-shards=10
          - --es.num-replicas=2
          - --collector.queue-size=10000
          - --collector.num-workers=100
          - --es.bulk.size=10000000
          - --es.bulk.workers=10
          - --es.bulk.flush-interval=1s
        ports:
          - containerPort: 14250
            name: grpc
          - containerPort: 14268
            name: http
        resources:
          requests:
            memory: "2Gi"
            cpu: "1000m"
          limits:
            memory: "4Gi"
            cpu: "2000m"

Adaptive Sampling

# Adaptive sampling configuration
sampling:
  strategies:
    - service: critical-service
      type: probabilistic
      param: 1.0  # 100% sampling
    - service: standard-service
      type: probabilistic
      param: 0.1  # 10% sampling
    - service: high-volume-service
      type: ratelimiting
      param: 1000  # max 1000 traces/sec

Storage Optimization

Elasticsearch Configuration

# Elasticsearch index template for Jaeger
{
  "template": "jaeger-span-*",
  "settings": {
    "number_of_shards": 10,
    "number_of_replicas": 2,
    "refresh_interval": "5s",
    "index.codec": "best_compression"
  },
  "mappings": {
    "properties": {
      "traceID": {"type": "keyword"},
      "spanID": {"type": "keyword"},
      "operationName": {"type": "keyword"},
      "startTime": {"type": "long"},
      "duration": {"type": "long"}
    }
  }
}

Query Performance

# Key Jaeger metrics to monitor
jaeger_collector_spans_received_total
jaeger_collector_spans_saved_by_svc_total
jaeger_collector_queue_length
jaeger_query_requests_total

Best Practices

  1. Use adaptive sampling to control data volume
  2. Optimize storage backend for write performance
  3. Implement proper indexing strategies
  4. Configure bulk operations for efficiency
  5. Monitor collector queue depth
  6. Use SSD storage for backends
  7. Implement proper retention policies
  8. Regular index optimization
  9. Query result caching
  10. Proper resource allocation

Conclusion

Optimizing Jaeger for enterprise-scale distributed tracing requires careful attention to sampling strategies, storage backend configuration, and query performance tuning to handle millions of spans while maintaining query responsiveness.