Container Image Vulnerability Scanning with Trivy: Enterprise Production Guide
Container image security represents a critical component of cloud-native security posture, with vulnerabilities in base images and dependencies posing significant risks to production workloads. Trivy, an open-source vulnerability scanner developed by Aqua Security, provides comprehensive scanning capabilities for containers, infrastructure as code, and configuration files. This guide explores enterprise-grade implementation patterns for Trivy across the complete software development lifecycle.
Container Image Vulnerability Scanning with Trivy: Enterprise Production Guide
Understanding Container Vulnerability Landscape
Container images inherit vulnerabilities from multiple sources:
Vulnerability Sources
Base Images: Operating system packages contain known CVEs
- Ubuntu, Alpine, Red Hat base images ship with OS packages
- Package managers (apt, apk, yum) may install vulnerable versions
- Legacy base images lack security patches
- Minimal images reduce attack surface but require maintenance
Application Dependencies: Language-specific libraries introduce vulnerabilities
- Python packages (pip, Poetry)
- Node.js modules (npm, yarn)
- Java libraries (Maven, Gradle)
- Go modules
- Ruby gems
Configuration Issues: Misconfigurations create security risks
- Running containers as root
- Exposed sensitive ports
- Missing security contexts
- Insecure environment variables
Trivy Architecture and Components
Core Scanning Engine
Trivy provides multiple scanning modes optimized for different use cases:
#!/bin/bash
# Trivy Installation and Basic Usage
set -euo pipefail
# Colors for output
RED='\033[0;31m'
GREEN='\033[0;32m'
YELLOW='\033[1;33m'
BLUE='\033[0;34m'
NC='\033[0m'
log() {
echo -e "${GREEN}[$(date +'%Y-%m-%d %H:%M:%S')]${NC} $*"
}
error() {
echo -e "${RED}[ERROR]${NC} $*" >&2
}
warn() {
echo -e "${YELLOW}[WARN]${NC} $*"
}
# Install Trivy
install_trivy() {
log "Installing Trivy..."
if command -v trivy &> /dev/null; then
local version=$(trivy --version | head -n1 | awk '{print $2}')
log "Trivy already installed: version ${version}"
return 0
fi
# Install from GitHub releases
local VERSION="0.48.0"
curl -sfL https://raw.githubusercontent.com/aquasecurity/trivy/main/contrib/install.sh | \
sh -s -- -b /usr/local/bin v${VERSION}
log "Trivy installed successfully"
trivy --version
}
# Basic image scanning
scan_image() {
local image="$1"
local severity="${2:-CRITICAL,HIGH}"
log "Scanning image: ${image}"
log "Severity filter: ${severity}"
trivy image \
--severity "${severity}" \
--format table \
--exit-code 1 \
"${image}"
}
# Scan with JSON output
scan_image_json() {
local image="$1"
local output_file="${2:-scan-results.json}"
log "Scanning image with JSON output: ${image}"
trivy image \
--format json \
--output "${output_file}" \
"${image}"
log "Results saved to: ${output_file}"
}
# Scan with SARIF output for GitHub Security
scan_image_sarif() {
local image="$1"
local output_file="${2:-scan-results.sarif}"
log "Scanning image with SARIF output: ${image}"
trivy image \
--format sarif \
--output "${output_file}" \
"${image}"
log "SARIF results saved to: ${output_file}"
}
# Scan filesystem
scan_filesystem() {
local path="${1:-.}"
local severity="${2:-CRITICAL,HIGH}"
log "Scanning filesystem: ${path}"
trivy fs \
--severity "${severity}" \
--format table \
--exit-code 1 \
"${path}"
}
# Scan Kubernetes manifests
scan_kubernetes() {
local manifest_path="${1:-.}"
log "Scanning Kubernetes manifests: ${manifest_path}"
trivy config \
--severity CRITICAL,HIGH,MEDIUM \
--format table \
--exit-code 1 \
"${manifest_path}"
}
# Scan with vulnerability database update
scan_with_db_update() {
local image="$1"
log "Updating vulnerability database..."
trivy image --download-db-only
log "Scanning image after DB update: ${image}"
trivy image \
--severity CRITICAL,HIGH \
--format table \
"${image}"
}
# Scan and ignore unfixed vulnerabilities
scan_ignore_unfixed() {
local image="$1"
log "Scanning image (ignoring unfixed vulnerabilities): ${image}"
trivy image \
--severity CRITICAL,HIGH \
--ignore-unfixed \
--format table \
"${image}"
}
# Generate HTML report
generate_html_report() {
local image="$1"
local output_file="${2:-scan-report.html}"
log "Generating HTML report for: ${image}"
trivy image \
--format template \
--template "@contrib/html.tpl" \
--output "${output_file}" \
"${image}"
log "HTML report saved to: ${output_file}"
}
# Scan with custom policy
scan_with_policy() {
local image="$1"
local policy_file="${2:-trivy-policy.rego}"
if [[ ! -f "${policy_file}" ]]; then
error "Policy file not found: ${policy_file}"
return 1
fi
log "Scanning with custom policy: ${policy_file}"
trivy image \
--config-policy "${policy_file}" \
--format table \
"${image}"
}
# Example usage
main() {
log "Trivy Container Security Scanner"
# Install Trivy if needed
install_trivy
# Example scans
# scan_image "nginx:latest"
# scan_image_json "nginx:latest" "nginx-scan.json"
# scan_filesystem "./application"
# scan_kubernetes "./k8s-manifests"
log "Scan operations completed"
}
# Run if executed directly
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
main "$@"
fi
Trivy Configuration File
# trivy.yaml - Comprehensive Configuration
# Place in project root or ~/.trivy/trivy.yaml
# General settings
cache:
backend: fs
dir: /tmp/trivy
# Database settings
db:
repository: ghcr.io/aquasecurity/trivy-db
skip-update: false
# Vulnerability settings
vulnerability:
type: os,library
ignore-unfixed: false
# Severity levels to report
severity:
- CRITICAL
- HIGH
- MEDIUM
# Output settings
format: table
output: ""
# Exit code settings
exit-code: 1
# Timeout settings
timeout: 5m0s
# Secret scanning
secret:
config: .trivy-secret.yaml
# License scanning
license:
full: false
# Ignore files
ignorefile: .trivyignore
# Custom headers (for private registries)
registry:
credentials:
- registry: registry.company.com
username: ${REGISTRY_USERNAME}
password: ${REGISTRY_PASSWORD}
Trivy Ignore File
# .trivyignore - Vulnerability Exceptions
# Format: CVE-ID or package-name
# Ignore specific CVEs with justification
# CVE-2023-1234 - Fixed in next release, workaround implemented
CVE-2023-1234
# Ignore by package
# pkg:golang/github.com/example/vulnerable-pkg
# Ignore with expiration date
CVE-2023-5678 exp:2025-12-31
# Ignore unfixable vulnerabilities in base image
# Will be addressed when base image updates
CVE-2023-9999
CI/CD Integration Patterns
GitLab CI Integration
# .gitlab-ci.yml - Trivy Integration
stages:
- build
- scan
- deploy
variables:
IMAGE_NAME: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
TRIVY_VERSION: "0.48.0"
TRIVY_CACHE_DIR: "$CI_PROJECT_DIR/.trivycache/"
# Build Docker image
build:
stage: build
image: docker:24.0
services:
- docker:24.0-dind
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
- docker build -t $IMAGE_NAME .
- docker push $IMAGE_NAME
only:
- main
- develop
- merge_requests
# Trivy vulnerability scan
trivy:vulnerability:
stage: scan
image: aquasec/trivy:$TRIVY_VERSION
cache:
paths:
- .trivycache/
variables:
TRIVY_CACHE_DIR: .trivycache/
before_script:
- trivy --version
script:
# Scan for vulnerabilities
- trivy image --exit-code 0 --severity LOW,MEDIUM $IMAGE_NAME
- trivy image --exit-code 1 --severity HIGH,CRITICAL $IMAGE_NAME
# Generate reports
- trivy image --format json --output trivy-report.json $IMAGE_NAME
- trivy image --format template --template "@contrib/gitlab.tpl" --output gl-container-scanning-report.json $IMAGE_NAME
artifacts:
reports:
container_scanning: gl-container-scanning-report.json
paths:
- trivy-report.json
expire_in: 30 days
dependencies:
- build
only:
- main
- develop
- merge_requests
# Trivy configuration scan
trivy:config:
stage: scan
image: aquasec/trivy:$TRIVY_VERSION
script:
# Scan Dockerfile
- trivy config --exit-code 1 --severity HIGH,CRITICAL Dockerfile
# Scan Kubernetes manifests
- trivy config --exit-code 1 --severity HIGH,CRITICAL k8s/
only:
- main
- develop
- merge_requests
# Trivy secret scan
trivy:secret:
stage: scan
image: aquasec/trivy:$TRIVY_VERSION
script:
- trivy fs --scanners secret --exit-code 1 .
allow_failure: false
only:
- main
- develop
- merge_requests
# Advanced scan with custom policy
trivy:policy:
stage: scan
image: aquasec/trivy:$TRIVY_VERSION
script:
- trivy image --config-policy .trivy-policy.rego --exit-code 1 $IMAGE_NAME
only:
- main
GitHub Actions Integration
# .github/workflows/security-scan.yml
name: Container Security Scan
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
schedule:
# Run daily at 2 AM UTC
- cron: '0 2 * * *'
env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
jobs:
build-and-scan:
runs-on: ubuntu-latest
permissions:
contents: read
packages: write
security-events: write
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to Container Registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Extract metadata
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
tags: |
type=ref,event=branch
type=ref,event=pr
type=sha,prefix={{branch}}-
- name: Build and push image
uses: docker/build-push-action@v5
with:
context: .
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
- name: Run Trivy vulnerability scanner
uses: aquasecurity/trivy-action@master
with:
image-ref: ${{ steps.meta.outputs.tags }}
format: 'sarif'
output: 'trivy-results.sarif'
severity: 'CRITICAL,HIGH'
exit-code: '1'
- name: Upload Trivy results to GitHub Security
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: 'trivy-results.sarif'
- name: Run Trivy configuration scan
uses: aquasecurity/trivy-action@master
with:
scan-type: 'config'
scan-ref: '.'
format: 'sarif'
output: 'trivy-config-results.sarif'
exit-code: '1'
- name: Upload configuration scan results
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: 'trivy-config-results.sarif'
- name: Generate HTML report
if: always()
uses: aquasecurity/trivy-action@master
with:
image-ref: ${{ steps.meta.outputs.tags }}
format: 'template'
template: '@/contrib/html.tpl'
output: 'trivy-report.html'
- name: Upload HTML report
if: always()
uses: actions/upload-artifact@v3
with:
name: trivy-report
path: trivy-report.html
retention-days: 30
secret-scan:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Run Trivy secret scanner
uses: aquasecurity/trivy-action@master
with:
scan-type: 'fs'
scan-ref: '.'
scanners: 'secret'
format: 'sarif'
output: 'trivy-secret-results.sarif'
exit-code: '1'
- name: Upload secret scan results
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: 'trivy-secret-results.sarif'
Jenkins Pipeline Integration
// Jenkinsfile - Trivy Integration
pipeline {
agent any
environment {
IMAGE_NAME = "registry.company.com/app"
IMAGE_TAG = "${env.BUILD_NUMBER}"
TRIVY_VERSION = "0.48.0"
SCAN_SEVERITY = "CRITICAL,HIGH"
}
stages {
stage('Build') {
steps {
script {
docker.build("${IMAGE_NAME}:${IMAGE_TAG}")
}
}
}
stage('Security Scan') {
parallel {
stage('Vulnerability Scan') {
steps {
script {
sh """
docker run --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v \$PWD:/workspace \
aquasec/trivy:${TRIVY_VERSION} \
image --exit-code 0 --severity MEDIUM,LOW \
${IMAGE_NAME}:${IMAGE_TAG}
"""
def scanResult = sh(
script: """
docker run --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
aquasec/trivy:${TRIVY_VERSION} \
image --exit-code 1 --severity ${SCAN_SEVERITY} \
--format json \
${IMAGE_NAME}:${IMAGE_TAG}
""",
returnStatus: true
)
if (scanResult != 0) {
error("Vulnerabilities found in image")
}
}
}
}
stage('Configuration Scan') {
steps {
sh """
docker run --rm \
-v \$PWD:/workspace \
aquasec/trivy:${TRIVY_VERSION} \
config --exit-code 1 --severity ${SCAN_SEVERITY} \
/workspace
"""
}
}
stage('Secret Scan') {
steps {
sh """
docker run --rm \
-v \$PWD:/workspace \
aquasec/trivy:${TRIVY_VERSION} \
fs --scanners secret --exit-code 1 \
/workspace
"""
}
}
}
}
stage('Generate Reports') {
steps {
sh """
docker run --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v \$PWD:/workspace \
aquasec/trivy:${TRIVY_VERSION} \
image --format json \
--output /workspace/trivy-report.json \
${IMAGE_NAME}:${IMAGE_TAG}
"""
sh """
docker run --rm \
-v /var/run/docker.sock:/var/run/docker.sock \
-v \$PWD:/workspace \
aquasec/trivy:${TRIVY_VERSION} \
image --format template \
--template "@/contrib/html.tpl" \
--output /workspace/trivy-report.html \
${IMAGE_NAME}:${IMAGE_TAG}
"""
archiveArtifacts artifacts: 'trivy-report.*', fingerprint: true
publishHTML([
allowMissing: false,
alwaysLinkToLastBuild: true,
keepAll: true,
reportDir: '.',
reportFiles: 'trivy-report.html',
reportName: 'Trivy Scan Report'
])
}
}
stage('Push Image') {
when {
expression { currentBuild.result == null || currentBuild.result == 'SUCCESS' }
}
steps {
script {
docker.withRegistry('https://registry.company.com', 'registry-credentials') {
docker.image("${IMAGE_NAME}:${IMAGE_TAG}").push()
docker.image("${IMAGE_NAME}:${IMAGE_TAG}").push('latest')
}
}
}
}
}
post {
failure {
emailext(
subject: "Security Scan Failed: ${env.JOB_NAME} - ${env.BUILD_NUMBER}",
body: "Security vulnerabilities found. Check ${env.BUILD_URL} for details.",
to: "security-team@company.com"
)
}
}
}
Kubernetes Runtime Scanning
Trivy Operator Deployment
# Trivy Operator for In-Cluster Scanning
apiVersion: v1
kind: Namespace
metadata:
name: trivy-system
labels:
app.kubernetes.io/name: trivy-operator
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: trivy-operator
namespace: trivy-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: trivy-operator
rules:
- apiGroups:
- ""
resources:
- pods
- pods/log
- replicationcontrollers
- services
- resourcequotas
- limitranges
verbs:
- get
- list
- watch
- apiGroups:
- apps
resources:
- deployments
- daemonsets
- statefulsets
- replicasets
verbs:
- get
- list
- watch
- apiGroups:
- batch
resources:
- jobs
- cronjobs
verbs:
- get
- list
- watch
- apiGroups:
- aquasecurity.github.io
resources:
- vulnerabilityreports
- configauditreports
- exposedsecretreports
- rbacassessmentreports
verbs:
- get
- list
- watch
- create
- update
- patch
- delete
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: trivy-operator
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: trivy-operator
subjects:
- kind: ServiceAccount
name: trivy-operator
namespace: trivy-system
---
apiVersion: v1
kind: ConfigMap
metadata:
name: trivy-operator-config
namespace: trivy-system
data:
scanJob.podTemplateLabels: "app=trivy-operator,scan=vulnerability"
scanJob.annotations: "sidecar.istio.io/inject=false"
vulnerabilityReports.scanner: "Trivy"
configAuditReports.scanner: "Trivy"
compliance.failEntriesLimit: "10"
scanJob.timeout: "5m"
scanJob.deleteAfterCompletion: "true"
trivy.severity: "CRITICAL,HIGH,MEDIUM"
trivy.ignoreUnfixed: "false"
trivy.dbRepository: "ghcr.io/aquasecurity/trivy-db"
trivy.javaDbRepository: "ghcr.io/aquasecurity/trivy-java-db"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: trivy-operator
namespace: trivy-system
spec:
replicas: 1
selector:
matchLabels:
app: trivy-operator
template:
metadata:
labels:
app: trivy-operator
spec:
serviceAccountName: trivy-operator
containers:
- name: trivy-operator
image: aquasec/trivy-operator:0.18.0
imagePullPolicy: IfNotPresent
args:
- operator
env:
- name: OPERATOR_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: OPERATOR_TARGET_NAMESPACES
value: "" # Empty means all namespaces
- name: OPERATOR_LOG_DEV_MODE
value: "false"
- name: OPERATOR_SCAN_JOB_TIMEOUT
value: "5m"
- name: OPERATOR_CONCURRENT_SCAN_JOBS_LIMIT
value: "10"
- name: OPERATOR_SCAN_JOB_RETRY_AFTER
value: "30s"
- name: OPERATOR_METRICS_BIND_ADDRESS
value: ":8080"
- name: OPERATOR_HEALTH_PROBE_BIND_ADDRESS
value: ":9090"
ports:
- containerPort: 8080
name: metrics
- containerPort: 9090
name: healthz
livenessProbe:
httpGet:
path: /healthz
port: healthz
initialDelaySeconds: 30
periodSeconds: 10
readinessProbe:
httpGet:
path: /readyz
port: healthz
initialDelaySeconds: 5
periodSeconds: 5
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
securityContext:
runAsNonRoot: true
runAsUser: 10000
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
Custom Resource Definitions
# VulnerabilityReport CRD Example
apiVersion: aquasecurity.github.io/v1alpha1
kind: VulnerabilityReport
metadata:
name: replicaset-nginx-deployment-7fb96c846b
namespace: production
labels:
resource-spec-hash: 7fb96c846b
trivy-operator.resource.kind: ReplicaSet
trivy-operator.resource.name: nginx-deployment-7fb96c846b
trivy-operator.resource.namespace: production
spec:
artifact:
repository: nginx
tag: "1.21"
registry:
server: docker.io
scanner:
name: Trivy
vendor: Aqua Security
version: "0.48.0"
summary:
criticalCount: 2
highCount: 5
mediumCount: 10
lowCount: 15
vulnerabilities:
- vulnerabilityID: CVE-2023-1234
resource: libssl1.1
installedVersion: 1.1.1n-0+deb11u3
fixedVersion: 1.1.1n-0+deb11u4
severity: CRITICAL
title: "OpenSSL vulnerability"
primaryLink: https://nvd.nist.gov/vuln/detail/CVE-2023-1234
score: 9.8
Scanning Policies
# Trivy Policy with OPA Rego
# trivy-policy.rego
package trivy
import data.lib.trivy
default ignore = false
# Ignore low and medium severity
ignore {
input.Severity == "LOW"
}
ignore {
input.Severity == "MEDIUM"
}
# Ignore specific CVEs with business justification
ignore {
input.VulnerabilityID == "CVE-2023-1234"
# Justification: Fixed in application layer, not exploitable
}
# Ignore vulnerabilities without fix
ignore {
input.Severity == "HIGH"
not input.FixedVersion
}
# Deny critical vulnerabilities in production images
deny[msg] {
input.Severity == "CRITICAL"
input.FixedVersion != ""
msg := sprintf("Critical vulnerability %s found in %s, fix available: %s",
[input.VulnerabilityID, input.PkgName, input.FixedVersion])
}
# Deny images with expired base OS
deny[msg] {
input.Class == "os-pkgs"
eol_date := data.eol[input.Type]
time.parse_rfc3339_ns(eol_date) < time.now_ns()
msg := sprintf("OS %s has reached end-of-life on %s", [input.Type, eol_date])
}
Admission Control Integration
Trivy Admission Controller
# Trivy Admission Controller Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: trivy-admission
namespace: trivy-system
spec:
replicas: 2
selector:
matchLabels:
app: trivy-admission
template:
metadata:
labels:
app: trivy-admission
spec:
serviceAccountName: trivy-operator
containers:
- name: admission
image: aquasec/trivy-operator:0.18.0
args:
- webhook
- --webhook-port=8443
- --webhook-cert-dir=/certs
ports:
- containerPort: 8443
name: https
volumeMounts:
- name: certs
mountPath: /certs
readOnly: true
livenessProbe:
httpGet:
path: /healthz
port: https
scheme: HTTPS
readinessProbe:
httpGet:
path: /readyz
port: https
scheme: HTTPS
resources:
limits:
cpu: 500m
memory: 512Mi
requests:
cpu: 100m
memory: 128Mi
volumes:
- name: certs
secret:
secretName: trivy-admission-tls
---
apiVersion: v1
kind: Service
metadata:
name: trivy-admission
namespace: trivy-system
spec:
ports:
- port: 443
targetPort: 8443
protocol: TCP
name: https
selector:
app: trivy-admission
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
name: trivy-admission
webhooks:
- name: trivy.admission.aquasec.github.io
admissionReviewVersions:
- v1
- v1beta1
clientConfig:
service:
name: trivy-admission
namespace: trivy-system
path: /validate
caBundle: <base64-encoded-ca-cert>
rules:
- operations:
- CREATE
- UPDATE
apiGroups:
- ""
apiVersions:
- v1
resources:
- pods
- operations:
- CREATE
- UPDATE
apiGroups:
- apps
apiVersions:
- v1
resources:
- deployments
- daemonsets
- statefulsets
failurePolicy: Ignore
matchPolicy: Equivalent
namespaceSelector:
matchExpressions:
- key: trivy-admission
operator: NotIn
values:
- disabled
sideEffects: None
timeoutSeconds: 10
Monitoring and Alerting
Prometheus Metrics
# ServiceMonitor for Trivy Operator
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
name: trivy-operator
namespace: trivy-system
spec:
selector:
matchLabels:
app: trivy-operator
endpoints:
- port: metrics
interval: 30s
path: /metrics
---
# PrometheusRule for Trivy Alerts
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: trivy-operator-alerts
namespace: trivy-system
spec:
groups:
- name: trivy-operator
interval: 30s
rules:
- alert: CriticalVulnerabilitiesDetected
expr: |
trivy_image_vulnerabilities{severity="CRITICAL"} > 0
for: 5m
labels:
severity: critical
annotations:
summary: "Critical vulnerabilities detected in {{ $labels.namespace }}/{{ $labels.name }}"
description: "Image has {{ $value }} critical vulnerabilities"
- alert: HighVulnerabilitiesThreshold
expr: |
trivy_image_vulnerabilities{severity="HIGH"} > 10
for: 10m
labels:
severity: warning
annotations:
summary: "High number of vulnerabilities in {{ $labels.namespace }}/{{ $labels.name }}"
description: "Image has {{ $value }} high severity vulnerabilities"
- alert: TrivyScanFailure
expr: |
rate(trivy_scan_errors_total[5m]) > 0.1
for: 5m
labels:
severity: warning
annotations:
summary: "Trivy scan failures detected"
description: "Trivy is experiencing scan failures at {{ $value }} errors/sec"
- alert: VulnerabilityReportOutdated
expr: |
time() - trivy_vulnerability_report_last_updated > 86400
for: 1h
labels:
severity: warning
annotations:
summary: "Vulnerability report outdated for {{ $labels.namespace }}/{{ $labels.name }}"
description: "Last scan was {{ $value | humanizeDuration }} ago"
Grafana Dashboard
{
"dashboard": {
"title": "Trivy Security Overview",
"panels": [
{
"title": "Vulnerability Summary",
"targets": [
{
"expr": "sum by (severity) (trivy_image_vulnerabilities)",
"legendFormat": "{{ severity }}"
}
],
"type": "graph"
},
{
"title": "Critical Vulnerabilities by Namespace",
"targets": [
{
"expr": "sum by (namespace) (trivy_image_vulnerabilities{severity=\"CRITICAL\"})",
"legendFormat": "{{ namespace }}"
}
],
"type": "bar"
},
{
"title": "Scan Success Rate",
"targets": [
{
"expr": "rate(trivy_scan_total[5m]) - rate(trivy_scan_errors_total[5m])",
"legendFormat": "Success Rate"
}
],
"type": "gauge"
},
{
"title": "Images by Risk Level",
"targets": [
{
"expr": "count by (risk) (trivy_image_vulnerabilities > 0)",
"legendFormat": "{{ risk }}"
}
],
"type": "pie"
}
]
}
}
Enterprise Best Practices
Multi-Stage Scanning Strategy
# Dockerfile with Multi-Stage Build and Scanning Points
# Stage 1: Build stage
FROM golang:1.21 AS builder
WORKDIR /app
COPY go.* ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o /app/server
# Stage 2: Security scan stage (for CI/CD)
FROM aquasec/trivy:0.48.0 AS scanner
COPY --from=builder /app /scan
RUN trivy fs --exit-code 1 --severity CRITICAL /scan
# Stage 3: Minimal runtime
FROM gcr.io/distroless/static-debian12:nonroot
COPY --from=builder /app/server /server
USER nonroot:nonroot
ENTRYPOINT ["/server"]
Automated Remediation Workflow
#!/bin/bash
# Automated vulnerability remediation script
set -euo pipefail
NAMESPACE="${1:-production}"
IMAGE="${2}"
REGISTRY="registry.company.com"
log() {
echo "[$(date +'%Y-%m-%d %H:%M:%S')] $*"
}
# Scan image
scan_image() {
local image="$1"
log "Scanning image: ${image}"
trivy image \
--format json \
--output scan-results.json \
--severity CRITICAL,HIGH \
"${image}"
}
# Parse scan results
parse_vulnerabilities() {
local results_file="$1"
jq -r '.Results[].Vulnerabilities[] |
select(.FixedVersion != null and .FixedVersion != "") |
"\(.PkgName):\(.InstalledVersion) -> \(.FixedVersion)"' \
"${results_file}"
}
# Generate remediation PR
create_remediation_pr() {
local image="$1"
local vulnerabilities="$2"
# Create branch
git checkout -b "security/remediate-${image}-$(date +%Y%m%d)"
# Update dependencies based on vulnerability report
# This is application-specific logic
# Commit changes
git add .
git commit -m "Security: Remediate vulnerabilities in ${image}
Vulnerabilities fixed:
${vulnerabilities}
Automated remediation by Trivy scanner"
# Create PR
gh pr create \
--title "Security: Remediate vulnerabilities in ${image}" \
--body "Automated vulnerability remediation" \
--label "security,automated"
}
# Main workflow
main() {
scan_image "${IMAGE}"
local vulns=$(parse_vulnerabilities scan-results.json)
if [[ -n "${vulns}" ]]; then
log "Found fixable vulnerabilities:"
echo "${vulns}"
create_remediation_pr "${IMAGE}" "${vulns}"
else
log "No fixable vulnerabilities found"
fi
}
main "$@"
Conclusion
Implementing comprehensive container vulnerability scanning with Trivy provides critical security insights throughout the software development lifecycle. By integrating Trivy into CI/CD pipelines, implementing runtime scanning with the Trivy Operator, and enforcing policies through admission control, organizations can significantly reduce their attack surface while maintaining development velocity.
The key to successful implementation lies in balancing security requirements with operational efficiency, automating remediation workflows where possible, and establishing clear policies for vulnerability management. Regular scanning, continuous monitoring, and proactive remediation ensure that container images remain secure throughout their lifecycle in production environments.