Viewing Logs from the Previous Container Instance When Your Container Is Crashing // Support Tools

When a Kubernetes Pod enters a CrashLoopBackOff state, it can be challenging to figure out what’s going wrong, especially since the container may be repeatedly crashing and restarting. In these cases, viewing the logs from the previous instance of the container can provide critical insight into why the container is failing.

In this post, we’ll walk through how to view the logs from a previous container instance using the kubectl logs command and provide troubleshooting tips for common issues.

Understanding Container Restart Behavior

When a container crashes, Kubernetes will attempt to restart it automatically. However, during the restart process, logs from the previous container instance may not be shown by default. Instead, you’ll need to explicitly request logs from the previous container to capture any crash-related information.

Step 1: Check Pod Status

First, confirm that your Pod is in a CrashLoopBackOff state by running:

kubectl get pod <pod-name>

Example output:

NAME      READY   STATUS             RESTARTS   AGE
my-app    0/1     CrashLoopBackOff    5          5m

The CrashLoopBackOff status indicates that the container has failed and is being restarted multiple times.

Step 2: View Logs from the Previous Container Instance

To view logs from the previous instance of the container, use the kubectl logs command with the --previous flag:

kubectl logs <pod-name> --previous

This command retrieves the logs from the last terminated container instance before the container was restarted.

Example

kubectl logs my-app --previous

This will output the logs from the previous container instance. For example:

Error: Cannot connect to database at 'mysql://invalid-url'

These logs can help you identify the root cause of the crash. In this case, the error indicates that the application is trying to connect to an invalid database URL, which is causing the container to crash.

Step 3: Troubleshoot Based on Previous Logs

Now that you have the logs from the previous container instance, you can begin troubleshooting. Common issues you may find include:

Environment Variable Misconfiguration: Incorrect environment variables (such as database URLs, API keys, etc.) causing the application to fail.
Missing Dependencies: The application might crash if certain dependencies or services it relies on are unavailable.
Resource Limitations: The container may crash due to resource exhaustion (memory or CPU).

In the example above, the issue was a misconfigured database URL. Updating the DATABASE_URL environment variable to point to the correct database service would resolve the crash.

Step 4: Redeploy and Verify

After troubleshooting and resolving the issue, redeploy the updated Pod or configuration. Use kubectl apply to deploy the corrected configuration:

kubectl apply -f <pod-config.yaml>

Then, verify that the Pod is now running without any crashes:

kubectl get pod <pod-name>

The Pod should transition from CrashLoopBackOff to Running:

NAME      READY   STATUS    RESTARTS   AGE
my-app    1/1     Running   0          2m

Step 5: Monitoring Future Logs

After fixing the crash, continue monitoring the Pod’s logs to ensure that the issue is fully resolved. You can use the kubectl logs command to view real-time logs from the currently running container:

kubectl logs <pod-name> -f

Conclusion

Viewing logs from the previous container instance using kubectl logs --previous is a simple yet powerful way to troubleshoot crashing containers in Kubernetes. By examining logs from the terminated container, you can identify the root cause of the crash and make necessary adjustments to prevent further issues. Whether you’re dealing with misconfigured environment variables, missing dependencies, or resource constraints, this method will help you get to the bottom of container crashes quickly.