Files
awesome-kubernetes/docs/kubernetes-troubleshooting.md
2026-05-14 20:30:43 +00:00

38 KiB
Raw Blame History

Kubernetes Troubleshooting

  1. Introduction
  2. Kubernetes Events
  3. Kubernetes Network Troubleshooting
  4. Exit Codes in Containers and Kubernetes
  5. ImagePullBackOff
  6. CrashLoopBackOff
  7. Failed to Create Pod Sandbox
  8. Terminated with exit code 1 error
  9. Pod in Terminating or Unknown Status
  10. OOM Kills
  11. Pause Container
  12. Preempted Pod
  13. Evited Pods
  14. Stuck Namespace
  15. Access PVC Data without the POD
  16. CoreDNS issues
  17. Debugging Techniques and Strategies. Debugging with ephemeral containers
  18. Troubleshooting Tools
    1. Komodor
    2. Palaemon
    3. cdebug and debug-ctr
    4. kubectl-debug
    5. Kubeshark
  19. Slides
  20. Images
  21. Tweets

Introduction

Kubernetes Events

Kubernetes Network Troubleshooting

Exit Codes in Containers and Kubernetes

ImagePullBackOff

  • 10 Real-World Kubernetes Troubleshooting Scenarios and Solutions 🌟 - This article provides practical, hands-on solutions for common Kubernetes production issues. It details 10 real-world scenarios, including ImagePullBackOff due to private registry authentication failure, and offers exact kubectl commands and steps for diagnosis and resolution. It also touches upon cloud-managed Kubernetes solutions and IAM roles for registry authentication.

  • blog.ediri.io: Kubernetes: ImagePullBackOff! How to keep your calm and fix this like a pro!

CrashLoopBackOff

Failed to Create Pod Sandbox

Terminated with exit code 1 error

Pod in Terminating or Unknown Status

OOM Kills

Pause Container

Preempted Pod

Evited Pods

Stuck Namespace

Access PVC Data without the POD

CoreDNS issues

Debugging Techniques and Strategies. Debugging with ephemeral containers

Troubleshooting Tools

Komodor

Palaemon

cdebug and debug-ctr

  • ==iximiuz/cdebug== a swiss army knife of container debugging. It's like "docker exec", but it works even for containers without a shell (scratch, distroless, slim, etc). The "cdebug exec" command allows you to bring your own toolkit and start a shell inside of a running container.
  • ==felipecruz91/debug-ctr== A commandline tool for interactive troubleshooting when a container has crashed or a container image doesn't include debugging utilities, such as distroless images. Heavily inspired by kubectl debug, but for containers instead of Pods.

kubectl-debug

  • github.com/JamesTGrant/kubectl-debug kubectl-debug is a tool that lets you debug a target container in a Kubernetes cluster by automatically creating a new, non-invasive, 'debug' container in the same PID, network, user, and IPC namespace as the target container without any disruption

Kubeshark

Slides

??? note "Click to expand!"

<center>
<iframe class="speakerdeck-iframe" frameborder="0" src="https://speakerdeck.com/player/e1c397b2aa67470e9204f82f938fec78?slide=1" title="KubeCologne keynote—Troubleshooting Kubernetes apps" allowfullscreen="true" mozallowfullscreen="true" webkitallowfullscreen="true" style="border: 0px; background: padding-box padding-box rgba(0, 0, 0, 0.1); margin: 0px; padding: 0px; border-radius: 6px; box-shadow: rgba(0, 0, 0, 0.2) 0px 5px 40px; width: 560px; height: 314px;" data-ratio="1.78343949044586"></iframe>
</center>

Images

??? note "Click to expand!"

<center>
[![learnk8s debug your pods](images/learnk8s_debug_your_pods.png){: style="width:30%"}](https://learnk8s.io/troubleshooting-deployments)
</center>

Tweets

??? note "Click to expand!"

<center>
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">My top 8 commands and tools for debugging applications running on <a href="https://twitter.com/kubernetesio?ref_src=twsrc%5Etfw">@kubernetesio</a> 🧵👇</p>&mdash; Daniel Bryant (@danielbryantuk) <a href="https://twitter.com/danielbryantuk/status/1492893332850237444?ref_src=twsrc%5Etfw">February 13, 2022</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
<blockquote class="twitter-tweet"><p lang="en" dir="ltr">What is your favourite Kubernetes troubleshooting command? Looking for some new ones 😉</p>&mdash; Saiyam Pathak (@SaiyamPathak) <a href="https://twitter.com/SaiyamPathak/status/1513572111721271298?ref_src=twsrc%5Etfw">April 11, 2022</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">I made a tool... to debug containers 🧙‍♂️<br><br>It&#39;s like &quot;docker exec&quot;, but it works even for containers without a shell (scratch, distroless, slim, etc).<br><br>The &quot;cdebug exec&quot; command allows you to bring your own toolkit and start a shell inside of a running container.<br><br>A short demo 👇 <a href="https://t.co/82m4vzPYJr">pic.twitter.com/82m4vzPYJr</a></p>&mdash; Ivan Velichko (@iximiuz) <a href="https://twitter.com/iximiuz/status/1584244173347074049?ref_src=twsrc%5Etfw">October 23, 2022</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">There is a Kubernetes deployment which processes items from a queue. Most items are very small and completed immediately. Occasionally a whopping big item comes along and causes an OOMKill. Retries don&#39;t help for obvious reasons.<br><br>How would you solve it?</p>&mdash; Natan Yellin (@aantn) <a href="https://twitter.com/aantn/status/1597653772255346691?ref_src=twsrc%5Etfw">November 29, 2022</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>

<blockquote class="twitter-tweet"><p lang="en" dir="ltr">How does Pod to Pod communication work in Kubernetes?<br><br>How does the traffic reach the pod?<br><br>Let&#39;s dive into how low-level networking works in Kubernetes. <a href="https://t.co/K8bBT8YiOf">pic.twitter.com/K8bBT8YiOf</a></p>&mdash; Daniele Polencic — @danielepolencic@hachyderm.io (@danielepolencic) <a href="https://twitter.com/danielepolencic/status/1655540892365889538?ref_src=twsrc%5Etfw">May 8, 2023</a></blockquote> <script async src="https://platform.twitter.com/widgets.js" charset="utf-8"></script>
</center>