{"id":4107,"date":"2026-11-16T23:55:00","date_gmt":"2026-11-16T23:55:00","guid":{"rendered":"https:\/\/chennaiorbit.com\/blog\/?p=4107"},"modified":"2026-11-19T22:55:00","modified_gmt":"2026-11-19T22:55:00","slug":"36-50-kubernetesops-restarts-timers-plain-words","status":"publish","type":"post","link":"https:\/\/chennaiorbit.com\/blog\/36-50-kubernetesops-restarts-timers-plain-words\/","title":{"rendered":"Why Pods Keep Restarting: KubernetesOps Timers in Plain Words"},"content":{"rendered":"<p>Anyone who has ordered from a busy neighbourhood kitchen knows the rhythm: if a dish fails, the cook tries again, but not instantly, and not forever. A cluster has a similar patience built in. Understanding those patience settings is one of the friendliest ways into KubernetesOps, the everyday upkeep of Kubernetes clusters, because they explain a lot of puzzling behaviour. Why does a broken app keep reappearing? Why does it take five minutes for work to move off a dead machine? Let us walk through it in plain language.<\/p>\n<h2>First, what is a Kubernetes cluster?<\/h2>\n<p>Think of it as a kitchen with a head office. Worker machines, called nodes, do the actual cooking, which here means running your programs packed into small units called pods. The head office, known as the control plane, keeps the order book and decides which worker takes which job. KubernetesOps is the work of keeping both parts healthy, upgraded and sensibly sized. For a short orientation, the <a href=\"https:\/\/www.xopsschool.com\/kubernetesops\/\">Ops Atlas explanation of KubernetesOps<\/a> is a useful companion, and it is where the figures below come from.<\/p>\n<h2>The restart that waits longer each time<\/h2>\n<p>Suppose a pod crashes as soon as it starts, perhaps because of a typo in a setting. Kubernetes will restart it, but it will not hammer away. Each failed attempt is followed by a longer pause, starting at ten seconds and growing until it reaches a ceiling of three hundred seconds, or five minutes. That is the origin of the status many people see: CrashLoopBackOff. It is not an error by itself; it is a message saying &#8220;I am waiting before I try again&#8221;. If you fix the typo, the next retry succeeds, though you may need to wait out the current pause.<\/p>\n<h2>Why does moving off a failed machine take so long?<\/h2>\n<p>When a node stops reporting, the cluster does not panic. Workloads carry a patience setting called a toleration, and for a node marked not ready the default is 300 seconds. Only after those five minutes do pods get moved elsewhere. This default prevents needless shuffling when a machine merely hiccups, but it surprises people who expect instant failover. If your service cannot wait five minutes, you can adjust the toleration, though you should do so knowingly because short fuses can cause churn.<\/p>\n<h2>The roll-out that gives up after ten minutes<\/h2>\n<p>Updating an application through a Deployment has a deadline too. The progress deadline default is 600 seconds. If new pods have not become ready by then, the Deployment is flagged with a reason called ProgressDeadlineExceeded. Notice what this does not do: it does not roll back automatically. It only reports that things are stuck, so a person or a pipeline still has to decide what to do next.<\/p>\n<h2>Health checks: keep them local<\/h2>\n<p>Here is the piece where friendly advice matters most. A liveness probe asks, &#8220;is this process still alive?&#8221; and restarts the pod if the answer is no. It sounds harmless, but the Atlas lists a classic mistake: making the liveness check depend on something outside the pod, like a database. When that database slows down, every copy of your app fails its liveness test together and gets restarted together, which makes the outage bigger. A liveness check should look only at the process itself.<\/p>\n<h2>Pitfall: protections that get in the way<\/h2>\n<p>Two more everyday traps appear in the Atlas. A disruption budget that allows zero pods to be unavailable will stall a node drain indefinitely, because there is nothing it is allowed to evict. And admission webhooks, which are checks that run when objects are created, fail closed by default: if the webhook service is down, requests can be refused. The Atlas advises keeping such checkers well clear of the cluster&#8217;s own system area and its network plugin so a broken checker cannot stop the cluster from mending itself.<\/p>\n<h2>How to read a stuck situation<\/h2>\n<p>When something looks frozen, ask which timer might be running. Is a pod in a back-off pause, so that the sensible thing is to fix the cause and wait a few minutes? Is a node newly gone quiet, meaning the five-minute tolerance is still counting down? Has a roll-out passed its ten-minute deadline and been marked, leaving a human to choose between waiting and reverting? Naming the timer turns a vague worry into a specific question, and specific questions are the ones that get answered.<\/p>\n<h2>A tiny cheat sheet to keep nearby<\/h2>\n<ul>\n<li>Crash restarts back off from 10 seconds up to a 300 second cap.<\/li>\n<li>A not-ready node is tolerated for 300 seconds before work moves.<\/li>\n<li>A stuck roll-out is flagged after 600 seconds.<\/li>\n<li>Liveness checks stay inside the pod; they never call other services.<\/li>\n<li>Zero-disruption budgets block drains.<\/li>\n<\/ul>\n<p>Pin that list where your team can see it. When a colleague asks why something is taking so long, one of these numbers is often the answer, and knowing it saves an hour of guessing.<\/p>\n<h2>Where to go from here<\/h2>\n<p>Once these timers feel familiar, try changing one on a practice cluster and watching what happens. That kind of tinkering teaches more than any article. And when you need a break from terminals, our guide to <a href=\"https:\/\/chennaiorbit.com\/blog\/how-can-visitors-plan-their-first-trip-to-chennai\/\">planning a first visit to Chennai<\/a> and the companion piece on <a href=\"https:\/\/chennaiorbit.com\/blog\/how-many-days-are-enough-to-explore-chennai\/\">how much time the city deserves<\/a> are good reading over a cup of filter coffee.<\/p>\n<p>Whatever you learn, remember the cook in the kitchen: patience with limits, retries with pauses, and a clear way to tell someone when something is stuck. Clusters are really just very organised kitchens.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A friendly explainer of the waiting times inside a Kubernetes cluster, from crash-restart delays to slow failover, using a neighbourhood kitchen.<\/p>\n","protected":false},"author":7,"featured_media":4108,"comment_status":"","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2432],"tags":[],"class_list":["post-4107","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech-how-to"],"_links":{"self":[{"href":"https:\/\/chennaiorbit.com\/blog\/wp-json\/wp\/v2\/posts\/4107","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/chennaiorbit.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/chennaiorbit.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/chennaiorbit.com\/blog\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/chennaiorbit.com\/blog\/wp-json\/wp\/v2\/comments?post=4107"}],"version-history":[{"count":1,"href":"https:\/\/chennaiorbit.com\/blog\/wp-json\/wp\/v2\/posts\/4107\/revisions"}],"predecessor-version":[{"id":4240,"href":"https:\/\/chennaiorbit.com\/blog\/wp-json\/wp\/v2\/posts\/4107\/revisions\/4240"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/chennaiorbit.com\/blog\/wp-json\/wp\/v2\/media\/4108"}],"wp:attachment":[{"href":"https:\/\/chennaiorbit.com\/blog\/wp-json\/wp\/v2\/media?parent=4107"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/chennaiorbit.com\/blog\/wp-json\/wp\/v2\/categories?post=4107"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/chennaiorbit.com\/blog\/wp-json\/wp\/v2\/tags?post=4107"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}