Kubernetes has quietly solved one of the most annoying problems in AI infrastructure: scheduling batch jobs that need all their pods up at once. Kubernetes v1.36, codenamed Haru, shipped native gang scheduling as part of workload aware scheduling, and it is the most AI focused release the project has produced. For anyone running distributed training or multi pod inference on AWS EKS, this changes how you plan clusters.
Why gang scheduling still matters
A distributed training job rarely fits in one pod. You split a model across four, eight, or sixteen workers, and every worker must start together. If the scheduler admits only some pods and queues the rest, the job never makes progress. Workers wait on missing peers, hold resources idle, and burn money on GPU time while doing nothing.
This is the all or nothing problem. Without gang scheduling, a partially scheduled job can deadlock. You pay for half a GPU fleet that computes nothing. In a shared EKS cluster with a mix of workloads, this waste is painful and hard to predict.
The old workaround fell short
For years the fix was a queueing layer on top. Operators like Volcano and Kueue handled batch priorities and gang semantics, and they still have a place for advanced policies. But they add moving parts. Every cluster you run needs the controller deployed, tuned, and upgraded. For a small platform team, that is real operational overhead.
Managing these operators across many EKS environments became a recurring cost. It worked, but it never felt like a primitive the platform could rely on by default.
What v1.36 ships
Kubernetes v1.36 brought workload aware scheduling with gang scheduling built into the core scheduler. The scheduler now understands that a group of pods belongs together and only starts the job when the full set can be placed. Partial admission is avoided at the source instead of patched around.
The release included around seventy enhancements, several aimed squarely at AI and ML workloads. The message from the SIG is clear: Kubernetes wants to be the operating system for model training and inference, not just web services.
Pair it with DRA for GPUs
Gang scheduling handles the all or nothing placement. Dynamic Resource Allocation, DRA, handles the other half of the puzzle. DRA went GA in Kubernetes v1.34 and gives workloads a declarative way to request GPUs by property, such as memory, interconnect, or topology, instead of the old device plugin crude counting.
The GPU vendors have standardized on it. On EKS you declare what the job needs and the resource manager satisfies it. Combined with gang scheduling, a training workload finally expresses both its placement and its GPU requirements in cluster native terms.
How to try it
Audit your existing GPU jobs and pick one proof of concept that is not business critical. Enable the relevant feature gates on your control plane and nodes, then declare your job as a single gang. Watch how often it previously started partially, then compare. The difference in utilization is usually obvious on the billing side.
Add the new scheduling fields to your Terraform generated manifests so the change is reviewable and repeatable across environments. Keep straight resource requests in place while you measure. Move to gang scheduling in a controlled rollout, then address GPUs through DRA once your driver is ready.
The bottom line
Native gang scheduling removes a whole category of AI batch failures from platform troubleshooting. It is free, in cluster, and shaped around how distributed workloads actually run. If you are already managing EKS for model training, this is the upgrade to test next. Fewer half started jobs means a cluster that earns its bill, and fewer late night incidents for the person on call, which is usually you.