Cloud infrastructure is no longer evolving in response to AI workloads. It is being rebuilt around them. Over the past twelve months, the changes have been structural: Kubernetes now treats GPUs as first-class citizens, serverless has hybridized to eliminate its most persistent tradeoffs, and the GPU market has fragmented into a multi-vendor race that fundamentally alters procurement strategy. For engineering teams making infrastructure decisions today, the landscape looks nothing like it did eighteen months ago.
Kubernetes Becomes AI-Native
The most significant shift is happening inside Kubernetes itself. With the release of v1.35 “Timbernetes” in December 2025, several features critical to AI workloads reached maturity. In-Place Pod Resource Resize hit GA, allowing teams to adjust CPU and memory allocations on running pods without restarts. For inference workloads with variable demand, this eliminates the disruptive restart cycles that previously degraded availability during scaling events.
More importantly for training workloads, Gang Scheduling entered Alpha. This feature enables scheduling groups of pods together as a unit rather than individually — a requirement for distributed training jobs where all workers must be co-located and start simultaneously. Without gang scheduling, teams have relied on custom controllers or third-party schedulers to achieve this, adding operational complexity. Having it upstream changes the calculus for teams that previously avoided running training on Kubernetes.
These are not incremental improvements. They represent Kubernetes acknowledging that AI workloads have different scheduling, resource, and lifecycle requirements than web services — and adapting its core primitives accordingly. The 419 contributors from 85 companies behind this release reflect the breadth of investment.
The CNCF formalized this direction further with the Kubernetes AI Conformance Program v1.0 in November 2025. This is the first industry standard for running AI workloads on Kubernetes, covering Dynamic Resource Allocation for GPUs, volume handling for large datasets, job-level networking for distributed training, and gang scheduling. Azure, GKE, CoreWeave, and Akamai are already certified. For teams evaluating managed Kubernetes providers, this conformance program provides a concrete checklist rather than relying on marketing claims.
Serverless Goes Hybrid
AWS Lambda Managed Instances, launched in December 2025, represents the most important evolution in serverless since its inception. The idea is straightforward but the implications are significant: run Lambda functions on EC2 compute while retaining the serverless programming model. The result is zero cold starts, EC2 pricing with up to 72% savings over standard Lambda, and multi-concurrency per environment.
This matters because it dissolves the traditional binary choice between serverless and server-based architectures. Teams no longer need to choose between the operational simplicity of Lambda and the cost efficiency and performance predictability of EC2. For AI inference workloads — where cold starts are unacceptable and per-invocation costs at scale are prohibitive — this hybrid model opens a path that did not exist before.
We see this as the beginning of a broader convergence. The distinction between serverless and traditional compute is becoming an implementation detail rather than an architectural decision. Teams that previously rejected serverless for cost or latency reasons should revisit their assumptions.
The GPU Market Fragments
The GPU infrastructure market has become genuinely multi-vendor for the first time. NVIDIA’s Rubin platform is entering deployment through CoreWeave in the second half of 2026, while Azure was first to deploy GB300 NVL72 racks at scale. Meanwhile, AWS Trainium3 is in preview, offering a credible alternative for teams willing to invest in the compilation toolchain.
The numbers tell the story: hyperscaler AI infrastructure spend is projected to exceed $180 billion in 2026, and inference has officially surpassed training in GPU revenue. That second point is critical. It signals that the industry bottleneck is shifting from model development to model deployment — which means infrastructure decisions around inference optimization, request routing, and cost management now have outsized impact on the bottom line.
For engineering teams, the practical implication is that GPU procurement is no longer a single-vendor decision. Workload characteristics should dictate provider choice: NVIDIA for maximum compatibility and ecosystem depth, custom silicon like Trainium for inference cost optimization at scale, and edge platforms like Cloudflare’s Infire engine for latency-sensitive deployments where data sovereignty matters. Locking into a single GPU vendor is now a strategic risk, not a simplification.
What This Means for Engineering Teams
Three concrete takeaways for teams making infrastructure decisions today:
Standardize on Kubernetes for AI, but demand conformance. The CNCF AI Conformance Program gives you a baseline to evaluate providers. If your managed Kubernetes offering is not certified, ask why. Features like Dynamic Resource Allocation and gang scheduling should not be afterthoughts.
Reevaluate your serverless boundaries. Lambda Managed Instances collapse the serverless-versus-server tradeoff. If you ruled out serverless for latency or cost reasons in the past, the constraints have changed. Audit your workloads with the new pricing and performance characteristics.
Build for multi-GPU-vendor from day one. Abstract your inference layer so you can shift workloads across NVIDIA, custom silicon, and edge providers based on cost, latency, and regulatory requirements. The teams that treat GPU infrastructure as a pluggable layer will have a significant advantage as the market continues to fragment.
The common thread across all of these shifts is that AI workloads are no longer a special case bolted onto existing infrastructure. They are the primary driver of infrastructure evolution. Engineering teams that recognize this and adapt their tooling, procurement, and architecture accordingly will build systems that are both more capable and more cost-efficient. The window to make these decisions well is now.