Pulumi Blog: Kubecon

KubeCon EU 2026 Recap: The Year AI Moved Into Production on Kubernetes

Engin Diri — Wed, 01 Apr 2026 00:00:00 -0700

Amsterdam in late March still has that sharp North Sea wind, but inside the RAI Convention Centre, 13,350 people generated enough energy to heat the building twice over. KubeCon + CloudNativeCon EU 2026 was the biggest European edition yet, and the shift from previous years was impossible to miss. AI dominated the conference.

I spent most of the conference at the Pulumi booth, and that turned out to be the best vantage point. Hundreds of visitors stopped by over four days, and I kept asking the same question: what are you actually running in production with AI on Kubernetes? The answers shaped this post more than any keynote did. Almost everyone had a proof of concept. Almost nobody had a production story they were happy with.

Here is the stat that framed the entire conference for me: 66% of organizations use Kubernetes to host generative AI workloads, but only 7% deploy to production daily. That gap between experimentation and actual production use matched what I was hearing at the booth. The CNCF’s own survey now counts 19.9 million cloud native developers worldwide, 7.3 million of them building AI workloads. The tooling and the infrastructure need to catch up.

My takeaway after four days on the ground: lots of working demos, very few production setups people trust. Teams are trying to scale inference, put guardrails around agents, and make GPU infrastructure behave like anything else they run.

Here is what I saw.

From training to inference: the big pivot

About 67% of AI compute now goes to inference, not training. The inference market is projected to hit $255 billion by 2030. It’s also where most of the operational complexity lives.

NVIDIA leaned into this hard. Their open-source stack around NeMo and Dynamo got significant stage time, but the bigger move was donating three projects to the CNCF: the DRA driver for fractional GPU allocation, the KAI Scheduler for GPU-aware scheduling, and Grove. Moving these to community governance signals that GPU infra is becoming part of the standard Kubernetes toolkit.

The CNCF donations that will reshape AI on Kubernetes

Every KubeCon has its crop of new CNCF projects, but this year’s batch felt different. We are starting to see the building blocks of an AI runtime for Kubernetes.

llm-d was the headline donation. Created by IBM Research, Red Hat, and Google Cloud, it splits inference workloads by separating prefill and decode phases across different pods. The collaborator list reads like an industry consortium: NVIDIA, CoreWeave, AMD, Cisco, Hugging Face, Intel, Lambda, Mistral AI, UC Berkeley, and UChicago. When that many organizations agree on a single approach to distributed inference, pay attention.

NVIDIA’s DRA driver enables fractional GPU allocation and multi-node NVLink support. GPU multi-tenancy is one of the hardest unsolved problems in Kubernetes right now. Scheduling, isolation, cost attribution — all of it breaks down when multiple workloads share a GPU. The DRA driver does not solve everything, but it gives the community a real starting point.

KAI Scheduler entered the CNCF Sandbox for GPU-aware scheduling. If llm-d handles the inference runtime and the DRA driver handles allocation, KAI Scheduler handles placement. Together, these three projects form the skeleton of a GPU-native Kubernetes stack.

Velero, donated by Broadcom, moved into CNCF Sandbox for backup and restore. AI workloads are stateful now (model weights, checkpoints, fine-tuning data), and backup is no longer optional. Good timing.

Microsoft AI Runway is an open-source Kubernetes API for inference that plugs in Hugging Face model discovery, GPU memory fit calculations, and cost estimates. Think of it as a model-aware control plane. HolmesGPT and Dalec, also from Microsoft, entered CNCF Sandbox for AI-powered troubleshooting and dependency analysis.

The Kubernetes AI Conformance Program is growing fast, with certifications nearly doubled and three new requirements proposed for Kubernetes 1.36. Conformance programs are boring until they are not. This one will determine which distributions can credibly claim AI readiness.

Agentic AI gets an identity layer

If inference was this year’s production story, agentic AI was the architecture story. Agents are proliferating, and nobody has quite figured out how to manage and secure them inside Kubernetes yet.

kagent, donated to CNCF Sandbox by Solo.io, defines agents as Kubernetes CRDs. It ships with pre-built MCP (Model Context Protocol) servers for Kubernetes, Istio, Helm, Argo, Prometheus, Grafana, and Cilium. An agent becomes a first-class Kubernetes resource, schedulable and observable and subject to RBAC, instead of a rogue process running in someone’s notebook.

kagenti from IBM goes after the identity problem directly. Using SPIFFE/SPIRE, it gives agents cryptographic identities. When an agent calls an API, you can verify exactly which agent made the call, what trust domain it belongs to, and whether it is authorized. This kind of security work needs to happen before agents proliferate across production clusters. Retrofitting identity later is ugly.

Dapr Agents took a different angle with the actor model and durable execution. Each agent gets reliable state management and exactly-once messaging semantics. If your workflows cannot tolerate lost messages or duplicate actions, this matters.

agentregistry showed up as a centralized discovery service for MCP servers and agents. As agents and tool servers multiply, you need a registry to find and manage them, the same way container registries became necessary for images.

David Soria Parra from Anthropic gave a talk on MCP evolving beyond simple tool-calling into richer interaction patterns (sched). Google announced the Kubernetes Agent Sandbox for running agentic AI workloads in secure, isolated environments — we’ve since published a full walkthrough of what it is and how to deploy it.

AI gateways and inference routing

Gateway infrastructure had its own mini-conference within KubeCon. The Gateway API Inference Extension from the Kubernetes SIG introduces model-aware routing and load balancing at the gateway level. Instead of routing by URL path, your gateway routes by model name, version, and capacity. That changes how inference traffic flows through a cluster in a fundamental way.

Envoy AI Gateway builds on Envoy’s existing proxy capabilities with token-aware rate limiting and provider failover. If your primary inference provider is saturated, traffic shifts to a secondary automatically. Rate limiting by token count rather than request count makes much more sense for LLM workloads, where a single request can consume vastly different amounts of compute.

I want to call out Agentgateway specifically. Written in Rust, it proxies LLM traffic, MCP connections, and agent-to-agent communication, with Cedar and CEL policy engines for fine-grained access control. Rust’s performance characteristics matter here because inference gateway latency adds directly to user-perceived response time.

Kuadrant, now in CNCF Sandbox, layers policy on top of gateway infrastructure and includes MCP server aggregation. Gateways are evolving from dumb traffic proxies into intelligent control planes for AI workloads, and these four projects are driving that shift.

Platform engineering absorbs LLMOps

The observability and platform engineering vendors showed up in force. The message was consistent: LLMOps is just platform engineering with new requirements.

Chronosphere demonstrated parallel AI investigation, with multiple agents analyzing different aspects of an incident simultaneously and combining their findings. SUSE Liz takes a domain-specialized approach, deploying different AI agents for different operational domains rather than one general-purpose assistant. groundcover combines eBPF with OpenTelemetry to give coding agents rich runtime context about the systems they are modifying. That last one is subtle but important: if an AI agent is writing code that touches a service, it should understand that service’s actual runtime behavior, not just its source code.

Dynatrace and DevCycle partnered to make feature flags observable primitives via OpenFeature. Rolling out AI features behind feature flags is table stakes, but having those flags show up in your observability pipeline as first-class signals closes a real gap.

Shadow AI governance emerged as its own theme. CAST AI’s Kimchi can route requests across 50+ models while providing centralized visibility into what models are being used, by whom, and at what cost. Every large organization I talked to had some version of the same problem: teams spinning up model endpoints without central oversight, burning through GPU budgets, creating compliance blind spots they did not even know about.

GPU multi-tenancy remains genuinely unsolved. Scheduling, workload isolation, cost attribution across shared GPUs — all of it breaks down at scale. Multiple talks addressed pieces of this, but nobody had a complete answer.

Sovereignty shapes infrastructure architecture

Regulation came up in almost every conversation. The EU Cyber Resilience Act is driving compliance requirements deep into software supply chains, and every European organization I spoke with is feeling the pressure. Teams are already changing how they build and deploy software.

Sovereign Kubernetes is a platform architecture requirement now, not something you can defer to next quarter. Organizations need Kubernetes distributions and cloud regions that guarantee data residency, and they need the tooling to enforce those guarantees programmatically. Self-hosted models are proliferating partly because of capability and cost, but data sovereignty is the accelerant. If your data cannot leave a jurisdiction, neither can your model.

Runtime isolation is expanding beyond containers. Several talks covered KVM-based isolation for AI workloads, which is heavier than containers but necessary when the threat model includes side-channel attacks on shared GPU memory. The sandboxing conversation has gotten more sophisticated since last year.

These constraints are not uniquely European. Any organization operating across jurisdictions faces similar pressures, and the regulatory direction globally is toward more data sovereignty requirements, not fewer.

What this means for your team

Four days in Amsterdam distilled into five things I would act on now:

Treat inference workloads like production services. If you are still deploying models with scripts and hope, stop. Inference infrastructure needs the same IaC discipline as any other production system: version-controlled, tested, policy-enforced.
Evaluate the Gateway API Inference Extension and llm-d. These are not speculative projects. They have broad industry backing and solve real problems around inference routing and distributed serving. Get them into your test environments.
Plan agent identity before agents proliferate. SPIFFE/SPIRE for agent identity is not optional if you are running agents in production. Retrofitting identity onto an existing agent fleet is painful. Start with kagenti now.
Platform teams should own AI infrastructure. Shadow AI is already happening in your organization. The platform engineering team needs to provide self-service AI infrastructure with guardrails before ungoverned model endpoints become a security and cost problem.
Sovereignty and GPU multi-tenancy are universal. Even if you are not subject to the EU Cyber Resilience Act today, data residency requirements are spreading globally. GPU multi-tenancy will affect every organization running inference at scale.

Kubernetes spent the past decade proving it could orchestrate containers. The next decade will test whether it can orchestrate intelligence. Based on what I saw in Amsterdam, the community is building the right pieces, but the gap between what exists and what production demands is still wide. That gap is where the interesting work happens.

YAML, Terraform, Pulumi: What’s the Smart Choice for Deployment Automation with Kubernetes?

Engin Diri — Mon, 18 Nov 2024 08:00:00 +0100

YAML and Kubernetes go together like peanut butter and jelly. While Kubernetes objects can be defined in JSON, YAML has emerged as the de facto standard.

It’s often the first tool developers encounter when diving into Kubernetes, and for good reason - its human-readable format makes it the preferred choice in most tutorials, documentation, and even production deployments.

Since this post was published, Pulumi has added first-class support for HashiCorp Configuration Language (HCL). You can now write Pulumi programs in HCL directly, alongside general-purpose languages like TypeScript, Python, Go, and C#. To see how it works, see Pulumi HCL.

But is it the best way to manage your Kubernetes resources? Let’s compare YAML, Terraform, and Pulumi and see which is best for managing your Kubernetes resources.

I also gave a talk before on No YAML! and what alternatives you have if you would only stick with the Go language:

Kubernetes Components and the Kubernetes API

Before we discuss the ways to manage Kubernetes objects, let’s look at the components of a typical Kubernetes cluster and their interactions.

Kubernetes Components. Credit: kubernetes.io

A Kubernetes cluster has two component categories: control plane and node. The control plane components manage the cluster. They consist of:

kube-apiserver - The API server is the front end for the Kubernetes control plane. It exposes the Kubernetes API.
etcd - A consistent, highly available key-value store. It is Kubernetes’ backing store for all API data.
kube-scheduler - Watches for newly created Pods with no assigned node and selects a node for them to run on.
kube-controller-manager - Runs a controller to implement the control loop.
cloud-controller-manager - Runs controllers that interact with the underlying cloud providers.

The node components run on every node in the cluster. They provide the runtime environment for the containers.

kubelet - An agent that runs on each node in the cluster. It ensures that pods are running.
kube-proxy - Maintains network rules on nodes.
Container Runtime - Is responsible for running containers.

The Kubernetes API is the central component in a Kubernetes cluster. It is the interface to the cluster and the only way to work with Kubernetes objects.

A Kubernetes object is a “record of intent”. It represents the desired state of the workload running on the cluster. The whole Kubernetes system will continuously work to ensure that the object exists.

YAML Manifests

If you are long enough exposed to Kubernetes, you will have seen a lot of YAML files. You can define many Kubernetes objects in a YAML file. These include Deployments, Services, ConfigMaps, Secrets, and more.

Let’s have a look at an example of a Deployment object in YAML for an Nginx web server:

Example

apiVersion: apps/v1
kind: Deployment
metadata:
 name: nginx-deployment
spec:
 replicas: 1
 selector:
 matchLabels:
 app: nginx
 template:
 metadata:
 labels:
 app: nginx
 spec:
 containers:
 - name: "nginx:latest"
 image: nginx

Pros and Cons of YAML

Pros

Simple: Kubernetes YAML files are easy to read and write.
The standard: YAML is the standard way to define Kubernetes objects. It is a first-class citizen in the kubectl command-line tool.

Cons

Multiple environments: If you have multiple environments, like development, staging, and production, you need to maintain multiple YAML files.
Too manual: You must create and update the YAML files by hand. Also, you must track changes across many files.
Not DRY: You cannot use the DRY principle (Don’t Repeat Yourself) in YAML files. If you have the same configuration in many files, you need to update all of them by hand.
No real abstraction: The advantage of a static configuration file is also its disadvantage. You cannot use dynamic values in the YAML files.
Need for tools from the ecosystem: You need to use tools like helm or kustomize to manage the complexity of many YAML files, environments, and templating.

To sum it up, YAML is a good choice for simple Kubernetes objects and small projects. But, if you have a complex project, and to be honest, most are, you need a better way to manage your Kubernetes resources.

Or to rephrase it: Friends don’t let friends manage Kubernetes resources with YAML.

Why should you avoid YAML? Credit: No YAML!

Terraform Kubernetes Provider

Terraform is an Infrastructure as Code tool. It lets you define and manage your infrastructure in a declarative way. It uses a high-level configuration language called HashiCorp Configuration Language (HCL). To manage Kubernetes objects with Terraform, use the Kubernetes provider. It allows you to define Kubernetes objects in Terraform HCL.

The Kubernetes provider allows you to define Kubernetes objects like kubernetes_deployment, kubernetes_service, kubernetes_config_map, kubernetes_secret, and more in code, leveraging the language features of Terraform. You can use, to a certain extent, loops, conditions, and variables in a way that is not possible with plain YAML files.

Let’s have a look at an example of a Deployment object in Terraform HCL for an Nginx web server:

Example

resource "kubernetes_deployment" "nginx" {
 metadata {
 name = "nginx-deployment"
 }

 spec {
 replicas = 1

 selector {
 match_labels = {
 app = "nginx"
 }
 }

 template {
 metadata {
 labels = {
 app = "nginx"
 }
 }

 spec {
 container {
 image = "nginx:latest"
 name = "nginx"
 }
 }
 }
 }
}

This is a very simple example of a Deployment object in Terraform HCL. Let’s add some complexity to the example by adding variables and loops:

Example with Variables and Loops

variable "namespace" {
 description = "The namespace to deploy the resources"
 type = string
 default = "default"
}

variable "nginx_image" {
 description = "The Nginx image to use"
 type = string
 default = "nginx:latest"
}

variable "replicas" {
 description = "The number of replicas to deploy"
 type = number
 default = 1
}

variable "number_of_deployments" {
 description = "The number of deployments to create"
 type = number
 default = 2
}

locals {
 nginx_image = var.nginx_image
 replicas = var.replicas
 namespace = var.namespace
 number_of_deployments = var.number_of_deployments
}

resource "kubernetes_deployment" "nginx" {
 for_each = {for i in range(local.number_of_deployments) : i => i}

 metadata {
 name = "nginx-deployment-${each.key}"
 namespace = local.namespace
 }

 spec {
 replicas = local.replicas
 selector {
 match_labels = {
 app = "nginx-${each.key}"
 }
 }

 template {
 metadata {
 labels = {
 app = "nginx-${each.key}"
 }
 }

 spec {
 container {
 image = local.nginx_image
 name = "nginx"
 }
 }
 }
 }
}

In this example, we can already see the advantage of Terraform over plain YAML files. We can use variables and loops to define multiple kubernetes_deployment objects in a more dynamic way.

Pros and Cons of Terraform

Pros

Dynamic: You can use variables, loops, and conditions in Terraform HCL. This lets you define your Kubernetes objects in a more dynamic way.
Abstraction: You can apply the DRY principle in Terraform HCL by defining and reusing modules across projects.
Environment management: By defining variables, you can manage multiple environments, like development, staging, and production. This is a more structured way to do it, without needing extra tools.

Cons

Complexity: Terraform is more complex than plain YAML files. You must learn Terraform’s HCL to define your Kubernetes objects.
DSL: Terraform is a Domain-Specific Language (DSL). You need to learn it to define your Kubernetes objects.
Testability: Testing Terraform code is harder. You need tools like Terratest and Kitchen-Terraform for it.
Helm Support: Terraform does not support Helm charts out of the box. You need to use the Helm provider to manage them with Terraform. This can add complexity to your Terraform code, and you need to keep track of the Helm provider dependency.
IDE Support: Terraform lacks the same IDE support as YAML files. You need plugins, like the Terraform Extension for Visual Studio Code, for syntax highlighting and autocompletion.

Terraform is a good choice for managing your Kubernetes resources if you have a complex project and you do not mind the complexity of the Terraform language. It gives you more flexibility and abstraction than plain YAML files, and you can manage multiple environments in a more structured way.

But is there a better way to manage your Kubernetes resources? Let’s have a look at Pulumi.

Pulumi and the Power of General Purpose Languages

Last but not least, let’s have a look at Pulumi. Pulumi is an Infrastructure as Code tool. It lets you define and manage your Kubernetes objects using general-purpose programming languages. Pulumi supports multiple languages like Python, Go, TypeScript, JavaScript, .NET, and JVM-based languages. This gives you all the benefits of a full programming language. It includes native IDE support, code completion, and debugging.

You need to install the Pulumi CLI. Then, create a new Pulumi project. Finally, add the Kubernetes library to it. It depends on the language you choose. You could use a package manager, like npm or pip.

Let’s have a look at an example of what a Deployment object would look like in all the supported languages.

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

const nginxDeployment = new k8s.apps.v1.Deployment("nginx-deployment", {
 metadata: {
 name: "nginx-deployment",
 },
 spec: {
 replicas: 1,
 selector: {
 matchLabels: {
 app: "nginx",
 },
 },
 template: {
 metadata: {
 labels: {
 app: "nginx",
 },
 },
 spec: {
 containers: [
 {
 name: "nginx",
 image: "nginx:latest",
 },
 ],
 },
 },
 },
});

import pulumi
import pulumi_kubernetes as k8s

nginx_deployment = k8s.apps.v1.Deployment(
 "nginx-deployment",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="nginx-deployment",
 ),
 spec=k8s.apps.v1.DeploymentSpecArgs(
 replicas=1,
 selector=k8s.meta.v1.LabelSelectorArgs(
 match_labels={
 "app": "nginx",
 },
 ),
 template=k8s.core.v1.PodTemplateSpecArgs(
 metadata=k8s.meta.v1.ObjectMetaArgs(
 labels={
 "app": "nginx",
 },
 ),
 spec=k8s.core.v1.PodSpecArgs(
 containers=[
 k8s.core.v1.ContainerArgs(
 name="nginx",
 image="nginx:latest",
 ),
 ],
 ),
 ),
 ),
)

package main

import (
 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apps/v1"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 _, err := v1.NewDeployment(ctx, "nginxDeployment", &v1.DeploymentArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("nginx-deployment"),
 },
 Spec: &v1.DeploymentSpecArgs{
 Replicas: pulumi.Int(1),
 Selector: &metav1.LabelSelectorArgs{
 MatchLabels: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 },
 Template: &corev1.PodTemplateSpecArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: pulumi.StringMap{
 "app": pulumi.String("nginx"),
 },
 },
 Spec: &corev1.PodSpecArgs{
 Containers: corev1.ContainerArray{
 &corev1.ContainerArgs{
 Name: pulumi.String("nginx"),
 Image: pulumi.String("nginx:latest"),
 },
 },
 },
 },
 },
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using Pulumi;
using Pulumi.Kubernetes.Types.Inputs.Core.V1;
using Pulumi.Kubernetes.Types.Inputs.Apps.V1;
using Pulumi.Kubernetes.Types.Inputs.Meta.V1;
using System.Threading.Tasks;
using Deployment = Pulumi.Deployment;

class MyStack : Stack
{
 public MyStack()
 {
 var nginxDeployment = new Pulumi.Kubernetes.Apps.V1.Deployment("nginxDeployment", new DeploymentArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Name = "nginx-deployment",
 },
 Spec = new DeploymentSpecArgs
 {
 Replicas = 1,
 Selector = new LabelSelectorArgs
 {
 MatchLabels =
 {
 { "app", "nginx" }
 }
 },
 Template = new PodTemplateSpecArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Labels =
 {
 { "app", "nginx" }
 }
 },
 Spec = new PodSpecArgs
 {
 Containers =
 {
 new ContainerArgs
 {
 Name = "nginx",
 Image = "nginx:latest",
 }
 }
 }
 }
 }
 });
 }
}

class Program
{
 static Task<int> Main() => Deployment.RunAsync<MyStack>();
}

package myproject;

import com.pulumi.Pulumi;
import com.pulumi.kubernetes.apps.v1.Deployment;
import com.pulumi.kubernetes.apps.v1.DeploymentArgs;
import com.pulumi.kubernetes.apps.v1.inputs.DeploymentSpecArgs;
import com.pulumi.kubernetes.core.v1.inputs.ContainerArgs;
import com.pulumi.kubernetes.core.v1.inputs.PodSpecArgs;
import com.pulumi.kubernetes.core.v1.inputs.PodTemplateSpecArgs;
import com.pulumi.kubernetes.meta.v1.inputs.LabelSelectorArgs;
import com.pulumi.kubernetes.meta.v1.inputs.ObjectMetaArgs;

import java.util.Map;

public class App {
 public static void main(String[] args) {
 Pulumi.run(ctx -> {
 var nginxDeployment = new Deployment("nginxDeployment", DeploymentArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .name("nginx-deployment")
 .build())
 .spec(DeploymentSpecArgs.builder()
 .replicas(1)
 .selector(LabelSelectorArgs.builder()
 .matchLabels(Map.of("app", "nginx"))
 .build())
 .template(PodTemplateSpecArgs.builder()
 .metadata(ObjectMetaArgs.builder()
 .labels(Map.of("app", "nginx"))
 .build())
 .spec(PodSpecArgs.builder()
 .containers(ContainerArgs.builder()
 .name("nginx")
 .image("nginx:latest")
 .build())
 .build())
 .build())
 .build())
 .build());
 });
 }

Now have a look into the more complex example with variables and loops in Pulumi. I am using TypeScript as an example, but you can of course use any of the Pulumi supported languages.

Example with Variables and Loops

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

let config = new pulumi.Config();

const namespace = config.get("namespace") || "namespace";
const nginxImage = config.get("nginxImage") || "nginx:latest";
const replicas = config.getNumber("replicas") || 1;
const numberOfDeployments = config.getNumber("numberOfDeployments") || 1;

for (let i = 0; i < numberOfDeployments; i++) {
 new k8s.apps.v1.Deployment(`nginxDeployment-${i}`, {
 metadata: {
 name: `nginx-deployment-${i}`,
 namespace: namespace,
 },
 spec: {
 replicas: replicas,
 selector: {
 matchLabels: {
 app: `nginx-${i}`,
 },
 },
 template: {
 metadata: {
 labels: {
 app: `nginx-${i}`,
 },
 },
 spec: {
 containers: [
 {
 name: "nginx",
 image: nginxImage,
 },
 ],
 },
 },
 },
 });
}

In this example, you can see how easy it is to define multiple Deployment objects with variables and loops syntax provided by TypeScript. This is a big advantage over plain YAML files and Terraform HCL. Blending the full power of a general-purpose programming language with the Kubernetes API is a natural process. Consider the possibilities of creating reusable functions, classes, and modules. You could share them across multiple projects and teams.

You might also like:

Pros and Cons of Pulumi

Pros

You can use general-purpose languages to define your Kubernetes objects. These include Python, Go, TypeScript, JavaScript, .NET, and JVM-based languages.

Full IDE support: You get full IDE support. It includes syntax highlighting, code completion, and debugging.
Testability: You can use the full power of your programming language to test your Pulumi code.
Abstraction: You can use the DRY principle in Pulumi. Define reusable functions, classes, and modules in your chosen programming language.
Environment management: You can manage multiple environments, like development, staging, and production, by defining config variables. This is a more structured way, without needing extra tools.
Kubernetes Helm Support: Pulumi supports Helm charts out of the box. You can manage Helm charts with Pulumi without needing extra tools or providers.
Programmatic execution of Pulumi: You can further blend Pulumi code into your projects using the Pulumi Automation API.

Cons

Runtime dependency: You must have the runtime of your chosen programming language installed on your machine, or use a containerised version of it in your CI/CD pipeline.
Follow best practices and compliance: Set up a code review process. It should ensure the Pulumi code follows the best practices of your chosen programming language and your organization’s additional best practices.
No possibility to convert one Pulumi language to another: If you start with one language, you can’t convert the Pulumi code to another language. You need to rewrite the Pulumi code in the new language.

YAML vs. Terraform vs. Pulumi - Results

Feature	YAML	Terraform	Pulumi
Readability	✅ Simple and human-readable	⚠️ Requires learning HCL	✅ Uses familiar programming languages
Environment Management	❌ Requires manual management of multiple files	✅ Structured management via variables	✅ Structured management via config variables
DRY Principle	❌ No reusability, requires manual updates	✅ Supports modules and reusable code	✅ Full support via functions, classes, modules
Dynamic Configuration	❌ Static configuration only	✅ Supports loops, conditions, variables	✅ Full programming language capabilities
IDE Support	✅ Good support	⚠️ Limited, requires plugins	✅ Full language-native IDE support
Testing	❌ Limited testing options	⚠️ Requires specialized tools (Terratest)	✅ Native language testing frameworks
Helm Support	✅ Native support	⚠️ Requires additional provider	✅ Built-in support
Learning Curve	✅ Low	⚠️ Medium (new DSL)	⚠️ Varies by language knowledge
Dependencies	✅ Minimal	⚠️ Terraform installation	❌ Language runtime required
Ecosystem Tools	❌ Requires additional tools (Helm, Kustomize)	✅ Integrated tooling	✅ Integrated tooling
Abstraction	❌ Limited	✅ Good	✅ Excellent

Legend:

✅ Good/Supported
⚠️ Partial/With limitations
❌ Poor/Not supported

Time to say Goodbye to YAML?

So, what’s the smart choice for deployment automation with Kubernetes? Is YAML still the best choice for Kubernetes? Or, is there a better way to manage your resources? I understand if you are a YAML fan. Despite complaints, it is still the most popular way to define Kubernetes objects. Many folks feel a sense of accomplishment when their YAML file works on the first try.

If you have a complex project, you will soon hit the limits of YAML. You want to manage your Kubernetes resources in a more structured, dynamic way while following best practices and compliance. Then, you keep adding more complexity to your YAML files. You do this by using tools like helm and kustomize to fix YAML’s flaws. Or, you adopt a hybrid approach with jsonnet, cuelang, or ytt. This makes it even more complex.

Giving up on YAML is not easy, but it does not mean that you have to give up on control. You can still have full control by using Terraform or Pulumi.

Conclusion

Now it’s time to make a decision. What’s the smart choice for deployment automation with Kubernetes? Is it YAML, Terraform, or Pulumi? For me, the smart choice is Pulumi. It gives you all the benefits of a general-purpose programming language. You gain full control, a more streamlined workflow, and the power to define all your infrastructure needs as code beyond Kubernetes. Especially on complex projects, use Pulumi. It’s better than dealing with many YAML files and the complexity of Terraform HCL.

Next time you start a new project, try Pulumi. See for yourself that it’s a smart choice. It’s an upgrade to your productivity and happiness.

But that’s my opinion. What’s your opinion? What’s your smart choice for deployment automation with Kubernetes?

If you want to try out Pulumi or play around to prove me wrong, you can start with:

Get Started with Pulumi

Use Pulumi's open-source SDK to create, deploy, and manage infrastructure on any cloud.

Pulumi's Latest Kubernetes Updates from KubeCon North America

Gavin Johnson — Tue, 12 Nov 2024 08:00:00 +0000

Pulumi is excited to be at KubeCon North America this week, the premier event for all things Kubernetes and cloud-native. KubeCon is the gathering place for developers, enterprises, and cloud native experts to meet and further the education and advancement of Kubernetes and cloud native computing. At Pulumi, we are strongly committed to Kubernetes and continue to support the ecosystem with infrastructure management solutions that empower teams to automate, secure, and manage Kubernetes at scale.

Come See Us at KubeCon

Stop by the Pulumi KubeCon booth R1 this week to chat with experts on the Pulumi team. If you can’t make it to KubeCon, join our workshop on November 18, 2024, to see how easy it is to manage Kubernetes with Pulumi.

Recent Pulumi Kubernetes Updates

We’ve launched several new and improved capabilities recently to serve Kubernetes users and to make it easier than ever to automate, secure, manage, and scale clusters and workloads.

Pulumi Kubernetes Operator 2.0

The Pulumi Kubernetes Operator automates the deployment and management of infrastructure by running Pulumi programs directly in Kubernetes clusters, enabling teams to manage cloud resources alongside Kubernetes-native resources. The Pulumi Kubernetes Operator 2.0 is a significant upgrade that introduces dedicated “workspace” pods for each Stack resource, effectively isolating each stack’s compute and memory resources, improving the isolation of secrets, and opening up new customization options. The operator now scales horizontally, enhancing performance and enabling teams to manage complex Kubernetes setups with greater reliability.

Pulumi ESC Secrets in Kubernetes with the External Secrets Operator

Pulumi ESC integrates with the External Secrets Operator (ESO) to securely access secrets from Pulumi ESC in Kubernetes. This enables a secure, automated way to manage and use secrets from Pulumi ESC across Kubernetes clusters, reducing the need for manual secrets maintenance and potential security risks from the local storage of secrets in manifest files.

Pulumi EKS Provider Version 3.0.0

The Pulumi EKS Provider Version 3.0.0 simplifies Amazon Elastic Kubernetes Service (EKS) cluster management with improvements in flexibility and security and the introduction of new features to enhance the Kubernetes experience on AWS. This update allows users to more easily customize networking, scaling, and security configurations for EKS clusters, streamlining deployment and management workflows.

Kubernetes-Native Support for Customer-Managed Pulumi Deployments Agents

Kubernetes-native support for customer-managed Pulumi Deployments agents allows organizations to host Pulumi Deployments agents within their Kubernetes environments enhancing the flexibility and control they have over their infrastructure deployments. Kubernetes-native support offers greater flexibility, scalability, and security for self-hosted Pulumi Deployments agents.

Improved Kubernetes await Logic in the Pulumi Kubernetes Provider

Improved Kubernetes await logic in the Pulumi Kubernetes provider enables more reliable handling of resource dependencies and synchronization when managing Kubernetes resources. This update makes it easier to ensure that resources are created or updated in the correct order, improving the overall predictability and stability of Kubernetes deployments.

Helm Chart v4 Resource in the Pulumi Kubernetes Provider

The Helm Chart v4 resource in the Pulumi Kubernetes provider introduces several new capabilities for users managing Helm charts. It allows for more sophisticated chart management, including enhanced support for dependencies between charts, applying transformations to chart resources, and the ability to handle Kubernetes namespaces more flexibly.

Revamped crd2pulumi

crd2pulumi is a CLI tool that generates strongly-typed Pulumi resources from Kubernetes CustomResourceDefinitions (CRDs), enabling better IDE support, type safety, and autocompletion for managing complex CRDs. It was revamped to enhance its handling of CRDs with multiple versions, refine the structure of its generated code, and offer more flexible output paths for smoother integration into Pulumi programs.

Learn about the crd2pulumi and its recent updates

Kubecon NA 2021 Roundup

Kat Cosgrove — Mon, 08 Nov 2021 22:50:38 +0000

KubeCon North America 2021 is over, but the recordings are now online! Every talk you wanted to attend and couldn’t is available on YouTube, so here’s some highlights—cloud native trends, updates from projects and SIGs, and a few of my favorite talks!

SBOM is Coming: Why You Should Care and How You Can Help - Frederick Kautz & Allan Friedman

A major trend throughout the conference was supply chain security and software bill of materials (SBOMs). There were quite a few talks on these subjects, but Frederick Kautz and Allan Friedman give a great overview of exactly what an SBOM is, and why it’s important.

Notary: State of the Container Supply Chain - Justin Cormack, Docker & Steve Lasker, Microsoft

On supply chain security, Steve Lasker presents updates to Notary v2. It’s changed quite a bit since v1 and there are more changes on the horizon. With the rising concerns surrounding supply chain security, I expect Notary and tools like it to be increasingly important. No worries if you aren’t already familiar with Notary—Steve explains it well.

A Vulnerable Tale About Burnout - Julia Simon, CloudOps

Taking things to a more human place, burnout was a common theme this year. It’s something that we don’t talk about as much as we should, but close to two years of isolation is forcing us all to confront it in ways we may not have expected. This talk by Julia Simon is important to watch not just so that you can learn to recognize these problems in yourself, but so that you can recognize them in your peers.

PodSecurityPolicy Replacement: Past, Present, and Future - Tabitha Sable & Tim Allclair

PodSecurityPolicy (PSP) has been deprecated. Many of you may be relying on this, and deprecations frequently sound scary, but Tabitha Sable and Tim Allclair are here to soothe your fears. They go to great lengths explaining what PSP is, why it was deprecated, what exactly this means for people relying on it, and where we’re going next.

SIG Contributor Experience Deep Dive - Alison Dowdney, Weaveworks & Christoph Blecker, Red Hat

An open source project lives or dies by its contributors, both technical and non-technical. Kubernetes is huge and requires a lot of contributors to keep it growing, so we have SIG Contributor Experience to make sure they stay happy and the community stays healthy. Alison Dowdney and Christoph Blecker go into just how SIG Contributor Experience operates, plus updates. If you’re in a leadership position for an open source project, there are some lessons to be learned from this talk.

Kubernetes Exposed! Seven of Nine Hidden Secrets That Will Give You… Ian Coldwater & Brad Geesaman

No Kubecon is complete without Ian Coldwater, this time joined by Brad Geesaman (and, briefly, a second Ian Coldwater) to go into some not-so-obvious but oh-so-important pitfalls and hidden security concerns within Kubernetes. Even if you caught this talk live, it’s worth watching again to make sure you’re secured against these issues, or you might find a goose haunting your cluster.

Beyond Kubernetes Security - Ellen Körbes, Tilt & Tabitha Sable, Datadog

Finally, it would be remiss of me to not mention Ellen and Tabitha’s cinematic contribution to this year’s Kubecon North America. Beyond Kubernetes Security is the spiritual successor to their talk from the last Kubecon Europe, once again going into how some security concepts and exploits work in Kubernetes, but presented as a short film involving crime, time travel, breakups, and Ellen’s collection of old electronics.

The quality of the content was incredible across the board this year, whether the speakers were live or presenting virtually, forcing some difficult decisions around selecting my schedule for talks to attend and how many I could feasibly put in this blog before it got way too long. Fortunately, all of them are on the CNCF YouTube channel now, including talks from the various co-located events. Dig through!

Kubecon North America was simultaneously a strange and familiar event. It’s been nearly two years since most of us have been at a conference, and while that certainly made things awkward at times while we all readjusted to seeing other humans in the flesh, our slow return to something vaguely familiar seems to have been a success. I sincerely hope to see you all in person at the next Kubecon EU in Valencia.

Honk <3,

Kat