How to Run AI Agents on Kubernetes with Pulumi

Kubernetes has become the default place teams run agentic AI workloads: CNCF’s 2026 annual survey found that 66% of organizations hosting generative AI models use Kubernetes to manage some or all of their inference workloads.¹ An entire ecosystem has grown up around that fact — agent runtimes, model servers, GPU schedulers — and most of it assumes the infrastructure underneath is already handled. It usually isn’t. An AI agent is not a stateless web service, and provisioning for one takes more than copying a Deployment YAML and swapping the image.

I spend a lot of my time these days thinking about what changes when the thing consuming your infrastructure isn’t a person or a fixed pipeline, but an agent making its own calls about what to do next — and Kubernetes is where I keep seeing that question show up first, because it’s already where most teams run everything else. This post is about that gap: what changes about Kubernetes infrastructure when the workload is an agent, what the current agentic-Kubernetes ecosystem actually looks like, and how to provision and govern that infrastructure with Pulumi using TypeScript and Python — plus where Pulumi Neo fits once the infrastructure itself has to answer to an agent. If you haven’t read our take on the broader shift toward agentic infrastructure, What Is Agentic Infrastructure? is a good companion piece; this post stays specific to the Kubernetes layer.

What makes agentic AI workloads different from ordinary Kubernetes workloads?

An agent’s resource and governance profile looks nothing like a typical web service, largely because it schedules accelerators, holds state across long sessions, and reaches out to systems a normal Deployment never touches.

None of this means Kubernetes is the wrong place to run agents — the opposite: its scheduler, autoscaling, and RBAC primitives are exactly the right building blocks for these problems. It means the infrastructure around an agent deserves more deliberate design than “reuse the Deployment template,” particularly around GPU scheduling, secrets, and the blast radius of what the agent is allowed to do.

What does the agentic Kubernetes stack look like in 2026?

There isn’t one “agents on Kubernetes” product; there’s a stack, and most of it is open source and CNCF-adjacent. Being fair to that ecosystem matters here, because Pulumi’s job is to provision and govern the infrastructure underneath these tools, not to replace them.

kagent is worth a closer look since it’s the piece most directly aimed at “run an agent on Kubernetes.” As of its current release (v0.9.9), it defines its own custom resources — Agent, ModelConfig, ModelProviderConfig, RemoteMCPServer, and SandboxAgent — and ships an MCP server with built-in tools for Kubernetes, Helm, Istio, Argo, and Prometheus, plus the ability to connect to any external MCP server via the RemoteMCPServer CRD.⁵ It’s a genuinely useful abstraction for defining what an agent can do; it isn’t, on its own, an answer for how the cluster the agent runs on got there, who’s allowed to change it, or how secrets get to it safely. That’s the layer Pulumi sits at.

How do you provision Kubernetes infrastructure for AI agents with Pulumi?

A minimal but representative path from “no cluster” to “an agent running on it, governed” looks like this:

1. Provision a Kubernetes cluster with a GPU-enabled node group sized for your model workload.
2. Install the kagent CRDs and controller onto the cluster using a Pulumi Helm chart resource.
3. Store the model provider’s API key as a Pulumi secret sourced from a Pulumi ESC environment, rather than a plaintext Kubernetes Secret.
4. Declare the agent itself as a custom resource so it’s versioned and reviewed the same way as everything else in the stack.
5. Attach a policy pack so specific classes of change — like exposing a service publicly, or granting a service account cluster-admin — require an explicit review.
6. Roll changes out through CI, or through the Pulumi Kubernetes Operator if you want the cluster itself to reconcile against a Pulumi stack.

Steps 1–4 in TypeScript:

import * as pulumi from "@pulumi/pulumi";
import * as kubernetes from "@pulumi/kubernetes";
import * as eks from "@pulumi/eks";

// 1. GPU-enabled node group alongside your existing EKS cluster.
const gpuNodeGroup = new eks.ManagedNodeGroup("agent-gpu-nodes", {
    cluster: eksCluster,
    nodeRole: nodeRole,
    instanceTypes: ["g6.8xlarge"],
    gpu: true,
    scalingConfig: { minSize: 1, desiredSize: 2, maxSize: 4 },
});

const k8sProvider = new kubernetes.Provider("k8s", {
    kubeconfig: eksCluster.kubeconfig,
});

// 2. kagent CRDs and controller via the Helm v4 Chart resource.
const kagentCrds = new kubernetes.helm.v4.Chart("kagent-crds", {
    chart: "oci://ghcr.io/kagent-dev/kagent/helm/kagent-crds",
    namespace: "kagent",
}, { provider: k8sProvider });

const kagent = new kubernetes.helm.v4.Chart("kagent", {
    chart: "oci://ghcr.io/kagent-dev/kagent/helm/kagent",
    namespace: "kagent",
    values: {
        providers: { default: "openAI" },
    },
}, { provider: k8sProvider, dependsOn: [kagentCrds] });

// 3. Model provider API key, sourced as a secret from a Pulumi ESC
// environment (declared in Pulumi.<stack>.yaml, not hardcoded here).
const config = new pulumi.Config();
const openAiApiKey = config.requireSecret("openAiApiKey");

const providerSecret = new kubernetes.core.v1.Secret("openai-credentials", {
    metadata: { namespace: "kagent" },
    stringData: { apiKey: openAiApiKey },
}, { provider: k8sProvider });

// 4. The agent itself, as a versioned custom resource.
const agent = new kubernetes.apiextensions.CustomResource("k8s-ops-agent", {
    apiVersion: "kagent.dev/v1alpha2",
    kind: "Agent",
    metadata: { namespace: "kagent" },
    spec: {
        description: "Answers questions about cluster state and proposes fixes",
        modelConfig: "openai-gpt",
        tools: ["kubernetes", "helm"],
    },
}, { provider: k8sProvider, dependsOn: [kagent] });

The same four steps in Python:

import pulumi
import pulumi_eks as eks
import pulumi_kubernetes as kubernetes
from pulumi_kubernetes import apiextensions

# 1. GPU-enabled node group alongside your existing EKS cluster.
gpu_node_group = eks.ManagedNodeGroup("agent-gpu-nodes",
    cluster=eks_cluster,
    node_role=node_role,
    instance_types=["g6.8xlarge"],
    gpu=True,
    scaling_config={"min_size": 1, "desired_size": 2, "max_size": 4})

k8s_provider = kubernetes.Provider("k8s", kubeconfig=eks_cluster.kubeconfig)

# 2. kagent CRDs and controller via the Helm v4 Chart resource.
kagent_crds = kubernetes.helm.v4.Chart("kagent-crds",
    chart="oci://ghcr.io/kagent-dev/kagent/helm/kagent-crds",
    namespace="kagent",
    opts=pulumi.ResourceOptions(provider=k8s_provider))

kagent = kubernetes.helm.v4.Chart("kagent",
    chart="oci://ghcr.io/kagent-dev/kagent/helm/kagent",
    namespace="kagent",
    values={"providers": {"default": "openAI"}},
    opts=pulumi.ResourceOptions(provider=k8s_provider, depends_on=[kagent_crds]))

# 3. Model provider API key, sourced as a secret from a Pulumi ESC
# environment (declared in Pulumi.<stack>.yaml, not hardcoded here).
config = pulumi.Config()
openai_api_key = config.require_secret("openAiApiKey")

provider_secret = kubernetes.core.v1.Secret("openai-credentials",
    metadata={"namespace": "kagent"},
    string_data={"apiKey": openai_api_key},
    opts=pulumi.ResourceOptions(provider=k8s_provider))

# 4. The agent itself, as a versioned custom resource.
agent = apiextensions.CustomResource("k8s-ops-agent",
    api_version="kagent.dev/v1alpha2",
    kind="Agent",
    metadata={"namespace": "kagent"},
    spec={
        "description": "Answers questions about cluster state and proposes fixes",
        "modelConfig": "openai-gpt",
        "tools": ["kubernetes", "helm"],
    },
    opts=pulumi.ResourceOptions(provider=k8s_provider, depends_on=[kagent]))

The Pulumi.<stack>.yaml for the stack pulls the API key from an ESC environment instead of a plaintext value in source control:

environment:
  - agents-team/production

Verify the exact CRD group/version (kagent.dev/v1alpha2 above) against kagent’s own CRD manifests before you ship this in production — CRD APIs move faster than blog posts do.

Why write agent infrastructure in a general-purpose language instead of templated YAML?

Because an agent’s infrastructure has exactly the kind of conditional, repeated, testable shape that general-purpose languages are built for, and templated YAML is not. A for loop that provisions one GPU node pool per region, a function that returns a standard “agent + its guardrail policy” bundle, a unit test that asserts a policy pack actually blocks an over-privileged service account before it ships — all of that is native to TypeScript, Python, Go, C#, and Java, and bolted-on at best in Helm templates or raw manifests.

This isn’t a knock on Helm or Kustomize, which remain genuinely good tools for packaging and distributing charts — including kagent’s own install, which is a Helm chart for a reason. The distinction is what you’re doing with the result. Distributing a reusable, versioned package is a job Helm is built for. Composing that package into your specific cluster, wiring its secrets, gating its rollout with policy, and testing the whole thing before it ships is software engineering, and it benefits from a real language: loops, functions, types, unit tests, and the same CI/CD your application code already runs through.

How do you keep agent workloads governed, least-privileged, and cost-aware?

An agent that can act on infrastructure needs infrastructure-shaped guardrails, not just a service account and a prayer. Three pieces of the Pulumi platform apply directly:

• Policy as code. Pulumi Policies let you write rules — in the same general-purpose languages, not a separate policy DSL — that run before a change is applied: block a Deployment requesting cluster-admin, require GPU node pools to carry a cost-center label, or flag any RemoteMCPServer pointed at a host outside an allowlist.
• Pulumi ESC for secrets and configuration, so model-provider API keys and other credentials are centrally managed, rotated, and scoped, rather than copy-pasted into Secret manifests across every cluster an agent touches.
• RBAC scoped to the agent’s actual job. The same custom-resource pattern used to declare the agent (shown above) is the natural place to also declare its ServiceAccount, Role, and RoleBinding — as code, reviewed the same way as the agent’s own configuration, not layered on afterward.

I want to be precise here about Pulumi Insights, because I see the two products get conflated more than almost anything else we ship: Insights gives you discovery (a searchable inventory of resources across Pulumi, Terraform, CloudFormation, and manually created infrastructure) and policy enforcement, including pre-built compliance packs. Cost-optimization recommendations are a Neo capability, not an Insights one.

Where does Pulumi Neo fit when the infrastructure itself is agentic?

Neo is Pulumi’s own infrastructure agent, and it’s worth being specific about what it actually does today rather than leaning on the general “AI for infrastructure” framing. Documented capabilities include answering questions about your existing infrastructure, proposing changes and opening a pull request for them, running a preview before anything lands, reviewing PRs, and taking on recurring maintenance work using your team’s established Pulumi practices.⁶ The Infrastructure AI docs cover each of these in more depth.

The thing I care about most with Neo is trust, and trust gets earned one reviewed pull request at a time, not granted upfront. That’s why every capability starts human-in-the-loop and only earns more autonomy once the guardrails have proven themselves in practice — the same discipline this post argues for everywhere else an agent touches your Kubernetes infrastructure. Pulumi’s own product blog has also described Neo handling things like Terraform-to-Pulumi migrations and containerizing and migrating a service onto Kubernetes.⁷

The relevant framing for this post: Neo doesn’t replace kagent, KServe, or any other workload-level agent framework running inside your cluster — it operates one layer up, on the infrastructure that provisions and governs the cluster itself. If your team is running kagent-based agents on Kubernetes, Neo is the thing that can propose the PR that adds a new GPU node pool, review the policy-pack change that locks down a RemoteMCPServer, or handle the maintenance toil around keeping that stack current.

Frequently asked questions

How do you deploy AI agents on Kubernetes?

Provision a cluster (with GPU node pools if your agent needs them), install an agent runtime — kagent is one of the more Kubernetes-native options as of 2026 — and declare the agent as a custom resource alongside its model configuration and any MCP tool servers it needs. Doing this with Pulumi means the cluster, the runtime install, the secrets, and the agent definition are all one reviewable, testable stack rather than a mix of kubectl apply commands run by hand.

Do you need GPUs to run AI agents on Kubernetes?

Not always. An agent that only calls a hosted model API (OpenAI, Anthropic, and similar) needs CPU nodes to run its own logic and tool calls; the GPU-heavy work happens on the provider’s infrastructure, not yours. GPUs matter once you’re self-hosting inference — running an open-weight model via vLLM, KServe, or Ray Serve — which is where projects like Kueue, LeaderWorkerSet, and Dynamic Resource Allocation come in.

How is kagent different from running an agent as a plain Deployment?

A plain Deployment gives you a running process; it doesn’t give you a standard way to describe an agent’s model configuration, its tool access, or its connections to MCP servers as first-class, versionable objects. kagent’s CRDs (Agent, ModelConfig, ModelProviderConfig, RemoteMCPServer, SandboxAgent) exist specifically to make those concepts declarative, so a policy engine, a reviewer, or another automation can reason about them the same way it reasons about a Deployment or a Service.

Can Pulumi manage Kubernetes resources that Helm or Terraform already created?

Yes. Pulumi Insights discovers and inventories resources regardless of what provisioned them — Pulumi, Terraform, CloudFormation, or manual changes — which is useful precisely because most clusters running agent workloads already have Helm-installed components like kagent, an ingress controller, or a GPU operator on them. You don’t need to rip those out to bring the rest of the stack under Pulumi; kubernetes.yaml.v2.ConfigFile and ConfigGroup can also adopt existing manifests directly into a Pulumi program.

What are Kubernetes best practices for AI workloads in 2026?

Match node pools to the workload (GPU pools for self-hosted inference, CPU pools for orchestration and tool-calling), use a queueing layer like Kueue for batch and training jobs so they don’t starve latency-sensitive agent traffic, keep model-provider credentials in a secrets manager (not plaintext Secret objects), and scope every agent’s RBAC to the specific tools it’s allowed to call. For general Kubernetes hygiene beyond the AI-specific pieces, see Kubernetes Best Practices I Wish I Had Known Before.

How does Pulumi Neo help manage Kubernetes infrastructure for AI agents?

Neo operates at the infrastructure layer underneath your agent workloads: it can answer questions about the state of your cluster, propose infrastructure changes as a reviewable pull request, run a preview before anything is applied, review incoming PRs, and take on the recurring maintenance that keeps a Kubernetes-plus-agents stack current — all using your team’s existing Pulumi practices rather than a separate workflow.

Where to go next

If you’re starting from zero, the Kubernetes get-started guide and the Kubernetes provider registry cover the provisioning basics this post builds on. If you’re already running agent workloads and want the governance layer, start with Pulumi Policies and Pulumi ESC. And if agent sprawl — not just agentic Kubernetes workloads, but the broader proliferation of agents across your org — is the more pressing problem, Agent Sprawl Is Here. Your IaC Platform Is the Answer. is the companion read.