Pulumi Blog: Pulumi

Why Choose Pulumi Over Terraform?

Pablo Seibelt — Tue, 02 Jun 2026 00:00:00 +0000

Terraform is a proven infrastructure as code tool with a large provider and module ecosystem. Many teams choose Pulumi when they want to keep that infrastructure as code model, but write and maintain infrastructure with general-purpose programming languages, familiar package managers, IDEs, testing, and software engineering patterns, while still understanding the refactoring tradeoffs in Terraform’s own module refactoring guidance.

Why choose Pulumi over Terraform? Pulumi’s language SDKs let teams define cloud infrastructure in TypeScript, Python, Go, C#, Java, or YAML while adding first-class workflows for refactoring with Pulumi aliases, secrets, protect, retainOnDelete, deleteBeforeReplace, replaceOnChanges, provider resources, Pulumi stacks, testing, and incremental migration with pulumi import. Pulumi does not remove every hard problem in cloud infrastructure, but it gives teams stronger tools for many day-to-day pain points.

Executive summary

Pulumi is often a better fit when infrastructure code needs to behave like application code: reviewed, tested, packaged, refactored, and shared across teams. The biggest advantages show up when teams need safer refactors, encrypted secret values, reusable components, clearer provider resources, and guardrails around destructive changes.

The tradeoff is important: Pulumi is still an infrastructure as code engine. Provider bugs, cloud API eventual consistency, drift, preview-time unknowns, and poorly designed abstractions still require engineering discipline. The advantage is not magic. The advantage is a stronger programming model and a more familiar developer workflow.

Need	Terraform pattern	Pulumi advantage	What still needs care
Languages and tooling	HCL plus Terraform-specific functions and expressions	Pulumi supports general-purpose languages, YAML, and now HCL natively, with the normal IDE, test, and package workflows for each	Teams still need code review and shared conventions
Refactoring	Moved blocks or state commands for resource identity changes	Pulumi aliases can map old resource identities to new ones during refactors	Aliases must model the old identity correctly
Secrets	Sensitive values can still require careful state and plan handling	Pulumi tracks secrets and encrypts secret values in state	Secrets are still available to code at runtime
Lifecycle safety	Lifecycle meta-arguments and plan review	Pulumi resource options such as protect, retainOnDelete, deleteBeforeReplace, and replaceOnChanges make intent explicit	Provider behavior and replacement semantics still matter
Environments	Workspaces or separate configurations	Pulumi stacks model environments with per-stack config, secrets, history, and outputs	Stack boundaries still need thoughtful design
Code reuse	Modules and HCL composition patterns	Pulumi components and packages use normal language abstractions	Over-abstracted components can become hard to use
Imports and migration	Import blocks, generated config, and state operations	`pulumi import` and migration tooling support gradual adoption	Imported code still needs review and cleanup
Terraform state	S3, Azure Blob, or Terraform Cloud backends	Pulumi Cloud can serve as a Terraform state backend with encrypted storage, locking, history, and RBAC, while you keep using the Terraform or OpenTofu CLI	State still needs a clean migration plan as resources move to Pulumi
Provider wiring	Provider inheritance and aliases inside modules	Explicit provider resources make multi-region and multi-account wiring visible in code review	Provider versions and bugs can still affect deployments
Testing	Validation, plan review, and external test harnesses	Pulumi programs can use normal unit and integration test frameworks	Tests complement previews, they do not replace them
Drift detection	Refresh and refresh-only plans, plus external scheduling	`pulumi refresh` reconciles state, and Pulumi Cloud adds scheduled drift detection and remediation on a configurable cadence	Detection still depends on provider behavior and a healthy state backend
Caveats	Declarative planning still has unknowns and drift	Pulumi improves the workflow around many pain points	It does not eliminate provider bugs or eventual consistency

Use programming languages and familiar tools

Terraform modules are powerful, but larger HCL codebases can require teams to maintain separate conventions for composition, validation, and reuse. Pulumi lets infrastructure teams use the features of whichever supported programming language they choose, such as classes, functions, types, loops, package managers, linters, and test frameworks.

Teams with HCL muscle memory are not left out, either. Pulumi now supports HCL natively as one of its languages, alongside Python, TypeScript, Go, C#, Java, and YAML. Part of a team can keep authoring in HCL while the project still benefits from Pulumi components, testing, and the same engine and package workflows as every other language.

For example, a platform team can wrap a standard storage pattern in a ComponentResource and share it like any other TypeScript abstraction:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

interface WorkQueueArgs {
 visibilityTimeoutSeconds: number;
}

class WorkQueue extends pulumi.ComponentResource {
 public readonly queueUrl: pulumi.Output<string>;

 constructor(name: string, args: WorkQueueArgs, opts?: pulumi.ComponentResourceOptions) {
 super("platform:queue:WorkQueue", name, {}, opts);

 const queue = new aws.sqs.Queue(`${name}-queue`, {
 visibilityTimeoutSeconds: args.visibilityTimeoutSeconds,
 }, { parent: this });

 this.queueUrl = queue.url;
 this.registerOutputs({ queueUrl: this.queueUrl });
 }
}

const jobs = new WorkQueue("image-jobs", {
 visibilityTimeoutSeconds: 60,
});

Refactor infrastructure without surprise replacement

Renaming a resource, moving it into a component, or reorganizing a project should not automatically mean replacing production infrastructure. Pulumi aliases let you tell Pulumi how a resource used to be addressed, so refactors can preserve resource identity when the old and new resources represent the same cloud object.

const queue = new aws.sqs.Queue("app-jobs", {
 visibilityTimeoutSeconds: 60,
}, {
 aliases: [{ name: "jobs" }],
});

Aliases are not a substitute for review. They work when the old identity is modeled correctly, including details such as name, parent, type, project, and stack when those changed.

Handle secrets with encrypted configuration and secret outputs

Secrets are one of the most common places where infrastructure workflows become risky. Terraform has sensitive values and backend guidance, but teams still need to understand how plans, state, outputs, and backend access interact. Pulumi treats secrets as first-class values, encrypts them in state, and preserves secrecy as values flow through outputs.

const config = new pulumi.Config();
const dbPassword = config.requireSecret("dbPassword");

const passwordParameter = new aws.ssm.Parameter("db-password", {
 type: "SecureString",
 value: dbPassword,
});

This improves the default experience, but it’s not runtime isolation. In the TypeScript SDK, config.requireSecret("dbPassword") retrieves secret configuration, and your program can still access the decrypted value while it runs, so reviews, least privilege, and secret-provider choices still matter. If a value starts as a plain input instead of coming from requireSecret, pulumi.secret(...) can mark it as secret.

For secrets that shouldn’t live in stack config at all, Pulumi ESC (Environments, Secrets, and Configuration) centralizes them in environments that your stacks consume through config. ESC can pull secrets dynamically from external stores such as HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, and 1Password, and mint short-lived OIDC credentials for AWS, Azure, and GCP. That lets you wire an environment into a Pulumi program instead of storing long-lived credentials in state.

Add safer lifecycle controls for destructive changes

Infrastructure mistakes are expensive when they replace or delete stateful resources. Pulumi gives teams explicit resource options for safety-sensitive intent, such as protect, retainOnDelete, deleteBeforeReplace, and replaceOnChanges.

const database = new aws.rds.Instance("orders-db", {
 instanceClass: "db.t4g.micro",
 allocatedStorage: 20,
 engine: "postgres",
 username: "app",
 password: dbPassword,
}, {
 protect: true,
 retainOnDelete: true,
});

These controls are guardrails, not guarantees. Provider bugs, cloud API eventual consistency, and replacement behavior still require preview review and operational judgment. replaceOnChanges applies to custom resources only, not component resources.

Model environments with stacks

Terraform workspaces can represent environments, but many teams eventually need stronger boundaries for configuration, secrets, history, and cross-environment outputs. Pulumi stacks make environment boundaries explicit and pair them with per-stack config and outputs.

const networking = new pulumi.StackReference("acme/networking/prod");
const vpcId = networking.requireOutput("vpcId");

const appSecurityGroup = new aws.ec2.SecurityGroup("app-sg", {
 vpcId,
 ingress: [{
 protocol: "tcp",
 fromPort: 443,
 toPort: 443,
 cidrBlocks: ["10.0.0.0/8"],
 }],
 egress: [{
 protocol: "-1",
 fromPort: 0,
 toPort: 0,
 cidrBlocks: ["0.0.0.0/0"],
 }],
});

Cross-stack references are cleaner than sharing an entire remote state file, but they are still dependencies. Keep stack outputs intentional and avoid turning one stack into a global variable bag.

Test infrastructure with normal language tooling

Because Pulumi programs are real programs, infrastructure testing can use familiar test runners and assertions. Teams can test component defaults, naming conventions, required tags, and policy assumptions before a deployment reaches production.

pulumi.runtime.setMocks({
 newResource: (args) => ({
 id: `${args.name}_id`,
 state: args.inputs,
 }),
 call: (args) => args.inputs,
});

Testing does not replace previews. It catches a different class of problems: broken abstractions, missing required inputs, invalid defaults, and regressions in shared components.

Wire providers explicitly

Provider configuration is another place where explicit code can reduce ambiguity. In Terraform, provider inheritance and aliases are often managed across module boundaries. In Pulumi, provider resources are normal resources that can be passed through resource options, which makes multi-region or multi-account deployments easier to follow in code review.

const west = new aws.Provider("west", {
 region: "us-west-2",
});

const east = new aws.Provider("east", {
 region: "us-east-1",
});

const primary = new aws.sqs.Queue("primary-jobs", {}, {
 provider: west,
});

const replica = new aws.sqs.Queue("replica-jobs", {}, {
 provider: east,
});

The provider object does not remove provider versioning or schema-change risk, but it makes provider wiring visible in the same language and review flow as the rest of the program.

Detect and reconcile drift

Infrastructure drifts when something changes outside your IaC tool: a hotfix in the console, another controller, or a manual break-glass change. Pulumi gives you first-class tools to find and fix it instead of leaving it to chance. pulumi refresh reconciles your state with what is actually running in the cloud, and pulumi preview --diff shows what the next update would change.

For teams that want this continuously, Pulumi Cloud adds scheduled drift detection and remediation that runs on a configurable cadence and can automatically remediate drift when it is detected. That turns finding and correcting out-of-band changes into a managed workflow rather than something a person has to remember to check.

Drift detection still depends on accurate provider behavior and a healthy state backend, so it complements review and operational discipline rather than replacing them.

Import and migrate incrementally

Teams rarely get to rebuild infrastructure from scratch. Pulumi supports incremental adoption with pulumi import, generated code, and Terraform interoperability paths. That makes it possible to start with one resource, one component, or one stack instead of forcing a big-bang migration.

Pulumi also supports interoperability paths for teams that need to bring existing Terraform assets forward over time, including Terraform provider access and migration workflows. That makes migration an engineering sequence, not an all-or-nothing rewrite.

const importedRepository = new aws.ecr.Repository("app", {
 name: "existing-app-repo",
}, {
 import: "existing-app-repo",
});

The CLI flow can also generate declarations with pulumi import. Generated code is a starting point, not a finished architecture. Review names, options, providers, secrets, and component boundaries before treating imported resources as production-ready Pulumi code.

You also don’t have to do it alone. Pulumi provides self-serve conversion tools and expert migration services, and for teams leaving HashiCorp, credits you can apply toward Pulumi to avoid paying for two tools at once, so budget and timing don’t have to block getting started.

Manage Terraform state in Pulumi Cloud

You don’t have to rewrite anything to start getting value from Pulumi. Pulumi Cloud can serve as a Terraform state backend, letting you store and manage Terraform state alongside your Pulumi stacks. Your team can keep using the Terraform or OpenTofu CLI for day-to-day operations while gaining encrypted state storage, update history, state locking, role-based access control, audit policies, and unified resource visibility through Pulumi Insights — with agentic infrastructure coding through Neo on top.

It uses Terraform’s standard remote backend, so you point the CLI at Pulumi Cloud without changing your infrastructure code:

terraform {
 backend "remote" {
 hostname = "api.pulumi.com"
 organization = "your-pulumi-org"

 workspaces {
 name = "_dev"
 }
 }
}

From there, Pulumi can also read outputs from that state, so a new Pulumi stack can consume Terraform-managed values such as VPC IDs and subnet IDs while you migrate one piece at a time. When you’re ready to bring resources fully across, pulumi import --from terraform reads a .tfstate file and generates matching declarations, and pulumi convert --from terraform translates the surrounding configuration into your chosen language.

Storing Terraform state in Pulumi Cloud keeps both tools pointed at one system of record, which is what makes a gradual, low-risk migration practical.

What Pulumi does not magically fix

Pulumi does not eliminate cloud API eventual consistency. A deployment can finish successfully while downstream reads, controllers, or other operators still see stale state for a short window. That is a property of the cloud control plane, not of the IaC tool.

Pulumi also does not eliminate provider bugs. If a provider has a schema issue, a bad default, or a flaky update path, Pulumi still has to ride that provider behavior. Drift will still happen when infrastructure changes outside Pulumi, too. The difference is that Pulumi gives you first-class tooling to detect and reconcile it, as covered in Detect and reconcile drift above — but the drift itself, and the discipline to act on it, do not disappear.

Pulumi does not eliminate preview-time unknowns either. Some values are not known until deployment, so the plan can still contain uncertainty. Bad project decomposition and side-effect-heavy deployment code remain risks too, which is why testing, clear stack boundaries, and disciplined component design still matter.

OpenTofu compatibility caveat

OpenTofu users face many of the same infrastructure-as-code concerns around state, providers, drift, secrets, lifecycle behavior, and migration planning. This post stays Terraform-focused because that is where most teams begin the Pulumi comparison, but the same Pulumi capabilities are relevant when evaluating OpenTofu alternatives or hybrid migration paths.

Pulumi can also work with Terraform provider ecosystems, including long-tail providers that may not have a native Pulumi package yet. Treat that as an interoperability path, not a promise that every Terraform or OpenTofu workflow maps one-for-one without design work.

Get started with Pulumi

If your team is evaluating infrastructure as code options, start with the workflow that creates the most leverage: write infrastructure in the language your team already uses, test shared components, protect critical resources, and migrate one stack at a time.

To go deeper, get started with Pulumi or read the Terraform migration guide.

Kubernetes ConfigMap Revisions with Pulumi

Matan Baruch — Tue, 13 Jan 2026 00:00:00 +0000

ConfigMaps in Kubernetes don’t have built-in revision support, which can create challenges when deploying applications with canary strategies.

When using Argo Rollouts with AWS Spot instances, ConfigMap deletions during canary deployments can cause older pods to fail when they try to reload configuration. We solved this by implementing a custom ConfigMap revision system using Pulumi’s ConfigMapPatch and Kubernetes owner references.

The Problem

When deploying applications to Kubernetes using canary strategies with Argo Rollouts, we encountered a specific challenge:

Pulumi ConfigMap replacement behavior: By default, when a ConfigMap’s data changes, Pulumi may replace it rather than update it in place, which for auto-named ConfigMaps results in a new generated name (suffix).
Canary deployment issues: During canary deployments, the old ConfigMap gets deleted, but older pods (especially on AWS Spot instances that can be replaced during canary) may fail to reload
No native revision support: Neither Kubernetes nor Pulumi natively supports ConfigMap revisions like they do for deployments

Going to KubeCon Europe 2026? Tame Kubernetes complexity with code.

Pulumi demos, platform engineering talks, AI-powered Neo in action, and yes, free plushies. Booth 784.

The solution: ConfigMap revisions with owner references

Our solution leverages Kubernetes’ garbage collection mechanism by using owner references to tie ConfigMaps to ReplicaSets created during canary deployments.

Pulumi makes this approach practical by letting us express patching logic, owner references, and rollout coordination directly in code, instead of encoding complex behavior in static YAML.

Key components

Pulumi’s ConfigMapPatch: Patches existing ConfigMaps with owner references
ReplicaSet Owner References: Links ConfigMaps to ReplicaSets for automatic cleanup
Kubernetes Garbage Collection: Automatically cleans up ConfigMaps when ReplicaSets are deleted
Retain on Delete: Protects ConfigMaps from immediate deletion during Pulumi updates

Implementation

Here’s how we implemented this solution in our rollout component:

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";
import * as k8sClient from "@kubernetes/client-node";

interface RolloutComponentArgs {
 namespace: string;
 configMapPatch?: boolean;
 kubeconfig: pulumi.Output<any>;
 configMapName: pulumi.Output<string>;
 rolloutSpec: k8s.types.input.apiextensions.CustomResourceArgs["spec"];
}

export class ConfigMapRevisionRollout extends pulumi.ComponentResource {
 public readonly rollout: k8s.apiextensions.CustomResource;

 constructor(
 name: string,
 args: RolloutComponentArgs,
 opts?: pulumi.ComponentResourceOptions
 ) {
 super("pulumi:component:ConfigMapRevisionRollout", name, {}, opts);

 // Create the Argo Rollout using CustomResource
 this.rollout = new k8s.apiextensions.CustomResource(
 `${name}-rollout`,
 {
 apiVersion: "argoproj.io/v1alpha1",
 kind: "Rollout",
 metadata: {
 name: name,
 namespace: args.namespace,
 },
 spec: args.rolloutSpec,
 },
 { parent: this, ...opts }
 );

 // Apply ConfigMap revision patching if enabled
 if (args.configMapPatch) {
 this.setupConfigMapRevisions(name, args);
 }

 this.registerOutputs({
 rollout: this.rollout,
 });
 }

 private setupConfigMapRevisions(name: string, args: RolloutComponentArgs): void {
 pulumi
 .all([args.kubeconfig, args.configMapName])
 .apply(async ([kubeconfig, configMapName]) => {
 try {
 // Create Server-Side Apply enabled provider
 const ssaProvider = new k8s.Provider(`${name}-ssa-provider`, {
 kubeconfig: JSON.stringify(kubeconfig),
 enableServerSideApply: true,
 });

 // Wait for rollout to stabilize and create ReplicaSets
 await this.waitForRolloutStabilization();

 // Get ReplicaSets associated with this rollout
 const replicaSets = await this.getAssociatedReplicaSets(
 args.namespace,
 configMapName,
 kubeconfig
 );

 if (replicaSets.length === 0) {
 pulumi.log.warn("No ReplicaSets found for ConfigMap patching");
 return;
 }

 // Create owner references for the ConfigMap
 const ownerReferences = replicaSets.map(rs => ({
 apiVersion: "apps/v1",
 kind: "ReplicaSet",
 name: rs.metadata?.name!,
 uid: rs.metadata?.uid!,
 controller: false,
 blockOwnerDeletion: false,
 }));

 // Patch the ConfigMap with owner references
 new k8s.core.v1.ConfigMapPatch(
 `${configMapName}-revision-patch`,
 {
 metadata: {
 name: configMapName,
 namespace: args.namespace,
 ownerReferences: ownerReferences,
 annotations: {
 "pulumi.com/patchForce": "true",
 "configmap.kubernetes.io/revision-managed": "true",
 },
 },
 },
 {
 provider: ssaProvider,
 retainOnDelete: true,
 parent: this,
 }
 );

 pulumi.log.info(
 `Successfully patched ConfigMap ${configMapName} with ${ownerReferences.length} owner references`
 );
 } catch (error) {
 pulumi.log.error(`Failed to setup ConfigMap revisions: ${error}`);
 throw error;
 }
 });
 }

 private async waitForRolloutStabilization(): Promise<void> {
 // Wait for rollout to create and stabilize ReplicaSets
 // In production, consider using a more sophisticated polling mechanism
 await new Promise(resolve => setTimeout(resolve, 10000));
 }

 private async getAssociatedReplicaSets(
 namespace: string,
 configMapName: string,
 kubeconfig: any
 ): Promise<k8sClient.V1ReplicaSet[]> {
 const kc = new k8sClient.KubeConfig();
 kc.loadFromString(JSON.stringify(kubeconfig));

 const appsV1Api = kc.makeApiClient(k8sClient.AppsV1Api);

 try {
 const response = await appsV1Api.listNamespacedReplicaSet(
 namespace,
 undefined, // pretty
 false, // allowWatchBookmarks
 undefined, // continue
 undefined, // fieldSelector
 `configMap=${configMapName}` // labelSelector
 );

 return response.body.items;
 } catch (error) {
 pulumi.log.error(`Failed to list ReplicaSets: ${error}`);
 return [];
 }
 }
}

How do ConfigMap revisions work with Argo Rollouts?

Rollout Creation: When a new rollout is created, Argo Rollouts generates new ReplicaSets for the canary deployment
ConfigMap Patching: Our code waits for the ReplicaSet creation, then patches the ConfigMap with owner references pointing to these ReplicaSets
Garbage Collection: Kubernetes automatically tracks the relationship between ConfigMaps and ReplicaSets
Automatic Cleanup: When ReplicaSets are cleaned up (based on the default 10 revision history), their associated ConfigMaps are also garbage collected

Benefits

Revision Control: ConfigMaps now have revision-like behavior tied to ReplicaSet history
Automatic Cleanup: No manual intervention needed for ConfigMap cleanup
Canary Safety: Old ConfigMaps remain available during canary deployments until ReplicaSets are cleaned up
Spot Instance Resilience: Pods that get replaced during canary deployments can still access their original ConfigMaps

Configuration options

interface RolloutComponentArgs {
 namespace: string;
 configMapPatch?: boolean;
 kubeconfig: pulumi.Output<any>;
 configMapName: pulumi.Output<string>;
 rolloutSpec: k8s.types.input.apiextensions.CustomResourceArgs["spec"];
}

To enable this feature in your rollout:

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

// Create your EKS cluster
const cluster = new k8s.Provider("k8s-provider", {
 kubeconfig: clusterKubeconfig,
});

// Create ConfigMap
const appConfig = new k8s.core.v1.ConfigMap("app-config", {
 metadata: {
 name: "my-app-config",
 namespace: "default",
 labels: {
 app: "my-app",
 configMap: "my-app-config", // Important for ReplicaSet selection
 },
 },
 data: {
 "app.properties": "key=value\nother=setting",
 },
}, { provider: cluster });

// Create rollout with ConfigMap revision management
const rollout = new ConfigMapRevisionRollout("my-app", {
 namespace: "default",
 configMapPatch: true,
 kubeconfig: clusterKubeconfig,
 configMapName: appConfig.metadata.name,
 rolloutSpec: {
 replicas: 3,
 selector: {
 matchLabels: { app: "my-app" },
 },
 template: {
 metadata: {
 labels: { app: "my-app" },
 },
 spec: {
 containers: [{
 name: "app",
 image: "nginx:latest",
 volumeMounts: [{
 name: "config",
 mountPath: "/etc/config",
 }],
 }],
 volumes: [{
 name: "config",
 configMap: {
 name: appConfig.metadata.name,
 },
 }],
 },
 },
 strategy: {
 canary: {
 maxSurge: 1,
 maxUnavailable: 0,
 steps: [
 { setWeight: 20 },
 { pause: { duration: "1m" } },
 { setWeight: 50 },
 { pause: { duration: "2m" } },
 ],
 },
 },
 },
});

Key dependencies

The solution uses several key packages:

@pulumi/kubernetes: For Kubernetes resources and ConfigMapPatch
@kubernetes/client-node: For direct Kubernetes API access
Argo Rollouts CRDs installed in your cluster

When should you use this approach?

This pattern is a good fit if you:

Use Argo Rollouts with canary deployments
Rely on ConfigMaps that must remain available across rollout revisions
Run workloads on Spot or preemptible instances
Need automatic cleanup without custom controllers

Conclusion

This approach gives us ConfigMap revision functionality that doesn’t exist natively in Kubernetes or Pulumi. By leveraging Kubernetes’ garbage collection mechanism and Pulumi’s patching capabilities, we created a robust solution for managing ConfigMap lifecycles during canary deployments.

The solution is particularly valuable when:

Running canary deployments with Argo Rollouts
Using AWS Spot instances that can be replaced during deployments
Needing automatic cleanup of old ConfigMaps without manual intervention
Wanting to maintain configuration availability for older pods during deployment transitions

This pattern can be extended to other scenarios where you need revision control for Kubernetes resources that don’t natively support it.

Want to learn how to put these practices into action? Meet us at KubeCon Europe 2026 (Booth 784) or register for our upcoming Zero to Production in Kubernetes workshop.

Try Pulumi for Free

From 'Works on My Machine' to Production-Ready: Building AI Agents with Amazon Bedrock AgentCore

Engin Diri — Mon, 15 Dec 2025 00:00:00 +0200

Every developer building AI agents knows the gap between a working prototype and production deployment. Your fraud detection agent works perfectly on your laptop, but how do you deploy it with proper authentication, memory persistence, observability, and guardrails? This post walks through a complete journey from local development to production-ready AI agents using Amazon Bedrock AgentCore, the Strands SDK, and Pulumi.

TL;DR: This post shows how to deploy AI agents from local prototype to production using:

Strands SDK for agent development
Amazon Bedrock AgentCore for managed runtime with Firecracker isolation
AgentCore Memory for short-term and long-term context
MCP Gateway for secure tool integration with OAuth2
Pulumi for infrastructure as code

Complete code: GitHub

The Challenge: Bridging the Gap

Building AI agents is getting easier. Tools like the Strands Agents SDK let you create functional agents in just a few lines of code. Production deployment introduces challenges:

Stateless to Stateful: How does your agent remember previous conversations?
Single-tenant to Multi-tenant: How do you isolate different users’ data?
Implicit Trust to Zero Trust: How do you authenticate requests and enforce boundaries?
Console Logs to Observability: How do you trace requests across distributed systems?
Ad-hoc Tools to Managed Services: How do you expose tools securely at scale?

The approach is progressive: start simple, add complexity only when needed, and use infrastructure as code to make it reproducible.

Agents Beyond Chat: Event-Driven AI

When most people think of AI agents, they imagine chatbots: interactive systems responding to user questions. But many production agents never interact with end users directly. Instead, they’re triggered by event streams: Kafka topics, SQS queues, webhook callbacks, or scheduled jobs.

Our fraud detection agent is a prime example. It doesn’t wait for customers to ask questions. It’s triggered automatically when the payment processing system detects a new transaction. The “user” is another system, and the “conversation” is a structured event payload.

This event-driven pattern is becoming the new stack for AI agents. As Kai Waehner describes in his analysis of agentic AI with Kafka and Flink:

“Protocols like A2A and MCP are essential for standardizing agent behavior and communication. But without an event-driven substrate like Kafka and a stream-native runtime like Flink, these agents remain stuck in isolated interactions.”

The emerging architecture looks like this:

In production, you might use Amazon MSK (Managed Kafka), Amazon SQS, or EventBridge to deliver transaction events to your agent. The agent processes each event autonomously, takes action (blocking cards, creating tickets), and publishes results back to another topic.

What We’re Building: A Fraud Detection Agent

Our demo is a fraud detection agent that:

Receives transaction alerts with user ID, amount, merchant, and location
Gathers context from tools (user profiles, transaction history)
Analyzes for fraud patterns (impossible travel, velocity attacks, amount anomalies)
Takes protective action (blocking cards) when fraud is detected

We’ll build this progressively:

Stage	What Changes	Key Additions
Local Development	Running on your laptop	Strands SDK, Claude Opus 4.5, mock tools
First Deployment	Running in AgentCore	Containerized, Firecracker isolation, guardrails
Production-Ready	Full enterprise features	Memory, MCP Gateway, X-Ray observability

Local Development with Strands SDK

The Strands Agents SDK is an open-source framework from AWS that takes a model-driven approach to agent development. Instead of hardcoding complex workflows, you let the LLM reason about tool usage autonomously.

Here’s our complete local agent in about 30 lines:

# agent.py
from strands import Agent
from strands.models import BedrockModel
from tools import get_user_profile, get_recent_transactions, block_credit_card

# Configure Bedrock model with cross-region inference for Claude Opus 4.5
# Note: Cross-region inference may have different pricing than single-region
bedrock_model = BedrockModel(
 model_id="us.anthropic.claude-opus-4-5-20251101-v1:0",
 region_name="us-east-1",
)

# Define the Fraud Detection Agent
fraud_agent = Agent(
 model=bedrock_model,
 tools=[get_user_profile, get_recent_transactions, block_credit_card],
 system_prompt="""
 You are a Senior Fraud Analyst Agent at a financial institution.
 Your mission is to protect customers from fraudulent transactions in real-time.

 When you receive a transaction alert:
 1. Gather context by checking the user's profile and recent transaction history.
 2. Analyze the transaction for fraud indicators:
 - Impossible travel: Can the user physically be at the new location?
 - Unusual amounts: Is this significantly larger than typical spending?
 - High-risk merchants: Electronics stores, gift cards, wire transfers.
 - Geographic anomalies: Transactions far from home location.
 3. Make a decision:
 - If fraud indicators are present, BLOCK the card immediately.
 - If the transaction appears legitimate, allow it to proceed.
 4. Document your reasoning clearly.

 Prioritize customer protection. When in doubt, block the card.
 """,
)

Tools are simple Python functions decorated with @tool:

from strands import tool

@tool
def get_user_profile(user_id: str) -> dict:
 """Get user profile including home location and account status."""
 return {
 "user_id": user_id,
 "name": "John Doe",
 "home_location": "London, UK",
 "account_status": "active",
 }

@tool
def block_credit_card(user_id: str, reason: str) -> dict:
 """Block a user's credit card and create a fraud ticket."""
 return {
 "status": "blocked",
 "ticket_id": f"FRAUD-{user_id}-001",
 "message": f"Card blocked for user {user_id}: {reason}",
 }

Running the Local Demo

Here’s how we simulate an incoming transaction event:

# demo_simulation.py
from agent import fraud_agent

# The "Trigger" Event (Simulated Payload from Kafka/SQS)
new_transaction_event = """
ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
"""

def run_demo():
 print("--- INCOMING EVENT STREAM ---")
 print(new_transaction_event)
 print("--- AGENT ACTIVATED ---")

 # Run the agent - processes the event autonomously
 result = fraud_agent(new_transaction_event)

 print("--- ANALYSIS COMPLETE ---")

if __name__ == "__main__":
 run_demo()

When we run this, the agent autonomously gathers context, analyzes the transaction, and takes action:

$ python demo_simulation.py
--- INCOMING EVENT STREAM ---
ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
-----------------------------------
AGENT ACTIVATED. Analyzing...
Tool #1: get_user_profile
Tool #2: get_recent_transactions
## 🚨 FRAUD ANALYSIS REPORT
**Customer:** John Doe (user_123)
### Analysis:
#### 1. ❌ IMPOSSIBLE TRAVEL DETECTED
- **Last Transaction:** 09:00 in **London, UK**
- **Current Transaction:** 09:15 in **Tokyo, Japan**
- **Time Elapsed:** 15 minutes
- **Distance:** ~9,500 km (5,900 miles)
- **Minimum Flight Time:** ~12 hours
**Verdict:** It is physically impossible for John Doe to travel from London
to Tokyo in 15 minutes. This is a critical fraud indicator.
#### 2. ⚠️ HIGH-RISK MERCHANT
- Electronics stores are frequently targeted for fraud due to high resale value
#### 3. ⚠️ HIGH-VALUE TRANSACTION
- $2,000 at an electronics store represents a substantial purchase
### Decision: 🛑 BLOCK CARD IMMEDIATELY
Tool #3: block_credit_card
## ✅ ACTION COMPLETED
| Action | Status |
|--------|--------|
| Card Blocked | ✅ Confirmed |
| Ticket ID | TICKET-999 |
--- ANALYSIS COMPLETE ---

Notice how the agent reasoned about the event using tools and took autonomous action. This works great for development, but it lacks everything needed for production: authentication, persistence, isolation, monitoring, and governance.

First Deployment with AgentCore Runtime

Our local prototype works, but where do you actually run it? Lambda times out after 15 minutes. ECS and Kubernetes add operational overhead. And AI agents have different requirements than typical microservices: they reason through multi-step workflows, sometimes taking minutes to analyze a complex fraud case. You need something built for this workload.

Amazon Bedrock AgentCore fills this gap. It’s a managed runtime specifically designed for AI agents, with support for tasks running up to 8 hours. No timeout workarounds, no cluster management.

The security model is particularly relevant for fraud detection. AgentCore runs each session in a dedicated microVM using AWS Firecracker (the same isolation technology behind Lambda). When your agent processes one customer’s transactions, it’s completely isolated from other sessions. No shared memory, no cross-tenant contamination.

The deployment model is straightforward: containerize your agent and point AgentCore at the image. It works with Strands, LangGraph, CrewAI, or custom code. The same agent we built locally runs unchanged. We just wrap it in FastAPI and add a Dockerfile.

Cold Start and Latency

For fraud detection, latency matters. AgentCore’s Firecracker microVMs introduce a cold start penalty on the first invocation, typically 2-5 seconds to spin up the container. Subsequent invocations within the same session are fast (sub-second overhead beyond model inference time).

For real-time fraud detection, this cold start is acceptable because:

The LLM reasoning itself takes 5-15 seconds for complex analysis
Most production deployments use session pooling to keep warm instances available
The alternative (always-on containers) trades latency for cost

If sub-second total latency is critical, consider Lambda for simple rule-based checks and reserve AgentCore for complex reasoning that justifies the cold start overhead.

Containerizing the Agent

First, we wrap our agent in a FastAPI endpoint:

# agent.py (production version)
from fastapi import FastAPI, Request
from strands import Agent
from strands.models import BedrockModel
import os

app = FastAPI()

bedrock_model = BedrockModel(
 model_id="us.anthropic.claude-opus-4-5-20251101-v1:0",
 region_name="us-east-1",
 guardrail_id=os.environ.get("BEDROCK_GUARDRAIL_ID"),
 guardrail_version=os.environ.get("BEDROCK_GUARDRAIL_VERSION"),
 guardrail_trace="enabled",
)

fraud_agent = Agent(
 model=bedrock_model,
 tools=[get_user_profile, get_recent_transactions, block_credit_card],
 system_prompt="...",
)

@app.post("/invocations")
async def invoke(request: Request):
 body = await request.json()
 prompt = body.get("input", {}).get("prompt", "")
 result = fraud_agent(prompt)
 return {"output": {"message": str(result)}}

Adding Guardrails

Amazon Bedrock Guardrails help ensure your agent stays on task. We configure content filters and topic restrictions using Pulumi:

fraud_detection_guardrail = bedrock.Guardrail(
 "fraud-detection-guardrail",
 name="fraud-detection-guardrail",
 description="Ensure agent only handles fraud detection tasks",
 blocked_input_messaging="This request is outside the scope of fraud detection.",
 blocked_outputs_messaging="This response was blocked.",
 content_policy_config=bedrock.GuardrailContentPolicyConfigArgs(
 filters_configs=[
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="SEXUAL", input_strength="HIGH", output_strength="HIGH",
 ),
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="VIOLENCE", input_strength="HIGH", output_strength="HIGH",
 ),
 bedrock.GuardrailContentPolicyConfigFiltersConfigArgs(
 type="PROMPT_ATTACK", input_strength="HIGH", output_strength="NONE",
 ),
 # ... additional filters
 ],
 ),
 topic_policy_config=bedrock.GuardrailTopicPolicyConfigArgs(
 topics_configs=[
 bedrock.GuardrailTopicPolicyConfigTopicsConfigArgs(
 name="entertainment-media",
 type="DENY",
 definition="Questions about movies, TV shows, celebrities...",
 examples=["What is LOTR?", "Tell me about Star Wars"],
 ),
 bedrock.GuardrailTopicPolicyConfigTopicsConfigArgs(
 name="general-assistance",
 type="DENY",
 definition="Requests for coding help, math problems...",
 examples=["Write me a poem", "Help me with Python code"],
 ),
 ],
 ),
)

Deploying with Pulumi

The infrastructure is straightforward: ECR repository, IAM roles, and AgentCore Runtime:

# Create ECR repository
repo = ecr.Repository(
 "strands-agent-repo",
 name="fraud-detection-agent",
 force_delete=True,
)

# Build and push Docker image
image = docker.Image(
 "strands-agent-image",
 image_name=repo.repository_url.apply(lambda url: f"{url}:latest"),
 build=docker.DockerBuildArgs(
 context="../fraud-detection-agent",
 platform="linux/arm64",
 ),
 registry=docker.RegistryArgs(
 server=repo.repository_url,
 username=auth_token.user_name,
 password=auth_token.password,
 ),
)

# Deploy AgentCore Runtime
agent_runtime = bedrock.AgentcoreAgentRuntime(
 "fraud-detection-agent-runtime",
 agent_runtime_name="fraud_detection_agent",
 description="Fraud Detection Agent with Guardrails",
 role_arn=agent_role.arn,
 agent_runtime_artifact=bedrock.AgentcoreAgentRuntimeAgentRuntimeArtifactArgs(
 container_configuration=bedrock.AgentcoreAgentRuntimeAgentRuntimeArtifactContainerConfigurationArgs(
 container_uri=image.image_name,
 ),
 ),
 network_configuration=bedrock.AgentcoreAgentRuntimeNetworkConfigurationArgs(
 network_mode="PUBLIC",
 ),
 environment_variables={
 "BEDROCK_GUARDRAIL_ID": fraud_detection_guardrail.guardrail_id,
 "BEDROCK_GUARDRAIL_VERSION": fraud_detection_guardrail.version,
 },
)

Testing the Basic Deployment

After pulumi up, we can invoke the deployed agent via the AWS SDK:

# invoke_agent.py
import boto3
import json

def invoke_agent(agent_runtime_arn: str, prompt: str) -> dict:
 client = boto3.client("bedrock-agentcore", region_name="us-east-1")

 response = client.invoke_agent_runtime(
 agentRuntimeArn=agent_runtime_arn,
 runtimeSessionId="session_abc123...",
 payload=json.dumps({"input": {"prompt": prompt}}),
 qualifier="DEFAULT",
 )

 return json.loads(response["response"].read())

Running the same fraud scenario against the deployed agent:

$ ./test.sh
Selecting Pulumi stack 'dev'...
Getting agent runtime ARN from Pulumi...
Agent Runtime ARN: arn:aws:bedrock-agentcore:us-east-1:052848974346:runtime/fraud_detection_agent-W7zFZT3Vcn
Invoking agent with fraud detection alert...
========================================
Sending prompt to agent...
Prompt: ALERT: New Transaction Attempt
User ID: user_123
Amount: $2000
Merchant: Electronics Store
Location: Tokyo, Japan
Time: 09:15
--------------------------------------------------
Agent Response:
{
"output": {
"message": {
"role": "assistant",
"content": [
{
"text": "**Analysis:** Impossible travel confirmed. User was in London at 09:00
and this transaction attempted in Tokyo at 09:15. Physically impossible.
Combined with high-risk merchant and $2000 amount, this is fraudulent.
Card blocked (TICKET-999)."
}
]
},
"timestamp": "2025-12-15T04:17:48.762738+00:00",
"model": "fraud-detection-agent"
}
}

The agent now runs in a managed, isolated environment with guardrails. Next, we add memory and external tool integration.

Production-Ready: Memory and MCP Gateway

Our basic deployment handles individual transactions, but fraud detection in the real world requires context. If a customer’s card was blocked five minutes ago, the agent shouldn’t try to block it again. If there’s a pattern of suspicious activity over the past week, the agent should factor that into its analysis.

This is where AgentCore Memory and the MCP Gateway come in.

AgentCore Memory: Short-term and Long-term

Memory in AgentCore works at two levels. Short-term memory stores raw conversation events within a session. If the agent blocks a card in one message, it remembers this in subsequent messages during the same session.

Long-term memory is more interesting. It uses semantic extraction to persist facts across sessions. The system runs your conversations through a smaller model (we use Claude Haiku) to extract key facts, then stores those facts in a namespace you define. When a new session starts, the agent can query for relevant facts about that user. Even if the customer calls back tomorrow, the agent knows their card was blocked yesterday.

Here’s how we configure the memory infrastructure:

# Create AgentCore Memory with 30-day event expiry
fraud_detection_memory = bedrock.AgentcoreMemory(
 "fraud-detection-memory",
 name="fraud_detection_memory",
 description="Memory for fraud detection agent",
 event_expiry_duration=30, # Events expire after 30 days
 memory_execution_role_arn=memory_execution_role.arn,
)

# Custom semantic extraction for fraud-related facts
FRAUD_EXTRACTION_PROMPT = """
Extract key fraud-related facts from this conversation about the user. Focus on:
1. Credit card status (blocked/active, ticket IDs if blocked)
2. Fraud alerts and patterns detected for this user
3. Transaction history patterns (locations, amounts, frequencies)
4. User profile details relevant to fraud detection
5. Previous fraud investigations and their outcomes

Format as concise facts that can help identify fraud patterns across sessions.
"""

# Create long-term memory strategy
fraud_semantic_strategy = bedrock.AgentcoreMemoryStrategy(
 "fraud-semantic-strategy",
 name="fraud_semantic_extraction",
 memory_id=fraud_detection_memory.id,
 type="CUSTOM",
 namespaces=["/fraud-detection/users/{actorId}"],
 configuration={
 "type": "SEMANTIC_OVERRIDE",
 "extraction": {
 "append_to_prompt": FRAUD_EXTRACTION_PROMPT,
 "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0",
 },
 "consolidation": {
 "model_id": "us.anthropic.claude-3-5-haiku-20241022-v1:0",
 },
 },
)

The namespace pattern /fraud-detection/users/{actorId} ensures each user’s facts are isolated. Claude Haiku handles the extraction efficiently, keeping costs low while maintaining accuracy.

Long-term memory extraction is asynchronous. Plan for a 20-40 second delay between storing conversations and having facts available for retrieval. Use short-term memory for immediate needs while long-term memories are being processed.

In the agent code, we integrate memory using Strands hooks:

from bedrock_agentcore.memory import MemoryClient

memory_client = MemoryClient(region_name="us-east-1")

# Retrieve long-term facts before processing
memories = memory_client.retrieve_memories(
 memory_id=memory_id,
 namespace=f"/fraud-detection/users/{actor_id}",
 query=f"fraud detection facts for user {actor_id}"
)

# Store conversation after processing
memory_client.create_event(
 memory_id=memory_id,
 actor_id=actor_id,
 session_id=session_id,
 messages=[
 (user_message, "USER"),
 (assistant_message, "ASSISTANT")
 ]
)

MCP Gateway: Tool Integration at Scale

The tools we’ve used so far are baked into the agent container. That works for simple cases, but what happens when you have dozens of tools across multiple teams? Or when a tool needs to call an external service that requires its own authentication?

The Model Context Protocol (MCP) is an open standard for AI systems to communicate with tools. AgentCore Gateway provides a managed MCP endpoint that handles authentication (JWT inbound, OAuth2 outbound) and lets agents discover tools using natural language queries rather than knowing exact tool names.

We deploy an MCP server for risk scoring with three tools:

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("Fraud Risk Scoring Server")

@mcp.tool()
def calculate_risk_score(user_id: str, amount: float, merchant: str, location: str) -> dict:
 """Calculate comprehensive fraud risk score (0-100)."""
 # Risk calculation logic
 return {
 "risk_score": 75,
 "factors": ["amount_anomaly", "high_risk_merchant"],
 "recommendation": "BLOCK"
 }

@mcp.tool()
def get_fraud_indicators(user_id: str) -> dict:
 """Get known fraud indicators for a user."""
 return {
 "indicators": ["velocity_alert", "device_inconsistency"],
 "severity": "HIGH"
 }

@mcp.tool()
def check_merchant_reputation(merchant_name: str) -> dict:
 """Check merchant risk rating and fraud history."""
 return {
 "risk_rating": "HIGH",
 "fraud_reports": 47,
 "chargeback_rate": 0.15
 }

The Gateway infrastructure requires careful attention to authentication. We use two separate Cognito user pools to handle the dual authentication flow:

Inbound authentication (Agent → Gateway): The agent authenticates to the Gateway using JWT tokens from the first Cognito pool. This ensures only authorized agents can discover and call tools.

# Gateway with MCP protocol and semantic search
agentcore_gateway = bedrock.AgentcoreGateway(
 "fraud-detection-gateway",
 name="fraud-detection-mcp-gateway",
 description="Gateway for fraud detection MCP tools",
 role_arn=gateway_role.arn,
 authorizer_type="CUSTOM_JWT",
 authorizer_configuration={
 "customJwtAuthorizer": {
 "discoveryUrl": gateway_discovery_url,
 "allowedClients": [gateway_app_client.id],
 },
 },
 protocol_type="MCP",
 protocol_configuration={
 "mcp": {
 "supportedVersions": ["2025-03-26"],
 "searchType": "SEMANTIC",
 "instructions": "Gateway for fraud detection risk scoring tools.",
 },
 },
)

The searchType: "SEMANTIC" configuration enables natural language tool discovery. Instead of the agent needing to know exact tool names, it can query for “tools that check merchant risk” and the Gateway returns relevant matches.

Outbound authentication (Gateway → MCP Server): When the Gateway calls the MCP server, it needs its own credentials. The OAuth2 Credential Provider manages this second authentication layer:

# OAuth2 Credential Provider for Gateway -> MCP Server auth
oauth2_credential_provider = bedrock.AgentcoreOauth2CredentialProvider(
 "mcp-runtime-oauth2-provider",
 name="fraud-mcp-runtime-oauth2",
 credential_provider_vendor="CustomOauth2",
 oauth2_provider_config={
 "customOauth2ProviderConfig": {
 "oauthDiscovery": {"discoveryUrl": mcp_runtime_discovery_url},
 "clientId": mcp_runtime_app_client.id,
 "clientSecret": mcp_runtime_app_client.client_secret,
 },
 },
)

Connecting the pieces: Finally, we create a Gateway Target. This links the Gateway to the MCP server endpoint. It also specifies which credential provider to use for outbound calls:

# Connect Gateway to MCP Server
gateway_target = bedrock.AgentcoreGatewayTarget(
 "mcp-risk-server-target",
 name="mcp-risk-server-target",
 gateway_identifier=agentcore_gateway.gateway_id,
 target_configuration={
 "mcp": {"mcpServer": {"endpoint": mcp_server_endpoint_url}},
 },
 credential_provider_configuration={
 "oauth": {
 "providerArn": oauth2_credential_provider.credential_provider_arn,
 "scopes": [mcp_runtime_scope_string],
 },
 },
)

This three-part configuration (Gateway, Credential Provider, Target) separates concerns cleanly: the Gateway handles inbound auth and protocol translation, the Credential Provider manages outbound secrets, and the Target defines routing.

Observability with X-Ray and OpenTelemetry

Production systems need observability. We configure CloudWatch Logs for application logs and X-Ray for distributed tracing:

# Agent application logs
agent_log_group = cloudwatch.LogGroup(
 "agent-runtime-log-group",
 name=pulumi.Output.concat(
 "/aws/vendedlogs/bedrock-agentcore/", agent_runtime.agent_runtime_id
 ),
 retention_in_days=30,
)

agent_logs_delivery = cloudwatch.LogDelivery(
 "agent-logs-delivery",
 delivery_source_name=agent_logs_delivery_source.name,
 delivery_destination_arn=agent_logs_delivery_destination.arn,
)

# X-Ray traces
agent_traces_delivery_destination = cloudwatch.LogDeliveryDestination(
 "agent-traces-delivery-destination",
 name="fraud-agent-traces-dest",
 delivery_destination_type="XRAY",
)

agent_traces_delivery = cloudwatch.LogDelivery(
 "agent-traces-delivery",
 delivery_source_name=agent_traces_delivery_source.name,
 delivery_destination_arn=agent_traces_delivery_destination.arn,
)

In the container, we use AWS Distro for OpenTelemetry for auto-instrumentation:

CMD ["uv", "run", "opentelemetry-instrument", "uvicorn", "agent:app", "--host", "0.0.0.0", "--port", "8080"]

This gives us end-to-end tracing across agent invocations, memory operations, and Gateway calls with zero code changes.

AWS has announced that X-Ray SDKs will be deprecated in favor of OpenTelemetry by February 2027. Using ADOT from the start ensures your observability stack is future-proof.

Running the Demo

Deploy the Infrastructure

cd advanced-bedrock-deployment/infra
uv sync
uv run pulumi up

This deploys approximately 40 AWS resources including:

ECR repositories for agent and MCP server
AgentCore Memory with semantic extraction strategy
Two Cognito user pools for authentication
AgentCore Gateway with MCP protocol
CloudWatch log groups and X-Ray trace delivery
IAM roles and policies

Test Short-term Memory

./test.sh demo

This runs 5 scenarios in the same session, demonstrating short-term memory:

======================================================================
FRAUD DETECTION DEMO - SHORT-TERM MEMORY (Same Session)
======================================================================
SCENARIO 1: John Doe - Impossible Travel Detection
Expected: FRAUD - impossible travel from London to Tokyo in 15 minutes
Actor ID: user_123
Session ID: session_cf171803eb324a2aa9ac456b3013fe45...
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**FRAUD CONFIRMED - CARD BLOCKED**
| Finding | Details |
|---------|---------|
| **User** | John Doe |
| **Ticket** | TICKET-123-999 |
| **Fraud Type** | Impossible Travel + Spending Anomaly |
**Analysis:** Transaction in Tokyo 15 minutes after London purchase
is physically impossible. The $2,000 amount is 10x the user's typical
spending range ($50-200). Card blocked immediately."
},
"memory_enabled": true,
"long_term_facts_retrieved": 3
}
}
======================================================================
SCENARIO 3: John Doe - Follow-up Transaction (Short-term Memory Test)
Expected: FRAUD - card should already be blocked (SHORT-TERM MEMORY)
Actor ID: user_123
Session ID: session_cf171803eb324a2aa9ac456b3013fe45... (SAME SESSION!)
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**Card for user_123 (John Doe) is ALREADY BLOCKED
(Ticket: TICKET-123-999). No further action needed.**
This appears to be another fraudulent attempt on an already-compromised
account. The card was previously blocked due to impossible travel fraud.
The current transaction follows the same pattern: Gift Card Kiosk
purchases are a common fraud tactic for converting stolen card access
to untraceable value."
},
"memory_enabled": true,
"long_term_facts_retrieved": 3
}
}
======================================================================
SHORT-TERM MEMORY DEMO SUMMARY
======================================================================
[OK] John Doe - Impossible Travel Detection
[OK] Jane Smith - Normal Transaction
[OK] John Doe - Follow-up Transaction (Short-term Memory Test)
[OK] Alice Chen - Impossible Travel Detection
[OK] Jane Smith - Second Normal Transaction

The agent remembers within the same session that John’s card is already blocked, so there’s no need to block it twice.

Test Long-term Memory

./test.sh longterm-fresh

This demonstrates cross-session memory with two phases:

======================================================================
PHASE 1: INITIAL FRAUD DETECTION
These interactions create long-term memory facts about users.
======================================================================
SCENARIO: John Doe - Initial Fraud (Card Blocked)
Session ID: session_xyz789... (NEW SESSION)
→ Card blocked, fact stored to long-term memory
======================================================================
WAITING FOR LONG-TERM MEMORY EXTRACTION...
The semantic strategy needs time to process conversations into facts.
Waiting 30 seconds...
======================================================================
======================================================================
PHASE 2: NEW SESSIONS - LONG-TERM MEMORY RETRIEVAL
These are COMPLETELY NEW sessions. The agent should remember
fraud-related facts about users from Phase 1.
======================================================================
SCENARIO: John Doe - NEW SESSION (Long-term Memory Test)
Session ID: session_new456... (DIFFERENT SESSION!)
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": "CARD ALREADY BLOCKED. According to my records,
this user's card was previously blocked due to fraud.
Ticket FRAUD-user_123-001 exists from a previous investigation.
The card remains blocked.",
"memory_enabled": true,
"long_term_facts_retrieved": 2
}
}

Even in a completely new session, the agent retrieves facts from long-term memory and knows John’s card was blocked yesterday.

Test MCP Gateway

./test.sh gateway-fresh

This runs 6 diverse fraud scenarios using both local tools and MCP Gateway tools:

======================================================================
LOADING MCP GATEWAY CONFIGURATION...
======================================================================
Gateway URL: https://fraud-detection-mcp-gateway-xxx.gateway.bedrock-agentcore.us-east-1.amazonaws.com/mcp
Token Endpoint: https://fraud-gateway-xxx.auth.us-east-1.amazoncognito.com/oauth2/token
Scope: fraud-detection-gateway/invoke
MCP Gateway configuration loaded successfully!
======================================================================
FRAUD DETECTION DEMO - MCP GATEWAY INTEGRATION
======================================================================
LOCAL TOOLS:
- get_user_profile: User details and home location
- get_recent_transactions: Last known transaction
- block_credit_card: Block card and create ticket
MCP GATEWAY TOOLS (via Risk Scoring Service):
- calculate_risk_score: Fraud risk score 0-100
- get_fraud_indicators: Known fraud indicators
- check_merchant_reputation: Merchant risk rating
======================================================================
SCENARIO 1: HIGH-RISK MERCHANT - Jane at CryptoExchange123
MCP Gateway: ENABLED
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "**✅ CARD BLOCKED** - Ticket: TICKET-456-999
**Summary:** Jane Smith's card blocked due to a $5,000 crypto exchange
transaction that is 10x her typical spend and outside her normal merchant
categories. Location matched home, but amount and merchant anomalies
indicate likely fraud or compromised credentials."
},
"mcp_gateway_enabled": true,
"mcp_tools_count": 4
}
}
======================================================================
SCENARIO 5: COMBINED SIGNALS - Alice's Multi-Factor Fraud
MCP Gateway: ENABLED
----------------------------------------------------------------------
Agent Response:
{
"output": {
"message": {
"text": "## 🚨 FRAUD CONFIRMED - CARD BLOCKED
**User:** Alice Chen (user_321)
**Ticket ID:** TICKET-321-999
**Fraud Indicators Detected:**
| Factor | Finding |
|--------|---------|
| ⛔ Impossible Travel | Singapore (10:00) → Los Angeles (11:30) = 90 min |
| ⛔ Amount Anomaly | $2,500 vs typical $50-300 (8x+ normal) |
| ⛔ Merchant Mismatch | Cash advance vs usual food/retail |
| ⛔ High-Risk Merchant | QuickCashAdvance (cash advance services) |
**Action Taken:** Card blocked immediately. This transaction exhibits
classic signs of stolen card credentials being used for rapid cash extraction."
},
"mcp_gateway_enabled": true,
"mcp_tools_count": 4
}
}
======================================================================
MCP GATEWAY DEMO SUMMARY
======================================================================
[OK] 1. HIGH-RISK MERCHANT - Jane at CryptoExchange123
Gateway: YES (4 tools)
[OK] 2. VELOCITY ATTACK - Bob's Rapid Transactions
Gateway: YES (4 tools)
[OK] 3. AMOUNT ANOMALY - Alice's Large Purchase
Gateway: YES (4 tools)
[OK] 4. KNOWN FRAUD PATTERNS - John's Indicator Check
Gateway: YES (4 tools)
[OK] 5. COMBINED SIGNALS - Alice's Multi-Factor Fraud
Gateway: YES (4 tools)
[OK] 6. CLEAN TRANSACTION - Jane's Normal Purchase
Gateway: YES (4 tools)

The MCP Gateway tools provide additional context (merchant reputation, risk scores, fraud indicators) that helps the agent make better decisions.

Architecture Summary

Here’s the complete architecture:

Key Takeaways

Event-driven first: Production agents are triggered by Kafka, SQS, or webhooks, not chat interfaces
Start simple: Build with Strands SDK locally, add AgentCore complexity only when needed
Guardrails early: Content filtering and topic restrictions prevent prompt injection
Memory has two levels: Short-term (session) and long-term (semantic extraction across sessions)
MCP for tools at scale: Gateway handles auth and discovery; you focus on tool logic
Observability required: X-Ray + CloudWatch are essential for debugging distributed AI
IaC for reproducibility: Pulumi deploys 40+ resources with proper dependencies

Cost Considerations

When evaluating AgentCore for production, consider these cost factors:

AgentCore Runtime charges per session-second. For fraud detection agents that typically run 10-30 seconds per transaction, this is cost-effective compared to maintaining always-on ECS containers. However, if your agents process thousands of transactions per second, the per-session costs add up. Consider batching or using Lambda for simple, fast agents.

Model costs dominate most deployments. Claude Opus 4.5 (used for reasoning) costs significantly more than Claude Haiku (used for memory extraction). We deliberately use Haiku for semantic extraction to keep memory costs low while maintaining accuracy. For high-volume production, consider whether Haiku or even smaller models could handle your primary reasoning.

Memory and Gateway have their own pricing tiers. AgentCore Memory charges for storage and retrieval operations. The Gateway adds overhead for OAuth2 token management and routing. For simple agents that don’t need cross-session memory or external tools, skip these components entirely.

Cognito user pools for authentication have a free tier (50,000 MAU) that covers most development and moderate production workloads. Beyond that, costs scale with active users.

Start with a basic deployment and add Memory/Gateway only when you have a clear need. The infrastructure cost savings of skipping unnecessary components often exceed the engineering cost of adding them later.

Conclusion

The fraud detection agent in this post grew from 30 lines of local Python to a production deployment with guardrails, dual-layer memory, MCP Gateway integration, and full observability, all defined as infrastructure as code.

The gap between prototype and production doesn’t have to be a chasm. AgentCore provides the managed runtime, Pulumi makes it reproducible, and the progressive approach lets you add complexity only when your use case demands it.

Get Started with Pulumi

Use Pulumi's open-source SDK to create, deploy, and manage infrastructure on any cloud.

I Tried Jenkins in 2025 with Pulumi: Here's How It Went

Engin Diri — Tue, 29 Jul 2025 10:35:00 +0200

It’s funny how technology has a way of sneaking back into your life just when you think you’ve moved on for good. Jenkins and I have quite the history. Think of it as that reliable but slightly temperamental friend from your college days who you haven’t seen in years.

A Blast from the Jenkins Past

The last time Jenkins and I were on speaking terms was during my tenure at my former workplace, back when the CI/CD landscape looked very different than it does today. We weren’t just casual acquaintances either. We were deep in the trenches together, orchestrating builds, managing deployments, and occasionally cursing at each other when things went sideways (which, let’s be honest, happened more often than I’d like to admit).

Those were the days when Jenkins was transitioning from its scrappy, plugin-heavy adolescence into something resembling maturity. I was part of a significant modernization effort that would make any DevOps engineer’s heart race: migrating our entire Jenkins infrastructure from Jenkins 1.x to the shiny new Jenkins 2.0. If you’ve never experienced a Jenkins major version upgrade in a production environment, imagine performing heart surgery while the patient is running a marathon. Technically possible, but requiring nerves of steel and copious amounts of coffee.

The migration wasn’t just about version numbers. Jenkins 2.0 brought with it the Pipeline plugin as a first-class citizen, transforming how we thought about continuous integration. Gone were the days of clicking through endless GUI configurations and praying that someone had documented the build process correctly. Instead, we embraced the revolutionary concept of “Pipeline as Code”, storing our build definitions in Jenkinsfile scripts that lived alongside our application code.

But the real game-changer came when we decided to containerize our Jenkins worker nodes using Docker. This was back when Docker was still the new kid on the block, and the idea of running build agents in containers felt both cutting-edge and slightly reckless. We implemented the Docker plugin for Jenkins, which allowed us to spin up fresh, isolated environments for each build. No more “but it works on my machine” syndrome. Every build got its own pristine container, complete with exactly the dependencies it needed.

The transformation was remarkable. What used to take us hours of manual environment setup and dependency management became as simple as defining a Dockerfile. Our build agents went from being precious, hand-crafted snowflakes that required constant care and feeding to becoming ephemeral, reproducible environments that could be destroyed and recreated at will.

Here’s what a typical Jenkinsfile looked like during that era. Notice how we could seamlessly combine different technologies and services, each running in their own containers:

pipeline {
 agent none
 stages {
 stage('Back-end') {
 agent {
 docker { image 'maven:3.9.10-eclipse-temurin-21-alpine' }
 }
 steps {
 sh 'mvn clean compile test'
 junit 'target/surefire-reports/*.xml'
 }
 }
 stage('Front-end') {
 agent {
 docker { image 'node:22.17.0-alpine3.22' }
 }
 steps {
 sh 'npm install && npm test'
 }
 }
 stage('Integration Tests') {
 agent any
 steps {
 script {
 docker.image('postgres:13').withRun('-e POSTGRES_PASSWORD=test') { db ->
 docker.image('redis:7-alpine').withRun() { redis ->
 docker.build('myapp:test').inside("--link ${db.id}:postgres --link ${redis.id}:redis") {
 sh 'npm run integration-tests'
 }
 }
 }
 }
 }
 }
 }
}

This pipeline definition encapsulated everything we’d learned about containerized CI/CD: isolated environments, service orchestration, and the beautiful simplicity of treating infrastructure as disposable. Each stage could use exactly the tools it needed, without worrying about conflicts or environmental drift.

Fast forward to 2025, and here I am, staring at a Jenkins instance again. But this time, there’s a twist: instead of the traditional XML configurations and GUI-heavy setup processes I remember, I’m armed with Pulumi. This tool promises to bring infrastructure-as-code practices to Jenkins deployment itself. The irony isn’t lost on me: using modern infrastructure tooling to deploy a CI/CD platform that helped define what modern infrastructure tooling could become.

So buckle up for this nostalgic yet forward-looking journey. We’re about to find out whether Jenkins has kept up with the times, and more importantly, whether the combination of Jenkins and Pulumi can create something greater than the sum of its parts.

How I’d Set Up Jenkins in 2025: The Kubernetes Way

Fast forward to 2025, and the infrastructure landscape has evolved dramatically. While my old Jenkins setup was running on virtual machines with Docker containers as build agents, the modern approach demands something more cloud-native. Today, if you’re serious about CI/CD at scale, Kubernetes isn’t just an option. It’s practically a requirement.

The beauty of running Jenkins on Kubernetes lies in its perfect marriage of concepts. Jenkins already understood the value of ephemeral build agents, and Kubernetes takes that philosophy to its logical conclusion: everything is ephemeral, everything is disposable, and everything can be recreated on demand. Your Jenkins controller becomes just another workload in your cluster, and your build agents become pods that spin up, do their work, and disappear without a trace.

But here’s where it gets interesting. Instead of clicking through installation wizards or crafting complex shell scripts, I’m going to use Pulumi to define my entire Jenkins infrastructure as code. This isn’t just about following the latest trends. It’s about bringing the same “everything as code” philosophy that made Jenkins 2.0 so compelling to the infrastructure layer itself.

The Pulumi Advantage

Why Pulumi for this task? After years of wrestling with YAML manifests and Helm charts, I’ve come to appreciate Pulumi’s approach to infrastructure as code. With Pulumi, I can leverage real programming languages (in this case, Python) to define my infrastructure, complete with loops, conditionals, and all the abstractions that make complex deployments manageable.

Let me show you what a modern Jenkins deployment looks like when we combine Pulumi’s infrastructure-as-code capabilities with Kubernetes’ orchestration power. This isn’t just theoretical. This is production-ready code that you could deploy in any Kubernetes cluster.

Setting Up Our Local Kubernetes Cluster with Kind

Before we dive into the Pulumi code, we need a Kubernetes cluster to deploy to. For this demonstration, I’m using Kind (Kubernetes in Docker), which creates a local Kubernetes cluster using Docker containers as nodes. It’s perfect for development, testing, and learning scenarios like this.

Kind has become the de facto standard for local Kubernetes development because it’s lightweight, fast, and provides a genuine multi-node Kubernetes experience without the overhead of virtual machines. Plus, it’s what the Kubernetes project itself uses for testing, so you know it’s reliable.

Creating the Cluster:

Once Kind is installed, creating a cluster is straightforward:

kind create cluster --name jenkins-cluster

This command spins up a single-node Kubernetes cluster running in a Docker container. The beauty of Kind is that it configures your kubectl context automatically, so you’re immediately ready to start deploying workloads.

You can verify your cluster is running:

kubectl cluster-info --context kind-jenkins-cluster
kubectl get nodes

For our Jenkins deployment, the default single-node cluster is sufficient, but Kind also supports multi-node clusters if you need to test more complex scenarios. The cluster will persist until you explicitly delete it with

kind delete cluster --name jenkins-cluster

Now that we have our Kubernetes cluster ready, let’s deploy Jenkins using Pulumi.

import pulumi
import pulumi_kubernetes as k8s
from pulumi_kubernetes.helm.v3 import Chart, ChartOpts, FetchOpts

# Configuration - making it parameterizable
config = pulumi.Config()
namespace_name = config.get("namespace", "jenkins")
storage_class = config.get("storageClass", "standard") # Kind uses 'standard' by default
jenkins_admin_password = config.require_secret("adminPassword")

# Create a dedicated namespace for Jenkins
jenkins_namespace = k8s.core.v1.Namespace(
 "jenkins-namespace",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name=namespace_name,
 labels={"app": "jenkins", "managed-by": "pulumi"}
 )
)

# Create persistent storage for Jenkins
jenkins_pvc = k8s.core.v1.PersistentVolumeClaim(
 "jenkins-pvc",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="jenkins-pv-claim",
 namespace=jenkins_namespace.metadata.name,
 ),
 spec=k8s.core.v1.PersistentVolumeClaimSpecArgs(
 access_modes=["ReadWriteOnce"],
 storage_class_name=storage_class,
 resources=k8s.core.v1.ResourceRequirementsArgs(
 requests={"storage": "20Gi"}
 )
 )
)

# Create a service account for Jenkins with proper RBAC
jenkins_service_account = k8s.core.v1.ServiceAccount(
 "jenkins-service-account",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="jenkins",
 namespace=jenkins_namespace.metadata.name,
 )
)

# Create cluster role for Jenkins to manage pods (for Kubernetes plugin)
jenkins_cluster_role = k8s.rbac.v1.ClusterRole(
 "jenkins-cluster-role",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="jenkins-cluster-role",
 ),
 rules=[
 k8s.rbac.v1.PolicyRuleArgs(
 api_groups=[""],
 resources=["pods", "pods/exec", "pods/log", "persistentvolumeclaims"],
 verbs=["get", "list", "watch", "create", "update", "patch", "delete"]
 ),
 k8s.rbac.v1.PolicyRuleArgs(
 api_groups=["apps"],
 resources=["deployments", "replicasets"],
 verbs=["get", "list", "watch", "create", "update", "patch", "delete"]
 )
 ]
)

# Bind the cluster role to the service account
jenkins_cluster_role_binding = k8s.rbac.v1.ClusterRoleBinding(
 "jenkins-cluster-role-binding",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="jenkins-cluster-role-binding",
 ),
 role_ref=k8s.rbac.v1.RoleRefArgs(
 api_group="rbac.authorization.k8s.io",
 kind="ClusterRole",
 name=jenkins_cluster_role.metadata.name
 ),
 subjects=[
 k8s.rbac.v1.SubjectArgs(
 kind="ServiceAccount",
 name=jenkins_service_account.metadata.name,
 namespace=jenkins_namespace.metadata.name
 )
 ]
)

# Create a secret for Jenkins admin credentials
jenkins_admin_secret = k8s.core.v1.Secret(
 "jenkins-admin-secret",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="jenkins-admin-secret",
 namespace=jenkins_namespace.metadata.name,
 ),
 string_data={
 "jenkins-admin-user": "admin",
 "jenkins-admin-password": jenkins_admin_password,
 }
)

# Deploy Jenkins using the official Helm chart from https://charts.jenkins.io
jenkins_chart = Chart(
 "jenkins",
 ChartOpts(
 chart="jenkins",
 version="5.8.72", # Using latest stable version
 namespace=jenkins_namespace.metadata.name,
 fetch_opts=FetchOpts(
 repo="https://charts.jenkins.io" # Official Jenkins Helm repository
 ),
 values={
 "controller": {
 "admin": {
 "existingSecret": jenkins_admin_secret.metadata.name,
 "userKey": "jenkins-admin-user",
 "passwordKey": "jenkins-admin-password",
 },
 "resources": {
 "requests": {
 "cpu": "500m",
 "memory": "1Gi"
 },
 "limits": {
 "cpu": "2000m",
 "memory": "4Gi"
 }
 },
 "installPlugins": [
 "kubernetes",
 "workflow-aggregator",
 "git",
 "configuration-as-code",
 "blueocean" # Modern Jenkins UI
 ],
 "JCasC": { # Jenkins Configuration as Code
 "enabled": True,
 "defaultConfig": True,
 "configScripts": {
 "kubernetes-config": '''
 jenkins:
 clouds:
 - kubernetes:
 name: "kubernetes"
 serverUrl: "https://kubernetes.default:443"
 namespace: "''' + namespace_name + '''"
 credentialsId: ""
 jenkinsTunnel: "jenkins-agent:50000"
 jenkinsUrl: "http://jenkins:8080"
 maxRequestsPerHostStr: 64
 retentionTimeout: 5
 connectTimeout: 10
 templates:
 - name: "pulumi-agent"
 namespace: "''' + namespace_name + '''"
 label: "pulumi"
 nodeUsageMode: NORMAL
 containers:
 - name: "pulumi"
 image: "pulumi/pulumi:latest"
 alwaysPullImage: true
 workingDir: "/home/jenkins/agent"
 command: ""
 args: ""
 resourceRequestCpu: "500m"
 resourceRequestMemory: "1Gi"
 resourceLimitCpu: "2000m"
 resourceLimitMemory: "4Gi"
 serviceAccount: "jenkins"
 - name: "default"
 namespace: "''' + namespace_name + '''"
 label: "jenkins-agent"
 nodeUsageMode: NORMAL
 containers:
 - name: "jnlp"
 image: "jenkins/inbound-agent:latest"
 alwaysPullImage: true
 workingDir: "/home/jenkins/agent"
 command: ""
 args: ""
 resourceRequestCpu: "500m"
 resourceRequestMemory: "512Mi"
 resourceLimitCpu: "1000m"
 resourceLimitMemory: "1Gi"
 '''
 }
 }
 },
 "serviceAccount": {
 "create": False,
 "name": jenkins_service_account.metadata.name
 },
 "persistence": {
 "existingClaim": jenkins_pvc.metadata.name
 },
 "agent": {
 "enabled": True,
 "namespace": jenkins_namespace.metadata.name
 }
 }
 )
)

# Export useful information
pulumi.export("namespace", jenkins_namespace.metadata.name)
pulumi.export("jenkins_url", "http://jenkins.{}.svc.cluster.local:8080".format(namespace_name))
pulumi.export("admin_user", "admin")

In the browser, you can access Jenkins at http://localhost:8080 (or the appropriate URL for your cluster).

What Makes This Approach Special

This Pulumi program does several things that would be much more cumbersome with traditional approaches:

Infrastructure as Code: Every aspect of the Jenkins deployment is version-controlled and reproducible. No more “it works on my machine” scenarios with infrastructure.

Kubernetes-Native: The deployment leverages Kubernetes’ native capabilities for service discovery, persistent storage, and RBAC, making Jenkins a first-class citizen in your cluster.

Configuration as Code: Using Jenkins Configuration as Code (JCasC), the Jenkins instance comes pre-configured with the Kubernetes plugin and agent templates, eliminating manual setup steps.

Security by Default: Proper RBAC configurations ensure Jenkins has exactly the permissions it needs, no more and no less.

Scalable Agent Model: Build agents are ephemeral pods that scale based on demand, just like the containerized agents we used years ago, but with Kubernetes orchestration.

The beauty of this approach is that it combines the lessons learned from our Docker-based Jenkins setup with modern cloud-native practices. Agents are still containerized and ephemeral, but now they’re managed by Kubernetes rather than the Docker plugin. Storage is persistent but managed by Kubernetes PVCs. And everything is defined as code using a real programming language rather than YAML.

This setup gives you a production-ready Jenkins instance that can handle serious workloads while being completely reproducible and maintainable. But more importantly, it sets the foundation for the kind of advanced CI/CD practices that modern teams need. These practices go far beyond what we could achieve with our old VM-based approach.

Configuring Our First Pipeline to Deploy a Pulumi Program

Now comes the fun part: putting our Kubernetes-based Jenkins setup to work with a real-world scenario. What better way to test our new infrastructure than by creating a pipeline that deploys infrastructure itself? We’re going to configure Jenkins to automatically deploy Pulumi programs using Kubernetes agents, creating a complete infrastructure-as-code CI/CD workflow.

This scenario hits close to home for any modern DevOps team. Gone are the days when infrastructure changes required manual server configurations or clicking through cloud provider consoles. Today’s teams need automated, reliable ways to manage infrastructure changes through the same rigorous processes they use for application code, including peer reviews, automated testing, and controlled deployments.

The Power of Kubernetes Agents

One of the most compelling features of our Kubernetes-based Jenkins setup is its ability to spawn dedicated agent pods for specific workloads. Unlike traditional Jenkins agents that run on persistent VMs or containers, Kubernetes agents are truly ephemeral. They exist only for the duration of a single build, then disappear without a trace.

This approach solves several problems that plagued our old Docker-based setup:

Resource Efficiency: Agents consume cluster resources only when actively building, not sitting idle waiting for work.

Perfect Isolation: Each pipeline run gets its own dedicated environment with no risk of interference from previous builds.

Unlimited Scalability: Need to run 50 concurrent builds? Kubernetes will happily schedule 50 agent pods across your cluster.

Tool-Specific Environments: Different pipelines can use completely different toolchains without conflicts.

For Pulumi deployments, this means we can create specialized agent pods that include exactly the tools we need: the Pulumi CLI, cloud provider SDKs, and any additional dependencies our infrastructure code requires.

A Complete Pulumi Deployment Pipeline

Now let’s create a Jenkinsfile that demonstrates the full power of our setup. This pipeline will:

Check out a Pulumi program from source control
Install dependencies and validate the Pulumi code
Run a preview to show planned changes
Deploy the infrastructure changes (with manual approval for production)
Clean up temporary resources

pipeline {
 agent {
 kubernetes {
 label 'pulumi-deployment'
 yaml """
apiVersion: v1
kind: Pod
spec:
 serviceAccountName: jenkins
 containers:
 - name: pulumi
 image: pulumi/pulumi:latest
 command:
 - sleep
 args:
 - 99d
 env:
 - name: PULUMI_SKIP_UPDATE_CHECK
 value: "true"
 resources:
 requests:
 memory: "1Gi"
 cpu: "500m"
 limits:
 memory: "4Gi"
 cpu: "2000m"
"""
 }
 }

 parameters {
 string(name: 'GIT_URL', defaultValue: 'https://github.com/pulumi/examples', description: 'Git repository URL') // Official Pulumi examples
 string(name: 'PULUMI_FOLDER', defaultValue: 'aws-ts-s3-folder', description: 'Folder containing Pulumi program')
 }

 environment {
 PULUMI_ACCESS_TOKEN = credentials('pulumi-access-token')
 }

 stages {
 stage('Checkout') {
 steps {
 git url: "${params.GIT_URL}",
 branch: "master"
 }
 }

 stage('Install Dependencies') {
 steps {
 container('pulumi') {
 sh '''
 # Navigate to the Pulumi program directory
 cd ${PULUMI_FOLDER}/

 # Install Python dependencies if using Python
 if [ -f "requirements.txt" ]; then
 pip install -r requirements.txt
 fi

 # Install Node.js dependencies if using TypeScript/JavaScript
 if [ -f "package.json" ]; then
 npm install
 fi

 # Install Go dependencies if using Go
 if [ -f "go.mod" ]; then
 go mod download
 fi

 # Login to Pulumi service
 pulumi login

 echo "environment:\n- pulumi-ultimate-gitops/dev" > Pulumi.dev.yaml
 cat Pulumi.dev.yaml
 '''
 }
 }
 }

 stage('Preview Changes') {
 steps {
 container('pulumi') {
 sh '''
 cd ${PULUMI_FOLDER}/
 pulumi stack select dev

 # Generate a preview of changes
 pulumi preview --diff --cwd . > preview.txt

 # Display the preview
 cat preview.txt
 '''
 }

 // Archive the preview for later reference
 archiveArtifacts artifacts: "${params.PULUMI_FOLDER}/preview.txt", allowEmptyArchive: true
 }
 }

 stage('Approval') {
 steps {
 script {
 def preview = readFile "${params.PULUMI_FOLDER}/preview.txt"
 def userInput = input(
 id: 'deploy-approval',
 message: 'Deploy these infrastructure changes?',
 parameters: [
 text(name: 'PREVIEW', defaultValue: preview, description: 'Preview of changes')
 ],
 submitterParameter: 'APPROVER'
 )
 env.APPROVER = userInput
 }
 }
 }

 stage('Deploy Infrastructure') {
 steps {
 container('pulumi') {
 sh '''
 cd ${PULUMI_FOLDER}/
 pulumi stack select dev
 pulumi up --yes --cwd .
 '''
 }
 }
 }
 }

 post {
 always {
 sh "rm -f ${params.PULUMI_FOLDER}/preview.txt"
 }

 success {
 script {
 if (env.APPROVER) {
 echo "Infrastructure deployment completed successfully for stack ${env.PULUMI_STACK}. Approved by: ${env.APPROVER}"
 }
 }
 }

 failure {
 echo "Infrastructure deployment failed for stack ${env.PULUMI_STACK}. Check the build logs for details."
 }
 }
}

Open the Jenkins UI, create a new pipeline job, and paste this Jenkinsfile into the pipeline configuration.

Now we can run this pipeline to deploy a Pulumi program.

Show the preview of changes to be applied:

And wait for manual approval before proceeding with the deployment:

Key Features of This Pipeline

This Jenkinsfile demonstrates several advanced concepts that make modern CI/CD pipelines robust and production-ready:

Kubernetes-Native Agent Pods: The pipeline uses a dedicated Kubernetes pod with a Pulumi-specific container image, providing an isolated environment with all necessary tools pre-installed. The pod exists only for the duration of the build, ensuring perfect isolation between pipeline runs.

Parameterized Deployments: The pipeline accepts parameters for Git repository URL and Pulumi folder path, making it reusable across different Pulumi projects and repositories. This flexibility allows teams to standardize their deployment process while supporting diverse infrastructure codebases.

Multi-Language Support: The dependency installation stage automatically detects and installs dependencies for Python (requirements.txt), Node.js/TypeScript (package.json), and Go (go.mod) projects, making the pipeline adaptable to any Pulumi program regardless of the chosen language.

Manual Approval Gates: All deployments require explicit manual approval, with the complete preview of infrastructure changes displayed to the approver. This ensures that infrastructure modifications are always reviewed by a human before being applied to live environments.

Stack Configuration Management: The pipeline dynamically creates and configures Pulumi stack files, demonstrating how to manage environment-specific configurations programmatically within the CI/CD process.

Comprehensive Artifact Management: Preview outputs are automatically archived for audit trails and debugging, providing a historical record of what changes were planned and approved for each deployment.

You might also like:

Wrap Up

After this deep dive into Jenkins with Pulumi in 2025, I’m reminded of why Jenkins became the workhorse of CI/CD in the first place. It’s reliable, it’s battle-tested, and most importantly, it adapts. Just as we successfully containerized our build agents years ago, Jenkins has seamlessly evolved to thrive in the Kubernetes era.

The transformation from those early days of XML configurations and GUI-heavy setup processes to today’s infrastructure-as-code approach is remarkable. What we’ve built here (a fully automated Jenkins deployment with Kubernetes agents and Pulumi integration) represents the kind of setup that would have been a months-long project back in the day. Now, it’s defined in a single Python file and deployable in minutes.

But here’s the honest truth: Would I choose Jenkins for a greenfield project today? Probably not.

The modern DevOps landscape offers compelling alternatives that were purpose-built for cloud-native environments. GitHub Actions provides seamless integration with source control and requires zero infrastructure management. GitLab CI/CD offers similar convenience with its integrated approach. Cloud-native solutions like Tekton or Argo Workflows were designed from the ground up for Kubernetes, without the historical baggage that Jenkins carries.

These newer platforms offer several advantages over Jenkins:

Zero infrastructure overhead: No need to manage controllers, agents, or storage
Native cloud integration: Built-in support for modern authentication, secrets management, and scaling
Simpler configuration: Pipelines without the complexity of Jenkins’ plugin ecosystem
Better resource efficiency: No persistent controllers consuming resources when idle

However, Jenkins still has its place. If you’re working with legacy systems, have extensive existing Jenkins infrastructure, or need the ultimate flexibility that comes from its massive plugin ecosystem, Jenkins remains a solid choice. The setup we’ve demonstrated proves that Jenkins can absolutely hold its own in modern environments.

The beauty of what we’ve accomplished here isn’t just about Jenkins. It’s about the power of infrastructure-as-code. Whether you choose Jenkins, GitHub Actions, or any other CI/CD platform, the principles remain the same: define your infrastructure declaratively, version control everything, and automate the deployment process.

Jenkins may be the old workhorse, but as we’ve seen, this workhorse still has plenty of life left in it.

Next Steps

This post demonstrated how to modernize Jenkins deployment using Pulumi and Kubernetes. By leveraging infrastructure-as-code principles, we created a completely reproducible CI/CD platform that bridges traditional practices with cloud-native approaches.

If you want to explore more modern CI/CD approaches or dive deeper into infrastructure automation with Pulumi, check out our documentation and try building your own infrastructure-as-code solutions. For more insights on DevOps best practices, explore our guide on IaC best practices.

Try Pulumi for Free

Enforcing Policy as Code on Discovered Resources with Pulumi

Tyler D — Tue, 04 Feb 2025 00:00:00 +0000

In this post, we’re introducing a powerful new capability in Pulumi Insights that extends policy as code (PaC) beyond infrastructure as code to automatically govern all cloud resources in your environment. By unifying policy enforcement across both IaC and discovered resources, you can now write policies once and apply them universally - dramatically simplifying how organizations maintain security and compliance standards at scale.

What is Policy as Code (PaC)?

Policy as Code (PaC) is a way to define and enforce infrastructure policies using code. Instead of relying on manual processes or static configuration files, PaC enables automated policy enforcement in a consistent, versioned way. Pulumi’s CrossGuard feature allows you to write policies in general purpose programming languages, applying them across various cloud platforms such as AWS, Azure, GCP, and Kubernetes.

You can read more about CrossGuard in Pulumi’s CrossGuard documentation.

Accounts and Insights

Before diving into how you can configure PaC for discovered resources, it’s important to understand how Pulumi discovers resources within your cloud environment regardless of how they were created. Pulumi Insights provides visibility into the state of your cloud resources by using read only access to your cloud environment in order to catalog the resources it contains. This is done through the Accounts page in Pulumi, which shows all the accounts for discovering resources you have added to your Pulumi organization. On this page, you can add new accounts which you can then add to policy groups to begin evaluating all of your cloud resources using your organization’s policy as code.

After you have added an account, you can connect it with one or multiple Policy Groups to enforce the policies within those groups across your stacks and Insights accounts. Once linked to a Policy Group, your resources will be automatically evaluated against the policies you define whenever those policies or resources are updated.

Setting Up Policy Groups for Discovered Resources

To apply policies to your discovered resources, you first need to set up a Policy Group. A Policy Group in Pulumi enforces policies across a group of stacks and accounts in your organization. Each Policy Group can contain multiple stacks and accounts, and you can assign multiple Policy Packs to these groups.

Here’s an example of the Policy Group configuration page, where you can create and manage policy groups:

Once your Policy Group is set up, you can add Insights Accounts to the group, which will ensure that the resources in those accounts are evaluated against the policies in the group.

Running Policies on Discovered Resources

Now that your Policy Group is set up, you can start running policies against your discovered resources. Currently, Pulumi Insights supports running resource policies against all resource types supported by Pulumi Insights with support for stack policies coming soon. Pulumi Insights will evaluate each resource against the policies defined in your Policy Pack. If the resources violate any of the defined policies, those violations will show up on the Policy Violations page, which gives you a detailed view of any non-compliant resources.

Each violation includes details about the resource and the reason for the violation, helping you quickly identify and address issues.

Key Takeaways

With Pulumi’s new PaC over discovered resources feature, you can now apply the same policies to both IaC and discovered resources across your cloud environment. This enables you to write a policy once and have it apply across all of your cloud resources. This results in consistent, compliant infrastructure no matter how resources are provisioned.

To begin securing your cloud infrastructure, here are the steps to take:

Set up Policy Groups to enforce policies across stacks and accounts.
Link Insights Accounts to your Policy Groups.
View violations on the Policy Violations page and take action to resolve them.

This new capability is available for AWS, Azure, OCI, and Kubernetes. Stay tuned for more updates on policy enforcement and discover how you can leverage Pulumi for cloud security and compliance.

Try it yourself: Start enforcing policies across your discovered resources in minutes with Pulumi Insights. Check out our getting started guide to begin securing your entire cloud environment with policy as code.

For more information on getting started with Policy as Code, check out the following resources:

Run Open-Source LLMs on AWS EC2 with Ollama and Pulumi

Engin Diri — Mon, 27 Jan 2025 00:00:00 +0000

TL;DR. Want to self-host an open-source LLM on AWS? Use a g4dn.xlarge ($0.526/hr on-demand, 16 GB GPU memory) for 7B/8B models, a g5.xlarge ($1.006/hr, 24 GB) for 13B–14B models, a g5.2xlarge ($1.212/hr, 24 GB) for 32B models, or a g6e.2xlarge ($2.242/hr, 48 GB) for 70B models. Deploy with the Pulumi program below and Ollama will run any model from its library: DeepSeek-R1, Llama 3, Qwen, or Mistral, with a one-line change.

This guide walks through that deployment end-to-end: a single Pulumi program that provisions a GPU-enabled EC2 instance, installs Ollama and Open WebUI via cloud-init, and exposes both a chat UI and an OpenAI-compatible API. The model is configurable, so you can swap DeepSeek-R1 for Llama 3.1, Qwen 2.5, or Mistral without touching the infrastructure code.

Why run open-source LLMs on AWS EC2?

Self-hosting an open-source LLM on AWS gives you three things hosted APIs can’t: data stays inside your VPC, per-token costs collapse to a flat hourly rate at high volume, and you can fine-tune or quantize models freely under permissive licenses. Ollama handles all three concerns from a single binary: it downloads, manages, and serves models behind an OpenAI-compatible API on port 11434.

The original version of this post focused on DeepSeek-R1 because it landed in late January 2025 and reset expectations for what an open-weight reasoning model could do. DeepSeek-R1 is still an excellent default (MIT-licensed, strong on math and coding, with distilled 1.5B–70B variants) but the same infrastructure runs Meta’s Llama 3, Alibaba’s Qwen, and Mistral equally well. Picking a model is now a config change, not an infrastructure decision.

For reference, DeepSeek-R1 scores 79.8% on AIME 2024 (vs. 79.2% for OpenAI o1-1217), 97.3% on MATH-500 (vs. 96.4%), 96.3% on Codeforces (vs. 96.6%), and 49.2% on SWE-bench Verified (vs. 48.9%)—near-parity with closed frontier models on most reasoning benchmarks, with the same caveat that benchmark scores age fast.

You might also like:

Which models can I run, and which EC2 instance do I need?

The bottleneck for inference is GPU memory (VRAM): the model weights have to fit in it, plus a few GB of headroom for the KV cache. The table below maps the most common Ollama models to the smallest AWS GPU instance that comfortably runs them at 4-bit (Q4) quantization, which is what ollama pull <model> gives you by default.

Model family	Sizes	Approx. VRAM (Q4)	Smallest EC2 instance	On-demand price (us-east-1)
DeepSeek-R1 (distill)	1.5B / 7B / 8B	1–6 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Llama 3.1 / Llama 3.2	8B	~5 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Qwen 2.5	7B	~5 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
Mistral 7B / Mistral Nemo	7B / 12B	5–8 GB	`g4dn.xlarge` (T4, 16 GB)	$0.526/hr
DeepSeek-R1 (distill)	14B	~10 GB	`g5.xlarge` (A10G, 24 GB)	$1.006/hr
Llama 3.3 / DeepSeek-R1	32B / 32B distill	~20 GB	`g5.2xlarge` (A10G, 24 GB)	$1.212/hr
Llama 3.1 / DeepSeek-R1	70B	~42 GB	`g6e.2xlarge` (L40S, 48 GB)	$2.242/hr
DeepSeek-R1 (full)	671B (MoE)	400 GB+	`p5.48xlarge` or multi-node	$98.32/hr

For most workloads—internal tools, RAG backends, code assistants—a g4dn.xlarge running an 8B model is the right starting point. Move up only if quality is the bottleneck.

How much does this cost vs. hosted APIs?

A g4dn.xlarge running 24/7 costs ~$378/month on-demand, or ~$237/month with a 1-year reserved instance. Whether that’s cheaper than a hosted API depends entirely on your token volume.

Compare against hosted pricing as of April 2026 (input + output blended, rough numbers):

Provider	Model	Approx. blended price
OpenAI	GPT-4o-mini	~$0.30 per 1M tokens
OpenAI	GPT-4o	~$5.00 per 1M tokens
Anthropic	Claude Sonnet 4	~$6.00 per 1M tokens
DeepSeek (hosted)	DeepSeek-V3	~$0.50 per 1M tokens

A g4dn.xlarge running Llama 3.1 8B sustains roughly 40–60 tokens/sec under single-user load, or about 100–155M tokens/month at 100% utilization. At that ceiling the effective rate is ~$2.40/M tokens—cheaper than GPT-4o or Claude, more expensive than GPT-4o-mini or DeepSeek’s own hosted API.

The takeaway: self-hosting wins on data residency, latency, and predictable cost at high utilization. Hosted APIs win below ~10M tokens/month or when you need frontier-class quality. Run the math against your actual token volume before committing.

How do I deploy Ollama on AWS EC2 with Pulumi?

Prerequisites

Before we start, make sure you have the following:

An AWS account
Pulumi CLI installed
AWS CLI installed and configured
A working understanding of Ollama

What is Ollama?

Ollama is an open-source runtime that downloads, manages, and serves large language models from a single binary. It runs on macOS, Linux, and Windows, supports GPU acceleration through CUDA and Metal, and exposes both a native HTTP API and an OpenAI-compatible API on port 11434. Most clients written for the OpenAI SDK work against an Ollama endpoint with only a base-URL change.

It supports the major open-weight families—DeepSeek-R1, Llama 3, Qwen, Mistral, Gemma, Phi—plus quantized and distilled variants for each. You pull a model by name and tag (ollama pull llama3.1:8b) and run it (ollama run llama3.1:8b); Ollama handles the rest.

Architecture

Create a new Pulumi project

First, scaffold a new Pulumi project. Run the following from an empty directory:

# Replace <language> with typescript, python, go, csharp, or yaml
pulumi new aws-<language>

Pick the language you are most comfortable with. The template installs the AWS provider and creates a working sample. You can delete the sample code—we’ll replace it with the snippets below.

Step 1: Create an instance role with S3 access

The EC2 instance needs to download NVIDIA drivers from a public AWS-managed S3 bucket. Create an IAM role with S3 read access and attach it to an instance profile:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as fs from "fs";

const role = new aws.iam.Role("deepSeekRole", {
 name: "deepseek-role",
 assumeRolePolicy: JSON.stringify({
 Version: "2012-10-17",
 Statement: [
 {
 Action: "sts:AssumeRole",
 Effect: "Allow",
 Principal: {
 Service: "ec2.amazonaws.com",
 },
 },
 ],
 }),
});

new aws.iam.RolePolicyAttachment("deepSeekS3Policy", {
 policyArn: "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 role: role.name,
});

const instanceProfile = new aws.iam.InstanceProfile("deepSeekProfile", {
 name: "deepseek-profile",
 role: role.name,
});

import pulumi
import pulumi_aws as aws
import json
import os

# IAM Role for EC2 instances
role = aws.iam.Role(
 "deepSeekRole",
 name="deepseek-role",
 assume_role_policy=json.dumps(
 {
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "ec2.amazonaws.com",
 },
 }
 ],
 }
 ),
)

# Attach S3 read-only policy to the IAM Role
iam_policy_attachment = aws.iam.RolePolicyAttachment(
 "deepSeekS3Policy",
 policy_arn="arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 role=role.name,
)

# Instance Profile containing the IAM Role
instance_profile = aws.iam.InstanceProfile(
 "deepSeekProfile", name="deepseek-profile", role=role.name
)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 rolePolicy, err := json.Marshal(map[string]interface{}{
 "Version": "2012-10-17",
 "Statement": []map[string]interface{}{
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": map[string]interface{}{"Service": "ec2.amazonaws.com"},
 },
 },
 })
 if err != nil {
 return err
 }

 role, err := iam.NewRole(ctx, "deepSeekRole", &iam.RoleArgs{
 Name: pulumi.String("deepseek-role"),
 AssumeRolePolicy: pulumi.String(rolePolicy),
 })
 if err != nil {
 return err
 }

 // Attach S3 read-only policy to the IAM Role
 _, err = iam.NewRolePolicyAttachment(ctx, "deepSeekS3Policy", &iam.RolePolicyAttachmentArgs{
 PolicyArn: pulumi.String("arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"),
 Role: role.Name,
 })
 if err != nil {
 return err
 }

 // Instance Profile containing the IAM Role
 instanceProfile, err := iam.NewInstanceProfile(ctx, "deepSeekProfile", &iam.InstanceProfileArgs{
 Name: pulumi.String("deepseek-profile"),
 Role: role.Name,
 })
 if err != nil {
 return err
 }
 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 {
 ["Version"] = "2012-10-17",
 ["Statement"] = new[]
 {
 new Dictionary<string, object>
 {
 ["Action"] = "sts:AssumeRole",
 ["Effect"] = "Allow",
 ["Principal"] = new Dictionary<string, string>
 {
 ["Service"] = "ec2.amazonaws.com"
 }
 }
 }
 });

 var role = new Role("deepSeekRole", new RoleArgs
 {
 Name = "deepseek-role",
 AssumeRolePolicy = rolePolicy
 });

 var rolePolicyAttachment = new RolePolicyAttachment("deepSeekS3Policy", new RolePolicyAttachmentArgs
 {
 PolicyArn = "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
 Role = role.Name
 });

 var instanceProfile = new InstanceProfile("deepSeekProfile", new InstanceProfileArgs
 {
 Name = "deepseek-profile",
 Role = role.Name
 });

 var vpc = new Vpc("deepSeekVpc", new VpcArgs
 {
 CidrBlock = "10.0.0.0/16",
 EnableDnsHostnames = true,
 EnableDnsSupport = true
 });

 var subnet = new Subnet("deepSeekSubnet", new SubnetArgs
 {
 VpcId = vpc.Id,
 CidrBlock = "10.0.48.0/20",
 AvailabilityZone = "eu-central-1a",
 MapPublicIpOnLaunch = true
 });

 var internetGateway = new InternetGateway("deepSeekInternetGateway", new InternetGatewayArgs
 {
 VpcId = vpc.Id
 });

 var routeTable = new RouteTable("deepSeekRouteTable", new RouteTableArgs

name: deepseek-ollama-yaml
description: DeepSeek Ollama AWS example
runtime: yaml

variables:
 publicKey:
 fn::readFile: ./deepseek.rsa
 userData:
 fn::readFile: ./cloud-init.yaml
 amiFilter: "amzn2-ami-hvm-*-x86_64-gp2"
 amiOwner: "137112412989"
 amiId:
 fn::invoke:
 function: aws:ec2:getAmi
 arguments:
 filters:
 - name: name
 values: ["${amiFilter}"]
 owners: ["${amiOwner}"]
 mostRecent: true
 return: id

resources:
 deepSeekRole:
 type: aws:iam:Role
 properties:
 name: deepseek-role
 assumeRolePolicy: |
 {
 "Version": "2012-10-17",
 "Statement": [
 {
 "Action": "sts:AssumeRole",
 "Effect": "Allow",
 "Principal": {
 "Service": "ec2.amazonaws.com"
 }
 }
 ]
 }

 deepSeekS3Policy:
 type: aws:iam:RolePolicyAttachment
 properties:
 role: ${deepSeekRole.name}
 policyArn: arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess

 deepSeekProfile:
 type: aws:iam:InstanceProfile
 properties:
 name: deepseek-profile
 role: ${deepSeekRole.name}

Step 2: Create the network

Next, create the VPC, subnet, internet gateway, and route table. The security group opens ports 22 (SSH), 3000 (Open WebUI), and 11434 (Ollama API):

const vpc = new aws.ec2.Vpc("deepSeekVpc", {
 cidrBlock: "10.0.0.0/16",
 enableDnsHostnames: true,
 enableDnsSupport: true,
});

const subnet = new aws.ec2.Subnet("deepSeekSubnet", {
 vpcId: vpc.id,
 cidrBlock: "10.0.48.0/20",
 availabilityZone: pulumi.interpolate`${aws.getAvailabilityZones().then(it => it.names[0])}`,
 mapPublicIpOnLaunch: true,
});

const internetGateway = new aws.ec2.InternetGateway("deepSeekInternetGateway", {
 vpcId: vpc.id,
});

const routeTable = new aws.ec2.RouteTable("deepSeekRouteTable", {
 vpcId: vpc.id,
 routes: [
 {
 cidrBlock: "0.0.0.0/0",
 gatewayId: internetGateway.id,
 },
 ],
});

const routeTableAssociation = new aws.ec2.RouteTableAssociation("deepSeekRouteTableAssociation", {
 subnetId: subnet.id,
 routeTableId: routeTable.id,
});

const securityGroup = new aws.ec2.SecurityGroup("deepSeekSecurityGroup", {
 vpcId: vpc.id,
 egress: [
 {
 fromPort: 0,
 toPort: 0,
 protocol: "-1",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
 ingress: [
 {
 fromPort: 22,
 toPort: 22,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 {
 fromPort: 3000,
 toPort: 3000,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 {
 fromPort: 11434,
 toPort: 11434,
 protocol: "tcp",
 cidrBlocks: ["0.0.0.0/0"],
 },
 ],
});


# Create a VPC
vpc = aws.ec2.Vpc(
 "deepSeekVpc",
 cidr_block="10.0.0.0/16",
 enable_dns_hostnames=True,
 enable_dns_support=True,
)

# Create a subnet
subnet = aws.ec2.Subnet(
 "deepSeekSubnet",
 vpc_id=vpc.id,
 cidr_block="10.0.48.0/20",
 availability_zone="eu-central-1a",
 map_public_ip_on_launch=True,
)

# Create an internet gateway
internet_gateway = aws.ec2.InternetGateway("deepSeekInternetGateway", vpc_id=vpc.id)

# Create a route table and route table association
route_table = aws.ec2.RouteTable(
 "deepSeekRouteTable",
 vpc_id=vpc.id,
 routes=[
 aws.ec2.RouteTableRouteArgs(
 cidr_block="0.0.0.0/0", gateway_id=internet_gateway.id
 )
 ],
)

route_table_association = aws.ec2.RouteTableAssociation(
 "deepSeekRouteTableAssociation", subnet_id=subnet.id, route_table_id=route_table.id
)

# Create a security group
security_group = aws.ec2.SecurityGroup(
 "deepSeekSecurityGroup",
 vpc_id=vpc.id,
 egress=[
 {
 "from_port": 0,
 "to_port": 0,
 "protocol": "-1",
 "cidr_blocks": ["0.0.0.0/0"],
 }
 ],
 ingress=[
 {
 "from_port": 22,
 "to_port": 22,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 {
 "from_port": 3000,
 "to_port": 3000,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 {
 "from_port": 11434,
 "to_port": 11434,
 "protocol": "tcp",
 "cidr_blocks": ["0.0.0.0/0"],
 },
 ],
)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // omitted for brevity
 }

 // Create a VPC
 vpc, err := ec2.NewVpc(ctx, "deepSeekVpc", &ec2.VpcArgs{
 CidrBlock: pulumi.String("10.0.0.0/16"),
 EnableDnsHostnames: pulumi.Bool(true),
 EnableDnsSupport: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }

 // Create a subnet
 subnet, err := ec2.NewSubnet(ctx, "deepSeekSubnet", &ec2.SubnetArgs{
 VpcId: vpc.ID(),
 CidrBlock: pulumi.String("10.0.48.0/20"),
 AvailabilityZone: pulumi.String("eu-central-1a"),
 MapPublicIpOnLaunch: pulumi.Bool(true),
 })
 if err != nil {
 return err
 }

 // Create an internet gateway
 internetGateway, err := ec2.NewInternetGateway(ctx, "deepSeekInternetGateway", &ec2.InternetGatewayArgs{
 VpcId: vpc.ID(),
 })
 if err != nil {
 return err
 }

 // Create a route table and route table association
 routeTable, err := ec2.NewRouteTable(ctx, "deepSeekRouteTable", &ec2.RouteTableArgs{
 VpcId: vpc.ID(),
 Routes: ec2.RouteTableRouteArray{
 &ec2.RouteTableRouteArgs{
 CidrBlock: pulumi.String("0.0.0.0/0"),
 GatewayId: internetGateway.ID(),
 },
 },
 })
 if err != nil {
 return err
 }

 _, err = ec2.NewRouteTableAssociation(ctx, "deepSeekRouteTableAssociation", &ec2.RouteTableAssociationArgs{
 SubnetId: subnet.ID(),
 RouteTableId: routeTable.ID(),
 })
 if err != nil {
 return err
 }

 // Create a security group
 securityGroup, err := ec2.NewSecurityGroup(ctx, "deepSeekSecurityGroup", &ec2.SecurityGroupArgs{
 VpcId: vpc.ID(),
 Egress: ec2.SecurityGroupEgressArray{
 &ec2.SecurityGroupEgressArgs{
 FromPort: pulumi.Int(0),
 ToPort: pulumi.Int(0),
 Protocol: pulumi.String("-1"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 },
 Ingress: ec2.SecurityGroupIngressArray{
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(22),
 ToPort: pulumi.Int(22),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(3000),
 ToPort: pulumi.Int(3000),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 &ec2.SecurityGroupIngressArgs{
 FromPort: pulumi.Int(11434),
 ToPort: pulumi.Int(11434),
 Protocol: pulumi.String("tcp"),
 CidrBlocks: pulumi.StringArray{pulumi.String("0.0.0.0/0")},
 },
 },
 })
 if err != nil {
 return err
 }

 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 // omitted for brevity
 {
 VpcId = vpc.Id,
 Routes =
 {
 new RouteTableRouteArgs
 {
 CidrBlock = "0.0.0.0/0",
 GatewayId = internetGateway.Id
 }
 }
 });

 var routeTableAssociation = new RouteTableAssociation("deepSeekRouteTableAssociation", new RouteTableAssociationArgs
 {
 SubnetId = subnet.Id,
 RouteTableId = routeTable.Id
 });

 var securityGroup = new SecurityGroup("deepSeekSecurityGroup", new SecurityGroupArgs
 {
 VpcId = vpc.Id,
 Egress =
 {
 new SecurityGroupEgressArgs
 {
 FromPort = 0,
 ToPort = 0,
 Protocol = "-1",
 CidrBlocks = { "0.0.0.0/0" }
 }
 },
 Ingress =
 {
 new SecurityGroupIngressArgs
 {
 FromPort = 22,
 ToPort = 22,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 },
 new SecurityGroupIngressArgs
 {
 FromPort = 3000,
 ToPort = 3000,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 },
 new SecurityGroupIngressArgs
 {
 FromPort = 11434,
 ToPort = 11434,
 Protocol = "tcp",
 CidrBlocks = { "0.0.0.0/0" }
 }
 }
 });

 var publicKey = File.ReadAllText("deepseek.rsa");
 var keyPair = new KeyPair("deepSeekKey", new KeyPairArgs
 {
 PublicKey = publicKey
 });

 var amazonLinux = Pulumi.Aws.Ec2.GetAmi.Invoke(new()
 {
 MostRecent = true,
 Filters = new[]
 {
 new GetAmiFilterInputArgs
 {
 Name = "name",
 Values = new[] { "amzn2-ami-hvm-*-x86_64-gp2" }
 },
 new GetAmiFilterInputArgs
 {
 Name = "architecture",


 deepSeekVpc:
 type: aws:ec2:Vpc
 properties:
 cidrBlock: 10.0.0.0/16
 enableDnsHostnames: true
 enableDnsSupport: true

 deepSeekSubnet:
 type: aws:ec2:Subnet
 properties:
 vpcId: ${deepSeekVpc.id}
 cidrBlock: 10.0.48.0/20
 availabilityZone: eu-central-1a
 mapPublicIpOnLaunch: true

 deepSeekInternetGateway:
 type: aws:ec2:InternetGateway
 properties:
 vpcId: ${deepSeekVpc.id}

 deepSeekRouteTable:
 type: aws:ec2:RouteTable
 properties:
 vpcId: ${deepSeekVpc.id}
 routes:
 - cidrBlock: 0.0.0.0/0
 gatewayId: ${deepSeekInternetGateway.id}

 deepSeekRouteTableAssociation:
 type: aws:ec2:RouteTableAssociation
 properties:
 subnetId: ${deepSeekSubnet.id}
 routeTableId: ${deepSeekRouteTable.id}

 deepSeekSecurityGroup:
 type: aws:ec2:SecurityGroup
 properties:
 vpcId: ${deepSeekVpc.id}
 ingress:
 - fromPort: 22
 toPort: 22
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 - fromPort: 3000
 toPort: 3000
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 - fromPort: 11434
 toPort: 11434
 protocol: tcp
 cidrBlocks:
 - 0.0.0.0/0
 egress:
 - fromPort: 0
 toPort: 0
 protocol: -1
 cidrBlocks:
 - 0.0.0.0/0

Step 3: Launch the GPU EC2 instance

Now create the EC2 instance itself. The example uses Amazon Linux because the NVIDIA driver install path is well-trodden, plus an SSH key pair you generate locally.

The default instance type is g4dn.xlarge—the cheapest option that fits any 7B/8B model. Bump it up if you picked a larger model from the table above: g5.xlarge for 13B–14B, g5.2xlarge for 32B, g6e.2xlarge for 70B. AWS publishes full specs for the G4, G5, and G6 families.

Generate the key pair if you don’t already have one:

openssl genrsa -out deepseek.pem 2048
openssl rsa -in deepseek.pem -pubout > deepseek.pub
ssh-keygen -f mykey.pub -i -mPKCS8 > deepseek.pem

const keyPair = new aws.ec2.KeyPair("deepSeekKey", {
 publicKey: pulumi.output(fs.readFileSync("deepseek.rsa", "utf-8")),
});

const deepSeekAmi = aws.ec2
 .getAmi({
 filters: [
 {
 name: "name",
 values: ["amzn2-ami-hvm-2.0.*-x86_64-gp2"],
 },
 {
 name: "architecture",
 values: ["x86_64"],
 },
 ],
 owners: ["137112412989"], // Amazon
 mostRecent: true,
 })
 .then(ami => ami.id);

const deepSeekInstance = new aws.ec2.Instance("deepSeekInstance", {
 ami: deepSeekAmi,
 instanceType: "g4dn.xlarge",
 keyName: keyPair.keyName,
 rootBlockDevice: {
 volumeSize: 100,
 volumeType: "gp3",
 },
 subnetId: subnet.id,
 vpcSecurityGroupIds: [securityGroup.id],
 iamInstanceProfile: instanceProfile.name,
 userData: fs.readFileSync("cloud-init.yaml", "utf-8"),
 tags: {
 Name: "deepSeek-server",
 },
});

export const amiId = deepSeekAmi;
export const instanceId = deepSeekInstance.id;
export const instancePublicDns = deepSeekInstance.publicIp;


# Key pair for SSH access
public_key = open("deepseek.rsa", "r").read()
key_pair = aws.ec2.KeyPair("deepSeekKey", public_key=public_key)


# Get the latest Amazon Linux 2 AMI
ami = aws.ec2.get_ami(
 filters=[
 {"name": "name", "values": ["amzn2-ami-hvm-2.0.*-x86_64-gp2"]},
 {"name": "architecture", "values": ["x86_64"]},
 ],
 owners=["137112412989"], # Amazon
 most_recent=True,
).id

# Create an EC2 instance
user_data = open("cloud-init.yaml", "r").read()
instance = aws.ec2.Instance(
 "deepSeekInstance",
 ami=ami,
 instance_type="g4dn.xlarge",
 key_name=key_pair.key_name,
 root_block_device=aws.ec2.InstanceRootBlockDeviceArgs(
 volume_size=100, volume_type="gp3"
 ),
 subnet_id=subnet.id,
 vpc_security_group_ids=[security_group.id],
 iam_instance_profile=instance_profile.name,
 user_data=user_data,
 tags={"Name": "deepSeek-server"},
)

pulumi.export("amiId", ami)
pulumi.export("instanceId", instance.id)
pulumi.export("instancePublicDns", instance.public_ip)

package main

import (
 "encoding/json"
 "os"

 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/ec2"
 "github.com/pulumi/pulumi-aws/sdk/v6/go/aws/iam"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // omitted for brevity

 // Key pair for SSH access
 publicKey, err := os.ReadFile("deepseek.rsa")
 if err != nil {
 return err
 }

 keyPair, err := ec2.NewKeyPair(ctx, "deepSeekKey", &ec2.KeyPairArgs{
 PublicKey: pulumi.String(string(publicKey)),
 })
 if err != nil {
 return err
 }

 // Get the latest Amazon Linux 2 AMI
 mostRecent := true
 ami, err := ec2.LookupAmi(ctx, &ec2.LookupAmiArgs{
 Filters: []ec2.GetAmiFilter{
 {
 Name: "name",
 Values: []string{"amzn2-ami-hvm-2.0.*-x86_64-gp2"},
 },
 {
 Name: "architecture",
 Values: []string{"x86_64"},
 },
 },
 Owners: []string{"137112412989"},
 MostRecent: &mostRecent,
 })
 if err != nil {
 return err
 }

 // Create an EC2 instance
 userData, err := os.ReadFile("cloud-init.yaml")
 if err != nil {
 return err
 }

 instance, err := ec2.NewInstance(ctx, "deepSeekInstance", &ec2.InstanceArgs{
 Ami: pulumi.String(ami.Id),
 InstanceType: pulumi.String("g4dn.xlarge"),
 KeyName: keyPair.KeyName,
 RootBlockDevice: &ec2.InstanceRootBlockDeviceArgs{
 VolumeSize: pulumi.Int(100),
 VolumeType: pulumi.String("gp3"),
 },
 SubnetId: subnet.ID(),
 VpcSecurityGroupIds: pulumi.StringArray{securityGroup.ID()},
 IamInstanceProfile: instanceProfile.Name,
 UserData: pulumi.String(string(userData)),
 Tags: pulumi.StringMap{
 "Name": pulumi.String("deepSeek-server"),
 },
 })
 if err != nil {
 return err
 }

 ctx.Export("amiId", pulumi.String(ami.Id))
 ctx.Export("instanceId", instance.ID())
 ctx.Export("instancePublicDns", instance.PublicIp)

 return nil
 })
}

using Pulumi;
using Pulumi.Aws.Ec2;
using Pulumi.Aws.Iam;
using System.Collections.Generic;
using System.IO;
using Pulumi.Aws.Ec2.Inputs;
using System.Text.Json;

return await Deployment.RunAsync(() =>
{
 var rolePolicy = JsonSerializer.Serialize(new Dictionary<string, object>
 {
 // omitted for brevity
 Name = "architecture",
 Values = new[] { "x86_64" }
 }
 },
 Owners = new[] { "137112412989" }
 });

 var userData = File.ReadAllText("cloud-init.yaml");
 var instance = new Instance("deepSeekInstance", new InstanceArgs
 {
 Ami = amazonLinux.Apply(r => r.Id),
 InstanceType = "g4dn.xlarge",
 KeyName = keyPair.KeyName,
 RootBlockDevice = new InstanceRootBlockDeviceArgs
 {
 VolumeSize = 100,
 VolumeType = "gp3"
 },
 SubnetId = subnet.Id,
 VpcSecurityGroupIds = { securityGroup.Id },
 IamInstanceProfile = instanceProfile.Name,
 UserData = userData,
 Tags = { { "Name", "deepSeek-server" } }
 });

 return new Dictionary<string, object?>
 {
 ["amiId"] = amazonLinux.Apply(r => r.Id),
 ["instanceId"] = instance.Id,
 ["instancePublicDns"] = instance.PublicIp
 };
});


 deepSeekKey:
 type: aws:ec2:KeyPair
 properties:
 publicKey: ${publicKey}

 deepSeekInstance:
 type: aws:ec2:Instance
 properties:
 ami: ${amiId}
 instanceType: "g4dn.xlarge"
 keyName: ${deepSeekKey.keyName}
 rootBlockDevice:
 volumeSize: 100
 volumeType: gp3
 subnetId: ${deepSeekSubnet.id}
 vpcSecurityGroupIds:
 - ${deepSeekSecurityGroup.id}
 iamInstanceProfile: ${deepSeekProfile.name}
 userData: ${userData}
 tags:
 Name: deepSeek-server

outputs:
 AmiId: ${amiId}
 InstanceId: ${deepSeekInstance.id}
 InstancePublicDns: ${deepSeekInstance.publicIp}

Step 4: Install Ollama via cloud-init

The EC2 instance is a blank box until cloud-init runs. The user-data script below installs the NVIDIA GRID drivers, Docker, and the NVIDIA Container Toolkit, then starts the Ollama and Open WebUI containers. To switch models, edit the ollama run line—the rest is identical regardless of which model you want.

#cloud-config
users:
- default

package_update: true

packages:
- apt-transport-https
- ca-certificates
- curl
- openjdk-17-jre-headless
- gcc

runcmd:
- yum install -y gcc kernel-devel-$(uname -r)
- aws s3 cp --recursive s3://ec2-linux-nvidia-drivers/latest/ .
- chmod +x NVIDIA-Linux-x86_64*.run
- /bin/sh ./NVIDIA-Linux-x86_64*.run --tmpdir . --silent
- touch /etc/modprobe.d/nvidia.conf
- echo "options nvidia NVreg_EnableGpuFirmware=0" | sudo tee --append /etc/modprobe.d/nvidia.conf
- yum install -y docker
- usermod -a -G docker ec2-user
- systemctl enable docker.service
- systemctl start docker.service
- curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
- yum install -y nvidia-container-toolkit
- nvidia-ctk runtime configure --runtime=docker
- systemctl restart docker
- docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama --restart always ollama/ollama
- sleep 120
- docker exec ollama ollama run deepseek-r1:7b
- docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

You might also like:

Step 5: Deploy the infrastructure

Make sure your AWS credentials are configured:

aws configure

Pulumi supports several other authentication methods for the AWS provider. Once credentials are in place, deploy the infrastructure:

pulumi up

Pulumi previews the changes; type yes to confirm.

pulumi up
Please choose a stack, or create a new one: <create a new stack>
Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`): dev
Please enter your desired stack name.
Created stack 'dev'
Previewing update (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/dirien/deepseek-ollama-typescript/dev/previews/1dbb18ea-ba31-4d5b-9510-5dce19eb8ee8
Type Name Plan
+ pulumi:pulumi:Stack deepseek-ollama-typescript-dev create
+ ├─ aws:ec2:KeyPair deepSeekKey create
+ ├─ aws:ec2:Vpc deepSeekVpc create
+ ├─ aws:iam:Role deepSeekRole create
+ ├─ aws:iam:InstanceProfile deepSeekProfile create
+ ├─ aws:ec2:SecurityGroup deepSeekSecurityGroup create
+ ├─ aws:ec2:RouteTable deepSeekRouteTable create
+ ├─ aws:ec2:InternetGateway deepSeekInternetGateway create
+ ├─ aws:ec2:Subnet deepSeekSubnet create
+ ├─ aws:iam:RolePolicyAttachment deepSeekS3Policy create
+ ├─ aws:ec2:RouteTableAssociation deepSeekRouteTableAssociation create
+ └─ aws:ec2:Instance deepSeekInstance create
Outputs:
amiId : "ami-085131ff43045c877"
instanceId : output<string>
instancePublicDns: output<string>
Resources:
+ 12 to create
Do you want to perform this update? yes
Updating (dev)
View in Browser (Ctrl+O): https://app.pulumi.com/dirien/deepseek-ollama-typescript/dev/updates/1
Type Name Status
+ pulumi:pulumi:Stack deepseek-ollama-typescript-dev created (40s)
+ ├─ aws:ec2:KeyPair deepSeekKey created (0.47s)
+ ├─ aws:iam:Role deepSeekRole created (1s)
+ ├─ aws:ec2:Vpc deepSeekVpc created (12s)
+ ├─ aws:iam:InstanceProfile deepSeekProfile created (6s)
+ ├─ aws:iam:RolePolicyAttachment deepSeekS3Policy created (0.90s)
+ ├─ aws:ec2:InternetGateway deepSeekInternetGateway created (0.69s)
+ ├─ aws:ec2:Subnet deepSeekSubnet created (11s)
+ ├─ aws:ec2:SecurityGroup deepSeekSecurityGroup created (2s)
+ ├─ aws:ec2:RouteTable deepSeekRouteTable created (1s)
+ ├─ aws:ec2:RouteTableAssociation deepSeekRouteTableAssociation created (0.92s)
+ └─ aws:ec2:Instance deepSeekInstance created (12s)
Outputs:
amiId : "ami-085131ff43045c877"
instanceId : "i-0ae7495781ace3e81"
instancePublicDns: "18.159.211.136"
Resources:
+ 12 created
Duration: 42s

The infrastructure provisions in under a minute, but the cloud-init script needs another 5–10 minutes to install drivers, pull container images, and download the model weights. SSH in to watch the progress, or just wait and load the Web UI when it’s ready.

ssh -i deepseek.pem ec2-user@<instance-public-ip>

Check the container status:

sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c8714335e205 ghcr.io/open-webui/open-webui:main "bash start.sh" 6 minutes ago Up 6 minutes (healthy) 0.0.0.0:3000->8080/tcp, :::3000->8080/tcp open-webui
bf4bb3b7ede1 ollama/ollama "/bin/ollama serve" 8 minutes ago Up 7 minutes 0.0.0.0:11434->11434/tcp, :::11434->11434/tcp ollama
[ec2-user@ip-10-0-58-122 ~]$

Step 6: Access the Web UI or API

Once the containers are healthy, open http://<instance-public-ip>:3000 in your browser for Open WebUI:

Open WebUI is not secured by default. Restrict the security group to your IP, put it behind an authenticated proxy, or terminate TLS at an ALB before exposing it to the internet.

For programmatic access, Ollama exposes an OpenAI-compatible API on port 11434. Most clients written for the OpenAI SDK only need a base-URL change:

from openai import OpenAI

client = OpenAI(
 base_url="http://<instance-public-ip>:11434/v1",
 api_key="ollama", # required by the SDK, ignored by Ollama
)

response = client.chat.completions.create(
 model="llama3.1:8b",
 messages=[{"role": "user", "content": "Why is the sky blue?"}],
)
print(response.choices[0].message.content)

How do I switch models?

Ollama hosts every major open-weight family in its model library. Pulling a different model is two commands inside the EC2 instance—or a one-line edit to the cloud-init script if you want it provisioned automatically:

# DeepSeek-R1 distill (default in this guide)
docker exec ollama ollama run deepseek-r1:7b

# Llama 3.1 (Meta, 8B)
docker exec ollama ollama run llama3.1:8b

# Qwen 2.5 (Alibaba, 7B)
docker exec ollama ollama run qwen2.5:7b

# Mistral (7B)
docker exec ollama ollama run mistral:7b

# Larger reasoning model (needs g5.2xlarge or larger)
docker exec ollama ollama run deepseek-r1:32b

Tags follow a <size> or <size>-<quantization> pattern—8b, 8b-instruct-q4_K_M, 8b-instruct-q8_0. Q4 is the default and the right starting point; bump to Q8 only if you have spare VRAM and notice quality issues with Q4. Browse the full tag list for any model on its Ollama library page.

Cleaning up

When you’re done, tear everything down:

pulumi destroy

What are the next steps?

You now have a reproducible, IaC-managed deployment of any open-source LLM on AWS. The infrastructure is fixed; the model is a parameter. From here, the natural extensions are wiring this up to a real application, adding RAG over your own data, or moving the deployment behind an authenticated load balancer.

If you want to go further with AI on Pulumi, here are some related guides:

Deploy LangServe Apps with Pulumi on AWS (RAG & Chatbot) — Build a retrieval-augmented chatbot that could front-end this Ollama instance.
Deploy AI Models on Amazon SageMaker using Pulumi Python IaC — A SageMaker alternative when you’d rather not manage the EC2 host yourself.
Build an AI Slack Bot on AWS Using Embedchain & Pulumi — Wire an LLM into Slack as an internal assistant.
What is Infrastructure as Code? — Background on the IaC approach used throughout this guide.

Try Pulumi for Free

Changelog

2026-04-30 — Broadened scope from DeepSeek-only to any Ollama-supported model (Llama, Qwen, Mistral). Added TL;DR, instance-type recommendation table, cost-vs-hosted-API comparison, and HowTo structured data. Restructured headings as user questions. Verified Ollama and cloud-init commands against current versions.
2025-03-10 — Minor edits and corrections.
2025-01-27 — Original post: Run DeepSeek-R1 on AWS EC2 Using Ollama.

Master Kubernetes Secrets with Pulumi ESC + Secrets Store CSI Driver

Engin Diri — Wed, 18 Dec 2024 09:58:01 +0100

Welcome to the second blog post of the Pulumi ESC and Kubernetes secrets series. If you haven’t had the chance to read the first blog post, go ahead and read it here.

In the previous blog post, we have learned how to manage secrets with Pulumi ESC and the External Secrets Operator. While the External Secrets Operator is a great tool to manage secrets in a cloud-native way, it still creates Kubernetes secrets in the cluster. Depending on your security requirements, you might want to avoid the use of Kubernetes secrets in your cluster at all. This is the point where you hit the limits of the External Secrets Operator.

But don’t worry, we also have a solution for this problem. In this blog post, we will introduce the Secrets Store CSI Driver.

TL;DR?

Jump straight to the comparison of the External Secrets Operator and the Secrets Store CSI Driver here.

Refresher: Why Should You Avoid Using Kubernetes Secrets?

Before we dive into the Secrets Store CSI Driver, let’s quickly recap why you might want to avoid using native Kubernetes secrets. I wrote in depth about this topic in the previous blog post, but here is a quick summary:

By default, etcd stores them in a base64-encoded format, which is not secure.
Developers create them either manually by using kubectl commands or by using a manifest file, which makes them hard to manage at scale.
Hard to manage and synchronise secrets across different environments and clusters.
There is no default way to rotate secrets automatically.

Point 1 is the most critical one, as there is even a warning in the Kubernetes documentation:

Kubernetes Secrets. Credit: kubernetes.io/docs/concepts/configuration/secret/

What is the Secrets Store CSI Driver?

The first thing you may notice is that in the name of the Secret Store CSI Driver, there is a reference to the Container Storage Interface (CSI) standard.

What is CSI and why it is used in the Secret Store CSI Driver*?*

How to write a Container Storage Interface (CSI) plugin. Credit: arslan.io/2018/06/21/how-to-write-a-container-storage-interface-csi-plugin/

Container Storage Interface (CSI)

The Container Storage Interface (CSI) is a standard to unify the interface between container orchestrators (like Kubernetes) and different storage vendors (like NetApp, Ceph, etc.). This helps to guarantee that implementing a CSI for a storage vendor is going to work with all orchestrators that support CSI.

Before CSI, we needed to create volume plugins for every orchestrator and storage vendor. This meant that developers coupled volume plugin development with the Kubernetes version and made it dependent on it. Bugs in volume plugins could break the Kubernetes components, instead of the volume plugin. Besides, volume plugins had full privileges on the Kubernetes components like the kubelet.

Combining CSI, Kubernetes and Secrets = Secret Store CSI Driver

The Secret Store CSI Driver is a CSI driver that allows you to mount many secrets, certificates, and keys from external secret stores into Kubernetes pods as volumes. After attaching the volume, the system mounts the secrets into the container file system.

The benefits of using the Secret Store CSI Driver are that you manage the lifecycle of the secrets outside of Kubernetes while still providing a Kubernetes-native experience of using the secrets in your pods.

The Architecture of the Secret Store CSI Driver

The Secret Store CSI Driver is a daemonset that lets all kublets communicate with each other and uses gRPC to talk to a provider. A SecretProviderClass custom resource specifies the definition of the external Secret Store. Then the system mounts a volume in the pod as tmpfs, and it injects the secret into the volume. When you delete the pod, the system cleans up the volume and removes the secret from the tmpfs volume.

How to Use the Secrets Store CSI Driver with Pulumi ESC

Before we start, make sure you have the following prerequisites:

A Pulumi Cloud account. If you don’t have one, you can create one for free.
A Kubernetes cluster (I will be using a local KinD cluster, but you can use any Kubernetes cluster)
Pulumi CLI installed. You can use the pulumi env commands to manage secrets and configurations.
kubectl CLI installed for some debugging

Step 1: Deploy the Secrets Store CSI Driver and Pulumi ESC CSI Provider

We will use Pulumi with the pulumi-kubernetes provider to deploy the Secrets Store CSI Driver and the Pulumi ESC CSI Provider to our Kubernetes cluster.

And here we also use the integration of Pulumi ESC with Pulumi IaC to supply the Pulumi access token in a secure way.

First, we need to create a new Pulumi project:

# Choose your favorite Pulumi supported language
pulumi new kubernetes-<your-programming-language> --name

Before we dig into the code, let’s head to the Pulumi Cloud Console and create a new Pulumi ESC project with the name esc-secrets-store-csi-driver-demo.

And create the environment dev:

In the editor add the following yaml into the Environment definition:

values:
 pulumiConfig:
 pulumi-pat:
 fn::secret: <your-pulumi-pat>

Don’t know how to create a PAT? Check out the official Pulumi documentation.

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/esc-secrets-store-csi-driver-demo/dev

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/esc-secrets-store-csi-driver-demo/dev

Now, we need to link the Pulumi ESC project to the Pulumi IaC project. To do this, we need to add the following to your Pulumi.dev.yaml:

environment:
- esc-secrets-store-csi-driver-demo/dev

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

const secretsStoreCSIDriver = new k8s.helm.v4.Chart("secrets-store-csi-driver", {
 chart: "secrets-store-csi-driver",
 namespace: "kube-system",
 repositoryOpts: {
 repo: "https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts",
 },
 values: {
 nodeSelector: {
 "kubernetes.io/os": "linux",
 },
 },
});

const secretsStoreCSIPulumiESCProvider = new k8s.helm.v4.Chart(
 "secrets-store-csi-pulumi-esc-provider",
 {
 chart: "oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider",
 namespace: "kube-system",
 values: {
 nodeSelector: {
 "kubernetes.io/os": "linux",
 },
 },
 },
 { dependsOn: secretsStoreCSIDriver },
);

const config = new pulumi.Config();

const mySecret = new k8s.core.v1.Secret(
 "my-secret",
 {
 metadata: {
 namespace: "default",
 name: "pulumi-access-token",
 },
 stringData: {
 "pulumi-access-token": config.require("pulumi-pat"),
 },
 type: "Opaque",
 },
 { dependsOn: secretsStoreCSIPulumiESCProvider },
);

import pulumi
import pulumi_kubernetes as k8s

secrets_store_csi_driver = k8s.helm.v4.Chart(
 "secrets-store-csi-driver",
 k8s.helm.v4.ChartArgs(
 chart="secrets-store-csi-driver",
 namespace="kube-system",
 repository_opts=k8s.helm.v4.RepositoryOptsArgs(
 repo="https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts",
 ),
 values={
 "nodeSelector": {
 "kubernetes.io/os": "linux",
 },
 },
 ),
)

secrets_store_csi_pulumi_esc_provider = k8s.helm.v4.Chart(
 "secrets-store-csi-pulumi-esc-provider",
 k8s.helm.v4.ChartArgs(
 chart="oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider",
 namespace="kube-system",
 values={
 "nodeSelector": {
 "kubernetes.io/os": "linux",
 },
 },
 ),
 opts=pulumi.ResourceOptions(depends_on=[secrets_store_csi_driver]),
)

config = pulumi.Config()

my_secret = k8s.core.v1.Secret(
 "my-secret",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 namespace="default", name="pulumi-access-token"
 ),
 string_data={
 "pulumi-access-token": config.require("pulumi-pat"),
 },
 type="Opaque",
 opts=pulumi.ResourceOptions(depends_on=[secrets_store_csi_pulumi_esc_provider]),
)

package main

import (
 k8s "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes"
 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apiextensions"
 appv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apps/v1"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 chartv4 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v4"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi/config"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 secretsStoreCsiDriver, err := chartv4.NewChart(ctx, "secrets-store-csi-driver", &chartv4.ChartArgs{
 Chart: pulumi.String("secrets-store-csi-driver"),
 Namespace: pulumi.String("kube-system"),
 RepositoryOpts: chartv4.RepositoryOptsArgs{
 Repo: pulumi.String("https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts"),
 },
 Values: pulumi.Map{
 "nodeSelector": pulumi.Map{
 "kubernetes.io/os": pulumi.String("linux"),
 },
 },
 })
 if err != nil {
 return err
 }

 secretsStoreCsiPulumiEscProvider, err := chartv4.NewChart(ctx, "secrets-store-csi-pulumi-esc-provider", &chartv4.ChartArgs{
 Chart: pulumi.String("oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider"),
 Namespace: pulumi.String("kube-system"),
 Values: pulumi.Map{
 "nodeSelector": pulumi.Map{
 "kubernetes.io/os": pulumi.String("linux"),
 },
 },
 }, pulumi.DependsOn([]pulumi.Resource{secretsStoreCsiDriver}))
 if err != nil {
 return err
 }

 pulumiPAT := config.Get(ctx, "pulumi-pat")

 mySecret, err := corev1.NewSecret(ctx, "my-secret", &corev1.SecretArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: pulumi.String("default"),
 Name: pulumi.String("pulumi-access-token"),
 },
 StringData: pulumi.StringMap{
 "pulumi-access-token": pulumi.String(pulumiPAT),
 },
 Type: pulumi.String("Opaque"),
 }, pulumi.DependsOn([]pulumi.Resource{secretsStoreCsiPulumiEscProvider}))
 if err != nil {
 return err
 }


 return nil
 })
}

using Pulumi;
using Pulumi.Kubernetes.Core.V1;
using Pulumi.Kubernetes.Types.Inputs.Core.V1;
using Pulumi.Kubernetes.Helm.V3;
using Pulumi.Kubernetes.Helm;
using Pulumi.Kubernetes.Types.Inputs.Meta.V1;
using Pulumi.Kubernetes.ApiExtensions;
using System.Collections.Generic;



return await Deployment.RunAsync(() =>
{
 var secretsStoreCsiDriver = new Release("secrets-store-csi-driver", new()
 {
 Chart = "secrets-store-csi-driver",
 Namespace = "kube-system",
 RepositoryOpts = new Pulumi.Kubernetes.Types.Inputs.Helm.V3.RepositoryOptsArgs
 {
 Repo = "https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts",
 },
 Values = new Dictionary<string, object>
 {
 { "nodeSelector", new Dictionary<string, object>
 {
 { "kubernetes.io/os", "linux" },
 } },
 },
 });

 var secretsStoreCsiPulumiEscProvider = new Release("secrets-store-csi-pulumi-esc-provider", new()
 {
 Chart = "oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider",
 Namespace = "kube-system",
 Values = new Dictionary<string, object>
 {
 { "nodeSelector", new Dictionary<string, object>
 {
 { "kubernetes.io/os", "linux" },
 } },
 },
 }, new CustomResourceOptions { DependsOn = { secretsStoreCsiDriver } });

 var config = new Config();
 var pulumiPAT = config.Require("pulumi-pat");

 var mySecret = new Secret("my-secret", new SecretArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Namespace = "default",
 Name = "pulumi-access-token",
 },
 StringData =
 {
 { "pulumi-access-token", pulumiPAT }
 },
 Type = "Opaque",
 }, new CustomResourceOptions { DependsOn = { secretsStoreCsiPulumiEscProvider } });

});


class SecretProviderClassArgs : CustomResourceArgs
{
 public SecretProviderClassArgs(): base("secrets-store.csi.x-k8s.io/v1", "SecretProviderClass")
 {
 }

 [Input("spec")]
 public Input<SecretProviderClassSpecArgs>? Spec { get; set; }
}


class SecretProviderClassSpecArgs : ResourceArgs
{
 [Input("provider")]
 public Input<string>? Provider { get; set; }

 [Input("parameters")]
 public Input<InputMap<object>>? Parameters { get; set; }
}

class SecretProviderParametersArgs : ResourceArgs
{
 [Input("apiUrl")]
 public Input<string>? ApiUrl { get; set; }

 [Input("organization")]
 public Input<string>? Organization { get; set; }

 [Input("project")]
 public Input<string>? Project { get; set; }

 [Input("environment")]
 public Input<string>? Environment { get; set; }

 [Input("authSecretName")]
 public Input<string>? AuthSecretName { get; set; }

 [Input("authSecretNamespace")]
 public Input<string>? AuthSecretNamespace { get; set; }

 [Input("secrets")]
 public Input<string>? Secrets { get; set; }
}

Deploy the stack by running:

pulumi up

And you should see that the secret was created in the Kubernetes cluster and the ESO instance was deployed successfully.

kubectl get secret pulumi-access-token -o jsonpath='{.data.PULUMI_ACCESS_TOKEN}' | base64 -d

Step 2: Create a SecretProviderClass

Now, we will create a secret in the Pulumi ESC project and synchronize it into the Kubernetes cluster by creating an SecretProviderClass:

Create a new ESC environment called csi-secrets-store-app in the esc-secrets-store-csi-driver-demo project:

values:
 app:
 hello: world
 hello-secret:
 fn::secret: world

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/esc-secrets-store-csi-driver-demo/csi-secrets-store-app

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/esc-secrets-store-csi-driver-demo/csi-secrets-store-app

Either way, you should see following environment configuration in the Pulumi Cloud Console:

We can now create the SecretProviderClass in the Kubernetes cluster:

const secretProviderClass = new k8s.apiextensions.CustomResource(
 "example-provider-pulumi-esc",
 {
 apiVersion: "secrets-store.csi.x-k8s.io/v1",
 kind: "SecretProviderClass",
 metadata: {
 name: "example-provider-pulumi-esc",
 namespace: "default",
 },
 spec: {
 provider: "pulumi",
 parameters: {
 apiUrl: "https://api.pulumi.com/api/esc",
 organization: "dirien",
 project: "esc-secrets-store-csi-driver-demo",
 environment: "csi-secrets-store-app",
 authSecretName: mySecret.metadata.name,
 authSecretNamespace: mySecret.metadata.namespace,
 secrets: `- secretPath: "/"
 fileName: "hello"
 secretKey: "app.hello"
`,
 },
 },
 },
 { dependsOn: secretsStoreCSIPulumiESCProvider },
);

secret_provider_class = k8s.apiextensions.CustomResource(
 "example-provider-pulumi-esc",
 api_version="secrets-store.csi.x-k8s.io/v1",
 kind="SecretProviderClass",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="example-provider-pulumi-esc", namespace="default"
 ),
 spec={
 "provider": "pulumi",
 "parameters": {
 "apiUrl": "https://api.pulumi.com/api/esc",
 "organization": "dirien",
 "project": "esc-secrets-store-csi-driver-demo",
 "environment": "csi-secrets-store-app",
 "authSecretName": my_secret.metadata["name"],
 "authSecretNamespace": my_secret.metadata["namespace"],
 "secrets": '- secretPath: "/"\n fileName: "hello"\n secretKey: "app.hello"\n',
 },
 },
 opts=pulumi.ResourceOptions(depends_on=[secrets_store_csi_pulumi_esc_provider]),
)

 secretProviderClass, err := apiextensions.NewCustomResource(ctx, "example-provider-pulumi-esc", &apiextensions.CustomResourceArgs{
 ApiVersion: pulumi.String("secrets-store.csi.x-k8s.io/v1"),
 Kind: pulumi.String("SecretProviderClass"),
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("example-provider-pulumi-esc"),
 Namespace: pulumi.String("default"),
 },
 OtherFields: k8s.UntypedArgs{
 "provider": pulumi.String("pulumi"),
 "parameters": pulumi.Map{
 "apiUrl": pulumi.String("https://api.pulumi.com/api/esc"),
 "organization": pulumi.String("dirien"),
 "project": pulumi.String("esc-secrets-store-csi-driver-demo"),
 "environment": pulumi.String("csi-secrets-store-app"),
 "authSecretName": mySecret.Metadata.Name().Elem(),
 "authSecretNamespace": mySecret.Metadata.Namespace().Elem(),
 "secrets": pulumi.String(`- secretPath: "/"
 fileName: "hello"
 secretKey: "app.hello"
`),
 },
 },
 }, pulumi.DependsOn([]pulumi.Resource{secretsStoreCsiPulumiEscProvider}))
 if err != nil {
 return err
 }

 var secretProviderClass = new Pulumi.Kubernetes.ApiExtensions.CustomResource("example-provider-pulumi-esc", new SecretProviderClassArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Name = "example-provider-pulumi-esc",
 Namespace = "default",
 },
 Spec = new SecretProviderClassSpecArgs
 {
 Provider = "pulumi",
 Parameters = new InputMap<object>
 {
 { "apiUrl", "https://api.pulumi.com/api/esc" },
 { "organization", "dirien" },
 { "project", "esc-secrets-store-csi-driver-demo" },
 { "environment", "csi-secrets-store-app" },
 { "authSecretName", mySecret.Metadata.Apply(metadata => metadata.Name) },
 { "authSecretNamespace", mySecret.Metadata.Apply(metadata => metadata.Namespace) },
 { "secrets", "- secretPath: \"/\"\n fileName: \"hello\"\n secretKey: \"app.hello\"\n" }
 },
 },
 }, new CustomResourceOptions { DependsOn = { secretsStoreCsiPulumiEscProvider } });

We can check that the secret was successfully synchronized by running:

kubectl get secretproviderclasses example-provider-pulumi-esc

You might also like:

Step 3: Deploy an Application and Mount the Secret

Now, we can deploy an application that references the secret from the Kubernetes cluster. I am going to use busybox that reads mounted file in.

const deployment = new k8s.apps.v1.Deployment("example-provider-pulumi-esc", {
 metadata: {
 name: "example-provider-pulumi-esc",
 namespace: "default",
 labels: {
 app: "example-provider-pulumi-esc",
 },
 },
 spec: {
 replicas: 1,
 selector: {
 matchLabels: {
 app: "example-provider-pulumi-esc",
 },
 },
 template: {
 metadata: {
 labels: {
 app: "example-provider-pulumi-esc",
 },
 },
 spec: {
 containers: [
 {
 name: "client",
 image: "busybox:latest",
 command: ["sh", "-c"],
 args: [
 `set -eux
 ls /run/secrets
 find /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo "$(cat "{}")"'
 tail -f /dev/null`,
 ],
 volumeMounts: [
 {
 name: "data",
 mountPath: "/run/secrets",
 },
 ],
 },
 ],
 volumes: [
 {
 name: "data",
 csi: {
 driver: "secrets-store.csi.k8s.io",
 readOnly: true,
 volumeAttributes: {
 secretProviderClass: secretProviderClass.metadata.name,
 },
 },
 },
 ],
 },
 },
 },
});

export const deploymentName = deployment.metadata.name;

deployment = k8s.apps.v1.Deployment(
 "example-provider-pulumi-esc",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 namespace="default",
 name="example-provider-pulumi-esc",
 labels={
 "app": "example-provider-pulumi-esc",
 },
 ),
 spec=k8s.apps.v1.DeploymentSpecArgs(
 replicas=1,
 selector=k8s.meta.v1.LabelSelectorArgs(
 match_labels={
 "app": "example-provider-pulumi-esc",
 },
 ),
 template=k8s.core.v1.PodTemplateSpecArgs(
 metadata=k8s.meta.v1.ObjectMetaArgs(
 labels={
 "app": "example-provider-pulumi-esc",
 },
 ),
 spec=k8s.core.v1.PodSpecArgs(
 containers=[
 k8s.core.v1.ContainerArgs(
 name="client",
 image="busybox:latest",
 command=["sh", "-c"],
 args=[
 "set -eux\nls /run/secrets\nfind /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo \"$(cat \"{}\")\"'\ntail -f /dev/null",
 ],
 volume_mounts=[
 k8s.core.v1.VolumeMountArgs(
 name="data",
 mount_path="/run/secrets",
 ),
 ],
 ),
 ],
 volumes=[
 k8s.core.v1.VolumeArgs(
 name="data",
 csi=k8s.core.v1.CSIVolumeSourceArgs(
 driver="secrets-store.csi.k8s.io",
 read_only=True,
 volume_attributes={
 "secretProviderClass": secret_provider_class.metadata[
 "name"
 ],
 },
 ),
 ),
 ],
 ),
 ),
 ),
)

pulumi.export("deploymentName", deployment.metadata["name"])

 deployment, err := appv1.NewDeployment(ctx, "example-provider-pulumi-esc", &appv1.DeploymentArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("example-provider-pulumi-esc"),
 Namespace: pulumi.String("default"),
 Labels: pulumi.StringMap{
 "app": pulumi.String("example-provider-pulumi-esc"),
 },
 },
 Spec: &appv1.DeploymentSpecArgs{
 Replicas: pulumi.Int(1),
 Selector: &metav1.LabelSelectorArgs{
 MatchLabels: pulumi.StringMap{
 "app": pulumi.String("example-provider-pulumi-esc"),
 },
 },
 Template: &corev1.PodTemplateSpecArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: pulumi.StringMap{
 "app": pulumi.String("example-provider-pulumi-esc"),
 },
 },
 Spec: &corev1.PodSpecArgs{
 Containers: corev1.ContainerArray{
 &corev1.ContainerArgs{
 Name: pulumi.String("client"),
 Image: pulumi.String("busybox:latest"),
 Command: pulumi.StringArray{
 pulumi.String("sh"),
 pulumi.String("-c"),
 },
 Args: pulumi.StringArray{
 pulumi.String(`set -eux
 ls /run/secrets
 find /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo "$(cat "{}")"'
 tail -f /dev/null`),
 },
 VolumeMounts: &corev1.VolumeMountArray{
 &corev1.VolumeMountArgs{
 Name: pulumi.String("data"),
 MountPath: pulumi.String("/run/secrets"),
 },
 },
 },
 },
 Volumes: &corev1.VolumeArray{
 &corev1.VolumeArgs{
 Name: pulumi.String("data"),
 Csi: corev1.CSIVolumeSourceArgs{
 Driver: pulumi.String("secrets-store.csi.k8s.io"),
 ReadOnly: pulumi.Bool(true),
 VolumeAttributes: pulumi.StringMap{
 "secretProviderClass": secretProviderClass.Metadata.Name().Elem(),
 },
 },
 },
 },
 },
 },
 },
 }, pulumi.DependsOn([]pulumi.Resource{secretProviderClass}))
 if err != nil {
 return err
 }

 var deployment = new Pulumi.Kubernetes.Apps.V1.Deployment("example-provider-pulumi-esc", new Pulumi.Kubernetes.Types.Inputs.Apps.V1.DeploymentArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Name = "example-provider-pulumi-esc",
 Namespace = "default",
 Labels =
 {
 { "app", "example-provider-pulumi-esc" },
 },
 },
 Spec = new Pulumi.Kubernetes.Types.Inputs.Apps.V1.DeploymentSpecArgs
 {
 Replicas = 1,
 Selector = new Pulumi.Kubernetes.Types.Inputs.Meta.V1.LabelSelectorArgs
 {
 MatchLabels =
 {
 { "app", "example-provider-pulumi-esc" },
 },
 },
 Template = new Pulumi.Kubernetes.Types.Inputs.Core.V1.PodTemplateSpecArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Labels =
 {
 { "app", "example-provider-pulumi-esc" },
 },
 },
 Spec = new Pulumi.Kubernetes.Types.Inputs.Core.V1.PodSpecArgs
 {
 Containers =
 {
 new Pulumi.Kubernetes.Types.Inputs.Core.V1.ContainerArgs
 {
 Name = "client",
 Image = "busybox:latest",
 Command =
 {
 "sh",
 "-c",
 },
 Args =
 {
 "set -eux\nls /run/secrets\nfind /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo \"$(cat \"{}\")\"'\ntail -f /dev/null",
 },
 VolumeMounts =
 {
 new Pulumi.Kubernetes.Types.Inputs.Core.V1.VolumeMountArgs
 {
 Name = "data",
 MountPath = "/run/secrets",
 },
 },
 },
 },
 Volumes =
 {
 new Pulumi.Kubernetes.Types.Inputs.Core.V1.VolumeArgs
 {
 Name = "data",
 Csi = new Pulumi.Kubernetes.Types.Inputs.Core.V1.CSIVolumeSourceArgs
 {
 Driver = "secrets-store.csi.k8s.io",
 ReadOnly = true,
 VolumeAttributes =
 {
 { "secretProviderClass", secretProviderClass.Metadata.Apply(metadata => metadata.Name) },
 },
 },
 },
 },
 },
 },
 },
 }, new CustomResourceOptions { DependsOn = { secretProviderClass } });

After deploying the stack, you can get the logs of the busybox pod to see that the secret was successfully mounted:

NAME=$(kubectl get pods -o name | grep example-provider-pulumi-esc | cut -d'/' -f2)
kubectl logs $NAME

You should see the following output:

+ ls /run/secrets
hello
+ find /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*'
+ xargs -t -I '{}' sh -c 'echo "$(cat "{}")"'
sh -c echo "$(cat "/run/secrets/hello")"
world
+ tail -f /dev/null

Or you can exec into the pod and check the content of the mounted secret:

kubectl exec -it $NAME -- cat /run/secrets/hello

You should see the following output:

world

Step 4: Clean Up

After you are done with the demo, you can clean up the resources by running:

pulumi destroy

Conclusion

The Secret Store CSI Driver is a great option for managing secrets in a cloud-native way when the use of Kubernetes secrets is not an option due to enhanced security requirements.

Below is a quick comparison of the External Secrets Operator and the Secrets Store CSI Driver to help you decide which one is the best fit for your use case. As always, the best way to find out is to try both solutions and see which one fits your requirements best.

The good part is, whatever you choose, you can use Pulumi ESC to manage your secrets and avoid any secret sprawl in your organization as both solutions are supported by Pulumi ESC.

Comparison of the External Secrets Operator and the Secrets Store CSI Driver

Feature/Aspect	External Secrets Operator	CSI Secrets Driver
Primary Use Case	Synchronizing external secrets into Kubernetes as native secrets.	Mounting secrets directly to pods as files or environment variables.
Integration with External Secret Stores	Supports multiple external secret stores like AWS Secrets Manager, Google Secret Manager, Vault, Azure Key Vault, etc.	Primarily supports integrations defined by the CSI specification (e.g., Vault, AWS, Azure Key Vault, Pulumi ESC).
Mechanism for Secrets Delivery	Secrets are pulled and stored as Kubernetes `Secret` resources.	Secrets are mounted directly into pods as files or injected as environment variables.
Kubernetes Resource Requirements	Requires a custom Kubernetes resource (`ExternalSecret`) and the External Secrets Operator.	Relies on the Kubernetes CSI (Container Storage Interface) standard and requires a specific CSI driver.
Secret Rotation	Supports automatic rotation by polling the external secret store at defined intervals.	Supports rotation, but the application needs to handle reloading mounted secrets.
Security Considerations	Secrets are stored temporarily in Kubernetes as native secrets, which may pose a risk if not properly managed.	Secrets are mounted directly and not stored as native Kubernetes secrets, reducing exposure but relying on file permissions.
Customization and Features	Highly customizable with fine-grained mappings and transformation options.	Limited customization, focused on mounting secrets as-is.
Application Compatibility	Applications access secrets as Kubernetes secrets (e.g., via env variables or volumes).	Applications access secrets directly via mounted files or injected env variables.
Performance Considerations	Potential slight delay due to syncing secrets to Kubernetes secrets.	Faster access as secrets are directly mounted.
Community and Ecosystem	Supported by a growing community; integrates well with diverse secret management solutions.	Part of the Kubernetes ecosystem; supports CSI-compliant secret storage solutions.
Best Use Cases	Ideal for scenarios where secrets need to be accessed as Kubernetes native secrets.	Best for applications requiring direct access to secrets without intermediate storage in Kubernetes secrets.

You can find detailed information about all current available Pulumi ESC Kubernetes integrations in the Pulumi ESC documentation.

Let us know what you think about the Secret Store CSI Driver and how you are managing secrets in your Kubernetes clusters. We would love to hear your feedback and experiences.

Try Pulumi for Free

How Secrets Sprawl is Slowing You Down—And What to Do About It.

Engin Diri — Mon, 28 Oct 2024 12:40:10 +0200

Only a few things are certain in the lives of developers and DevOps engineers: taxes, yearly performance reviews, and secret sprawl. While the first two are inevitable, the last one is something that can be managed.

As we keep adding new cloud resources and releasing new applications, the number of secrets we need to manage keeps growing: passwords, API keys, certificates, and more. And as if this isn’t enough, we need to manage secrets across different systems and environments with different teams that need to access them; we end up with duplicates.

Let’s have a look at a typical environment and see what teams use: Everything container-related may use Docker secrets or Kubernetes secrets. There is also a high chance that they might use cloud-specific secrets managers like Azure Key Vault, AWS Secrets Manager, or Google Secret Manager. In addition to these, there is a high chance that they might use HashiCorp Vault or some enterprise-grade secrets manager like CyberArk.

There are still a lot of organizations that store their secrets (like the API key for a cloud service) in environment variables to configure their applications on virtual machines, containers, or serverless functions. And as if that weren’t risky enough, some developers still hardcode sensitive information in their application code. Or use plain unencrypted text files such as .env files and share them inside the team. To be fair, this is not something that is done on purpose, and most teams realize the risks of these practices. But the problem is that they have not found a better way to manage their secrets that won’t slow them down and impact their productivity.

As the IT infrastructure grows, it becomes increasingly challenging for organizations to secure, organize, and distribute their secrets. This is where secrets sprawl comes into play. Secrets sprawl is the uncontrolled and unmanaged distribution of secrets across different systems and environments.

Modern architectures are adding to the problem of secret sprawl as modern applications are more distributed and consist of smaller building blocks with a growing number of different technologies.

While teams are ensuring that each of their applications has everything it needs to run, it’s also easy for them to lose track of the secrets they are using. Everything can be fine until they need to rotate a secret or revoke a compromised one. Then they need to go through all the systems and environments to find and update the secret. Tracking down all the places where a secret is used can be a time-consuming and error-prone process with a high risk of missing some places.

So what are some of the pain points of secret sprawl, and how can we overcome them to regain our productivity?

Pain Points of Secrets Sprawl

With secrets sprawled all across our teams (development and operations) and infrastructure, we face several points:

It is challenging to keep track of all the secrets and where they are used. When we need, for example, to find a specific secret, we need to go through different secret managers, code, and, in the worst case, the local files of one of the developers.

Do we need to rotate a secret? Now we need to go through all the systems and environments to find and update every duplicated secret.
Security risk is another pain point to mention. When a secret is spread throughout the organization, it’s hard to manage access control and enforce specific access policies.

Security best practices also recommend that secrets should be rotated regularly. But with secrets sprawled all over the place, it makes it a lot of work to rotate them. Time we could spend on more important tasks like developing new features.
Secrets sprawl makes it extremely challenging to distribute secrets. Having a secret, for example, in a cloud provider’s secret manager is cool and great, but what if we need to use it in a Kubernetes cluster? What is the process for that?

Also, having secrets in multiple places means there are different owners for them. These secrets are now some kind of gatekeeping by the team that owns them. This makes it hard for other teams to access them when they need them. And even if they get access, they will then be mostly shared in plain text as the other team does not have access to the secret manager itself.

As we can see, secret sprawl is a real problem that can slow down our teams and impact our productivity. But it also affects how fast an organization can scale its development and operations teams.

With an “organic” way of managing secrets and no real transparency, it’s hard to keep up with the pace of the business, and secret sprawl becomes a major bottleneck. Additionally, that growth means a whole bunch of new secrets that need to be managed.

Rinse and repeat.

But behold, there is a solution to this problem. And it’s called centralized secrets management.

Centralized Secrets Management against Secrets Sprawl

The cure for secrets sprawl is centralized secrets management. When you centralize your secrets in one place, you know where they are and who has access to them.

This makes it easier to manage access control and enforce specific access policies. It also makes it easier to rotate secrets as you only need to do it in one place.

Have a look at an example scaling of an organization or team:

When the company is small, the team decides to use Kubernetes to deploy their applications, and they store their secrets in Kubernetes secrets. This first step is already a centralization of secrets as it tightens access to the secrets and makes it easier to audit them. Maintaining an encrypted secrets store and centralizing the secrets in one place is a good start and ensures the organization is following a consistent threat protection policy.

As the company grows, one DevOps engineer is responsible for multiple Kubernetes clusters. In these cases, individual users can’t enforce their own application-wide policies, meaning secrets are better managed and controlled.

But what if the company grows even more and the team decides to use multiple cloud providers? There are many reasons for this, like costs, different service offerings, and better disaster recovery when using multiple cloud providers.

Sticking to the built-in secrets managers of the cloud providers is not a good idea as it will lead to secrets sprawl.

You might also like:

Why Pulumi ESC is the right choice for centralized multi-cloud secrets management

Pulumi ESC is the right choice for centralized multi-cloud capable secrets management as it reduces secrets sprawl by centralizing secrets storage with a lot of integrations for a wide range of platforms.

Pulumi ESC delivers a seamless integration so you can still use your existing secrets managers like Azure Key Vault, AWS Secrets Manager, Google Secret Manager, or HashiCorp Vault. Secrets are automatically synced to Pulumi ESC, saving developers a lot of time so they can focus on their main tasks like developing and building new products and features.

Pulumi ESC is a completely managed service that again saves time and resources for the organization as they don’t need to worry about the infrastructure, maintenance, and support of their self-hosted secrets manager.

Wrapping Up

Every organization faces the risk of secrets sprawl as they grow and add more cloud resources and applications, which they may not even be aware of.

Multi-cloud scenarios make it even more challenging to manage secrets across different systems and environments.

As teams have to juggle between delivering and building new products and features, managing secrets in a secure and efficient way becomes a major bottleneck on productivity and a major security risk. A ticking time bomb that can explode at any time.

Secrets managers can help you reduce secrets sprawl by centralizing secrets storage and providing a seamless way to use, change, and distribute secrets across different systems and environments. Independent of the size of your organization, a startup or a large enterprise, it’s never too late to start managing your secrets in a secure and efficient way.

Pulumi ESC is on the way to becoming the best choice to solve the problem of secrets sprawl. Making both developers and security teams happy by providing a secure and efficient way to manage secrets across different systems and environments.

Start using Pulumi ESC and ESO today by creating an account on the Pulumi Cloud Console and begin managing your secrets in a secure and efficient way.

Create an Account

Pulumi ESC and External Secrets Operator: The Perfect Solution for Today's Cloud-Native Secret Management

Engin Diri — Fri, 04 Oct 2024 00:00:00 +0000

Managing secrets in a cloud-native environment can be challenging, but it is crucial for ensuring the security and integrity of any application or infrastructure. We encounter a lot of different types of secrets, from API keys, database passwords, and certificates to tokens and passwords. These secrets need to be stored securely and accessed by different services in a secure way without exposing any sensitive information to unauthorized users.

Here is where Pulumi ESC and External Secrets Operator come into play by providing a secure and efficient solution for cloud-native secret management.

Pulumi ESC (Environments, Secrets, and Configuration)

Pulumi ESC is our new secrets management & orchestration service for environments, secrets, and configurations. It can be used as part of your Pulumi IaC (Infrastructure as Code) projects using pulumiConfig to expose the stored values to your Pulumi stacks.

But it can also be used as a standalone service to manage secrets and configurations. Pulumi ESC provides support for applications through its dedicated SDKs built for different programming languages. It can also be managed from the command line with the pulumi env commands in the Pulumi CLI, which can inject secrets and configuration as a context for your CLI tools (e.g. terraform).

For more information on Pulumi ESC, you can check out the official documentation.

External Secrets Operator (ESO)

The Open Source project External Secrets Operator (ESO) is a Kubernetes operator that helps manage secrets in a highly secure and scalable way. ESO synchronizes secrets stored in external secret management systems like Pulumi ESC, HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, and more into Kubernetes secrets. This approach allows us to control secret sprawl and centralize the management of secrets in a single place while providing a secure and controlled way to access them.

Why Kubernetes secrets are not enough?

Kubernetes secrets are a great way to store sensitive information in a Kubernetes cluster. Kubernetes secrets are created and stored as a Kubernetes resource object and then used by pods or other resources inside the cluster.

Here is an example of a Kubernetes secret manifest file:

apiVersion: v1
kind: Secret
metadata:
 name: mysecret
type: Opaque
data:
 username: YWRtaW4= # admin
 password: YWRtaW4= # admin

or using kubectl:

kubectl create secret generic mysecret --from-literal=username=admin --from-literal=password=admin

Kubernetes secrets have several limitations. Here are the top ones:

They are stored by default in etcd in base64 encoded format, which is not secure.
They are created either manually by using kubectl commands or by using a manifest file, making them hard to manage at scale.
Hard to manage and synchronize secrets across different environments and clusters.
There is no default way to rotate secrets automatically.

ESO shines by tackling all the challenges listed and providing a way to manage secrets in a more secure and efficient manner. Benefits include:

Mitigate security risks as secrets are stored in a secure external secret management system like Pulumi ESC, which provides more robust security features like encryption, access control, and auditing.
Eliminate manual management of secrets by synchronizing them from external secret management systems into the Kubernetes cluster, reducing the need for manual intervention and minimizing the risk of human error.
ESO provides a unified interface, making it easy to manage secrets across clusters and multi-cloud environments.
ESO integrates with external secret management systems which ensure that access to secrets is tracked and an audit trail is maintained. This is crucial for security compliance and adhering to regulatory standards.
Automate secrets rotation when the external secret management system has support it.

External Secrets Operator architecture

The central part of the ESO architecture is the extension of the Kubernetes API with a new Custom Resource Definition (CRD), which defines where the secrets are stored and how they are synchronized into Kubernetes secrets. The ESO controller then retrieves the secrets from the external secret management system and creates or updates the Kubernetes secrets. Every time a secret is updated in the external secret management system, the ESO controller will synchronize the secret in the Kubernetes cluster and update the secret accordingly.

Core resources of ESO

(Cluster)SecretStore: Defines the external secret management system where the secrets are stored. It contains the necessary information to connect to the external secret management system, such as the endpoint, authentication credentials, and other configuration options. The SecretStore is namespaced while the ClusterSecretStore is cluster wide, which allows you cross-namespaced access to the secrets.
(Cluster)ExternalSecret: Defines the secret that needs to be synchronized into the Kubernetes cluster. It contains the reference to the SecretStore and the secret key and how the data should be transformed and saved as a Kubernetes secret.
PushSecret: With the PushSecret resource, you can push secrets from the Kubernetes cluster to the external secret management system. The Pulumi ESC provider for ESO supports this feature, which allows you to push secrets from the cluster into Pulumi ESC. This feature is useful when you create secrets through other means and want to synchronize them into Pulumi ESC to manage them centrally.

How to Use Pulumi ESC and ESO Together

To demonstrate how to use Pulumi ESC and ESO together, we will store some secrets and configuration values in Pulumi ESC, then synchronize them into a Kubernetes cluster using ESO.

To follow along, you’ll need the following prerequisites:

A Kubernetes cluster (I will be using a local KinD cluster, but you can use any Kubernetes cluster)
Pulumi CLI installed. You can use the pulumi env commands to manage secrets and configurations.
kubectl CLI installed for some debugging

Step 1: Install ESO and Link to Pulumi ESC

We will use Pulumi with the pulumi-kubernetes provider to deploy ESO into the Kubernetes cluster. The power of Pulumi ESC integration comes in two ways: First, to supply the Pulumi access token to ESO via Pulumi IaC, allowing the new ESO instance to access Pulumi ESC later. Second, to provide configuration and secrets from Pulumi ESC, via ESO, to the PodInfo application within the Kubernetes cluster.

pulumi new kubernetes-typescript --name eso-pulumi-esc

You should see the following output:

This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

Project description (A minimal Kubernetes TypeScript Pulumi program):
Created project 'eso-pulumi-esc'

Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`).
Stack name (dev):
Created stack 'dev'

The package manager to use for installing dependencies npm
Installing dependencies...


added 310 packages, and audited 311 packages in 12s

44 packages are looking for funding
 run `npm fund` for details

found 0 vulnerabilities
Finished installing dependencies

Your new project is ready to go! ✨

To perform an initial deployment, run `pulumi up`

Head to Pulumi Cloud Console and create a new Pulumi ESC project with the name eso-esc-demo:

And the environment eso-esc-dev:

In the editor add the following yaml into the Environment definition:

values:
 pulumiConfig:
 pulumi-pat:
 fn::secret: <your-pulumi-pat>

Don’t know how to create a PAT? Check out the official Pulumi documentation.

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/eso-esc-demo/eso-esc-dev

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/eso-esc-demo/eso-esc-dev

Now, we need to link the Pulumi ESC project to the Pulumi IaC project. To do this, we need to add the following to your Pulumi.dev.yaml:

environment:
- eso-esc-demo/eso-esc-dev

Now that all the wiring is done, we can deploy the ESO instance into the Kubernetes cluster. Open the index.ts file and add the following code:

import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

const config = new pulumi.Config();

// Create a Kubernetes namespace
const ns = new k8s.core.v1.Namespace("external-secrets", {
 metadata: {
 name: "external-secrets",
 },
});

// Deploy a Helm release into the namespace
const externalSecrets = new k8s.helm.v3.Release("external-secrets", {
 chart: "external-secrets",
 version: "0.10.4", // Specify the version of the chart
 namespace: ns.metadata.name,
 repositoryOpts: {
 repo: "https://charts.external-secrets.io",
 },
});

// Deploy a secret into the namespace
const mySecret = new k8s.core.v1.Secret("my-secret", {
 metadata: {
 namespace: ns.metadata.name,
 name: "pulumi-access-token",
 },
 stringData: {
 "PULUMI_ACCESS_TOKEN": config.require("pulumi-pat"),
 },
 type: "Opaque",
});

// Export the namespace name
export const namespaceName = ns.metadata.name;

// Export the Helm release name
export const helmReleaseName = externalSecrets.name;

Deploy the stack by running:

pulumi up

And you should see that the secret was created in the Kubernetes cluster and the ESO instance was deployed successfully.

kubectl get secret -n external-secrets pulumi-access-token -o jsonpath='{.data.PULUMI_ACCESS_TOKEN}' | base64 -d

Step 2: Create an External Secret

Now, we will create a secret in the Pulumi ESC project and synchronize it into the Kubernetes cluster by creating an ESO reference to the external secret.

Create a new ESC environment called esp-esc-app in the eso-esc-demo ESC project.

Add the following YAML into the Environment definition:

values:
 app:
 hello: world
 hello-secret:
 fn::secret: world

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/eso-esc-demo/esp-esc-app

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/eso-esc-demo/esp-esc-app

Either way, you should see following environment configuration in the Pulumi Cloud Console:

We can now create ClusterExternalSecret in the index.ts and reference the ESC secret.

import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

// cut for brevity

const crd = new k8s.apiextensions.CustomResource("cluster-secret-store", {
 apiVersion: "external-secrets.io/v1beta1",
 kind: "ClusterSecretStore",
 metadata: {
 name: "secret-store",
 },
 spec: {
 provider: {
 pulumi: {
 organization: pulumi.getOrganization(),
 project: "eso-esc-demo",
 environment: "esp-esc-app",
 accessToken: {
 secretRef: {
 name: mySecret.metadata.name,
 key: "PULUMI_ACCESS_TOKEN",
 namespace: mySecret.metadata.namespace,
 },
 },
 },
 },
 },
});

We can check that the secret was successfully synchronized by running:

kubectl get clustersecretstore secret-store -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'

Step 3: Deploy an Application and Reference the Secret

Now, we can deploy an application that references the secret from the Kubernetes cluster. I am going to deploy the app called PodInfo, which is an application that exposes the pod information as an API. Add the following code to the index.ts:

import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

// cut for brevity
const podInfo = new k8s.helm.v3.Release("podinfo", {
 chart: "podinfo",
 version: "6.7.0",
 namespace: "podinfo",
 createNamespace: true,
 repositoryOpts: {
 repo: "https://stefanprodan.github.io/podinfo",
 },
 values: {
 extraEnvs: [
 {
 name: "FROM_ESC_VIA_ESO",
 valueFrom: {
 secretKeyRef: {
 name: "esc-secret-store",
 key: "hello-secret",
 }
 }
 },
 {
 name: "FROM_ESC_VIA_ESO_2",
 valueFrom: {
 secretKeyRef: {
 name: "esc-secret-store",
 key: "hello",
 }
 }
 }
 ]
 },
});

const externalSecretPodInfo = new k8s.apiextensions.CustomResource("external-secret-podinfo", {
 apiVersion: "external-secrets.io/v1beta1",
 kind: "ExternalSecret",
 metadata: {
 name: "esc-secret-store",
 namespace: podInfo.namespace,
 },
 spec: {
 dataFrom: [
 {
 extract: {
 conversionStrategy: "Default",
 key: "app",
 }
 }
 ],
 refreshInterval: "1h",
 secretStoreRef: {
 kind: clusterSecretStore.kind,
 name: clusterSecretStore.metadata.name,
 }
 },
});

This will deploy the PodInfo application and create an ExternalSecret resource that references the secret from the Pulumi ESC project. We mount the secret into the pod as an environment variable by using the extraEnvs field in the Helm chart.

In a real-world application, you might use alternate ways to reference the secret, like mounting it as a volume or using fromEnv field in the Helm chart. It depends on your application and how you want to consume the secret.

After deploying the stack, you should see that the PodInfo application is running. To access the application, we can port-forward the service by running:

kubectl port-forward svc/podinfo 9898:9898 -n podinfo

Our PodInfo demo application provides a large selection of different endpoints to interact with. We are interested in the /env endpoint to check if the secret was successfully mounted into the pod. Let’s try that:

curl -s localhost:9898/env | grep FROM
 "FROM_ESC_VIA_ESO=world",
 "FROM_ESC_VIA_ESO_2=world",

And here we see secret values from the ESC project, proxied by ESO, and passed to the PodInfo application as environment variables. That means the secret was successfully mounted into the pod and the application is running as expected.

Step 4: Clean Up

After you are done with the demo, you can clean up the resources by running:

pulumi destroy

Conclusion

The combination of Pulumi ESC and External Secrets Operator provides a secure and efficient way to manage secrets in a cloud-native environment, improving security, efficiency, and compliance when consuming secrets in your Kubernetes cluster. By using Pulumi ESC, you can manage secrets and configurations in a centralized way, and with ESO, you now have a streamlined process to synchronize secrets into your Kubernetes cluster.

Start using Pulumi ESC and ESO today by creating an account on the Pulumi Cloud Console and begin managing your secrets in a secure and efficient way.

Create an Account

Pulumi in a Cloud Native World

Engin Diri — Fri, 02 Aug 2024 00:00:00 +0000

In today’s complex digital landscape, organizations are increasingly turning to platform engineering to optimize their software delivery processes and maximize efficiency. The growing complexity of modern applications, coupled with the need for rapid, secure, and scalable deployments, has created a pressing demand for robust Internal Developer Platforms (IDPs).

IDPs are a key component of modern platform engineering strategies. An IDP is a self-service layer that sits on top of an organization’s infrastructure, abstracting away complexity and providing developers with the tools and environments they need to build, test, and deploy applications efficiently.

Key features of an IDP:

Developer Control Plane. Curated experiences that empower developers by meeting them at their level of expertise, whether it’s an abstracted developer portal, custom CLI, or shared IaC templates.
Integration & Delivery. Automations to version control, test, trace, and deploy all infrastructure from resources, configurations, environments, and secrets as well as orchestration automations to manage provisioning workflows.
Monitoring & Logging. Components to log, monitor, and observe all infrastructure for greater operational control as well as optimize against unnecessary costs.
Security & Identity. Security and compliance guardrails that regulate every piece of infrastructure from policies to fine-grained access controls to secrets.
Resources. Providers that support modern cloud architectures such as Kubernetes, containers, serverless, generative AI, machine learning, data lakes, hybrid cloud/on-premises, and more.

By implementing an IDP, organizations can significantly reduce cognitive load on developers, accelerate time-to-market, improve security and compliance, and foster a culture of innovation. IDPs act as a force multiplier, allowing development teams to focus on creating business value while the platform handles the underlying complexity of cloud-native infrastructure.

The Cloud Native Computing Foundation (CNCF) landscape offers a large ecosystem of tools and technologies that can be leveraged to build these platforms from scratch. However, the sheer number of options can be overwhelming, making it difficult for organizations to determine the best combination of tools for their specific needs.

At Pulumi’s Customer Experience Team, we’ve observed that adopting Infrastructure as Code often serves as a catalyst for organizations to reassess their entire software delivery process. This reassessment frequently leads to the implementation of more streamlined, automated, and secure methodologies – the core principles of platform engineering mentioned above.

To provide guidance for our customers, we have created a reference architecture called the KEBAP stack, this stack offers a structured approach to integrate these technologies into an effective platform engineering solution using Pulumi.

The KEBAP Stack Reference Architecture

The KEBAP stack consists of the following (most commonly used) CNCF projects:

Kubernetes (with Kyverno as a bonus)
External Secrets Operator
Backstage
Argo CD
Pulumi

Have a look on the high-level architectural diagram of the KEBAP stack to get a first idea of what we are going to talk about in the following sections:

Breaking Down the KEBAP Stack

Let’s take a closer look at each component of the KEBAP stack and understand what their role. The KEBAP is layered in such a way that each component builds on the capabilities of the previous one. The five layers of the KEBAP stack are designed to address different capabilities of our internal developer platform:

Kubernetes

In modern cloud computing, the hub-and-spoke architecture pattern is widely used to manage multiple Kubernetes clusters. In this pattern, a central control plane (hub) manages a set of clusters (spokes) that run applications and services. The hub provides a unified view of the entire infrastructure, enabling centralized management, monitoring, and security.

The control plane cluster in the KEBAP stack is the hub cluster and contains the control plane logic for the entire IDP.

Pulumi

Pulumi manages the lifecycle of all cloud infrastructure, providing developers with self-service capabilities. It allows you to use familiar programming languages to define your infrastructure as code, offering more flexibility and power than traditional YAML or domain-specific languages.

The Pulumi Kubernetes Operator enables you to manage your infrastructure directly from within Kubernetes, seamlessly integrating with your existing Kubernetes workflows.

Since we use GitOps as our deployment method, we define the infrastructure in a Git repository and then let Argo CD deploy it to the hub cluster. As soon as the infrastructure definition is deployed, the Pulumi Kubernetes Operator will take over and create the defined resources in the selected cloud provider.

Here is an example Pulumi definition of a new Kubernetes cluster using DigitalOcean:

apiVersion: pulumi.com/v1
kind: Program
metadata:
 name: ${{values.name}}
program:
 variables:
 clusterName: "do-cluster"
 clusterRegion: "nyc1"
 nodePoolName: "default"
 nodeCount: 1
 version: 1.30.1-do.0

 resources:
 do_cluster:
 type: digitalocean:KubernetesCluster
 properties:
 name: "${clusterName}"
 region: "${clusterRegion}"
 version: "${version}"
 destroyAllAssociatedResources: true
 nodePool:
 name: "${nodePoolName}"
 size: "s-2vcpu-2gb"
 nodeCount: "${nodeCount}"
 # cut for brevity
---
apiVersion: pulumi.com/v1
kind: Stack
metadata:
 name: ${{values.name}}
spec:
 stack: ${{values.stack}}
 envRefs:
 DIGITALOCEAN_TOKEN:
 type: Env
 env:
 name: DIGITALOCEAN_TOKEN
 programRef:
 name: ${{values.name}}
 destroyOnFinalize: false

Argo CD

Argo CD handles the deployment layer, ensuring that the current deployment matches what is declaratively defined in the source. It’s emerging as the de facto standard due to its rich front-end and powerful features. Argo CD implements the GitOps model, continuously monitoring your Git repositories and automatically updating the deployed applications to match the desired state.

In the KEBAP stack, we use Argo CD to deploy the Pulumi infrastructure defined in the Git repository to our hub cluster.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
 name: ${{values.name}}-${{values.stage}}-${{ (values.owner | parseEntityRef).name }}
 namespace: argocd
spec:
 destination:
 namespace: pulumi-operator
 server: https://kubernetes.default.svc
 project: ${{ (values.owner | parseEntityRef).name }}
 source:
 path: "gitops/teams/clusters/${{values.name}}-${{values.stage}}-cluster"
 directory:
 recurse: false
 exclude: '{catalog-info.yaml}'
 targetRevision: main
 repoURL: https://github.com/my-backstage-demo/backstage-infrastructure-provisioning-templates-workshop.git
 syncPolicy:
 automated:
 prune: true
 selfHeal: true
 syncOptions:
 - ServerSideApply=true
 - CreateNamespace=true

External Secrets Operator (ESO)

The External Secrets Operator (ESO) addresses the challenge of securely managing secrets in Kubernetes. It integrates with various secret stores (such as Pulumi ESC, AWS Secrets Manager, HashiCorp Vault, Google Secrets Manager, Azure Key Vault). This allows you to configure access to the secret store once for all your required secrets, enhancing security and simplifying management.

We even take it a step further by using ESO to manage the secrets from Pulumi ESC (Environments, Secrets, and Configurations) with the Pulumi Provider for ESO. This way, we use the Pulumi not only for the infrastructure but also for the secrets management reducing the number of tools we need to manage.

To give the Pulumi Kubernetes Operator access to the secrets, we create following ClusterSecretStore object:

apiVersion: external-secrets.io/v1beta1
kind: ClusterSecretStore
metadata:
 name: secret-store
spec:
 provider:
 pulumi:
 organization: <org>
 environment: <env>
 accessToken:
 secretRef:
 name: pulumi-access-token
 key: PULUMI_ACCESS_TOKEN
 namespace: argocd

And then we can define the ExternalSecret with all the secrets that we want to manage and pass to our Operator as environment variables extraEnv:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
 name: pulumi-operator-secrets
spec:
 refreshInterval: 1h
 secretStoreRef:
 kind: ClusterSecretStore
 name: secret-store
 data:
 - secretKey: pulumi-access-token
 remoteRef:
 key: "backstage.pulumi-pat"
 - secretKey: do-token
 remoteRef:
 key: "backstage.do"

Backstage

Backstage is an open platform for building developer portals. It provides a centralized place for managing software catalogs, documentation, and tooling. This layer helps in organizing microservices and infrastructure, streamlining the process for developers to create, manage, and explore services.

In our KEBAB stack, we installed the Pulumi plugin for Backstage, which allows us to manage our infrastructure as code directly from the Backstage UI by cookie-cutting the Pulumi project and making the result available in our source control repository. From there, Argo CD takes over and deploys the definition to the hub cluster as described above.

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
 name: kubernetes-template
 title: Kubernetes Cluster
 description: |
 A template for creating a new Kubernetes Cluster.
 tags:
 - pulumi
 - kubernetes
spec:
 steps:
 - id: pulumi-new-component
 name: Cookie cut the component Pulumi project
 action: pulumi:new
 input:
 name: "${{ parameters.component_id }}-infrastructure"
 description: ${{ parameters.description | dump }}
 organization: ediri
 stack: ${{ parameters.stack }}
 template: "https://github.com/my-silly-organisation/microservice-civo/tree/main/infrastructure-${{ parameters.cloud }}-${{ parameters.language }}"
 config:
 "node:node_count": "${{ parameters.nodeCount }}"
 folder: .

Bonus: Kyverno

Kyverno can be used to enforce policies and automate security and operational best practices in Kubernetes clusters. Kyverno is a policy engine designed for Kubernetes that allows you to define policies as code and enforce them at runtime. This ensures that your Kubernetes clusters are secure and compliant with your organization’s policies.

In the KEBEP stack, we use Kyverno to create a ClusterPolicy object to automatically add vCluster clusters to the Argo CD as spoke clusters. This will generate a Secret object in the argocd namespace for each vCluster cluster the dev teams create.

apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
 name: sync-secret-vcluster
 annotations:
 argocd.argoproj.io/sync-wave: "5"
spec:
 generateExistingOnPolicyUpdate: true
 rules:
 - name: sync-secret-vcluster-nonprod
 match:
 any:
 - resources:
 names:
 - "*-dev-*"
 - "*-staging-*"
 - "*-prod-*"
 kinds:
 - Secret
 exclude:
 any:
 - resources:
 namespaces:
 - kube-system
 - default
 - kube-public
 - kyverno
 context:
 - name: namespace
 variable:
 value: "{{ request.object.metadata.namespace }}"
 - name: name
 variable:
 value: "{{ request.object.metadata.name }}"
 - name: ca
 variable:
 value: "{{ request.object.data.\"certificate-authority\" }}"
 - name: cert
 variable:
 value: "{{ request.object.data.\"client-certificate\" }}"
 - name: key
 variable:
 value: "{{ request.object.data.\"client-key\" }}"
 - name: vclusterName
 variable:
 value: "{{ replace_all(name, 'vc-', '') }}"
 jmesPath: 'to_string(@)'
 - name: stage
 variable:
 value: "{{ split(vclusterName, '-')[2] }}"
 jmesPath: 'to_string(@)'
 - name: team
 variable:
 value: "{{ split(vclusterName, '-')[3] }}"
 jmesPath: 'to_string(@)'
 generate:
 kind: Secret
 apiVersion: v1
 name: "{{ vclusterName }}"
 namespace: argocd
 synchronize: true
 data:
 kind: Secret
 metadata:
 labels:
 argocd.argoproj.io/secret-type: cluster
 clusterType: "vcluster"
 env: "{{ stage }}"
 team: "{{ team }}"
 stringData:
 name: "{{ vclusterName }}"
 server: "https://{{ vclusterName }}.{{ namespace }}:443"
 config: |
 {
 "tlsClientConfig": {
 "insecure": false,
 "caData": "{{ ca }}",
 "certData": "{{ cert }}",
 "keyData": "{{ key }}"
 }
 }

Putting all the Pieces Together

As we have seen, each component of the KEBAP stack plays a crucial role in the overall architecture. Let’s how the different workflows look like when we put all the pieces together. We go through the following scenarios:

Ignite the Control Plane Cluster With Pulumi
Ordering a New Development Kubernetes Cluster
Ordering a New Production Kubernetes Cluster
Ordering Other Infrastructure

Ignite the Control Plane Cluster With Pulumi

We use Pulumi code to create the control plane cluster in our cloud provider of choice. After the creation of the control plane, we install the Argo CD in a very bare-bones way:

const argocd = new ArgoCD("argocd", {
 initialObjects: initialObjects
}, {
 providers: {
 kubernetes: k8sProvider,
 },
});

The idea is to let then Argo CD fetch the rest of the configuration from the Git repository and apply it to the cluster. And here comes the power of GitOps: It will also manage the Argo CD itself. This way, we separated the workload running on the cluster from the creation of the cluster itself.

Ordering a New Development Kubernetes Cluster

Starting point is Backstage, where we can order a new development Kubernetes cluster. As we want to keep the costs low for development cluster, we opted here for the vCluster project. The vCluster project allows us to create a Kubernetes cluster on demand, which runs as containers on the control plane cluster. The moment the cluster is created, we add it to the Argo CD as spoke cluster. This way, we can roll out applications to the development cluster in the same way as we would do for production clusters.

See the definition of the ClusterPolicy object in the Kyverno section.

Ordering a New Production Kubernetes Cluster

The Backstage catalog is again the starting point to order a new production Kubernetes cluster. This time, we use the Pulumi code to create the cluster in the cloud provider of choice. After the creation of the cluster, we add it to the Argo CD as a production cluster. This way, we can roll out applications to the production cluster in the same way as we would do for development clusters.

Ordering Other Infrastructure

Any other infrastructure components such as databases, message queues, and storage will be also ordered via Backstage. And similar to the ordering of a new production Kubernetes cluster, we use Pulumi to create the cloud infrastructure.

Conclusion

A lot of organizations are building their internal developer platforms using CNCF projects to fulfill the needs of their internal customers. The KEBAP stack provides a structured approach to integrate these technologies into an effective platform engineering solution. We also saw that Pulumi plays a crucial role in this stack by taking care of the management of the cloud infrastructure.

This KEBAP stack offers several benefits, including:

A secure, automated, and auditable environment
Reproducible and programmable infrastructure-as-code
Unified management of cloud resources, Kubernetes, and application deployments
Faster onboarding for new team members
Easier compliance with regulatory requirements due to increased auditability

But there is also a downside: The KEBAP stack requires a significant investment in time and resources to set up and maintain as you are in full charge of the integration of the components as well as operating them. All the projects under the umbrella of CNCF are not automatically implying that they are compatible with each other out of the box. Any platform engineering team will need to invest continuously time and resources to keep the stack up-to-date and running. This is a not negligible investment to consider for any organization.

For this reason, we have another reference architecture that uses components from all of the Pulumi Platform for an end to end fully-managed solution. This solution takes care of the integration of the different components of an internal developer portal and provides a carefree experience of managing your infrastructure at scale.

Continue to upskill and learn about platform engineering:

Read Pulumi Patterns and Practices (P3): A Pulumi-based reference architecture for large-scale organizations
Learn how to build Developer Portals with Pulumi.
Register for one of our Platform Engineering or DevOps workshops.

Pulumi vs HCL: Understanding the Language Differences in Infrastructure as Code

Engin Diri — Tue, 09 Jul 2024 00:00:00 +0000

The Java Language Architect at Oracle, Brian Goetz, author of Java Concurrency in Practice, has commented how declarative languages can be a double-edged sword:

HashiCorp’s infrastructure as code solution, Terraform, uses a domain-specific language (DSL) to declare cloud resources. Pulumi’s infrastructure as code solution, on the other hand, lets you choose from any number of modern languages – C#, Java, JavaScript, Go, Python, or TypeScript – or the industry-standard markup language YAML, to declare cloud resources. Although both Terraform and Pulumi are declarative infrastructure as code engines at their core, this fundamentally different approach to expression languages has significant consequences.

Real-World Consequences

We’ve worked with 1,000s of companies and 100,000+ developers and infrastructure experts alike to adopt infrastructure as code using Pulumi and modern languages, and we see several significant recurring themes as they migrate away from Terraform and HCL.

Tapping Into Millions of Developers

Pulumi allows developers to use widely-adopted general-purpose languages for infrastructure as code. This approach taps into vast existing developer communities, with JavaScript boasting over 17 million developers, Python 15 million, and Go 3 million worldwide. By utilizing these languages, Pulumi enables teams to leverage their current programming expertise and access extensive resources, libraries, and tools. Meanwhile, HCL maintains a dedicated user base of several hundred thousand developers who appreciate its specialized syntax for infrastructure description.

Pulumi Leverages Rich Ecosystems of General-Purpose Languages

Pulumi harnesses the extensive ecosystems surrounding popular programming languages for infrastructure as code. While domain-specific languages like HCL have their strengths, Pulumi’s approach offers developers access to:

Comprehensive IDE support, including features like autocompletion, syntax highlighting, and real-time error detection, which may be more advanced than those available for specialized IaC languages
Wide array of established linting tools to enforce coding standards and catch potential issues early in the development process
Robust testing frameworks that allow for thorough verification and validation of infrastructure configurations, often more mature than those specific to IaC languages
Extensive libraries and packages that can be seamlessly integrated into infrastructure code, expanding possibilities beyond traditional IaC toolsets

This approach allows teams to utilize familiar tools and workflows, potentially increasing productivity and code quality. Developers can leverage their existing programming expertise and the vast resources available in these language communities when working on infrastructure projects, which may require less additional learning compared to adopting a specialized syntax

Pulumi Embraces Open-Source Languages and Principles

Languages like Python, JavaScript, and Go are developed and maintained by diverse global communities, ensuring transparency and collective innovation. These languages have remained open-source throughout their history, with their core components freely available for use, modification, and distribution.

In contrast, some domain-specific IaC languages have different ownership and licensing models. For example, HCL has always been under the control of a single company. Recently, there have been changes in its licensing, with parts of its execution engine moving to a “source available” license that is no longer considered fully open-source.

By using mainstream open-source languages, Pulumi offers teams the benefits of well-established ecosystems, community-driven development, and the assurance of continued open availability. This approach can provide additional confidence for organizations concerned about long-term access and evolution of their IaC tooling.

Here’s a list of languages Pulumi commonly uses, along with their licenses:

Language	License
C#	MIT (.NET Core)
Java	GNU v2.0 (OpenJDK)
JavaScript	MIT (ECMAScript), BSD-style 3-Clause (V8)
Go	BSD-style 3-Clause
Python	BSD-style Python Software Foundation
TypeScript	Apache 2.0
YAML	Creative Commons Attribution

Developers Onboard Quicker with Pulumi

While HCL serves its purpose in infrastructure as code, many developers find Pulumi’s approach more appealing. Pulumi allows engineers to use familiar programming languages, making it easier to leverage existing skills and tools.

This familiarity often leads to faster onboarding and more efficient debugging processes. Pulumi’s flexibility enables developers to create custom solutions with ease, tapping into the full power of general-purpose languages. Additionally,

Pulumi’s open architecture provides developers with the freedom to integrate various tools and services, fostering a diverse and adaptable ecosystem for infrastructure management.

Enhancing Team Collaboration With Pulumi

Pulumi offers several advantages for fostering better collaboration across teams compared to HCL. While HCL is a powerful tool primarily utilized by operations and DevOps teams, its specialized nature for infrastructure tasks might limit broader engagement with development teams who use other programming languages.

Pulumi, on the other hand, supports multiple languages, allowing development teams to use familiar tools and languages which can enhance collaboration and streamline processes. Additionally, Pulumi’s flexibility in integrating with a variety of tools can help bridge gaps between different teams, ensuring smoother communication and more effective teamwork.

This inclusive approach not only accelerates progress but also democratizes access to infrastructure management, avoiding bottlenecks and enhancing overall team efficiency.

Reducing Redundancy and Copy-Paste Challenges

HCL often encounters a significant challenge with “copy-pasta,” where developers frequently copy and paste code. This practice can lead to redundant configurations across environments, bloating the codebase and complicating management.

Additionally, this increases the risk of errors as small mistakes can propagate across multiple instances, and each one must be manually updated when changes are needed, heightening the chance for inconsistencies. As projects evolve, maintaining a codebase with extensive copy-pasta becomes burdensome, slowing development and elevating the likelihood of bugs.

In contrast, Pulumi reduces the need for such redundancies by enabling more dynamic code reuse and better resource management, streamlining updates and fostering more robust and error-resistant infrastructure development.

Reduce Need for Custom Tooling

As projects expand, managing HCL configurations becomes increasingly complex, often necessitating custom tools for efficient large-scale management. While HCL is adept at defining infrastructure as code, it lacks built-in features to orchestrate, validate, and deploy configurations across extensive networks smoothly. Teams usually need to develop bespoke tools or integrate external solutions like Terragrunt, Atlantis, and Sentinel to automate tasks, ensure consistency, and facilitate integration with other systems. This process can demand significant resources and expertise.

In contrast, Pulumi is designed to scale seamlessly with projects, reducing the need for external tools or extensive custom solutions. Pulumi provides robust capabilities for managing large-scale projects efficiently, streamlining the deployment process, and ensuring consistency across various environments without the overhead of additional tooling. This makes Pulumi a more straightforward and resource-efficient choice for teams looking to manage their infrastructure effectively as they grow.

Future-Proofing Your Infrastructure

HCL, while robust, lacks the future-proof qualities of Pulumi SDKs due to its fixed syntax and usage constraints. In contrast, Pulumi’s support for multiple programming languages like Python, TypeScript, and Go, coupled with the potential to adopt new languages over time, offers unparalleled flexibility and adaptability.

This allows developers to write infrastructure as code in familiar languages, facilitating easier integration with existing projects and workflows. As technology progresses, the capacity to embrace new programming languages and paradigms becomes essential. Pulumi’s adaptable model supports this continuous evolution, ensuring that infrastructure as code practices can keep pace with the dynamic nature of development environments and preferences, thereby offering a more sustainable and scalable solution for long-term projects.

Making Infrastructure Code Simpler

As HCL continues to evolve, its fixed syntax and structured approach are likely to lead to greater complexity. This evolution could make the language increasingly specialized and challenging for newcomers, particularly those already proficient in more widely-used programming languages. As HCL adapts more bespoke features to meet specific needs, it may become less intuitive and harder to handle effectively, potentially resulting in a language that is cumbersome to use and maintain, especially in long-term projects.

In contrast, Pulumi offers a more streamlined and adaptable alternative. Supporting multiple mainstream programming languages like Python, TypeScript, and Go, Pulumi enables developers to work with tools and languages they are already familiar with, greatly reducing the learning curve and enhancing productivity. This flexibility allows Pulumi to integrate seamlessly into existing projects and workflows, making it a more sustainable and scalable choice for modern infrastructure needs.

HCL in the Wild: Practical Comparisons With Pulumi

In this section, we’ll explore real-life examples to compare and contrast HCL with Pulumi programs. By examining how each approach handles similar infrastructure tasks, you’ll quickly appreciate the advantages of leveraging a familiar programming language through Pulumi.

This comparison aims to highlight the practical benefits and increased efficiency that Pulumi offers over traditional HCL configurations.

`for_each` Loop With Conditional Expression

First, let’s look at an HCL example featuring a for_each loop with a conditional statement. At first glance, it may seem straightforward, but we can immediately see the unique way a loop must be defined in HCL.

Initially, you must decide whether to use count or for_each. Following this, you must adhere to the strict format for implementing conditional statements.

This contrasts starkly with how you would use these constructs in a generic programming language.

variable "storage_account_names" {
 type = list(string)
 default = ["mickey", "donald", "goofy"]
}

resource "azurerm_resource_group" "example" {
 name = "storage-rg"
 location = "West Europe"
}

resource "azurerm_storage_account" "my_storage" {
 for_each = {
 for name in var.storage_account_names :
 name => name
 if name != "goofy"
 }
 name = each.value
 resource_group_name = azurerm_resource_group.example.name
 location = azurerm_resource_group.example.location
 account_tier = "Standard"
 account_replication_type = "GRS"
}

Now let’s look at how we can approach the same example using Pulumi.

const storageAccountNames: string[] = [
 "mickey",
 "donald",
 "goofy",
];
const example = new azure.core.ResourceGroup("example", {
 name: "storage-rg",
 location: "West Europe",
});

storageAccountNames.filter((name: string) => name !== "goofy").forEach((name: string) => {
 new azure.storage.Account(`my_storage-${name}`, {
 name: name,
 resourceGroupName: example.name,
 location: example.location,
 accountTier: "Standard",
 accountReplicationType: "GRS",
 });
});

import pulumi
import pulumi_azure as azure

storage_account_names = ["mickey", "donald", "goofy"]

example = azure.core.ResourceGroup(
 "example",
 name="storage-rg",
 location="West Europe"
)

for name in filter(lambda n: n != "goofy", storage_account_names):
 azure.storage.Account(
 f"my_storage-{name}",
 name=name,
 resource_group_name=example.name,
 location=example.location,
 account_tier="Standard",
 account_replication_type="GRS"
 )

package main

import (
 "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/core"
 "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/storage"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 storageAccountNames := []string{"mickey", "donald", "goofy"}

 example, err := core.NewResourceGroup(ctx, "example", &core.ResourceGroupArgs{
 Name: pulumi.String("storage-rg"),
 Location: pulumi.String("West Europe"),
 })
 if err != nil {
 return err
 }

 for _, name := range storageAccountNames {
 if name != "goofy" {
 _, err := storage.NewAccount(ctx, "my_storage-"+name, &storage.AccountArgs{
 Name: pulumi.String(name),
 ResourceGroupName: example.Name,
 Location: example.Location,
 AccountTier: pulumi.String("Standard"),
 AccountReplicationType: pulumi.String("GRS"),
 })
 if err != nil {
 return err
 }
 }
 }

 return nil
 })
}

using Pulumi;
using Pulumi.Azure.Core;
using Pulumi.Azure.Storage;
using System.Collections.Generic;

class Program
{
 static Task<int> Main() => Deployment.RunAsync<MyStack>();
}

class MyStack : Stack
{
 public MyStack()
 {
 var storageAccountNames = new List<string> { "mickey", "donald", "goofy" };

 var resourceGroup = new ResourceGroup("example", new ResourceGroupArgs
 {
 Name = "storage-rg",
 Location = "West Europe",
 });

 foreach (var name in storageAccountNames)
 {
 if (name != "goofy")
 {
 var storageAccount = new Account($"my_storage-{name}", new AccountArgs
 {
 Name = name,
 ResourceGroupName = resourceGroup.Name,
 Location = resourceGroup.Location,
 AccountTier = "Standard",
 AccountReplicationType = "GRS",
 });
 }
 }
 }
}

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.azurenative.core.ResourceGroup;
import com.pulumi.azurenative.core.ResourceGroupArgs;
import com.pulumi.azurenative.storage.Account;
import com.pulumi.azurenative.storage.AccountArgs;

import java.util.List;

public class Main {
 public static void main(String[] args) {
 Pulumi.run(Main::stack);
 }

 public static void stack(Context ctx) {
 var storageAccountNames = List.of("mickey", "donald", "goofy");

 var resourceGroup = new ResourceGroup("example", ResourceGroupArgs.builder()
 .name("storage-rg")
 .location("West Europe")
 .build());

 for (var name : storageAccountNames) {
 if (!name.equals("goofy")) {
 var storageAccount = new Account("my_storage-" + name, AccountArgs.builder()
 .name(name)
 .resourceGroupName(resourceGroup.name())
 .location(resourceGroup.location())
 .kind("StorageV2")
 .sku(AccountSkuArgs.builder()
 .name("Standard_GRS")
 .tier("Standard")
 .build())
 .build());
 }
 }
 }
}

We can see that we leverage the full potential that the generic programming language offers, using filter to streamline our array without the need for conditional statements, and then iterating through the reduced array with forEach. This approach is less error-prone, faster, and easier to debug.

However, implementing a for_each in HCL can become even more complicated.

`for_each` Loop With Multi-Level Nested Dynamic Blocks

To give you an idea of how HCL handles cascading nested dynamic providers, consider this: dynamic providers are intended to dynamically construct repeatable nested blocks. However, this complexity can make the code difficult to read and maintain. We will examine a specific resource from this GitHub repository to provide a clearer illustration:

variable "enable_auto_scaling" {
 type = bool
 default = false
 description = "Enable node pool autoscaling"
}

variable "agents_pool_kubelet_configs" {
 type = list(object({
 cpu_manager_policy = optional(string)
 cpu_cfs_quota_enabled = optional(bool, true)
 cpu_cfs_quota_period = optional(string)
 image_gc_high_threshold = optional(number)
 image_gc_low_threshold = optional(number)
 topology_manager_policy = optional(string)
 allowed_unsafe_sysctls = optional(set(string))
 container_log_max_size_mb = optional(number)
 container_log_max_line = optional(number)
 pod_max_pid = optional(number)
 }))
 default = [
 {
 cpu_manager_policy = "none"
 cpu_cfs_quota_enabled = true
 cpu_cfs_quota_period = "100ms"
 image_gc_high_threshold = 85
 image_gc_low_threshold = 65
 topology_manager_policy = "none"
 allowed_unsafe_sysctls = ["net.*", "kernel.*"]
 container_log_max_size_mb = 10
 container_log_max_line = 10
 pod_max_pid = 100
 }
 ]
 nullable = false
}

resource "azurerm_resource_group" "example" {
 name = "storage-rg"
 location = "West Europe"
}

resource "azurerm_kubernetes_cluster" "main" {
 name = "example-aks"
 location = azurerm_resource_group.example.location
 resource_group_name = azurerm_resource_group.example.name
 dns_prefix = "exampleaks"

 dynamic "default_node_pool" {
 for_each = var.enable_auto_scaling == true ? [] : ["default_node_pool_manually_scaled"]
 content {
 name = "default"
 vm_size = "Standard_DS2_v2"
 zones = [1, 2, 3]
 enable_auto_scaling = true
 min_count = 1
 max_count = 3
 dynamic "kubelet_config" {
 for_each = var.agents_pool_kubelet_configs
 content {
 allowed_unsafe_sysctls = kubelet_config.value.allowed_unsafe_sysctls
 }
 }
 }
 }
 identity {
 type = "SystemAssigned"
 }
}

As we can see, incorporating even one nested dynamic provider block can make the code difficult to read. We recommend using dynamic blocks sparingly to maintain readability and ease of maintenance, particularly when creating an abstraction with the help of a module. Overuse of these blocks, especially when defining a resource’s arguments and nested blocks directly from input variables, might not provide a useful abstraction.

Have look how this would look like when using Pulumi:

const config = new pulumi.Config();
const enableAutoScaling = config.getBoolean("enableAutoScaling") || false;

const agentsPoolKubeletConfigs = config.getObject("agentsPoolKubeletConfigs") || {
 allowedUnsafeSysctls: [
 "kernel.*",
 "net.*",
 ],
 containerLogMaxLine: 10,
 containerLogMaxSizeMb: 10,
 cpuCfsQuotaEnabled: true,
 cpuCfsQuotaPeriod: "100ms",
 cpuManagerPolicy: "none",
 imageGcHighThreshold: 85,
 imageGcLowThreshold: 65,
 podMaxPid: 100,
 topologyManagerPolicy: "none",
};
const example = new azure.core.ResourceGroup("example", {
 name: "storage-rg",
 location: "West Europe",
});

let kubeletConfigs: azure.types.input.containerservice.KubernetesClusterDefaultNodePoolKubeletConfig = agentsPoolKubeletConfigs;

let defaultNodePool: azure.types.input.containerservice.KubernetesClusterDefaultNodePool = {
 name: "default",
 vmSize: "Standard_DS2_v2",
 zones: [
 "1",
 "2",
 "3",
 ],
 enableAutoScaling: enableAutoScaling,
 minCount: 1,
 maxCount: 3,
 kubeletConfig: kubeletConfigs,
};

import pulumi
import pulumi_azure as azure

config = pulumi.Config()
enable_auto_scaling = config.get_bool("enableAutoScaling") or False

agents_pool_kubelet_configs = config.get_object("agentsPoolKubeletConfigs") or {
 "allowedUnsafeSysctls": [
 "kernel.*",
 "net.*",
 ],
 "containerLogMaxLine": 10,
 "containerLogMaxSizeMb": 10,
 "cpuCfsQuotaEnabled": True,
 "cpuCfsQuotaPeriod": "100ms",
 "cpuManagerPolicy": "none",
 "imageGcHighThreshold": 85,
 "imageGcLowThreshold": 65,
 "podMaxPid": 100,
 "topologyManagerPolicy": "none",
}

example = azure.core.ResourceGroup("example",
 name="storage-rg",
 location="West Europe")

kubelet_configs = azure.containerservice.KubernetesClusterDefaultNodePoolKubeletConfigArgs(
 **agents_pool_kubelet_configs
)

default_node_pool = azure.containerservice.KubernetesClusterDefaultNodePoolArgs(
 name="default",
 vm_size="Standard_DS2_v2",
 zones=["1", "2", "3"],
 enable_auto_scaling=enable_auto_scaling,
 min_count=1,
 max_count=3,
 kubelet_config=kubelet_configs,
)

package main

import (
 "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/core"
 "github.com/pulumi/pulumi-azure/sdk/v5/go/azure/containerservice"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 enableAutoScaling := false

 agentsPoolKubeletConfigs := map[string]interface{}{
 "allowedUnsafeSysctls": []string{
 "kernel.*",
 "net.*",
 },
 "containerLogMaxLine": 10,
 "containerLogMaxSizeMb": 10,
 "cpuCfsQuotaEnabled": true,
 "cpuCfsQuotaPeriod": "100ms",
 "cpuManagerPolicy": "none",
 "imageGcHighThreshold": 85,
 "imageGcLowThreshold": 65,
 "podMaxPid": 100,
 "topologyManagerPolicy": "none",
 }

 example, err := core.NewResourceGroup(ctx, "example", &core.ResourceGroupArgs{
 Name: pulumi.String("storage-rg"),
 Location: pulumi.String("West Europe"),
 })
 if err != nil {
 return err
 }

 kubeletConfigs := &containerservice.KubernetesClusterDefaultNodePoolKubeletConfigArgs{
 AllowedUnsafeSysctls: pulumi.StringArray{
 pulumi.String("kernel.*"),
 pulumi.String("net.*"),
 },
 ContainerLogMaxLine: pulumi.Int(agentsPoolKubeletConfigs["containerLogMaxLine"].(int)),
 ContainerLogMaxSizeMb: pulumi.Int(agentsPoolKubeletConfigs["containerLogMaxSizeMb"].(int)),
 CpuCfsQuotaEnabled: pulumi.Bool(agentsPoolKubeletConfigs["cpuCfsQuotaEnabled"].(bool)),
 CpuCfsQuotaPeriod: pulumi.String(agentsPoolKubeletConfigs["cpuCfsQuotaPeriod"].(string)),
 CpuManagerPolicy: pulumi.String(agentsPoolKubeletConfigs["cpuManagerPolicy"].(string)),
 ImageGcHighThreshold: pulumi.Int(agentsPoolKubeletConfigs["imageGcHighThreshold"].(int)),
 ImageGcLowThreshold: pulumi.Int(agentsPoolKubeletConfigs["imageGcLowThreshold"].(int)),
 PodMaxPid: pulumi.Int(agentsPoolKubeletConfigs["podMaxPid"].(int)),
 TopologyManagerPolicy: pulumi.String(agentsPoolKubeletConfigs["topologyManagerPolicy"].(string)),
 }

 _, err = containerservice.NewKubernetesClusterDefaultNodePool(ctx, "defaultNodePool", &containerservice.KubernetesClusterDefaultNodePoolArgs{
 Name: pulumi.String("default"),
 VmSize: pulumi.String("Standard_DS2_v2"),
 Zones: pulumi.StringArray{pulumi.String("1"), pulumi.String("2"), pulumi.String("3")},
 EnableAutoScaling: pulumi.Bool(enableAutoScaling),
 MinCount: pulumi.Int(1),
 MaxCount: pulumi.Int(3),
 KubeletConfig: kubeletConfigs,
 }, pulumi.Parent(example))
 if err != nil {
 return err
 }

 return nil
 })
}

using Pulumi;
using Pulumi.AzureNative.Resources;
using Pulumi.AzureNative.ContainerService;
using System.Collections.Generic;

class MyStack : Stack
{
 public MyStack()
 {
 var enableAutoScaling = false;

 var agentsPoolKubeletConfigs = new Dictionary<string, object>
 {
 { "allowedUnsafeSysctls", new string[] { "kernel.*", "net.*" } },
 { "containerLogMaxLine", 10 },
 { "containerLogMaxSizeMb", 10 },
 { "cpuCfsQuotaEnabled", true },
 { "cpuCfsQuotaPeriod", "100ms" },
 { "cpuManagerPolicy", "none" },
 { "imageGcHighThreshold", 85 },
 { "imageGcLowThreshold", 65 },
 { "podMaxPid", 100 },
 { "topologyManagerPolicy", "none" }
 };

 var resourceGroup = new ResourceGroup("example", new ResourceGroupArgs
 {
 Name = "storage-rg",
 Location = "West Europe"
 });

 var kubeletConfigArgs = new KubernetesClusterDefaultNodePoolKubeletConfigArgs
 {
 AllowedUnsafeSysctls = agentsPoolKubeletConfigs["allowedUnsafeSysctls"] as string[],
 ContainerLogMaxLine = agentsPoolKubeletConfigs["containerLogMaxLine"] as int?,
 ContainerLogMaxSizeMb = agentsPoolKubeletConfigs["containerLogMaxSizeMb"] as int?,
 CpuCfsQuotaEnabled = agentsPoolKubeletConfigs["cpuCfsQuotaEnabled"] as bool?,
 CpuCfsQuotaPeriod = agentsPoolKubeletConfigs["cpuCfsQuotaPeriod"] as string,
 CpuManagerPolicy = agentsPoolKubeletConfigs["cpuManagerPolicy"] as string,
 ImageGcHighThreshold = agentsPoolKubeletConfigs["imageGcHighThreshold"] as int?,
 ImageGcLowThreshold = agentsPoolKubeletConfigs["imageGcLowThreshold"] as int?,
 PodMaxPid = agentsPoolKubeletConfigs["podMaxPid"] as int?,
 TopologyManagerPolicy = agentsPoolKubeletConfigs["topologyManagerPolicy"] as string
 };

 var defaultNodePool = new KubernetesClusterDefaultNodePool("defaultNodePool", new KubernetesClusterDefaultNodePoolArgs
 {
 Name = "default",
 VmSize = "Standard_DS2_v2",
 Zones = new[] { "1", "2", "3" },
 EnableAutoScaling = enableAutoScaling,
 MinCount = 1,
 MaxCount = 3,
 KubeletConfig = kubeletConfigArgs
 }, new CustomResourceOptions { Parent = resourceGroup });
 }
}

class Program
{
 static Task<int> Main() => Deployment.RunAsync<MyStack>();
}

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.azure.core.ResourceGroup;
import com.pulumi.azure.core.inputs.ResourceGroupArgs;
import com.pulumi.azure.containerservice.KubernetesClusterDefaultNodePool;
import com.pulumi.azure.containerservice.inputs.KubernetesClusterDefaultNodePoolArgs;
import com.pulumi.azure.containerservice.inputs.KubernetesClusterDefaultNodePoolKubeletConfigArgs;

import java.util.Map;
import java.util.HashMap;

public class MyStack {
 public static void main(String[] args) {
 Pulumi.run(ctx -> {
 boolean enableAutoScaling = false;

 Map<String, Object> agentsPoolKubeletConfigs = new HashMap<>();
 agentsPoolKubeletConfigs.put("allowedUnsafeSysctls", new String[]{"kernel.*", "net.*"});
 agentsPoolKubeletConfigs.put("containerLogMaxLine", 10);
 agentsPoolKubeletConfigs.put("containerLogMaxSizeMb", 10);
 agentsPoolKubeletConfigs.put("cpuCfsQuotaEnabled", true);
 agentsPoolKubeletConfigs.put("cpuCfsQuotaPeriod", "100ms");
 agentsPoolKubeletConfigs.put("cpuManagerPolicy", "none");
 agentsPoolKubeletConfigs.put("imageGcHighThreshold", 85);
 agentsPoolKubeletConfigs.put("imageGcLowThreshold", 65);
 agentsPoolKubeletConfigs.put("podMaxPid", 100);
 agentsPoolKubeletConfigs.put("topologyManagerPolicy", "none");

 var resourceGroup = new ResourceGroup("example", ResourceGroupArgs.builder()
 .name("storage-rg")
 .location("West Europe")
 .build());

 var kubeletConfigArgs = KubernetesClusterDefaultNodePoolKubeletConfigArgs.builder()
 .allowedUnsafeSysctls((String[]) agentsPoolKubeletConfigs.get("allowedUnsafeSysctls"))
 .containerLogMaxLine((Integer) agentsPoolKubeletConfigs.get("containerLogMaxLine"))
 .containerLogMaxSizeMb((Integer) agentsPoolKubeletConfigs.get("containerLogMaxSizeMb"))
 .cpuCfsQuotaEnabled((Boolean) agentsPoolKubeletConfigs.get("cpuCfsQuotaEnabled"))
 .cpuCfsQuotaPeriod((String) agentsPoolKubeletConfigs.get("cpuCfsQuotaPeriod"))
 .cpuManagerPolicy((String) agentsPoolKubeletConfigs.get("cpuManagerPolicy"))
 .imageGcHighThreshold((Integer) agentsPoolKubeletConfigs.get("imageGcHighThreshold"))
 .imageGcLowThreshold((Integer) agentsPoolKubeletConfigs.get("imageGcLowThreshold"))
 .podMaxPid((Integer) agentsPoolKubeletConfigs.get("podMaxPid"))
 .topologyManagerPolicy((String) agentsPoolKubeletConfigs.get("topologyManagerPolicy"))
 .build();

 var defaultNodePool = new KubernetesClusterDefaultNodePool("defaultNodePool", KubernetesClusterDefaultNodePoolArgs.builder()
 .name("default")
 .vmSize("Standard_DS2_v2")
 .zones(new String[]{"1", "2", "3"})
 .enableAutoScaling(enableAutoScaling)
 .minCount(1)
 .maxCount(3)
 .kubeletConfig(kubeletConfigArgs)
 .build(), new CustomResourceOptions().parent(resourceGroup));
 });
 }
}

In Pulumi, using TypeScript as an example, we can easily create new objects of a specific type and populate them with values from our configuration. If we need to add conditions, we can include our usual if statements. This approach greatly simplifies the code, making it easy to read and maintain for any future changes.

Execute Programming Code With `null_resource`, `local_file` and `provisioners`

There are sometimes situations where there is a strong need to execute a script written in a programming language. Most prominent use case is packaging up the source code for an AWS lambda function. Let’s have a look into an example from the Terraform AWS Lambda module

locals {
 python = (substr(pathexpand("~"), 0, 1) == "/") ? "python3" : "python.exe"
}

# Generates a filename for the zip archive based on the content of the files
# in source_path. The filename will change when the source code changes.
data "external" "archive_prepare" {
 count = var.create && var.create_package ? 1 : 0
 program = [local.python, "${path.module}/package.py", "prepare"]
 query = {
 paths = jsonencode({
 module = path.module
 root = path.root
 cwd = path.cwd
 })
 docker = var.build_in_docker ? jsonencode({
 docker_pip_cache = var.docker_pip_cache
 docker_build_root = var.docker_build_root
 docker_file = var.docker_file
 docker_image = var.docker_image
 with_ssh_agent = var.docker_with_ssh_agent
 docker_additional_options = var.docker_additional_options
 docker_entrypoint = var.docker_entrypoint
 }) : null
 artifacts_dir = var.artifacts_dir
 runtime = var.runtime
 source_path = jsonencode(var.source_path)
 hash_extra = var.hash_extra
 hash_extra_paths = jsonencode(
 [
 # Temporary fix when building from multiple locations
 # We should take into account content of package.py when counting hash
 # Related issue: https://github.com/terraform-aws-modules/terraform-aws-lambda/issues/63
 # "${path.module}/package.py"
 ]
 )
 recreate_missing_package = var.recreate_missing_package
 }
}

# This transitive resource used as a bridge between a state stored
# in a Terraform plan and a call of a build command on the apply stage
# to transfer a noticeable amount of data
resource "local_file" "archive_plan" {
 count = var.create && var.create_package ? 1 : 0
 content = data.external.archive_prepare[0].result.build_plan
 filename = data.external.archive_prepare[0].result.build_plan_filename
 directory_permission = "0755"
 file_permission = "0644"
}

# Build the zip archive whenever the filename changes.
resource "null_resource" "archive" {
 count = var.create && var.create_package ? 1 : 0
 triggers = {
 filename = data.external.archive_prepare[0].result.filename
 timestamp = var.trigger_on_package_timestamp ? data.external.archive_prepare[0].result.timestamp : null
 }
 provisioner "local-exec" {
 interpreter = [
 local.python, "${path.module}/package.py", "build",
 "--timestamp", data.external.archive_prepare[0].result.timestamp
 ]
 command = data.external.archive_prepare[0].result.build_plan_filename
 }
 depends_on = [local_file.archive_plan]
}

We see a confusing mix of different providers (local and null_resource) and even the use of provisioner here. This setup is overly complex to read and maintain, leading to a split-brain situation where some logic is in the code and the rest in HCL. It also becomes very challenging to debug, especially if the team lacks experience with the Python code.

In Pulumi, simplicity is key: simply write your Python code and seamlessly integrate it with Pulumi SDK calls. No additional steps are necessary, and there’s no need to run the Python binary separately to execute the file. It’s all about streamlining the process to make your life easier. For instance, consider this example where we read the name of an Azure Resource Group from a JSON file (config.json, illustrating how effortlessly Python can blend with Pulumi’s infrastructure management capabilities.

import json
import pulumi
import pulumi_azure_native as azure_native

# Load the JSON file
with open('config.json') as json_file:
 config = json.load(json_file)

# Extract values from the JSON file
resource_group_name = config['resourceGroupName']
location = config['location']

# Create an Azure Resource Group
resource_group = azure_native.resources.ResourceGroup(
 "resourceGroup",
 resource_group_name=resource_group_name,
 location=location
)

# Export the name of the resource group
pulumi.export('resourceGroupName', resource_group.name)

If Statements Constructed With `count`

variable "mode" {
 default = "not-spoke"
}

variable "restricted_net_hub_project_id" {
 default = "xxx"
}

data "google_compute_network" "vpc_restricted_net_hub" {
 count = var.mode == "spoke" ? 1 : 0
 name = "vpc-c-shared-restricted-hub"
 project = var.restricted_net_hub_project_id
}

In this HCL example, we see the count parameter being used in a way that stretches its original intent—essentially using it to conditionally create or skip resources based on the value of var.mode. This method, while functional, can make the code more difficult to understand and maintain, as it repurposes Terraform constructs in unintended ways.

Contrast this with Pulumi, which allows you to use native if statements from the programming language, such as TypeScript or Python. This native approach enhances readability and maintainability because it aligns with standard programming practices, making it easier for developers to follow and debug. Pulumi’s use of familiar programming constructs means you can write more intuitive and straightforward infrastructure code without bending the tool to fit unusual patterns.

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

const config = new pulumi.Config();
const mode = config.get("mode") || "";
const restrictedNetHubProjectId = config.get("restrictedNetHubProjectId") || "";
if (mode == "spoke") {
 const vpcSharedSpoke = gcp.compute.getNetworkOutput({
 name: "vpc-c-shared-spoke",
 project: restrictedNetHubProjectId,
 });
}

import pulumi
import pulumi_gcp as gcp

config = pulumi.Config()
mode = config.get("mode") or ""
restricted_net_hub_project_id = config.get("restrictedNetHubProjectId") or ""

if mode == "spoke":
 vpc_shared_spoke = gcp.compute.get_network_output(
 name="vpc-c-shared-spoke",
 project=restricted_net_hub_project_id,
 )

package main

import (
 "github.com/pulumi/pulumi-gcp/sdk/v7/go/gcp/compute"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 mode := ""

 if mode == "spoke" {
 _, err := compute.LookupNetwork(ctx, &compute.LookupNetworkArgs{
 Name: "vpc-c-shared-spoke",
 Project: "your-restrictedNetHubProjectId",
 })

 if err != nil {
 return err
 }
 }

 return nil
 })
}

using Pulumi;
using Pulumi.Gcp.Compute;

class Program
{
 static Task<int> Main() => Deployment.RunAsync<ComputeStack>();

 class ComputeStack : Stack
 {
 public ComputeStack()
 {
 var mode = "";

 if (mode == "spoke")
 {
 var network = Output.Tuple(Output.Create("vpc-c-shared-spoke"), Output.Create("your-restrictedNetHubProjectId"))
 .Apply(tuple => GetNetwork.InvokeAsync(new GetNetworkArgs
 {
 Name = tuple.Item1,
 Project = tuple.Item2
 }));

 // Handle potential errors if necessary
 network.Apply(n =>
 {
 if (n == null)
 {
 throw new Exception("Network not found");
 }
 });
 }
 }
 }
}

import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.gcp.compute.ComputeFunctions;
import com.pulumi.gcp.compute.inputs.GetNetworkArgs;

public class Main {
 public static void main(String[] args) {
 Pulumi.run(ctx -> {
 String mode = "";

 if ("spoke".equals(mode)) {
 Output<Tuple<String, String>> networkData = Output.tuple(Output.of("vpc-c-shared-spoke"), Output.of("your-restrictedNetHubProjectId"));

 Output<ComputedNetwork> network = networkData.applyValue(tuple ->
 ComputeFunctions.getNetwork(GetNetworkArgs.builder()
 .name(tuple.t1)
 .project(tuple.t2)
 .build()
 )
 );

 network.applyValue(n -> {
 if (n == null) {
 throw new RuntimeException("Network not found");
 }
 return null;
 });
 }
 });
 }
}

HCL Function Frenzy

Another source of extreme confusion and constant re-learning is the heavy usage of built-in functions within HCL expressions. At first glance, while it seems helpful for performing complex operations quickly, it can lead to overly complex expressions.

Here is an example of very complex expressions involving extensive function usage from the Google Network module:

locals {
 local_network_name = element(reverse(split("/", var.local_network)), 0)
 peer_network_name = element(reverse(split("/", var.peer_network)), 0)

 local_network_peering = "${var.prefix}-${local.local_network_name}-${local.peer_network_name}"
 local_network_peering_name = length(local.local_network_peering) < 63 ? local.local_network_peering :
 "${substr(local.local_network_peering, 0, min(58, length(local.local_network_peering)))}-${random_string.network_peering_suffix.result}"
 peer_network_peering = "${var.prefix}-${local.peer_network_name}-${local.local_network_name}"
 peer_network_peering_name = length(local.peer_network_peering) < 63 ? local.peer_network_peering :
 "${substr(local.peer_network_peering, 0, min(58, length(local.peer_network_peering)))}-${random_string.network_peering_suffix.result}"
}

In Pulumi, we can use the same functions from the programming language we pick for our infrastructure. This makes things easier because we don’t need to learn new, special functions like in HCL. We can also use extra libraries if we need them. Compared to HCL, where you have to learn unique functions, Pulumi lets you work with tools you might already know, making the whole process simpler and more flexible.

import * as pulumi from "@pulumi/pulumi";

const config = new pulumi.Config();
const localNetwork = config.require("localNetwork");
const peerNetwork = config.require("peerNetwork");
const prefix = config.require("prefix");

const localNetworkName = localNetwork.split("/").reverse()[0];
const peerNetworkName = peerNetwork.split("/").reverse()[0];

const localNetworkPeering = `${prefix}-${localNetworkName}-${peerNetworkName}`;
const peerNetworkPeering = `${prefix}-${peerNetworkName}-${localNetworkName}`;
const networkPeeringSuffix = "peering";
const localNetworkPeeringName = localNetworkPeering.length < 63
 ? localNetworkPeering
 : `${localNetworkPeering.substring(0, Math.min(58, localNetworkPeering.length))}-${networkPeeringSuffix}`;

const peerNetworkPeeringName = peerNetworkPeering.length < 63
 ? peerNetworkPeering
 : `${peerNetworkPeering.substring(0, Math.min(58, peerNetworkPeering.length))}-${networkPeeringSuffix}`;

export const outputs = {
 localNetworkName,
 peerNetworkName,
 localNetworkPeeringName,
 peerNetworkPeeringName,
};

import pulumi

config = pulumi.Config()
local_network = config.require("localNetwork")
peer_network = config.require("peerNetwork")
prefix = config.require("prefix")

local_network_name = local_network.split("/")[::-1][0]
peer_network_name = peer_network.split("/")[::-1][0]

local_network_peering = f"{prefix}-{local_network_name}-{peer_network_name}"
peer_network_peering = f"{prefix}-{peer_network_name}-{local_network_name}"
network_peering_suffix = "peering"

local_network_peering_name = (
 local_network_peering if len(local_network_peering) < 63
 else f"{local_network_peering[:min(58, len(local_network_peering))]}-{network_peering_suffix}"
)

peer_network_peering_name = (
 peer_network_peering if len(peer_network_peering) < 63
 else f"{peer_network_peering[:min(58, len(peer_network_peering))]}-{network_peering_suffix}"
)

pulumi.export("localNetworkName", local_network_name)
pulumi.export("peerNetworkName", peer_network_name)
pulumi.export("localNetworkPeeringName", local_network_peering_name)
pulumi.export("peerNetworkPeeringName", peer_network_peering_name)

package main

import (
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 localNetwork := "your-local-network"
 peerNetwork := "your-peer-network"
 prefix := "your-prefix"

 localNetworkName := reverseString(getNetworkName(localNetwork))
 peerNetworkName := reverseString(getNetworkName(peerNetwork))

 localNetworkPeering := prefix + "-" + localNetworkName + "-" + peerNetworkName
 peerNetworkPeering := prefix + "-" + peerNetworkName + "-" + localNetworkName
 networkPeeringSuffix := "peering"

 localNetworkPeeringName := condenseName(localNetworkPeering, networkPeeringSuffix)
 peerNetworkPeeringName := condenseName(peerNetworkPeering, networkPeeringSuffix)

 ctx.Export("localNetworkName", pulumi.String(localNetworkName))
 ctx.Export("peerNetworkName", pulumi.String(peerNetworkName))
 ctx.Export("localNetworkPeeringName", pulumi.String(localNetworkPeeringName))
 ctx.Export("peerNetworkPeeringName", pulumi.String(peerNetworkPeeringName))

 return nil
 })
}

func getNetworkName(network string) string {
 parts := strings.Split(network, "/")
 return parts[len(parts)-1]
}

func reverseString(s string) string {
 runes := []rune(s)
 for i, j := 0, len(runes)-1; i < j; i, j = i+1, j-1 {
 runes[i], runes[j] = runes[j], runes[i]
 }
 return string(runes)
}

func condenseName(name string, suffix string) string {
 if len(name) < 63 {
 return name
 }
 return name[:min(58, len(name))] + "-" + suffix
}

func min(a, b int) int {
 if a < b {
 return a
 }
 return b
}

using System;
using System.Linq;
using Pulumi;

class Program
{
 static Task<int> Main() => Deployment.RunAsync<MyStack>();

 class MyStack : Stack
 {
 public MyStack()
 {
 var localNetwork = "your-local-network";
 var peerNetwork = "your-peer-network";
 var prefix = "your-prefix";

 var localNetworkName = ReverseString(GetNetworkName(localNetwork));
 var peerNetworkName = ReverseString(GetNetworkName(peerNetwork));

 var localNetworkPeering = $"{prefix}-{localNetworkName}-{peerNetworkName}";
 var peerNetworkPeering = $"{prefix}-{peerNetworkName}-{localNetworkName}";
 var networkPeeringSuffix = "peering";

 var localNetworkPeeringName = CondenseName(localNetworkPeering, networkPeeringSuffix);
 var peerNetworkPeeringName = CondenseName(peerNetworkPeering, networkPeeringSuffix);

 this.Export("localNetworkName", localNetworkName);
 this.Export("peerNetworkName", peerNetworkName);
 this.Export("localNetworkPeeringName", localNetworkPeeringName);
 this.Export("peerNetworkPeeringName", peerNetworkPeeringName);
 }

 private string GetNetworkName(string network)
 {
 var parts = network.Split('/');
 return parts[parts.Length - 1];
 }

 private string ReverseString(string s)
 {
 var chars = s.ToCharArray();
 Array.Reverse(chars);
 return new string(chars);
 }

 private string CondenseName(string name, string suffix)
 {
 if (name.Length < 63) return name;
 return name.Substring(0, Math.Min(58, name.Length)) + "-" + suffix;
 }
 }
}

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.resources.Stack;

import java.util.Arrays;

public class Program {
 public static void main(String[] args) {
 Pulumi.run(Program::stack);
 }

 public static void stack(Context ctx) {
 var localNetwork = "your-local-network";
 var peerNetwork = "your-peer-network";
 var prefix = "your-prefix";

 var localNetworkName = reverseString(getNetworkName(localNetwork));
 var peerNetworkName = reverseString(getNetworkName(peerNetwork));

 var localNetworkPeering = String.format("%s-%s-%s", prefix, localNetworkName, peerNetworkName);
 var peerNetworkPeering = String.format("%s-%s-%s", prefix, peerNetworkName, localNetworkName);
 var networkPeeringSuffix = "peering";

 var localNetworkPeeringName = condenseName(localNetworkPeering, networkPeeringSuffix);
 var peerNetworkPeeringName = condenseName(peerNetworkPeering, networkPeeringSuffix);

 ctx.export("localNetworkName", Output.of(localNetworkName));
 ctx.export("peerNetworkName", Output.of(peerNetworkName));
 ctx.export("localNetworkPeeringName", Output.of(localNetworkPeeringName));
 ctx.export("peerNetworkPeeringName", Output.of(peerNetworkPeeringName));
 }

 private static String getNetworkName(String network) {
 var parts = network.split("/");
 return parts[parts.length - 1];
 }

 private static String reverseString(String s) {
 var chars = s.toCharArray();
 for (int i = 0, j = chars.length - 1; i < j; i++, j--) {
 char temp = chars[i];
 chars[i] = chars[j];
 chars[j] = temp;
 }
 return new String(chars);
 }

 private static String condenseName(String name, String suffix) {
 if (name.length() < 63) return name;
 return name.substring(0, Math.min(58, name.length())) + "-" + suffix;
 }
}

Pitfalls of HCL’s `try` Function

HCL’s try function can be problematic because it evaluates each argument until one does not produce an error. While it seems useful for navigating complex data structures, it often masks underlying issues by ignoring errors rather than addressing them.

This behavior can make debugging difficult because it can hide the real reasons behind failures, leaving configurations brittle and error-prone. Here is an example of the overuse of the try function

platform = try(each.value.platform, var.eks_managed_node_group_defaults.platform, "linux")
cluster_endpoint = try(time_sleep.this[0].triggers["cluster_endpoint"], "")
cluster_auth_base64 = try(time_sleep.this[0].triggers["cluster_certificate_authority_data"], "")
cluster_service_ipv4_cidr = var.cluster_service_ipv4_cidr
cluster_ip_family = var.cluster_ip_family
cluster_service_cidr = try(time_sleep.this[0].triggers["cluster_service_cidr"], "")
enable_bootstrap_user_data = try(each.value.enable_bootstrap_user_data, var.eks_managed_node_group_defaults.enable_bootstrap_user_data, false)
pre_bootstrap_user_data = try(each.value.pre_bootstrap_user_data, var.eks_managed_node_group_defaults.pre_bootstrap_user_data, "")
post_bootstrap_user_data = try(each.value.post_bootstrap_user_data, var.eks_managed_node_group_defaults.post_bootstrap_user_data, "")
bootstrap_extra_args = try(each.value.bootstrap_extra_args, var.eks_managed_node_group_defaults.bootstrap_extra_args, "")
user_data_template_path = try(each.value.user_data_template_path, var.eks_managed_node_group_defaults.user_data_template_path, "")
cloudinit_pre_nodeadm = try(each.value.cloudinit_pre_nodeadm, var.eks_managed_node_group_defaults.cloudinit_pre_nodeadm, [])
cloudinit_post_nodeadm = try(each.value.cloudinit_post_nodeadm, var.eks_managed_node_group_defaults.cloudinit_post_nodeadm, [])

Switching to Pulumi provides a clearer, more maintainable approach:

import * as pulumi from "@pulumi/pulumi";

// Example configuration using TypeScript for better error handling
const config = new pulumi.Config();
const platform = config.get("platform") || "linux"; // Default to "linux" if undefined

// Output using straightforward TypeScript logic
pulumi.output({ platform });

import pulumi

# Example configuration using Python
config = pulumi.Config()
platform = config.get("platform") or "linux" # Default to "linux" if undefined

# Output using straightforward Python logic
pulumi.export("platform", platform)

package main

import (
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 platform, ok := ctx.GetConfig("platform")
 if !ok {
 platform = "linux" // Default to "linux" if undefined
 }

 ctx.Export("platform", pulumi.String(platform))
 return nil
 })
}

using Pulumi;

class Program
{
 static Task<int> Main() => Pulumi.Deployment.RunAsync(() =>
 {
 var config = new Config();
 var platform = config.Get("platform") ?? "linux"; // Default to "linux" if undefined

 return new Dictionary<string, object?>
 {
 { "platform", platform }
 };
 });
}

import com.pulumi.Config;
import com.pulumi.Pulumi;

import java.util.HashMap;
import java.util.Map;

public class Main {
 public static void main(String[] args) {
 Pulumi.run(ctx -> {
 Config config = Config.auto();
 String platform = config.get("platform").orElse("linux"); // Default to "linux" if undefined

 Map<String, Object> result = new HashMap<>();
 result.put("platform", platform);

 ctx.export("result", result);
 });
 }
}

This Pulumi example shows how to handle configuration values in a more straightforward way, avoiding the need for complex try functions which potentially sweep errors under the rug. By using standard programming constructs, you can improve the readability and maintainability of your infrastructure code.

In Conclusion

In conclusion, Pulumi’s approach to infrastructure as code offers significant advantages over HCL, positioning it as a more developer-friendly and future-proof solution. By leveraging general-purpose programming languages, Pulumi addresses many of the limitations and complexities inherent in HCL.

Pulumi’s use of languages like TypeScript, Python, and Go brings several key benefits:

Larger ecosystem and community support, providing better tooling, debugging capabilities, and resources.
Familiar syntax and constructs for developers, reducing the learning curve and improving code readability.
Greater flexibility and expressiveness, allowing for more elegant solutions to complex infrastructure challenges.
Better integration with existing development workflows and practices.
Improved collaboration between development and operations teams, helping to break down silos.

The examples provided throughout this comparison demonstrate how Pulumi’s approach leads to cleaner, more intuitive code. Complex operations that require convoluted HCL expressions or multiple custom resources can often be accomplished with straightforward programming constructs in Pulumi.

Moreover, Pulumi’s model is inherently more adaptable to future changes in cloud technologies and programming paradigms. As new languages or cloud services emerge, Pulumi can easily incorporate them, ensuring that your infrastructure-as-code practices remain current and effective.

For organizations looking to modernize their infrastructure management, streamline their DevOps processes, and empower their development teams, Pulumi presents the only alternative to HCL. Its ability to unify application and infrastructure code under a single programming model offers a path to more efficient, maintainable, and scalable cloud engineering practices.

By choosing Pulumi, teams can enjoy the benefits of true programming languages for infrastructure, leading to more robust, flexible, and developer-friendly cloud deployments.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Interested in migrating? Our cloud experts can help. Learn more about Pulumi’s Migration Services.

Uploading documents to your AI Slackbot in real-time using S3, SQS and Pulumi on AWS

Tyler Mulligan — Mon, 03 Jun 2024 17:21:02 +0100

In the introductory blog post, we learned to Create an AI Slack Bot to Chat with Your Data Using Embedchain, Pulumi on AWS, and continued with Adding data to Pinecone using S3, Embedchain, and Pulumi on AWS for an AI Slack bot.

For reference, here’s what our architecture looked like at the end of the second blog post.

To follow along, clone the project, git clone https://github.com/catmeme/arti.git or view it on GitHub.

S3 Events and SQS

AWS provides a few ways to integrate with AWS S3 Events, for this project, we’re going to focus on the latest and most straight-forward, S3 Event Notifications. Depending on your use case, integrating with EventBridge may make more sense.

While we can integrate S3 events directly with Lambda, what happens when we add 100’s of files at the same time? Potentially, hundreds of Lambdas could be invoked simultaneously, which could become an issue depending on other factors. We’ll solve for this by Using Lambda with Amazon SQS, and reserved concurrency.

Our Pulumi program is updated to add a Queue and integrate it with our new Lambda SQS handler (policies left out for brevity, you can view them in the deploy/pulumi/__main__.py in the project).

# Add a queue
sqs_queue = aws.sqs.Queue(...)

# Add a docker image runtime for our Lambda SQS handler
docker_image_lambda_sqs_handler = docker.Image(...)

# Add Lambda SQS handler
app_sqs_lambda = aws.lambda_.Function(
 resource_name=f"{app_name}-sqs-lambda",
 environment=lambda_environment, # We've added the sqs_queue.url here as well
 image_uri=docker_image_lambda_sqs_handler.repo_digest,
 opts=pulumi.ResourceOptions(depends_on=[docker_image_lambda_sqs_handler]),
 **common_lambda_options,
)

# Add event source mapping
event_source_mapping = aws.lambda_.EventSourceMapping(
 f"{app_name}-lambda-sqs-event-source-mapping",
 event_source_arn=sqs_queue.arn,
 function_name=app_sqs_lambda.name,
 enabled=True,
 batch_size=10,
)

Add the application code responsible for processing S3 events coming from SQS.

def handler(event, context):
 for record in event.get("Records", []):
 s3_event = json.loads(record.get("body", "{}"))
 for s3_record in s3_event.get("Records", []):
 object_key = s3_record.get("s3", {}).get("object", {}).get("key")
 if object_key:
 logger.info("Processed %s in bucket %s.", object_key, app_bucket_name)
 load_data(asset_location=f"{app_bucket_name}/{object_key}")

 return {"statusCode": 200, "body": json.dumps("Successfully processed S3 event(s) from SQS messages.")}

We also have a scheduled event that we set up for ingestion in the last post, which we’re going to disable in favor of real-time events.

That’s pretty much it on the infrastructure side, here’s the updated diagram.

Slack to S3 file upload bridge

So what do we do with this new functionality? How do we expose it to the user? The most obvious approach is to reuse our existing Slack integration.

We’ll start by creating a Slack file upload bridge to S3, the bot watches for file_shared events on Slack and re-uploads files to our S3 bucket.

Using Slack Bolt, we add a new function handle_file_upload with the @app.event("file_shared") decorator, binding it to said event. When that event occurs, it’s processed, and the details of our file are retrieved, a new payload is constructed using requests, and the file gets uploaded to S3, this is our bridge.

@app.event("file_shared")
def handle_file_uploads(event, say, client, logger):
 file_id = event['file']['id']
 channel_id = event['channel_id']

 # Fetch file details using the Slack API
 file_info = client.files_info(token=slack_bot_token, file=file_id)['file']
 file_url = file_info['url_private_download']

 # Download file content from Slack
 headers = {'Authorization': f"Bearer {slack_bot_token}"}
 response = requests.get(file_url, headers=headers, stream=True)

 if response.status_code == 200:
 # Upload the file to S3
 s3_client.upload_fileobj(
 Fileobj=response.raw,
 Bucket=os.getenv("APP_BUCKET_NAME"),
 Key=file_info['name']
 )

 # Fetch recent messages to find the correct timestamp
 messages = client.conversations_history(channel=channel_id, limit=10)['messages']
 for msg in messages:
 if msg.get('files') and any(f['id'] == file_id for f in msg.get('files', [])):
 # Found the message with the correct file
 message_ts = msg['ts']
 client.reactions_add(channel=channel_id, name="white_check_mark", timestamp=message_ts)
 # Respond in a thread to the message containing the file
 say(text=f"File `{file_info['name']}` uploaded to S3 successfully! :tada:", thread_ts=message_ts)
 return

 logger.error("No valid message found for reaction.")

Keep in mind, we’re not doing any validation here yet. We’re blindly copying files from Slack to S3 and having our data loader process them. It’s probably not a good idea to deploy this bot to a public Slack channel without more moderation/validation in place.

Here’s what it looks like when we drag/drop a file into Slack now. On the right, a file was shared with the bot in private message. The bot acknowledged this upload with the checkmark reaction, the job is placed on the queue. Shortly after, on the left, the SQS Lambda handler has finished processing the uploaded file. We’re now able to use the bot to talk about the uploaded files.

Slack interactivity

In minutes, we were able to move from a scheduled task for processing files in a bucket, to real-time file processing integrated with Slack. We have our /arti command we can use to interrogate the documents we just uploaded, but we’re going to have a little more fun.

Thanks to Slack Block Kit Builder, we can build a form that exposes all of the configuration options for the query method in Embedchain’s app, providing the user with more options for interacting with the RAG model.

Our code to utilize the JSON we’ve crafted in the builder looks something like (we’ll call this /smarti):

@app.command("/smarti")
def open_smarti_modal(ack, body, client):
 ack()
 print("Opening SMARTI modal...")
 modal_file_path = Path(config.project_root) / 'config' / 'slack_modals' / 'chat.json'

 with open(modal_file_path, 'r') as json_file:
 modal_view = json.load(json_file)

 client.views_open(
 trigger_id=body["trigger_id"],
 view=modal_view
 )

And the code for handling that submission looks something like:

def handle_smarti_submission(client, logger, body, view):
 values = view["state"]["values"]
 question = values["question"]["input"]["value"]
 model = values["model"]["input"]["selected_option"]["value"]
 temperature = float(values["temperature"]["input"]["value"])
 max_tokens = int(values["max_tokens"]["input"]["value"])
 top_p = float(values["top_p"]["input"]["value"])
 prompt = values["prompt"]["input"]["value"]
 system_prompt = values["system_prompt"]["input"]["value"]
 citation_options = values["citations"]["input"].get("selected_options", [])
 citations = any(option['value'] == 'true' for option in citation_options)

 ai_response = ask_ai(
 question=question,
 model=model,
 temperature=temperature,
 max_tokens=max_tokens,
 top_p=top_p,
 prompt=prompt,
 system_prompt=system_prompt,
 citations=citations
 )
 ...

Submitting that form results in a response such as below. The bot honored our prompts, talking like Shakespeare, a thread is created, and citations were noted, showing where the data was retrieved from.

Lastly, if we want interact with the bot more publicly, then we can use the app_mention event.

@app.event("app_mention")
def handle_app_mention(event, say, client):
 text = event['text']
 bot_user_id = client.auth_test()["user_id"]

 # Strip the mention of the bot to isolate the command and arguments
 command_text = text.split(f'<@{bot_user_id}>')[1].strip() if f'<@{bot_user_id}>' in text else text

 # Parse the command and arguments
 parts = command_text.split(maxsplit=1)
 if not parts:
 say("I need a command to do something useful!")
 return

 # Get the command and the remainder of the message as arguments
 command = parts[0].lower()
 args = parts[1] if len(parts) > 1 else ""

 if command == "ask":
 if args:
 response = ask_ai(question=args)
 say(response)
 else:
 say("Please provide a question to ask.")
 elif command == "list":
 list_data_sources(say)
 else:
 say(f"Sorry, I don't recognize the command `{command}`.")

Resulting in an interaction such as

Of course, this same methodology can be used on any of the events exposed by Slack.

To learn more about extending this bot, see Slack Bolt’s documentation.

Conclusion

In this blog post, we took our existing Slack bot which has a data loading process that loads documents from an S3 bucket into a vector database on a schedule, and we wired it up to behave in real time.

To expose this functionality to the end user, we created a bridge between file uploads to Slack and S3, allowing drag/drop capabilities of files in conversational chat.

Furthering our interactions with our application, we leveraged Slack’s Block Kit to create a custom modal, and interact with events.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!

Adding data to Pinecone using S3, Embedchain and Pulumi on AWS for an AI Slack bot

Tyler Mulligan — Wed, 10 Apr 2024 17:00:00 +0100

In the introductory blog post, we learned to Create an AI Slack Bot to Chat with Your Data Using Embedchain, Pulumi on AWS. However, we made a few concessions in the application logic to illustrate the broader picture of what we were able to achieve combining these three technologies. Now that we have a solid foundation for deploying our Slack bot and querying our data, lets begin moving from proof-of-concept to production-ready, iteratively.

For reference, here’s what our architecture looked like at the end of the first iteration.

To follow along, clone the project, git clone https://github.com/catmeme/arti.git

Configuration

In our proof-of-concept, we defined our Embedchain config in our Python code. Best practices dictate strict separation of config from code, so the first thing we must address is getting the embedchain configuration out of our code.

A config/config.yaml made from our dict, is much easier to read, maintain and share.

app:
 config:
 id: arti
 collect_metrics: False
 log_level: INFO
llm:
 provider: openai
 config:
 model: 'gpt-3.5-turbo-1106'
 temperature: 0.5
 max_tokens: 1000
 top_p: 1
embedder:
 provider: openai
chunker:
 chunk_size: 2000
 chunk_overlap: 0
 length_function: len
vectordb:
 provider: chroma
 config:
 collection_name: arti-ai
 dir: /tmp/db
 allow_reset: True

Our goal is to replace our big inline dict, and read a YAML file instead, following this configuration specification.

def ask_ai(question):
 """Ask the AI a question."""
 print(f"Processing question: {question}")

 app_config = {
 # hardcoded configuration we're moving out to YAML
 }

 loader = DirectoryLoader(config={"recursive": True})
 app = App.from_config(config=app_config)
 app.add("assets", loader=loader)

 return app.query(question)

To do that, though, we don’t want to implement it directly as we already have a config module, src/arti_ai/config.py. Let’s define a new method to read that YAML file, and pass it to our application at startup.

class Config:
 def __init__(self, config_file="config/config.yaml", secrets_manager_client=None):
 self.secrets_manager_client = secrets_manager_client or boto3.client("secretsmanager")
 self.project_root = Path(__file__).resolve().parents[2]
 self.config_file = self.project_root / config_file
 load_dotenv()

 def get_embedchain_config(self):
 try:
 with open(self.config_file, 'r') as file:
 return yaml.safe_load(file)
 except FileNotFoundError as e:
 raise FileNotFoundError(f"Configuration file not found: {self.config_file}") from e
 except yaml.YAMLError as e:
 raise yaml.YAMLError(f"Error parsing YAML configuration: {self.config_file}") from e

Our app.py function simplifies, becoming more readable and flexible, we’ve even fixed a bug, making our application more portable with config.project_root, now arti can be run from any directory.

def ask_ai(question: str):
 """Ask the AI a question."""
 print(f"Processing question: {question}")

 from embedchain import App
 from embedchain.loaders.directory_loader import DirectoryLoader

 loader = DirectoryLoader(config={"recursive": True})
 app = App.from_config(config=config.get_embedchain_config()) # inline dict has become this
 app.add(f"{config.project_root}/assets", loader=loader)

 return app.query(question)

Now that we’ve separated out our configuration, we can have multiple scenarios preconfigured. In the repository, there are a few examples, including a private ai configuration, so that embedchain can be run 100% locally on your machine using gpt4all. The configuration difference boils down to updating the LLM and Embedder to open-source models.

llm:
 provider: gpt4all
 config:
 model: 'orca-mini-3b-gguf2-q4_0.gguf'
 max_tokens: 1000
 top_p: 1
embedder:
 provider: huggingface
 config:
 model: 'sentence-transformers/all-MiniLM-L6-v2'

Now that we have a dynamic approach to loading configuration from YAML, lets move on to improving the application architecture, which was not built to scale beyond relatively small data sets.

Application architecture

Knowing your limits

The current application architecture uses a single process with the expectation that a response is generated within 29 seconds, before the API Gateway timeout.

Slack also has an expectation, that we must acknowledge (or ack) a request within 3 seconds.

Currently, when the application receives a request, the API Gateway Lambda integration starts the application. The lifecycle of this single process approach is diagrammed below.

If we break this into two processes, Data Processing, and API Handling, we give our API handler more dedicated time within that 29 seconds. Simultaneously, we overcome another limitation, large amounts of data being loaded will take more than 29 seconds, by breaking out data processing into its own process, we can now have up to 15 minutes to load data using Lambda.

We’re going to use the same docker image for both Lambdas by adding an entrypoint script that can take an argument to select the handler, essentially deploying the same application twice as different Lambda Functions.

Our entrypoint.sh

#!/usr/bin/env sh
set -e

. /app/venv/bin/activate

if [ -n "$LAMBDA_HANDLER" ]; then
 /app/venv/bin/python -m awslambdaric $LAMBDA_HANDLER
else
 arti "$@"
fi

Here’s what the two process lifecycles look like separated

We have, however, introduced a new issue, and that is persistence. Our single process approach always loads the data, and warm requests will have the data cached, so it was okay that the Chroma database was ephemeral, it could always be recreated on the fly.

When our corpus is many gigabytes or terabytes, losing and recreating those embeddings for the vector database is a costly endeavor, both in time, and very likely monetarily.

How do we solve for that?

Multi-tier architecture and data persistence

By putting our vector database into another tier, we don’t couple its lifecycle with our application’s. It’s pretty cool that we can have an ephemeral vector database in Lambda for demonstration purposes, but it’s not an optimal configuration for production.

One option, is to deploy Chroma ourselves using a service such as EC2 or ECS, for example, but that incurs costs. For the ease of this demonstration, we’re going to use Pinecone, which Pulumi conveniently has a provider for. Though, there are many alternative options.

Navigate to Pinecone’s homepage, and click “Get started.” After confirming your account, login to Pinecone, and click “API Keys” in the sidebar. Create a new API key for Arti.

After configuring our Pinecone API key in AWS Secrets Manager, we are able to define our Pinecone index in our __main__.py Pulumi program.

import pulumi
import pinecone_pulumi as pinecone

pinecone_index = pinecone.PineconeIndex(
 f"{app_name}-pinecone-index",
 name="main-index",
 dimension=1536,
 metric=pinecone.IndexMetric.COSINE,
 spec=pinecone.PineconeSpecArgs(
 pod=pinecone.PineconePodSpecArgs(
 environment="gcp-starter",
 pod_type="starter",
 replicas=1,
 )
 ),
)

pulumi.export("output", {"value": pinecone_index.host})

Handling data load events

In the proof-of-concept, we are abusing docker to distribute our data sources, this isn’t optimal for a few reasons, the size of the docker image increases, which also increases build times. Do our documents really belong in the container, what about privacy, triggering builds, lead time to get new data added, etc.?

The obvious next place to put our documents is AWS S3 (Simple Storage Service). We don’t want to regress our existing implementation, which works locally as well as deployed to Lambda, so we abstract the concept of loading a path of assets, whether it’s a local path or S3. In the presence of an S3 bucket name, assume we intend to use that loader, otherwise use the directory loader.

def get_loader_and_asset_root():
 """Select and configures the appropriate loader based on AWS S3 bucket presence."""
 app_bucket_name = os.getenv("APP_BUCKET_NAME")
 root_path = os.getenv("ASSETS_ROOT_PATH", "assets")
 asset_location = root_path

 # pylint: disable=import-outside-toplevel
 if app_bucket_name:
 from arti_ai.s3_loader import S3BucketLoader # type: ignore

 loader = S3BucketLoader()
 asset_location = f"{app_bucket_name}/{root_path}"
 else:
 from embedchain.loaders.directory_loader import DirectoryLoader # type: ignore

 loader = DirectoryLoader(config={"recursive": True})

 # pylint: enable=import-outside-toplevel

 return loader, asset_location


def load_data():
 """Load data from data sources."""
 print("Loading data")

 # pylint: disable=import-outside-toplevel
 from embedchain import App

 # pylint: enable=import-outside-toplevel

 app = App.from_config(config=config.get_embedchain_config())

 loader, asset_location = get_loader_and_asset_root()
 response = app.add(asset_location, loader=loader)

 return response

With our data loading logic figured out, we’ll pass the APP_BUCKET_NAME in our Pulumi program to our Lambda Function, so it can retrieve the data.

app_bucket = aws.s3.Bucket(
 f"{app_name}-bucket",
 bucket=f"{app_name}-bucket",
 # ...
)

lambda_environment = pulumi.Output.all(
 app_bucket.id,
).apply(
 lambda args: {
 "variables": {
 # ...
 "APP_BUCKET_NAME": args[0],
 # ...
 }
 }
)

app_event_lambda = aws.lambda_.Function(
 f"{app_name}-lambda",
 architectures=["x86_64"],
 environment=lambda_environment,
 image_uri=docker_image_lambda_event_handler.repo_digest,
 # ...
)

If you’re new to Pulumi, you might be wondering what’s going on with the lambda_environment, why isn’t app_bucket.id referenced directly, and what is pulumi.Output.all? Simply put, this tells Pulumi to wait until the bucket is created before trying to use its name in a string we’re building. This is an important core feature of Pulumi that largely happens by nesting Outputs. To learn more, see Inputs & Outputs.

Triggering the data loader

When deployed, we’re going to need something that doesn’t require a local machine. One method for loading data is to use a scheduled task. AWS EventBridge makes that task simple, our Pulumi resource declarations are below. We define a rule/schedule (pulled from stack configuration), add our target (the event handler Lambda), and setup permissions to invoke the Lambda function.

event_rule_schedule = aws.cloudwatch.EventRule(
 f"{app_name}-event-rule-schedule",
 schedule_expression=data_load_event_rule_schedule,
 state=data_load_event_rule_state,
)

event_rule_target = aws.cloudwatch.EventTarget(
 f"{app_name}-event-target-schedule",
 rule=event_rule_schedule.name,
 arn=app_event_lambda.arn,
)

event_rule_target_permission = aws.lambda_.Permission(
 f"{app_name}-lambda-event-target-permissions-schedule",
 action="lambda:InvokeFunction",
 function=app_event_lambda.name,
 principal="events.amazonaws.com",
 source_arn=event_rule_schedule.arn,
)

Our event lambda handler looks something like

import json

from arti_ai.app import load_data


def handler(event, context):
 print("Received event: " + json.dumps(event, indent=2))
 print("Context AWS request ID: " + json.dumps(context, indent=2))

 load_data()

 return {"statusCode": 200, "body": json.dumps("Event processed successfully!")}

Locally, we can use our command-line interface (CLI), defined in __main__.py

if args.command == "ask":
 handle_ask(args.question)
elif args.command == "load":
 load_data()
elif args.command == "reset":
 reset_data()
else:
 parser.print_help()

Any of our YAML configuration files should work with the following commands:

arti ask "what can you tell me about fruits and vegetables?"
arti load
arti reset
arti -h

You could manage your live Pinecone data this way, whether you should is highly dependent on the use case.

Wrapping up

We reflected on the project that was presented in the original post, and reviewed short-comings with the proof-of-concept, working through improvements that touched upon architecture, application design, and infrastructure. Specifically, we have moved application configuration out of code into YAML, decoupled data-loading from querying, broken out our vector database to another tier, and integrated our application with AWS EventBridge to trigger scheduled tasks.

Our delivery on the concept was expedited by utilizing services that aligned with our improvements, and Pulumi helped us achieve our architecture changes using familiar patterns in just a few lines of code.

In the next post in this series, we’ll continue to make iterative improvements to both our applications and infrastructure, and start getting deeper into RAG.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!

Build an AI Slack Bot on AWS Using Embedchain & Pulumi

Tyler Mulligan — Mon, 18 Mar 2024 17:21:02 +0100

The integration of artificial intelligence (AI) to improve user experiences is gaining popularity in today’s world. One fascinating application of AI is the creation of chatbots, which can engage users in conversation and provide helpful information or services.

In this blog post, we’ll explore the process of building an AI-powered Slack bot using Embedchain, a Retrieval-Augmented Generation (RAG) framework powered by LangChain. Additionally, we’ll deploy our bot on AWS using Pulumi, a modern infrastructure as code (IaC) platform.

What are LLMs and RAG chatbots?

Large Language Models (LLMs) are sophisticated AI models trained on vast amounts of text data, enabling them to generate human-like text responses. Retrieval-Augmented Generated (RAG) is a cutting-edge approach to chatbot development that combines the capabilities of LLMs with information retrieval techniques. RAG chatbots can generate responses based on both the input query and relevant information retrieved from a knowledge base, resulting in more contextually relevant and informative interactions.

What’s an Embedchain?

Embedchain is a RAG framework powered by LangChain. It simplifies the process of creating RAG applications by providing high-level abstractions that reduce the complexity of AI technologies. With Embedchain, developers can focus on defining conversational flows and configuring the bot’s behavior using YAML configuration files, rather than dealing with low-level implementation details.

Embedchain supports configuration for various data sources, LLMs, vector databases, embedding models, and evaluation.

Creating a Slack bot

Start by cloning the project:

git clone https://github.com/catmeme/arti.git
cd arti

The files within the src/arti_ai directory can be summarized as:

The fourth file isn’t strictly necessary, but it illustrates a design pattern. The main application, app.py, is imported by slack_app but also __main__.py. This makes it easy to add additional user interfaces, such as APIs, or other chat services with minimal code.

The AI application

Let’s start with “the brain”, our Embedchain application. Below is a simple example (using explicit configuration for clarity). This code expects that OPENAI_API_KEY is populated in your environment. export OPENAI_API_KEY=<your key> to experiment with the code below.

from embedchain import App

app_config = {
 "app": {
 "config": {
 "collect_metrics": False,
 "log_level": "INFO",
 }
 },
 "llm": {
 "provider": "openai",
 "config": {
 "model": "gpt-3.5-turbo-1106",
 "temperature": 0.5,
 "max_tokens": 1000,
 "top_p": 1,
 },
 },
 "embedder": {"provider": "openai"},
 "chunker": {"chunk_size": 2000, "chunk_overlap": 0, "length_function": "len"},
 "vectordb": {
 "provider": "chroma", # we're writing to a vectordb in memory in lambda
 "config": {"collection_name": "arti-ai", "dir": "/tmp/db", "allow_reset": True},
 },
}

app = App.from_config(config=app_config)
app.add("https://www.pulumi.com/docs/", data_type="docs_site")
app.query("What is Pulumi?")

See how easy it was for us to configure our application, an LLM, an embedding model, chunker and vector database? Then we added a datasource, which is abstracting away more complex logic for us. The simple line app.add("https://www.pulumi.com/docs/", "docs_site") ends up crawling the website, chunking the documents, generating embeddings and inserting them into our vector database.

The app_config reference is the Embedchain configuration schema.

The Slack application

Now that our brain is built, we need a way for our users to interact with it. Slack Bolt is an official library for building Slack bots.

Follow their instructions to create a new Slack app.

OAuth & permissions

chat:write
channels:read
commands
im:read
im:write
users:read
users:write

Event subscriptions

message.channels
message.im

Code breakdown

The example below demonstrates two commands. knock knock is a basic ping/pong test that you expect to respond immediately. It’s not doing any complex logic, so that’s easy. What’s going on with this /arti command then? Slack requires your bot respond in 3 seconds or less. In our serverless architecture utilizing AWS API Gateway and Lambda, we’re operating on thin margins when we have cold starts. This enables us to respond quickly to Slack, but start our longer running process.

It is worth noting that in this architecture we’re still operating under 29 seconds for API Gateway’s response timeout. This can be overcome in a number of ways, but that’s out of scope for this demonstration.

from slack_bolt import App

from arti_ai.app import ask_ai, config

slack_bot_token, slack_bot_signing_secret = config.get_slack_credentials()
app = App(process_before_response=True, token=slack_bot_token, signing_secret=slack_bot_signing_secret)


def process_request(respond, body):
 """Process the request and respond to the user."""
 question = body["text"]
 response = ask_ai(question)
 respond(f"Q: _{question}_ A: {response}")


def respond_to_slack_within_3_seconds(ack):
 """Respond to the user within 3 seconds."""
 ack("Thinking...")


app.command("/arti")(ack=respond_to_slack_within_3_seconds, lazy=[process_request])


@app.message("knock knock")
def ask_who(_message, say):
 """Ask who's there."""
 say("_Who's there?_")

Configuration

The configuration module in this application normalizes retrieval of secrets from environment variables or AWS Secrets Manager. You may have noticed the Embedchain application configuration is hardcoded into the app.py. This isn’t considered best practice, and may be a fun first step in making this project yours. Otherwise, stay tuned for future posts in this series.

Deploying on AWS with Pulumi

Once our bot is ready, we’ll deploy it on AWS using Pulumi. Pulumi allows us to define our cloud infrastructure using familiar programming languages like TypeScript or Python, making it easy to manage and scale our deployment.

It’s important to note that in this selected architecture, we’re working within a few limitations. Lambda’s only writable directory is /tmp and it’s limited to 500MB, and we need to use a Docker container to overcome the 250MB filesize limitation of our application, which we’ll be bundling our data with.

The Pulumi program is broken up into the following sections:

Config: Get configuration from stack configuration, environment, and set up resource tagging
Networking: Set up VPC, VPC endpoints, security groups
S3: Bucket for later use
Application: Docker, Lambda, policies
API: API Gateway
Output: Stack outputs

Resources are contained within the VPC, policies and security groups exercise principle of least privilege, and logging is enabled. lambda_api_handler.py is responsible for handling API Gateway requests and imports the Slack app.

Prerequisites

Create the following keys in Secrets Manager. These names are configurable in deploy/pulumi/Pulumi.<stack>.yaml. Slack and Pinecone are only necessary if they are configured for use.

Secret Name	Schema
/catmeme/cloudplatform/sandbox/arti/access-token/openai	`{ "apiKey": "" }`
/catmeme/cloudplatform/sandbox/arti/access-token/slack	`{ "apiKey": "", "signingSecret": "" }`

Makefile

Makefiles offer a powerful, flexible, and efficient way to manage project tasks, contributing to smoother development processes, better team collaboration, and more reliable product builds. See the project README.md for more information.

cp -R /path/to/my/assets ./assets
cp sample.env .env
python3 -m venv venv
. venv/bin/activate
make develop

Ensure you have a matching AWS profile name to the one in the stack. The Makefile assumes <environment>-deployment. This script will attempt to copy the deploy/pulumi/Pulumi.catmeme.arti.sandbox.us-east-1.yaml as a base for configuration of the stack you deploy.

make deploy DEPLOY_ENVIRONMENT=dev

At this point, you should be able to use /arti to interact with your data via Slack.

Conclusion

In this blog post, we’ve explored the process of building an AI-powered Slack bot using Embedchain and deploying it on AWS with Pulumi. By harnessing the capabilities of LLMs and RAG chatbots, we can create intelligent conversational agents that enhance user interactions and provide valuable insights. With Embedchain’s high-level abstractions and Pulumi’s infrastructure as code platform, developers can streamline the development and deployment of AI-powered applications, unlocking new possibilities for innovation in the digital space.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!

The Pulumi 'Push to start' GitOps Experience

Kat Morgan — Tue, 23 Jan 2024 19:41:13 +0000

As a skeptic of “quick starts” myself, I approach most marketing promises with a measure of cautious excitement. If the great and powerful algorithm, friends, or a peer brought your attention here, then I invite you to take this one seriously.

Pulumi, with its full support of many general-purpose programming languages, can appear like a chore to get started with. The feeling can haunt seasoned developers as much as practitioners new to infrastructure code.

However, I’ll show you that finding the proverbial easy street is easier than you might believe. The pulumi new developer story just gets sweeter when combined with a few other nice-to-have conveniences.

This is a “bring your own GitHub login” event, so refresh that login. You will want to grab a Pulumi Cloud login as well. With credentials in hand, you are set with everything that you need to start. Let’s go!

Hint: all of the steps in this demo are ready to run yourself from the Pulumi Devcontainer using just your web browser and a Github account. Even better, they are real activities that Pulumians use every day to orchestrate the cloud too!

New technology learning advice

When it comes to ice cream, sometimes just selecting your flavor can be too much. Similarly, when choosing where to start with cloud tooling the onset of “choice paralysis” can be chronic. Pulumi’s powerful toolset is both a worthy endeavor to learn and a rewarding challenge to master. There is no better time to learn than the present!

“If you are short on time and have a bike, do you stop to hop on the bike, or choose to walk because there is no time to stop and get on?”

Freezing, while great for ice cream, is not quite as satisfying when building the next cloud solution and putting it off for next week or next year can be about as useful as walking fast and pushing a perfectly good bike because there is no time to stop and get on.

How can we overcome the learning curve? Picking up and mastering a new tool can be easy if we choose to start small.

Tech learning hacks:

🏆 Do: Start
🔨 Use: Helpful Tools
🥅 Make: Achievable Goals
🎉 Win: Celebrate Successes

Recognizing achievable goals can be tricky when you are learning something new. Always over-estimate the complexity of each step. If you overestimate how complicated a task might be, the biggest risk is accomplishing the goal faster than you think! Either way, the learning journey should be enjoyable and rewarding by following this rule.

Helpful tools are more abundant than ever. I recommend Pulumi as one of the most helpful tools but that is my bias showing through. Consider how intuitive, durable, and well-integrated any tool selection you make will be. Today we’re using multiple features and services of both the Pulumi and Pulumi Cloud offering, as well as demonstrating the benefits of a tightly integrated Git, Github, Devcontainers, and VSCode workflow to make light work of a traditionally considerably complicated achievement.

The resulting developer stack left me feeling like I would be successful automating anything. Be sure to share your experience with us in the Pulumi Community Slack or find us on Twitter/X @PulumiCorp too!

A DevOps journey

The best solutions are often buried in the details. The only way to reach them is to dig.

In true DevOps fashion, development for this blog post featured a roller coaster of chronic scope creep, pushing deadlines, discovering details that scrapped the original plan, and during the final finish line approach too! The worry and doubt was palpable.

Have we taken on too much? Did project ambitions exceed the achievable? The solution emerged while hacking on a side quest with a friend. A single vision came into focus, tying everything together into a simple “Push to Start” Pulumi developer workflow.

In this story, it took two tries to achieve the goal. Often it can take several tries. With that backstory let’s begin this next learning adventure together.

DevOps day 0

Let’s track our DevOps journey in typical “DevOps Day” style. Because we’re counting like computers, of course, we start with zero. In other words, we need a goal, and we will be starting from scratch.

Day zero is going to be your design, planning, and prep phase. Sometimes this includes various stages of research and other times it is a matter of just getting started with hands-on investigation.

Every journey includes a beginning and an end. To demonstrate the Pulumi Push to Start developer workflow, we need a project to develop! For this adventure let’s use the Pulumi Kubernetes Provider from the Pulumi Providers registry to deploy an app on Kubernetes. This provider is one of our most popular providers, as you might guess.

In this case, we can practice with a new project from the Community Engineering team. The GitHub pulumi/devcontainer repository is a GitHub template repository containing all the fundamentals of a new Pulumi infrastructure as code GitOps project. Let’s begin with a quick tour of the GitOps repo boilerplate.

“What belongs in a GitOps git repository?”

I have seen this question raised several times during the past year. Most recently it was brought up during a great customer conversation at AWS re:Invent.

Filtering through the conflicting opinions and advice about proper GitOps repo hygiene, a few fundamentals remain consistent.

Ops Environment
- Configuration
- Dependencies
- Artifacts
Documentation
- Who it is for
- What it does
- How to Operate
- Why it is important
Actions
- Testing: Continuous Integration
- Deployment: Continuous Deployment
- Safety: Version Controls

You may have assembled projects piece by piece before. In the pulumi/devcontainer repository, we provide a turnkey “known good” starter kit which will even improve over time. As a community resource, we welcome bugs and contributions!

And now for the feature presentation, let’s get to dev’ing!

What’s in a DevOps day 0?

Day 0 usually includes aspects of planning and preparing. For this exercise, we have just a few requirements to get started. Let’s knock these out and get to the fun stuff!

Prerequisites

Now, even the fanciest push-to-start car still requires a fob, so let’s cover those bases super quick.

Web browser
GitHub account
10 minutes (approx.)

Next, let’s review the tools we will depend on for developing and deploying our project.

Commandline Utilities:

*Dependencies supplied in the Pulumi devcontainer ghcr.io/pulumi/devcontainer

Tool	Description
A Programming Language	The code part of IaC
Pulumi CLI	Infrastructure as Code Tools
Pulumi ESC	Secrets and Configuration Store
Kubectl	Kubernetes CLI
Direnv	Automated Environment Variables
KinD	Local Kubernetes Infrastructure
Github CLI	Github & Git Repository Binaries
curl, jq, act, awk, etc	Other useful tools

Cloud-native … cloud developer environment?

Besides our command line tools and the programming language of your choice, we need an editor to write our code with. Many practitioners are fluent with Vim or Emacs; others turn to editors like VSCode, IntelliJ, and others. We are going to choose the GitHub Codespaces VSCode IDE for its deep integration with GitHub, and its convenience of requiring nothing from the user other than a web browser to use.

If you are already familiar with GitHub Codespaces, then you might know what to expect next. If not, you are in for a treat, and yes that is right, we are developing for the cloud, in the cloud, with the cloud.

Alright, let’s jump in and get started by launching a new GitHub Codespaces.

👇 Click this button to begin.👇

If you prefer a more informed approach you can launch the same codespaces by navigating to the Pulumi Devcontainer Repository or you can learn more about Codespaces at this Github Codespaces Link.

Clicking the ‘Open in GitHub Codespaces’ button should take you directly into a new Codespaces session.

Note: This demonstration is deeply integrated into the Web based Codespaces environment offered by Github. Alternatively, VSCode also supports running Devcontainers with Docker locally allowing you the benefit of Pulumi Devcontainer on your own hardware.

With a new codespaces session launched you should see the VSCode IDE in your browser. If you are not already familiar with VSCode, you can learn more about it in the VSCode Docs.

Congratulations! Your developer environment is ready for some Pulumi action.

For our final Day 0 step, we still need to decide what app to deploy with our Infrastructure-as-Code (IaC) program.

Focusing on something fun, let’s deploy Minecraft as the app for our Kubernetes exercise. It’s a fun way to get started with Pulumi, a great way to show off your cloud skills to friends, and maybe even share with your favorite young future technologists! After all, Open Source DevOps is for everyone!

DevOps day 1

We have an environment! We have a plan! Now we need to code … or do we?

Don’t just take my word for it. The big brains of Pulumi’s AI Engineering team have delivered a brilliant Pulumi AI. Your Pulumi IaC developer sidekick makes learning and developing Pulumi just that much easier.

Q: What can Pulumi AI do? A: Today’s AI is all experimental. As an industry, we are only just beginning to discover the possibilities, but you are invited to try it yourself with the Pulumi Neo WebUI or with the pulumi new cli as demonstrated below.

`~$ pulumi new --ai`

Pulumi Neo is available via WebUI much like the ChatGPT console. Besides the familiar browser-based interface, Pulumi AI is also built into the pulumi new CLI too! Now you can prompt Pulumi AI interactively from the convenience of the command line tools you will already be using, conveniently configured in your new Codespaces IDE.

Let’s try it! (Reminder: Perform all these steps from within the Codespaces IDE.)

# First Login to Pulumi Cloud for a few features we will use in upcoming GitOps content.
pulumi login

Step 2. Create a new program

# Let's create a new pulumi with the Pulumi CLI
pulumi new

Step 3. YOLO for fun and profit!

Let’s go all in on this one and try Pulumi AI. We can always enhance the prompt with additional dialog back and forth during the pulumi new chat if it doesn’t meet our expectations right away.

It is good to keep in mind that even as advanced as AI is today, the practice is still very much in its infancy. We will save more deep dives into the differences Traditional Ops (Imperative Ops), GitOps (Automated Declarative Ops), and AI Ops (Probability Augmented GitOps) in another post, for now let’s see how close the Pulumi AI can take us.

Woohoo! We selected the AI option on CLI, let’s follow the user prompts and feed the AI a simple prompt of what we want our Pulumi project to create.

Let’s try something like this:

Write a pulumi infrastructure as code program which creates the namespce 'minecraft' and using the pulumi_kubernetes.helm.v3.Release to deploy the https://itzg.github.io/minecraft-server-charts 'minecraft-server' helm chart, sets EULA to true, deploys onto Kubernetes in the minecraft namespace."

How did we do? Let’s see what the AI came up with.

It’s looking pretty good for a simple deployment. In DevOps, nothing works till the CI testing turns green. Since fortune favors the bold let’s proceed. For my developer workflow I ran kind create cluster in the Codespaces terminal. You can do the same or add your own kubeconfig to deploy on your choice of cluster.

Of course, proceed through the remaining prompts from the pulumi new interactive CLI.

I chose Python for this project, but you can choose any of the supported languages. We can see that pulumi new creates a new Python virtual environment for us programmatically. In addition to the boilerplate supporting our new project, Pulumi AI wrote the first iteration of our Python Minecraft Deployment IaC! The next steps should be printed on the screen. Assuming your Kubernetes cluster kubeconfig is ready to roll we just have one step left.

Hint, this is where we get the name for our annual “Pulumi Up conference.

Follow the link for the full playlist of 2023 Pulumi Up sessions

pulumi up -y

DevOps day 2

Finally, it is day two. This is the beginning of a choose-your-own-adventure wonderland and the opportunities go almost as far as your imagination. This is where we commit our code and plan our next change requests and enhancements.

We can commit this code to git. Build a Github Actions Workflow to automate the CI testing of our IaC with every git pr or commit. Once you have a production-ready IaC, you can add another to deploy to production clusters. If you run make act you will see how comfortable testing new GitHub actions workflows can be in the convenience of your new enhanced IDE workflow. You can see the dev container workflow running yourself right away by executing make act as a starting point.

To conclude, Pulumi is not a single tool. Pulumi is a complete cloud developer ecosystem. Tools, SDKs, and cloud services come together forming the most accelerated way to create your own cloud success legends.

Conclusion

Did it deploy?

Let’s check on the status of our deployment.

# Check for your new Minecraft pod
kubectl get po

With any luck, our Pulumi Neo gave you a working program right out of the gate. For references on Kubernetes Pulumi package resources to expand your infrastructure from here head to the Pulumi Registry Kubernetes Provider page for more.

Sharing Pulumi with people is a genuine passion found among Pulumians all around the globe but we all know evidence reigns king among engineers who, like any good practitioner, expect proof. Evolving your developer workflow with Pulumi gets even better than just easy, it’s remarkably powerful and ready for you to adopt in your daily workflows right away too.

Follow PulumiTV for upcoming examples built on this solution and to see how Pulumi ESC brings safety to your secrets and more.

What’s in the recipe?

Because no hero’s journey is complete without a gruesome account of what came before, I have included many of the manual steps required to reach this state one command at a time for the ~~morbidly~~ curious as well.

click ‘details’ to expand

Git Code Repository

Now create a git code repository to version control this project infrastructure code in Github.

# Authenticate with Github before proceeding.
gh auth login --web --git-protocol https --scopes "repo,gist,read:packages,admin:org,delete_repo,codespace"

# Create new Git Repository
gh repo create workshop --public \
 --gitignore Python --license apache-2.0 \
 --description "pulumi iac developer workflow workshop"

# Initialize the git repository
git clone --recurse-submodules https://github.com/${GITHUB_USER}/workshop .

# Configure `git` cli to use the `gh` cli for authentication with Github
gh auth setup-git

# Configure the username and email associated with your Github account.
git config --global user.email ${GIT_COMMITTER_EMAIL}
git config --global user.name ${GITHUB_USER}

# List files being tracked by git and their current status.
git status

Direnv: Automatic Environment Variables

Various non-secret environment variables may be worth maintaining in code locally as well. There are many ways to do this, but here we are going to use Direnv to automatically load environment variables from a .envrc file.

# Add environment variables useful during development to .envrc
cat <<EOF >> .envrc
export NO_COLOR=true
export PULUMI_HOME=\${HOME}/.pulumi
export PULUMI_SKIP_UPDATE_CHECK=true
export PULUMI_SKIP_CONFIRMATIONS=true
export KUBECONFIG=\${PWD}/.kube/config
source venv/bin/activate 2>/dev/null || true
EOF

Now you can enable direnv on this directory.

# Enable direnv in this directory.
direnv allow

Note:

Env variables shown are for educational purposes only and should be used with care. Read more about available Pulumi Environment variables here: https://www.pulumi.com/docs/cli/environment-variables

Pulumi Devcontainer

Now, to make sure we have all of our CLI dependencies, let’s grab the Pulumi Devcontainer.

# Add the Pulumi Devcontainer git submodule
git submodule add https://github.com/pulumi/devcontainer .devcontainer
git submodule update --init --recursive .devcontainer

# Copy the devcontainer config of your choice into the top level of your project
# so it will auto select by default when you open Codespaces in the future.
cp -f .devcontainer/devcontainer.json .devcontainer.json

# Rebuild the codespaces using the newly added Pulumi Devcontainer
gh codespace rebuild --codespace ${CODESPACE_NAME}

Pulumi Cloud

Let’s login to Pulumi Cloud and initialize a new ESC environment to store our environment variables, secrets, and configuration which should be preserved and distributed securely from your Pulumi Cloud account.

# Login to Pulumi Cloud
pulumi login

# Create a new ESC Environment (optionally use another already existing environment)
pulumi env init workshop

KinD: Kubernetes-in-docker

We are going to use Kubernetes to demonstrate our Pulumi IaC. Let’s go ahead and create a new cluster now.

# Create the directory for your new kind kubeconfig
mkdir .kube

# Create a Kind Cluster
kind create cluster

# Let's make sure our kubeconfig works and pods are starting up
kubectl get pods -A

# Load Kubeconfig into Pulumi ESC as an encrypted secret for safe keeping.
# This is an example use case for storing kubeconfig which would not usually be practical
# for disposable kind kubernetes clusters like this however the approach is practical in many
# other scenarios which you may come across.
pulumi env set workshop secrets.kubeconfig.kind --secret "$(jq . -R -s < $KUBECONFIG)"
pulumi env set workshop files.KUBECONFIG --plaintext \${secrets.kubeconfig.kind}

In our environment, we maintain our secrets including API personal access tokens, kubeconfigs, and such all with Pulumi’s Environments, Secrets, and Configuration cloud service.

# Load the newly updated Pulumi ESC Environment in the local shell
eval $(pulumi env open workshop --format shell)

Now let’s create the deployment code!

If fortune favors the bold, let’s be bold on this next step and let AI write our sample code on the fly!

As this is a “hello world” style demonstration intended to showcase the developer workflow more than any specific cloud technologies, let’s deploy something fun like a Minecraft server! Be sure to watch the video demonstration of this exercise on Pulumi’s PulumiTV YouTube channel for an extra easter egg bonus step!

# Write a new Pulumi Python IaC program to deploy Minecraft on Kubernetes
pulumi new \
 --ai "Write a program using pulumi kubernetes helm v3 Release to deploy the itzg/minecraft-server helm chart on Kubernetes, and include a pulumi output to show the helm release status." \
 --description "A pulumi infrastructure as code (iac) program for deploying and serving minecraft on kubernetes" \
 --name "minecraft-on-kubernetes" \
 --language python \
 --stack "workshop" \
 --force \
 --dir .

Now let’s see if our new Infrastructure as Code Pulumi Python Codebase will deploy!

# Deploy your new Pulumi IaC on Kubernetes
pulumi stack select workshop
pulumi up

Cleanup

pulumi destroy -y --skip-preview
pulumi stack rm workshop
kind delete cluster --name kind
gh repo delete ${GITHUB_USER}/workshop

Pulumi CLI now displays time elapsed per resource

Kyle Dixler — Thu, 03 Nov 2022 11:19:51 -0600

If you’ve deployed resources to your favorite cloud provider, you have probably found yourself sitting in the console thinking: “I don’t know how long this is going to take.” Then you deploy the resource and think: “When did I even start this?” When using Pulumi, the CLI prints out how long the update took after it ran, but while you’re in the moment, it feels like ages.

We’re excited to announce a CLI usability enhancement

You can now see how long each of your resources are taking to deploy.

When you run pulumi up, you’ll see that individual resources have the time taken displayed during updates including up, destroy, import, and refresh.

Comparing resource deployment times

I have a contrived Pulumi program that deploys comparable resources in AWS and GCP and you can see the differences in deployment times between them. It provides a rough idea of the orders of magnitude in deployment time between different resource classes.

Here’s a screenrecording of the output of the program (it took fairly long):

For those interested, the code is located here.

This is far from a perfect example. These numbers have many sources of error, including the load on the cloud provider, the chosen availability zone, quotas and throttling, whether the resource is ready for requests, inefficient resource options, and size of the images (for compute resources). Nonetheless, it provides more perspective as instead of a feeling short or long, we now see concrete numbers to help us quantify it.

Thanks for reading. If you have any feedback on how we can further improve our CLI experience, let us know by commenting or upvoting open issues tracking enhancements on GitHub. If you can’t find an existing issue, feel free to open a new one!

Snowflake Provider Launch

Kat Cosgrove — Wed, 15 Sep 2021 17:16:09 -0500

Snowflake support is here! Pulumi’s new Snowflake Provider gives you the ability to easily set up cloud storage and manage your connections to Snowflake, right alongside the rest of your code.

Pulumi is bringing management of your Snowflake resources into the same place as the rest of your infrastructure! Whether you’re already a big fan or you were just exploring the possibility of using Snowflake, Pulumi has you. For example, you can:

Set up storage in your cloud provider and add it to Snowflake as an external stage
Add storage and connect it to Snowpipe
Create a service user and push the key into the secrets manager of your choice
Rotate keys

You can check out the details in the official Snowflake Provider documentation , with references for Python, Go, TypeScript, and C#!