Pulumi Blog: Csi

Why Every Cloud Engineer Needs Pulumi ESC for Secrets Management

Sara Huddleston — Mon, 17 Mar 2025 07:28:56 +0000

Managing secrets is one of the most critical responsibilities in cloud engineering. Secrets like API keys, database credentials, and encryption tokens are the backbone of secure and seamless cloud operations. Yet they are so often an afterthought. They get replicated across cloud-specific secrets managers and stuffed in GitHub secrets, compromising security for the sake of simplicity. ¿Por que no los dos? Why can’t secrets management be secure and simple?

Enter Pulumi ESC (Environments, Secrets, and Configuration)—a breakthrough in taming secrets sprawl and streamlining configuration management across infrastructure. Let’s explore why Pulumi ESC is a necessity for cloud engineers, helping make secrets management secure while keeping it simple.

The Challenge of Secrets Management in Modern Cloud Environments

Secrets management has changed significantly over the past decade. Gone are the days when secrets could be hardcoded, manually maintained in static configuration files, or left floating around in plaintext on dev machines. Nowadays, cloud environments demand a higher level of sophistication:

Distributed Systems: Even in multi-cloud and hybrid setups, secrets must be accessible across varied platforms without exposing vulnerabilities.
Dynamic Infrastructure: Kubernetes, serverless architectures, and ephemeral environments must have secrets that adapt dynamically.
Security Risks: Hardcoded or poorly managed secrets can lead to catastrophic breaches, costing companies millions in data recovery and compliance penalties.
Operational Burdens: Manual secrets management is error-prone, inefficient, unscalable, and needs to align with DevOps and DevSecOps best practices.

Pulumi ESC addresses these issues head-on, redefining how cloud engineers manage secrets efficiently, securely, and at scale.

Jk Jensen, Team Lead at Mysten Labs, said:

“Pulumi ESC has been a lifesaver for us. It’s nice to throw everything behind an ESC environment and eliminate one-off granting/IAM permissions and other issues related to static credentials. It gives us peace of mind knowing that we can grant permissions quickly and revoke easily limiting blast radius for any access.”

What is Pulumi ESC?

Pulumi ESC simplifies how organizations manage secrets and configurations. It is designed to secure sensitive configurations across modern cloud environments and supports seamless integration, enabling engineers to:

Access, share, and manage secrets, passwords, API keys, and configuration settings like network and deployment options.
Synchronize secrets and configuration from any secrets manager to any app, tool, or CI/CD platform.
Access secrets securely through CLI, API, Kubernetes operator, the Pulumi Cloud UI, and in-code with Typescript/Javascript, Python, and Go SDKs.
Connect to cloud providers and secrets stores via OIDC to generate dynamic, short-lived, auto-expiring credentials.
Set role-based access controls (RBAC), making securing secrets and configurations easy by assigning permissions to users based on their roles.

Whether integrated with Pulumi’s Infrastructure as Code (IaC) platform or used as a standalone service, Pulumi ESC enables cloud engineers to streamline secrets management with centralized control.

Key Features of Pulumi ESC

1. Seamless Integration with External Platforms

Pulumi ESC integrates with popular secrets providers, including AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, 1Password, and HashiCorp Vault, making it adaptable for multi-cloud and hybrid cloud architectures.

2. Dynamic Secrets Synchronization with ESO

Partnering with the External Secrets Operator, Pulumi ESC synchronizes secrets securely into Kubernetes clusters. This eliminates hardcoding secrets into manifests or relying on unsecured manual processes.

3. Safely Roll Back to a Previous Version

Pulumi ESC Versioning gives you unprecedented control over your secrets and configuration. Every change is captured in an immutable revision history, allowing you to audit modifications, compare versions, and safely roll back.

4. Secure by Design

Pulumi ESC follows a “secure by default” model, employing encryption, access control, and detailed audit trails. Engineers can meet compliance regulations effortlessly while gaining full visibility into secret access patterns.

5. Automated Rotation and Expiry

Pulumi ESC minimizes security risks by automating the rotation of secrets. This feature aligns secrets management with CI/CD processes for cloud engineers focused on DevOps, ensuring credentials remain valid only when needed.

6. Configuration-as-Code, Automation, and Integration Everywhere

Pulumi ESC embraces an “as-code” approach, enabling configuration and secrets management using TypeScript, JavaScript, Go, Python, or YAML. The ’esc’ CLI and API support automation in CI/CD environments, reducing credential duplication and ensuring a single source of truth.

7. Dev Tools Integrations

Pulumi ESC’s metadata and support for popular configuration formats enable seamless integration with tools like Direnv, Docker, and GitHub, allowing easy management of environment variables, secrets, and configurations.

8. Infrastructure Tools Integrations

Pulumi ESC extends its capabilities beyond Pulumi IaC by integrating with other infrastructure tools such as Cloudflare, Terraform, and OpenTofu. These integrations enable seamless provisioning of cloud credentials and input variables directly from ESC environments.

Why Cloud Engineers Need Pulumi ESC

Enhanced Security and Compliance

Cloud engineers often have to balance security and operations. Pulumi ESC centralizes secrets in a secure vault, ensuring encrypted storage, access control, and audit visibility. It also complies with industry standards like SOC 2, PCI-DSS, and MTCS.

Streamlined Multi-Cloud Management

Managing secrets across AWS, Azure, and GCP can quickly become chaotic. Pulumi ESC’s cross-platform integration simplifies this process, ensuring engineers can manage and sync secrets from a single interface.

Simplified Kubernetes Secrets

Kubernetes’ default secrets offer limited security and scalability. Pulumi ESC with ESO and Secrets Store CSI Driver overcomes these limitations by securely synchronizing and managing secrets within Kubernetes clusters. This prevents unauthorized access and provides automated updates.

Zero Downtime Through Automation

Manual secrets management often leads to errors such as expired credentials or outdated tokens. Pulumi ESC automates the entire lifecycle of secrets—creation, rotation, replication, and expiry—guaranteeing uninterrupted services.

Developer-Friendly Workflows

Tools should make engineers’ lives easier, not harder. Pulumi ESC’s CLI, SDKs, and API provide intuitive ways to integrate into existing workflows. For cloud engineers leveraging Infrastructure as Code with Pulumi, managing secrets alongside the stack becomes effortless.

Thomas Meckel, Senior Cloud Architect at Ophios GmbH, explains:

“Pulumi’s built-in encryption of secrets sets it apart as a leader in secure infrastructure management. Unlike other tools that leave sensitive data exposed or require complex configurations, Pulumi ensures secrets are encrypted by default, leveraging flexible options like Azure Key Vault or AWS KMS. This eliminates the risk of plaintext exposure and simplifies secure deployments. For any organization serious about cloud security, Pulumi’s approach to secrets management is a critical differentiator, combining ease of use with robust protection against breaches.”

Kubernetes Secrets Management

For secrets management in Kubernetes environments, Pulumi ESC becomes even more powerful when paired with External Secrets Operator (ESO) or the Secrets Store CSI Driver. Here’s how it works:

Using Pulumi ESC with External Secrets Operator

External Secrets Operator is an open-source Kubernetes operator that syncs secrets from external providers (like Pulumi ESC) into Kubernetes as native secrets.

Centralized Storage: Store secrets securely in Pulumi ESC and synchronize them with Kubernetes across multiple clusters.
Dynamic Updates: Whenever a secret is updated in ESC, ESO automatically replicates the changes into Kubernetes, eliminating manual intervention.
Granular Management: Define which secrets are synchronized and to which namespaces, ensuring tight access controls and minimizing risk.
Automated Secret Rotation: By leveraging ESC’s rotation capabilities, ESO ensures Kubernetes receives refreshed credentials without requiring downtime.

Learn how to manage Kubernetes secrets with Pulumi ESC and External Secrets Operator: The Perfect Solution for Today’s Cloud-Native Secret Management.

Using Pulumi ESC with Secrets Store CSI Driver

The Secret Store CSI Driver mounts secrets, certificates, and keys from external stores into Kubernetes pods as volumes.

Automated Secrets Injection: Secrets Store CSI Driver enables Pulumi ESC to automatically inject secrets into Kubernetes pods as mounted volumes or environment variables. This reduces the manual overhead of managing secrets directly in Kubernetes.
Secure and Dynamic Secret Access: By leveraging Pulumi ESC, Kubernetes applications can securely fetch secrets from external providers, ensuring dynamic access without exposing credentials in pod specifications.
Streamlined Operations: This integration simplifies the process of synchronizing secrets from Pulumi ESC with Kubernetes-native constructs. The automation reduces errors and boosts efficiency.

Learn how to Master Kubernetes Secrets with Pulumi ESC + Secrets Store CSI Driver.

The Future of Secrets Management with Pulumi ESC

Pulumi ESC is more than just a secrets management tool — it’s the foundation of secure, scalable, and agile cloud operations. For cloud engineers grappling with multi-cloud complexity, Kubernetes deployments, and compliance concerns, it provides a streamlined, automated approach that enhances efficiency and security.

With rising cyber threats and stricter data regulations, adopting tools like Pulumi ESC is no longer optional. It’s a competitive advantage for cloud engineers and organizations aiming to lead in today’s cloud-native world.

Next Steps to Secure Your Secrets

Secrets management doesn’t have to be a headache! With Pulumi ESC, you can safeguard your infrastructure, ensure compliance, and eliminate manual errors—all while enhancing security and agility.

Get started with Pulumi ESC today and experience the future of secrets management. Take control of your cloud environment—securely and effortlessly.

Master Kubernetes Secrets with Pulumi ESC + Secrets Store CSI Driver

Engin Diri — Wed, 18 Dec 2024 09:58:01 +0100

Welcome to the second blog post of the Pulumi ESC and Kubernetes secrets series. If you haven’t had the chance to read the first blog post, go ahead and read it here.

In the previous blog post, we have learned how to manage secrets with Pulumi ESC and the External Secrets Operator. While the External Secrets Operator is a great tool to manage secrets in a cloud-native way, it still creates Kubernetes secrets in the cluster. Depending on your security requirements, you might want to avoid the use of Kubernetes secrets in your cluster at all. This is the point where you hit the limits of the External Secrets Operator.

But don’t worry, we also have a solution for this problem. In this blog post, we will introduce the Secrets Store CSI Driver.

TL;DR?

Jump straight to the comparison of the External Secrets Operator and the Secrets Store CSI Driver here.

Refresher: Why Should You Avoid Using Kubernetes Secrets?

Before we dive into the Secrets Store CSI Driver, let’s quickly recap why you might want to avoid using native Kubernetes secrets. I wrote in depth about this topic in the previous blog post, but here is a quick summary:

By default, etcd stores them in a base64-encoded format, which is not secure.
Developers create them either manually by using kubectl commands or by using a manifest file, which makes them hard to manage at scale.
Hard to manage and synchronise secrets across different environments and clusters.
There is no default way to rotate secrets automatically.

Point 1 is the most critical one, as there is even a warning in the Kubernetes documentation:

Kubernetes Secrets. Credit: kubernetes.io/docs/concepts/configuration/secret/

What is the Secrets Store CSI Driver?

The first thing you may notice is that in the name of the Secret Store CSI Driver, there is a reference to the Container Storage Interface (CSI) standard.

What is CSI and why it is used in the Secret Store CSI Driver*?*

How to write a Container Storage Interface (CSI) plugin. Credit: arslan.io/2018/06/21/how-to-write-a-container-storage-interface-csi-plugin/

Container Storage Interface (CSI)

The Container Storage Interface (CSI) is a standard to unify the interface between container orchestrators (like Kubernetes) and different storage vendors (like NetApp, Ceph, etc.). This helps to guarantee that implementing a CSI for a storage vendor is going to work with all orchestrators that support CSI.

Before CSI, we needed to create volume plugins for every orchestrator and storage vendor. This meant that developers coupled volume plugin development with the Kubernetes version and made it dependent on it. Bugs in volume plugins could break the Kubernetes components, instead of the volume plugin. Besides, volume plugins had full privileges on the Kubernetes components like the kubelet.

Combining CSI, Kubernetes and Secrets = Secret Store CSI Driver

The Secret Store CSI Driver is a CSI driver that allows you to mount many secrets, certificates, and keys from external secret stores into Kubernetes pods as volumes. After attaching the volume, the system mounts the secrets into the container file system.

The benefits of using the Secret Store CSI Driver are that you manage the lifecycle of the secrets outside of Kubernetes while still providing a Kubernetes-native experience of using the secrets in your pods.

The Architecture of the Secret Store CSI Driver

The Secret Store CSI Driver is a daemonset that lets all kublets communicate with each other and uses gRPC to talk to a provider. A SecretProviderClass custom resource specifies the definition of the external Secret Store. Then the system mounts a volume in the pod as tmpfs, and it injects the secret into the volume. When you delete the pod, the system cleans up the volume and removes the secret from the tmpfs volume.

How to Use the Secrets Store CSI Driver with Pulumi ESC

Before we start, make sure you have the following prerequisites:

A Pulumi Cloud account. If you don’t have one, you can create one for free.
A Kubernetes cluster (I will be using a local KinD cluster, but you can use any Kubernetes cluster)
Pulumi CLI installed. You can use the pulumi env commands to manage secrets and configurations.
kubectl CLI installed for some debugging

Step 1: Deploy the Secrets Store CSI Driver and Pulumi ESC CSI Provider

We will use Pulumi with the pulumi-kubernetes provider to deploy the Secrets Store CSI Driver and the Pulumi ESC CSI Provider to our Kubernetes cluster.

And here we also use the integration of Pulumi ESC with Pulumi IaC to supply the Pulumi access token in a secure way.

First, we need to create a new Pulumi project:

# Choose your favorite Pulumi supported language
pulumi new kubernetes-<your-programming-language> --name

Before we dig into the code, let’s head to the Pulumi Cloud Console and create a new Pulumi ESC project with the name esc-secrets-store-csi-driver-demo.

And create the environment dev:

In the editor add the following yaml into the Environment definition:

values:
 pulumiConfig:
 pulumi-pat:
 fn::secret: <your-pulumi-pat>

Don’t know how to create a PAT? Check out the official Pulumi documentation.

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/esc-secrets-store-csi-driver-demo/dev

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/esc-secrets-store-csi-driver-demo/dev

Now, we need to link the Pulumi ESC project to the Pulumi IaC project. To do this, we need to add the following to your Pulumi.dev.yaml:

environment:
- esc-secrets-store-csi-driver-demo/dev

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

const secretsStoreCSIDriver = new k8s.helm.v4.Chart("secrets-store-csi-driver", {
 chart: "secrets-store-csi-driver",
 namespace: "kube-system",
 repositoryOpts: {
 repo: "https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts",
 },
 values: {
 nodeSelector: {
 "kubernetes.io/os": "linux",
 },
 },
});

const secretsStoreCSIPulumiESCProvider = new k8s.helm.v4.Chart(
 "secrets-store-csi-pulumi-esc-provider",
 {
 chart: "oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider",
 namespace: "kube-system",
 values: {
 nodeSelector: {
 "kubernetes.io/os": "linux",
 },
 },
 },
 { dependsOn: secretsStoreCSIDriver },
);

const config = new pulumi.Config();

const mySecret = new k8s.core.v1.Secret(
 "my-secret",
 {
 metadata: {
 namespace: "default",
 name: "pulumi-access-token",
 },
 stringData: {
 "pulumi-access-token": config.require("pulumi-pat"),
 },
 type: "Opaque",
 },
 { dependsOn: secretsStoreCSIPulumiESCProvider },
);

import pulumi
import pulumi_kubernetes as k8s

secrets_store_csi_driver = k8s.helm.v4.Chart(
 "secrets-store-csi-driver",
 k8s.helm.v4.ChartArgs(
 chart="secrets-store-csi-driver",
 namespace="kube-system",
 repository_opts=k8s.helm.v4.RepositoryOptsArgs(
 repo="https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts",
 ),
 values={
 "nodeSelector": {
 "kubernetes.io/os": "linux",
 },
 },
 ),
)

secrets_store_csi_pulumi_esc_provider = k8s.helm.v4.Chart(
 "secrets-store-csi-pulumi-esc-provider",
 k8s.helm.v4.ChartArgs(
 chart="oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider",
 namespace="kube-system",
 values={
 "nodeSelector": {
 "kubernetes.io/os": "linux",
 },
 },
 ),
 opts=pulumi.ResourceOptions(depends_on=[secrets_store_csi_driver]),
)

config = pulumi.Config()

my_secret = k8s.core.v1.Secret(
 "my-secret",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 namespace="default", name="pulumi-access-token"
 ),
 string_data={
 "pulumi-access-token": config.require("pulumi-pat"),
 },
 type="Opaque",
 opts=pulumi.ResourceOptions(depends_on=[secrets_store_csi_pulumi_esc_provider]),
)

package main

import (
 k8s "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes"
 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apiextensions"
 appv1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/apps/v1"
 corev1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/core/v1"
 chartv4 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/helm/v4"
 metav1 "github.com/pulumi/pulumi-kubernetes/sdk/v4/go/kubernetes/meta/v1"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi/config"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 secretsStoreCsiDriver, err := chartv4.NewChart(ctx, "secrets-store-csi-driver", &chartv4.ChartArgs{
 Chart: pulumi.String("secrets-store-csi-driver"),
 Namespace: pulumi.String("kube-system"),
 RepositoryOpts: chartv4.RepositoryOptsArgs{
 Repo: pulumi.String("https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts"),
 },
 Values: pulumi.Map{
 "nodeSelector": pulumi.Map{
 "kubernetes.io/os": pulumi.String("linux"),
 },
 },
 })
 if err != nil {
 return err
 }

 secretsStoreCsiPulumiEscProvider, err := chartv4.NewChart(ctx, "secrets-store-csi-pulumi-esc-provider", &chartv4.ChartArgs{
 Chart: pulumi.String("oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider"),
 Namespace: pulumi.String("kube-system"),
 Values: pulumi.Map{
 "nodeSelector": pulumi.Map{
 "kubernetes.io/os": pulumi.String("linux"),
 },
 },
 }, pulumi.DependsOn([]pulumi.Resource{secretsStoreCsiDriver}))
 if err != nil {
 return err
 }

 pulumiPAT := config.Get(ctx, "pulumi-pat")

 mySecret, err := corev1.NewSecret(ctx, "my-secret", &corev1.SecretArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Namespace: pulumi.String("default"),
 Name: pulumi.String("pulumi-access-token"),
 },
 StringData: pulumi.StringMap{
 "pulumi-access-token": pulumi.String(pulumiPAT),
 },
 Type: pulumi.String("Opaque"),
 }, pulumi.DependsOn([]pulumi.Resource{secretsStoreCsiPulumiEscProvider}))
 if err != nil {
 return err
 }


 return nil
 })
}

using Pulumi;
using Pulumi.Kubernetes.Core.V1;
using Pulumi.Kubernetes.Types.Inputs.Core.V1;
using Pulumi.Kubernetes.Helm.V3;
using Pulumi.Kubernetes.Helm;
using Pulumi.Kubernetes.Types.Inputs.Meta.V1;
using Pulumi.Kubernetes.ApiExtensions;
using System.Collections.Generic;



return await Deployment.RunAsync(() =>
{
 var secretsStoreCsiDriver = new Release("secrets-store-csi-driver", new()
 {
 Chart = "secrets-store-csi-driver",
 Namespace = "kube-system",
 RepositoryOpts = new Pulumi.Kubernetes.Types.Inputs.Helm.V3.RepositoryOptsArgs
 {
 Repo = "https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts",
 },
 Values = new Dictionary<string, object>
 {
 { "nodeSelector", new Dictionary<string, object>
 {
 { "kubernetes.io/os", "linux" },
 } },
 },
 });

 var secretsStoreCsiPulumiEscProvider = new Release("secrets-store-csi-pulumi-esc-provider", new()
 {
 Chart = "oci://ghcr.io/pulumi/helm-charts/pulumi-esc-csi-provider",
 Namespace = "kube-system",
 Values = new Dictionary<string, object>
 {
 { "nodeSelector", new Dictionary<string, object>
 {
 { "kubernetes.io/os", "linux" },
 } },
 },
 }, new CustomResourceOptions { DependsOn = { secretsStoreCsiDriver } });

 var config = new Config();
 var pulumiPAT = config.Require("pulumi-pat");

 var mySecret = new Secret("my-secret", new SecretArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Namespace = "default",
 Name = "pulumi-access-token",
 },
 StringData =
 {
 { "pulumi-access-token", pulumiPAT }
 },
 Type = "Opaque",
 }, new CustomResourceOptions { DependsOn = { secretsStoreCsiPulumiEscProvider } });

});


class SecretProviderClassArgs : CustomResourceArgs
{
 public SecretProviderClassArgs(): base("secrets-store.csi.x-k8s.io/v1", "SecretProviderClass")
 {
 }

 [Input("spec")]
 public Input<SecretProviderClassSpecArgs>? Spec { get; set; }
}


class SecretProviderClassSpecArgs : ResourceArgs
{
 [Input("provider")]
 public Input<string>? Provider { get; set; }

 [Input("parameters")]
 public Input<InputMap<object>>? Parameters { get; set; }
}

class SecretProviderParametersArgs : ResourceArgs
{
 [Input("apiUrl")]
 public Input<string>? ApiUrl { get; set; }

 [Input("organization")]
 public Input<string>? Organization { get; set; }

 [Input("project")]
 public Input<string>? Project { get; set; }

 [Input("environment")]
 public Input<string>? Environment { get; set; }

 [Input("authSecretName")]
 public Input<string>? AuthSecretName { get; set; }

 [Input("authSecretNamespace")]
 public Input<string>? AuthSecretNamespace { get; set; }

 [Input("secrets")]
 public Input<string>? Secrets { get; set; }
}

Deploy the stack by running:

pulumi up

And you should see that the secret was created in the Kubernetes cluster and the ESO instance was deployed successfully.

kubectl get secret pulumi-access-token -o jsonpath='{.data.PULUMI_ACCESS_TOKEN}' | base64 -d

Step 2: Create a SecretProviderClass

Now, we will create a secret in the Pulumi ESC project and synchronize it into the Kubernetes cluster by creating an SecretProviderClass:

Create a new ESC environment called csi-secrets-store-app in the esc-secrets-store-csi-driver-demo project:

values:
 app:
 hello: world
 hello-secret:
 fn::secret: world

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/esc-secrets-store-csi-driver-demo/csi-secrets-store-app

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/esc-secrets-store-csi-driver-demo/csi-secrets-store-app

Either way, you should see following environment configuration in the Pulumi Cloud Console:

We can now create the SecretProviderClass in the Kubernetes cluster:

const secretProviderClass = new k8s.apiextensions.CustomResource(
 "example-provider-pulumi-esc",
 {
 apiVersion: "secrets-store.csi.x-k8s.io/v1",
 kind: "SecretProviderClass",
 metadata: {
 name: "example-provider-pulumi-esc",
 namespace: "default",
 },
 spec: {
 provider: "pulumi",
 parameters: {
 apiUrl: "https://api.pulumi.com/api/esc",
 organization: "dirien",
 project: "esc-secrets-store-csi-driver-demo",
 environment: "csi-secrets-store-app",
 authSecretName: mySecret.metadata.name,
 authSecretNamespace: mySecret.metadata.namespace,
 secrets: `- secretPath: "/"
 fileName: "hello"
 secretKey: "app.hello"
`,
 },
 },
 },
 { dependsOn: secretsStoreCSIPulumiESCProvider },
);

secret_provider_class = k8s.apiextensions.CustomResource(
 "example-provider-pulumi-esc",
 api_version="secrets-store.csi.x-k8s.io/v1",
 kind="SecretProviderClass",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 name="example-provider-pulumi-esc", namespace="default"
 ),
 spec={
 "provider": "pulumi",
 "parameters": {
 "apiUrl": "https://api.pulumi.com/api/esc",
 "organization": "dirien",
 "project": "esc-secrets-store-csi-driver-demo",
 "environment": "csi-secrets-store-app",
 "authSecretName": my_secret.metadata["name"],
 "authSecretNamespace": my_secret.metadata["namespace"],
 "secrets": '- secretPath: "/"\n fileName: "hello"\n secretKey: "app.hello"\n',
 },
 },
 opts=pulumi.ResourceOptions(depends_on=[secrets_store_csi_pulumi_esc_provider]),
)

 secretProviderClass, err := apiextensions.NewCustomResource(ctx, "example-provider-pulumi-esc", &apiextensions.CustomResourceArgs{
 ApiVersion: pulumi.String("secrets-store.csi.x-k8s.io/v1"),
 Kind: pulumi.String("SecretProviderClass"),
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("example-provider-pulumi-esc"),
 Namespace: pulumi.String("default"),
 },
 OtherFields: k8s.UntypedArgs{
 "provider": pulumi.String("pulumi"),
 "parameters": pulumi.Map{
 "apiUrl": pulumi.String("https://api.pulumi.com/api/esc"),
 "organization": pulumi.String("dirien"),
 "project": pulumi.String("esc-secrets-store-csi-driver-demo"),
 "environment": pulumi.String("csi-secrets-store-app"),
 "authSecretName": mySecret.Metadata.Name().Elem(),
 "authSecretNamespace": mySecret.Metadata.Namespace().Elem(),
 "secrets": pulumi.String(`- secretPath: "/"
 fileName: "hello"
 secretKey: "app.hello"
`),
 },
 },
 }, pulumi.DependsOn([]pulumi.Resource{secretsStoreCsiPulumiEscProvider}))
 if err != nil {
 return err
 }

 var secretProviderClass = new Pulumi.Kubernetes.ApiExtensions.CustomResource("example-provider-pulumi-esc", new SecretProviderClassArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Name = "example-provider-pulumi-esc",
 Namespace = "default",
 },
 Spec = new SecretProviderClassSpecArgs
 {
 Provider = "pulumi",
 Parameters = new InputMap<object>
 {
 { "apiUrl", "https://api.pulumi.com/api/esc" },
 { "organization", "dirien" },
 { "project", "esc-secrets-store-csi-driver-demo" },
 { "environment", "csi-secrets-store-app" },
 { "authSecretName", mySecret.Metadata.Apply(metadata => metadata.Name) },
 { "authSecretNamespace", mySecret.Metadata.Apply(metadata => metadata.Namespace) },
 { "secrets", "- secretPath: \"/\"\n fileName: \"hello\"\n secretKey: \"app.hello\"\n" }
 },
 },
 }, new CustomResourceOptions { DependsOn = { secretsStoreCsiPulumiEscProvider } });

We can check that the secret was successfully synchronized by running:

kubectl get secretproviderclasses example-provider-pulumi-esc

You might also like:

Step 3: Deploy an Application and Mount the Secret

Now, we can deploy an application that references the secret from the Kubernetes cluster. I am going to use busybox that reads mounted file in.

const deployment = new k8s.apps.v1.Deployment("example-provider-pulumi-esc", {
 metadata: {
 name: "example-provider-pulumi-esc",
 namespace: "default",
 labels: {
 app: "example-provider-pulumi-esc",
 },
 },
 spec: {
 replicas: 1,
 selector: {
 matchLabels: {
 app: "example-provider-pulumi-esc",
 },
 },
 template: {
 metadata: {
 labels: {
 app: "example-provider-pulumi-esc",
 },
 },
 spec: {
 containers: [
 {
 name: "client",
 image: "busybox:latest",
 command: ["sh", "-c"],
 args: [
 `set -eux
 ls /run/secrets
 find /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo "$(cat "{}")"'
 tail -f /dev/null`,
 ],
 volumeMounts: [
 {
 name: "data",
 mountPath: "/run/secrets",
 },
 ],
 },
 ],
 volumes: [
 {
 name: "data",
 csi: {
 driver: "secrets-store.csi.k8s.io",
 readOnly: true,
 volumeAttributes: {
 secretProviderClass: secretProviderClass.metadata.name,
 },
 },
 },
 ],
 },
 },
 },
});

export const deploymentName = deployment.metadata.name;

deployment = k8s.apps.v1.Deployment(
 "example-provider-pulumi-esc",
 metadata=k8s.meta.v1.ObjectMetaArgs(
 namespace="default",
 name="example-provider-pulumi-esc",
 labels={
 "app": "example-provider-pulumi-esc",
 },
 ),
 spec=k8s.apps.v1.DeploymentSpecArgs(
 replicas=1,
 selector=k8s.meta.v1.LabelSelectorArgs(
 match_labels={
 "app": "example-provider-pulumi-esc",
 },
 ),
 template=k8s.core.v1.PodTemplateSpecArgs(
 metadata=k8s.meta.v1.ObjectMetaArgs(
 labels={
 "app": "example-provider-pulumi-esc",
 },
 ),
 spec=k8s.core.v1.PodSpecArgs(
 containers=[
 k8s.core.v1.ContainerArgs(
 name="client",
 image="busybox:latest",
 command=["sh", "-c"],
 args=[
 "set -eux\nls /run/secrets\nfind /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo \"$(cat \"{}\")\"'\ntail -f /dev/null",
 ],
 volume_mounts=[
 k8s.core.v1.VolumeMountArgs(
 name="data",
 mount_path="/run/secrets",
 ),
 ],
 ),
 ],
 volumes=[
 k8s.core.v1.VolumeArgs(
 name="data",
 csi=k8s.core.v1.CSIVolumeSourceArgs(
 driver="secrets-store.csi.k8s.io",
 read_only=True,
 volume_attributes={
 "secretProviderClass": secret_provider_class.metadata[
 "name"
 ],
 },
 ),
 ),
 ],
 ),
 ),
 ),
)

pulumi.export("deploymentName", deployment.metadata["name"])

 deployment, err := appv1.NewDeployment(ctx, "example-provider-pulumi-esc", &appv1.DeploymentArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Name: pulumi.String("example-provider-pulumi-esc"),
 Namespace: pulumi.String("default"),
 Labels: pulumi.StringMap{
 "app": pulumi.String("example-provider-pulumi-esc"),
 },
 },
 Spec: &appv1.DeploymentSpecArgs{
 Replicas: pulumi.Int(1),
 Selector: &metav1.LabelSelectorArgs{
 MatchLabels: pulumi.StringMap{
 "app": pulumi.String("example-provider-pulumi-esc"),
 },
 },
 Template: &corev1.PodTemplateSpecArgs{
 Metadata: &metav1.ObjectMetaArgs{
 Labels: pulumi.StringMap{
 "app": pulumi.String("example-provider-pulumi-esc"),
 },
 },
 Spec: &corev1.PodSpecArgs{
 Containers: corev1.ContainerArray{
 &corev1.ContainerArgs{
 Name: pulumi.String("client"),
 Image: pulumi.String("busybox:latest"),
 Command: pulumi.StringArray{
 pulumi.String("sh"),
 pulumi.String("-c"),
 },
 Args: pulumi.StringArray{
 pulumi.String(`set -eux
 ls /run/secrets
 find /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo "$(cat "{}")"'
 tail -f /dev/null`),
 },
 VolumeMounts: &corev1.VolumeMountArray{
 &corev1.VolumeMountArgs{
 Name: pulumi.String("data"),
 MountPath: pulumi.String("/run/secrets"),
 },
 },
 },
 },
 Volumes: &corev1.VolumeArray{
 &corev1.VolumeArgs{
 Name: pulumi.String("data"),
 Csi: corev1.CSIVolumeSourceArgs{
 Driver: pulumi.String("secrets-store.csi.k8s.io"),
 ReadOnly: pulumi.Bool(true),
 VolumeAttributes: pulumi.StringMap{
 "secretProviderClass": secretProviderClass.Metadata.Name().Elem(),
 },
 },
 },
 },
 },
 },
 },
 }, pulumi.DependsOn([]pulumi.Resource{secretProviderClass}))
 if err != nil {
 return err
 }

 var deployment = new Pulumi.Kubernetes.Apps.V1.Deployment("example-provider-pulumi-esc", new Pulumi.Kubernetes.Types.Inputs.Apps.V1.DeploymentArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Name = "example-provider-pulumi-esc",
 Namespace = "default",
 Labels =
 {
 { "app", "example-provider-pulumi-esc" },
 },
 },
 Spec = new Pulumi.Kubernetes.Types.Inputs.Apps.V1.DeploymentSpecArgs
 {
 Replicas = 1,
 Selector = new Pulumi.Kubernetes.Types.Inputs.Meta.V1.LabelSelectorArgs
 {
 MatchLabels =
 {
 { "app", "example-provider-pulumi-esc" },
 },
 },
 Template = new Pulumi.Kubernetes.Types.Inputs.Core.V1.PodTemplateSpecArgs
 {
 Metadata = new ObjectMetaArgs
 {
 Labels =
 {
 { "app", "example-provider-pulumi-esc" },
 },
 },
 Spec = new Pulumi.Kubernetes.Types.Inputs.Core.V1.PodSpecArgs
 {
 Containers =
 {
 new Pulumi.Kubernetes.Types.Inputs.Core.V1.ContainerArgs
 {
 Name = "client",
 Image = "busybox:latest",
 Command =
 {
 "sh",
 "-c",
 },
 Args =
 {
 "set -eux\nls /run/secrets\nfind /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*' | xargs -t -I {} sh -c 'echo \"$(cat \"{}\")\"'\ntail -f /dev/null",
 },
 VolumeMounts =
 {
 new Pulumi.Kubernetes.Types.Inputs.Core.V1.VolumeMountArgs
 {
 Name = "data",
 MountPath = "/run/secrets",
 },
 },
 },
 },
 Volumes =
 {
 new Pulumi.Kubernetes.Types.Inputs.Core.V1.VolumeArgs
 {
 Name = "data",
 Csi = new Pulumi.Kubernetes.Types.Inputs.Core.V1.CSIVolumeSourceArgs
 {
 Driver = "secrets-store.csi.k8s.io",
 ReadOnly = true,
 VolumeAttributes =
 {
 { "secretProviderClass", secretProviderClass.Metadata.Apply(metadata => metadata.Name) },
 },
 },
 },
 },
 },
 },
 },
 }, new CustomResourceOptions { DependsOn = { secretProviderClass } });

After deploying the stack, you can get the logs of the busybox pod to see that the secret was successfully mounted:

NAME=$(kubectl get pods -o name | grep example-provider-pulumi-esc | cut -d'/' -f2)
kubectl logs $NAME

You should see the following output:

+ ls /run/secrets
hello
+ find /run/secrets/ -mindepth 1 -maxdepth 1 -not -name '.*'
+ xargs -t -I '{}' sh -c 'echo "$(cat "{}")"'
sh -c echo "$(cat "/run/secrets/hello")"
world
+ tail -f /dev/null

Or you can exec into the pod and check the content of the mounted secret:

kubectl exec -it $NAME -- cat /run/secrets/hello

You should see the following output:

world

Step 4: Clean Up

After you are done with the demo, you can clean up the resources by running:

pulumi destroy

Conclusion

The Secret Store CSI Driver is a great option for managing secrets in a cloud-native way when the use of Kubernetes secrets is not an option due to enhanced security requirements.

Below is a quick comparison of the External Secrets Operator and the Secrets Store CSI Driver to help you decide which one is the best fit for your use case. As always, the best way to find out is to try both solutions and see which one fits your requirements best.

The good part is, whatever you choose, you can use Pulumi ESC to manage your secrets and avoid any secret sprawl in your organization as both solutions are supported by Pulumi ESC.

Comparison of the External Secrets Operator and the Secrets Store CSI Driver

Feature/Aspect	External Secrets Operator	CSI Secrets Driver
Primary Use Case	Synchronizing external secrets into Kubernetes as native secrets.	Mounting secrets directly to pods as files or environment variables.
Integration with External Secret Stores	Supports multiple external secret stores like AWS Secrets Manager, Google Secret Manager, Vault, Azure Key Vault, etc.	Primarily supports integrations defined by the CSI specification (e.g., Vault, AWS, Azure Key Vault, Pulumi ESC).
Mechanism for Secrets Delivery	Secrets are pulled and stored as Kubernetes `Secret` resources.	Secrets are mounted directly into pods as files or injected as environment variables.
Kubernetes Resource Requirements	Requires a custom Kubernetes resource (`ExternalSecret`) and the External Secrets Operator.	Relies on the Kubernetes CSI (Container Storage Interface) standard and requires a specific CSI driver.
Secret Rotation	Supports automatic rotation by polling the external secret store at defined intervals.	Supports rotation, but the application needs to handle reloading mounted secrets.
Security Considerations	Secrets are stored temporarily in Kubernetes as native secrets, which may pose a risk if not properly managed.	Secrets are mounted directly and not stored as native Kubernetes secrets, reducing exposure but relying on file permissions.
Customization and Features	Highly customizable with fine-grained mappings and transformation options.	Limited customization, focused on mounting secrets as-is.
Application Compatibility	Applications access secrets as Kubernetes secrets (e.g., via env variables or volumes).	Applications access secrets directly via mounted files or injected env variables.
Performance Considerations	Potential slight delay due to syncing secrets to Kubernetes secrets.	Faster access as secrets are directly mounted.
Community and Ecosystem	Supported by a growing community; integrates well with diverse secret management solutions.	Part of the Kubernetes ecosystem; supports CSI-compliant secret storage solutions.
Best Use Cases	Ideal for scenarios where secrets need to be accessed as Kubernetes native secrets.	Best for applications requiring direct access to secrets without intermediate storage in Kubernetes secrets.

You can find detailed information about all current available Pulumi ESC Kubernetes integrations in the Pulumi ESC documentation.

Let us know what you think about the Secret Store CSI Driver and how you are managing secrets in your Kubernetes clusters. We would love to hear your feedback and experiences.

Try Pulumi for Free

Pulumi Blog: Csi

Why Every Cloud Engineer Needs Pulumi ESC for Secrets Management

In this article:

The Challenge of Secrets Management in Modern Cloud Environments

What is Pulumi ESC?

Key Features of Pulumi ESC

1. Seamless Integration with External Platforms

2. Dynamic Secrets Synchronization with ESO

3. Safely Roll Back to a Previous Version

4. Secure by Design

5. Automated Rotation and Expiry

6. Configuration-as-Code, Automation, and Integration Everywhere

7. Dev Tools Integrations

8. Infrastructure Tools Integrations

Why Cloud Engineers Need Pulumi ESC

Enhanced Security and Compliance

Streamlined Multi-Cloud Management

Simplified Kubernetes Secrets

Zero Downtime Through Automation

Developer-Friendly Workflows

Kubernetes Secrets Management

Using Pulumi ESC with External Secrets Operator

Using Pulumi ESC with Secrets Store CSI Driver

The Future of Secrets Management with Pulumi ESC

Next Steps to Secure Your Secrets

Master Kubernetes Secrets with Pulumi ESC + Secrets Store CSI Driver

TL;DR?

Refresher: Why Should You Avoid Using Kubernetes Secrets?

What is the Secrets Store CSI Driver?

Container Storage Interface (CSI)

Combining CSI, Kubernetes and Secrets = Secret Store CSI Driver

The Architecture of the Secret Store CSI Driver

How to Use the Secrets Store CSI Driver with Pulumi ESC

Step 1: Deploy the Secrets Store CSI Driver and Pulumi ESC CSI Provider

Step 2: Create a SecretProviderClass

Step 3: Deploy an Application and Mount the Secret

Step 4: Clean Up

Conclusion

Comparison of the External Secrets Operator and the Secrets Store CSI Driver