Using kubernetes policy with secrets-store.csi.x-k8s.io

Question

Pulumi · Accepted Answer

To interact with Kubernetes policies and use the `secrets-store.csi.x-k8s.io` CSI driver, you'll generally be dealing with a few different Kubernetes resources:

1. **CustomResourceDefinitions (CRDs)** - These define custom resources such as the Secrets Store CSI driver's `SecretProviderClass` resource, which is how you associate external secret stores with your Kubernetes pods.
2. **ClusterRole and RoleBinding or ClusterRoleBinding** - These are RBAC (Role-Based Access Control) resources that dictate what permissions the CSI driver's ServiceAccount has within your Kubernetes cluster.
3. **DaemonSet or Deployment** - This resource type is used to deploy the CSI driver into your cluster.

Below is a high-level overview of the steps you would typically take:

1. Install the CSI Secret Store driver in your cluster. This is often done via `kubectl apply` or a tool like `Helm`, but could theoretically be handled by Pulumi as well.
2. Create a `SecretProviderClass` resource that specifies which secrets you want to fetch and how to fetch them.
3. Create a pod that references the `SecretProviderClass` and mounts the secrets as volumes or exposes them as environment variables.

Since Pulumi can be used to manage Kubernetes resources through code, you can define these requirements programmatically through Pulumi's Kubernetes provider. The example below will walk you through a basic Pulumi TypeScript setup to start using the `secrets-store.csi.x-k8s.io` CSI driver with Kubernetes policies.

Before starting, make sure you have installed:
- Pulumi CLI
- Node.js and npm or yarn
- `kubectl` configured to access your Kubernetes cluster

Now, let’s look at the program:

```typescript
import * as k8s from '@pulumi/kubernetes';

// A Pulumi program to use Kubernetes policies with the secrets-store.csi.x-k8s.io driver.

// Create a service account for the CSI driver.
const csiServiceAccount = new k8s.core.v1.ServiceAccount('csi-secrets-store-service-account', {
  metadata: {
    name: 'csi-secrets-store',
    namespace: 'kube-system',
  },
}, { provider: k8sProvider });

// Define the ClusterRole with necessary permissions for the CSI driver.
const csiClusterRole = new k8s.rbac.v1.ClusterRole('csi-secrets-store-cluster-role', {
  metadata: {
    name: 'secrets-store-csi-driver',
  },
  rules: [
    // Additional rules might be necessary depending on what the driver needs to do.
    {
      apiGroups: [""],
      resources: ["secrets"],
      verbs: ["get", "list"],
    },
    // ... other rules ...
  ],
}, { provider: k8sProvider });

// Bind the ClusterRole to the service account.
const csiClusterRoleBinding = new k8s.rbac.v1.ClusterRoleBinding('csi-secrets-store-role-binding', {
  metadata: {
    name: 'secrets-store-csi-driver',
  },
  subjects: [
    {
      kind: 'ServiceAccount',
      name: csiServiceAccount.metadata.name,
      namespace: 'kube-system',
    },
  ],
  roleRef: {
    kind: 'ClusterRole',
    name: csiClusterRole.metadata.name,
    apiGroup: 'rbac.authorization.k8s.io',
  },
}, { provider: k8sProvider });

// Create a SecretProviderClass resource that specifies the secret store to use
// and the secrets to retrieve.
const secretProviderClass = new k8s.apiextensions.CustomResource('my-secret-provider-class', {
  apiVersion: 'secrets-store.csi.x-k8s.io/v1alpha1',
  kind: 'SecretProviderClass',
  metadata: {
    name: 'my-secret-provider',
  },
  spec: {
    provider: 'azure', // or your provider of choice
    // Parameters relevant to your secret store provider
    parameters: {
      // Placeholder values; replace with your specific secret store config
      keyvaultName: 'my-keyvault',
      objects: [{
        objectName: 'mySecret',
        objectType: 'secret',
        objectVersion: '', // optional; if empty will fetch latest version
      }],
      tenantId: 'my-tenant-id',
    },
    // Define how the secrets will be made accessible to the pod
    secretObjects: [
      {
        secretName: 'my-secret',
        type: 'Opaque',
        data: [
          {
            key: 'mySecret',
            objectName: 'mySecret',
          },
        ],
      },
    ],
  },
}, { provider: k8sProvider });

// Then, in your pod spec, you would use something like the following:
const myPod = new k8s.core.v1.Pod('my-pod', {
  metadata: {
    name: 'my-app',
    namespace: 'default',
  },
  spec: {
    containers: [
      {
        name: 'my-app',
        image: 'my-app-image',
        volumeMounts: [
          {
            name: 'secret-vol',
            mountPath: '/mnt/secrets',
            readOnly: true,
          },
        ],
      },
    ],
    volumes: [
      {
        name: 'secret-vol',
        csi: {
          driver: 'secrets-store.csi.k8s.io',
          readOnly: true,
          volumeAttributes: {
            secretProviderClass: 'my-secret-provider',
          },
        },
      },
    ],
  },
}, { provider: k8sProvider });

// Export the ServiceAccount name
export const serviceAccountName = csiServiceAccount.metadata.name;
```

This program creates the Kubernetes resources necessary to set up the `secrets-store.csi.x-k8s.io` CSI driver and use it within a pod to access secrets from an external secret store such as Azure Key Vault.

- The `csiServiceAccount` is a `ServiceAccount` that will be used to run the CSI driver pods.
- The `csiClusterRole` and `csiClusterRoleBinding` define and apply the permissions needed by the CSI driver to function properly within the cluster.
- The `secretProviderClass` is a custom resource that tells the CSI driver which secret store to connect to, and what secrets to fetch from that store.
- The `myPod` resource is an example of how to define a pod that uses a mounted volume to access secrets through the CSI driver.

Remember to replace placeholder values like `my-keyvault`, `mySecret`, `my-tenant-id`, and `my-app-image` with your actual values. The `volumeAttributes` in the volume within the pod spec should reference the `secretProviderClass` that was created.

Each of the resources in this example should be modified to fit your specific use case, such as the choice of secret provider and the required RBAC permissions. You should also replace `k8sProvider` with an instance of a Pulumi Kubernetes provider that is configured for your cluster.