Using kubernetes secrets-store.csi.x-k8s.io with k8s.nginx.org

Question

Pulumi · Accepted Answer

When integrating Kubernetes secrets with an NGINX Ingress controller, you would typically use a Kubernetes Secret resource to hold your sensitive data, such as TLS certificates, and then reference these secrets in your Ingress resource to secure your endpoints.

The Secrets Store CSI driver (`secrets-store.csi.x-k8s.io`) allows you to retrieve secrets from external Secrets Managers (like Azure Key Vault, AWS Secrets Manager, etc.) and project them into your Kubernetes pods. The NGINX Ingress controller (`k8s.nginx.org`) can then use these secrets by referencing the projected volume in your Ingress definitions.

Below is a Pulumi program written in TypeScript that demonstrates how you would set up this scenario. The program assumes that you have already set up the Secrets Store CSI driver and have an external secrets provider where your secrets are stored. We will create two main resources:

1. A SecretProviderClass resource which configures how secrets should be retrieved from the external provider.
2. An Ingress resource which uses an NGINX controller to route traffic to your service and references the mounted secrets.

Before you can run this Pulumi program, you should have the following prerequisites in place:
- A Kubernetes cluster with the Secrets Store CSI driver installed.
- An external secrets provider (like Azure Key Vault) with the necessary permissions set up for the Secrets Store CSI driver to access the secrets.
- The NGINX Ingress controller installed in your Kubernetes cluster.

Here is a program that sets up an Ingress resource with a reference to the secret mounted by the Secrets Store CSI driver:

```typescript
import * as kubernetes from "@pulumi/kubernetes";

// Create a SecretProviderClass resource.
// This resource references the external secret store and the specific secrets within it.
// Replace `my-provider`, `my-secret`, etc., with your specific provider and secret details.
const secretProviderClass = new kubernetes.apiextensions.CustomResource("my-secret-provider-class", {
    apiVersion: "secrets-store.csi.x-k8s.io/v1",
    kind: "SecretProviderClass",
    metadata: {
        name: "my-secret-provider-class",
    },
    spec: {
        provider: "my-provider", // Replace with your secret provider, e.g., azure, gcp, aws, etc.
        parameters: {
            // Provider-specific parameters for accessing the secret store.
        },
        secretObjects: [
            {
                secretName: "my-tls-secret",
                type: "kubernetes.io/tls",
                data: [
                    {
                        key: "tls.crt",
                        objectName: "my-cert" // Name of the certificate object in the secret store.
                    },
                    {
                        key: "tls.key",
                        objectName: "my-cert-key" // Name of the certificate key object in the secret store.
                    },
                ],
            },
        ],
    },
}, {dependsOn: [/* dependencies */]});

// Define an Ingress resource that uses the NGINX Ingress controller.
// It will reference the TLS secret to secure the endpoint.
const ingress = new kubernetes.networking.v1.Ingress("my-ingress", {
    metadata: {
        name: "my-ingress",
        annotations: {
            "kubernetes.io/ingress.class": "nginx",
            "nginx.ingress.kubernetes.io/ssl-redirect": "true",
        },
    },
    spec: {
        tls: [
            {
                // hostnames to be covered by the TLS certificate.
                hosts: ["example.com"],
                // Name of the secret containing the TLS certificate and key.
                // This should match the `secretObjects` field in the SecretProviderClass.
                secretName: "my-tls-secret",
            },
        ],
        // The rules defining how traffic is routed to your services.
        rules: [
            // ...your routing rules...
        ],
    },
}, {dependsOn: [secretProviderClass]});

// Export the ingress' endpoint so you can access it.
export const ingressUrl = ingress.status.loadBalancer.ingress[0].hostname;
```

This program will create an Ingress resource using the NGINX Ingress controller to manage incoming traffic to your services. It leverages a SecretProviderClass to retrieve the TLS certificate and key from an external secret store, which are mounted in the Kubernetes secret `my-tls-secret`. This secret is then referenced in the Ingress resource under `spec.tls`, enabling TLS termination with your own certificates for secure HTTPS communication.

You could adapt this Pulumi program according to your actual cloud provider and the specifics of your setup, such as secret names, secret providers, domain names for the Ingress resource, and the service that it routes traffic to.

After deploying this program with Pulumi, you should have an Ingress that uses TLS certificates fetched from an external secret store, securing the communication to your services.