Using kubernetes external-secrets.io with admissionregistration.k8s.io

Question

Pulumi · Accepted Answer

To use `external-secrets` with Kubernetes, you need to set up the `external-secrets` Kubernetes operator which allows you to manage secrets outside of Kubernetes (for example, in AWS Secrets Manager or GCP Secrets Manager). When you deploy `external-secrets` in Kubernetes, it may require a webhook to validate or mutate the Kubernetes resources upon their creation or update. This webhook can be implemented through the Kubernetes `admissionregistration.k8s.io` API.

Here's a practical example of how we can achieve this using Pulumi with TypeScript:

1. We'll create a `MutatingWebhookConfiguration` or `ValidatingWebhookConfiguration` depending on our need to either mutate or validate the Kubernetes resources, respectively.
2. For simplicity, we'll use a dummy configuration in the code but in a real-world scenario, you would configure it to target your `external-secrets` webhook service.
3. We'll also need a `Service` and potentially a `Deployment` for the webhook server. For `external-secrets.io`, the actual Deployment may have been created by installing the operator using Helm or a similar package manager, so in this example, we will just create the Service.
4. A `Secret` resource may be used if you need to store the certificates for the webhook's HTTPS server. Typically, this would be a TLS secret type.

Let's set up a validating webhook as part of the `external-secrets` setup:

```typescript
import * as k8s from "@pulumi/kubernetes";

// Creating a namespace for external-secrets
const namespace = new k8s.core.v1.Namespace("external-secrets-ns", {
    metadata: { 
        name: "external-secrets",
    },
});

// Placeholder for the TLS secret necessary for the webhook service
// Certificates must be issued and managed to allow for secure communication to the webhook
const tlsSecret = new k8s.core.v1.Secret("webhook-tls-secret", {
    metadata: {
        namespace: namespace.metadata.name,
    },
    type: "kubernetes.io/tls",
    data: {
        "tls.crt": "BASE64_ENCODED_TLS_CERT",
        "tls.key": "BASE64_ENCODED_TLS_PRIVATE_KEY",
    }
});

// Creating a service that points to the webhook server
const webhookService = new k8s.core.v1.Service("webhook-service", {
    metadata: {
        namespace: namespace.metadata.name,
    },
    spec: {
        ports: [
            { port: 443, targetPort: 8443 },
        ],
        selector: {
            app: "external-secrets", // Should match the labels of the external-secrets deployment
        },
    },
});

// Creating a ValidatingWebhookConfiguration for the external-secrets webhook
const validatingWebhookConfiguration = new k8s.admissionregistration.v1.ValidatingWebhookConfiguration("external-secrets-validating-webhook", {
    metadata: { 
        name: "external-secrets-validator",
    },
    webhooks: [{
        name: "external-secrets.validation.io",
        clientConfig: {
            service: {
                name: webhookService.metadata.name,
                namespace: namespace.metadata.name,
                path: "/validate", // This path may vary depending on external-secrets webhook configuration
            },
            caBundle: tlsSecret.data["tls.crt"],
        },
        rules: [{
            operations: ["CREATE", "UPDATE"],
            apiGroups: [""], // Core API group, adjust depending on the external secrets operator
            apiVersions: ["v1"],
            resources: ["secrets"], // Define the resources that require validation
        }],
        failurePolicy: "Fail", // Specifies what to do if the webhook is unavailable
        sideEffects: "None", // Specifies if the webhooks have side effects
        admissionReviewVersions: ["v1"], // API versions that the webhook understands
    }],
},{
    dependsOn: [tlsSecret, webhookService], // Ensures the webhook service and TLS secret are created first
});
```

This Pulumi code establishes the environment you would need to integrate `external-secrets.io` with Kubernetes webhooks under the `admissionregistration.k8s.io` API group.

Here's what each section of the Pulumi code does:

- The `Namespace` block is where you define a new namespace for the `external-secrets` resources. 
- The `Service` block defines a Kubernetes Service which directs traffic to your webhook server pods.
- The `Secret` block is where you would place the TLS certificates that the webhook uses to ensure secure communication. These values should be base64-encoded.
- The `ValidatingWebhookConfiguration` block defines the configuration of your validating webhook, including the path to your webhook's service and the `caBundle` which is a base64-encoded string of your webhook's CA certificate.
- `dependsOn` ensures that certain resources are created in the correct order.

If you haven’t already set up the `external-secrets` operator, you might also need to install the operator in your cluster to manage and create `ExternalSecret` objects that synchronize secrets from external APIs (like AWS Secrets Manager or Azure Key Vault).

Please remember to replace the following placeholders with your actual data:
- `BASE64_ENCODED_TLS_CERT` with your base64-encoded TLS certificate.
- `BASE64_ENCODED_TLS_PRIVATE_KEY` with your base64-encoded TLS private key.
- `/validate` with the actual path to your webhook's validate function if it's different.

Note that you would also need the appropriate Role-Based Access Control (RBAC) configurations in place, and actually deploying the webhook endpoint is beyond the scope of this example.

Lastly, this is a high-level, declarative specification of the desired state of your system, so running this Pulumi script will ensure the necessary Kubernetes objects exist in the right configuration, and Pulumi takes care of orchestrating the creation, updating, or deletion of these resources.