Using gcp secretmanager with gke-autoneg-controller

Question

Pulumi · Accepted Answer

When integrating Google Cloud Secret Manager with Google Kubernetes Engine (GKE), you can leverage the secrets to hold sensitive information that your application within GKE may need. In the case of using the GKE auto-negotiation controller, which is typically used to automatically manage network endpoint groups based on the service annotations, you might want to provide the controller with the necessary credentials or other sensitive configuration data stored securely in Secret Manager.

The following program will guide you through:

Creating a secret in Google Cloud Secret Manager.
Granting the GKE auto-negotiation controller the necessary IAM permissions to access the secret.
Creating a Kubernetes Secret resource in GKE, which references the Google Cloud Secret Manager secret.
Assigning the Kubernetes Secret to the GKE auto-negotiation controller so it can use the sensitive data securely.

First, we’ll create a secret in GCP Secret Manager:

import * as gcp from "@pulumi/gcp";
import * as k8s from "@pulumi/kubernetes";

// Step 1: Create a GCP Secret
const secret = new gcp.secretmanager.Secret("my-secret", {
    replication: {
        automatic: true,
    },
    // other properties like secretId, project etc., can be specified as needed
});

This defines a new secret with automatic replication, ensuring it's available across all regions. If you need to place this on a specific project, you can specify the project field accordingly.

Next, we will create a secret version to hold the actual secret data:

const secretVersion = new gcp.secretmanager.SecretVersion("my-secret-version", {
    secret: secret.id,
    secretData: "my-super-secret-data", // Your secret data here
});

The secret data you provide here is the actual value you want to keep safe (like API keys, passwords, etc.).

Then, we assign the necessary IAM bindings to allow the GKE auto-negotiation controller to access the secret:

const secretAccessor = new gcp.secretmanager.SecretIamBinding("secret-accessor", {
    secretId: secret.id,
    role: "roles/secretmanager.secretAccessor",
    members: [
        // Assuming that your auto-negotiation controller runs under a specific service account
        // Replace "<YOUR_SERVICE_ACCOUNT>" with the actual service account email
        `serviceAccount:<YOUR_SERVICE_ACCOUNT>`,
    ],
});

Here, the role roles/secretmanager.secretAccessor grants read-only access to the secret value. Be sure to replace <YOUR_SERVICE_ACCOUNT> with the service account your auto-negotiation controllers run under.

Finally, in your GKE cluster, you’ll need to create a Kubernetes secret that references the GCP secret. This allows the auto-negotiation controller to use the data from Secret Manager.

// Ensure that your K8s provider is properly configured to connect to your GKE cluster
const provider = new k8s.Provider("provider", {
    kubeconfig: "<YOUR_KUBECONFIG_CONTENT>", // Your GKE cluster's kubeconfig
});

const k8sSecret = new k8s.core.v1.Secret("k8s-secret", {
    // Secret data here should reference your GCP Secret Manager's secret
    stringData: {
        key: secretVersion.secretData.apply(data => data), // This will fetch the data from your created GCP Secret
    },
    // other properties like metadata, type, etc., can be specified as needed
}, { provider });

Make sure to replace <YOUR_KUBECONFIG_CONTENT> with the actual kubeconfig content of your GKE cluster. The Secret resource created will be referenced in your auto-negotiation controller service configuration. The secretVersion.secretData.apply() method retrieves the secret data from GCP and applies it to the Kubernetes secret.

To summarize:

A GCP secret and its version are created to securely hold your sensitive data.
IAM permissions are set up to allow the auto-neg-controller to access the secret.
A Kubernetes secret is created to reference the GCP secret, which can then be used in your Kubernetes workloads.

This Pulumi program will set up the infrastructure you need to securely manage sensitive data for your auto-negotiation controller in GKE using Google Cloud Secret Manager.