Using kubernetes velero.io with generators.external-secrets.io

Question

Pulumi · Accepted Answer

To integrate Velero with Kubernetes for backups and ExternalSecrets for managing secrets, you'll use the Kubernetes provider in Pulumi. Velero is an open source tool to safely backup and restore, perform disaster recovery, and migrate Kubernetes cluster resources and persistent volumes. ExternalSecrets is a Kubernetes operator that reads information from a backend (like AWS Secrets Manager, Azure Key Vault, or GCP Secret Manager) to securely inject secrets into your Kubernetes resources.

First, you will need to set up Velero in your Kubernetes cluster using its Helm chart, configure `BackupStorageLocation` and `VolumeSnapshotLocation` with your cloud provider's details, and deploy the necessary CRDs for ExternalSecrets. Then, you'll create an `ExternalSecret` that Velero would use for accessing your cloud storage for backups.

Now let's write a Pulumi program in TypeScript to deploy Velero and set up ExternalSecrets in your cluster. To use the following setup, ensure you have Pulumi and kubectl installed and configured to connect to your Kubernetes cluster.

### Detailed Pulumi Program To Deploy Velero and ExternalSecrets

```typescript
import * as kubernetes from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

// We'll start by setting up the Helm chart for Velero. For this example, let's assume we're using AWS.
const veleroChart = new kubernetes.helm.v3.Chart("velero", {
    chart: "velero",
    version: "2.23.3", // Please use the latest applicable version
    fetchOpts: {
        repo: "https://vmware-tanzu.github.io/helm-charts",
    },
    values: {
        // Credentials and configuration for AWS S3 as a backup storage provider
        configuration: {
            provider: "aws",
            backupStorageLocation: {
                bucket: "my-velero-backups",
                config: {
                    region: "us-west-2", // Specify your AWS region
                    // More configuration here as needed
                },
            },
            volumeSnapshotLocation: {
                config: {
                    region: "us-west-2", // Specify your AWS region
                    // More configuration here as needed
                },
            },
        },
        // Provide the service account or IAM role with the necessary permissions
        serviceAccount: {
            server: {
                annotations: {
                    // The annotation depends on your cloud provider. This example uses AWS's annotation for IRSA (IAM roles for service accounts)
                    "eks.amazonaws.com/role-arn": "arn:aws:iam::123456789012:role/velero-server-role" // Replace with the correct ARN
                },
            },
        },
        // Specify your own secrets (e.g., cloud provider credentials)
        credentials: {
            useSecret: true,
            secretContents: {
                cloud: pulumi.secret("<your cloud provider credentials here>"), // Securely inject the credential file
            },
        },
        // You can set more values here as you need
    },
});

// Now let's deploy the ExternalSecrets Helm chart.
const externalSecretsChart = new kubernetes.helm.v3.Chart("external-secrets", {
    chart: "external-secrets",
    version: "0.3.6", // Please use the latest applicable version
    fetchOpts: {
        repo: "https://external-secrets.github.io/kubernetes-external-secrets/",
    },
    values: {
        // Configuration values for the ExternalSecrets Helm chart
        // This setup depends on your exact backend and requirements
        // For AWS, you might set the ROLE_ARN here for instance
        env: {
            AWS_REGION: "us-west-2",
            // possibly other environment variables needed for ExternalSecrets
        },
        // You can set more values here as per your setup
    },
});

// Having Velero and ExternalSecrets deployed, you now can create an ExternalSecret resource
// to hold the credentials for Velero to access the storage backend.
const veleroBackendCredentials = new kubernetes.apiextensions.CustomResource("velero-backend-creds", {
    apiVersion: "kubernetes-client.io/v1",
    kind: "ExternalSecret",
    metadata: { name: "velero-backend-creds" },
    spec: {
        backendType: "secretsManager", // Your backend type
        // Specification for data coming from the backend
        data: [{
            key: "my-velero-backups/credentials", // Key for the secret in the backend
            name: "cloud", // Name of the key to be created in the Kubernetes secret
        }],
    },
});

// Export the URL for the Velero server
export const veleroServerURL = veleroChart.getResourceProperty(
    "v1/Service",
    "velero/velero",
    "status"
).apply(status => status.loadBalancer.ingress[0].hostname || status.loadBalancer.ingress[0].ip);

```

This program deploys Velero and sets up ExternalSecrets in your Kubernetes cluster by following these steps:

1. It defines a Helm chart resource for Velero, specifying AWS as the backup storage provider.
2. It configures the necessary AWS S3 bucket details and AWS region for both backup storage and volume snapshots.
3. It customizes annotations for the Velero server's service account to use an AWS IAM role via IRSA (IAM Role for Service Account).
4. It sets up the ExternalSecrets Helm chart with the necessary environment variables.
5. It creates a custom resource for an `ExternalSecret`, pointing to where Velero's credentials are stored.

Make sure to replace placeholders like `"my-velero-backups"`, `us-west-2`, and `"arn:aws:iam::123456789012:role/velero-server-role"` with the actual values for your AWS S3 bucket name, region, and IAM role ARN. The IAM role should have the necessary permissions to access S3 for backups and snapshotting.

The `pulumi.secret()` function is used to ensure that cloud provider credentials are encrypted and securely managed by Pulumi.

Finally, the program exports the URL for the Velero server service, which can be used to access the Velero dashboard if you have set up an ingress for it.

Feel free to tailor the configuration as per your actual cloud provider setup and specific values for ExternalSecrets. Ensure that the versions for Velero and ExternalSecrets charts used are compatible with your cluster version and cloud provider specifics.