Using kubernetes opentelemetry.io with generators.external-secrets.io

Question

Pulumi · Accepted Answer

If you're looking to integrate OpenTelemetry on Kubernetes with External Secrets, you're combining observability with secure management of secrets. OpenTelemetry provides a suite of tools to capture, export, and analyze telemetry data (like metrics, logs, and traces) from your Kubernetes cluster. Meanwhile, External Secrets allows you to manage your secrets securely outside of your Kubernetes cluster, such as in cloud provider secret management services.

Here's what you would generally want to accomplish with such an integration:

1. Deploy an OpenTelemetry collector in your Kubernetes cluster to gather telemetry data.
2. Securely manage access to the telemetry backends (such as Prometheus, Jaeger, etc.) or other services that OpenTelemetry might send data to, by using secrets that are fetched from an external secret management service (like AWS Secrets Manager, Azure Key Vault, etc.) using External Secrets.

To get you started, I’ll outline a Pulumi program in TypeScript that would deploy these components to a Kubernetes cluster. Note that it assumes you are already familiar with both Kubernetes and Pulumi, and that you have Pulumi installed and configured for use with your Kubernetes cluster.

### Program Explanation

The program will do the following:

- Define a Kubernetes `Deployment` for the OpenTelemetry collector, which collects telemetry data from various sources within your cluster.
- Define a `CustomResource` for an ExternalSecret that will reference a secret in an external secret manager to be used by OpenTelemetry for authentication when exporting data.

### Prerequisites

Before you start, you need to have:

- A Kubernetes cluster up and running.
- `kubectl` configured to interact with your Kubernetes cluster.
- Pulumi CLI installed and logged in.
- An external secret management service where your actual secrets are stored.

Below is the Pulumi program that sets up OpenTelemetry Collector and External Secrets on Kubernetes:

```typescript
import * as k8s from '@pulumi/kubernetes';

const namespace = "monitoring"; // Namespace where components will be deployed

// OpenTelemetry Collector Deployment
const otelCollectorName = "otel-collector";
const otelCollectorDeployment = new k8s.apps.v1.Deployment(otelCollectorName, {
    metadata: {
        namespace: namespace,
        labels: { app: otelCollectorName },
    },
    spec: {
        replicas: 1,
        selector: { matchLabels: { app: otelCollectorName } },
        template: {
            metadata: { labels: { app: otelCollectorName } },
            spec: {
                containers: [{
                    name: otelCollectorName,
                    image: "otel/opentelemetry-collector:latest", // Use the appropriate collector image
                    // You may need to configure the collector by specifying config volumes, env vars, etc.
                }],
            },
        },
    },
}, { provider: /* your k8s provider here */ });

// External Secrets Custom Resource Definition (CRD)
// Assuming the External Secrets Operator is already installed and set up in your cluster
const externalSecretName = "otel-backend-secret";
const externalSecret = new k8s.apiextensions.CustomResource(externalSecretName, {
    apiVersion: "kubernetes-client.io/v1",
    kind: "ExternalSecret",
    metadata: { 
        namespace: namespace,
    },
    spec: {
        backendType: "secretsManager", // Specify the type of your external secret storage
        data: [{
            key: "external-secret-key-name", // Key name in the external secret store
            name: "internal-secret-name", // Key name within the Kubernetes secret
        }],
        // Specify other properties like roleArn, region, etc., depending on your external secret backend
    },
}, { provider: /* your k8s provider here */, dependsOn: otelCollectorDeployment });

// Export the name of the namespace
export const monitoringNamespace = namespace;
```

### Explanation of the Code

- `@pulumi/kubernetes`: This is the Pulumi Kubernetes SDK, which allows you to define Kubernetes resources using code.
- `otelCollectorDeployment`: Kubernetes deployment for the OpenTelemetry Collector. The collector is responsible for gathering telemetry data. The image and configuration should be adjusted according to your requirements for the OpenTelemetry Collector.
- `externalSecret`: Custom resource of kind `ExternalSecret`, which instructs the External Secrets Operator to fetch the secret from the external secret management service (like AWS Secrets Manager, Azure Key Vault, etc.) and create a Kubernetes secret with the same values.

### Notes

- This program assumes that the necessary CRD (`ExternalSecret`) for the External Secrets Operator is already installed in your cluster. You would need to install the External Secrets Operator beforehand. This can often be done using a package manager like Helm.
- You need to replace the placeholder comment `/* your k8s provider here */` with actual Pulumi Kubernetes provider if you are managing multiple clusters with Pulumi and need to specify a particular one.
- The `apiVersion`, `details on the backendType`, and `spec` for the `ExternalSecret` should align with your actual external secret backend configuration.

Ensure to replace placeholders with real values suited to your cluster and external secrets setup. After you write this code into a `index.ts` file, you can create and manage these Kubernetes resources using Pulumi command line tools:

1. Run `pulumi up` to preview and deploy the changes.
2. Confirm the deployment, and Pulumi will manage the deployment to Kubernetes.

Remember, proper access management and permissions between Pulumi and your Kubernetes cluster, as well as between the Kubernetes cluster and the external secrets backend, are required for this code to work.