Automated Certificate Renewal for AI APIs on DigitalOcean Kubernetes

Question

Pulumi · Accepted Answer

In this Pulumi program, we will set up a Kubernetes cluster in DigitalOcean and configure cert-manager, an open-source tool used in Kubernetes to automate the management and renewal of TLS certificates. Cert-manager will handle the certificates for AI APIs, ensuring they are always valid and up-to-date without manual intervention.

### What we will do

1. **Create a DigitalOcean Kubernetes Cluster**: We will set up a Kubernetes cluster where our AI APIs will be hosted. DigitalOcean provides managed Kubernetes clusters that are easy to set up and scale.
2. **Configure cert-manager**: After setting up the cluster, we will deploy cert-manager to our Kubernetes cluster. Cert-manager will automate the certificate management within our Kubernetes cluster.

### Pre-requisites

- Make sure you have Pulumi CLI installed and configured for use with DigitalOcean.
- Make sure you have `kubectl` installed, as it is required to manage Kubernetes clusters.
- Ensure you have a DigitalOcean API token with the necessary permissions to create and manage Kubernetes clusters.

### The Pulumi Program

Below is the Python Pulumi program that accomplishes the above steps:

```python
import pulumi
import pulumi_digitalocean as digitalocean
import pulumi_kubernetes as kubernetes
from pulumi_kubernetes.helm.v3 import Chart, ChartOpts
from pulumi_kubernetes.apiextensions import CustomResource
from pulumi_kubernetes.yaml import ConfigGroup

# Create a DigitalOcean Kubernetes Cluster
cluster = digitalocean.KubernetesCluster(
    "do-cluster",
    region="nyc3",
    version="1.22.5-do.0",
    node_pool={
        "name": "default",
        "size": "s-2vcpu-2gb",
        "nodeCount": 2,
    }
)

# Export the cluster's kubeconfig
pulumi.export('kubeconfig', cluster.kube_configs[0]['rawConfig'])

# Set up a Kubernetes provider using the DigitalOcean cluster's kubeconfig
k8s_provider = pulumi.ProviderResource(
    "k8s-provider",
    kubeconfig=cluster.kube_configs[0]['rawConfig'],
)

# Install cert-manager using Helm
cert_manager_chart = Chart(
    "cert-manager",
    ChartOpts(
        chart="cert-manager",
        version="v1.5.3",
        fetch_opts={
            "repo": "https://charts.jetstack.io"
        },
        namespace="cert-manager",
        values={
            "installCRDs": True,
        },
    ),
    opts=pulumi.ResourceOptions(provider=k8s_provider),
)

# After deploying cert-manager, we could set up a ClusterIssuer or Issuer
# to start handling certificates for our services. This can be done by
# defining Certificate and Issuer resources as shown below.

# Deploy a ClusterIssuer using a previously created CustomResource (example)
# Note: Before deploying, make sure you have the proper DNS setup and it's
# accessible by cert-manager for the validations.
cluster_issuer = CustomResource(
    "letsencrypt-issuer",
    api_version="cert-manager.io/v1",
    kind="ClusterIssuer",
    metadata={"name": "letsencrypt-staging"},
    spec={
        "acme": {
            "server": "https://acme-staging-v02.api.letsencrypt.org/directory",
            "email": "your-email@example.com",
            "privateKeySecretRef": {
                "name": "letsencrypt-private-key"
            },
            "solvers": [
                {
                    "http01": {
                        "ingress": {
                            "class": "nginx",
                        }
                    }
                }
            ]
        }
    },
    opts=pulumi.ResourceOptions(provider=k8s_provider),
)

# Ensure the cert-manager Chart deployment is ready before creating any issuers
pulumi.output(cert_manager_chart.ready).apply(
    lambda is_ready: cluster_issuer if is_ready else None)

# Now, any certificates issued for services in your cluster will be
# automatically renewed by cert-manager. You can use the ClusterIssuer
# we created here as part of a Certficate resource to manage TLS for specific services.

```

### Explanation

1. **DigitalOcean Kubernetes Cluster**: We define a `KubernetesCluster` resource with the desired region, version, and node pool configuration. This will create a new Kubernetes cluster in your DigitalOcean account.

2. **Kubernetes Provider**: We set up a Pulumi Kubernetes provider that uses the kubeconfig of the newly created DigitalOcean cluster to interact with it.

3. **cert-manager Installation**: We use the Pulumi Kubernetes Helm Chart resource to deploy cert-manager to our cluster. The `installCRDs` value is set to true to ensure that cert-manager’s Custom Resource Definitions (CRDs) are also installed.

4. **ClusterIssuer**: The ClusterIssuer is a cert-manager resource that represents a certificate authority from which to request certificates. This example uses Let's Encrypt’s staging environment for demonstration purposes. You would switch to the production environment for actual deployments by changing the ACME server URL.

5. **Certificate Renewal**: Once you have the ClusterIssuer in place, you can define Certificate resources for each service that needs TLS. Cert-manager automatically renews certificates before they expire and ensures that your services are always using valid certificates.

### Stack Exports

At the end of the Pulumi program, we export the kubeconfig of the cluster, which we'll need for `kubectl` or other Kubernetes tooling to interact with our cluster.

After setting up the resources with Pulumi, you will be able to use the Kubernetes cluster for hosting your AI APIs and let cert-manager handle the certificates lifecycle automatically.