Configuring persistent storage in Kubernetes for Upsolver stateful data

Question

Pulumi · Accepted Answer

To configure persistent storage in Kubernetes for stateful applications like Upsolver, we use PersistentVolumes (PVs) and PersistentVolumeClaims (PVCs). A `PersistentVolume` is a piece of storage in the cluster that has been provisioned by an administrator or dynamically provisioned using Storage Classes. A `PersistentVolumeClaim` is a request for storage by a user. StatefulSets are the standard way to deploy stateful applications in Kubernetes with a stable and unique network identifier.

Below is a Pulumi TypeScript program that creates a `PersistentVolume`, a `PersistentVolumeClaim`, and a `StatefulSet` to manage stateful applications like Upsolver in Kubernetes.

1. **PersistentVolume** - A volume that outlives the pod and needs to be manually created by a cluster administrator or dynamically provisioned using `StorageClass`.

2. **PersistentVolumeClaim** - A request for storage by a user; pods consume node resources and PVCs consume PV resources.

3. **StatefulSet** - Manages the deployment and scaling of a set of Pods and provides guarantees about the ordering and uniqueness of these Pods.

Here is a step-by-step Pulumi TypeScript program to configure persistent storage for stateful data:

```typescript
import * as k8s from "@pulumi/kubernetes";

// Create a PersistentVolume that will provide storage persisted outside the lifecycle of individual pods.
const persistentVolume = new k8s.core.v1.PersistentVolume("upsolver-storage", {
    // Define the capacity of the PersistentVolume.
    spec: {
        capacity: {
            storage: "10Gi", // Define the size of the storage volume.
        },
        accessModes: ["ReadWriteOnce"], // The access mode of the volume, ReadWriteOnce is commonly used for single pod access.
        persistentVolumeReclaimPolicy: "Retain", // Retain the volume after the claim is deleted, could also be set to Delete or Recycle.
        storageClassName: "standard", // The name of the StorageClass associated with this volume.
        // Define how the volume can be accessed - e.g., NFS, iSCSI, or a cloud provider-specific storage system.
        nfs: {
            path: "/path/to/nfs", // Replace with the path to your NFS directory.
            server: "nfs-server-ip", // Replace with the IP of your NFS server.
        },
    },
    // Metadata for the PersistentVolume, used to define labels, annotations, etc.
    metadata: {
        name: "upsolver-pv",
        labels: { app: "upsolver" },
    },
});

// Create a PersistentVolumeClaim to request a certain size of the PersistentVolume.
const persistentVolumeClaim = new k8s.core.v1.PersistentVolumeClaim("upsolver-pvc", {
    spec: {
        accessModes: ["ReadWriteOnce"], // Must match the access modes of the PersistentVolume.
        resources: {
            requests: {
                storage: "10Gi", // Request a volume of the specified size.
            },
        },
        // Ensure the PersistentVolumeClaim will bind to the specific PersistentVolume.
        volumeName: persistentVolume.metadata.name,
    },
    // Metadata for the PersistentVolumeClaim.
    metadata: {
        name: "upsolver-pvc",
        labels: { app: "upsolver" },
    },
});

// Create a StatefulSet that will use the PersistentVolumeClaim for storage.
const statefulSet = new k8s.apps.v1.StatefulSet("upsolver-statefulset", {
    spec: {
        serviceName: "upsolver", // The name of the service that gets created for this StatefulSet.
        replicas: 1, // The number of desired StatefulSet replicas.
        selector: { matchLabels: { app: "upsolver" } },
        template: { // Template for the pods controlled by the StatefulSet.
            metadata: { labels: { app: "upsolver" } },
            spec: {
                containers: [{
                    name: "upsolver",
                    image: "upsolver-image", // The container image to use for the Upsolver application.
                    ports: [{ containerPort: 80 }], // The ports that the container will listen on.
                    volumeMounts: [{ // Mount the PersistentVolumeClaim into the pod.
                        name: "upsolver-storage",
                        mountPath: "/data", // The path where the volume should be mounted inside the pod.
                    }],
                }],
                volumes: [{ // Reference the PersistentVolumeClaim to be used by the pod.
                    name: "upsolver-storage",
                    persistentVolumeClaim: {
                        claimName: persistentVolumeClaim.metadata.name,
                    },
                }],
            },
        },
    },
    // Metadata for the StatefulSet.
    metadata: {
        name: "upsolver-statefulset",
        labels: { app: "upsolver" },
    },
});

// Export the name of the StatefulSet
export const statefulSetName = statefulSet.metadata.name;
```

In this program, we created a `PersistentVolume` backed by NFS (which needs to be pre-configured and running in your environment). We also created a `PersistentVolumeClaim` that requests this storage and a `StatefulSet` to deploy an Upsolver application that uses this persistent storage claim.

The key to note here is the volumeName in PersistentVolumeClaim that is set to the name of the PersistentVolume. This ensures that the PVC binds to the correct PV. The `StatefulSet` uses the claim through the volumes section in the pod template.

To apply this Pulumi program, you would run `pulumi up` using the Pulumi CLI after setting up your Pulumi project and installing necessary npm packages like `@pulumi/kubernetes`. This command will provision the resources defined above in your currently configured Kubernetes cluster.