Defining Custom Controllers for Custom Resource operations

Question

Pulumi · Accepted Answer

When you're using Kubernetes with Pulumi, you might find yourself needing to define custom controllers to perform operations on custom resources. Custom controllers are a key part of the Kubernetes ecosystem, allowing users to extend Kubernetes functionality in a way that's native to Kubernetes API constructs.

A common scenario is using the CustomResourceDefinition (CRD) in Kubernetes which allows for the creation of custom objects. Once these CRDs are available, you can create custom controllers that watch for changes to instances of your CRD and perform operations based on those changes. These controllers typically run inside your cluster as a pod and are packaged and deployed alongside any other application code.

Here’s an outline of the steps needed to use Pulumi to define a CustomResourceDefinition and set the stage for a custom controller:

1. Create a new Pulumi project using TypeScript.
2. Define the CustomResourceDefinition using Pulumi’s Kubernetes package.
3. Implement custom logic in the form of a controller that watches for changes to your CRD's instances. (This typically involves writing the controller's code and creating a Deployment for it in the cluster, which is beyond Pulumi's scope and should be written in a language suitable for Kubernetes controllers like Go).

Below is a TypeScript program with Pulumi that defines a simple CustomResourceDefinition. For simplicity, the program will only cover steps one and two. Implementing a custom controller involves writing actual application code, usually in a language like Go using client libraries for Kubernetes like client-go.

```typescript
import * as k8s from '@pulumi/kubernetes';

// Define a CustomResourceDefinition for a CRD named "CronTab".
const cronTabCRD = new k8s.apiextensions.v1.CustomResourceDefinition("cronTabCrd", {
    metadata: {
        // Name of the CustomResourceDefinition
        name: "crontabs.stable.example.com"
    },
    spec: {
        // Group name which is used to logically group the resources
        group: "stable.example.com",
        // Version details and schema for the Custom Resource
        versions: [{
            name: "v1",
            served: true,
            storage: true,
            schema: {
                openAPIV3Schema: {
                    type: "object",
                    properties: {
                        spec: {
                            type: "object",
                            properties: {
                                cronSpec: {
                                    type: "string"
                                },
                                image: {
                                    type: "string"
                                },
                                replicas: {
                                    type: "integer"
                                }
                            }
                        }
                    }
                }
            }
        }],
        // This defines what kind of object the CRD refers to, in this case, "CronTab".
        names: {
            plural: "crontabs",
            singular: "crontab",
            kind: "CronTab",
        },
        // Scope can either be "Namespaced" or "Cluster".
        scope: "Namespaced"
    }
});

// Export the name of the CRD
export const crdName = cronTabCRD.metadata.name;
```

This code uses the `@pulumi/kubernetes` SDK to define a CRD for a made-up resource `CronTab`. The `CronTab` objects' specification includes fields such as `cronSpec`, `image`, and `replicas`. The CRD metadata gives the plural and singular names for the resources, their kind, and their scope within Kubernetes.

The `spec` structure follows the OpenAPI v3 schema, which is used for validation of Custom Resources managed by this CRD. By setting `served` to true and `storage` to true for a version, you're specifying that it is served by the Kubernetes API and stored in etcd.

The `export` at the bottom provides the CRD's name, which could then be used elsewhere if needed.

After running this code with Pulumi, a CRD will be created in your Kubernetes cluster. Following the creation of the CRD, you'd write a custom controller separately to handle the business logic whenever a CronTab resource is created, updated, or deleted.

Creating custom controllers is a more advanced topic that involves setting up a codebase listening for Kubernetes events and acting accordingly. There are frameworks and libraries such as kube-builder and operator-sdk in the Go programming language that can help you in developing custom controllers. With Pulumi, you lay the foundation by defining the resources that your controller will manage.