Kubernetes Custom Resource Definitions for Model Serving

Question

Pulumi · Accepted Answer

To create a Kubernetes Custom Resource Definition (CRD) for model serving, you would first define the CRD to specify your custom resource, including the schema that the resource will adhere to. This schema serves as a validation mechanism for the custom resources and forms the basis of the RESTful API that Kubernetes will serve.

In Kubernetes, a CRD is a powerful way to extend Kubernetes capabilities with your custom resource that behaves like any standard Kubernetes object. This allows you to define custom controllers that can provide domain-specific handling for your Kubernetes applications.

Pulumi allows you to manage Kubernetes resources in a declarative manner using familiar programming languages. In this case, we'll use Python to define a CRD for model serving. Here's how you can do it with Pulumi:

1. Define the schema for your CRD.
2. Create the CRD using Pulumi's Kubernetes package.
3. Instantiate the custom resources based on the defined CRD.

Below is a Pulumi Python program that defines a simple model serving CRD and creates an instance of it:

```python
import pulumi
import pulumi_kubernetes as k8s

# Define the schema for the Custom Resource Definition for model serving.
model_serving_crd = k8s.apiextensions.v1.CustomResourceDefinition(
    "model-serving-crd",
    metadata=k8s.meta.v1.ObjectMetaArgs(
        name="modelservings.myorg.com"
    ),
    spec=k8s.apiextensions.v1.CustomResourceDefinitionSpecArgs(
        group="myorg.com",
        versions=[k8s.apiextensions.v1.CustomResourceDefinitionVersionArgs(
            name="v1",
            served=True,
            storage=True,
            schema=k8s.apiextensions.v1.CustomResourceValidationArgs(
                openAPIV3Schema=k8s.apiextensions.v1.JSONSchemaPropsArgs(
                    type="object",
                    properties={
                        "spec": k8s.apiextensions.v1.JSONSchemaPropsArgs(
                            type="object",
                            properties={
                                "modelType": k8s.apiextensions.v1.JSONSchemaPropsArgs(
                                    type="string",
                                    description="The type of the model to serve."
                                ),
                                "modelURI": k8s.apiextensions.v1.JSONSchemaPropsArgs(
                                    type="string",
                                    description="The URI of the model to serve."
                                ),
                                "resources": k8s.core.v1.ResourceRequirementsArgs(
                                    requests={"cpu": "500m", "memory": "128Mi"},
                                    limits={"cpu": "1000m", "memory": "256Mi"}
                                )
                            },
                            required=["modelType", "modelURI"]
                        )
                    }
                )
            )
        )],
        scope="Namespaced",
        names=k8s.apiextensions.v1.CustomResourceDefinitionNamesArgs(
            plural="modelservings",
            singular="modelserving",
            kind="ModelServing",
            short_names=["ms"]
        )
    )
)

# Define an instance of the custom resource ModelServing.
model_serving_instance = k8s.apiextensions.CustomResource(
    "example-model-serving",
    api_version="myorg.com/v1",
    kind="ModelServing",
    metadata=k8s.meta.v1.ObjectMetaArgs(
        name="example-model",
        namespace="default"
    ),
    spec={
        "modelType": "TensorFlow",
        "modelURI": "gs://my-model-bucket/tensorflow-model",
        "resources": {
            "requests": {"cpu": "1", "memory": "512Mi"},
            "limits": {"cpu": "2", "memory": "1024Mi"}
        }
    }
)

pulumi.export('custom_resource_definition', model_serving_crd.metadata.name)
pulumi.export('model_serving_instance', model_serving_instance.metadata.name)
```

In this program:
- We define a CRD named `model-serving-crd` with the domain `myorg.com` and version `v1`. It will be identified in Kubernetes by the name `modelservings.myorg.com`.
- The CRD schema specifies that each `ModelServing` custom resource have a `spec` with required `modelType` and `modelURI` properties, and optional `resources` for the Kubernetes resource requirements.
- Then, we create an instance of `ModelServing`, providing values for the required fields and specifying Kubernetes resource requests and limits.

The `pulumi.export` commands output the names of the created resources. Once applied, these resources will be part of your Kubernetes cluster, and you can interact with them using `kubectl` or the Kubernetes API just like any other resource.

Please note that when you run this Pulumi program, you'll need a configured Kubernetes cluster, and Pulumi needs to be set up to connect to it. To apply this configuration to your cluster, you'd run `pulumi up` in the directory where this program is saved. This command initiates the deployment process where Pulumi will create the defined resources in your Kubernetes cluster.