1. Kubernetes Custom Resource Definitions for Model Serving


    To create a Kubernetes Custom Resource Definition (CRD) for model serving, you would first define the CRD to specify your custom resource, including the schema that the resource will adhere to. This schema serves as a validation mechanism for the custom resources and forms the basis of the RESTful API that Kubernetes will serve.

    In Kubernetes, a CRD is a powerful way to extend Kubernetes capabilities with your custom resource that behaves like any standard Kubernetes object. This allows you to define custom controllers that can provide domain-specific handling for your Kubernetes applications.

    Pulumi allows you to manage Kubernetes resources in a declarative manner using familiar programming languages. In this case, we'll use Python to define a CRD for model serving. Here's how you can do it with Pulumi:

    1. Define the schema for your CRD.
    2. Create the CRD using Pulumi's Kubernetes package.
    3. Instantiate the custom resources based on the defined CRD.

    Below is a Pulumi Python program that defines a simple model serving CRD and creates an instance of it:

    import pulumi import pulumi_kubernetes as k8s # Define the schema for the Custom Resource Definition for model serving. model_serving_crd = k8s.apiextensions.v1.CustomResourceDefinition( "model-serving-crd", metadata=k8s.meta.v1.ObjectMetaArgs( name="modelservings.myorg.com" ), spec=k8s.apiextensions.v1.CustomResourceDefinitionSpecArgs( group="myorg.com", versions=[k8s.apiextensions.v1.CustomResourceDefinitionVersionArgs( name="v1", served=True, storage=True, schema=k8s.apiextensions.v1.CustomResourceValidationArgs( openAPIV3Schema=k8s.apiextensions.v1.JSONSchemaPropsArgs( type="object", properties={ "spec": k8s.apiextensions.v1.JSONSchemaPropsArgs( type="object", properties={ "modelType": k8s.apiextensions.v1.JSONSchemaPropsArgs( type="string", description="The type of the model to serve." ), "modelURI": k8s.apiextensions.v1.JSONSchemaPropsArgs( type="string", description="The URI of the model to serve." ), "resources": k8s.core.v1.ResourceRequirementsArgs( requests={"cpu": "500m", "memory": "128Mi"}, limits={"cpu": "1000m", "memory": "256Mi"} ) }, required=["modelType", "modelURI"] ) } ) ) )], scope="Namespaced", names=k8s.apiextensions.v1.CustomResourceDefinitionNamesArgs( plural="modelservings", singular="modelserving", kind="ModelServing", short_names=["ms"] ) ) ) # Define an instance of the custom resource ModelServing. model_serving_instance = k8s.apiextensions.CustomResource( "example-model-serving", api_version="myorg.com/v1", kind="ModelServing", metadata=k8s.meta.v1.ObjectMetaArgs( name="example-model", namespace="default" ), spec={ "modelType": "TensorFlow", "modelURI": "gs://my-model-bucket/tensorflow-model", "resources": { "requests": {"cpu": "1", "memory": "512Mi"}, "limits": {"cpu": "2", "memory": "1024Mi"} } } ) pulumi.export('custom_resource_definition', model_serving_crd.metadata.name) pulumi.export('model_serving_instance', model_serving_instance.metadata.name)

    In this program:

    • We define a CRD named model-serving-crd with the domain myorg.com and version v1. It will be identified in Kubernetes by the name modelservings.myorg.com.
    • The CRD schema specifies that each ModelServing custom resource have a spec with required modelType and modelURI properties, and optional resources for the Kubernetes resource requirements.
    • Then, we create an instance of ModelServing, providing values for the required fields and specifying Kubernetes resource requests and limits.

    The pulumi.export commands output the names of the created resources. Once applied, these resources will be part of your Kubernetes cluster, and you can interact with them using kubectl or the Kubernetes API just like any other resource.

    Please note that when you run this Pulumi program, you'll need a configured Kubernetes cluster, and Pulumi needs to be set up to connect to it. To apply this configuration to your cluster, you'd run pulumi up in the directory where this program is saved. This command initiates the deployment process where Pulumi will create the defined resources in your Kubernetes cluster.