Using kubernetes deployment with machinelearning.seldon.io

Question

Pulumi · Accepted Answer

To use Kubernetes for deploying a machine learning model with Seldon, you need to have a Seldon Deployment that specifies the model you want to serve. Seldon is an open-source platform that allows you to deploy, scale, and manage thousands of machine learning models on Kubernetes.

Assuming you already have a Docker image for your machine learning model, the Seldon Deployment would be defined using a Kubernetes Custom Resource (CR) provided by Seldon. This Custom Resource would define the components necessary for serving the model, such as the predictors, the graph of computations, and the service orchestrator.

Below is a Pulumi TypeScript program that demonstrates how to deploy a machine learning model using Seldon on a Kubernetes cluster. For this example, let's say we have a Docker image called `my-model:v1.0` that serves predictions over HTTP.

Firstly, ensure you have the Seldon CRD (Custom Resource Definition) installed on your cluster. This CRD defines the Seldon Deployment schema. You would usually install Seldon Core Operator in your cluster which would include the CRD, using Helm or directly applying YAML files provided by Seldon.

Here's the Pulumi program that describes a basic Seldon Deployment resource:

```typescript
import * as k8s from "@pulumi/kubernetes";

// Your machine learning model Docker image
const dockerImage = "my-model:v1.0";

// Create a Seldon Deployment resource
const seldonDeployment = new k8s.apiextensions.CustomResource("seldon-deployment", {
    apiVersion: "machinelearning.seldon.io/v1",
    kind: "SeldonDeployment",
    metadata: {
        name: "my-model-deployment",
        labels: {
            app: "seldon",
        },
    },
    spec: {
        name: "my-model",
        predictors: [
            {
                name: "model-predictor",
                replicas: 1,
                componentSpecs: [{
                    spec: {
                        containers: [
                            {
                                name: "model-container",
                                image: dockerImage,
                                resources: { requests: { cpu: "0.5", memory: "1Gi" }},
                            },
                        ],
                    },
                }],
                graph: {
                    name: "model-container",
                    type: "MODEL",
                    endpoint: { type: "REST" },
                },
                svcOrchSpec: {
                    env: [{
                        name: "SELDON_LOG_LEVEL",
                        value: "INFO",
                    }],
                },
            },
        ],
    },
});

// Export the Seldon Deployment name
export const seldonDeploymentName = seldonDeployment.metadata.name;
```

In the above code, we start by importing the Kubernetes Pulumi package.

We then declare a `CustomResource` using the Pulumi Kubernetes SDK, specifying the required `apiVersion`, `kind`, and `metadata`. The `spec` object defines the configuration of our Seldon Deployment:

- `name`: A unique name for the Seldon Deployment.
- `predictors`: The spec for each predictor, which serves the model. Here, we have a single predictor.
  - `name`: The name of the predictor.
  - `replicas`: The number of replicas to scale our model to. Setting it to `1` means a single pod will be running.
  - `componentSpecs`: Defines the specification for our model component(s).
    - `containers`: The container in which our model is running. We specify its name and Docker image.
      - `resources`: The resource requests for our container. We request `0.5` CPU and `1Gi` of memory.
  - `graph`: Describes how requests are routed to the containers.
    - `name`: The name of the container as specified in `containers`.
    - `type`: The type of the component in the graph, which is a `MODEL`.
    - `endpoint`: The type of endpoint, here we use `REST`.

- `svcOrchSpec`: The specifications for the service orchestrator. We're setting the log level to `INFO`.

Please ensure that you have the necessary permissions to create such resources on your Kubernetes cluster and that the Seldon Core Operator is installed properly for the custom resource to be recognized.

This configuration assumes you have a single container serving a model. If you have a more complex setup, such as a model ensemble, you might have multiple containers and a more complex graph specified under `graph`.

Given that Pulumi observes real cloud resources, running the above code with Pulumi would create a SeldonDeployment in your Kubernetes cluster, which orchestrates the deployment of your machine learning model.