1. Containerized Model Serving on Red Hat OpenShift


    To containerize a model for serving on Red Hat OpenShift, you need to take several steps, which include containerizing your application, creating a deployment configuration for OpenShift, and exposing the application via a route so it can be accessed from outside the OpenShift cluster. Below you’ll find a Pulumi program written in Python that accomplishes the setup of a containerized model serving on Azure Red Hat OpenShift.


    1. An Azure account with permissions to create resources.
    2. Pulumi CLI installed and configured with an appropriate Azure subscription.
    3. pulumi-azure-native and pulumi-docker packages installed for Pulumi with Python.
    4. Docker installed for building and pushing your container image.
    5. Your machine learning model ready to be containerized.
    6. Your Red Hat OpenShift cluster already up and running on Azure.

    Containerizing the Model

    Before we dive into the Pulumi code, your model needs to be containerized. This typically involves:

    • Writing a Dockerfile that specifies how to build the container image, including installing dependencies, copying your model to the image, and defining the command or process that runs when the container starts.
    • Building the docker image and pushing it to a container registry, such as Azure Container Registry (ACR) or Docker Hub.

    Here's an example of what a Dockerfile might look like:

    # Use an official Python runtime as a base image FROM python:3.8-slim # Set the working directory in the container WORKDIR /usr/src/app # Copy the current directory contents into the container at /usr/src/app COPY . . # Install any needed packages specified in requirements.txt RUN pip install --no-cache-dir -r requirements.txt # Make port 80 available to the world outside this container EXPOSE 80 # Define environment variable ENV NAME World # Run app.py when the container launches CMD ["python", "./app.py"]

    The Pulumi Program

    Next, here's a simplified Pulumi program that sets up the necessary Azure Red Hat OpenShift (ARO) resources. The code does not include the actual model serving application but assumes you have a container image ready for deployment.

    import pulumi import pulumi_azure_native as azure_native # Configurations for the OpenShift deployment image = "your-container-registry/your-image" # Replace with your container image app_name = "model-serving-app" # Name of the application resource_group_name = "myResourceGroup" # Replace with your resource group name # Fetch the OpenShift cluster resource from Azure open_shift_cluster = azure_native.redhatopenshift.get_open_shift_cluster( resource_group_name=resource_group_name, resource_name="myCluster" # Replace with your cluster name ) # Define an Azure Container Registry to store the image if not already present acr = azure_native.containerregistry.Registry("myContainerRegistry", resource_group_name=resource_group_name, sku=azure_native.containerregistry.SkuArgs( name="Basic" ), ) # Build and Push to Azure Container Registry (in reality, you would do this outside of Pulumi) # Here, we are simply demonstrating how you might define a custom resource for building and # pushing a container image using the `pulumi-docker` package. import pulumi_docker as docker docker.Image("model-serving-image", image_name=acr.login_server.apply(lambda server: f"{server}/{app_name}:v1"), build=docker.DockerBuild(context="path/to/Dockerfile/folder"), registry=docker.ImageRegistry( server=acr.login_server, username=acr.admin_username, password=acr.admin_password, ), ) # Export the container registry URL pulumi.export('container_registry_url', acr.login_server)

    Steps in the Program:

    1. Define Container Image: We have a placeholder for the image that needs to be deployed. You would replace 'your-container-registry/your-image' with the path to your image in a container registry.

    2. Configure OpenShift Cluster & ACR: We fetch an existing OpenShift cluster (azure_native.redhatopenshift.get_open_shift_cluster) from Azure and create an Azure Container Registry where our container image will reside.

    3. Build & Push Container Image: The Docker image is defined using pulumi_docker.Image. This part assumes you have a folder containing your Dockerfile and all necessary code. The code will build the image and push it to the newly created ACR. In practice, this usually happens in a CI/CD pipeline outside of your infrastructure code. Here we illustrate the concept.

    Deploying the Containerized Model to OpenShift

    Deploying your application on OpenShift typically involves creating an Application resource that references your container image. This could be done using the kubectl tool, OpenShift CLI (oc), or through clients that talk to the OpenShift API.

    As of my knowledge cutoff in 2023, direct management of OpenShift resources through Pulumi is limited to fetching existing clusters and using custom resources for specific OpenShift-related tasks within Azure. For deeper OpenShift management such as defining DeploymentConfigs, Services, or Routes, you would need to use kubectl or oc.


    The provided Pulumi program helps you to set up an Azure Container Registry and build & push your containerized model serving application image. You would then complete the setup by deploying the application on an OpenShift cluster using the OpenShift tools and referring to the image stored in ACR.