Containerized Machine Learning Model Deployment

Question

Pulumi · Accepted Answer

In order to deploy a containerized machine learning model, we generally need to perform the following steps:

1. **Prepare the model**: Train a machine learning model and make sure it's saved in a way that can be loaded into a container.
2. **Containerize the model**: Create a Docker container image with the necessary environment to run the model.
3. **Push the image to a registry**: Upload the container image to a container registry, so it can be pulled from there for deployment.
4. **Deploy the container**: Run the container in a cloud environment, such as AWS, Azure, or GCP, and expose it as a service.

For the purposes of this walkthrough, let's assume you have already trained your machine learning model, and you have a Docker image containing your model ready to be deployed. We'll focus on how to deploy this image using a service on Azure called Azure Machine Learning (Azure ML).

We'll use Pulumi to define and deploy an `Azure ML Compute` instance to host the containerized model, which allows us to scale the model service as needed. Here's how you could do it with Pulumi in Python:

```python
import pulumi
import pulumi_azure_native as azure_native

# Define a resource group where all our resources will be created.
resource_group = azure_native.resources.ResourceGroup('ml-resource-group')

# Specify the details of the machine learning workspace.
# The workspace is the top-level resource for Azure Machine Learning, providing
# a centralized place to work with all the artifacts you create.
ml_workspace = azure_native.machinelearningservices.Workspace(
    'ml-workspace',
    resource_group_name=resource_group.name,
    location=resource_group.location,
    sku=azure_native.machinelearningservices.SkuArgs(name="Standard"),
    identity=azure_native.machinelearningservices.IdentityArgs(type="SystemAssigned"),
)

# Define the Azure ML Compute instance where the machine learning model will be deployed.
# This compute target can be used to host your model as a web service.
ml_compute = azure_native.machinelearningservices.Compute(
    'ml-compute',
    resource_group_name=resource_group.name,
    compute_name="gpu-compute",
    location=resource_group.location,
    compute_type="AmlCompute",  # A managed compute resource for training and inference workloads.
    properties=azure_native.machinelearningservices.ComputePropertiesArgs(
        vm_size="STANDARD_NC6",  # An example VM size. Adjust according to your model's requirements.
        scale_settings=azure_native.machinelearningservices.ScaleSettingsArgs(
            max_node_count=1,
        ),
        vm_priority="LowPriority",  # To minimize cost, using low-priority VM.
    ),
    workspace_name=ml_workspace.name,
)

# Output the primary key of the workspace, which is necessary for interacting
# with the Azure Machine Learning SDK or REST API.
primary_key_output = pulumi.Output.secret(ml_workspace.list_workspace_keys().apply(lambda keys: keys.keys[0].value))

# When you're ready to deploy the actual model, you'll need to make use of the AzureML Python SDK
# to register the model, create a container image for it, and finally deploy that image to the compute we've just created.

# For now, let's export the ID of the Compute resource.
pulumi.export('compute_id', ml_compute.id)
pulumi.export('primary_key', primary_key_output)
```

This program sets up an Azure ML workspace and a compute instance within that workspace where your model will be served. The Azure ML workspace groups all related resources for an experiment, including datasets, notebooks, models, runs, and deployments.

The program doesn't deploy the model itself, as this process typically requires using the Azure ML SDK or the REST API to handle tasks such as registering the model, setting up an InferenceConfig, and deploying the web service. These steps are best performed outside of Pulumi, using the Azure ML SDK, which provides more granular control over the deployment steps and allows for more complex configurations.

Depending on your model and requirements, you could scale up or down the compute resources, specify different VM sizes, and specify other advanced settings. It’s recommended to use a Managed Identity with `SystemAssigned` for Azure ML workspace for better security as it avoids the need to handle Azure Principal credentials manually.

After this infrastructure is deployed with Pulumi, you would then typically use the Azure Machine Learning SDK in Python as a next step to register the Docker image containing your model with the workspace, create a scoring script, create an environment file with necessary dependencies, and deploy the machine learning model as a web service on Azure Kubernetes Service (AKS), or as an Azure Container Instance (ACI) for lightweight, isolated duties.

Lastly, we export the compute ID for later reference, and the primary key of the workspace, which we mark as a secret since it should not be exposed in plain text. This key is needed to interact with your deployed services programmatically.