Continuous Deployment Pipelines for AI Services on Cloud Run

Question

Pulumi · Accepted Answer

To create continuous deployment (CD) pipelines for AI services on Google Cloud Run, you will typically follow a series of steps in your Pulumi program to define the infrastructure and the deployment process. Below I'll guide you through the necessary components and how they fit within a Pulumi program written in Python.

### Google Cloud Run Service

Google Cloud Run is a managed platform that enables you to run stateless containers that are invocable via HTTP requests. It's a great fit for AI services since it abstracts away all the infrastructure management, and you pay only for the compute resources your service consumes while it's running.

In Pulumi, to define a Cloud Run service, you use the `Service` resource provided by the `pulumi_gcp` module. You define the container image to run, along with other configurations such as environment variables, memory limits, and request concurrency settings.

### Continuous Deployment

Continuous Deployment involves automatically deploying your new code changes after they pass through the testing stages in your CI/CD pipeline. To implement CD, you need to integrate your version control system (like GitHub) with your CD platform (like Google Cloud Build or GitHub Actions) to trigger a new deployment whenever new commits are pushed to your main branch.

Pulumi does not directly handle the CD pipeline's runtime behavior (like listening for git commits). Still, it can be used to describe and configure the infrastructure that your CD pipeline will interact with.

### The Pulumi Program

Below is a Pulumi program that will set up a Google Cloud Run service ready for continuous deployment. It doesn't configure your CI/CD pipeline logic but sets up a Cloud Run service to which your CI/CD pipeline can deploy.

In this example, the commented sections provide explanations for each step in the process:

```python
import pulumi
import pulumi_gcp as gcp

# Replace 'PROJECT' and 'LOCATION' with your GCP project id and desired location
project_id = 'PROJECT'
location = 'LOCATION'
service_name = 'my-ai-service'
container_image = 'gcr.io/my-project/my-ai-service'  # Replace with the path to your container image

# Create a Cloud Run service with the given specifications.
cloud_run_service = gcp.cloudrun.Service("my-ai-service",
    location=location,
    project=project_id,
    template=gcp.cloudrun.ServiceTemplateArgs(
        spec=gcp.cloudrun.ServiceTemplateSpecArgs(
            containers=[
                gcp.cloudrun.ServiceTemplateSpecContainerArgs(
                    image=container_image,
                ),
            ],
            # If you need to specify the number of requests a single Cloud Run instance can handle
            # container_concurrency=80,
        ),
    ),
)

# Here we are exporting the URL of the Cloud Run service, which can be used to trigger the AI service.
pulumi.export("service_url", cloud_run_service.statuses[0].url)

```

What this Pulumi program does is to declare a Cloud Run service with a specified container image. Once this infrastructure is deployed with Pulumi, your CD pipeline's responsibility will be to push new container images to the specified GCR repository and then redeploy the Cloud Run service using Pulumi or directly with gcloud CLI or Google Cloud Build triggers.

For a complete CI/CD setup, you will need additional steps:
1. Set up a source repository with your AI service's code and a Dockerfile.
2. Configure a build pipeline (with Google Cloud Build, GitHub Actions, GitLab CI, etc.) to build the container image and push it to Google Container Registry (GCR) on code changes.
3. Configure a deployment pipeline that triggers after the successful image build, which will execute `pulumi up` to apply the new image to your Cloud Run service.

Please remember that you'll need to authenticate Pulumi with Google Cloud for the above code to run successfully. This typically involves setting up a service account with appropriate permissions and then running `pulumi config set gcp:project PROJECT` and `pulumi config set gcp:credentials <KEY_FILE_PATH>` to set your project and credentials for Pulumi to use. Additionally, it's good practice to use Stack References in Pulumi for sharing information between stacks, such as passing the Cloud Run URL from the infrastructure stack to the application stack.

Remember to replace placeholder strings like 'PROJECT', 'LOCATION', and 'gcr.io/my-project/my-ai-service' with actual values appropriate for your setup.