Load Balanced AI APIs on DigitalOcean App Platform

Question

Pulumi · Accepted Answer

To deploy a load balanced AI API on DigitalOcean's App Platform using Pulumi, you can utilize the digitalocean.App resource which handles deployments and scaling on the platform. The App Platform automates infrastructure management tasks such as provisioning, scaling, and updating.

Here is a step-by-step guide followed by the Pulumi program to achieve this:

Create a new Python Pulumi project: You'll need to have Pulumi CLI installed. Once installed, you can create a new project using pulumi new python.
Define the App Platform service: We will use the digitalocean.App class from the pulumi_digitalocean provider to define our API service. This class allows you to specify the details of the service such as source code repository, environment variables, and the desired instance size.
Specify the scaling options: In the digitalocean.App service definition, we can define the scaling options that automatically adjust the number of running instances based on traffic.
Add load balancing options: By default, the DigitalOcean App Platform automatically balances the traffic across instances. However, you can customize the load balancing behavior by setting up routes and health checks.
Setting up environment variables: If your API requires any environment variables, they can be defined within the digitalocean.App specification.
Deploy: Once the Pulumi program is ready and the Pulumi stack is configured, execute pulumi up to deploy your AI API to the DigitalOcean App Platform.

Below is a Pulumi program in Python that specifies an application on DigitalOcean's App Platform with load balancing capabilities:

import pulumi
import pulumi_digitalocean as digitalocean

# Define a DigitalOcean App Platform app with a service that runs a container.
# The service will be automatically managed, scaled, and load-balanced by the platform.
# Replace `<your_docker_image>` with your Docker image source.
# Add any required environment variables in the `envs` list.

app = digitalocean.App("ai-api-app",
    spec=digitalocean.AppSpecArgs(
        name="ai-api-app",
        services=[
            digitalocean.AppServiceSpecArgs(
                name="ai-api-service",
                image=digitalocean.AppImageSpecArgs(
                    repository="<your_docker_image>",
                    registry_type="DOCKER_HUB"  # Or another registry type if applicable.
                ),
                # Define custom health checks (optional)
                health_check=digitalocean.AppServiceSpecHealthCheckArgs(
                    http_path="/healthz",  # replace with the health check path
                    check_interval_seconds=10,
                    failure_threshold=5,
                    timeout_seconds=3,
                    healthy_threshold=3,
                ),
                # Define deployment rules and scaling options
                instance_size_slug="basic-xxs",
                instance_count=2,  # start with two instances and scale automatically
            ),
        ],
    ))

# Export the live URL of the app
pulumi.export("live_url", app.live_url)

This program creates a scalable service on DigitalOcean's App Platform that runs a provided Docker image. The health_check sets up a custom health check endpoint, which the platform uses to verify that an instance is healthy. If you don’t provide a health check, DigitalOcean will use default settings.

Make sure you replace <your_docker_image> with the Docker image of your AI API. If you have your image on Docker Hub, it’s the name of the image; if it’s hosted on another registry, provide the full path.

Note: The instance_count parameter is set to 2 to signify that initially, you want two instances of your API running. DigitalOcean App Platform will handle the load balancing automatically across these instances.

Run pulumi up to deploy the AI API on the DigitalOcean App Platform after setting up the program. Once deployed, Pulumi will provide you with a live URL where your API service is accessible.

Remember, before running pulumi up, ensure you've authenticated with DigitalOcean and configured your Pulumi stack with the necessary credentials to communicate with DigitalOcean's API.