Scalable AI APIs with Kong and Docker

Question

Pulumi · Accepted Answer

Scalable AI APIs often require a robust API gateway to manage requests and Docker to containerize and deploy the individual AI services. Kong is a popular open-source API Gateway and Microservice Management layer. Docker is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers.

To set up scalable AI APIs using Kong and Docker, you will need a Docker environment where you can deploy Kong, and a method to orchestrate and scale your Docker containers. One common approach for orchestration is to use a container orchestration platform such as Kubernetes, which can manage and scale your containers according to the workload.

In this Pulumi program, we will create a basic setup for running Kong in a Docker container on a local machine. This setup can be extended to deploy on any cloud provider that supports managed Kubernetes services or Docker hosts. For a production setup, you would need a more complex configuration, possibly including a managed Kubernetes service, a load balancer, more complex networking, and persistent storage.

The below Pulumi Python program creates a Docker service with Kong:

1. **Docker Image**: We pull the official Kong image from Docker Hub.
2. **Docker Service**: We set up a Docker service to run the Kong container.
3. **Networking**: We define a Docker network for Kong to enable communication with other services.
4. **Volumes**: Persistent volumes for Kong to store data across multiple sessions and scalability.

Let's write the Pulumi program:

```python
import pulumi
import pulumi_docker as docker

# Define a Docker network for the Kong service.
network = docker.Network("network", name="kong-net")

# Define a volume for the Kong service to persist data.
volume = docker.Volume("volume", name="kong-data")

# Pull the latest Kong Docker image.
kong_image = docker.RemoteImage("kong-image",
    name="kong:latest")

# Create a Docker service for Kong.
kong_service = docker.Service("kong-service",
    name="kong",
    task_spec=docker.ServiceTaskSpecArgs(
        container_spec=docker.ServiceTaskSpecContainerSpecArgs(
            image=kong_image.repo_digest,
            env=[
                "KONG_DATABASE=off",
                "KONG_DECLARATIVE_CONFIG=/etc/kong/kong.yml",
                "KONG_PROXY_ACCESS_LOG=/dev/stdout",
                "KONG_ADMIN_ACCESS_LOG=/dev/stdout",
                "KONG_PROXY_ERROR_LOG=/dev/stderr",
                "KONG_ADMIN_ERROR_LOG=/dev/stderr",
                "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl",
            ],
            mounts=[docker.ServiceTaskSpecContainerSpecMountArgs(
                type="volume",
                source=volume.name,
                target="/etc/kong"
            )],
        ),
    ),
    networks=[docker.ServiceNetworkArgs(
        target=network.name,
    )],
    endpoint_spec=docker.ServiceEndpointSpecArgs(
        ports=[docker.ServiceEndpointSpecPortArgs(
            published_port=8000,
            target_port=8000,
            protocol="tcp",
        ), docker.ServiceEndpointSpecPortArgs(
            published_port=8443,
            target_port=8443,
            protocol="tcp",
        ), docker.ServiceEndpointSpecPortArgs(
            published_port=8001,
            target_port=8001,
            protocol="tcp",
        ), docker.ServiceEndpointSpecPortArgs(
            published_port=8444,
            target_port=8444,
            protocol="tcp",
        )]
    ))

# Export the Kong service name and endpoints.
pulumi.export("service_name", kong_service.name)
pulumi.export("service_id", kong_service.id)
pulumi.export("service_endpoint", pulumi.Output.concat("http://localhost:", kong_service.endpoint[0].ports[0].published_port))
```

In this program:
- We define a `docker.Network` and `docker.Volume` to isolate our Kong service and allow it to persist configuration data.
- We import the official Kong image using `docker.RemoteImage`.
- We create a `docker.Service`, which is the component responsible for running our containers. Within this service:
    - We specify the environment variables required by Kong to operate in a DB-less mode. This is for simplicity's sake and wouldn't be appropriate for all use cases.
    - We define a mount point using the created volume to store Kong's configuration files.
    - We expose the appropriate ports (8000, 8443 for proxy traffic, and 8001, 8444 for admin traffic).
- We export the service's name and endpoint so we can interact with it after deployment.

To proceed with this Pulumi program, you would need Docker installed on your machine or your cloud environment, and Pulumi CLI set up to run Pulumi programs. When you run this program with `pulumi up`, it would create these resources on your local Docker instance. For cloud deployments, you need to define appropriate providers and resources that target your cloud of choice.