Containerized Model Serving Using ECS with Fargate Capacity Providers

Question

Pulumi · Accepted Answer

To serve a containerized model using AWS ECS with Fargate capacity providers, you'll have to do the following:

1. Set up an ECS Cluster: This is the orchestration service for your containers.
2. Define a Task Definition: Specifies how your containers should run, including the image to use, CPU, and memory requirements.
3. Create a Fargate Service: Runs and maintains a specified number of instances of the task definition.

To get your containerized model up and running on AWS Fargate, I'll guide you through a simple Pulumi program in Python. This program will create all necessary resources mentioned above.

First, we need to import the required Pulumi packages for AWS. Then, we'll start by creating an ECS Cluster, followed by an ECS Task Definition with a container specification. Finally, we'll define an ECS Service which will orchestrate the running of our tasks.

Let's get started with the Pulumi program:

```python
import pulumi
import pulumi_aws as aws

# Create an ECS cluster
cluster = aws.ecs.Cluster("app-cluster")

# Define a role for the ECS Tasks - this role will define the AWS permissions that the running tasks are granted
task_execution_role = aws.iam.Role("task-exec-role", assume_role_policy=(
    """{
        "Version": "2012-10-17",
        "Statement": [{
            "Action": "sts:AssumeRole",
            "Effect": "Allow",
            "Principal": {
                "Service": "ecs-tasks.amazonaws.com"
            }
        }]
    }"""))

# Attach the policy to the execution role created above that allows the ECS tasks to pull images and store logs in CloudWatch
task_exec_policy_attachment = aws.iam.RolePolicyAttachment("task-exec-policy",
    role=task_execution_role.name,
    policy_arn="arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy")

# Model container definition
container_definition = pulumi.Output.all(cluster.name).apply(lambda args: aws.ecs.TaskDefinitionContainerDefinitionArgs(
    name="my-model",
    image="my-model-image", # Replace with your actual Docker image for the model
    cpu=512,
    memory=1024,
    essential=True,
    port_mappings=[
        aws.ecs.TaskDefinitionPortMappingArgs(
            container_port=80,
            host_port=80,
            protocol="tcp"
        )
    ]
))

# Define the ECS task definition with a single container
task_definition = aws.ecs.TaskDefinition("app-task-def",
    family="app-task-family",
    cpu="512",
    memory="1024",
    network_mode="awsvpc",
    requires_compatibilities=["FARGATE"],
    execution_role_arn=task_execution_role.arn,
    container_definitions=container_definition.apply(lambda c: pulumi.Output.all(c).apply(pulumi.json.encode))) # Encodes container definition to JSON format

# Create a Fargate service to run the task definition
service = aws.ecs.Service("app-svc",
    cluster=cluster.arn,
    task_definition=task_definition.arn,
    desired_count=1,
    launch_type="FARGATE",
    network_configuration=aws.ecs.ServiceNetworkConfigurationArgs(
        assign_public_ip=True,
        subnets=["subnet-XXXXXXXX"], # Replace with your actual subnet IDs
        security_groups=["sg-XXXXXXXX"] # Replace with your actual security group IDs
    ))

# Export the cluster name and service name
pulumi.export("cluster_name", cluster.name)
pulumi.export("service_name", service.name)
```

Here's a breakdown of what this program does:

- It creates an ECS cluster using the `aws.ecs.Cluster` class.
- It sets up an IAM role for ECS task execution, including a policy that grants permissions needed by the tasks.
- It defines the task definition including the container that will run your model. You'll need to ensure that the image parameter in the container definition points to the Docker image that contains your model.
- It creates an ECS service with type `FARGATE`, specifying the network configuration that allows the tasks to be accessed through the public internet.

Remember to replace placeholders like `my-model-image`, `subnet-XXXXXXXX`, and `sg-XXXXXXXX` with your actual Docker image name, VPC subnet ids, and security group ids.

Before running this code, make sure you have the Pulumi AWS provider setup and configured with the necessary AWS credentials. To run the program, use the Pulumi CLI and execute `pulumi up` within the directory of your Pulumi project. If you're new to Pulumi, you'll need to first install Pulumi, create a project, and configure the AWS provider. You can follow [Quickstart](https://www.pulumi.com/docs/get-started/aws/) for the instructions to get started.