1. Containerized Model Serving Using ECS with Fargate Capacity Providers


    To serve a containerized model using AWS ECS with Fargate capacity providers, you'll have to do the following:

    1. Set up an ECS Cluster: This is the orchestration service for your containers.
    2. Define a Task Definition: Specifies how your containers should run, including the image to use, CPU, and memory requirements.
    3. Create a Fargate Service: Runs and maintains a specified number of instances of the task definition.

    To get your containerized model up and running on AWS Fargate, I'll guide you through a simple Pulumi program in Python. This program will create all necessary resources mentioned above.

    First, we need to import the required Pulumi packages for AWS. Then, we'll start by creating an ECS Cluster, followed by an ECS Task Definition with a container specification. Finally, we'll define an ECS Service which will orchestrate the running of our tasks.

    Let's get started with the Pulumi program:

    import pulumi import pulumi_aws as aws # Create an ECS cluster cluster = aws.ecs.Cluster("app-cluster") # Define a role for the ECS Tasks - this role will define the AWS permissions that the running tasks are granted task_execution_role = aws.iam.Role("task-exec-role", assume_role_policy=( """{ "Version": "2012-10-17", "Statement": [{ "Action": "sts:AssumeRole", "Effect": "Allow", "Principal": { "Service": "ecs-tasks.amazonaws.com" } }] }""")) # Attach the policy to the execution role created above that allows the ECS tasks to pull images and store logs in CloudWatch task_exec_policy_attachment = aws.iam.RolePolicyAttachment("task-exec-policy", role=task_execution_role.name, policy_arn="arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy") # Model container definition container_definition = pulumi.Output.all(cluster.name).apply(lambda args: aws.ecs.TaskDefinitionContainerDefinitionArgs( name="my-model", image="my-model-image", # Replace with your actual Docker image for the model cpu=512, memory=1024, essential=True, port_mappings=[ aws.ecs.TaskDefinitionPortMappingArgs( container_port=80, host_port=80, protocol="tcp" ) ] )) # Define the ECS task definition with a single container task_definition = aws.ecs.TaskDefinition("app-task-def", family="app-task-family", cpu="512", memory="1024", network_mode="awsvpc", requires_compatibilities=["FARGATE"], execution_role_arn=task_execution_role.arn, container_definitions=container_definition.apply(lambda c: pulumi.Output.all(c).apply(pulumi.json.encode))) # Encodes container definition to JSON format # Create a Fargate service to run the task definition service = aws.ecs.Service("app-svc", cluster=cluster.arn, task_definition=task_definition.arn, desired_count=1, launch_type="FARGATE", network_configuration=aws.ecs.ServiceNetworkConfigurationArgs( assign_public_ip=True, subnets=["subnet-XXXXXXXX"], # Replace with your actual subnet IDs security_groups=["sg-XXXXXXXX"] # Replace with your actual security group IDs )) # Export the cluster name and service name pulumi.export("cluster_name", cluster.name) pulumi.export("service_name", service.name)

    Here's a breakdown of what this program does:

    • It creates an ECS cluster using the aws.ecs.Cluster class.
    • It sets up an IAM role for ECS task execution, including a policy that grants permissions needed by the tasks.
    • It defines the task definition including the container that will run your model. You'll need to ensure that the image parameter in the container definition points to the Docker image that contains your model.
    • It creates an ECS service with type FARGATE, specifying the network configuration that allows the tasks to be accessed through the public internet.

    Remember to replace placeholders like my-model-image, subnet-XXXXXXXX, and sg-XXXXXXXX with your actual Docker image name, VPC subnet ids, and security group ids.

    Before running this code, make sure you have the Pulumi AWS provider setup and configured with the necessary AWS credentials. To run the program, use the Pulumi CLI and execute pulumi up within the directory of your Pulumi project. If you're new to Pulumi, you'll need to first install Pulumi, create a project, and configure the AWS provider. You can follow Quickstart for the instructions to get started.