Centralized AI Model Repository on AWS ECR

Question

Pulumi · Accepted Answer

To create a centralized AI Model Repository on AWS using Amazon Elastic Container Registry (ECR), you would need to establish a repository for storing container images, ensure thorough image scanning for vulnerabilities, and configure appropriate access policies.

Here's what each Pulumi resource does in this context:

- `aws.ecr.Repository`: Creates an ECR repository to store Docker container images. These images can be of your AI models which are containerized.
- `aws.ecr.RepositoryPolicy`: Manages access control to the ECR Repository by using IAM policies.
- `aws.ecr.ReplicationConfiguration`: Configures replication so that your images can be synchronized across multiple AWS regions for high availability or other reasons.

Now, let's set up an ECR repository, apply a repository policy to manage access, and configure replication for availability in multiple regions. Below is the Pulumi program in Python that achieves these goals:

```python
import pulumi
import pulumi_aws as aws

# Create an ECR Repository to store AI models
ai_model_repo = aws.ecr.Repository("aiModelRepo",
    image_scanning_configuration=aws.ecr.RepositoryImageScanningConfigurationArgs(
        scan_on_push=True, # Enable scanning of images on push for vulnerabilities
    ),
    image_tag_mutability="MUTABLE", # Allows tags to be overwritten
)

# Define a repository policy - replace 'policy_json' with your actual policy
repo_policy = {
    "Version": "2012-10-17",
    "Statement": [{
        "Sid": "AllowPushPull",
        "Effect": "Allow",
        "Principal": "*", # Define principals appropriately
        "Action": [
            "ecr:GetDownloadUrlForLayer",
            "ecr:BatchGetImage",
            "ecr:BatchCheckLayerAvailability",
            "ecr:PutImage",
            "ecr:InitiateLayerUpload",
            "ecr:UploadLayerPart",
            "ecr:CompleteLayerUpload",
        ],
        "Resource": f"{ai_model_repo.arn}",
    }]
}

# Apply the repository policy to the ECR Repository
ai_model_repo_policy = aws.ecr.RepositoryPolicy("aiModelRepoPolicy",
    policy=pulumi.Output.from_input(repo_policy).apply(pulumi.json.encode),
    repository=ai_model_repo.name,
)

# Replication configuration example to replicate to us-east-1
replication_config = aws.ecr.ReplicationConfiguration("aiModelRepoReplication",
    replication_configuration=aws.ecr.ReplicationConfigurationReplicationConfigurationArgs(
        rules=[
            aws.ecr.ReplicationConfigurationReplicationConfigurationRuleArgs(
                destinations=[
                    aws.ecr.ReplicationConfigurationReplicationConfigurationRuleDestinationArgs(
                        region="us-east-1",
                        registry_id=aws.get_caller_identity().account_id,
                    ),
                ],
            ),
        ],
    ),
    repository=ai_model_repo.name,
)

# Export the repository URL
pulumi.export("repository_url", ai_model_repo.repository_url)
# Export the replication status
pulumi.export("replication_status", replication_config.status)
```

This Pulumi program performs the following actions:

1. It creates an ECR repository named `aiModelRepo`, which will host your AI model images.
2. It enables scanning on pushing the images to the repository, a security best practice.
3. It sets the image tag mutability to 'MUTABLE', which allows image tags to be overwritten; depending on your workflow, you may choose 'IMMUTABLE' to prevent overwriting.
4. It defines a repository policy to control access to the repository operations such as uploading and downloading images. The example policy grants wide permissions for example purposes; you should restrict the `Principal` to relevant IAM entities in your environment.
5. It applies the repository policy to your ECR repository.
6. It sets up a replication configuration to replicate the images to another AWS region, providing higher availability and reducing latency for users in different geographic areas. In this example, we replicate to the `us-east-1` region, and you would use your account ID.
7. Finally, the program exports the repository URL and the replication status as outputs. The repository URL can be used in your CI/CD pipelines to push or pull images, and the replication status provides insight into the replication configuration's operational status.

Replace `'policy_json'` with your actual IAM policy JSON. Ensure the `Principal` is set correctly to specify who can access the repository. Replace `'us-east-1'` and other arguments with the proper values according to your requirements.