Isolated Subnetworks for Secure AI Training Environments

Question

Pulumi · Accepted Answer

Creating isolated subnetworks within a cloud provider's virtual network is crucial for ensuring secure environments, especially in scenarios like AI training where sensitive data might be processed. These isolated subnets can help enforce strict access controls and segregate network traffic to and from the AI training resources. In this case, we will utilize cloud resources to create such an environment.

We'll use Pulumi with the AWS provider, as Amazon Virtual Private Cloud (VPC) allows for the creation of isolated subnetworks (subnets) that provide control over sections of a network with specific access policies and rules.

Here is how you can create an isolated subnetwork in an AWS VPC using Pulumi in Python:

1. **VPC Creation**: First, we create a virtual private cloud (VPC) to provide a virtual network dedicated to your AWS account.

2. **Subnet Creation**: Within the VPC, we create one or more subnets. Each subnet is isolated within the VPC, and you can control access via network ACLs (Access Control Lists) and security groups.

3. **Security Groups and ACLs**: We define security groups and network ACLs to enforce strict security rules that control the traffic to and from the subnets.

4. **IAM Roles and Policies**: To add an additional layer of security, we create IAM roles with associated policies that grant the necessary permissions for AI training while restricting unnecessary access.

Let's proceed with the code to achieve this. Ensure you have configured the Pulumi AWS provider with the necessary credentials before running the following program.

```python
import pulumi
import pulumi_aws as aws

# Create a new VPC for your AI training environment
ai_vpc = aws.ec2.Vpc("aiVpc",
                     cidr_block="10.0.0.0/16",
                     enable_dns_support=True,
                     enable_dns_hostnames=True,
                     tags={
                         "Name": "ai_vpc"
                     })

# Create subnets within the VPC for different aspects of your AI environment
# Assuming we want one subnet for training and another for related services
training_subnet = aws.ec2.Subnet("trainingSubnet",
                                 vpc_id=ai_vpc.id,
                                 cidr_block="10.0.1.0/24",
                                 availability_zone="us-west-2a",
                                 tags={
                                     "Name": "training_subnet"
                                 })

services_subnet = aws.ec2.Subnet("servicesSubnet",
                                 vpc_id=ai_vpc.id,
                                 cidr_block="10.0.2.0/24",
                                 availability_zone="us-west-2b",
                                 tags={
                                     "Name": "services_subnet"
                                 })

# Create security groups to tightly control access to the training subnet
training_sg = aws.ec2.SecurityGroup("trainingSg",
                                    vpc_id=ai_vpc.id,
                                    description="Allow traffic for AI training",
                                    ingress=[{
                                        'from_port': 22,
                                        'to_port': 22,
                                        'protocol': 'tcp',
                                        'cidr_blocks': ["0.0.0.0/0"], # For demo purposes only, restrict this in production!
                                    }],
                                    egress=[{
                                        'from_port': 0,
                                        'to_port': 0,
                                        'protocol': '-1', # Allow all outbound traffic
                                        'cidr_blocks': ["0.0.0.0/0"],
                                    }],
                                    tags={
                                        "Name": "training_sg"
                                    })

# Create IAM roles and policies for access control
# This is just a placeholder showing where to add detailed IAM roles and policies
ai_training_role = aws.iam.Role("aiTrainingRole",
                                assume_role_policy={
                                    "Version": "2012-10-17",
                                    "Statement": [{
                                        "Action": "sts:AssumeRole",
                                        "Effect": "Allow",
                                        "Principal": {
                                            "Service": "ec2.amazonaws.com"
                                        }
                                    }]
                                })

# Expose the IDs of the resources created
pulumi.export('vpc_id', ai_vpc.id)
pulumi.export('training_subnet_id', training_subnet.id)
pulumi.export('services_subnet_id', services_subnet.id)
pulumi.export('training_sg_id', training_sg.id)
pulumi.export('ai_training_role_arn', ai_training_role.arn)
```

Explanation:

1. **Creating the VPC (`ai_vpc`)**: A VPC is set up with a CIDR block that determines its IP address range. We've enabled both DNS support and DNS hostnames within the VPC, which could be useful for communication and access purposes.

2. **Creating Subnets (`training_subnet` and `services_subnet`)**: Two subnets are defined with different CIDR blocks for training and related services. Note how each subnet is tied to the VPC via `vpc_id` and its availability zone is specified for resilience.

3. **Security Group Setup (`training_sg`)**: A security group for the training subnet is created that allows ingress traffic only on port 22 (SSH) and allows all egress traffic. This is meant for demonstration purposes; you should restrict the CIDR blocks and tailor the rules to your specific security needs.

4. **IAM Roles and Policies (`ai_training_role`)**: Although this code snippet is a placeholder, it represents where you would define IAM roles and associated policies for granular access control and principle of least privilege.

Running this program with the Pulumi CLI will create these resources. Remember to follow the AWS best practices and replace open CIDR blocks and permissive security group settings with more restricted values matching your organizational policies.