Network Isolation for OCI AI Model Training Jobs

Question

Pulumi · Accepted Answer

To achieve network isolation for AI model training jobs on Oracle Cloud Infrastructure (OCI), you typically employ a combination of Virtual Cloud Networks (VCN), subnets, Network Security Groups (NSG), and potentially Private Endpoints if you need private access to OCI services. These resources help ensure that the AI model training environment is isolated from public networks and that the data and training jobs are secure.

In this setup with Pulumi, we'll create the following resources for a secure and isolated network environment in OCI:

- A Virtual Cloud Network (VCN) to provide a virtual network dedicated to your cloud resources.
- A subnet which will be a subdivision of the VCN to host your resources.
- A Network Security Group (NSG) for defining security rules to control traffic to and from resources associated with the NSG.
- An `AiPrivateEndpoint` which represents a private endpoint within the subnet to allow the AI service to be accessed privately.
- A `Model` resource for the AI model training job.
- A `Project` for organizing all related resources for anomaly detection.

Let's proceed with the Pulumi program to set up the network isolation for OCI AI model training jobs.

### Pulumi Program for Network Isolation

```python
import pulumi
import pulumi_oci as oci

# Initialize OCI provider configuration (replace with appropriate values)
oci_provider = oci.Provider("oci",
                            region="us-phoenix-1",
                            tenancy_ocid="ocid1.tenancy.oc1..xxxxx",
                            user_ocid="ocid1.user.oc1..xxxxx",
                            private_key_path="/path/to/private/key",
                            fingerprint="xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx")

# Create a Virtual Cloud Network (VCN)
vcn = oci.core.Vcn("ai-train-vcn",
                   cidr_block="10.0.0.0/16",
                   display_name="AI_Training_VCN",
                   compartment_id=oci_provider.compartment_ocid,
                   provider=oci_provider,
                   opts=pulumi.ResourceOptions(provider=oci_provider))

# Create a subnet within the VCN
subnet = oci.core.Subnet("ai-train-subnet",
                         cidr_block="10.0.1.0/24",
                         display_name="AI_Training_Subnet",
                         vcn_id=vcn.id,
                         compartment_id=oci_provider.compartment_ocid,
                         provider=oci_provider,
                         opts=pulumi.ResourceOptions(provider=oci_provider))

# Create a Network Security Group (NSG) for our subnet
nsg = oci.core.NetworkSecurityGroup("ai-train-nsg",
                                    display_name="AI_Training_NSG",
                                    vcn_id=vcn.id,
                                    compartment_id=oci_provider.compartment_ocid,
                                    provider=oci_provider,
                                    opts=pulumi.ResourceOptions(provider=oci_provider))

# Create a private endpoint for AI services within the subnet
ai_private_endpoint = oci.aianomalydetection.AiPrivateEndpoint("ai-private-endpoint",
                                                               display_name="AI_Private_Endpoint",
                                                               subnet_id=subnet.id,
                                                               compartment_id=oci_provider.compartment_ocid,
                                                               provider=oci_provider,
                                                               opts=pulumi.ResourceOptions(provider=oci_provider))

# Define a project for AI jobs
ai_project = oci.aianomalydetection.Project("ai-train-project",
                                            display_name="AI_Training_Project",
                                            compartment_id=oci_provider.compartment_ocid,
                                            provider=oci_provider,
                                            opts=pulumi.ResourceOptions(provider=oci_provider))

# Define the AI model (replace with appropriate values for model training details)
ai_model = oci.aianomalydetection.Model("ai-model",
                                        project_id=ai_project.id,
                                        display_name="Anomaly_Detection_Model",
                                        compartment_id=oci_provider.compartment_ocid,
                                        model_training_details={
                                            "data_asset_ids": ["ocid1.dataasset.oc1..xxxx"],
                                            "target_fap": 0.05,
                                            "window_size": 120
                                        },
                                        provider=oci_provider,
                                        opts=pulumi.ResourceOptions(provider=oci_provider))

# Output the private endpoint's details
pulumi.export('private_endpoint_url', ai_private_endpoint.private_endpoint_ip)
```

In the above program, we perform the following steps:

1. **Initialize OCI Provider**: We set up the provider with the necessary OCI tenant, user, and key information to authenticate the Pulumi program with your OCI account.

2. **Create VCN and Subnet**: The VCN is the backbone of your network on OCI, with the subnet being a segmented portion of the VCN. This allows us to define a controllable network area for our AI services.

3. **Network Security Group (NSG)**: We define an NSG to control the inbound and outbound network traffic rules for your AI training compute instances that are associated with this NSG.

4. **Private Endpoint**: The `AiPrivateEndpoint` resource ensures that your AI services can be accessed privately within your VCN, thus not exposed to the public internet.

5. **AI Project and Model**: We create an AI project to organize your resources and a model configuration for your AI anomaly detection job.

6. **Output**: Finally, we output the private endpoint's URL, which confirms the creation of the resource.

Please ensure that you replace placeholder values (like OCID values and private key path) with actual values from your OCI account before running the program. The provided example is a blueprint and needs real values to work correctly.

To run this program, you'll need to have Pulumi installed and configured for use with OCI. Once you've set up your Pulumi configuration with the correct credentials, you can use the command `pulumi up` to deploy these resources to your OCI account. After the deployment, you can use the outputted private endpoint URL for accessing the isolated AI services for model training.