Private Access to Azure Cosmos DB for AI Workloads

Question

Pulumi · Accepted Answer

To set up private access to Azure Cosmos DB for AI workloads, you'll need to create an instance of Azure Cosmos DB configured for private access. This typically involves setting up a Cosmos DB account, creating a private endpoint, and configuring your AI workloads to connect to Cosmos DB using this private endpoint.

Below is a Pulumi program written in Python that demonstrates how to create an Azure Cosmos DB account with a private endpoint. This program assumes that you already have an Azure Resource Group created where we can deploy these resources.

The following resources will be used:
- `DatabaseAccount`: This creates an Azure Cosmos DB account.
- `Subnet`: A subnet within an Azure Virtual Network where you'll place your Cosmos DB.
- `PrivateEndpoint`: This creates a private endpoint within the specified subnet that connects securely to the Cosmos DB service.
- `PrivateDnsZoneGroup`: Associates the private endpoint with a private DNS zone for name resolution within the VNet.

Here is the Pulumi program that accomplishes this:

```python
import pulumi
import pulumi_azure_native as azure_native

# Configure these to match the existing Virtual Network and Resource Group
existing_virtual_network_name = "your-vnet-name"
existing_subnet_name = "your-subnet-name"
existing_resource_group_name = "your-resource-group-name"

# An Azure Cosmos DB account with MongoDB compatibility
cosmosdb_account = azure_native.documentdb.DatabaseAccount("cosmosdbAccount",
    resource_group_name=existing_resource_group_name,
    location="West US",  # Use the appropriate Azure region
    database_account_offer_type="Standard",
    consistency_policy={
        "defaultConsistencyLevel": "Session",
    },
    capabilities=[{"name": "EnableMongo"}]  # If your AI workloads require MongoDB API
)

# A subnet specifically for the Azure Cosmos DB
cosmosdb_subnet = azure_native.network.Subnet("cosmosdbSubnet",
    address_prefix="10.0.0.0/24",  # Choose an appropriate CIDR range for your subnet
    virtual_network_name=existing_virtual_network_name,
    resource_group_name=existing_resource_group_name,
    service_endpoints=[{"service": "Microsoft.AzureCosmosDB"}]
)

# A private endpoint to the Cosmos DB account within the designated subnet
cosmosdb_private_endpoint = azure_native.network.PrivateEndpoint("cosmosdbPrivateEndpoint",
    private_link_service_connections=[{
        "name": "cosmosdbPrivateLinkServiceConnection",
        "private_link_service_id": cosmosdb_account.id,
        "group_ids": ["Sql"],  # Use the corresponding group for your chosen API
    }],
    subnet={
        "id": cosmosdb_subnet.id,
    },
    resource_group_name=existing_resource_group_name,
    location="West US",  # Match the Azure region with Cosmos DB and VNet
)

# Group for associating the private endpoint with a DNS zone
dns_zone_group = azure_native.network.PrivateDnsZoneGroup("dnsZoneGroup",
    private_dns_zone_group_name="default",  # Default group name; can be customized
    private_endpoint_name=cosmosdb_private_endpoint.name,
    private_dns_zone_configs=[{
        "private_dns_zone_id": "/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.Network/privateDnsZones/privatelink.documents.azure.com"  # Use the correct subscription and resource group names
    }],
    resource_group_name=existing_resource_group_name,
)

# Export the outputs that we might want to use or reference
pulumi.export("cosmosdb_endpoint", cosmosdb_private_endpoint.private_endpoint_ip_configurations[0].private_ip_address)
```

This program begins by setting up a Cosmos DB account configured for MongoDB API compatibility, which is often needed for AI workloads. We then create a subnet dedicated to the Cosmos DB account and establish a private endpoint within this subnet. The private endpoint is then associated with a private DNS zone to ensure the Cosmos DB account is accessible securely within the Virtual Network.

Please replace the placeholders ('your-vnet-name', 'your-subnet-name', etc.) with your actual resource group, virtual network, and subnet details.

Once the Pulumi script is ready, you can run it using the Pulumi CLI. Ensure you have set up your Azure credentials for use with Pulumi and then run `pulumi up` to create the resources. The resources can be updated or destroyed with the `pulumi up` and `pulumi destroy` commands, respectively.

Remember to always review the changes provided by Pulumi before applying them to understand what will happen in your Azure environment.