Versioned Model Artifact Storage with Azure Blob

Question

Pulumi · Accepted Answer

To create versioned model artifact storage with Azure Blob Storage, we will use Pulumi to provision the necessary infrastructure. Azure Blob Storage is an object storage solution for the cloud that enables you to store large amounts of unstructured data, such as text or binary data.

We will create a storage account, a blob container within that storage account, and then enable versioning on the blob container to maintain versioned artifacts.

Here's a detailed breakdown of the steps we will follow in the Pulumi program:

1. **Create an Azure Resource Group**: A resource group is a container that holds related resources for an Azure solution.
2. **Create an Azure Storage Account**: This account will be used to create the Blob service where your artifacts will be stored.
3. **Create a Blob Container within the Storage Account**: This container will hold your blobs (artifacts) in Azure Storage.
4. **Enable Versioning on the Blob Container**: This will allow you to keep different versions of your blobs whenever they get overwritten or deleted.

Now let's write the Pulumi Python program to implement this:

```python
import pulumi
import pulumi_azure_native.storage as storage

# Replace these variables with your specific values
resource_group_name = "my-resource-group"
storage_account_name = "mystorageaccount"
container_name = "myartifactcontainer"

# Create an Azure Resource Group
resource_group = storage.ResourceGroup(resource_group_name)

# Create a storage account
account = storage.StorageAccount(storage_account_name,
    resource_group_name=resource_group.name,
    sku=storage.SkuArgs(name=storage.SkuName.STANDARD_LRS),
    kind=storage.Kind.STORAGE_V2)

# Create a storage container inside the storage account
container = storage.BlobContainer(container_name,
    account_name=account.name,
    resource_group_name=resource_group.name,
    public_access=storage.PublicAccess.NONE,
    immutable_storage_with_versioning=storage.ImmutableStorageWithVersioningArgs(enabled=True)) # Enable versioning here

# Export the connection string for the storage account to access it later
primary_connection_string = account.primary_connection_string.apply(
    lambda connection_string: connection_string)

# Export the endpoint of the blob container for access
container_endpoint = pulumi.Output.concat(
    "https://", account.name, ".blob.core.windows.net/", container.name)

pulumi.export('PrimaryStorageConnectionString', primary_connection_string)
pulumi.export('ContainerEndpoint', container_endpoint)
```

In this program, `StorageAccount` and `BlobContainer` are Pulumi resource classes from the `pulumi_azure_native.storage` module. We are using the `STANDARD_LRS` SKU for the storage account, which stands for "Locally Redundant Storage", and setting our `kind` to `STORAGE_V2` to use the latest storage type with support for all storage capabilities including blobs, files, queues, and tables.

The `BlobContainer` resource is configured with `public_access` set to `None` for privacy and to enable versioning, we set the `immutable_storage_with_versioning` property to `True`. Versioning allows you to store, retrieve, and restore every version of a blob to help you protect your data.

The connection string to the storage account and the endpoint to the blob container are exported using Pulumi's `export` function so they can be used outside of Pulumi, such as in applications that need to store or retrieve artifacts from this container.

With this program, you will create a secure and versioned storage solution suitable for managing model artifacts or other data that requires versioning for change management or compliance reasons. Remember to replace the placeholder resource names with the actual names that you wish to use for your resource group, storage account, and blob container.