Versioned Model Artifact Storage with Azure Blob
PythonTo create versioned model artifact storage with Azure Blob Storage, we will use Pulumi to provision the necessary infrastructure. Azure Blob Storage is an object storage solution for the cloud that enables you to store large amounts of unstructured data, such as text or binary data.
We will create a storage account, a blob container within that storage account, and then enable versioning on the blob container to maintain versioned artifacts.
Here's a detailed breakdown of the steps we will follow in the Pulumi program:
- Create an Azure Resource Group: A resource group is a container that holds related resources for an Azure solution.
- Create an Azure Storage Account: This account will be used to create the Blob service where your artifacts will be stored.
- Create a Blob Container within the Storage Account: This container will hold your blobs (artifacts) in Azure Storage.
- Enable Versioning on the Blob Container: This will allow you to keep different versions of your blobs whenever they get overwritten or deleted.
Now let's write the Pulumi Python program to implement this:
import pulumi import pulumi_azure_native.storage as storage # Replace these variables with your specific values resource_group_name = "my-resource-group" storage_account_name = "mystorageaccount" container_name = "myartifactcontainer" # Create an Azure Resource Group resource_group = storage.ResourceGroup(resource_group_name) # Create a storage account account = storage.StorageAccount(storage_account_name, resource_group_name=resource_group.name, sku=storage.SkuArgs(name=storage.SkuName.STANDARD_LRS), kind=storage.Kind.STORAGE_V2) # Create a storage container inside the storage account container = storage.BlobContainer(container_name, account_name=account.name, resource_group_name=resource_group.name, public_access=storage.PublicAccess.NONE, immutable_storage_with_versioning=storage.ImmutableStorageWithVersioningArgs(enabled=True)) # Enable versioning here # Export the connection string for the storage account to access it later primary_connection_string = account.primary_connection_string.apply( lambda connection_string: connection_string) # Export the endpoint of the blob container for access container_endpoint = pulumi.Output.concat( "https://", account.name, ".blob.core.windows.net/", container.name) pulumi.export('PrimaryStorageConnectionString', primary_connection_string) pulumi.export('ContainerEndpoint', container_endpoint)
In this program,
StorageAccount
andBlobContainer
are Pulumi resource classes from thepulumi_azure_native.storage
module. We are using theSTANDARD_LRS
SKU for the storage account, which stands for "Locally Redundant Storage", and setting ourkind
toSTORAGE_V2
to use the latest storage type with support for all storage capabilities including blobs, files, queues, and tables.The
BlobContainer
resource is configured withpublic_access
set toNone
for privacy and to enable versioning, we set theimmutable_storage_with_versioning
property toTrue
. Versioning allows you to store, retrieve, and restore every version of a blob to help you protect your data.The connection string to the storage account and the endpoint to the blob container are exported using Pulumi's
export
function so they can be used outside of Pulumi, such as in applications that need to store or retrieve artifacts from this container.With this program, you will create a secure and versioned storage solution suitable for managing model artifacts or other data that requires versioning for change management or compliance reasons. Remember to replace the placeholder resource names with the actual names that you wish to use for your resource group, storage account, and blob container.