Custom AI Models Output Storage on Azure Blob
PythonTo store the output of custom AI models on Azure Blob Storage, you would typically create an Azure Storage Account and then within it, a blob container where you can store the output.
Here's how you generally accomplish this using Pulumi:
- Create an Azure Resource Group: Resource groups in Azure are a way to organize your resources (like blob storage) within Azure.
- Create an Azure Storage Account: A storage account is the parent resource for the blob container. It's a dedicated space in Azure where all your blobs live.
- Create a Blob Container: Within the storage account, you create a container which is a namespace in which your blobs are stored.
Now, let's write a Pulumi program in Python to set this up:
import pulumi import pulumi_azure_native as azure_native # Step 1: Create an Azure Resource Group resource_group = azure_native.resources.ResourceGroup('ai-models-resource-group') # Step 2: Create an Azure Storage Account storage_account = azure_native.storage.StorageAccount("aioutputstorageaccount", # Required properties for a Storage Account resource resource_group_name=resource_group.name, # The name of the Azure storage account account_name="aimodelsoutputstorage", # Sku defines the type of the storage account, here we choose 'Standard_LRS' # LRS means Locally-redundant storage. sku=azure_native.storage.SkuArgs(name=azure_native.storage.SkuName.STANDARD_LRS), # The kind of the Storage Account, BlobStorage can be used specifically for storing blobs kind=azure_native.storage.Kind.BLOB_STORAGE, # The location of the resource group which is taken from the resource group's location location=resource_group.location) # Step 3: Create a Blob Container blob_container = azure_native.storage.BlobContainer("aioutputcontainer", account_name=storage_account.name, resource_group_name=resource_group.name, # Public access level, we choose None to prevent public access to our AI models' outputs public_access=azure_native.storage.PublicAccess.NONE) # Export the primary endpoint primary_endpoint = pulumi.Output.all(resource_group.name, storage_account.name).apply( lambda args: f"https://{args[1]}.blob.core.windows.net") pulumi.export('primary_blob_endpoint', primary_endpoint)
Let's break down what the code does:
- We import the necessary Pulumi libraries.
- We create an Azure resource group that acts as a logical container for our storage resource. The creation of the resource group is abstracted away by Pulumi's
ResourceGroup
class within theazure_native.resources
module. - We then define an Azure Storage Account by instantiating
StorageAccount
from theazure_native.storage
module. The storage account stores and provides access to our blob data. - Next, we create a Blob Container named
aioutputcontainer
which will store all our AI model outputs. This container is part of the storage account we just created. - Lastly, we export the primary endpoint URL, which can be used to access the blob storage. Pulumi's
Output
object is used for capturing and exporting runtime information. Here, we concatenate the parts to form the full URL.
This program provides a simple, secure way to store the outputs of your AI models on Azure Blob Storage. Once deployed, your AI models can directly upload output files to this container for storage.
Remember to set up the Pulumi Azure Native provider with the necessary credentials and defaults before running this program. You would typically deploy this with
pulumi up
on the command line, and Pulumi will handle provisioning and configuring the Azure resources for you.