Automated Lifecycle Management for Azure Blob Storage

Question

Pulumi · Accepted Answer

Automated Lifecycle Management in Azure Blob Storage is a feature you can use to create rule-based policies that transition your data to the optimal access tier or expire your data at the end of its lifecycle. Azure Blob Storage supports lifecycle management for block blobs, append blobs, and page blobs. Azure storage lifecycle management offers a rich, rule-based policy for GPv2 and Blob storage accounts.

Here's a Pulumi program that demonstrates how to set up lifecycle management policies for an Azure Blob Storage Container. This policy will, for example, delete blobs that are older than 365 days and transition blobs to a cooler access tier if they haven't been accessed in more than 30 days.

Firstly, we will need a Resource Group and a Storage Account, which will host our Blob Container. Then we specify the Blob Container and the management rules.

Let's break down the code into steps:

1. Create a new Resource Group, which acts as a logical container for your Azure resources.
2. Create a Storage Account where the Blob Container will reside.
3. Define a Blob Container within the Storage Account.
4. Apply a Management Policy to the Storage Account that defines the lifecycle rules for blobs within the Blob Container.

The program below demonstrates these steps:

```python
import pulumi
import pulumi_azure_native as azure_native

# Step 1: Create an Azure Resource Group
resource_group = azure_native.resources.ResourceGroup('my-resource-group')

# Step 2: Create an Azure Storage Account
storage_account = azure_native.storage.StorageAccount('my-storage-account',
    resource_group_name=resource_group.name,
    sku=azure_native.storage.SkuArgs(name=azure_native.storage.SkuName.STANDARD_LRS),
    kind=azure_native.storage.Kind.STORAGE_V2)

# Step 3: Create a Blob Container inside the Storage Account
container = azure_native.storage.BlobContainer('my-container',
    account_name=storage_account.name,
    resource_group_name=resource_group.name)

# Step 4: Set up the Storage Account Lifecycle Management Policy
management_policy = azure_native.storage.ManagementPolicy('my-management-policy',
    account_name=storage_account.name,
    resource_group_name=resource_group.name,
    policy=azure_native.storage.ManagementPolicySchemaArgs(
        rules=[
            azure_native.storage.ManagementPolicyRuleArgs(
                enabled=True,
                name="deleteOldBlobs",
                type="Lifecycle",
                definition=azure_native.storage.ManagementPolicyDefinitionArgs(
                    actions=azure_native.storage.ManagementPolicyActionArgs(
                        base_blob=azure_native.storage.ManagementPolicyBaseBlobArgs(
                            delete=azure_native.storage.DateAfterModificationArgs(
                                days_after_modification_greater_than=365
                            ),
                        ),
                        snapshot=azure_native.storage.ManagementPolicySnapShotArgs(
                            delete=azure_native.storage.DateAfterCreationArgs(
                                days_after_creation_greater_than=365
                            )
                        )
                    ),
                    filters=azure_native.storage.ManagementPolicyFilterArgs(
                        prefix_match=["container1"]
                    )
                )
            ),
        ],
    )
)

# Export the primary endpoint of the Storage Account
pulumi.export('primary_storage_endpoint', storage_account.primary_endpoints)
```

In the above code, I've used:

- `ManagementPolicy` to define the lifecycle management rule. The `rule` component—which consists of filters and actions—specifies how the lifecycle of blobs should be managed.
- `prefix_match` within `filters` to select blobs in the specified Blob Container.

This code would manage the lifecycle of blobs within the "container1" prefix, deleting any blobs not modified for over 365 days. The policy also deletes any snapshots created over 365 days ago.

Before running this program with Pulumi, you would need to have sure you have the Azure CLI installed and configured with the credentials that provide access to your Azure subscription. Make sure you are logged in using `az login`.

After setting up your Azure credentials, you can run this Pulumi program like any Python script. Pulumi CLI will take care of deploying the resources as specified in the program.