Optimizing AI Compute Costs using Azure Consumption Insights

Question

Pulumi · Accepted Answer

To optimize AI compute costs on Azure, you can leverage Azure Cost Management capabilities. One of the tools provided by Azure for monitoring and controlling costs is Azure Consumption Insights. These insights can help you understand where your budget is being spent, and on what resources.

To utilize Azure Consumption Insights for cost optimization in the context of AI compute, you could do the following:

Set Budgets: Define budgets for specific scopes, such as subscription or resource groups, to control spending. A budget can trigger actions or notifications as your cost nears the predefined limit.
Cost Allocation: Allocate costs to different departments or projects to create transparency and accountability.
Commitment Plans: If using Azure Machine Learning, leverage Commitment Plans, which provide discounted rates for compute in exchange for committing to a certain level of usage.
Cost Reporting: Generate detailed reports by billing account to identify trends and potential savings.
Scheduled Actions: Automate cost management tasks using scheduled actions based on predefined conditions or thresholds.

Below is a program written in Python using Pulumi that illustrates how you might set up a budget and retrieve consumption reports. This gives you the building blocks to start optimizing AI compute costs using Azure Consumption Insights.

First, let’s set up a budget:

import pulumi
import pulumi_azure_native as azure_native

# Example of setting up a budget for a subscription
budget = azure_native.costmanagement.Budget(
    "budget",
    scope=f"/subscriptions/{pulumi.config.get('azure:subscriptionId')}",
    amount=1000.0,
    time_period=azure_native.costmanagement.BudgetTimePeriodArgs(
        start_date="2023-04-01T00:00:00Z",
        # Assuming we want the budget to roll over monthly.
        # Set your desired end date or leave it open-ended.
        end_date="2024-04-01T00:00:00Z",
    ),
    time_grain="Monthly",
    category="Cost",  # or use "Usage"
    notifications={
        # Define the threshold at which a notification is sent.
        # 0.8 implies a notification will be sent at 80% budget usage.
        "threshold": 80,
        # You can set up an email action or trigger a webhook to an external system.
        "contact_emails": ["admin@example.com"],
        "enabled": True,
    }
)

# Export the budget name and ID for later reference
pulumi.export('budget_name', budget.name)
pulumi.export('budget_id', budget.id)

In this example, we’ve created a monthly budget of $1000 for your subscription and set notifications to be sent to admin@example.com when 80% of the budget is reached. You can also set up webhooks to notify or trigger other systems.

To analyze AI compute cost, we would set up reports:

# This is a placeholder demonstrating how you might set up a cost report.
# You would replace this with the actual details relevant to your Azure subscription and AI resources.
cost_report = azure_native.costmanagement.ReportByBillingAccount(
    "costReport",
    billing_account_id=pulumi.config.get('azure:billingAccountId'),
    format="Csv",
    delivery_info=azure_native.costmanagement.DeliveryInfoArgs(
        destination=azure_native.costmanagement.DestinationArgs(
            container="reportcontainer",
            resource_id="/subscriptions/{subscription_id}/resourceGroups/{resource_group}/providers/Microsoft.Storage/storageAccounts/{storage_account}",
            # Optionally, specify a root folder path in the storage account container.
            root_folder_path="reports/",
        ),
    ),
    schedule=azure_native.costmanagement.ScheduleArgs(
        # Define how often you want the report to be generated.
        recurrence="Monthly",
        # Define the period for the report data.
        recurrence_period=azure_native.costmanagement.RecurrencePeriodArgs(
            from_="2023-04-01T00:00:00Z",
            to="2024-04-01T00:00:00Z",
        ),
    )
)

# Export the report name for later reference.
pulumi.export('cost_report_name', cost_report.name)

The cost_report resource defines a recurring cost report for a billing account that sends data to a specified storage container in CSV format. The report will help you analyze costs within the defined period.

Remember, to run this Pulumi program, you will need to have the Azure CLI installed and configured with the appropriate credentials. Also, before running the code, make sure to replace <subscription_id>, <resource_group>, <storage_account>, and <billing_account_id> with your actual Azure subscription ID, resource group name, storage account name, and billing account ID respectively.

This example demonstrates how to automate cost control mechanisms in Azure. Pulumi enables you to define these mechanisms as code, which can be version-controlled and integrated into your existing CI/CD systems.