Forecasting AI cloud expenditure with Azure Cost Management

Question

Pulumi · Accepted Answer

Azure Cost Management is a suite of tools provided by Microsoft to help manage and optimize cloud costs. It can forecast spending, identify underutilized resources, and provide recommendations for cost savings. Azure Cost Management tools can be created and managed programmatically using Pulumi, an infrastructure as code tool, which enables you to define cloud resources using familiar programming languages like Python.

Here I will guide you through creating two resources in Azure using Pulumi for forecasting AI cloud expenditure:

1. **Cost Management Budget**: Budgets in Azure Cost Management help you plan for and drive organizational accountability. With budgets, you can account for the spending for your AI services and prevent cost overruns.

2. **Cost Management Export**: This sets up the ability to export cost management data to a storage account for further analysis, which can be useful for forecasting when combined with your own forecasting tools or AI models.

Let's dive into the Pulumi code to create these resources:

```python
import pulumi
import pulumi_azure_native as azure_native

# Cost Management Budget - used for setting a budget for your AI services.
budget = azure_native.consumption.Budget(
    "aiCostBudget",
    amount=1000.0,  # The total amount of cost to budget for (adjust this value as needed).
    budget_name="aiServicesMonthly",  # A helpful name for identifying the budget.
    category="Cost",  # Budget category - could also be "Usage" if you were tracking quantity of service, etc.
    time_grain="Monthly",  # The time span for which to look for changes in costs.
    time_period=azure_native.consumption.BudgetTimePeriodArgs(
        start_date="2023-01-01T00:00:00Z",  # When the budget should start monitoring costs.
        end_date="2023-12-31T23:59:59Z"  # An optional end date for the budget.
    ),
)

# Cost Management Export - used for exporting cost data for analysis.
export = azure_native.costmanagement.Export(
    "aiCostExport",
    definition=azure_native.costmanagement.ExportDefinitionArgs(
        type="Usage",  # This defines the type of data you want to export. "Cost" is also an option.
        time_period=azure_native.costmanagement.ExportTimePeriodArgs(
            from_property="2023-01-01T00:00:00Z",  # Start time for the data you want to export.
            to="2023-12-31T23:59:59Z"  # End time for the data you want to export.
        ),
        dataset=azure_native.costmanagement.ExportDatasetArgs(
            granularity="Daily",  # The granularity of the data in the report. Could also be Monthly.
            configuration=azure_native.costmanagement.ExportDatasetConfigurationArgs(
                columns=[
                    "ResourceType", 
                    "ResourceLocation", 
                    "PreTaxCost", 
                    "UsageQuantity", 
                    "ResourceName"  # And any other columns that are relevant to your analysis.
                ],
            ),
        ),
    ),
    format="Csv",  # The format of the exported data file.
    # We need to provide a storage account details where the export would be saved.
    delivery_info=azure_native.costmanagement.ExportDeliveryInfoArgs(
        destination=azure_native.costmanagement.ExportDeliveryDestinationArgs(
            container="aiexportcontainer",  # The container in your storage account where exports would be stored.
            resource_id="/subscriptions/{subscription_id}/resourceGroups/{resource_group_name}/providers/Microsoft.Storage/storageAccounts/{storage_account_name}",  # Substitute your actual subscription, resource group, and storage account details here.
        ),
    ),
    schedule=azure_native.costmanagement.ExportScheduleArgs(
        recurrence="Monthly",  # How often the export should run. Could be "Daily", "Weekly", or "Annually".
        recurrence_period=azure_native.costmanagement.ExportRecurrencePeriodArgs(
            from_property="2023-01-01T00:00:00Z",
            to="2023-12-31T23:59:59Z",
        ),
    ),
)

# Export the names of the budget and export resources just created
pulumi.export("budgetName", budget.name)
pulumi.export("exportName", export.name)
```

This code defines two resources using the `pulumi_azure_native` module:

- `aiCostBudget` is a budget that tracks spending on AI services in Azure, defined by the `azure_native.consumption.Budget` class.
- `aiCostExport` is an export that allows for the usage data to be exported to a CSV file for further analysis. The export is defined using the `azure_native.costmanagement.Export` class.

When you run the code with `pulumi up`, it will:

1. Create a budget that limits the monthly spending for Azure AI services to a specified amount.
2. Define an export that routinely sends cost and usage data to a designated container in a storage account. The exported data is in CSV format for ease of analysis, with a set schedule and period covering the desired timeframe.

You must replace placeholder values, such as the storage container information and the date ranges with your specific requirements.
  
To execute this Pulumi code, you will need to have the Pulumi CLI installed, be logged in, have an existing Azure subscription, and have set up your Azure credentials for Pulumi to use. Please read [Pulumi's Azure setup guide](https://www.pulumi.com/docs/intro/cloud-providers/azure/setup/) to get started. Once set up, you can run `pulumi up` to deploy your infrastructure.