Cost analysis of Machine Learning services using Azure Cost Management

Question

Pulumi · Accepted Answer

To perform cost analysis of Machine Learning services using Azure Cost Management, you can utilize the Azure Cost Management APIs to create reports, set up budgets, and analyze costs. We'll demonstrate how to programmatically set up a cost management report with Pulumi to track the costs associated with your Azure Machine Learning services.

The resources we will use include:
- `azure-native.costmanagement.Report`: This resource allows us to define and create a cost management report specific to our Azure Machine Learning services. We will configure it to provide a regular report of costs.
- `azure-native.costmanagement.Export`: This optional resource can be used to schedule the automatic export of reports for external analysis or record-keeping.

Here's how to define a cost management report in Pulumi using Python:

```python
import pulumi
import pulumi_azure_native as azure_native

# Create a cost management report for Azure Machine Learning services
report = azure_native.costmanagement.Report("machine_learning_cost_report",
    definition=azure_native.costmanagement.ReportDefinitionArgs(
        type="Usage",
        timeframe="MonthToDate",
        dataset=azure_native.costmanagement.ReportDatasetArgs(
            granularity="None",  # No granularity for total cost
            filter=azure_native.costmanagement.ReportFilterArgs(
                # Filter costs for the Machine Learning service resource type
                resource_type="Microsoft.MachineLearningServices/workspaces",
            ),
            aggregation={
                "totalCost": azure_native.costmanagement.ReportAggregationArgs(
                    name="PreTaxCost",
                    function="Sum"
                )
            },
        ),
    ),
    schedule=azure_native.costmanagement.ReportScheduleArgs(
        status="Active",
        recurrence="Monthly",
        recurrence_period=azure_native.costmanagement.ReportRecurrencePeriodArgs(
            from_="start_of_the_month_str",  # Replace with the actual start date string
            to="end_of_the_month_str",  # Replace with the actual end date string
        )
    ),
    format="Csv",  # The format of the report
    delivery_info=azure_native.costmanagement.ReportDeliveryInfoArgs(
        destination=azure_native.costmanagement.ReportDeliveryDestinationArgs(
            container= "reports",  # The container where reports will be stored
            root_folder_path="ml-cost-reports"  # The folder within the container where reports will be placed
        )
    )
)

# Export the URL of the storage container where reports are saved for easy access
pulumi.export('report_storage_url', pulumi.Output.concat("https://", report.delivery_info.destination.container, ".blob.core.windows.net/", report.delivery_info.destination.root_folder_path))
```

In this program:

1. We define a cost report with the `azure-native.costmanagement.Report` resource that is specifically filtered to track costs associated with Azure Machine Learning services.
2. The `timeframe` is set to "MonthToDate" which means the report will cover costs from the start of the current month to the date the report is executed.
3. The `format` is "Csv," allowing for easy readability and compatibility with various analysis tools.
4. We schedule the report to be generated monthly using the `schedule` property with a status of "Active".
5. The `delivery_info` specifies where the generated report will be stored within Azure Blob Storage for retrieval.
6. Finally, the URL to the storage location is exported, making it easy to access the reports once they are generated.

Please replace `"start_of_the_month_str"` and `"end_of_the_month_str"` with actual date strings defining the report's start and end period, and configure the storage container and folder according to your environment.

This Pulumi program provides a structured approach to automating cost reports generation and can be adjusted to fit more specific analysis requirements through additional filtering and dataset configuration.