AI Infrastructure Cost Analysis using Azure Resource Graph

Question

Pulumi · Accepted Answer

To perform AI Infrastructure Cost Analysis using Azure Resource Graph, we will use Pulumi with the `azure-native` package. This will allow us to create reports and view analytics regarding our Azure resources with the help of Azure Cost Management and Azure Resource Graph services.

Firstly, we will leverage the `ReportByResourceGroupName` class to create a cost analysis report for a specific resource group on Azure. We'll define the type of the report, set up the dataset with appropriate filters or groupings, define the time period for cost analysis, and configure a delivery method for the report (for example, sending the report to a storage container).

Here's an outline of what will be done:

1. Use `ReportByResourceGroupName` to establish a report configuration for your AI infrastructure within a particular resource group.
2. Define filters and grouping to focus on the AI services and resources you're interested in.
3. Schedule the delivery of these reports to an Azure Storage Account container.

Below is a Pulumi program written in Python that implements the outlined steps:

```python
import pulumi
import pulumi_azure_native as azure_native

# Assume that the resource group and storage account are already created and the user has their names.

# Define the report configuration for Azure Cost Management with the necessary filters
# and parameters tailored to AI infrastructure within a particular resource group.
cost_management_report = azure_native.costmanagement.ReportByResourceGroupName(
    "costManagementReport",
    resource_group_name="your_resource_group_name",
    report_name="AI_Infrastructure_Cost_Analysis",
    format="Csv",
    schedule=azure_native.costmanagement.ReportConfigScheduleArgs(
        status="Active",  # The report is actively generated based on the schedule.
        recurrence="Monthly",
        recurrence_period=azure_native.costmanagement.ReportConfigRecurrencePeriodArgs(
            from="2023-04-01T00:00:00Z",
            to="2023-04-30T23:59:59Z",
        ),
    ),
    definition=azure_native.costmanagement.ReportConfigDefinitionArgs(
        type="Usage",
        dataset=azure_native.costmanagement.ReportConfigDatasetArgs(
            aggregation={
                "totalCost": azure_native.costmanagement.ReportConfigAggregationArgs(
                    name="Cost",
                    function="Sum"
                )
            },
            grouping=[
                azure_native.costmanagement.ReportConfigGroupingArgs(
                    type="Dimension",
                    name="ResourceType",
                ),
            ],
            # Filter by resource types related to AI (e.g., Machine Learning, Cognitive Services)
            filter=azure_native.costmanagement.ReportConfigFilterArgs(
                and=[
                    azure_native.costmanagement.ReportFilterArgs(
                        dimension=azure_native.costmanagement.ReportComparisonExpressionArgs(
                            name="ResourceType",
                            operator="In",
                            values=[
                                "Microsoft.MachineLearningServices/workspaces",
                                "Microsoft.CognitiveServices/accounts",
                                # Add more AI-related resource types here
                            ],
                        ),
                    ),
                ],
            ),
        ),
        timeframe="MonthToDate",
        time_period=azure_native.costmanagement.ReportConfigTimePeriodArgs(
            from="2023-04-01T00:00:00Z",
            to="2023-04-30T23:59:59Z",
        ),
    ),
    delivery_info=azure_native.costmanagement.ReportConfigDeliveryInfoArgs(
        destination=azure_native.costmanagement.ReportConfigDeliveryDestinationArgs(
            # Store the generated report in a blob container
            container="reports",
            resource_id="your_storage_account_resource_id", # The Resource ID of your storage account
        ),
    ),
)

# The following command exports the ID of the created report configuration,
# which can be used to retrieve or manage the report.
pulumi.export("report_id", cost_management_report.id)
```

This program sets up cost tracking tailored specifically to AI infrastructure resources, and will generate a CSV report to a designated blob container monthly. The setup includes:

- **Report Format**: CSV, which is a common format for data analytics and can be imported into various tools for further analysis.
- **Schedule**: Monthly reports generated for the current month.
- **Dataset Aggregation**: Summing up costs under a totalCost label.
- **Dataset Grouping**: Grouping costs by resource type allows us to see costs broken down per AI service used.
- **Dataset Filters**: Focusing the report on specific AI resource types like Machine Learning and Cognitive Services.
- **Timeframe**: Set to provide the current month's data.
- **Delivery Info**: Configured to deliver the report to a specific Azure Storage Container.

Remember to replace `your_resource_group_name` with the name of your Azure Resource Group and `your_storage_account_resource_id` with the full Azure Resource ID of your storage account where you wish to keep the reports.

Ensure your Pulumi configuration is set for Azure, and you have the necessary permissions to perform these actions. Run `pulumi up` to deploy and start the cost tracking. The resulting reports provide visibility into your AI infrastructure costs that can be used to analyze spending patterns and implement cost-saving strategies.