Periodic log analytics through saved searches

Question

Pulumi · Accepted Answer

To perform periodic log analytics through saved searches, we will create a Pulumi program using the `azure-native` provider. The main resource we'll focus on is `azure-native.operationalinsights.SavedSearch`, which enables you to save log queries in Azure Monitor Logs. Saved Searches are a feature within Azure's Log Analytics service that allows you to create, save, and manage commonly used log queries, making it easier for you to periodically run these queries against your log data.

In the context of Azure, Log Analytics is a tool within the Azure Monitor suite that lets you query and analyze logs collected from various resources in Azure. These logs could include metrics and data related to the operation of Azure services, applications, and infrastructure.

The following is a breakdown of the steps the Pulumi program will perform:

1. Configure the Azure provider: Set up necessary imports and configuration for the Azure provider.
2. Create a Log Analytics Workspace: The saved search needs a workspace in which to store the logs.
3. Create a Saved Search: Define a query that you want to save and periodically run.

Here's a detailed Pulumi TypeScript program that sets up a Log Analytics Workspace and then adds a Saved Search to it for periodic log analysis:

```typescript
import * as pulumi from "@pulumi/pulumi";
import * as azure_native from "@pulumi/azure-native";

// Create an Azure Resource Group
const resourceGroup = new azure_native.resources.ResourceGroup("logResourceGroup");

// Create an Azure Log Analytics Workspace 
const workspace = new azure_native.operationalinsights.Workspace("logWorkspace", {
    resourceGroupName: resourceGroup.name,
    sku: {
        name: "PerGB2018", // Choose the appropriate pricing tier
    },
    retentionInDays: 30, // Set the data retention policy (in days)
});

// Saved Search for the Workspace
const savedSearch = new azure_native.operationalinsights.SavedSearch("mySavedSearch", {
    resourceGroupName: resourceGroup.name,
    workspaceName: workspace.name,
    category: "MyCategory", // Category for the saved search
    displayName: "My Saved Search", // Display name for the saved search
    query: "Heartbeat | summarize Count() by Computer", // Example query to run periodically
    // Assign any relevant metadata tags to the saved search
    tags: [
        {
            name: "Environment",
            value: "Production",
        },
    ],
});

// Export the Workspace ID and Saved Search ID
export const workspaceId = workspace.id;
export const savedSearchId = savedSearch.id;
```

In this program:

- We import the necessary `pulumi` and `azure-native` modules to interact with Azure resources.
- We create a new resource group named `logResourceGroup` to contain our Log Analytics resources.
- We then define a new `Workspace` resource named `logWorkspace` within the created resource group. The workspace will store and manage the log data that is being collected.
    - The `sku` property is set to `"PerGB2018"`, which corresponds to a specific pricing tier for Log Analytics. Make sure to choose the right SKU for your use case.
    - `retentionInDays` is set to `30`, which instructs Azure to keep the logs for 30 days before they are purged.
- A `SavedSearch` named `mySavedSearch` is created within the workspace to define a saved log query. 
    - `query` contains an example Kusto Query Language (KQL) command that summarizes the count of heartbeat records by Computer.
    - `category` and `displayName` provide organization and user-friendly naming for the saved search.
    - `tags` can be used to assign metadata to the search for better resource management.
- At the end, we export the generated Workspace and Saved Search IDs. These values are dynamically assigned by Azure and are made available after the resources have been created. You can use these exports to interact with the resources through other tools or reference them in other Pulumi programs.

When you run this program using `pulumi up`, it will perform all the necessary actions to create these resources within your Azure subscription.

Remember to replace the placeholder query with your actual log query that you wish to run periodically. The Kusto Query Language (KQL) is powerful and allows for a wide range of queries, so tailor this to your specific analysis needs.

Also, be mindful of the costs associated with the chosen SKU and the amount of data retention required for your use case. Azure Monitor Logs pricing is based on the amount of data ingested into Log Analytics and retained in the workspace.