Permission Control for Queue-Triggered AI Functions in Azure

Question

Pulumi · Accepted Answer

When setting up queue-triggered functions in Azure to run AI workloads, you generally require a combination of Azure services including Azure Functions and Azure Queue Storage. The Azure Functions service will host the execution environment where your codified AI logic runs when triggered, and the Queue Storage will serve as the trigger point where messages initiate the execution of your Functions.

To control permissions for this setup, Azure uses Role-Based Access Control (RBAC), where you apply roles to service principals, users, or groups, defining what actions they can perform on Azure resources. Azure Function Apps and Queue Storage have their own set of RBAC roles.

In Pulumi, the configuration for Azure Function Apps, Queue Storage, and corresponding permissions can be declared using the `pulumi_azure_native` provider. Below is a Python program using Pulumi SDK to create an Azure Function App, an Azure Queue Storage, and set up permissions for a queue-triggered function:

1. **Azure Function App**: This is the compute resource that will host the executable code of your AI Function.
2. **Azure Storage Account and Queue**: This represents where the Function will listen for messages (events) to trigger execution.
3. **Role Assignments**: These define the permissions that identities (like a service principal) have with respect to the Function App and Queue Storage.

Here is a Pulumi program in Python to demonstrate how you would set up these resources and control permissions for an Azure queue-triggered AI function.

```python
import pulumi
import pulumi_azure_native as azure_native

# Define an existing resource group or create a new one where all resources will be allocated
resource_group = azure_native.resources.ResourceGroup('my-resource-group')

# Define the storage account where the queue will reside
storage_account = azure_native.storage.StorageAccount('mystorageaccount',
    resource_group_name=resource_group.name,
    sku=azure_native.storage.SkuArgs(
        name=azure_native.storage.SkuName.STANDARD_LRS,
    ),
    kind=azure_native.storage.Kind.STORAGE_V2)

# The storage queue which triggers the execution of AI function
storage_queue = azure_native.storage.Queue('myqueue',
    resource_group_name=resource_group.name,
    account_name=storage_account.name)

# Define the Function App which will contain the executable AI workload
app_service_plan = azure_native.web.AppServicePlan('myappserviceplan',
    resource_group_name=resource_group.name,
    sku=azure_native.web.SkuDescriptionArgs(
        name="Y1",
        tier="Dynamic",
    ),
    kind="FunctionApp")

function_app = azure_native.web.WebApp('myfunctionapp',
    resource_group_name=resource_group.name,
    server_farm_id=app_service_plan.id,
    kind="FunctionApp",
    site_config=azure_native.web.SiteConfigArgs(
        app_settings=[
            # Application settings goes here
        ],
    ))

# Obtain the principal ID for the function which will be given permissions on the queue
function_app_identity = function_app.identity.apply(lambda identity: identity.principal_id)

# Function App requires a storage connection string to manage triggers and logging
# Construct the connection string using the account name and keys
storage_connection_string = pulumi.Output.all(storage_account.name, storage_account.primary_key).apply(
    lambda args: f"DefaultEndpointsProtocol=https;AccountName={args[0]};AccountKey={args[1]};EndpointSuffix=core.windows.net")

# Assign appropriate RBAC roles for the function app to interact with the queue
role_assignment = azure_native.authorization.RoleAssignment('myroleassignment',
    scope=storage_queue.id,
    role_assignment_name=pulumi.uuid(),
    principal_type=azure_native.authorization.PrincipalType.SERVICE_PRINCIPAL,
    principal_id=function_app_identity,
    role_definition_id=pulumi.Output.concat(
        "/subscriptions/",
        pulumi.Config("azure").require("subscription_id"),
        "/providers/Microsoft.Authorization/roleDefinitions/",
        # Azure Roles like Storage Queue Data Contributor have fixed IDs you find in Azure docs
        "974c5e8b-45b9-4653-af93-4877683168e1"  # Role ID for Storage Queue Data Contributor
    ))

pulumi.export("storage_account_name", storage_account.name)
pulumi.export("function_app_name", function_app.name)
```

Now let's examine the key parts of this program:

- `ResourceGroup` is the container that holds related resources for an Azure solution. The `mystorageaccount` and `myfunctionapp` need to be created in a resource group.
- Next, we create a `StorageAccount` and a `Queue`, which is where our function will listen for new messages to process.
- We then set up an `AppServicePlan` and a `WebApp` with the kind as `FunctionApp`. This is where our AI function code will live.
- After deploying the Function App, we capture its `principal_id` which allows us to define role assignments.
- The `RoleAssignment` resource is then used to grant the Function App the necessary permissions to read messages from the Queue. For this, we use the `Storage Queue Data Contributor` built-in role which provides read/write/delete permissions to queue data.
- Finally, we export the names of our storage account and function app for easy access and reference.

By controlling permissions using RBAC and setting up the resources with Pulumi, you create a secure and automated deployment of Azure resources for your AI-based applications.