Automated User Provisioning and Role Assignments in Databricks

Question

Pulumi · Accepted Answer

To achieve automated user provisioning and role assignments in Databricks with Pulumi, we'll primarily use two resources from the `pulumi_databricks` provider:

- `databricks.User`: This resource is used to create a new user in the Databricks workspace. You can specify the user's name, display name, whether the user is active, and other settings.
- `databricks.Group`: Managing user groups is an essential part of access control in Databricks. This resource allows you to create and configure groups.
- `databricks.Permissions`: This resource allows you to define permissions for various Databricks resources, including assigning users or groups to roles.

In the example below, the Pulumi program will:

1. Create a new user.
2. Create a new group.
3. Assign the user to the group.
4. Assign the group specific permissions on a Databricks cluster.

Here's a Pulumi Python program that carries out these steps:

```python
import pulumi
import pulumi_databricks as databricks

# Create a new Databricks user
new_user = databricks.User("new-user",
    userName="jane.doe@example.com",
    displayName="Jane Doe",
    active=True
)

# Create a "Data Scientists" group
data_scientists_group = databricks.Group("data-scientists-group",
    display_name="Data Scientists"
)

# Add the new user to the "Data Scientists" group
user_group_membership = databricks.GroupMember("user-group-membership",
    group_name=data_scientists_group.display_name,
    member_name=new_user.userName
)

# Assume we already have a cluster created and its ID is available.
# This ID can be hardcoded, dynamically retrieved, or referenced from another stack, depending on your setup.
cluster_id = "your-databricks-cluster-id"

# Assign the "Data Scientists" group to a cluster with CAN_RESTART permissions.
cluster_permissions = databricks.Permissions("cluster-permissions",
    cluster_id=cluster_id,
    access_control_list=[
        databricks.PermissionsAccessControlListArgs(
            group_name=data_scientists_group.display_name,
            permission_level="CAN_RESTART"
        )
    ]
)

# Export some of the resource IDs
pulumi.export("user_id", new_user.id)
pulumi.export("group_id", data_scientists_group.id)
pulumi.export("cluster_permissions_id", cluster_permissions.id)
```

This program defines the desired state, which Pulumi then works to achieve when you run the `pulumi up` command. It ensures that Pulumi will create the resources in the appropriate order based on their dependencies; for example, it will not attempt to add a user to a group before both the user and the group have been created.

Please replace `"your-databricks-cluster-id"` with the actual ID of the cluster you want to assign permissions to. This could be obtained from a different part of your Pulumi stack or as an output from a cluster created in the same program.

You can run this program using the Pulumi CLI after setting up your Pulumi project and stack. Ensure you've configured the `pulumi_databricks` provider with the necessary credentials to interact with your Databricks workspace. If you're new to Pulumi, you can check the [Pulumi Getting Started Guide](https://www.pulumi.com/docs/get-started/) and the [Pulumi Databricks Provider documentation](https://www.pulumi.com/registry/packages/databricks/) to help you get started.