Databricks Workspace Security with Permissions

Question

Pulumi · Accepted Answer

To manage security and permissions for a Databricks workspace, you can use the Pulumi Databricks provider to define and apply resource configurations that fit your security requirements. Databricks allows you to define permission assignments for various principals (like users and groups) across different-scoped objects within Databricks, such as clusters, jobs, and notebooks.

Here's what you can do with Pulumi in terms of Databricks workspace security:

1. **Workspace Management**: You can create and configure a Databricks workspace, specifying network settings, storage configurations, and more.
2. **Permission Assignment**: After defining your workspace, you can use resources like `PermissionAssignment` to specify what actions principals can perform on different objects within Databricks.
3. **Security Configuration**: Pulumi also lets you set up more granular security settings, like SQL permissions, using `SqlPermissions` to control access to databases, tables, views, and so on.

Below, I'll show you a basic example that includes creating a Databricks workspace and setting up a simple user permission assignment. This example assumes that you're working within Azure, given the resource information provided.

Please note that sensitive information like `accountId` must be securely managed. In a production environment, you would use Pulumi's secret management or integrate with a service like Azure KeyVault.

```python
import pulumi
import pulumi_databricks as databricks

# Create a Databricks workspace
databricks_workspace = databricks.MwsWorkspaces("myDatabricksWorkspace",
    account_id="my-account-id",  # Replace with your account ID
    aws_region="us-west-2",
    workspace_name="myWorkspace",
    pricing_tier="premium",
    is_no_public_ip_enabled=True,
    deployment_name="myDeployment",
)

# Create a user in Databricks
user = databricks.User("alice",
    user_name="alice@example.com",
    display_name="Alice",
)

# Assign permissions to the user on a workspace
permission_assignment = databricks.PermissionAssignment("aliceWorkspacePermissions",
    principal_id=user.acl_principal_id,
    permissions=["CAN_USE"],  # List of permissions you want to assign
)

# Export the workspace URL and user ID to the Pulumi stack output
pulumi.export("workspaceUrl", databricks_workspace.workspace_url)
pulumi.export("userId", user.user_name)

```

In this Pulumi program:

- We instantiate a `MwsWorkspaces` class to create a new Databricks workspace.
- We create a new user using the `User` resource.
- We assign basic 'CAN_USE' permissions to the user for the workspace using the `PermissionAssignment` resource.

This is just a starting point. In practice, you'd establish more detailed permisions, users, roles, and potentially integrate with identity providers.

Remember to replace placeholders like `"my-account-id"` with your actual Databricks account ID or other proper configurations. For more sophisticated setups, you would manage access across clusters, notebooks, and other entities with similar resources.

You can refer to the [pulumi_databricks documentation](https://www.pulumi.com/registry/packages/databricks/api-docs/mwsworkspaces/) for detailed information on the resources and their configurations.