Authenticating REST API requests to Databricks workspace

Question

Pulumi · Accepted Answer

To authenticate REST API requests to a Databricks workspace, you typically use a bearer token which you can obtain through the Databricks REST API. In Pulumi, you can manage this token by using the `Token` resource from the Databricks provider.

Here's how authentication would typically work with Databricks:

1. Create the Databricks workspace.
2. Generate a PAT (Personal Access Token) for secure authentication requests.
3. Use this token to authenticate REST API calls.

Below is a Pulumi TypeScript program that demonstrates how to create a Databricks workspace and then generate an API token that you could use for authenticating REST API requests to the workspace:

```typescript
import * as databricks from "@pulumi/databricks";

// Create a Databricks workspace.
const workspace = new databricks.Workspace("my-databricks-workspace", {
    // Replace with the appropriate values for your setup
    region: "westus",
    sku: "standard",
});

// Generate an API token for the workspace
const token = new databricks.Token("my-databricks-token", {
    // Tokens will have a defined lifetime, after which they can't be used.
    // For example, `lifetimeSeconds: 3600` defines a token that lasts one hour.
    // You can adjust this value to your requirements.
    lifetimeSeconds: 3600,
    // Optional: If you want to annotate your token with a label for easier management,
    // you can add a `comment` property.
    comment: "token for REST API access",
});

// Output the token value, you will use this as the bearer token for authentication.
// Please ensure you handle this value securely as it provides access to your Databricks workspace.
export const tokenValue = token.tokenValue;

```

Explanation of the resources used:

- `databricks.Workspace`: This resource creates a Databricks workspace in the chosen region with the specified SKU.

- `databricks.Token`: This resource generates a personal access token that can be used to authenticate with the Databricks REST API. You should handle the `tokenValue` securely, as it allows access to your Databricks workspace.

Remember that you need to install the Pulumi Databricks provider to use this code by running `npm install @pulumi/databricks`.

**Security Note**: The program exports the generated token value, which will be visible in the Pulumi stack's output. This output should be treated as a sensitive value. In production scenarios, consider using [Pulumi secrets](https://www.pulumi.com/docs/intro/concepts/secrets/) to manage such sensitive values securely.

After the Pulumi program has been run and the token has been obtained, you can use this token to authenticate HTTP requests to the Databricks REST API by including it in the `Authorization` header as a Bearer token (e.g., `Authorization: Bearer <token_value>`).