Docs Packages
  1. Packages
  2. Databricks
  3. API Docs
  4. Volume
Databricks v1.38.0 published on Monday, Apr 29, 2024 by Pulumi
pulumi/pulumi-databricks

databricks.Volume

Explore with Pulumi AI

databricks logo
Databricks v1.38.0 published on Monday, Apr 29, 2024 by Pulumi
pulumi/pulumi-databricks

    Public Preview This feature is in Public Preview.

    Note This resource could be only used with workspace-level provider!

    Volumes are Unity Catalog objects representing a logical volume of storage in a cloud object storage location. Volumes provide capabilities for accessing, storing, governing, and organizing files. While tables provide governance over tabular datasets, volumes add governance over non-tabular datasets. You can use volumes to store and access files in any format, including structured, semi-structured, and unstructured data.

    A volume resides in the third layer of Unity Catalog’s three-level namespace. Volumes are siblings to tables, views, and other objects organized under a schema in Unity Catalog.

    A volume can be managed or external.

    A managed volume is a Unity Catalog-governed storage volume created within the default storage location of the containing schema. Managed volumes allow the creation of governed storage for working with files without the overhead of external locations and storage credentials. You do not need to specify a location when creating a managed volume, and all file access for data in managed volumes is through paths managed by Unity Catalog.

    An external volume is a Unity Catalog-governed storage volume registered against a directory within an external location.

    A volume can be referenced using its identifier: <catalogName>.<schemaName>.<volumeName>, where:

    • <catalogName>: The name of the catalog containing the Volume.
    • <schemaName>: The name of the schema containing the Volume.
    • <volumeName>: The name of the Volume. It identifies the volume object.

    The path to access files in volumes uses the following format:

    /Volumes/<catalog>/<schema>/<volume>/<path>/<file_name>

    Databricks also supports an optional dbfs:/ scheme, so the following path also works:

    dbfs:/Volumes/<catalog>/<schema>/<volume>/<path>/<file_name>

    This resource manages Volumes in Unity Catalog.

    Example Usage

    import * as pulumi from "@pulumi/pulumi";
import * as databricks from "@pulumi/databricks";

const sandbox = new databricks.Catalog("sandbox", {
    name: "sandbox",
    comment: "this catalog is managed by terraform",
    properties: {
        purpose: "testing",
    },
});
const things = new databricks.Schema("things", {
    catalogName: sandbox.name,
    name: "things",
    comment: "this schema is managed by terraform",
    properties: {
        kind: "various",
    },
});
const external = new databricks.StorageCredential("external", {
    name: "creds",
    awsIamRole: {
        roleArn: externalDataAccess.arn,
    },
});
const some = new databricks.ExternalLocation("some", {
    name: "external-location",
    url: `s3://${externalAwsS3Bucket.id}/some`,
    credentialName: external.name,
});
const _this = new databricks.Volume("this", {
    name: "quickstart_volume",
    catalogName: sandbox.name,
    schemaName: things.name,
    volumeType: "EXTERNAL",
    storageLocation: some.url,
    comment: "this volume is managed by terraform",
});

    import pulumi
import pulumi_databricks as databricks

sandbox = databricks.Catalog("sandbox",
    name="sandbox",
    comment="this catalog is managed by terraform",
    properties={
        "purpose": "testing",
    })
things = databricks.Schema("things",
    catalog_name=sandbox.name,
    name="things",
    comment="this schema is managed by terraform",
    properties={
        "kind": "various",
    })
external = databricks.StorageCredential("external",
    name="creds",
    aws_iam_role=databricks.StorageCredentialAwsIamRoleArgs(
        role_arn=external_data_access["arn"],
    ))
some = databricks.ExternalLocation("some",
    name="external-location",
    url=f"s3://{external_aws_s3_bucket['id']}/some",
    credential_name=external.name)
this = databricks.Volume("this",
    name="quickstart_volume",
    catalog_name=sandbox.name,
    schema_name=things.name,
    volume_type="EXTERNAL",
    storage_location=some.url,
    comment="this volume is managed by terraform")

    package main

import (
	"fmt"

	"github.com/pulumi/pulumi-databricks/sdk/go/databricks"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		sandbox, err := databricks.NewCatalog(ctx, "sandbox", &databricks.CatalogArgs{
			Name:    pulumi.String("sandbox"),
			Comment: pulumi.String("this catalog is managed by terraform"),
			Properties: pulumi.Map{
				"purpose": pulumi.Any("testing"),
			},
		})
		if err != nil {
			return err
		}
		things, err := databricks.NewSchema(ctx, "things", &databricks.SchemaArgs{
			CatalogName: sandbox.Name,
			Name:        pulumi.String("things"),
			Comment:     pulumi.String("this schema is managed by terraform"),
			Properties: pulumi.Map{
				"kind": pulumi.Any("various"),
			},
		})
		if err != nil {
			return err
		}
		external, err := databricks.NewStorageCredential(ctx, "external", &databricks.StorageCredentialArgs{
			Name: pulumi.String("creds"),
			AwsIamRole: &databricks.StorageCredentialAwsIamRoleArgs{
				RoleArn: pulumi.Any(externalDataAccess.Arn),
			},
		})
		if err != nil {
			return err
		}
		some, err := databricks.NewExternalLocation(ctx, "some", &databricks.ExternalLocationArgs{
			Name:           pulumi.String("external-location"),
			Url:            pulumi.String(fmt.Sprintf("s3://%v/some", externalAwsS3Bucket.Id)),
			CredentialName: external.Name,
		})
		if err != nil {
			return err
		}
		_, err = databricks.NewVolume(ctx, "this", &databricks.VolumeArgs{
			Name:            pulumi.String("quickstart_volume"),
			CatalogName:     sandbox.Name,
			SchemaName:      things.Name,
			VolumeType:      pulumi.String("EXTERNAL"),
			StorageLocation: some.Url,
			Comment:         pulumi.String("this volume is managed by terraform"),
		})
		if err != nil {
			return err
		}
		return nil
	})
}

    using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Databricks = Pulumi.Databricks;

return await Deployment.RunAsync(() => 
{
    var sandbox = new Databricks.Catalog("sandbox", new()
    {
        Name = "sandbox",
        Comment = "this catalog is managed by terraform",
        Properties = 
        {
            { "purpose", "testing" },
        },
    });

    var things = new Databricks.Schema("things", new()
    {
        CatalogName = sandbox.Name,
        Name = "things",
        Comment = "this schema is managed by terraform",
        Properties = 
        {
            { "kind", "various" },
        },
    });

    var external = new Databricks.StorageCredential("external", new()
    {
        Name = "creds",
        AwsIamRole = new Databricks.Inputs.StorageCredentialAwsIamRoleArgs
        {
            RoleArn = externalDataAccess.Arn,
        },
    });

    var some = new Databricks.ExternalLocation("some", new()
    {
        Name = "external-location",
        Url = $"s3://{externalAwsS3Bucket.Id}/some",
        CredentialName = external.Name,
    });

    var @this = new Databricks.Volume("this", new()
    {
        Name = "quickstart_volume",
        CatalogName = sandbox.Name,
        SchemaName = things.Name,
        VolumeType = "EXTERNAL",
        StorageLocation = some.Url,
        Comment = "this volume is managed by terraform",
    });

});

    package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.databricks.Catalog;
import com.pulumi.databricks.CatalogArgs;
import com.pulumi.databricks.Schema;
import com.pulumi.databricks.SchemaArgs;
import com.pulumi.databricks.StorageCredential;
import com.pulumi.databricks.StorageCredentialArgs;
import com.pulumi.databricks.inputs.StorageCredentialAwsIamRoleArgs;
import com.pulumi.databricks.ExternalLocation;
import com.pulumi.databricks.ExternalLocationArgs;
import com.pulumi.databricks.Volume;
import com.pulumi.databricks.VolumeArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    public static void stack(Context ctx) {
        var sandbox = new Catalog("sandbox", CatalogArgs.builder()        
            .name("sandbox")
            .comment("this catalog is managed by terraform")
            .properties(Map.of("purpose", "testing"))
            .build());

        var things = new Schema("things", SchemaArgs.builder()        
            .catalogName(sandbox.name())
            .name("things")
            .comment("this schema is managed by terraform")
            .properties(Map.of("kind", "various"))
            .build());

        var external = new StorageCredential("external", StorageCredentialArgs.builder()        
            .name("creds")
            .awsIamRole(StorageCredentialAwsIamRoleArgs.builder()
                .roleArn(externalDataAccess.arn())
                .build())
            .build());

        var some = new ExternalLocation("some", ExternalLocationArgs.builder()        
            .name("external-location")
            .url(String.format("s3://%s/some", externalAwsS3Bucket.id()))
            .credentialName(external.name())
            .build());

        var this_ = new Volume("this", VolumeArgs.builder()        
            .name("quickstart_volume")
            .catalogName(sandbox.name())
            .schemaName(things.name())
            .volumeType("EXTERNAL")
            .storageLocation(some.url())
            .comment("this volume is managed by terraform")
            .build());

    }
}

    resources:
  sandbox:
    type: databricks:Catalog
    properties:
      name: sandbox
      comment: this catalog is managed by terraform
      properties:
        purpose: testing
  things:
    type: databricks:Schema
    properties:
      catalogName: ${sandbox.name}
      name: things
      comment: this schema is managed by terraform
      properties:
        kind: various
  external:
    type: databricks:StorageCredential
    properties:
      name: creds
      awsIamRole:
        roleArn: ${externalDataAccess.arn}
  some:
    type: databricks:ExternalLocation
    properties:
      name: external-location
      url: s3://${externalAwsS3Bucket.id}/some
      credentialName: ${external.name}
  this:
    type: databricks:Volume
    properties:
      name: quickstart_volume
      catalogName: ${sandbox.name}
      schemaName: ${things.name}
      volumeType: EXTERNAL
      storageLocation: ${some.url}
      comment: this volume is managed by terraform

    Create Volume Resource

    Resources are created with functions called constructors. To learn more about declaring and configuring resources, see Resources.

    Constructor syntax

    new Volume(name: string, args: VolumeArgs, opts?: CustomResourceOptions);
    @overload
def Volume(resource_name: str,
           args: VolumeArgs,
           opts: Optional[ResourceOptions] = None)

@overload
def Volume(resource_name: str,
           opts: Optional[ResourceOptions] = None,
           catalog_name: Optional[str] = None,
           schema_name: Optional[str] = None,
           volume_type: Optional[str] = None,
           comment: Optional[str] = None,
           name: Optional[str] = None,
           owner: Optional[str] = None,
           storage_location: Optional[str] = None)
    func NewVolume(ctx *Context, name string, args VolumeArgs, opts ...ResourceOption) (*Volume, error)
    public Volume(string name, VolumeArgs args, CustomResourceOptions? opts = null)
    public Volume(String name, VolumeArgs args)
public Volume(String name, VolumeArgs args, CustomResourceOptions options)

    type: databricks:Volume
properties: # The arguments to resource properties.
options: # Bag of options to control resource's behavior.

    Parameters

    name string
    The unique name of the resource.
    args VolumeArgs
    The arguments to resource properties.
    opts CustomResourceOptions
    Bag of options to control resource's behavior.
    resource_name str
    The unique name of the resource.
    args VolumeArgs
    The arguments to resource properties.
    opts ResourceOptions
    Bag of options to control resource's behavior.
    ctx Context
    Context object for the current deployment.
    name string
    The unique name of the resource.
    args VolumeArgs
    The arguments to resource properties.
    opts ResourceOption
    Bag of options to control resource's behavior.
    name string
    The unique name of the resource.
    args VolumeArgs
    The arguments to resource properties.
    opts CustomResourceOptions
    Bag of options to control resource's behavior.
    name String
    The unique name of the resource.
    args VolumeArgs
    The arguments to resource properties.
    options CustomResourceOptions
    Bag of options to control resource's behavior.

    Example

    The following reference example uses placeholder values for all input properties.

    var volumeResource = new Databricks.Volume("volumeResource", new()
{
    CatalogName = "string",
    SchemaName = "string",
    VolumeType = "string",
    Comment = "string",
    Name = "string",
    Owner = "string",
    StorageLocation = "string",
});

    example, err := databricks.NewVolume(ctx, "volumeResource", &databricks.VolumeArgs{
	CatalogName:     pulumi.String("string"),
	SchemaName:      pulumi.String("string"),
	VolumeType:      pulumi.String("string"),
	Comment:         pulumi.String("string"),
	Name:            pulumi.String("string"),
	Owner:           pulumi.String("string"),
	StorageLocation: pulumi.String("string"),
})

    var volumeResource = new Volume("volumeResource", VolumeArgs.builder()        
    .catalogName("string")
    .schemaName("string")
    .volumeType("string")
    .comment("string")
    .name("string")
    .owner("string")
    .storageLocation("string")
    .build());

    volume_resource = databricks.Volume("volumeResource",
    catalog_name="string",
    schema_name="string",
    volume_type="string",
    comment="string",
    name="string",
    owner="string",
    storage_location="string")

    const volumeResource = new databricks.Volume("volumeResource", {
    catalogName: "string",
    schemaName: "string",
    volumeType: "string",
    comment: "string",
    name: "string",
    owner: "string",
    storageLocation: "string",
});

    type: databricks:Volume
properties:
    catalogName: string
    comment: string
    name: string
    owner: string
    schemaName: string
    storageLocation: string
    volumeType: string

    Volume Resource Properties

    To learn more about resource properties and how to use them, see Inputs and Outputs in the Architecture and Concepts docs.

    Inputs

    The Volume resource accepts the following input properties:

    CatalogName string
    Name of parent Catalog. Change forces creation of a new resource.
    SchemaName string
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    VolumeType string
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.
    Comment string
    Free-form text.
    Name string
    Name of the Volume
    Owner string
    Name of the volume owner.
    StorageLocation string
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.
    CatalogName string
    Name of parent Catalog. Change forces creation of a new resource.
    SchemaName string
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    VolumeType string
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.
    Comment string
    Free-form text.
    Name string
    Name of the Volume
    Owner string
    Name of the volume owner.
    StorageLocation string
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.
    catalogName String
    Name of parent Catalog. Change forces creation of a new resource.
    schemaName String
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    volumeType String
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.
    comment String
    Free-form text.
    name String
    Name of the Volume
    owner String
    Name of the volume owner.
    storageLocation String
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.
    catalogName string
    Name of parent Catalog. Change forces creation of a new resource.
    schemaName string
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    volumeType string
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.
    comment string
    Free-form text.
    name string
    Name of the Volume
    owner string
    Name of the volume owner.
    storageLocation string
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.
    catalog_name str
    Name of parent Catalog. Change forces creation of a new resource.
    schema_name str
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    volume_type str
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.
    comment str
    Free-form text.
    name str
    Name of the Volume
    owner str
    Name of the volume owner.
    storage_location str
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.
    catalogName String
    Name of parent Catalog. Change forces creation of a new resource.
    schemaName String
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    volumeType String
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.
    comment String
    Free-form text.
    name String
    Name of the Volume
    owner String
    Name of the volume owner.
    storageLocation String
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.

    Outputs

    All input properties are implicitly available as output properties. Additionally, the Volume resource produces the following output properties:

    Id string
    The provider-assigned unique ID for this managed resource.
    VolumePath string
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.
    Id string
    The provider-assigned unique ID for this managed resource.
    VolumePath string
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.
    id String
    The provider-assigned unique ID for this managed resource.
    volumePath String
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.
    id string
    The provider-assigned unique ID for this managed resource.
    volumePath string
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.
    id str
    The provider-assigned unique ID for this managed resource.
    volume_path str
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.
    id String
    The provider-assigned unique ID for this managed resource.
    volumePath String
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.

    Look up Existing Volume Resource

    Get an existing Volume resource’s state with the given name, ID, and optional extra properties used to qualify the lookup.

    public static get(name: string, id: Input<ID>, state?: VolumeState, opts?: CustomResourceOptions): Volume
    @staticmethod
def get(resource_name: str,
        id: str,
        opts: Optional[ResourceOptions] = None,
        catalog_name: Optional[str] = None,
        comment: Optional[str] = None,
        name: Optional[str] = None,
        owner: Optional[str] = None,
        schema_name: Optional[str] = None,
        storage_location: Optional[str] = None,
        volume_path: Optional[str] = None,
        volume_type: Optional[str] = None) -> Volume
    func GetVolume(ctx *Context, name string, id IDInput, state *VolumeState, opts ...ResourceOption) (*Volume, error)
    public static Volume Get(string name, Input<string> id, VolumeState? state, CustomResourceOptions? opts = null)
    public static Volume get(String name, Output<String> id, VolumeState state, CustomResourceOptions options)
    Resource lookup is not supported in YAML
    name
    The unique name of the resulting resource.
    id
    The unique provider ID of the resource to lookup.
    state
    Any extra arguments used during the lookup.
    opts
    A bag of options that control this resource's behavior.
    resource_name
    The unique name of the resulting resource.
    id
    The unique provider ID of the resource to lookup.
    name
    The unique name of the resulting resource.
    id
    The unique provider ID of the resource to lookup.
    state
    Any extra arguments used during the lookup.
    opts
    A bag of options that control this resource's behavior.
    name
    The unique name of the resulting resource.
    id
    The unique provider ID of the resource to lookup.
    state
    Any extra arguments used during the lookup.
    opts
    A bag of options that control this resource's behavior.
    name
    The unique name of the resulting resource.
    id
    The unique provider ID of the resource to lookup.
    state
    Any extra arguments used during the lookup.
    opts
    A bag of options that control this resource's behavior.
    The following state arguments are supported:
    CatalogName string
    Name of parent Catalog. Change forces creation of a new resource.
    Comment string
    Free-form text.
    Name string
    Name of the Volume
    Owner string
    Name of the volume owner.
    SchemaName string
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    StorageLocation string
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.
    VolumePath string
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.
    VolumeType string
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.
    CatalogName string
    Name of parent Catalog. Change forces creation of a new resource.
    Comment string
    Free-form text.
    Name string
    Name of the Volume
    Owner string
    Name of the volume owner.
    SchemaName string
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    StorageLocation string
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.
    VolumePath string
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.
    VolumeType string
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.
    catalogName String
    Name of parent Catalog. Change forces creation of a new resource.
    comment String
    Free-form text.
    name String
    Name of the Volume
    owner String
    Name of the volume owner.
    schemaName String
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    storageLocation String
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.
    volumePath String
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.
    volumeType String
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.
    catalogName string
    Name of parent Catalog. Change forces creation of a new resource.
    comment string
    Free-form text.
    name string
    Name of the Volume
    owner string
    Name of the volume owner.
    schemaName string
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    storageLocation string
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.
    volumePath string
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.
    volumeType string
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.
    catalog_name str
    Name of parent Catalog. Change forces creation of a new resource.
    comment str
    Free-form text.
    name str
    Name of the Volume
    owner str
    Name of the volume owner.
    schema_name str
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    storage_location str
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.
    volume_path str
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.
    volume_type str
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.
    catalogName String
    Name of parent Catalog. Change forces creation of a new resource.
    comment String
    Free-form text.
    name String
    Name of the Volume
    owner String
    Name of the volume owner.
    schemaName String
    Name of parent Schema relative to parent Catalog. Change forces creation of a new resource.
    storageLocation String
    Path inside an External Location. Only used for EXTERNAL Volumes. Change forces creation of a new resource.
    volumePath String
    base file path for this Unity Catalog Volume in form of /Volumes/<catalog>/<schema>/<name>.
    volumeType String
    Volume type. EXTERNAL or MANAGED. Change forces creation of a new resource.

    Import

    This resource can be imported by full_name which is the 3-level Volume identifier: <catalog>.<schema>.<name>

    bash

    $ pulumi import databricks:index/volume:Volume this <catalog_name>.<schema_name>.<name>

    To learn more about importing existing cloud resources, see Importing resources.

    Package Details

    Repository
    databricks pulumi/pulumi-databricks
    License
    Apache-2.0
    Notes
    This Pulumi package is based on the databricks Terraform Provider.
    databricks logo
    Databricks v1.38.0 published on Monday, Apr 29, 2024 by Pulumi
    pulumi/pulumi-databricks