Configure Azure Data Share ADLS Gen2 Folder Datasets

The azure-native:datashare:ADLSGen2FolderDataSet resource, part of the Pulumi Azure Native provider, registers an ADLS Gen2 folder as a dataset within an Azure Data Share, enabling folder-level sharing with data consumers. This guide focuses on one capability: dataset registration within existing shares.

Datasets belong to Data Share accounts and shares, and reference ADLS Gen2 storage accounts with specific folders. The example is intentionally minimal. Combine it with your own Data Share infrastructure and storage account details.

Add an ADLS Gen2 folder to a data share

Data providers register folders from ADLS Gen2 storage accounts as datasets within shares, making them available to consumers.

import * as pulumi from "@pulumi/pulumi";
import * as azure_native from "@pulumi/azure-native";

const adlsGen2FolderDataSet = new azure_native.datashare.ADLSGen2FolderDataSet("adlsGen2FolderDataSet", {
    accountName: "Account1",
    dataSetName: "Dataset1",
    resourceGroupName: "SampleResourceGroup",
    shareName: "Share1",
});

import pulumi
import pulumi_azure_native as azure_native

adls_gen2_folder_data_set = azure_native.datashare.ADLSGen2FolderDataSet("adlsGen2FolderDataSet",
    account_name="Account1",
    data_set_name="Dataset1",
    resource_group_name="SampleResourceGroup",
    share_name="Share1")

package main

import (
	datashare "github.com/pulumi/pulumi-azure-native-sdk/datashare/v3"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		_, err := datashare.NewADLSGen2FolderDataSet(ctx, "adlsGen2FolderDataSet", &datashare.ADLSGen2FolderDataSetArgs{
			AccountName:       pulumi.String("Account1"),
			DataSetName:       pulumi.String("Dataset1"),
			ResourceGroupName: pulumi.String("SampleResourceGroup"),
			ShareName:         pulumi.String("Share1"),
		})
		if err != nil {
			return err
		}
		return nil
	})
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using AzureNative = Pulumi.AzureNative;

return await Deployment.RunAsync(() => 
{
    var adlsGen2FolderDataSet = new AzureNative.DataShare.ADLSGen2FolderDataSet("adlsGen2FolderDataSet", new()
    {
        AccountName = "Account1",
        DataSetName = "Dataset1",
        ResourceGroupName = "SampleResourceGroup",
        ShareName = "Share1",
    });

});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.azurenative.datashare.ADLSGen2FolderDataSet;
import com.pulumi.azurenative.datashare.ADLSGen2FolderDataSetArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    public static void stack(Context ctx) {
        var adlsGen2FolderDataSet = new ADLSGen2FolderDataSet("adlsGen2FolderDataSet", ADLSGen2FolderDataSetArgs.builder()
            .accountName("Account1")
            .dataSetName("Dataset1")
            .resourceGroupName("SampleResourceGroup")
            .shareName("Share1")
            .build());

    }
}

resources:
  adlsGen2FolderDataSet:
    type: azure-native:datashare:ADLSGen2FolderDataSet
    properties:
      accountName: Account1
      dataSetName: Dataset1
      resourceGroupName: SampleResourceGroup
      shareName: Share1

The accountName and shareName properties identify the parent Data Share account and share. The dataSetName provides a unique identifier for this dataset within the share. This example shows the minimal structure; in practice, you must also specify fileSystem, folderPath, storageAccountName, subscriptionId, and resourceGroup to identify which ADLS Gen2 folder to share.

Beyond these examples

This snippet focuses on dataset registration within shares. It’s intentionally minimal rather than a complete data sharing configuration.

The example references pre-existing infrastructure such as Azure Data Share accounts and shares, ADLS Gen2 storage accounts with folders, and resource groups. It focuses on dataset registration rather than provisioning the surrounding infrastructure.

To keep things focused, common dataset patterns are omitted, including:

  • Storage account identification (fileSystem, folderPath, storageAccountName)
  • Cross-subscription sharing (subscriptionId, resourceGroup)
  • Dataset kind specification (kind property)
  • Import scenarios for existing datasets

These omissions are intentional: the goal is to illustrate how dataset registration is wired, not provide drop-in data sharing modules. See the ADLS Gen2 Folder DataSet resource reference for all available configuration options.

Let's configure Azure Data Share ADLS Gen2 Folder Datasets

Get started with Pulumi Cloud, then follow our quick setup guide to deploy this infrastructure.

Try Pulumi Cloud for FREE

Frequently Asked Questions

Resource Configuration
What properties are required to create an ADLS Gen2 folder dataset?
You need to specify both DataShare location and storage location properties. For DataShare, provide accountName, dataSetName, resourceGroupName, and shareName. For the ADLS Gen2 storage location, provide fileSystem, folderPath, storageAccountName, subscriptionId, resourceGroup, and kind.
What value should I use for the kind property?
Set kind to AdlsGen2Folder. This is the expected discriminator value for ADLS Gen2 folder datasets.
How do I get the unique identifier for my dataset?
The dataSetId output property provides a unique identifier for the dataset resource after creation.
Immutability & Lifecycle
What properties can't I change after creating the dataset?
Four properties are immutable: accountName, dataSetName, resourceGroupName, and shareName. Changing any of these requires replacing the resource.

Using a different cloud?

Explore storage guides for other cloud providers:

AWS Guides GCP Guides