Manage GCP Dataproc Metastore Service IAM Policies

The gcp:dataproc/metastoreServiceIamPolicy:MetastoreServiceIamPolicy resource, part of the Pulumi GCP provider, manages IAM access control for Dataproc Metastore services at three levels: complete policy replacement, role-level binding, or individual member grants. This guide focuses on three capabilities: complete policy replacement (MetastoreServiceIamPolicy), role-level member binding (MetastoreServiceIamBinding), and individual member grants (MetastoreServiceIamMember).

These resources reference an existing Dataproc Metastore service. MetastoreServiceIamPolicy cannot be used with MetastoreServiceIamBinding or MetastoreServiceIamMember, as they conflict over policy ownership. MetastoreServiceIamBinding and MetastoreServiceIamMember can coexist only if they manage different roles. The examples are intentionally small. Combine them with your own metastore service and identity management.

Replace the entire IAM policy for a service

When you need complete control over who can access a metastore service, you can set the entire IAM policy at once, replacing any existing permissions.

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

const admin = gcp.organizations.getIAMPolicy({
    bindings: [{
        role: "roles/viewer",
        members: ["user:jane@example.com"],
    }],
});
const policy = new gcp.dataproc.MetastoreServiceIamPolicy("policy", {
    project: _default.project,
    location: _default.location,
    serviceId: _default.serviceId,
    policyData: admin.then(admin => admin.policyData),
});

import pulumi
import pulumi_gcp as gcp

admin = gcp.organizations.get_iam_policy(bindings=[{
    "role": "roles/viewer",
    "members": ["user:jane@example.com"],
}])
policy = gcp.dataproc.MetastoreServiceIamPolicy("policy",
    project=default["project"],
    location=default["location"],
    service_id=default["serviceId"],
    policy_data=admin.policy_data)

package main

import (
	"github.com/pulumi/pulumi-gcp/sdk/v9/go/gcp/dataproc"
	"github.com/pulumi/pulumi-gcp/sdk/v9/go/gcp/organizations"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		admin, err := organizations.LookupIAMPolicy(ctx, &organizations.LookupIAMPolicyArgs{
			Bindings: []organizations.GetIAMPolicyBinding{
				{
					Role: "roles/viewer",
					Members: []string{
						"user:jane@example.com",
					},
				},
			},
		}, nil)
		if err != nil {
			return err
		}
		_, err = dataproc.NewMetastoreServiceIamPolicy(ctx, "policy", &dataproc.MetastoreServiceIamPolicyArgs{
			Project:    pulumi.Any(_default.Project),
			Location:   pulumi.Any(_default.Location),
			ServiceId:  pulumi.Any(_default.ServiceId),
			PolicyData: pulumi.String(admin.PolicyData),
		})
		if err != nil {
			return err
		}
		return nil
	})
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Gcp = Pulumi.Gcp;

return await Deployment.RunAsync(() => 
{
    var admin = Gcp.Organizations.GetIAMPolicy.Invoke(new()
    {
        Bindings = new[]
        {
            new Gcp.Organizations.Inputs.GetIAMPolicyBindingInputArgs
            {
                Role = "roles/viewer",
                Members = new[]
                {
                    "user:jane@example.com",
                },
            },
        },
    });

    var policy = new Gcp.Dataproc.MetastoreServiceIamPolicy("policy", new()
    {
        Project = @default.Project,
        Location = @default.Location,
        ServiceId = @default.ServiceId,
        PolicyData = admin.Apply(getIAMPolicyResult => getIAMPolicyResult.PolicyData),
    });

});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.gcp.organizations.OrganizationsFunctions;
import com.pulumi.gcp.organizations.inputs.GetIAMPolicyArgs;
import com.pulumi.gcp.dataproc.MetastoreServiceIamPolicy;
import com.pulumi.gcp.dataproc.MetastoreServiceIamPolicyArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    public static void stack(Context ctx) {
        final var admin = OrganizationsFunctions.getIAMPolicy(GetIAMPolicyArgs.builder()
            .bindings(GetIAMPolicyBindingArgs.builder()
                .role("roles/viewer")
                .members("user:jane@example.com")
                .build())
            .build());

        var policy = new MetastoreServiceIamPolicy("policy", MetastoreServiceIamPolicyArgs.builder()
            .project(default_.project())
            .location(default_.location())
            .serviceId(default_.serviceId())
            .policyData(admin.policyData())
            .build());

    }
}

resources:
  policy:
    type: gcp:dataproc:MetastoreServiceIamPolicy
    properties:
      project: ${default.project}
      location: ${default.location}
      serviceId: ${default.serviceId}
      policyData: ${admin.policyData}
variables:
  admin:
    fn::invoke:
      function: gcp:organizations:getIAMPolicy
      arguments:
        bindings:
          - role: roles/viewer
            members:
              - user:jane@example.com

The MetastoreServiceIamPolicy resource is authoritative: it replaces the entire IAM policy on the service. The policyData property accepts output from the getIAMPolicy data source, which defines roles and members in a structured format. This approach gives you full control but overwrites any permissions not included in the policy data.

Grant a role to multiple members at once

When multiple users or service accounts need the same level of access, you can bind them all to a single role without affecting other role assignments.

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

const binding = new gcp.dataproc.MetastoreServiceIamBinding("binding", {
    project: _default.project,
    location: _default.location,
    serviceId: _default.serviceId,
    role: "roles/viewer",
    members: ["user:jane@example.com"],
});

import pulumi
import pulumi_gcp as gcp

binding = gcp.dataproc.MetastoreServiceIamBinding("binding",
    project=default["project"],
    location=default["location"],
    service_id=default["serviceId"],
    role="roles/viewer",
    members=["user:jane@example.com"])

package main

import (
	"github.com/pulumi/pulumi-gcp/sdk/v9/go/gcp/dataproc"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		_, err := dataproc.NewMetastoreServiceIamBinding(ctx, "binding", &dataproc.MetastoreServiceIamBindingArgs{
			Project:   pulumi.Any(_default.Project),
			Location:  pulumi.Any(_default.Location),
			ServiceId: pulumi.Any(_default.ServiceId),
			Role:      pulumi.String("roles/viewer"),
			Members: pulumi.StringArray{
				pulumi.String("user:jane@example.com"),
			},
		})
		if err != nil {
			return err
		}
		return nil
	})
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Gcp = Pulumi.Gcp;

return await Deployment.RunAsync(() => 
{
    var binding = new Gcp.Dataproc.MetastoreServiceIamBinding("binding", new()
    {
        Project = @default.Project,
        Location = @default.Location,
        ServiceId = @default.ServiceId,
        Role = "roles/viewer",
        Members = new[]
        {
            "user:jane@example.com",
        },
    });

});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.gcp.dataproc.MetastoreServiceIamBinding;
import com.pulumi.gcp.dataproc.MetastoreServiceIamBindingArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    public static void stack(Context ctx) {
        var binding = new MetastoreServiceIamBinding("binding", MetastoreServiceIamBindingArgs.builder()
            .project(default_.project())
            .location(default_.location())
            .serviceId(default_.serviceId())
            .role("roles/viewer")
            .members("user:jane@example.com")
            .build());

    }
}

resources:
  binding:
    type: gcp:dataproc:MetastoreServiceIamBinding
    properties:
      project: ${default.project}
      location: ${default.location}
      serviceId: ${default.serviceId}
      role: roles/viewer
      members:
        - user:jane@example.com

The MetastoreServiceIamBinding resource is authoritative for a specific role: it replaces all members assigned to that role while preserving other roles on the service. The members property accepts a list of identities (users, service accounts, groups) that should have the specified role. Other roles remain unchanged.

Add a single member to a role incrementally

When you need to grant access to one user without disturbing existing permissions, you can add individual members to roles one at a time.

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

const member = new gcp.dataproc.MetastoreServiceIamMember("member", {
    project: _default.project,
    location: _default.location,
    serviceId: _default.serviceId,
    role: "roles/viewer",
    member: "user:jane@example.com",
});

import pulumi
import pulumi_gcp as gcp

member = gcp.dataproc.MetastoreServiceIamMember("member",
    project=default["project"],
    location=default["location"],
    service_id=default["serviceId"],
    role="roles/viewer",
    member="user:jane@example.com")

package main

import (
	"github.com/pulumi/pulumi-gcp/sdk/v9/go/gcp/dataproc"
	"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
		_, err := dataproc.NewMetastoreServiceIamMember(ctx, "member", &dataproc.MetastoreServiceIamMemberArgs{
			Project:   pulumi.Any(_default.Project),
			Location:  pulumi.Any(_default.Location),
			ServiceId: pulumi.Any(_default.ServiceId),
			Role:      pulumi.String("roles/viewer"),
			Member:    pulumi.String("user:jane@example.com"),
		})
		if err != nil {
			return err
		}
		return nil
	})
}

using System.Collections.Generic;
using System.Linq;
using Pulumi;
using Gcp = Pulumi.Gcp;

return await Deployment.RunAsync(() => 
{
    var member = new Gcp.Dataproc.MetastoreServiceIamMember("member", new()
    {
        Project = @default.Project,
        Location = @default.Location,
        ServiceId = @default.ServiceId,
        Role = "roles/viewer",
        Member = "user:jane@example.com",
    });

});

package generated_program;

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.core.Output;
import com.pulumi.gcp.dataproc.MetastoreServiceIamMember;
import com.pulumi.gcp.dataproc.MetastoreServiceIamMemberArgs;
import java.util.List;
import java.util.ArrayList;
import java.util.Map;
import java.io.File;
import java.nio.file.Files;
import java.nio.file.Paths;

public class App {
    public static void main(String[] args) {
        Pulumi.run(App::stack);
    }

    public static void stack(Context ctx) {
        var member = new MetastoreServiceIamMember("member", MetastoreServiceIamMemberArgs.builder()
            .project(default_.project())
            .location(default_.location())
            .serviceId(default_.serviceId())
            .role("roles/viewer")
            .member("user:jane@example.com")
            .build());

    }
}

resources:
  member:
    type: gcp:dataproc:MetastoreServiceIamMember
    properties:
      project: ${default.project}
      location: ${default.location}
      serviceId: ${default.serviceId}
      role: roles/viewer
      member: user:jane@example.com

The MetastoreServiceIamMember resource is non-authoritative: it adds one member to a role without affecting other members who already have that role. The member property specifies a single identity. This approach is safest when multiple teams manage access to the same service, as it avoids accidentally removing permissions.

Beyond these examples

These snippets focus on specific IAM management features: authoritative vs non-authoritative access control, and policy-level, role-level, and member-level configuration. They’re intentionally minimal rather than full access control systems.

The examples reference pre-existing infrastructure such as Dataproc Metastore services (serviceId reference) and GCP project and location configuration. They focus on configuring IAM policies rather than provisioning the metastore service itself.

To keep things focused, common IAM patterns are omitted, including:

Conditional IAM bindings (condition blocks)
Custom role definitions
Service account creation and management
Cross-project or organization-level policies

These omissions are intentional: the goal is to illustrate how each IAM resource type is wired, not provide drop-in access control modules. See the Dataproc Metastore Service IAM Policy resource reference for all available configuration options.

Let's manage GCP Dataproc Metastore Service IAM Policies

Get started with Pulumi Cloud, then follow our quick setup guide to deploy this infrastructure.

Try Pulumi Cloud for FREE

Frequently Asked Questions

Resource Selection & Conflicts

Which IAM resource should I use for managing Dataproc Metastore permissions?

You have three options:

MetastoreServiceIamPolicy - Authoritative. Replaces the entire IAM policy for the service.
MetastoreServiceIamBinding - Authoritative for a specific role. Preserves other roles in the policy.
MetastoreServiceIamMember - Non-authoritative. Adds a single member to a role, preserving other members.

Why am I seeing conflicts when using MetastoreServiceIamPolicy with other IAM resources?

MetastoreServiceIamPolicy cannot be used with MetastoreServiceIamBinding or MetastoreServiceIamMember because they compete to manage the same policy. Choose one approach: use Policy alone for full control, or use Binding/Member for granular management.

Can I use MetastoreServiceIamBinding and MetastoreServiceIamMember together?

Yes, but only if they manage different roles. If both resources grant privileges to the same role, they will conflict.

Configuration & Setup

How do I generate the policyData for MetastoreServiceIamPolicy?

Use the gcp.organizations.getIAMPolicy data source to generate policyData, then pass it to the MetastoreServiceIamPolicy resource as shown in the examples.

Properties & Defaults

What's the default location for a Dataproc Metastore service?

The default location is global. Note that location is immutable after creation.

Using a different cloud?

Explore security guides for other cloud providers:

AWS Guides Azure Guides