This page documents the language specification for the gcp package. If you're looking for help working with the inputs, outputs, or functions of gcp resources in a Pulumi program, please see the resource documentation for examples and API reference.

bigquery

This provider is a derived work of the Terraform Provider distributed under MPL 2.0. If you encounter a bug or missing feature, first check the pulumi/pulumi-gcp repo; however, if that doesn’t turn up anything, please consult the source terraform-providers/terraform-provider-google repo.

class pulumi_gcp.bigquery.AppProfile(resource_name, opts=None, app_profile_id=None, description=None, ignore_warnings=None, instance=None, multi_cluster_routing_use_any=None, project=None, single_cluster_routing=None, __props__=None, __name__=None, __opts__=None)

App profile is a configuration object describing how Cloud Bigtable should treat traffic from a particular end user application.

import pulumi
import pulumi_gcp as gcp

instance = gcp.bigtable.Instance("instance",
    cluster=[{
        "cluster_id": "bt-instance",
        "zone": "us-central1-b",
        "num_nodes": 3,
        "storageType": "HDD",
    }],
    deletion_protection="true")
ap = gcp.bigquery.AppProfile("ap",
    instance=instance.name,
    app_profile_id="bt-profile",
    multi_cluster_routing_use_any=True,
    ignore_warnings=True)
import pulumi
import pulumi_gcp as gcp

instance = gcp.bigtable.Instance("instance",
    cluster=[{
        "cluster_id": "bt-instance",
        "zone": "us-central1-b",
        "num_nodes": 3,
        "storageType": "HDD",
    }],
    deletion_protection="true")
ap = gcp.bigquery.AppProfile("ap",
    instance=instance.name,
    app_profile_id="bt-profile",
    single_cluster_routing={
        "cluster_id": "bt-instance",
        "allowTransactionalWrites": True,
    },
    ignore_warnings=True)
Parameters
  • resource_name (str) – The name of the resource.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • app_profile_id (pulumi.Input[str]) – The unique name of the app profile in the form [_a-zA-Z0-9][-_.a-zA-Z0-9]*.

  • description (pulumi.Input[str]) – Long form description of the use case for this app profile.

  • ignore_warnings (pulumi.Input[bool]) – If true, ignore safety checks when deleting/updating the app profile.

  • instance (pulumi.Input[str]) – The name of the instance to create the app profile within.

  • multi_cluster_routing_use_any (pulumi.Input[bool]) – If true, read/write requests are routed to the nearest cluster in the instance, and will fail over to the nearest cluster that is available in the event of transient errors or delays. Clusters in a region are considered equidistant. Choosing this option sacrifices read-your-writes consistency to improve availability.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • single_cluster_routing (pulumi.Input[dict]) – Use a single-cluster routing policy. Structure is documented below.

The single_cluster_routing object supports the following:

  • allowTransactionalWrites (pulumi.Input[bool]) - If true, CheckAndMutateRow and ReadModifyWriteRow requests are allowed by this app profile. It is unsafe to send these requests to the same table/row/column in multiple clusters.

  • cluster_id (pulumi.Input[str]) - The cluster to which read/write requests should be routed.

app_profile_id: pulumi.Output[str] = None

The unique name of the app profile in the form [_a-zA-Z0-9][-_.a-zA-Z0-9]*.

description: pulumi.Output[str] = None

Long form description of the use case for this app profile.

ignore_warnings: pulumi.Output[bool] = None

If true, ignore safety checks when deleting/updating the app profile.

instance: pulumi.Output[str] = None

The name of the instance to create the app profile within.

multi_cluster_routing_use_any: pulumi.Output[bool] = None

If true, read/write requests are routed to the nearest cluster in the instance, and will fail over to the nearest cluster that is available in the event of transient errors or delays. Clusters in a region are considered equidistant. Choosing this option sacrifices read-your-writes consistency to improve availability.

name: pulumi.Output[str] = None

The unique name of the requested app profile. Values are of the form ‘projects//instances//appProfiles/’.

project: pulumi.Output[str] = None

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

single_cluster_routing: pulumi.Output[dict] = None

Use a single-cluster routing policy. Structure is documented below.

  • allowTransactionalWrites (bool) - If true, CheckAndMutateRow and ReadModifyWriteRow requests are allowed by this app profile. It is unsafe to send these requests to the same table/row/column in multiple clusters.

  • cluster_id (str) - The cluster to which read/write requests should be routed.

static get(resource_name, id, opts=None, app_profile_id=None, description=None, ignore_warnings=None, instance=None, multi_cluster_routing_use_any=None, name=None, project=None, single_cluster_routing=None)

Get an existing AppProfile resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters
  • resource_name (str) – The unique name of the resulting resource.

  • id (str) – The unique provider ID of the resource to lookup.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • app_profile_id (pulumi.Input[str]) – The unique name of the app profile in the form [_a-zA-Z0-9][-_.a-zA-Z0-9]*.

  • description (pulumi.Input[str]) – Long form description of the use case for this app profile.

  • ignore_warnings (pulumi.Input[bool]) – If true, ignore safety checks when deleting/updating the app profile.

  • instance (pulumi.Input[str]) – The name of the instance to create the app profile within.

  • multi_cluster_routing_use_any (pulumi.Input[bool]) – If true, read/write requests are routed to the nearest cluster in the instance, and will fail over to the nearest cluster that is available in the event of transient errors or delays. Clusters in a region are considered equidistant. Choosing this option sacrifices read-your-writes consistency to improve availability.

  • name (pulumi.Input[str]) – The unique name of the requested app profile. Values are of the form ‘projects//instances//appProfiles/’.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • single_cluster_routing (pulumi.Input[dict]) – Use a single-cluster routing policy. Structure is documented below.

The single_cluster_routing object supports the following:

  • allowTransactionalWrites (pulumi.Input[bool]) - If true, CheckAndMutateRow and ReadModifyWriteRow requests are allowed by this app profile. It is unsafe to send these requests to the same table/row/column in multiple clusters.

  • cluster_id (pulumi.Input[str]) - The cluster to which read/write requests should be routed.

translate_output_property(prop)

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

translate_input_property(prop)

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

class pulumi_gcp.bigquery.AwaitableGetDefaultServiceAccountResult(email=None, id=None, project=None)
class pulumi_gcp.bigquery.Connection(resource_name, opts=None, cloud_sql=None, connection_id=None, description=None, friendly_name=None, location=None, project=None, __props__=None, __name__=None, __opts__=None)

A connection allows BigQuery connections to external data sources..

To get more information about Connection, see:

Warning: All arguments including cloud_sql.credential.password will be stored in the raw state as plain-text. Read more about sensitive data in state.

import pulumi
import pulumi_gcp as gcp
import pulumi_random as random

instance = gcp.sql.DatabaseInstance("instance",
    database_version="POSTGRES_11",
    region="us-central1",
    settings={
        "tier": "db-f1-micro",
    })
db = gcp.sql.Database("db", instance=instance.name)
pwd = random.RandomPassword("pwd",
    length=16,
    special=False)
user = gcp.sql.User("user",
    instance=instance.name,
    password=pwd.result)
connection = gcp.bigquery.Connection("connection",
    friendly_name="👋",
    description="a riveting description",
    cloud_sql={
        "instance_id": instance.connection_name,
        "database": db.name,
        "type": "POSTGRES",
        "credential": {
            "username": user.name,
            "password": user.password,
        },
    })
import pulumi
import pulumi_gcp as gcp
import pulumi_random as random

instance = gcp.sql.DatabaseInstance("instance",
    database_version="POSTGRES_11",
    region="us-central1",
    settings={
        "tier": "db-f1-micro",
    })
db = gcp.sql.Database("db", instance=instance.name)
pwd = random.RandomPassword("pwd",
    length=16,
    special=False)
user = gcp.sql.User("user",
    instance=instance.name,
    password=pwd.result)
connection = gcp.bigquery.Connection("connection",
    connection_id="my-connection",
    location="US",
    friendly_name="👋",
    description="a riveting description",
    cloud_sql={
        "instance_id": instance.connection_name,
        "database": db.name,
        "type": "POSTGRES",
        "credential": {
            "username": user.name,
            "password": user.password,
        },
    })
Parameters
  • resource_name (str) – The name of the resource.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • cloud_sql (pulumi.Input[dict]) – Cloud SQL properties. Structure is documented below.

  • connection_id (pulumi.Input[str]) – Optional connection id that should be assigned to the created connection.

  • description (pulumi.Input[str]) – A descriptive description for the connection

  • friendly_name (pulumi.Input[str]) – A descriptive name for the connection

  • location (pulumi.Input[str]) – The geographic location where the connection should reside. Cloud SQL instance must be in the same location as the connection with following exceptions: Cloud SQL us-central1 maps to BigQuery US, Cloud SQL europe-west1 maps to BigQuery EU. Examples: US, EU, asia-northeast1, us-central1, europe-west1. The default value is US.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

The cloud_sql object supports the following:

  • credential (pulumi.Input[dict]) - Cloud SQL properties. Structure is documented below.

    • password (pulumi.Input[str]) - Password for database. Note: This property is sensitive and will not be displayed in the plan.

    • username (pulumi.Input[str]) - Username for database.

  • database (pulumi.Input[str]) - Database name.

  • instance_id (pulumi.Input[str]) - Cloud SQL instance ID in the form project:location:instance.

  • type (pulumi.Input[str]) - Type of the Cloud SQL database.

cloud_sql: pulumi.Output[dict] = None

Cloud SQL properties. Structure is documented below.

  • credential (dict) - Cloud SQL properties. Structure is documented below.

    • password (str) - Password for database. Note: This property is sensitive and will not be displayed in the plan.

    • username (str) - Username for database.

  • database (str) - Database name.

  • instance_id (str) - Cloud SQL instance ID in the form project:location:instance.

  • type (str) - Type of the Cloud SQL database.

connection_id: pulumi.Output[str] = None

Optional connection id that should be assigned to the created connection.

description: pulumi.Output[str] = None

A descriptive description for the connection

friendly_name: pulumi.Output[str] = None

A descriptive name for the connection

has_credential: pulumi.Output[bool] = None

True if the connection has credential assigned.

location: pulumi.Output[str] = None

The geographic location where the connection should reside. Cloud SQL instance must be in the same location as the connection with following exceptions: Cloud SQL us-central1 maps to BigQuery US, Cloud SQL europe-west1 maps to BigQuery EU. Examples: US, EU, asia-northeast1, us-central1, europe-west1. The default value is US.

name: pulumi.Output[str] = None

The resource name of the connection in the form of: “projects/{project_id}/locations/{location_id}/connections/{connectionId}”

project: pulumi.Output[str] = None

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

static get(resource_name, id, opts=None, cloud_sql=None, connection_id=None, description=None, friendly_name=None, has_credential=None, location=None, name=None, project=None)

Get an existing Connection resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters
  • resource_name (str) – The unique name of the resulting resource.

  • id (str) – The unique provider ID of the resource to lookup.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • cloud_sql (pulumi.Input[dict]) – Cloud SQL properties. Structure is documented below.

  • connection_id (pulumi.Input[str]) – Optional connection id that should be assigned to the created connection.

  • description (pulumi.Input[str]) – A descriptive description for the connection

  • friendly_name (pulumi.Input[str]) – A descriptive name for the connection

  • has_credential (pulumi.Input[bool]) – True if the connection has credential assigned.

  • location (pulumi.Input[str]) – The geographic location where the connection should reside. Cloud SQL instance must be in the same location as the connection with following exceptions: Cloud SQL us-central1 maps to BigQuery US, Cloud SQL europe-west1 maps to BigQuery EU. Examples: US, EU, asia-northeast1, us-central1, europe-west1. The default value is US.

  • name (pulumi.Input[str]) – The resource name of the connection in the form of: “projects/{project_id}/locations/{location_id}/connections/{connectionId}”

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

The cloud_sql object supports the following:

  • credential (pulumi.Input[dict]) - Cloud SQL properties. Structure is documented below.

    • password (pulumi.Input[str]) - Password for database. Note: This property is sensitive and will not be displayed in the plan.

    • username (pulumi.Input[str]) - Username for database.

  • database (pulumi.Input[str]) - Database name.

  • instance_id (pulumi.Input[str]) - Cloud SQL instance ID in the form project:location:instance.

  • type (pulumi.Input[str]) - Type of the Cloud SQL database.

translate_output_property(prop)

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

translate_input_property(prop)

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

class pulumi_gcp.bigquery.DataTransferConfig(resource_name, opts=None, data_refresh_window_days=None, data_source_id=None, destination_dataset_id=None, disabled=None, display_name=None, location=None, params=None, project=None, schedule=None, service_account_name=None, __props__=None, __name__=None, __opts__=None)

Represents a data transfer configuration. A transfer configuration contains all metadata needed to perform a data transfer.

To get more information about Config, see:

import pulumi
import pulumi_gcp as gcp

project = gcp.organizations.get_project()
permissions = gcp.projects.IAMMember("permissions",
    role="roles/iam.serviceAccountShortTermTokenMinter",
    member=f"serviceAccount:service-{project.number}@gcp-sa-bigquerydatatransfer.iam.gserviceaccount.com")
my_dataset = gcp.bigquery.Dataset("myDataset",
    dataset_id="my_dataset",
    friendly_name="foo",
    description="bar",
    location="asia-northeast1")
query_config = gcp.bigquery.DataTransferConfig("queryConfig",
    display_name="my-query",
    location="asia-northeast1",
    data_source_id="scheduled_query",
    schedule="first sunday of quarter 00:00",
    destination_dataset_id=my_dataset.dataset_id,
    params={
        "destination_table_name_template": "my_table",
        "write_disposition": "WRITE_APPEND",
        "query": "SELECT name FROM tabl WHERE x = 'y'",
    })
Parameters
  • resource_name (str) – The name of the resource.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • data_refresh_window_days (pulumi.Input[float]) – The number of days to look back to automatically refresh the data. For example, if dataRefreshWindowDays = 10, then every day BigQuery reingests data for [today-10, today-1], rather than ingesting data for just [today-1]. Only valid if the data source supports the feature. Set the value to 0 to use the default value.

  • data_source_id (pulumi.Input[str]) – The data source id. Cannot be changed once the transfer config is created.

  • destination_dataset_id (pulumi.Input[str]) – The BigQuery target dataset id.

  • disabled (pulumi.Input[bool]) – When set to true, no runs are scheduled for a given transfer.

  • display_name (pulumi.Input[str]) – The user specified display name for the transfer config.

  • location (pulumi.Input[str]) – The geographic location where the transfer config should reside. Examples: US, EU, asia-northeast1. The default value is US.

  • params (pulumi.Input[dict]) – These parameters are specific to each data source.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • schedule (pulumi.Input[str]) – Data transfer schedule. If the data source does not support a custom schedule, this should be empty. If it is empty, the default value for the data source will be used. The specified times are in UTC. Examples of valid format: 1st,3rd monday of month 15:30, every wed,fri of jan, jun 13:15, and first sunday of quarter 00:00. See more explanation about the format here: https://cloud.google.com/appengine/docs/flexible/python/scheduling-jobs-with-cron-yaml#the_schedule_format NOTE: the granularity should be at least 8 hours, or less frequent.

  • service_account_name (pulumi.Input[str]) – Optional service account name. If this field is set, transfer config will be created with this service account credentials. It requires that requesting user calling this API has permissions to act as this service account.

data_refresh_window_days: pulumi.Output[float] = None

The number of days to look back to automatically refresh the data. For example, if dataRefreshWindowDays = 10, then every day BigQuery reingests data for [today-10, today-1], rather than ingesting data for just [today-1]. Only valid if the data source supports the feature. Set the value to 0 to use the default value.

data_source_id: pulumi.Output[str] = None

The data source id. Cannot be changed once the transfer config is created.

destination_dataset_id: pulumi.Output[str] = None

The BigQuery target dataset id.

disabled: pulumi.Output[bool] = None

When set to true, no runs are scheduled for a given transfer.

display_name: pulumi.Output[str] = None

The user specified display name for the transfer config.

location: pulumi.Output[str] = None

The geographic location where the transfer config should reside. Examples: US, EU, asia-northeast1. The default value is US.

name: pulumi.Output[str] = None

The resource name of the transfer config. Transfer config names have the form projects/{projectId}/locations/{location}/transferConfigs/{configId}. Where configId is usually a uuid, but this is not required. The name is ignored when creating a transfer config.

params: pulumi.Output[dict] = None

These parameters are specific to each data source.

project: pulumi.Output[str] = None

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

schedule: pulumi.Output[str] = None

Data transfer schedule. If the data source does not support a custom schedule, this should be empty. If it is empty, the default value for the data source will be used. The specified times are in UTC. Examples of valid format: 1st,3rd monday of month 15:30, every wed,fri of jan, jun 13:15, and first sunday of quarter 00:00. See more explanation about the format here: https://cloud.google.com/appengine/docs/flexible/python/scheduling-jobs-with-cron-yaml#the_schedule_format NOTE: the granularity should be at least 8 hours, or less frequent.

service_account_name: pulumi.Output[str] = None

Optional service account name. If this field is set, transfer config will be created with this service account credentials. It requires that requesting user calling this API has permissions to act as this service account.

static get(resource_name, id, opts=None, data_refresh_window_days=None, data_source_id=None, destination_dataset_id=None, disabled=None, display_name=None, location=None, name=None, params=None, project=None, schedule=None, service_account_name=None)

Get an existing DataTransferConfig resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters
  • resource_name (str) – The unique name of the resulting resource.

  • id (str) – The unique provider ID of the resource to lookup.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • data_refresh_window_days (pulumi.Input[float]) – The number of days to look back to automatically refresh the data. For example, if dataRefreshWindowDays = 10, then every day BigQuery reingests data for [today-10, today-1], rather than ingesting data for just [today-1]. Only valid if the data source supports the feature. Set the value to 0 to use the default value.

  • data_source_id (pulumi.Input[str]) – The data source id. Cannot be changed once the transfer config is created.

  • destination_dataset_id (pulumi.Input[str]) – The BigQuery target dataset id.

  • disabled (pulumi.Input[bool]) – When set to true, no runs are scheduled for a given transfer.

  • display_name (pulumi.Input[str]) – The user specified display name for the transfer config.

  • location (pulumi.Input[str]) – The geographic location where the transfer config should reside. Examples: US, EU, asia-northeast1. The default value is US.

  • name (pulumi.Input[str]) – The resource name of the transfer config. Transfer config names have the form projects/{projectId}/locations/{location}/transferConfigs/{configId}. Where configId is usually a uuid, but this is not required. The name is ignored when creating a transfer config.

  • params (pulumi.Input[dict]) – These parameters are specific to each data source.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • schedule (pulumi.Input[str]) – Data transfer schedule. If the data source does not support a custom schedule, this should be empty. If it is empty, the default value for the data source will be used. The specified times are in UTC. Examples of valid format: 1st,3rd monday of month 15:30, every wed,fri of jan, jun 13:15, and first sunday of quarter 00:00. See more explanation about the format here: https://cloud.google.com/appengine/docs/flexible/python/scheduling-jobs-with-cron-yaml#the_schedule_format NOTE: the granularity should be at least 8 hours, or less frequent.

  • service_account_name (pulumi.Input[str]) – Optional service account name. If this field is set, transfer config will be created with this service account credentials. It requires that requesting user calling this API has permissions to act as this service account.

translate_output_property(prop)

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

translate_input_property(prop)

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

class pulumi_gcp.bigquery.Dataset(resource_name, opts=None, accesses=None, dataset_id=None, default_encryption_configuration=None, default_partition_expiration_ms=None, default_table_expiration_ms=None, delete_contents_on_destroy=None, description=None, friendly_name=None, labels=None, location=None, project=None, __props__=None, __name__=None, __opts__=None)

Datasets allow you to organize and control access to your tables.

To get more information about Dataset, see:

import pulumi
import pulumi_gcp as gcp

bqowner = gcp.service_account.Account("bqowner", account_id="bqowner")
dataset = gcp.bigquery.Dataset("dataset",
    dataset_id="example_dataset",
    friendly_name="test",
    description="This is a test description",
    location="EU",
    default_table_expiration_ms=3600000,
    labels={
        "env": "default",
    },
    access=[
        {
            "role": "OWNER",
            "user_by_email": bqowner.email,
        },
        {
            "role": "READER",
            "domain": "hashicorp.com",
        },
    ])
import pulumi
import pulumi_gcp as gcp

key_ring = gcp.kms.KeyRing("keyRing", location="us")
crypto_key = gcp.kms.CryptoKey("cryptoKey", key_ring=key_ring.id)
dataset = gcp.bigquery.Dataset("dataset",
    dataset_id="example_dataset",
    friendly_name="test",
    description="This is a test description",
    location="US",
    default_table_expiration_ms=3600000,
    default_encryption_configuration={
        "kms_key_name": crypto_key.id,
    })
Parameters
  • resource_name (str) – The name of the resource.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • accesses (pulumi.Input[list]) – An array of objects that define dataset access for one or more entities. Structure is documented below.

  • dataset_id (pulumi.Input[str]) – The ID of the dataset containing this table.

  • default_encryption_configuration (pulumi.Input[dict]) – The default encryption key for all tables in the dataset. Once this property is set, all newly-created partitioned tables in the dataset will have encryption key set to this value, unless table creation request (or query) overrides the key. Structure is documented below.

  • default_partition_expiration_ms (pulumi.Input[float]) – The default partition expiration for all partitioned tables in the dataset, in milliseconds.

  • default_table_expiration_ms (pulumi.Input[float]) – The default lifetime of all tables in the dataset, in milliseconds. The minimum value is 3600000 milliseconds (one hour).

  • delete_contents_on_destroy (pulumi.Input[bool]) – If set to true, delete all the tables in the dataset when destroying the resource; otherwise, destroying the resource will fail if tables are present.

  • description (pulumi.Input[str]) – A user-friendly description of the dataset

  • friendly_name (pulumi.Input[str]) – A descriptive name for the dataset

  • labels (pulumi.Input[dict]) – The labels associated with this dataset. You can use these to organize and group your datasets

  • location (pulumi.Input[str]) – The geographic location where the dataset should reside. See official docs.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

The accesses object supports the following:

  • domain (pulumi.Input[str]) - A domain to grant access to. Any users signed in with the domain specified will be granted the specified access

  • group_by_email (pulumi.Input[str]) - An email address of a Google Group to grant access to.

  • role (pulumi.Input[str]) - Describes the rights granted to the user specified by the other member of the access object. Primitive, Predefined and custom roles are supported. Predefined roles that have equivalent primitive roles are swapped by the API to their Primitive counterparts. See official docs.

  • special_group (pulumi.Input[str]) - A special group to grant access to. Possible values include:

  • user_by_email (pulumi.Input[str]) - An email address of a user to grant access to. For example: fred@example.com

  • view (pulumi.Input[dict]) - A view from a different dataset to grant access to. Queries executed against that view will have read access to tables in this dataset. The role field is not required when this field is set. If that view is updated by any user, access to the view needs to be granted again via an update operation. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this table.

    • project_id (pulumi.Input[str]) - The ID of the project containing this table.

    • table_id (pulumi.Input[str]) - The ID of the table. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.

The default_encryption_configuration object supports the following:

  • kms_key_name (pulumi.Input[str]) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

accesses: pulumi.Output[list] = None

An array of objects that define dataset access for one or more entities. Structure is documented below.

  • domain (str) - A domain to grant access to. Any users signed in with the domain specified will be granted the specified access

  • group_by_email (str) - An email address of a Google Group to grant access to.

  • role (str) - Describes the rights granted to the user specified by the other member of the access object. Primitive, Predefined and custom roles are supported. Predefined roles that have equivalent primitive roles are swapped by the API to their Primitive counterparts. See official docs.

  • special_group (str) - A special group to grant access to. Possible values include:

  • user_by_email (str) - An email address of a user to grant access to. For example: fred@example.com

  • view (dict) - A view from a different dataset to grant access to. Queries executed against that view will have read access to tables in this dataset. The role field is not required when this field is set. If that view is updated by any user, access to the view needs to be granted again via an update operation. Structure is documented below.

    • dataset_id (str) - The ID of the dataset containing this table.

    • project_id (str) - The ID of the project containing this table.

    • table_id (str) - The ID of the table. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.

creation_time: pulumi.Output[float] = None

The time when this dataset was created, in milliseconds since the epoch.

dataset_id: pulumi.Output[str] = None

The ID of the dataset containing this table.

default_encryption_configuration: pulumi.Output[dict] = None

The default encryption key for all tables in the dataset. Once this property is set, all newly-created partitioned tables in the dataset will have encryption key set to this value, unless table creation request (or query) overrides the key. Structure is documented below.

  • kms_key_name (str) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

default_partition_expiration_ms: pulumi.Output[float] = None

The default partition expiration for all partitioned tables in the dataset, in milliseconds.

default_table_expiration_ms: pulumi.Output[float] = None

The default lifetime of all tables in the dataset, in milliseconds. The minimum value is 3600000 milliseconds (one hour).

delete_contents_on_destroy: pulumi.Output[bool] = None

If set to true, delete all the tables in the dataset when destroying the resource; otherwise, destroying the resource will fail if tables are present.

description: pulumi.Output[str] = None

A user-friendly description of the dataset

etag: pulumi.Output[str] = None

A hash of the resource.

friendly_name: pulumi.Output[str] = None

A descriptive name for the dataset

labels: pulumi.Output[dict] = None

The labels associated with this dataset. You can use these to organize and group your datasets

last_modified_time: pulumi.Output[float] = None

The date when this dataset or any of its tables was last modified, in milliseconds since the epoch.

location: pulumi.Output[str] = None

The geographic location where the dataset should reside. See official docs.

project: pulumi.Output[str] = None

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

The URI of the created resource.

static get(resource_name, id, opts=None, accesses=None, creation_time=None, dataset_id=None, default_encryption_configuration=None, default_partition_expiration_ms=None, default_table_expiration_ms=None, delete_contents_on_destroy=None, description=None, etag=None, friendly_name=None, labels=None, last_modified_time=None, location=None, project=None, self_link=None)

Get an existing Dataset resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters
  • resource_name (str) – The unique name of the resulting resource.

  • id (str) – The unique provider ID of the resource to lookup.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • accesses (pulumi.Input[list]) – An array of objects that define dataset access for one or more entities. Structure is documented below.

  • creation_time (pulumi.Input[float]) – The time when this dataset was created, in milliseconds since the epoch.

  • dataset_id (pulumi.Input[str]) – The ID of the dataset containing this table.

  • default_encryption_configuration (pulumi.Input[dict]) – The default encryption key for all tables in the dataset. Once this property is set, all newly-created partitioned tables in the dataset will have encryption key set to this value, unless table creation request (or query) overrides the key. Structure is documented below.

  • default_partition_expiration_ms (pulumi.Input[float]) – The default partition expiration for all partitioned tables in the dataset, in milliseconds.

  • default_table_expiration_ms (pulumi.Input[float]) – The default lifetime of all tables in the dataset, in milliseconds. The minimum value is 3600000 milliseconds (one hour).

  • delete_contents_on_destroy (pulumi.Input[bool]) – If set to true, delete all the tables in the dataset when destroying the resource; otherwise, destroying the resource will fail if tables are present.

  • description (pulumi.Input[str]) – A user-friendly description of the dataset

  • etag (pulumi.Input[str]) – A hash of the resource.

  • friendly_name (pulumi.Input[str]) – A descriptive name for the dataset

  • labels (pulumi.Input[dict]) – The labels associated with this dataset. You can use these to organize and group your datasets

  • last_modified_time (pulumi.Input[float]) – The date when this dataset or any of its tables was last modified, in milliseconds since the epoch.

  • location (pulumi.Input[str]) –

    The geographic location where the dataset should reside. See official docs.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • self_link (pulumi.Input[str]) – The URI of the created resource.

The accesses object supports the following:

  • domain (pulumi.Input[str]) - A domain to grant access to. Any users signed in with the domain specified will be granted the specified access

  • group_by_email (pulumi.Input[str]) - An email address of a Google Group to grant access to.

  • role (pulumi.Input[str]) - Describes the rights granted to the user specified by the other member of the access object. Primitive, Predefined and custom roles are supported. Predefined roles that have equivalent primitive roles are swapped by the API to their Primitive counterparts. See official docs.

  • special_group (pulumi.Input[str]) - A special group to grant access to. Possible values include:

  • user_by_email (pulumi.Input[str]) - An email address of a user to grant access to. For example: fred@example.com

  • view (pulumi.Input[dict]) - A view from a different dataset to grant access to. Queries executed against that view will have read access to tables in this dataset. The role field is not required when this field is set. If that view is updated by any user, access to the view needs to be granted again via an update operation. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this table.

    • project_id (pulumi.Input[str]) - The ID of the project containing this table.

    • table_id (pulumi.Input[str]) - The ID of the table. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.

The default_encryption_configuration object supports the following:

  • kms_key_name (pulumi.Input[str]) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

translate_output_property(prop)

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

translate_input_property(prop)

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

class pulumi_gcp.bigquery.DatasetAccess(resource_name, opts=None, dataset_id=None, domain=None, group_by_email=None, iam_member=None, project=None, role=None, special_group=None, user_by_email=None, view=None, __props__=None, __name__=None, __opts__=None)

Gives dataset access for a single entity. This resource is intended to be used in cases where it is not possible to compile a full list of access blocks to include in a bigquery.Dataset resource, to enable them to be added separately.

Note: If this resource is used alongside a bigquery.Dataset resource, the dataset resource must either have no defined access blocks or a lifecycle block with ignore_changes = [access] so they don’t fight over which accesses should be on the dataset.

To get more information about DatasetAccess, see:

import pulumi
import pulumi_gcp as gcp

dataset = gcp.bigquery.Dataset("dataset", dataset_id="example_dataset")
bqowner = gcp.service_account.Account("bqowner", account_id="bqowner")
access = gcp.bigquery.DatasetAccess("access",
    dataset_id=dataset.dataset_id,
    role="OWNER",
    user_by_email=bqowner.email)
import pulumi
import pulumi_gcp as gcp

private = gcp.bigquery.Dataset("private", dataset_id="example_dataset")
public_dataset = gcp.bigquery.Dataset("publicDataset", dataset_id="example_dataset2")
public_table = gcp.bigquery.Table("publicTable",
    dataset_id=public_dataset.dataset_id,
    table_id="example_table",
    view={
        "query": "SELECT state FROM [lookerdata:cdc.project_tycho_reports]",
        "useLegacySql": False,
    })
access = gcp.bigquery.DatasetAccess("access",
    dataset_id=private.dataset_id,
    view={
        "project_id": public_table.project,
        "dataset_id": public_dataset.dataset_id,
        "table_id": public_table.table_id,
    })
Parameters
  • resource_name (str) – The name of the resource.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • dataset_id (pulumi.Input[str]) – The ID of the dataset containing this table.

  • domain (pulumi.Input[str]) – A domain to grant access to. Any users signed in with the domain specified will be granted the specified access

  • group_by_email (pulumi.Input[str]) – An email address of a Google Group to grant access to.

  • iam_member (pulumi.Input[str]) – Some other type of member that appears in the IAM Policy but isn’t a user, group, domain, or special group. For example: allUsers

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • role (pulumi.Input[str]) –

    Describes the rights granted to the user specified by the other member of the access object. Primitive, Predefined and custom roles are supported. Predefined roles that have equivalent primitive roles are swapped by the API to their Primitive counterparts, and will show a diff post-create. See official docs.

  • special_group (pulumi.Input[str]) – A special group to grant access to. Possible values include:

  • user_by_email (pulumi.Input[str]) – An email address of a user to grant access to. For example: fred@example.com

  • view (pulumi.Input[dict]) – A view from a different dataset to grant access to. Queries executed against that view will have read access to tables in this dataset. The role field is not required when this field is set. If that view is updated by any user, access to the view needs to be granted again via an update operation. Structure is documented below.

The view object supports the following:

  • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this table.

  • project_id (pulumi.Input[str]) - The ID of the project containing this table.

  • table_id (pulumi.Input[str]) - The ID of the table. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.

dataset_id: pulumi.Output[str] = None

The ID of the dataset containing this table.

domain: pulumi.Output[str] = None

A domain to grant access to. Any users signed in with the domain specified will be granted the specified access

group_by_email: pulumi.Output[str] = None

An email address of a Google Group to grant access to.

iam_member: pulumi.Output[str] = None

Some other type of member that appears in the IAM Policy but isn’t a user, group, domain, or special group. For example: allUsers

project: pulumi.Output[str] = None

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

role: pulumi.Output[str] = None

Describes the rights granted to the user specified by the other member of the access object. Primitive, Predefined and custom roles are supported. Predefined roles that have equivalent primitive roles are swapped by the API to their Primitive counterparts, and will show a diff post-create. See official docs.

special_group: pulumi.Output[str] = None

A special group to grant access to. Possible values include:

user_by_email: pulumi.Output[str] = None

An email address of a user to grant access to. For example: fred@example.com

view: pulumi.Output[dict] = None

A view from a different dataset to grant access to. Queries executed against that view will have read access to tables in this dataset. The role field is not required when this field is set. If that view is updated by any user, access to the view needs to be granted again via an update operation. Structure is documented below.

  • dataset_id (str) - The ID of the dataset containing this table.

  • project_id (str) - The ID of the project containing this table.

  • table_id (str) - The ID of the table. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.

static get(resource_name, id, opts=None, dataset_id=None, domain=None, group_by_email=None, iam_member=None, project=None, role=None, special_group=None, user_by_email=None, view=None)

Get an existing DatasetAccess resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters
  • resource_name (str) – The unique name of the resulting resource.

  • id (str) – The unique provider ID of the resource to lookup.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • dataset_id (pulumi.Input[str]) – The ID of the dataset containing this table.

  • domain (pulumi.Input[str]) – A domain to grant access to. Any users signed in with the domain specified will be granted the specified access

  • group_by_email (pulumi.Input[str]) – An email address of a Google Group to grant access to.

  • iam_member (pulumi.Input[str]) – Some other type of member that appears in the IAM Policy but isn’t a user, group, domain, or special group. For example: allUsers

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • role (pulumi.Input[str]) –

    Describes the rights granted to the user specified by the other member of the access object. Primitive, Predefined and custom roles are supported. Predefined roles that have equivalent primitive roles are swapped by the API to their Primitive counterparts, and will show a diff post-create. See official docs.

  • special_group (pulumi.Input[str]) – A special group to grant access to. Possible values include:

  • user_by_email (pulumi.Input[str]) – An email address of a user to grant access to. For example: fred@example.com

  • view (pulumi.Input[dict]) – A view from a different dataset to grant access to. Queries executed against that view will have read access to tables in this dataset. The role field is not required when this field is set. If that view is updated by any user, access to the view needs to be granted again via an update operation. Structure is documented below.

The view object supports the following:

  • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this table.

  • project_id (pulumi.Input[str]) - The ID of the project containing this table.

  • table_id (pulumi.Input[str]) - The ID of the table. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.

translate_output_property(prop)

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

translate_input_property(prop)

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

class pulumi_gcp.bigquery.DatasetIamBinding(resource_name, opts=None, condition=None, dataset_id=None, members=None, project=None, role=None, __props__=None, __name__=None, __opts__=None)

Three different resources help you manage your IAM policy for BigQuery dataset. Each of these resources serves a different use case:

  • bigquery.DatasetIamPolicy: Authoritative. Sets the IAM policy for the dataset and replaces any existing policy already attached.

  • bigquery.DatasetIamBinding: Authoritative for a given role. Updates the IAM policy to grant a role to a list of members. Other roles within the IAM policy for the dataset are preserved.

  • bigquery.DatasetIamMember: Non-authoritative. Updates the IAM policy to grant a role to a new member. Other members for the role for the dataset are preserved.

Note: These resources cannot be used with bigquery.DatasetAccess resources or the access field on bigquery.Dataset or they will fight over what the policy should be.

Note: Using any of these resources will remove any authorized view permissions from the dataset. To assign and preserve authorized view permissions use the bigquery.DatasetAccess instead.

Note: Legacy BigQuery roles OWNER WRITER and READER cannot be used with any of these IAM resources. Instead use the full role form of: roles/bigquery.dataOwner roles/bigquery.dataEditor and roles/bigquery.dataViewer.

Note: bigquery.DatasetIamPolicy cannot be used in conjunction with bigquery.DatasetIamBinding and bigquery.DatasetIamMember or they will fight over what your policy should be.

Note: bigquery.DatasetIamBinding resources can be used in conjunction with bigquery.DatasetIamMember resources only if they do not grant privilege to the same role.

import pulumi
import pulumi_gcp as gcp

owner = gcp.organizations.get_iam_policy(binding=[{
    "role": "roles/dataOwner",
    "members": ["user:jane@example.com"],
}])
dataset = gcp.bigquery.DatasetIamPolicy("dataset",
    dataset_id="your-dataset-id",
    policy_data=owner.policy_data)
import pulumi
import pulumi_gcp as gcp

reader = gcp.bigquery.DatasetIamBinding("reader",
    dataset_id="your-dataset-id",
    members=["user:jane@example.com"],
    role="roles/bigquery.dataViewer")
import pulumi
import pulumi_gcp as gcp

editor = gcp.bigquery.DatasetIamMember("editor",
    dataset_id="your-dataset-id",
    member="user:jane@example.com",
    role="roles/bigquery.dataEditor")
Parameters
  • resource_name (str) – The name of the resource.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • dataset_id (pulumi.Input[str]) – The dataset ID, in the form projects/{project}/datasets/{dataset_id}

  • role (pulumi.Input[str]) – The role that should be applied. Only one bigquery.DatasetIamBinding can be used per role. Note that custom roles must be of the format [projects|organizations]/{parent-name}/roles/{role-name}.

The condition object supports the following:

  • description (pulumi.Input[str])

  • expression (pulumi.Input[str])

  • title (pulumi.Input[str])

dataset_id: pulumi.Output[str] = None

The dataset ID, in the form projects/{project}/datasets/{dataset_id}

etag: pulumi.Output[str] = None

(Computed) The etag of the dataset’s IAM policy.

role: pulumi.Output[str] = None

The role that should be applied. Only one bigquery.DatasetIamBinding can be used per role. Note that custom roles must be of the format [projects|organizations]/{parent-name}/roles/{role-name}.

static get(resource_name, id, opts=None, condition=None, dataset_id=None, etag=None, members=None, project=None, role=None)

Get an existing DatasetIamBinding resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters
  • resource_name (str) – The unique name of the resulting resource.

  • id (str) – The unique provider ID of the resource to lookup.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • dataset_id (pulumi.Input[str]) – The dataset ID, in the form projects/{project}/datasets/{dataset_id}

  • etag (pulumi.Input[str]) – (Computed) The etag of the dataset’s IAM policy.

  • role (pulumi.Input[str]) – The role that should be applied. Only one bigquery.DatasetIamBinding can be used per role. Note that custom roles must be of the format [projects|organizations]/{parent-name}/roles/{role-name}.

The condition object supports the following:

  • description (pulumi.Input[str])

  • expression (pulumi.Input[str])

  • title (pulumi.Input[str])

translate_output_property(prop)

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

translate_input_property(prop)

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

class pulumi_gcp.bigquery.DatasetIamMember(resource_name, opts=None, condition=None, dataset_id=None, member=None, project=None, role=None, __props__=None, __name__=None, __opts__=None)

Three different resources help you manage your IAM policy for BigQuery dataset. Each of these resources serves a different use case:

  • bigquery.DatasetIamPolicy: Authoritative. Sets the IAM policy for the dataset and replaces any existing policy already attached.

  • bigquery.DatasetIamBinding: Authoritative for a given role. Updates the IAM policy to grant a role to a list of members. Other roles within the IAM policy for the dataset are preserved.

  • bigquery.DatasetIamMember: Non-authoritative. Updates the IAM policy to grant a role to a new member. Other members for the role for the dataset are preserved.

Note: These resources cannot be used with bigquery.DatasetAccess resources or the access field on bigquery.Dataset or they will fight over what the policy should be.

Note: Using any of these resources will remove any authorized view permissions from the dataset. To assign and preserve authorized view permissions use the bigquery.DatasetAccess instead.

Note: Legacy BigQuery roles OWNER WRITER and READER cannot be used with any of these IAM resources. Instead use the full role form of: roles/bigquery.dataOwner roles/bigquery.dataEditor and roles/bigquery.dataViewer.

Note: bigquery.DatasetIamPolicy cannot be used in conjunction with bigquery.DatasetIamBinding and bigquery.DatasetIamMember or they will fight over what your policy should be.

Note: bigquery.DatasetIamBinding resources can be used in conjunction with bigquery.DatasetIamMember resources only if they do not grant privilege to the same role.

import pulumi
import pulumi_gcp as gcp

owner = gcp.organizations.get_iam_policy(binding=[{
    "role": "roles/dataOwner",
    "members": ["user:jane@example.com"],
}])
dataset = gcp.bigquery.DatasetIamPolicy("dataset",
    dataset_id="your-dataset-id",
    policy_data=owner.policy_data)
import pulumi
import pulumi_gcp as gcp

reader = gcp.bigquery.DatasetIamBinding("reader",
    dataset_id="your-dataset-id",
    members=["user:jane@example.com"],
    role="roles/bigquery.dataViewer")
import pulumi
import pulumi_gcp as gcp

editor = gcp.bigquery.DatasetIamMember("editor",
    dataset_id="your-dataset-id",
    member="user:jane@example.com",
    role="roles/bigquery.dataEditor")
Parameters
  • resource_name (str) – The name of the resource.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • dataset_id (pulumi.Input[str]) – The dataset ID, in the form projects/{project}/datasets/{dataset_id}

  • role (pulumi.Input[str]) – The role that should be applied. Only one bigquery.DatasetIamBinding can be used per role. Note that custom roles must be of the format [projects|organizations]/{parent-name}/roles/{role-name}.

The condition object supports the following:

  • description (pulumi.Input[str])

  • expression (pulumi.Input[str])

  • title (pulumi.Input[str])

dataset_id: pulumi.Output[str] = None

The dataset ID, in the form projects/{project}/datasets/{dataset_id}

etag: pulumi.Output[str] = None

(Computed) The etag of the dataset’s IAM policy.

role: pulumi.Output[str] = None

The role that should be applied. Only one bigquery.DatasetIamBinding can be used per role. Note that custom roles must be of the format [projects|organizations]/{parent-name}/roles/{role-name}.

static get(resource_name, id, opts=None, condition=None, dataset_id=None, etag=None, member=None, project=None, role=None)

Get an existing DatasetIamMember resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters
  • resource_name (str) – The unique name of the resulting resource.

  • id (str) – The unique provider ID of the resource to lookup.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • dataset_id (pulumi.Input[str]) – The dataset ID, in the form projects/{project}/datasets/{dataset_id}

  • etag (pulumi.Input[str]) – (Computed) The etag of the dataset’s IAM policy.

  • role (pulumi.Input[str]) – The role that should be applied. Only one bigquery.DatasetIamBinding can be used per role. Note that custom roles must be of the format [projects|organizations]/{parent-name}/roles/{role-name}.

The condition object supports the following:

  • description (pulumi.Input[str])

  • expression (pulumi.Input[str])

  • title (pulumi.Input[str])

translate_output_property(prop)

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

translate_input_property(prop)

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

class pulumi_gcp.bigquery.DatasetIamPolicy(resource_name, opts=None, dataset_id=None, policy_data=None, project=None, __props__=None, __name__=None, __opts__=None)

Three different resources help you manage your IAM policy for BigQuery dataset. Each of these resources serves a different use case:

  • bigquery.DatasetIamPolicy: Authoritative. Sets the IAM policy for the dataset and replaces any existing policy already attached.

  • bigquery.DatasetIamBinding: Authoritative for a given role. Updates the IAM policy to grant a role to a list of members. Other roles within the IAM policy for the dataset are preserved.

  • bigquery.DatasetIamMember: Non-authoritative. Updates the IAM policy to grant a role to a new member. Other members for the role for the dataset are preserved.

Note: These resources cannot be used with bigquery.DatasetAccess resources or the access field on bigquery.Dataset or they will fight over what the policy should be.

Note: Using any of these resources will remove any authorized view permissions from the dataset. To assign and preserve authorized view permissions use the bigquery.DatasetAccess instead.

Note: Legacy BigQuery roles OWNER WRITER and READER cannot be used with any of these IAM resources. Instead use the full role form of: roles/bigquery.dataOwner roles/bigquery.dataEditor and roles/bigquery.dataViewer.

Note: bigquery.DatasetIamPolicy cannot be used in conjunction with bigquery.DatasetIamBinding and bigquery.DatasetIamMember or they will fight over what your policy should be.

Note: bigquery.DatasetIamBinding resources can be used in conjunction with bigquery.DatasetIamMember resources only if they do not grant privilege to the same role.

import pulumi
import pulumi_gcp as gcp

owner = gcp.organizations.get_iam_policy(binding=[{
    "role": "roles/dataOwner",
    "members": ["user:jane@example.com"],
}])
dataset = gcp.bigquery.DatasetIamPolicy("dataset",
    dataset_id="your-dataset-id",
    policy_data=owner.policy_data)
import pulumi
import pulumi_gcp as gcp

reader = gcp.bigquery.DatasetIamBinding("reader",
    dataset_id="your-dataset-id",
    members=["user:jane@example.com"],
    role="roles/bigquery.dataViewer")
import pulumi
import pulumi_gcp as gcp

editor = gcp.bigquery.DatasetIamMember("editor",
    dataset_id="your-dataset-id",
    member="user:jane@example.com",
    role="roles/bigquery.dataEditor")
Parameters
  • resource_name (str) – The name of the resource.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • dataset_id (pulumi.Input[str]) – The dataset ID, in the form projects/{project}/datasets/{dataset_id}

  • policy_data (pulumi.Input[str]) – The policy data generated by a organizations.getIAMPolicy data source.

dataset_id: pulumi.Output[str] = None

The dataset ID, in the form projects/{project}/datasets/{dataset_id}

etag: pulumi.Output[str] = None

(Computed) The etag of the dataset’s IAM policy.

policy_data: pulumi.Output[str] = None

The policy data generated by a organizations.getIAMPolicy data source.

static get(resource_name, id, opts=None, dataset_id=None, etag=None, policy_data=None, project=None)

Get an existing DatasetIamPolicy resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters
  • resource_name (str) – The unique name of the resulting resource.

  • id (str) – The unique provider ID of the resource to lookup.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • dataset_id (pulumi.Input[str]) – The dataset ID, in the form projects/{project}/datasets/{dataset_id}

  • etag (pulumi.Input[str]) – (Computed) The etag of the dataset’s IAM policy.

  • policy_data (pulumi.Input[str]) – The policy data generated by a organizations.getIAMPolicy data source.

translate_output_property(prop)

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

translate_input_property(prop)

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

class pulumi_gcp.bigquery.GetDefaultServiceAccountResult(email=None, id=None, project=None)

A collection of values returned by getDefaultServiceAccount.

email = None

The email address of the service account. This value is often used to refer to the service account in order to grant IAM permissions.

id = None

The provider-assigned unique ID for this managed resource.

class pulumi_gcp.bigquery.Job(resource_name, opts=None, copy=None, extract=None, job_id=None, job_timeout_ms=None, labels=None, load=None, location=None, project=None, query=None, __props__=None, __name__=None, __opts__=None)

Jobs are actions that BigQuery runs on your behalf to load data, export data, query data, or copy data. Once a BigQuery job is created, it cannot be changed or deleted.

import pulumi
import pulumi_gcp as gcp

bar = gcp.bigquery.Dataset("bar",
    dataset_id="job_query_dataset",
    friendly_name="test",
    description="This is a test description",
    location="US")
foo = gcp.bigquery.Table("foo",
    dataset_id=bar.dataset_id,
    table_id="job_query_table")
job = gcp.bigquery.Job("job",
    job_id="job_query",
    labels={
        "example-label": "example-value",
    },
    query={
        "query": "SELECT state FROM [lookerdata:cdc.project_tycho_reports]",
        "destination_table": {
            "project_id": foo.project,
            "dataset_id": foo.dataset_id,
            "table_id": foo.table_id,
        },
        "allowLargeResults": True,
        "flattenResults": True,
        "script_options": {
            "keyResultStatement": "LAST",
        },
    })
import pulumi
import pulumi_gcp as gcp

bar = gcp.bigquery.Dataset("bar",
    dataset_id="job_query_dataset",
    friendly_name="test",
    description="This is a test description",
    location="US")
foo = gcp.bigquery.Table("foo",
    dataset_id=bar.dataset_id,
    table_id="job_query_table")
job = gcp.bigquery.Job("job",
    job_id="job_query",
    labels={
        "example-label": "example-value",
    },
    query={
        "query": "SELECT state FROM [lookerdata:cdc.project_tycho_reports]",
        "destination_table": {
            "table_id": foo.id,
        },
        "default_dataset": {
            "dataset_id": bar.id,
        },
        "allowLargeResults": True,
        "flattenResults": True,
        "script_options": {
            "keyResultStatement": "LAST",
        },
    })
import pulumi
import pulumi_gcp as gcp

bar = gcp.bigquery.Dataset("bar",
    dataset_id="job_load_dataset",
    friendly_name="test",
    description="This is a test description",
    location="US")
foo = gcp.bigquery.Table("foo",
    dataset_id=bar.dataset_id,
    table_id="job_load_table")
job = gcp.bigquery.Job("job",
    job_id="job_load",
    labels={
        "my_job": "load",
    },
    load={
        "sourceUris": ["gs://cloud-samples-data/bigquery/us-states/us-states-by-date.csv"],
        "destination_table": {
            "project_id": foo.project,
            "dataset_id": foo.dataset_id,
            "table_id": foo.table_id,
        },
        "skipLeadingRows": 1,
        "schemaUpdateOptions": [
            "ALLOW_FIELD_RELAXATION",
            "ALLOW_FIELD_ADDITION",
        ],
        "writeDisposition": "WRITE_APPEND",
        "autodetect": True,
    })
import pulumi
import pulumi_gcp as gcp

source_one_dataset = gcp.bigquery.Dataset("source-oneDataset",
    dataset_id="job_extract_dataset",
    friendly_name="test",
    description="This is a test description",
    location="US")
source_one_table = gcp.bigquery.Table("source-oneTable",
    dataset_id=source_one_dataset.dataset_id,
    table_id="job_extract_table",
    schema="""[
  {
    "name": "name",
    "type": "STRING",
    "mode": "NULLABLE"
  },
  {
    "name": "post_abbr",
    "type": "STRING",
    "mode": "NULLABLE"
  },
  {
    "name": "date",
    "type": "DATE",
    "mode": "NULLABLE"
  }
]
""")
dest = gcp.storage.Bucket("dest", force_destroy=True)
job = gcp.bigquery.Job("job",
    job_id="job_extract",
    extract={
        "destinationUris": [dest.url.apply(lambda url: f"{url}/extract")],
        "source_table": {
            "project_id": source_one_table.project,
            "dataset_id": source_one_table.dataset_id,
            "table_id": source_one_table.table_id,
        },
        "destinationFormat": "NEWLINE_DELIMITED_JSON",
        "compression": "GZIP",
    })
Parameters
  • resource_name (str) – The name of the resource.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • copy (pulumi.Input[dict]) – Copies a table. Structure is documented below.

  • extract (pulumi.Input[dict]) – Configures an extract job. Structure is documented below.

  • job*id (pulumi.Input[str]) –

    The ID of the job. The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (*), or dashes (-). The maximum length is 1,024 characters.

  • job_timeout_ms (pulumi.Input[str]) – Job timeout in milliseconds. If this time limit is exceeded, BigQuery may attempt to terminate the job.

  • labels (pulumi.Input[dict]) – The labels associated with this job. You can use these to organize and group your jobs.

  • load (pulumi.Input[dict]) – Configures a load job. Structure is documented below.

  • location (pulumi.Input[str]) – The geographic location of the job. The default value is US.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • query (pulumi.Input[dict]) – Configures a query job. Structure is documented below.

The copy object supports the following:

  • createDisposition (pulumi.Input[str]) - Specifies whether the job is allowed to create new tables. The following values are supported: CREATE_IF_NEEDED: If the table does not exist, BigQuery creates the table. CREATE_NEVER: The table must already exist. If it does not, a ‘notFound’ error is returned in the job result. Creation, truncation and append actions occur as one atomic update upon job completion

  • destinationEncryptionConfiguration (pulumi.Input[dict]) - Custom encryption configuration (e.g., Cloud KMS keys) Structure is documented below.

    • kms_key_name (pulumi.Input[str]) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

  • destinationTable (pulumi.Input[dict]) - The destination table. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

    • table_id (pulumi.Input[str]) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • sourceTables (pulumi.Input[list]) - Source tables to copy. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

    • table_id (pulumi.Input[str]) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • writeDisposition (pulumi.Input[str]) - Specifies the action that occurs if the destination table already exists. The following values are supported: WRITE_TRUNCATE: If the table already exists, BigQuery overwrites the table data and uses the schema from the query result. WRITE_APPEND: If the table already exists, BigQuery appends the data to the table. WRITE_EMPTY: If the table already exists and contains data, a ‘duplicate’ error is returned in the job result. Each action is atomic and only occurs if BigQuery is able to complete the job successfully. Creation, truncation and append actions occur as one atomic update upon job completion.

The extract object supports the following:

  • compression (pulumi.Input[str]) - The compression type to use for exported files. Possible values include GZIP, DEFLATE, SNAPPY, and NONE. The default value is NONE. DEFLATE and SNAPPY are only supported for Avro.

  • destinationFormat (pulumi.Input[str]) - The exported file format. Possible values include CSV, NEWLINE_DELIMITED_JSON and AVRO for tables and SAVED_MODEL for models. The default value for tables is CSV. Tables with nested or repeated fields cannot be exported as CSV. The default value for models is SAVED_MODEL.

  • destinationUris (pulumi.Input[list]) - A list of fully-qualified Google Cloud Storage URIs where the extracted table should be written.

  • fieldDelimiter (pulumi.Input[str]) - When extracting data in CSV format, this defines the delimiter to use between fields in the exported data. Default is ‘,’

  • printHeader (pulumi.Input[bool]) - Whether to print out a header row in the results. Default is true.

  • sourceModel (pulumi.Input[dict]) - A reference to the model being exported. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • modelId (pulumi.Input[str]) - The ID of the model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

  • sourceTable (pulumi.Input[dict]) - A reference to the table being exported. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

    • table_id (pulumi.Input[str]) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • useAvroLogicalTypes (pulumi.Input[bool]) - Whether to use logical types when extracting to AVRO format.

The load object supports the following:

  • allowJaggedRows (pulumi.Input[bool]) - Accept rows that are missing trailing optional columns. The missing values are treated as nulls. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. Only applicable to CSV, ignored for other formats.

  • allowQuotedNewlines (pulumi.Input[bool]) - Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.

  • autodetect (pulumi.Input[bool]) - Indicates if we should automatically infer the options and schema for CSV and JSON sources.

  • createDisposition (pulumi.Input[str]) - Specifies whether the job is allowed to create new tables. The following values are supported: CREATE_IF_NEEDED: If the table does not exist, BigQuery creates the table. CREATE_NEVER: The table must already exist. If it does not, a ‘notFound’ error is returned in the job result. Creation, truncation and append actions occur as one atomic update upon job completion

  • destinationEncryptionConfiguration (pulumi.Input[dict]) - Custom encryption configuration (e.g., Cloud KMS keys) Structure is documented below.

    • kms_key_name (pulumi.Input[str]) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

  • destinationTable (pulumi.Input[dict]) - The destination table. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

    • table_id (pulumi.Input[str]) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • encoding (pulumi.Input[str]) - The character encoding of the data. The supported values are UTF-8 or ISO-8859-1. The default value is UTF-8. BigQuery decodes the data after the raw, binary data has been split using the values of the quote and fieldDelimiter properties.

  • fieldDelimiter (pulumi.Input[str]) - When extracting data in CSV format, this defines the delimiter to use between fields in the exported data. Default is ‘,’

  • ignoreUnknownValues (pulumi.Input[bool]) - Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The sourceFormat property determines what BigQuery treats as an extra value: CSV: Trailing columns JSON: Named values that don’t match any column names

  • maxBadRecords (pulumi.Input[float]) - The maximum number of bad records that BigQuery can ignore when running the job. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid.

  • nullMarker (pulumi.Input[str]) - Specifies a string that represents a null value in a CSV file. The default value is the empty string. If you set this property to a custom value, BigQuery throws an error if an empty string is present for all data types except for STRING and BYTE. For STRING and BYTE columns, BigQuery interprets the empty string as an empty value.

  • projectionFields (pulumi.Input[list]) - If sourceFormat is set to “DATASTORE_BACKUP”, indicates which entity properties to load into BigQuery from a Cloud Datastore backup. Property names are case sensitive and must be top-level properties. If no properties are specified, BigQuery loads all properties. If any named property isn’t found in the Cloud Datastore backup, an invalid error is returned in the job result.

  • quote (pulumi.Input[str]) - The value that is used to quote data sections in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. The default value is a double-quote (‘”’). If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allowQuotedNewlines property to true.

  • schemaUpdateOptions (pulumi.Input[list]) - Allows the schema of the destination table to be updated as a side effect of the load job if a schema is autodetected or supplied in the job configuration. Schema update options are supported in two cases: when writeDisposition is WRITE_APPEND; when writeDisposition is WRITE_TRUNCATE and the destination table is a partition of a table, specified by partition decorators. For normal tables, WRITE_TRUNCATE will always overwrite the schema. One or more of the following values are specified: ALLOW_FIELD_ADDITION: allow adding a nullable field to the schema. ALLOW_FIELD_RELAXATION: allow relaxing a required field in the original schema to nullable.

  • skipLeadingRows (pulumi.Input[float]) - The number of rows at the top of a CSV file that BigQuery will skip when loading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped. When autodetect is on, the behavior is the following: skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.

  • sourceFormat (pulumi.Input[str]) - The format of the data files. For CSV files, specify “CSV”. For datastore backups, specify “DATASTORE_BACKUP”. For newline-delimited JSON, specify “NEWLINE_DELIMITED_JSON”. For Avro, specify “AVRO”. For parquet, specify “PARQUET”. For orc, specify “ORC”. The default value is CSV.

  • sourceUris (pulumi.Input[list]) - The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one ‘’ wildcard character and it must come after the ‘bucket’ name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups: Exactly one URI can be specified. Also, the ‘’ wildcard character is not allowed.

  • time_partitioning (pulumi.Input[dict]) - Time-based partitioning specification for the destination table. Structure is documented below.

    • expirationMs (pulumi.Input[str]) - Number of milliseconds for which to keep the storage for a partition. A wrapper is used here because 0 is an invalid value.

    • field (pulumi.Input[str]) - If not set, the table is partitioned by pseudo column ‘_PARTITIONTIME’; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. A wrapper is used here because an empty string is an invalid value.

    • type (pulumi.Input[str]) - The only type supported is DAY, which will generate one partition per day. Providing an empty string used to cause an error, but in OnePlatform the field will be treated as unset.

  • writeDisposition (pulumi.Input[str]) - Specifies the action that occurs if the destination table already exists. The following values are supported: WRITE_TRUNCATE: If the table already exists, BigQuery overwrites the table data and uses the schema from the query result. WRITE_APPEND: If the table already exists, BigQuery appends the data to the table. WRITE_EMPTY: If the table already exists and contains data, a ‘duplicate’ error is returned in the job result. Each action is atomic and only occurs if BigQuery is able to complete the job successfully. Creation, truncation and append actions occur as one atomic update upon job completion.

The query object supports the following:

  • allowLargeResults (pulumi.Input[bool]) - If true and query uses legacy SQL dialect, allows the query to produce arbitrarily large result tables at a slight cost in performance. Requires destinationTable to be set. For standard SQL queries, this flag is ignored and large results are always allowed. However, you must still set destinationTable when result size exceeds the allowed maximum response size.

  • createDisposition (pulumi.Input[str]) - Specifies whether the job is allowed to create new tables. The following values are supported: CREATE_IF_NEEDED: If the table does not exist, BigQuery creates the table. CREATE_NEVER: The table must already exist. If it does not, a ‘notFound’ error is returned in the job result. Creation, truncation and append actions occur as one atomic update upon job completion

  • defaultDataset (pulumi.Input[dict]) - Specifies the default dataset to use for unqualified table names in the query. Note that this does not alter behavior of unqualified dataset names. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

  • destinationEncryptionConfiguration (pulumi.Input[dict]) - Custom encryption configuration (e.g., Cloud KMS keys) Structure is documented below.

    • kms_key_name (pulumi.Input[str]) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

  • destinationTable (pulumi.Input[dict]) - The destination table. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

    • table_id (pulumi.Input[str]) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • flattenResults (pulumi.Input[bool]) - If true and query uses legacy SQL dialect, flattens all nested and repeated fields in the query results. allowLargeResults must be true if this is set to false. For standard SQL queries, this flag is ignored and results are never flattened.

  • maximumBillingTier (pulumi.Input[float]) - Limits the billing tier for this job. Queries that have resource usage beyond this tier will fail (without incurring a charge). If unspecified, this will be set to your project default.

  • maximumBytesBilled (pulumi.Input[str]) - Limits the bytes billed for this job. Queries that will have bytes billed beyond this limit will fail (without incurring a charge). If unspecified, this will be set to your project default.

  • parameterMode (pulumi.Input[str]) - Standard SQL only. Set to POSITIONAL to use positional (?) query parameters or to NAMED to use named (@myparam) query parameters in this query.

  • priority (pulumi.Input[str]) - Specifies a priority for the query.

  • query (pulumi.Input[str]) - Configures a query job. Structure is documented below.

  • schemaUpdateOptions (pulumi.Input[list]) - Allows the schema of the destination table to be updated as a side effect of the load job if a schema is autodetected or supplied in the job configuration. Schema update options are supported in two cases: when writeDisposition is WRITE_APPEND; when writeDisposition is WRITE_TRUNCATE and the destination table is a partition of a table, specified by partition decorators. For normal tables, WRITE_TRUNCATE will always overwrite the schema. One or more of the following values are specified: ALLOW_FIELD_ADDITION: allow adding a nullable field to the schema. ALLOW_FIELD_RELAXATION: allow relaxing a required field in the original schema to nullable.

  • scriptOptions (pulumi.Input[dict]) - Options controlling the execution of scripts. Structure is documented below.

    • keyResultStatement (pulumi.Input[str]) - Determines which statement in the script represents the “key result”, used to populate the schema and query results of the script job.

    • statementByteBudget (pulumi.Input[str]) - Limit on the number of bytes billed per statement. Exceeding this budget results in an error.

    • statementTimeoutMs (pulumi.Input[str]) - Timeout period for each statement in a script.

  • useLegacySql (pulumi.Input[bool]) - Specifies whether to use BigQuery’s legacy SQL dialect for this query. The default value is true. If set to false, the query will use BigQuery’s standard SQL.

  • useQueryCache (pulumi.Input[bool]) - Whether to look for the result in the query cache. The query cache is a best-effort cache that will be flushed whenever tables in the query are modified. Moreover, the query cache is only available when a query does not have a destination table specified. The default value is true.

  • userDefinedFunctionResources (pulumi.Input[list]) - Describes user-defined function resources used in the query. Structure is documented below.

    • inlineCode (pulumi.Input[str]) - An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code.

    • resource_uri (pulumi.Input[str]) - A code resource to load from a Google Cloud Storage URI (gs://bucket/path).

  • writeDisposition (pulumi.Input[str]) - Specifies the action that occurs if the destination table already exists. The following values are supported: WRITE_TRUNCATE: If the table already exists, BigQuery overwrites the table data and uses the schema from the query result. WRITE_APPEND: If the table already exists, BigQuery appends the data to the table. WRITE_EMPTY: If the table already exists and contains data, a ‘duplicate’ error is returned in the job result. Each action is atomic and only occurs if BigQuery is able to complete the job successfully. Creation, truncation and append actions occur as one atomic update upon job completion.

copy: pulumi.Output[dict] = None

Copies a table. Structure is documented below.

  • createDisposition (str) - Specifies whether the job is allowed to create new tables. The following values are supported: CREATE_IF_NEEDED: If the table does not exist, BigQuery creates the table. CREATE_NEVER: The table must already exist. If it does not, a ‘notFound’ error is returned in the job result. Creation, truncation and append actions occur as one atomic update upon job completion

  • destinationEncryptionConfiguration (dict) - Custom encryption configuration (e.g., Cloud KMS keys) Structure is documented below.

    • kms_key_name (str) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

  • destinationTable (dict) - The destination table. Structure is documented below.

    • dataset_id (str) - The ID of the dataset containing this model.

    • project_id (str) - The ID of the project containing this model.

    • table_id (str) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • sourceTables (list) - Source tables to copy. Structure is documented below.

    • dataset_id (str) - The ID of the dataset containing this model.

    • project_id (str) - The ID of the project containing this model.

    • table_id (str) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • writeDisposition (str) - Specifies the action that occurs if the destination table already exists. The following values are supported: WRITE_TRUNCATE: If the table already exists, BigQuery overwrites the table data and uses the schema from the query result. WRITE_APPEND: If the table already exists, BigQuery appends the data to the table. WRITE_EMPTY: If the table already exists and contains data, a ‘duplicate’ error is returned in the job result. Each action is atomic and only occurs if BigQuery is able to complete the job successfully. Creation, truncation and append actions occur as one atomic update upon job completion.

extract: pulumi.Output[dict] = None

Configures an extract job. Structure is documented below.

  • compression (str) - The compression type to use for exported files. Possible values include GZIP, DEFLATE, SNAPPY, and NONE. The default value is NONE. DEFLATE and SNAPPY are only supported for Avro.

  • destinationFormat (str) - The exported file format. Possible values include CSV, NEWLINE_DELIMITED_JSON and AVRO for tables and SAVED_MODEL for models. The default value for tables is CSV. Tables with nested or repeated fields cannot be exported as CSV. The default value for models is SAVED_MODEL.

  • destinationUris (list) - A list of fully-qualified Google Cloud Storage URIs where the extracted table should be written.

  • fieldDelimiter (str) - When extracting data in CSV format, this defines the delimiter to use between fields in the exported data. Default is ‘,’

  • printHeader (bool) - Whether to print out a header row in the results. Default is true.

  • sourceModel (dict) - A reference to the model being exported. Structure is documented below.

    • dataset_id (str) - The ID of the dataset containing this model.

    • modelId (str) - The ID of the model.

    • project_id (str) - The ID of the project containing this model.

  • sourceTable (dict) - A reference to the table being exported. Structure is documented below.

    • dataset_id (str) - The ID of the dataset containing this model.

    • project_id (str) - The ID of the project containing this model.

    • table_id (str) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • useAvroLogicalTypes (bool) - Whether to use logical types when extracting to AVRO format.

job_id: pulumi.Output[str] = None

The ID of the job. The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (_), or dashes (-). The maximum length is 1,024 characters.

job_timeout_ms: pulumi.Output[str] = None

Job timeout in milliseconds. If this time limit is exceeded, BigQuery may attempt to terminate the job.

job_type: pulumi.Output[str] = None

The type of the job.

labels: pulumi.Output[dict] = None

The labels associated with this job. You can use these to organize and group your jobs.

load: pulumi.Output[dict] = None

Configures a load job. Structure is documented below.

  • allowJaggedRows (bool) - Accept rows that are missing trailing optional columns. The missing values are treated as nulls. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. Only applicable to CSV, ignored for other formats.

  • allowQuotedNewlines (bool) - Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.

  • autodetect (bool) - Indicates if we should automatically infer the options and schema for CSV and JSON sources.

  • createDisposition (str) - Specifies whether the job is allowed to create new tables. The following values are supported: CREATE_IF_NEEDED: If the table does not exist, BigQuery creates the table. CREATE_NEVER: The table must already exist. If it does not, a ‘notFound’ error is returned in the job result. Creation, truncation and append actions occur as one atomic update upon job completion

  • destinationEncryptionConfiguration (dict) - Custom encryption configuration (e.g., Cloud KMS keys) Structure is documented below.

    • kms_key_name (str) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

  • destinationTable (dict) - The destination table. Structure is documented below.

    • dataset_id (str) - The ID of the dataset containing this model.

    • project_id (str) - The ID of the project containing this model.

    • table_id (str) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • encoding (str) - The character encoding of the data. The supported values are UTF-8 or ISO-8859-1. The default value is UTF-8. BigQuery decodes the data after the raw, binary data has been split using the values of the quote and fieldDelimiter properties.

  • fieldDelimiter (str) - When extracting data in CSV format, this defines the delimiter to use between fields in the exported data. Default is ‘,’

  • ignoreUnknownValues (bool) - Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The sourceFormat property determines what BigQuery treats as an extra value: CSV: Trailing columns JSON: Named values that don’t match any column names

  • maxBadRecords (float) - The maximum number of bad records that BigQuery can ignore when running the job. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid.

  • nullMarker (str) - Specifies a string that represents a null value in a CSV file. The default value is the empty string. If you set this property to a custom value, BigQuery throws an error if an empty string is present for all data types except for STRING and BYTE. For STRING and BYTE columns, BigQuery interprets the empty string as an empty value.

  • projectionFields (list) - If sourceFormat is set to “DATASTORE_BACKUP”, indicates which entity properties to load into BigQuery from a Cloud Datastore backup. Property names are case sensitive and must be top-level properties. If no properties are specified, BigQuery loads all properties. If any named property isn’t found in the Cloud Datastore backup, an invalid error is returned in the job result.

  • quote (str) - The value that is used to quote data sections in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. The default value is a double-quote (‘”’). If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allowQuotedNewlines property to true.

  • schemaUpdateOptions (list) - Allows the schema of the destination table to be updated as a side effect of the load job if a schema is autodetected or supplied in the job configuration. Schema update options are supported in two cases: when writeDisposition is WRITE_APPEND; when writeDisposition is WRITE_TRUNCATE and the destination table is a partition of a table, specified by partition decorators. For normal tables, WRITE_TRUNCATE will always overwrite the schema. One or more of the following values are specified: ALLOW_FIELD_ADDITION: allow adding a nullable field to the schema. ALLOW_FIELD_RELAXATION: allow relaxing a required field in the original schema to nullable.

  • skipLeadingRows (float) - The number of rows at the top of a CSV file that BigQuery will skip when loading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped. When autodetect is on, the behavior is the following: skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.

  • sourceFormat (str) - The format of the data files. For CSV files, specify “CSV”. For datastore backups, specify “DATASTORE_BACKUP”. For newline-delimited JSON, specify “NEWLINE_DELIMITED_JSON”. For Avro, specify “AVRO”. For parquet, specify “PARQUET”. For orc, specify “ORC”. The default value is CSV.

  • sourceUris (list) - The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one ‘’ wildcard character and it must come after the ‘bucket’ name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups: Exactly one URI can be specified. Also, the ‘’ wildcard character is not allowed.

  • time_partitioning (dict) - Time-based partitioning specification for the destination table. Structure is documented below.

    • expirationMs (str) - Number of milliseconds for which to keep the storage for a partition. A wrapper is used here because 0 is an invalid value.

    • field (str) - If not set, the table is partitioned by pseudo column ‘_PARTITIONTIME’; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. A wrapper is used here because an empty string is an invalid value.

    • type (str) - The only type supported is DAY, which will generate one partition per day. Providing an empty string used to cause an error, but in OnePlatform the field will be treated as unset.

  • writeDisposition (str) - Specifies the action that occurs if the destination table already exists. The following values are supported: WRITE_TRUNCATE: If the table already exists, BigQuery overwrites the table data and uses the schema from the query result. WRITE_APPEND: If the table already exists, BigQuery appends the data to the table. WRITE_EMPTY: If the table already exists and contains data, a ‘duplicate’ error is returned in the job result. Each action is atomic and only occurs if BigQuery is able to complete the job successfully. Creation, truncation and append actions occur as one atomic update upon job completion.

location: pulumi.Output[str] = None

The geographic location of the job. The default value is US.

project: pulumi.Output[str] = None

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

query: pulumi.Output[dict] = None

Configures a query job. Structure is documented below.

  • allowLargeResults (bool) - If true and query uses legacy SQL dialect, allows the query to produce arbitrarily large result tables at a slight cost in performance. Requires destinationTable to be set. For standard SQL queries, this flag is ignored and large results are always allowed. However, you must still set destinationTable when result size exceeds the allowed maximum response size.

  • createDisposition (str) - Specifies whether the job is allowed to create new tables. The following values are supported: CREATE_IF_NEEDED: If the table does not exist, BigQuery creates the table. CREATE_NEVER: The table must already exist. If it does not, a ‘notFound’ error is returned in the job result. Creation, truncation and append actions occur as one atomic update upon job completion

  • defaultDataset (dict) - Specifies the default dataset to use for unqualified table names in the query. Note that this does not alter behavior of unqualified dataset names. Structure is documented below.

    • dataset_id (str) - The ID of the dataset containing this model.

    • project_id (str) - The ID of the project containing this model.

  • destinationEncryptionConfiguration (dict) - Custom encryption configuration (e.g., Cloud KMS keys) Structure is documented below.

    • kms_key_name (str) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

  • destinationTable (dict) - The destination table. Structure is documented below.

    • dataset_id (str) - The ID of the dataset containing this model.

    • project_id (str) - The ID of the project containing this model.

    • table_id (str) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • flattenResults (bool) - If true and query uses legacy SQL dialect, flattens all nested and repeated fields in the query results. allowLargeResults must be true if this is set to false. For standard SQL queries, this flag is ignored and results are never flattened.

  • maximumBillingTier (float) - Limits the billing tier for this job. Queries that have resource usage beyond this tier will fail (without incurring a charge). If unspecified, this will be set to your project default.

  • maximumBytesBilled (str) - Limits the bytes billed for this job. Queries that will have bytes billed beyond this limit will fail (without incurring a charge). If unspecified, this will be set to your project default.

  • parameterMode (str) - Standard SQL only. Set to POSITIONAL to use positional (?) query parameters or to NAMED to use named (@myparam) query parameters in this query.

  • priority (str) - Specifies a priority for the query.

  • query (str) - Configures a query job. Structure is documented below.

  • schemaUpdateOptions (list) - Allows the schema of the destination table to be updated as a side effect of the load job if a schema is autodetected or supplied in the job configuration. Schema update options are supported in two cases: when writeDisposition is WRITE_APPEND; when writeDisposition is WRITE_TRUNCATE and the destination table is a partition of a table, specified by partition decorators. For normal tables, WRITE_TRUNCATE will always overwrite the schema. One or more of the following values are specified: ALLOW_FIELD_ADDITION: allow adding a nullable field to the schema. ALLOW_FIELD_RELAXATION: allow relaxing a required field in the original schema to nullable.

  • scriptOptions (dict) - Options controlling the execution of scripts. Structure is documented below.

    • keyResultStatement (str) - Determines which statement in the script represents the “key result”, used to populate the schema and query results of the script job.

    • statementByteBudget (str) - Limit on the number of bytes billed per statement. Exceeding this budget results in an error.

    • statementTimeoutMs (str) - Timeout period for each statement in a script.

  • useLegacySql (bool) - Specifies whether to use BigQuery’s legacy SQL dialect for this query. The default value is true. If set to false, the query will use BigQuery’s standard SQL.

  • useQueryCache (bool) - Whether to look for the result in the query cache. The query cache is a best-effort cache that will be flushed whenever tables in the query are modified. Moreover, the query cache is only available when a query does not have a destination table specified. The default value is true.

  • userDefinedFunctionResources (list) - Describes user-defined function resources used in the query. Structure is documented below.

    • inlineCode (str) - An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code.

    • resource_uri (str) - A code resource to load from a Google Cloud Storage URI (gs://bucket/path).

  • writeDisposition (str) - Specifies the action that occurs if the destination table already exists. The following values are supported: WRITE_TRUNCATE: If the table already exists, BigQuery overwrites the table data and uses the schema from the query result. WRITE_APPEND: If the table already exists, BigQuery appends the data to the table. WRITE_EMPTY: If the table already exists and contains data, a ‘duplicate’ error is returned in the job result. Each action is atomic and only occurs if BigQuery is able to complete the job successfully. Creation, truncation and append actions occur as one atomic update upon job completion.

user_email: pulumi.Output[str] = None

Email address of the user who ran the job.

static get(resource_name, id, opts=None, copy=None, extract=None, job_id=None, job_timeout_ms=None, job_type=None, labels=None, load=None, location=None, project=None, query=None, user_email=None)

Get an existing Job resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters
  • resource_name (str) – The unique name of the resulting resource.

  • id (str) – The unique provider ID of the resource to lookup.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • copy (pulumi.Input[dict]) – Copies a table. Structure is documented below.

  • extract (pulumi.Input[dict]) – Configures an extract job. Structure is documented below.

  • job*id (pulumi.Input[str]) –

    The ID of the job. The ID must contain only letters (a-z, A-Z), numbers (0-9), underscores (*), or dashes (-). The maximum length is 1,024 characters.

  • job_timeout_ms (pulumi.Input[str]) – Job timeout in milliseconds. If this time limit is exceeded, BigQuery may attempt to terminate the job.

  • job_type (pulumi.Input[str]) – The type of the job.

  • labels (pulumi.Input[dict]) – The labels associated with this job. You can use these to organize and group your jobs.

  • load (pulumi.Input[dict]) – Configures a load job. Structure is documented below.

  • location (pulumi.Input[str]) – The geographic location of the job. The default value is US.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • query (pulumi.Input[dict]) – Configures a query job. Structure is documented below.

  • user_email (pulumi.Input[str]) – Email address of the user who ran the job.

The copy object supports the following:

  • createDisposition (pulumi.Input[str]) - Specifies whether the job is allowed to create new tables. The following values are supported: CREATE_IF_NEEDED: If the table does not exist, BigQuery creates the table. CREATE_NEVER: The table must already exist. If it does not, a ‘notFound’ error is returned in the job result. Creation, truncation and append actions occur as one atomic update upon job completion

  • destinationEncryptionConfiguration (pulumi.Input[dict]) - Custom encryption configuration (e.g., Cloud KMS keys) Structure is documented below.

    • kms_key_name (pulumi.Input[str]) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

  • destinationTable (pulumi.Input[dict]) - The destination table. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

    • table_id (pulumi.Input[str]) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • sourceTables (pulumi.Input[list]) - Source tables to copy. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

    • table_id (pulumi.Input[str]) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • writeDisposition (pulumi.Input[str]) - Specifies the action that occurs if the destination table already exists. The following values are supported: WRITE_TRUNCATE: If the table already exists, BigQuery overwrites the table data and uses the schema from the query result. WRITE_APPEND: If the table already exists, BigQuery appends the data to the table. WRITE_EMPTY: If the table already exists and contains data, a ‘duplicate’ error is returned in the job result. Each action is atomic and only occurs if BigQuery is able to complete the job successfully. Creation, truncation and append actions occur as one atomic update upon job completion.

The extract object supports the following:

  • compression (pulumi.Input[str]) - The compression type to use for exported files. Possible values include GZIP, DEFLATE, SNAPPY, and NONE. The default value is NONE. DEFLATE and SNAPPY are only supported for Avro.

  • destinationFormat (pulumi.Input[str]) - The exported file format. Possible values include CSV, NEWLINE_DELIMITED_JSON and AVRO for tables and SAVED_MODEL for models. The default value for tables is CSV. Tables with nested or repeated fields cannot be exported as CSV. The default value for models is SAVED_MODEL.

  • destinationUris (pulumi.Input[list]) - A list of fully-qualified Google Cloud Storage URIs where the extracted table should be written.

  • fieldDelimiter (pulumi.Input[str]) - When extracting data in CSV format, this defines the delimiter to use between fields in the exported data. Default is ‘,’

  • printHeader (pulumi.Input[bool]) - Whether to print out a header row in the results. Default is true.

  • sourceModel (pulumi.Input[dict]) - A reference to the model being exported. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • modelId (pulumi.Input[str]) - The ID of the model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

  • sourceTable (pulumi.Input[dict]) - A reference to the table being exported. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

    • table_id (pulumi.Input[str]) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • useAvroLogicalTypes (pulumi.Input[bool]) - Whether to use logical types when extracting to AVRO format.

The load object supports the following:

  • allowJaggedRows (pulumi.Input[bool]) - Accept rows that are missing trailing optional columns. The missing values are treated as nulls. If false, records with missing trailing columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. Only applicable to CSV, ignored for other formats.

  • allowQuotedNewlines (pulumi.Input[bool]) - Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.

  • autodetect (pulumi.Input[bool]) - Indicates if we should automatically infer the options and schema for CSV and JSON sources.

  • createDisposition (pulumi.Input[str]) - Specifies whether the job is allowed to create new tables. The following values are supported: CREATE_IF_NEEDED: If the table does not exist, BigQuery creates the table. CREATE_NEVER: The table must already exist. If it does not, a ‘notFound’ error is returned in the job result. Creation, truncation and append actions occur as one atomic update upon job completion

  • destinationEncryptionConfiguration (pulumi.Input[dict]) - Custom encryption configuration (e.g., Cloud KMS keys) Structure is documented below.

    • kms_key_name (pulumi.Input[str]) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

  • destinationTable (pulumi.Input[dict]) - The destination table. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

    • table_id (pulumi.Input[str]) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • encoding (pulumi.Input[str]) - The character encoding of the data. The supported values are UTF-8 or ISO-8859-1. The default value is UTF-8. BigQuery decodes the data after the raw, binary data has been split using the values of the quote and fieldDelimiter properties.

  • fieldDelimiter (pulumi.Input[str]) - When extracting data in CSV format, this defines the delimiter to use between fields in the exported data. Default is ‘,’

  • ignoreUnknownValues (pulumi.Input[bool]) - Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false. The sourceFormat property determines what BigQuery treats as an extra value: CSV: Trailing columns JSON: Named values that don’t match any column names

  • maxBadRecords (pulumi.Input[float]) - The maximum number of bad records that BigQuery can ignore when running the job. If the number of bad records exceeds this value, an invalid error is returned in the job result. The default value is 0, which requires that all records are valid.

  • nullMarker (pulumi.Input[str]) - Specifies a string that represents a null value in a CSV file. The default value is the empty string. If you set this property to a custom value, BigQuery throws an error if an empty string is present for all data types except for STRING and BYTE. For STRING and BYTE columns, BigQuery interprets the empty string as an empty value.

  • projectionFields (pulumi.Input[list]) - If sourceFormat is set to “DATASTORE_BACKUP”, indicates which entity properties to load into BigQuery from a Cloud Datastore backup. Property names are case sensitive and must be top-level properties. If no properties are specified, BigQuery loads all properties. If any named property isn’t found in the Cloud Datastore backup, an invalid error is returned in the job result.

  • quote (pulumi.Input[str]) - The value that is used to quote data sections in a CSV file. BigQuery converts the string to ISO-8859-1 encoding, and then uses the first byte of the encoded string to split the data in its raw, binary state. The default value is a double-quote (‘”’). If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allowQuotedNewlines property to true.

  • schemaUpdateOptions (pulumi.Input[list]) - Allows the schema of the destination table to be updated as a side effect of the load job if a schema is autodetected or supplied in the job configuration. Schema update options are supported in two cases: when writeDisposition is WRITE_APPEND; when writeDisposition is WRITE_TRUNCATE and the destination table is a partition of a table, specified by partition decorators. For normal tables, WRITE_TRUNCATE will always overwrite the schema. One or more of the following values are specified: ALLOW_FIELD_ADDITION: allow adding a nullable field to the schema. ALLOW_FIELD_RELAXATION: allow relaxing a required field in the original schema to nullable.

  • skipLeadingRows (pulumi.Input[float]) - The number of rows at the top of a CSV file that BigQuery will skip when loading the data. The default value is 0. This property is useful if you have header rows in the file that should be skipped. When autodetect is on, the behavior is the following: skipLeadingRows unspecified - Autodetect tries to detect headers in the first row. If they are not detected, the row is read as data. Otherwise data is read starting from the second row. skipLeadingRows is 0 - Instructs autodetect that there are no headers and data should be read starting from the first row. skipLeadingRows = N > 0 - Autodetect skips N-1 rows and tries to detect headers in row N. If headers are not detected, row N is just skipped. Otherwise row N is used to extract column names for the detected schema.

  • sourceFormat (pulumi.Input[str]) - The format of the data files. For CSV files, specify “CSV”. For datastore backups, specify “DATASTORE_BACKUP”. For newline-delimited JSON, specify “NEWLINE_DELIMITED_JSON”. For Avro, specify “AVRO”. For parquet, specify “PARQUET”. For orc, specify “ORC”. The default value is CSV.

  • sourceUris (pulumi.Input[list]) - The fully-qualified URIs that point to your data in Google Cloud. For Google Cloud Storage URIs: Each URI can contain one ‘’ wildcard character and it must come after the ‘bucket’ name. Size limits related to load jobs apply to external data sources. For Google Cloud Bigtable URIs: Exactly one URI can be specified and it has be a fully specified and valid HTTPS URL for a Google Cloud Bigtable table. For Google Cloud Datastore backups: Exactly one URI can be specified. Also, the ‘’ wildcard character is not allowed.

  • time_partitioning (pulumi.Input[dict]) - Time-based partitioning specification for the destination table. Structure is documented below.

    • expirationMs (pulumi.Input[str]) - Number of milliseconds for which to keep the storage for a partition. A wrapper is used here because 0 is an invalid value.

    • field (pulumi.Input[str]) - If not set, the table is partitioned by pseudo column ‘_PARTITIONTIME’; if set, the table is partitioned by this field. The field must be a top-level TIMESTAMP or DATE field. Its mode must be NULLABLE or REQUIRED. A wrapper is used here because an empty string is an invalid value.

    • type (pulumi.Input[str]) - The only type supported is DAY, which will generate one partition per day. Providing an empty string used to cause an error, but in OnePlatform the field will be treated as unset.

  • writeDisposition (pulumi.Input[str]) - Specifies the action that occurs if the destination table already exists. The following values are supported: WRITE_TRUNCATE: If the table already exists, BigQuery overwrites the table data and uses the schema from the query result. WRITE_APPEND: If the table already exists, BigQuery appends the data to the table. WRITE_EMPTY: If the table already exists and contains data, a ‘duplicate’ error is returned in the job result. Each action is atomic and only occurs if BigQuery is able to complete the job successfully. Creation, truncation and append actions occur as one atomic update upon job completion.

The query object supports the following:

  • allowLargeResults (pulumi.Input[bool]) - If true and query uses legacy SQL dialect, allows the query to produce arbitrarily large result tables at a slight cost in performance. Requires destinationTable to be set. For standard SQL queries, this flag is ignored and large results are always allowed. However, you must still set destinationTable when result size exceeds the allowed maximum response size.

  • createDisposition (pulumi.Input[str]) - Specifies whether the job is allowed to create new tables. The following values are supported: CREATE_IF_NEEDED: If the table does not exist, BigQuery creates the table. CREATE_NEVER: The table must already exist. If it does not, a ‘notFound’ error is returned in the job result. Creation, truncation and append actions occur as one atomic update upon job completion

  • defaultDataset (pulumi.Input[dict]) - Specifies the default dataset to use for unqualified table names in the query. Note that this does not alter behavior of unqualified dataset names. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

  • destinationEncryptionConfiguration (pulumi.Input[dict]) - Custom encryption configuration (e.g., Cloud KMS keys) Structure is documented below.

    • kms_key_name (pulumi.Input[str]) - Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.

  • destinationTable (pulumi.Input[dict]) - The destination table. Structure is documented below.

    • dataset_id (pulumi.Input[str]) - The ID of the dataset containing this model.

    • project_id (pulumi.Input[str]) - The ID of the project containing this model.

    • table_id (pulumi.Input[str]) - The table. Can be specified {{table_id}} if project_id and dataset_id are also set, or of the form projects/{{project}}/datasets/{{dataset_id}}/tables/{{table_id}} if not.

  • flattenResults (pulumi.Input[bool]) - If true and query uses legacy SQL dialect, flattens all nested and repeated fields in the query results. allowLargeResults must be true if this is set to false. For standard SQL queries, this flag is ignored and results are never flattened.

  • maximumBillingTier (pulumi.Input[float]) - Limits the billing tier for this job. Queries that have resource usage beyond this tier will fail (without incurring a charge). If unspecified, this will be set to your project default.

  • maximumBytesBilled (pulumi.Input[str]) - Limits the bytes billed for this job. Queries that will have bytes billed beyond this limit will fail (without incurring a charge). If unspecified, this will be set to your project default.

  • parameterMode (pulumi.Input[str]) - Standard SQL only. Set to POSITIONAL to use positional (?) query parameters or to NAMED to use named (@myparam) query parameters in this query.

  • priority (pulumi.Input[str]) - Specifies a priority for the query.

  • query (pulumi.Input[str]) - Configures a query job. Structure is documented below.

  • schemaUpdateOptions (pulumi.Input[list]) - Allows the schema of the destination table to be updated as a side effect of the load job if a schema is autodetected or supplied in the job configuration. Schema update options are supported in two cases: when writeDisposition is WRITE_APPEND; when writeDisposition is WRITE_TRUNCATE and the destination table is a partition of a table, specified by partition decorators. For normal tables, WRITE_TRUNCATE will always overwrite the schema. One or more of the following values are specified: ALLOW_FIELD_ADDITION: allow adding a nullable field to the schema. ALLOW_FIELD_RELAXATION: allow relaxing a required field in the original schema to nullable.

  • scriptOptions (pulumi.Input[dict]) - Options controlling the execution of scripts. Structure is documented below.

    • keyResultStatement (pulumi.Input[str]) - Determines which statement in the script represents the “key result”, used to populate the schema and query results of the script job.

    • statementByteBudget (pulumi.Input[str]) - Limit on the number of bytes billed per statement. Exceeding this budget results in an error.

    • statementTimeoutMs (pulumi.Input[str]) - Timeout period for each statement in a script.

  • useLegacySql (pulumi.Input[bool]) - Specifies whether to use BigQuery’s legacy SQL dialect for this query. The default value is true. If set to false, the query will use BigQuery’s standard SQL.

  • useQueryCache (pulumi.Input[bool]) - Whether to look for the result in the query cache. The query cache is a best-effort cache that will be flushed whenever tables in the query are modified. Moreover, the query cache is only available when a query does not have a destination table specified. The default value is true.

  • userDefinedFunctionResources (pulumi.Input[list]) - Describes user-defined function resources used in the query. Structure is documented below.

    • inlineCode (pulumi.Input[str]) - An inline resource that contains code for a user-defined function (UDF). Providing a inline code resource is equivalent to providing a URI for a file containing the same code.

    • resource_uri (pulumi.Input[str]) - A code resource to load from a Google Cloud Storage URI (gs://bucket/path).

  • writeDisposition (pulumi.Input[str]) - Specifies the action that occurs if the destination table already exists. The following values are supported: WRITE_TRUNCATE: If the table already exists, BigQuery overwrites the table data and uses the schema from the query result. WRITE_APPEND: If the table already exists, BigQuery appends the data to the table. WRITE_EMPTY: If the table already exists and contains data, a ‘duplicate’ error is returned in the job result. Each action is atomic and only occurs if BigQuery is able to complete the job successfully. Creation, truncation and append actions occur as one atomic update upon job completion.

translate_output_property(prop)

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

translate_input_property(prop)

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

class pulumi_gcp.bigquery.Reservation(resource_name, opts=None, ignore_idle_slots=None, location=None, name=None, project=None, slot_capacity=None, __props__=None, __name__=None, __opts__=None)

A reservation is a mechanism used to guarantee BigQuery slots to users.

To get more information about Reservation, see:

import pulumi
import pulumi_gcp as gcp

reservation = gcp.bigquery.Reservation("reservation",
    location="asia-northeast1",
    slot_capacity=0,
    ignore_idle_slots=False)
Parameters
  • resource_name (str) – The name of the resource.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • ignore_idle_slots (pulumi.Input[bool]) – If false, any query using this reservation will use idle slots from other reservations within the same admin project. If true, a query using this reservation will execute with the slot capacity specified above at most.

  • location (pulumi.Input[str]) – The geographic location where the transfer config should reside. Examples: US, EU, asia-northeast1. The default value is US.

  • name (pulumi.Input[str]) – The name of the reservation. This field must only contain alphanumeric characters or dash.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • slot_capacity (pulumi.Input[float]) – Minimum slots available to this reservation. A slot is a unit of computational power in BigQuery, and serves as the unit of parallelism. Queries using this reservation might use more slots during runtime if ignoreIdleSlots is set to false.

ignore_idle_slots: pulumi.Output[bool] = None

If false, any query using this reservation will use idle slots from other reservations within the same admin project. If true, a query using this reservation will execute with the slot capacity specified above at most.

location: pulumi.Output[str] = None

The geographic location where the transfer config should reside. Examples: US, EU, asia-northeast1. The default value is US.

name: pulumi.Output[str] = None

The name of the reservation. This field must only contain alphanumeric characters or dash.

project: pulumi.Output[str] = None

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

slot_capacity: pulumi.Output[float] = None

Minimum slots available to this reservation. A slot is a unit of computational power in BigQuery, and serves as the unit of parallelism. Queries using this reservation might use more slots during runtime if ignoreIdleSlots is set to false.

static get(resource_name, id, opts=None, ignore_idle_slots=None, location=None, name=None, project=None, slot_capacity=None)

Get an existing Reservation resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters
  • resource_name (str) – The unique name of the resulting resource.

  • id (str) – The unique provider ID of the resource to lookup.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • ignore_idle_slots (pulumi.Input[bool]) – If false, any query using this reservation will use idle slots from other reservations within the same admin project. If true, a query using this reservation will execute with the slot capacity specified above at most.

  • location (pulumi.Input[str]) – The geographic location where the transfer config should reside. Examples: US, EU, asia-northeast1. The default value is US.

  • name (pulumi.Input[str]) – The name of the reservation. This field must only contain alphanumeric characters or dash.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • slot_capacity (pulumi.Input[float]) – Minimum slots available to this reservation. A slot is a unit of computational power in BigQuery, and serves as the unit of parallelism. Queries using this reservation might use more slots during runtime if ignoreIdleSlots is set to false.

translate_output_property(prop)

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

translate_input_property(prop)

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

class pulumi_gcp.bigquery.Table(resource_name, opts=None, clusterings=None, dataset_id=None, description=None, encryption_configuration=None, expiration_time=None, external_data_configuration=None, friendly_name=None, labels=None, project=None, range_partitioning=None, schema=None, table_id=None, time_partitioning=None, view=None, __props__=None, __name__=None, __opts__=None)

Creates a table resource in a dataset for Google BigQuery. For more information see the official documentation and API.

import pulumi
import pulumi_gcp as gcp

default_dataset = gcp.bigquery.Dataset("defaultDataset",
    dataset_id="foo",
    friendly_name="test",
    description="This is a test description",
    location="EU",
    default_table_expiration_ms=3600000,
    labels={
        "env": "default",
    })
default_table = gcp.bigquery.Table("defaultTable",
    dataset_id=default_dataset.dataset_id,
    table_id="bar",
    time_partitioning={
        "type": "DAY",
    },
    labels={
        "env": "default",
    },
    schema="""[
  {
    "name": "permalink",
    "type": "STRING",
    "mode": "NULLABLE",
    "description": "The Permalink"
  },
  {
    "name": "state",
    "type": "STRING",
    "mode": "NULLABLE",
    "description": "State where the head office is located"
  }
]
""")
sheet = gcp.bigquery.Table("sheet",
    dataset_id=default_dataset.dataset_id,
    table_id="sheet",
    external_data_configuration={
        "autodetect": True,
        "sourceFormat": "GOOGLE_SHEETS",
        "google_sheets_options": {
            "skipLeadingRows": 1,
        },
        "sourceUris": ["https://docs.google.com/spreadsheets/d/123456789012345"],
    })
Parameters
  • resource_name (str) – The name of the resource.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • clusterings (pulumi.Input[list]) – Specifies column names to use for data clustering. Up to four top-level columns are allowed, and should be specified in descending priority order.

  • dataset_id (pulumi.Input[str]) – The dataset ID to create the table in. Changing this forces a new resource to be created.

  • description (pulumi.Input[str]) – The field description.

  • encryption_configuration (pulumi.Input[dict]) – Specifies how the table should be encrypted. If left blank, the table will be encrypted with a Google-managed key; that process is transparent to the user. Structure is documented below.

  • expiration_time (pulumi.Input[float]) – The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.

  • external_data_configuration (pulumi.Input[dict]) – Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table. Structure is documented below.

  • friendly_name (pulumi.Input[str]) – A descriptive name for the table.

  • labels (pulumi.Input[dict]) – A mapping of labels to assign to the resource.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • range_partitioning (pulumi.Input[dict]) – If specified, configures range-based partitioning for this table. Structure is documented below.

  • schema (pulumi.Input[str]) – A JSON schema for the table. Schema is required for CSV and JSON formats and is disallowed for Google Cloud Bigtable, Cloud Datastore backups, and Avro formats when using external tables. For more information see the BigQuery API documentation. ~>NOTE: Because this field expects a JSON string, any changes to the string will create a diff, even if the JSON itself hasn’t changed. If the API returns a different value for the same schema, e.g. it switched the order of values or replaced STRUCT field type with RECORD field type, we currently cannot suppress the recurring diff this causes. As a workaround, we recommend using the schema as returned by the API.

  • table_id (pulumi.Input[str]) – A unique ID for the resource. Changing this forces a new resource to be created.

  • time_partitioning (pulumi.Input[dict]) – If specified, configures time-based partitioning for this table. Structure is documented below.

  • view (pulumi.Input[dict]) – If specified, configures this table as a view. Structure is documented below.

The encryption_configuration object supports the following:

  • kms_key_name (pulumi.Input[str]) - The self link or full name of a key which should be used to encrypt this table. Note that the default bigquery service account will need to have encrypt/decrypt permissions on this key - you may want to see the bigquery.getDefaultServiceAccount datasource and the kms.CryptoKeyIAMBinding resource.

The external_data_configuration object supports the following:

  • autodetect (pulumi.Input[bool]) - - Let BigQuery try to autodetect the schema and format of the table.

  • compression (pulumi.Input[str]) - The compression type of the data source. Valid values are “NONE” or “GZIP”.

  • csvOptions (pulumi.Input[dict]) - Additional properties to set if source_format is set to “CSV”. Structure is documented below.

    • allowJaggedRows (pulumi.Input[bool]) - Indicates if BigQuery should accept rows that are missing trailing optional columns.

    • allowQuotedNewlines (pulumi.Input[bool]) - Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.

    • encoding (pulumi.Input[str]) - The character encoding of the data. The supported values are UTF-8 or ISO-8859-1.

    • fieldDelimiter (pulumi.Input[str]) - The separator for fields in a CSV file.

    • quote (pulumi.Input[str]) - The value that is used to quote data sections in a CSV file. If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allow_quoted_newlines property to true. The API-side default is ", specified in the provider escaped as ". Due to limitations with default values, this value is required to be explicitly set.

    • skipLeadingRows (pulumi.Input[float]) - The number of rows at the top of the sheet that BigQuery will skip when reading the data. At least one of range or skip_leading_rows must be set.

  • googleSheetsOptions (pulumi.Input[dict]) - Additional options if source_format is set to “GOOGLE_SHEETS”. Structure is documented below.

    • range (pulumi.Input[str]) - Information required to partition based on ranges. Structure is documented below.

    • skipLeadingRows (pulumi.Input[float]) - The number of rows at the top of the sheet that BigQuery will skip when reading the data. At least one of range or skip_leading_rows must be set.

  • hivePartitioningOptions (pulumi.Input[dict]) - When set, configures hive partitioning support. Not all storage formats support hive partitioning – requesting hive partitioning on an unsupported format will lead to an error, as will providing an invalid specification.

    • mode (pulumi.Input[str]) - When set, what mode of hive partitioning to use when reading data. The following modes are supported.

      • AUTO: automatically infer partition key name(s) and type(s).

      • STRINGS: automatically infer partition key name(s). All types are Not all storage formats support hive partitioning. Requesting hive partitioning on an unsupported format will lead to an error. Currently supported formats are: JSON, CSV, ORC, Avro and Parquet.

      • CUSTOM: when set to CUSTOM, you must encode the partition key schema within the source_uri_prefix by setting source_uri_prefix to gs://bucket/path_to_table/{key1:TYPE1}/{key2:TYPE2}/{key3:TYPE3}.

    • sourceUriPrefix (pulumi.Input[str]) - When hive partition detection is requested, a common for all source uris must be required. The prefix must end immediately before the partition key encoding begins. For example, consider files following this data layout. gs://bucket/path_to_table/dt=2019-06-01/country=USA/id=7/file.avro gs://bucket/path_to_table/dt=2019-05-31/country=CA/id=3/file.avro When hive partitioning is requested with either AUTO or STRINGS detection, the common prefix can be either of gs://bucket/path_to_table or gs://bucket/path_to_table/. Note that when mode is set to CUSTOM, you must encode the partition key schema within the source_uri_prefix by setting source_uri_prefix to gs://bucket/path_to_table/{key1:TYPE1}/{key2:TYPE2}/{key3:TYPE3}.

  • ignoreUnknownValues (pulumi.Input[bool]) - Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false.

  • maxBadRecords (pulumi.Input[float]) - The maximum number of bad records that BigQuery can ignore when reading data.

  • sourceFormat (pulumi.Input[str]) - The data format. Supported values are: “CSV”, “GOOGLE_SHEETS”, “NEWLINE_DELIMITED_JSON”, “AVRO”, “PARQUET”, and “DATSTORE_BACKUP”. To use “GOOGLE_SHEETS” the scopes must include “https://www.googleapis.com/auth/drive.readonly”.

  • sourceUris (pulumi.Input[list]) - A list of the fully-qualified URIs that point to your data in Google Cloud.

The range_partitioning object supports the following:

  • field (pulumi.Input[str]) - The field used to determine how to create a range-based partition.

  • range (pulumi.Input[dict]) - Information required to partition based on ranges. Structure is documented below.

    • end (pulumi.Input[float]) - End of the range partitioning, exclusive.

    • interval (pulumi.Input[float]) - The width of each range within the partition.

    • start (pulumi.Input[float]) - Start of the range partitioning, inclusive.

The time_partitioning object supports the following:

  • expirationMs (pulumi.Input[float]) - Number of milliseconds for which to keep the storage for a partition.

  • field (pulumi.Input[str]) - The field used to determine how to create a range-based partition.

  • requirePartitionFilter (pulumi.Input[bool]) - If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.

  • type (pulumi.Input[str]) - The only type supported is DAY, which will generate one partition per day based on data loading time.

The view object supports the following:

  • query (pulumi.Input[str]) - A query that BigQuery executes when the view is referenced.

  • useLegacySql (pulumi.Input[bool]) - Specifies whether to use BigQuery’s legacy SQL for this view. The default value is true. If set to false, the view will use BigQuery’s standard SQL.

clusterings: pulumi.Output[list] = None

Specifies column names to use for data clustering. Up to four top-level columns are allowed, and should be specified in descending priority order.

creation_time: pulumi.Output[float] = None

The time when this table was created, in milliseconds since the epoch.

dataset_id: pulumi.Output[str] = None

The dataset ID to create the table in. Changing this forces a new resource to be created.

description: pulumi.Output[str] = None

The field description.

encryption_configuration: pulumi.Output[dict] = None

Specifies how the table should be encrypted. If left blank, the table will be encrypted with a Google-managed key; that process is transparent to the user. Structure is documented below.

  • kms_key_name (str) - The self link or full name of a key which should be used to encrypt this table. Note that the default bigquery service account will need to have encrypt/decrypt permissions on this key - you may want to see the bigquery.getDefaultServiceAccount datasource and the kms.CryptoKeyIAMBinding resource.

etag: pulumi.Output[str] = None

A hash of the resource.

expiration_time: pulumi.Output[float] = None

The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.

external_data_configuration: pulumi.Output[dict] = None

Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table. Structure is documented below.

  • autodetect (bool) - - Let BigQuery try to autodetect the schema and format of the table.

  • compression (str) - The compression type of the data source. Valid values are “NONE” or “GZIP”.

  • csvOptions (dict) - Additional properties to set if source_format is set to “CSV”. Structure is documented below.

    • allowJaggedRows (bool) - Indicates if BigQuery should accept rows that are missing trailing optional columns.

    • allowQuotedNewlines (bool) - Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.

    • encoding (str) - The character encoding of the data. The supported values are UTF-8 or ISO-8859-1.

    • fieldDelimiter (str) - The separator for fields in a CSV file.

    • quote (str) - The value that is used to quote data sections in a CSV file. If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allow_quoted_newlines property to true. The API-side default is ", specified in the provider escaped as ". Due to limitations with default values, this value is required to be explicitly set.

    • skipLeadingRows (float) - The number of rows at the top of the sheet that BigQuery will skip when reading the data. At least one of range or skip_leading_rows must be set.

  • googleSheetsOptions (dict) - Additional options if source_format is set to “GOOGLE_SHEETS”. Structure is documented below.

    • range (str) - Information required to partition based on ranges. Structure is documented below.

    • skipLeadingRows (float) - The number of rows at the top of the sheet that BigQuery will skip when reading the data. At least one of range or skip_leading_rows must be set.

  • hivePartitioningOptions (dict) - When set, configures hive partitioning support. Not all storage formats support hive partitioning – requesting hive partitioning on an unsupported format will lead to an error, as will providing an invalid specification.

    • mode (str) - When set, what mode of hive partitioning to use when reading data. The following modes are supported.

      • AUTO: automatically infer partition key name(s) and type(s).

      • STRINGS: automatically infer partition key name(s). All types are Not all storage formats support hive partitioning. Requesting hive partitioning on an unsupported format will lead to an error. Currently supported formats are: JSON, CSV, ORC, Avro and Parquet.

      • CUSTOM: when set to CUSTOM, you must encode the partition key schema within the source_uri_prefix by setting source_uri_prefix to gs://bucket/path_to_table/{key1:TYPE1}/{key2:TYPE2}/{key3:TYPE3}.

    • sourceUriPrefix (str) - When hive partition detection is requested, a common for all source uris must be required. The prefix must end immediately before the partition key encoding begins. For example, consider files following this data layout. gs://bucket/path_to_table/dt=2019-06-01/country=USA/id=7/file.avro gs://bucket/path_to_table/dt=2019-05-31/country=CA/id=3/file.avro When hive partitioning is requested with either AUTO or STRINGS detection, the common prefix can be either of gs://bucket/path_to_table or gs://bucket/path_to_table/. Note that when mode is set to CUSTOM, you must encode the partition key schema within the source_uri_prefix by setting source_uri_prefix to gs://bucket/path_to_table/{key1:TYPE1}/{key2:TYPE2}/{key3:TYPE3}.

  • ignoreUnknownValues (bool) - Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false.

  • maxBadRecords (float) - The maximum number of bad records that BigQuery can ignore when reading data.

  • sourceFormat (str) - The data format. Supported values are: “CSV”, “GOOGLE_SHEETS”, “NEWLINE_DELIMITED_JSON”, “AVRO”, “PARQUET”, and “DATSTORE_BACKUP”. To use “GOOGLE_SHEETS” the scopes must include “https://www.googleapis.com/auth/drive.readonly”.

  • sourceUris (list) - A list of the fully-qualified URIs that point to your data in Google Cloud.

friendly_name: pulumi.Output[str] = None

A descriptive name for the table.

labels: pulumi.Output[dict] = None

A mapping of labels to assign to the resource.

last_modified_time: pulumi.Output[float] = None

The time when this table was last modified, in milliseconds since the epoch.

location: pulumi.Output[str] = None

The geographic location where the table resides. This value is inherited from the dataset.

num_bytes: pulumi.Output[float] = None

The size of this table in bytes, excluding any data in the streaming buffer.

num_long_term_bytes: pulumi.Output[float] = None

The number of bytes in the table that are considered “long-term storage”.

num_rows: pulumi.Output[float] = None

The number of rows of data in this table, excluding any data in the streaming buffer.

project: pulumi.Output[str] = None

The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

range_partitioning: pulumi.Output[dict] = None

If specified, configures range-based partitioning for this table. Structure is documented below.

  • field (str) - The field used to determine how to create a range-based partition.

  • range (dict) - Information required to partition based on ranges. Structure is documented below.

    • end (float) - End of the range partitioning, exclusive.

    • interval (float) - The width of each range within the partition.

    • start (float) - Start of the range partitioning, inclusive.

schema: pulumi.Output[str] = None

A JSON schema for the table. Schema is required for CSV and JSON formats and is disallowed for Google Cloud Bigtable, Cloud Datastore backups, and Avro formats when using external tables. For more information see the BigQuery API documentation. ~>NOTE: Because this field expects a JSON string, any changes to the string will create a diff, even if the JSON itself hasn’t changed. If the API returns a different value for the same schema, e.g. it switched the order of values or replaced STRUCT field type with RECORD field type, we currently cannot suppress the recurring diff this causes. As a workaround, we recommend using the schema as returned by the API.

The URI of the created resource.

table_id: pulumi.Output[str] = None

A unique ID for the resource. Changing this forces a new resource to be created.

time_partitioning: pulumi.Output[dict] = None

If specified, configures time-based partitioning for this table. Structure is documented below.

  • expirationMs (float) - Number of milliseconds for which to keep the storage for a partition.

  • field (str) - The field used to determine how to create a range-based partition.

  • requirePartitionFilter (bool) - If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.

  • type (str) - The only type supported is DAY, which will generate one partition per day based on data loading time.

type: pulumi.Output[str] = None

The only type supported is DAY, which will generate one partition per day based on data loading time.

view: pulumi.Output[dict] = None

If specified, configures this table as a view. Structure is documented below.

  • query (str) - A query that BigQuery executes when the view is referenced.

  • useLegacySql (bool) - Specifies whether to use BigQuery’s legacy SQL for this view. The default value is true. If set to false, the view will use BigQuery’s standard SQL.

static get(resource_name, id, opts=None, clusterings=None, creation_time=None, dataset_id=None, description=None, encryption_configuration=None, etag=None, expiration_time=None, external_data_configuration=None, friendly_name=None, labels=None, last_modified_time=None, location=None, num_bytes=None, num_long_term_bytes=None, num_rows=None, project=None, range_partitioning=None, schema=None, self_link=None, table_id=None, time_partitioning=None, type=None, view=None)

Get an existing Table resource’s state with the given name, id, and optional extra properties used to qualify the lookup.

Parameters
  • resource_name (str) – The unique name of the resulting resource.

  • id (str) – The unique provider ID of the resource to lookup.

  • opts (pulumi.ResourceOptions) – Options for the resource.

  • clusterings (pulumi.Input[list]) – Specifies column names to use for data clustering. Up to four top-level columns are allowed, and should be specified in descending priority order.

  • creation_time (pulumi.Input[float]) – The time when this table was created, in milliseconds since the epoch.

  • dataset_id (pulumi.Input[str]) – The dataset ID to create the table in. Changing this forces a new resource to be created.

  • description (pulumi.Input[str]) – The field description.

  • encryption_configuration (pulumi.Input[dict]) – Specifies how the table should be encrypted. If left blank, the table will be encrypted with a Google-managed key; that process is transparent to the user. Structure is documented below.

  • etag (pulumi.Input[str]) – A hash of the resource.

  • expiration_time (pulumi.Input[float]) – The time when this table expires, in milliseconds since the epoch. If not present, the table will persist indefinitely. Expired tables will be deleted and their storage reclaimed.

  • external_data_configuration (pulumi.Input[dict]) – Describes the data format, location, and other properties of a table stored outside of BigQuery. By defining these properties, the data source can then be queried as if it were a standard BigQuery table. Structure is documented below.

  • friendly_name (pulumi.Input[str]) – A descriptive name for the table.

  • labels (pulumi.Input[dict]) – A mapping of labels to assign to the resource.

  • last_modified_time (pulumi.Input[float]) – The time when this table was last modified, in milliseconds since the epoch.

  • location (pulumi.Input[str]) – The geographic location where the table resides. This value is inherited from the dataset.

  • num_bytes (pulumi.Input[float]) – The size of this table in bytes, excluding any data in the streaming buffer.

  • num_long_term_bytes (pulumi.Input[float]) – The number of bytes in the table that are considered “long-term storage”.

  • num_rows (pulumi.Input[float]) – The number of rows of data in this table, excluding any data in the streaming buffer.

  • project (pulumi.Input[str]) – The ID of the project in which the resource belongs. If it is not provided, the provider project is used.

  • range_partitioning (pulumi.Input[dict]) – If specified, configures range-based partitioning for this table. Structure is documented below.

  • schema (pulumi.Input[str]) –

    A JSON schema for the table. Schema is required for CSV and JSON formats and is disallowed for Google Cloud Bigtable, Cloud Datastore backups, and Avro formats when using external tables. For more information see the BigQuery API documentation. ~>NOTE: Because this field expects a JSON string, any changes to the string will create a diff, even if the JSON itself hasn’t changed. If the API returns a different value for the same schema, e.g. it switched the order of values or replaced STRUCT field type with RECORD field type, we currently cannot suppress the recurring diff this causes. As a workaround, we recommend using the schema as returned by the API.

  • self_link (pulumi.Input[str]) – The URI of the created resource.

  • table_id (pulumi.Input[str]) – A unique ID for the resource. Changing this forces a new resource to be created.

  • time_partitioning (pulumi.Input[dict]) – If specified, configures time-based partitioning for this table. Structure is documented below.

  • type (pulumi.Input[str]) – The only type supported is DAY, which will generate one partition per day based on data loading time.

  • view (pulumi.Input[dict]) – If specified, configures this table as a view. Structure is documented below.

The encryption_configuration object supports the following:

  • kms_key_name (pulumi.Input[str]) - The self link or full name of a key which should be used to encrypt this table. Note that the default bigquery service account will need to have encrypt/decrypt permissions on this key - you may want to see the bigquery.getDefaultServiceAccount datasource and the kms.CryptoKeyIAMBinding resource.

The external_data_configuration object supports the following:

  • autodetect (pulumi.Input[bool]) - - Let BigQuery try to autodetect the schema and format of the table.

  • compression (pulumi.Input[str]) - The compression type of the data source. Valid values are “NONE” or “GZIP”.

  • csvOptions (pulumi.Input[dict]) - Additional properties to set if source_format is set to “CSV”. Structure is documented below.

    • allowJaggedRows (pulumi.Input[bool]) - Indicates if BigQuery should accept rows that are missing trailing optional columns.

    • allowQuotedNewlines (pulumi.Input[bool]) - Indicates if BigQuery should allow quoted data sections that contain newline characters in a CSV file. The default value is false.

    • encoding (pulumi.Input[str]) - The character encoding of the data. The supported values are UTF-8 or ISO-8859-1.

    • fieldDelimiter (pulumi.Input[str]) - The separator for fields in a CSV file.

    • quote (pulumi.Input[str]) - The value that is used to quote data sections in a CSV file. If your data does not contain quoted sections, set the property value to an empty string. If your data contains quoted newline characters, you must also set the allow_quoted_newlines property to true. The API-side default is ", specified in the provider escaped as ". Due to limitations with default values, this value is required to be explicitly set.

    • skipLeadingRows (pulumi.Input[float]) - The number of rows at the top of the sheet that BigQuery will skip when reading the data. At least one of range or skip_leading_rows must be set.

  • googleSheetsOptions (pulumi.Input[dict]) - Additional options if source_format is set to “GOOGLE_SHEETS”. Structure is documented below.

    • range (pulumi.Input[str]) - Information required to partition based on ranges. Structure is documented below.

    • skipLeadingRows (pulumi.Input[float]) - The number of rows at the top of the sheet that BigQuery will skip when reading the data. At least one of range or skip_leading_rows must be set.

  • hivePartitioningOptions (pulumi.Input[dict]) - When set, configures hive partitioning support. Not all storage formats support hive partitioning – requesting hive partitioning on an unsupported format will lead to an error, as will providing an invalid specification.

    • mode (pulumi.Input[str]) - When set, what mode of hive partitioning to use when reading data. The following modes are supported.

      • AUTO: automatically infer partition key name(s) and type(s).

      • STRINGS: automatically infer partition key name(s). All types are Not all storage formats support hive partitioning. Requesting hive partitioning on an unsupported format will lead to an error. Currently supported formats are: JSON, CSV, ORC, Avro and Parquet.

      • CUSTOM: when set to CUSTOM, you must encode the partition key schema within the source_uri_prefix by setting source_uri_prefix to gs://bucket/path_to_table/{key1:TYPE1}/{key2:TYPE2}/{key3:TYPE3}.

    • sourceUriPrefix (pulumi.Input[str]) - When hive partition detection is requested, a common for all source uris must be required. The prefix must end immediately before the partition key encoding begins. For example, consider files following this data layout. gs://bucket/path_to_table/dt=2019-06-01/country=USA/id=7/file.avro gs://bucket/path_to_table/dt=2019-05-31/country=CA/id=3/file.avro When hive partitioning is requested with either AUTO or STRINGS detection, the common prefix can be either of gs://bucket/path_to_table or gs://bucket/path_to_table/. Note that when mode is set to CUSTOM, you must encode the partition key schema within the source_uri_prefix by setting source_uri_prefix to gs://bucket/path_to_table/{key1:TYPE1}/{key2:TYPE2}/{key3:TYPE3}.

  • ignoreUnknownValues (pulumi.Input[bool]) - Indicates if BigQuery should allow extra values that are not represented in the table schema. If true, the extra values are ignored. If false, records with extra columns are treated as bad records, and if there are too many bad records, an invalid error is returned in the job result. The default value is false.

  • maxBadRecords (pulumi.Input[float]) - The maximum number of bad records that BigQuery can ignore when reading data.

  • sourceFormat (pulumi.Input[str]) - The data format. Supported values are: “CSV”, “GOOGLE_SHEETS”, “NEWLINE_DELIMITED_JSON”, “AVRO”, “PARQUET”, and “DATSTORE_BACKUP”. To use “GOOGLE_SHEETS” the scopes must include “https://www.googleapis.com/auth/drive.readonly”.

  • sourceUris (pulumi.Input[list]) - A list of the fully-qualified URIs that point to your data in Google Cloud.

The range_partitioning object supports the following:

  • field (pulumi.Input[str]) - The field used to determine how to create a range-based partition.

  • range (pulumi.Input[dict]) - Information required to partition based on ranges. Structure is documented below.

    • end (pulumi.Input[float]) - End of the range partitioning, exclusive.

    • interval (pulumi.Input[float]) - The width of each range within the partition.

    • start (pulumi.Input[float]) - Start of the range partitioning, inclusive.

The time_partitioning object supports the following:

  • expirationMs (pulumi.Input[float]) - Number of milliseconds for which to keep the storage for a partition.

  • field (pulumi.Input[str]) - The field used to determine how to create a range-based partition.

  • requirePartitionFilter (pulumi.Input[bool]) - If set to true, queries over this table require a partition filter that can be used for partition elimination to be specified.

  • type (pulumi.Input[str]) - The only type supported is DAY, which will generate one partition per day based on data loading time.

The view object supports the following:

  • query (pulumi.Input[str]) - A query that BigQuery executes when the view is referenced.

  • useLegacySql (pulumi.Input[bool]) - Specifies whether to use BigQuery’s legacy SQL for this view. The default value is true. If set to false, the view will use BigQuery’s standard SQL.

translate_output_property(prop)

Provides subclasses of Resource an opportunity to translate names of output properties into a format of their choosing before writing those properties to the resource object.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

translate_input_property(prop)

Provides subclasses of Resource an opportunity to translate names of input properties into a format of their choosing before sending those properties to the Pulumi engine.

Parameters

prop (str) – A property name.

Returns

A potentially transformed property name.

Return type

str

pulumi_gcp.bigquery.get_default_service_account(project=None, opts=None)

Get the email address of a project’s unique BigQuery service account.

Each Google Cloud project has a unique service account used by BigQuery. When using BigQuery with customer-managed encryption keys, this account needs to be granted the cloudkms.cryptoKeyEncrypterDecrypter IAM role on the customer-managed Cloud KMS key used to protect the data.

For more information see the API reference.

import pulumi
import pulumi_gcp as gcp

bq_sa = gcp.bigquery.get_default_service_account()
key_sa_user = gcp.kms.CryptoKeyIAMMember("keySaUser",
    crypto_key_id=google_kms_crypto_key["key"]["id"],
    role="roles/cloudkms.cryptoKeyEncrypterDecrypter",
    member=f"serviceAccount:{bq_sa.email}")
Parameters

project (str) – The project the unique service account was created for. If it is not provided, the provider project is used.