Importing Google Cloud Infrastructure

Most infrastructure as code projects require working with existing cloud resources, whether those resources were originally created with another Infrastructure as Code (IaC) tool or manually provisioned with a cloud provider console or CLI. In this tutorial, you will learn how to import your existing Google Cloud resources to bring it under the management of Pulumi.

In this tutorial, you'll learn:

How to import resources using the CLI
How to import resources in bulk
How to import resources in code

Prerequisites:

Create initial resources

To start, login to the Google Cloud Console and follow the instructions in the official Google Cloud documentation to create a new Cloud Storage bucket.

Example Storage Bucket

Then, login to the Pulumi CLI and ensure it is configured to use your Google Cloud account. Next, create a new project and stack that will be used to hold the resource definition for your imported resources.

# Example using Python
$ mkdir pulumi-tutorial-import
$ cd pulumi-tutorial-import
$ pulumi new python

This tutorial will define resources using the Google Cloud Classic provider, so you will also need to make sure to install the Google Cloud Classic dependency into your project.

Importing a resource

In Pulumi, there are three paths to take when importing resources:

the pulumi import CLI command for individual resources
the pulumi import CLI command with a special JSON file for bulk import
an import option in your Pulumi program code.

Import using the CLI

The pulumi import command looks up the resource using the specified type token and resource identifier, adds the resource to the stack’s current state, and emits the code required to manage the resource with Pulumi from that point forward. This option requires the least manual effort, so it is generally recommended and best suited to projects consisting consisting of only one stack.

To import an existing cloud resource with the Pulumi CLI, use the following syntax:

$ pulumi import <type> <name> <id>

The first argument, type, is the Pulumi type token to use for the imported resource. You can find the type token for a given resource by navigating to the Import section of the resource’s API documentation in the Pulumi Registry. For example, the type token of an Google Cloud Storage Bucket resource is gcp:storage/bucket:Bucket.
The second argument, name, is the resource name to apply to the resource once it’s imported.
The third argument, id, corresponds to the value you would use in Pulumi to lookup the resource in the cloud provider. This value should correspond to the designated lookup property specified in the Import section of the resource’s API documentation in the Registry. In the case of an Google Cloud Storage Bucket, this would be bucket_name or project_id/bucket_name.

If the project ID is not passed to the import command, it will be inferred from the provider block or environment variables.

When put all together, the import command should look something like the following example, where imported-storage-bucket is the resource name that will be applied to the Storage Bucket once imported.

$ pulumi import gcp:storage/bucket:Bucket imported-storage-bucket pulumi-tutorial-storage-bucket

The output should look something like the following:

$ pulumi import gcp:storage/bucket:Bucket imported-storage-bucket pulumi-tutorial-storage-bucket

Previewing import (dev)

     Type                   Name                     Plan
 +   pulumi:pulumi:Stack    dev                      create
 =   └─ gcp:storage:Bucket  imported-storage-bucket  import

Resources:
    + 1 to create
    = 1 to import
    2 changes

Do you want to perform this import?  [Use arrows to move, type to filter]
> yes
  no
  details

Notice the equals sign (=) instead of our usual plus sign (+) in the resource table and in the details. This is Pulumi’s way of telling you that it’s adding something to the state without modifying it.

Choose yes to complete the import. This will immediately add the resource to the current stack’s state and will emit a block of code to STDOUT to be added to your Pulumi program. If the current program were written in Python, for example, the resulting CLI output would resemble the following:

...
Importing (dev)

     Type                   Name                     Status
 +   pulumi:pulumi:Stack    dev                      created
 =   └─ gcp:storage:Bucket  imported-storage-bucket  imported (0.42s)

Resources:
    + 1 created
    = 1 imported
    2 changes

Duration: 4s

Please copy the following code into your Pulumi application. Not doing so
will cause Pulumi to report that an update will happen on the next update command.

Please note that the imported resources are marked as protected. To destroy them
you will need to remove the `protect` option and run `pulumi update` *before*
the destroy will take effect.

import pulumi
import pulumi_gcp as gcp

imported_storage_bucket = gcp.storage.Bucket("imported-storage-bucket",
    location="US",
    name="pulumi-tutorial-storage-bucket",
    project="pulumi-devrel",
    public_access_prevention="enforced",
    rpo="DEFAULT",
    uniform_bucket_level_access=True,
    opts = pulumi.ResourceOptions(protect=True))

Next, copy the emitted code snippet and replace the contents of your Pulumi program file with it. Then, save the file and run pulumi up. You should see that the update produces no changes:

$ pulumi up

Previewing update (dev)

     Type                 Name                     Plan
     pulumi:pulumi:Stack  dev

Resources:
    2 unchanged

Do you want to perform this update? yes

Updating (dev)

     Type                 Name                     Status
     pulumi:pulumi:Stack  dev

Resources:
    2 unchanged

Duration: 2s

The resource is now under management with Pulumi!

Resources imported with the CLI are marked as protected to guard against accidental deletion. If, for example, you forgot to append the generated code to your program before running another pulumi up, Pulumi would first interpret the missing code as an intention to delete the resource. The protect property will prevent this from happening, leaving the resource intact. If you ever want to delete this resource, you will have to set the protect property to false in the code. You can learn more by visiting the Resource option: protect documentation.

Import in bulk

The pulumi import command also enables you to import resources in bulk for scenarios in which you need to bring multiple resources under management with Pulumi. To do so, you will need to create a JSON file that has all of the required information for each resource: a type, a desired name, and an id.

To start, return to the Google Cloud console and create two additional Storage Buckets. Next, return to your program folder and create a new file called resources.json. Inside of this file, copy and paste the following JSON object, making sure to replace the values of the id parameters with the actual values for your environment:

{
    "resources": [
        {
            "type": "gcp:storage/bucket:Bucket",
            "name": "second-imported-storage-bucket",
            "id": "tutorial-storage-bucket-2" # REPLACE
        },
        {
            "type": "gcp:storage/bucket:Bucket",
            "name": "third-imported-storage-bucket",
            "id": "tutorial-storage-bucket-3" # REPLACE
        }
    ]
}

To import these resources, save the file and then run the pulumi import command with the -f flag, passing in the path to the resources.json file:

$ pulumi import -f ./resources.json

Previewing import (dev)

     Type                   Name                            Plan
 +   pulumi:pulumi:Stack    dev                             create
 =   ├─ gcp:storage:Bucket  second-imported-storage-bucket  import
 =   └─ gcp:storage:Bucket  third-imported-storage-bucket   import

Resources:
    = 2 to import
    2 unchanged

Do you want to perform this import? yes

Importing (dev)

     Type                   Name                              Status
     pulumi:pulumi:Stack    dev
 =   ├─ gcp:storage:Bucket  second-imported-storage-bucket    imported (1s)
 =   └─ gcp:storage:Bucket  third-imported-storage-bucket     imported (2s)

Resources:
    = 2 imported
    2 unchanged

Duration: 6s

Please copy the following code into your Pulumi application. Not doing so
will cause Pulumi to report that an update will happen on the next update command.

Please note that the imported resources are marked as protected. To destroy them
you will need to remove the `protect` option and run `pulumi update` *before*
the destroy will take effect.

# Code begins here...

Just like when running the command against a single resource, the pulumi import command will import the resources into your stack’s state file and will generate the code snippets for all of the resources in the JSON file. Once again, copy the generated code for the two new resources into your existing program file, save the file, and run the pulumi up command to bring these new resources under the management of Pulumi.

You only need to copy over the resource definitions of the two Storage Buckets and not the import statements again.

Import using code

The third method to import existing cloud resources into a Pulumi project is by defining the resource code yourself and configuring the import resource option in the resource’s definition. This approach may be better suited for scenarios that require importing multiple resources of the same type across multiple stacks and/or deployment environments as part of an automation workflow.

To demonstrate, you will start by creating a simple Service Account in the Google Cloud portal. Once that is complete, you will need to identify the lookup property of the Service Account resource. To do so, navigate to the Import section of the Google Cloud Service Account resource page in the Pulumi documentation. You will notice that the lookup property is the following, where {{project_id}} refers to the ID of your Google Cloud project, and {{email}} refers to the email address of the Service Account:

projects/{{project_id}}/serviceAccounts/{{email}}

Now, navigate to your program code file and update the code with the following resource definition, making sure to replace the values of the service account display name and service account email prefix variables with the values of your own Service Account:

"use strict";
const pulumi = require("@pulumi/pulumi");
const gcp = require("@pulumi/gcp");

const config = new pulumi.Config("gcp");
const projectId = config.require("project");

const serviceAcctEmailSuffix = `@${projectId}.iam.gserviceaccount.com`;
const serviceAcctDisplayName = "pulumi-tutorial-service-account"; // REPLACE
const serviceAcctEmailPrefix = "pulumi-tutorial-service-accoun"; // REPLACE

const importedTutorialServiceAccount = new gcp.serviceaccount.Account(
    "imported-tutorial-service-account",
    {
        accountId: serviceAcctEmailPrefix,
        displayName: serviceAcctDisplayName,
        project: projectId,
    },
    {
        import: `${serviceAcctEmailPrefix}${serviceAcctEmailSuffix}`,
    },
);

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

const config = new pulumi.Config("gcp");
const projectId = config.require("project");

const serviceAcctEmailSuffix = `@${projectId}.iam.gserviceaccount.com`;
const serviceAcctDisplayName = "pulumi-tutorial-service-account"; // REPLACE
const serviceAcctEmailPrefix = "pulumi-tutorial-service-accoun"; // REPLACE

const importedTutorialServiceAccount = new gcp.serviceaccount.Account(
    "imported-tutorial-service-account",
    {
        accountId: serviceAcctEmailPrefix,
        displayName: serviceAcctDisplayName,
        project: projectId,
    },
    {
        import: `${serviceAcctEmailPrefix}${serviceAcctEmailSuffix}`,
    },
);

import pulumi
from pulumi_gcp import serviceaccount, config

project_id = config.project
service_acct_email_suffix = f"@{project_id}.iam.gserviceaccount.com"
service_acct_display_name = "pulumi-tutorial-service-account" # REPLACE
service_acct_email_prefix = "pulumi-tutorial-service-accoun" # REPLACE

imported_tutorial_service_account = serviceaccount.Account("imported-tutorial-service-account",
    account_id=service_acct_email_prefix,
    display_name=service_acct_display_name, 
    project=project_id,
    opts = pulumi.ResourceOptions(
        import_= f"{service_acct_email_prefix}{service_acct_email_suffix}"
    )
)

package main

import (
	"fmt"
    
    "github.com/pulumi/pulumi-gcp/sdk/v6/go/gcp/serviceaccount"
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
    "github.com/pulumi/pulumi/sdk/v3/go/pulumi/config"
)

func main() {
	pulumi.Run(func(ctx *pulumi.Context) error {
        conf := config.New(ctx, "gcp")
        projectId := conf.Require("project")
        
        serviceAcctEmailSuffix := fmt.Sprintf("@%s.iam.gserviceaccount.com", projectId)
        serviceAcctDisplayName := "pulumi-tutorial-service-account" // REPLACE
        serviceAcctEmailPrefix := "pulumi-tutorial-service-accoun" // REPLACE

        _, err := serviceaccount.NewAccount(ctx, "imported-tutorial-service-account", 
            &serviceaccount.AccountArgs{
                AccountId:   pulumi.String(serviceAcctEmailPrefix),
                DisplayName: pulumi.String(serviceAcctDisplayName),
                Project:    pulumi.String(projectId),
            },
            pulumi.Import(pulumi.ID(fmt.Sprintf("%s%s", serviceAcctEmailPrefix, serviceAcctEmailSuffix))),
        )
        if err != nil {
            return err
        }
        
        return nil
    })
}

using Pulumi;
using Pulumi.Gcp.ServiceAccount;

return await Deployment.RunAsync(() =>
{
    var gcpConfig = new Config("gcp");
    var projectId = gcpConfig.Require("project");
    var serviceAcctEmailSuffix = $"@{projectId}.iam.gserviceaccount.com";
    var serviceAcctDisplayName = "pulumi-tutorial-service-account"; // REPLACE
    var serviceAcctEmailPrefix = "pulumi-tutorial-service-accoun"; // REPLACE

    var importedTutorialServiceAccount = new Account("imported-tutorial-service-account", new AccountArgs
    {
        AccountId = serviceAcctEmailPrefix,
        DisplayName = serviceAcctDisplayName,
        Project = projectId
    }, new CustomResourceOptions
    {
        ImportId = $"{serviceAcctEmailPrefix}{serviceAcctEmailSuffix}"
    });
});

name: gcp-import-service-account-yaml
description: An example that deploys the configuration necessary to import a Service Account resource in Google Cloud.
runtime: yaml

variables:
  serviceAcctEmailSuffix: "@${gcp:project}.iam.gserviceaccount.com"
  serviceAcctDisplayName: pulumi-tutorial-service-account # REPLACE
  serviceAcctEmailPrefix: pulumi-tutorial-service-accoun # REPLACE

resources:
  imported-tutorial-service-account:
    type: gcp:serviceaccount:Account
    properties:
      accountId: ${serviceAcctEmailPrefix}
      displayName: ${serviceAcctDisplayName}
      project: ${gcp:project}
    options:
      import: ${serviceAcctEmailPrefix}${serviceAcctEmailSuffix}

As you can see, the lookup property of the Resource Goup resource has been provided as the value of the import option in the resource definition.

At this point, the definition for the imported resources has only been written, meaning it has not yet been imported into your project’s state and is therefore not yet under management by Pulumi. To complete the import process using this method, you will need to save your file and run the pulumi up command. You should see output resembling the following example:

$ pulumi up -y
Previewing update (dev)

     Type                           Name                               Plan
 +   pulumi:pulumi:Stack            dev                                create
 =   └─ gcp:serviceaccount:Account  imported-tutorial-service-account  import

Resources:
    + 1 to create
    = 1 to import
    2 changes

Updating (dev)

     Type                           Name                               Status
 +   pulumi:pulumi:Stack            dev                                created (5s)
 =   └─ gcp:serviceaccount:Account  imported-tutorial-service-account  imported (0.71s)

Resources:
    + 1 created
    = 1 imported
    2 changes

Duration: 6s

It is important to note that when defining resources that you want to import using the import resource option method, the resource definition must match all properties of the existing resource. If you fail to include all of the existing properties, you will run into an error similar to the following:

Previewing update (dev)

     Type                           Name                      Plan       Info
 +   pulumi:pulumi:Stack            dev                       create
 =   └─ gcp:serviceaccount:Account  imported-service-acct     import     [diff: -tags]; 1 warning

Diagnostics:
  gcp:serviceaccount:Account (imported-service-acct):
    warning: inputs to import do not match the existing resource; importing this resource will fail

Resources:
    + 1 to create
    = 1 to import
    2 changes

Updating (dev)

     Type                           Name                     Status                       Info
 +   pulumi:pulumi:Stack            dev                      **creating failed (5s)**     1 error
 =   └─ gcp:serviceaccount:Account  imported-service-acct    **importing failed**         1 error

Diagnostics:
  gcp:serviceaccount:Account (imported-service-acct):
    error: inputs to import do not match the existing resource

  pulumi:pulumi:Stack (dev):
    error: update failed

Resources:
    + 1 created

Duration: 7s

The highlighted line in the preview section of the output indicates which property of the existing resource is missing in the resource definition. You can use this to correct your resource definition before re-deploying. Once a resource is successfully imported, make sure to remove the import option from your code because Pulumi is now managing the resource.

Clean-up

Before moving on, tear down the resources that are part of your stack to avoid incurring any charges.

Run pulumi destroy to tear down all resources. You'll be prompted to make sure you really want to delete these resources. A destroy operation may take some time, since Pulumi waits for the resources to finish shutting down before it considers the destroy operation to be complete.
To delete the stack itself, run pulumi stack rm. Note that this command deletes all deployment history from the Pulumi Service.

With the resources you imported via the CLI command, make sure to set the protect property to false in the code and run pulumi up to make the change before running the pulumi destroy command. Otherwise the deletion will fail.

Next steps

In this tutorial, you imported existing cloud resources via the CLI and updated the program code to include the definition of those imported resources. You also imported existing cloud resources by manually defining the resource definition using the import resource option method.

To learn more about creating and managing resources in Pulumi, take a look a the following resources:

Learn more about importing resources in Pulumi in the Adopting Pulumi -> Import resources documentation.
Learn more about migrating to Pulumi in the Migrating from other solutions to Pulumi documentation.
Learn more about other useful Pulumi CLI commands in the Pulumi CLI overview documentation.

Platform Engineering Workshop Series - Register Now