Pulumi ESC: Retrieve Secrets from External Sources

Overview

With Pulumi ESC, you can safely store configuration values as plaintext or as secrets directly in the environment file. You also have the option to dynamically import secrets from external sources such as:

AWS Secrets Manager
Azure Key Vault
Google Secret Manager

This enables you to centralize both new and existing secrets and configurations and use them in multiple places without duplicating values across providers or environments.

Import external secrets

If you have not done so already, make sure you have configured OIDC connectivity between Pulumi and a supported provider listed below. During the configuration, you will need to make sure that you add the permissions necessary to interact with secrets in your chosen provider.

To retrieve secret values from AWS Secrets Manager, you must first:

ensure that your IAM role has the appropriate Secrets Manager permissions
create a Secrets Manager secret

Once that is complete, you will need to update your environment definition to add the aws-secrets provider configuration. To do this, add the following configuration to your environment definition, making sure to:

nest it under the aws second-level key
replace the value of <your-aws-region-here> with your secret’s region
replace the value of <your-secret-name-here> with the name of your secret

values:
  aws: # your existing oidc configuration
    login:
      ...
      ...
    secrets: # add the aws-secrets provider config
      fn::open::aws-secrets:
        region: <your-aws-region-here>
        login: ${aws.login}
        get:
          app-secret:
            secretId: <your-secret-name-here>

Your updated environment file should look similar to the following:

# example environment definition
values:
  aws:
    login:
      fn::open::aws-login:
        oidc:
          roleArn: arn:aws:iam::0123456789:role/esc-oidc
          sessionName: pulumi-environments-session
          duration: 1h
    secrets:
      fn::open::aws-secrets:
        region: us-east-1
        login: ${aws.login}
        get:
          app-secret:
            secretId: my-app-secret

You can validate this configuration by opening the environment via the ESC console, clicking the Open button and then clicking the Show secrets slider.

Alternatively, you can validate the configuration by opening the environment via the ESC CLI. Run the esc env open <your-org>/<your-environment-name> command, making sure to replace the values of <your-org> and <your-environment-name> with the names of your Pulumi organization and ESC environment respectively.

$ esc env open pulumi/aws-secrets-example
{
  "aws": {
    "login": {
      "accessKeyId": "ASIA...",
      "secretAccessKey": "Jdt...",
      "sessionToken": "Fwo..."
    },
    "secrets": {
      "app-secret": "pulumi-esc-get-started-secret"
    }
  }
}

If you need to retrieve multiple Secrets Manager secrets, you can do so as shown below:

values:
  aws:
    login:
      ...
      ...
    secrets:
      fn::open::aws-secrets:
        region: us-east-1
        login: ${aws.login}
        get:
          app-secret:
            secretId: my-app-secret
          api-key: # additional AWS Secrets Manager secret
            secretId: my-api-key

To retrieve secret values from Azure Key Vault, you must first:

create an Azure Key Vault
add a Key Vault role assignment
- On your Key Vault’s service page, follow steps 3 - 8 in the Azure OIDC guide, making sure to select the “Key Vault Secrets Officer” role under the Job functions role tab
create an Azure Key Vault secret

Once that is complete, you will need to update your environment definition to add the azure-secrets provider configuration. To do this, add the following configuration to your environment definition, making sure to:

nest it under the azure second-level key
replace the value of <your-vault-name-here> with the name of your vault
replace the value of <your-secret-name-here> with the name of your secret

values:
  azure: # your existing oidc configuration
    login:
      ...
      ...
    secrets: # add the azure-secrets provider config
      fn::open::azure-secrets:
        login: ${azure.login}
        vault: <your-vault-name-here>
        get:
          app-secret:
            name: <your-secret-name-here>

Your updated environment file should look similar to the following:

# example environment definition
values:
  azure:
    login:
      fn::open::azure-login:
        clientId: 99b...
        tenantId: 706...
        subscriptionId: 028...
        oidc: true
    secrets:
      fn::open::azure-secrets:
        login: ${azure.login}
        vault: pulumi-esc-vault
        get:
          app-secret:
            name: my-app-secret

You can validate this configuration by opening the environment via the ESC console, clicking the Open button and then clicking the Show secrets slider.

$ esc env open pulumi/azure-secrets-example
{
  "azure": {
    "login": {
      "clientId": "99b...",
      "oidc": {
        "token": "eyJ..."
      },
      "subscriptionId": "028...",
      "tenantId": "706..."
    },
    "secrets": {
      "app-secret": "pulumi-esc-get-started-secret"
    }
  }
}

If you need to retrieve multiple Azure Key Vault secrets, you can do so as shown below:

values:
  azure:
    login:
      ...
      ...
    secrets:
      fn::open::azure-secrets:
        login: ${azure.login}
        vault: pulumi-esc-vault
        get:
          app-secret:
            name: my-app-secret
          api-key: # additional Azure Key Vault secret
            secretId: my-api-key

To retrieve secret values from Google Secret Manager, you must first:

ensure that your service account has the appropriate Secret Manager permissions
create a Google secret

Once that is complete, you will need to update your environment definition to add the gcp-secrets provider configuration. To do this, add the following configuration to your environment definition, making sure to:

nest it under the gcp second-level key
replace the value of <your-secret-name-here> with the name of your secret

values:
  gcp: # your existing oidc configuration
    login:
      ...
      ...
    secrets: # add the gcp-secrets provider config
      fn::open::gcp-secrets:
        login: ${gcp.login}
        access:
          app-secret:
            name: <your-secret-name-here>

Your updated environment file should look similar to the following:

# example environment definition
values:
  gcp:
    login:
      fn::open::gcp-login:
        project: 123456789
        oidc:
          workloadPoolId: pulumi-esc
          providerId: pulumi-esc
          serviceAccount: pulumi-esc@foo-bar-123456.iam.gserviceaccount.com
    secrets:
      fn::open::gcp-secrets:
        login: ${gcp.login}
        access:
          app-secret:
            name: my-app-secret

You can validate this configuration by opening the environment via the ESC console, clicking the Open button and then clicking the Show secrets slider.

$ esc env open pulumi/gcp-secrets-example
{
  "gcp": {
    "login": {
      "accessToken": "ya29....",
      "expiry": "2024-01-25T12:52:15Z",
      "project": 123456789,
      "tokenType": "Bearer"
    },
    "secrets": {
      "app-secret": "pulumi-esc-get-started-secret"
    }
  }
}

If you need to retrieve multiple Google Secret Manager secrets, you can do so as shown below:

values:
  gcp:
    login:
      ...
      ...
    secrets:
      fn::open::gcp-secrets:
        login: ${gcp.login}
        access:
          api-key: # additional Google Secret Manager secret
            name: api-key
          app-secret:
            name: app-secret

← Previous Step Integrate with Pulumi IaC →

Pulumi ESC: Retrieve Secrets from External Sources

On this page

On this page

Overview

Import external secrets

On this page

On this page