1. Docs
  2. Pulumi ESC
  3. Get started
  4. Retrieve secrets from external sources

Pulumi ESC: Retrieve Secrets from External Sources

    Overview

    With Pulumi ESC, you can safely store configuration values as plaintext or as secrets directly in the environment file. You also have the option to dynamically import secrets from external sources such as:

    • AWS Secrets Manager
    • Azure Key Vault
    • Google Secret Manager

    This enables you to centralize both new and existing secrets and configurations and use them in multiple places without duplicating values across providers or environments.

    Import external secrets

    If you have not done so already, make sure you have configured OIDC connectivity between Pulumi and a supported provider listed below. During the configuration, you will need to make sure that you add the permissions necessary to interact with secrets in your chosen provider.

    To retrieve secret values from AWS Secrets Manager, you must first:

    Once that is complete, you will need to update your environment definition to add the aws-secrets provider configuration. To do this, add the following configuration to your environment definition, making sure to:

    • nest it under the aws second-level key
    • replace the value of <your-aws-region-here> with your secret’s region
    • replace the value of <your-secret-name-here> with the name of your secret
    values:
      aws: # your existing oidc configuration
        login:
          ...
          ...
        secrets: # add the aws-secrets provider config
          fn::open::aws-secrets:
            region: <your-aws-region-here>
            login: ${aws.login}
            get:
              app-secret:
                secretId: <your-secret-name-here>
    

    Your updated environment file should look similar to the following:

    # example environment definition
    values:
      aws:
        login:
          fn::open::aws-login:
            oidc:
              roleArn: arn:aws:iam::0123456789:role/esc-oidc
              sessionName: pulumi-environments-session
              duration: 1h
        secrets:
          fn::open::aws-secrets:
            region: us-east-1
            login: ${aws.login}
            get:
              app-secret:
                secretId: my-app-secret
    

    You can validate this configuration by opening the environment via the ESC console, clicking the Open button and then clicking the Show secrets slider.

    Alternatively, you can validate the configuration by opening the environment via the ESC CLI. Run the esc env open <your-org>/<your-environment-name> command, making sure to replace the values of <your-org> and <your-environment-name> with the names of your Pulumi organization and ESC environment respectively.

    $ esc env open pulumi/aws-secrets-example
    {
      "aws": {
        "login": {
          "accessKeyId": "ASIA...",
          "secretAccessKey": "Jdt...",
          "sessionToken": "Fwo..."
        },
        "secrets": {
          "app-secret": "pulumi-esc-get-started-secret"
        }
      }
    }
    

    If you need to retrieve multiple Secrets Manager secrets, you can do so as shown below:

    values:
      aws:
        login:
          ...
          ...
        secrets:
          fn::open::aws-secrets:
            region: us-east-1
            login: ${aws.login}
            get:
              app-secret:
                secretId: my-app-secret
              api-key: # additional AWS Secrets Manager secret
                secretId: my-api-key
    

    To retrieve secret values from Azure Key Vault, you must first:

    Once that is complete, you will need to update your environment definition to add the azure-secrets provider configuration. To do this, add the following configuration to your environment definition, making sure to:

    • nest it under the azure second-level key
    • replace the value of <your-vault-name-here> with the name of your vault
    • replace the value of <your-secret-name-here> with the name of your secret
    values:
      azure: # your existing oidc configuration
        login:
          ...
          ...
        secrets: # add the azure-secrets provider config
          fn::open::azure-secrets:
            login: ${azure.login}
            vault: <your-vault-name-here>
            get:
              app-secret:
                name: <your-secret-name-here>
    

    Your updated environment file should look similar to the following:

    # example environment definition
    values:
      azure:
        login:
          fn::open::azure-login:
            clientId: 99b...
            tenantId: 706...
            subscriptionId: 028...
            oidc: true
        secrets:
          fn::open::azure-secrets:
            login: ${azure.login}
            vault: pulumi-esc-vault
            get:
              app-secret:
                name: my-app-secret
    

    You can validate this configuration by opening the environment via the ESC console, clicking the Open button and then clicking the Show secrets slider.

    Alternatively, you can validate the configuration by opening the environment via the ESC CLI. Run the esc env open <your-org>/<your-environment-name> command, making sure to replace the values of <your-org> and <your-environment-name> with the names of your Pulumi organization and ESC environment respectively.

    $ esc env open pulumi/azure-secrets-example
    {
      "azure": {
        "login": {
          "clientId": "99b...",
          "oidc": {
            "token": "eyJ..."
          },
          "subscriptionId": "028...",
          "tenantId": "706..."
        },
        "secrets": {
          "app-secret": "pulumi-esc-get-started-secret"
        }
      }
    }
    

    If you need to retrieve multiple Azure Key Vault secrets, you can do so as shown below:

    values:
      azure:
        login:
          ...
          ...
        secrets:
          fn::open::azure-secrets:
            login: ${azure.login}
            vault: pulumi-esc-vault
            get:
              app-secret:
                name: my-app-secret
              api-key: # additional Azure Key Vault secret
                secretId: my-api-key
    

    To retrieve secret values from Google Secret Manager, you must first:

    Once that is complete, you will need to update your environment definition to add the gcp-secrets provider configuration. To do this, add the following configuration to your environment definition, making sure to:

    • nest it under the gcp second-level key
    • replace the value of <your-secret-name-here> with the name of your secret
    values:
      gcp: # your existing oidc configuration
        login:
          ...
          ...
        secrets: # add the gcp-secrets provider config
          fn::open::gcp-secrets:
            login: ${gcp.login}
            access:
              app-secret:
                name: <your-secret-name-here>
    

    Your updated environment file should look similar to the following:

    # example environment definition
    values:
      gcp:
        login:
          fn::open::gcp-login:
            project: 123456789
            oidc:
              workloadPoolId: pulumi-esc
              providerId: pulumi-esc
              serviceAccount: pulumi-esc@foo-bar-123456.iam.gserviceaccount.com
        secrets:
          fn::open::gcp-secrets:
            login: ${gcp.login}
            access:
              app-secret:
                name: my-app-secret
    

    You can validate this configuration by opening the environment via the ESC console, clicking the Open button and then clicking the Show secrets slider.

    Alternatively, you can validate the configuration by opening the environment via the ESC CLI. Run the esc env open <your-org>/<your-environment-name> command, making sure to replace the values of <your-org> and <your-environment-name> with the names of your Pulumi organization and ESC environment respectively.

    $ esc env open pulumi/gcp-secrets-example
    {
      "gcp": {
        "login": {
          "accessToken": "ya29....",
          "expiry": "2024-01-25T12:52:15Z",
          "project": 123456789,
          "tokenType": "Bearer"
        },
        "secrets": {
          "app-secret": "pulumi-esc-get-started-secret"
        }
      }
    }
    

    If you need to retrieve multiple Google Secret Manager secrets, you can do so as shown below:

    values:
      gcp:
        login:
          ...
          ...
        secrets:
          fn::open::gcp-secrets:
            login: ${gcp.login}
            access:
              api-key: # additional Google Secret Manager secret
                name: api-key
              app-secret:
                name: app-secret
    
      Pulumi AI - What cloud infrastructure would you like to build? Generate Program