1. Docs
  2. Pulumi ESC
  3. Environments

Pulumi ESC Environments

    Pulumi ESC (Environments, Secrets, and Configuration) lets you define collections of configuration settings and secrets called environments and use them in any application or service. Environments are YAML documents composed of static key-value pairs, programmatic expressions, dynamically retrieved values from supported providers including all major clouds through OpenID Connect (OIDC), and other Pulumi ESC environments.

    Environments are accessible with the standalone esc CLI, the pulumi CLI, the Pulumi SDK, and the Pulumi Cloud console and REST API, and you can have as many environments as you need. Pulumi ESC is a service of Pulumi Cloud and is currently in public preview.

    The examples below use the new esc CLI, but all esc subcommands are available on the pulumi CLI as well. The pulumi CLI also provides fully integrated support for configuring Pulumi stacks with Pulumi ESC. See Using with Pulumi IaC below for details.

    Creating a new environment

    To create a new, empty environment, use esc env init [<org-name>/]<environment-name>, where <org-name> is optional and defaults to your Pulumi Cloud username.

    Environment names must be unique within an organization and may only contain alphanumeric characters, hyphens, underscores, and periods.

    $ esc env init myorg/test
    Environment created.
    

    Listing environments

    To list the environments you have access to, use esc env ls:

    $ esc env ls
    myorg/test
    

    You can filter this list to a particular organization by passing its name:

    $ esc env ls --organization myorg
    myorg/test
    

    Getting and setting environment values

    Setting a value

    To set a new value or update an existing value, use esc env set <key> <value>:

    $ esc env set myorg/test foo bar
    

    Getting a value

    To retrieve a single value and its definition, use esc env get <environment-name> <key>:

    $ esc env get myorg/test foo
    
       Value
    
        "bar"
    
       Definition
    
        bar
    
    
       Defined at
    
      • test:2:10
    

    Why is the value shown as [unknown]?

    The command esc env get returns statically defined plain-text values and definitions. It does not return secrets defined in the environment nor resolves values from specified provider configuration, therefore, attempting to get a secret results in an [unknown] output.

    Use esc env open to access secrets; this opens the environment and resolves dynamically retrieved values or secrets.

    Getting all values

    To retrieve all values in an environment, run esc env get <environment-name>:

    $ esc env get myorg/test
    
       Value
    
        {
          "foo": "bar"
        }
    
       Definition
    
        values:
          foo: bar
    

    You can also get values in other formats, such as json, dotenv, or shell:

    $ esc env get myorg/test --value json
    {
      "foo": "bar"
    }
    
    $ esc env get myorg/test foo --value json
    "bar"
    

    Note that the dotenv and shell options return values only when an environment defines one or more environment variables. See Projecting environment variables for details.

    Structured configuration

    Structured data like maps and arrays can also be set with esc env set <environment-name> <key> <value> using object-property (.) or array ([]) syntax:

    $ esc env set myorg/test 'data.active' true
    $ esc env set myorg/test 'data.nums[0]' 1
    $ esc env set myorg/test 'data.nums[1]' 2
    $ esc env get myorg/test
    
      Value
    
      {
        "data": {
          "active": true,
          "nums": [
            1,
            2
          ]
        },
        "foo": "bar"
      }
    
      Definition
    
      values:
        foo: bar
        data:
          active: true
          nums:
            - 1
            - 2
    

    Boolean and numeric values are implicitly converted and persisted in their respective types.

    You can also fetch individual values at any level in the hierarchy. For example, to fetch the second item from the nested array in the example above:

    $ esc env get myorg/test 'data.nums[1]'
    
       Value
    
        2
    
       Definition
    
        2
    
    
       Defined at
    
      • test:7:15
    

    Interpolating values

    Values don’t have to be static. You can also use string interpolation to transform values or compose new values from other values. For example:

    $ esc env set myorg/test salutation Hello
    $ esc env set myorg/test name World
    $ esc env set myorg/test greeting '${salutation}, ${name}'
    $ esc env get myorg/test greeting
    
       Value
    
        "Hello, World"
    
       Definition
    
        ${salutation}, ${name}
    
    
       Defined at
    
      • test:10:15
    

    Pulumi Contextual information

    Contextual information is available and can be used to interpolate on any environment value.

    values:
      personalNamespace: ${context.rootEnvironment.name}/${context.pulumi.user.login}
    

    It can be accessed through the context attribute with the following options:

    • context.rootEnvironment.name: the name of the root environment being evaluated
    • context.currentEnvironment.name: the name of the current environment being evaluated
    • context.user.login: the user login identifier
    • context.organization.login: the organization login identifier

    Editing environments

    Environments may be edited in a number of ways.

    With the Pulumi ESC CLI

    To edit an environment using your shell’s default text editor (as defined by the $EDITOR environment variable), use esc env edit <environment-name>:

    $ esc env edit myorg/test
    

    When $EDITOR is unset, esc uses vi as a fallback:

    values:
        foo: bar
        data:
            active: true
            nums:
                - 1
                - 2
        salutation: Hello
        name: World
        greeting: ${salutation}, ${name}
    ---
    # Please edit the environment definition above.
    # The object below is the current result of
    # evaluating the environment and will not be
    # saved. An empty definition aborts the edit.
    
    {
      "data": {
        "active": true,
        "nums": [
          1,
          2
        ]
      },
      "foo": "bar",
      "greeting": "Hello, World",
      "name": "World",
      "salutation": "Hello"
    }
    ~
    ~
    ~
    "/tmp/2198639483.yaml" 29L, 528B
    

    You can also specify the editor you want to use by passing its name with the --editor flag:

    $ esc env edit --editor="code" myorg/test
    

    Doing so overrides any value set in $EDITOR. See esc env edit --help for more options.

    In the Pulumi Cloud console

    To edit an environment in the Pulumi Cloud console, select your organization, choose Environments in the left-hand menu, and select the environment you wish to edit:

    Environments Intelligent YAML Editor

    Click Save to apply your changes immediately or Delete to remove the environment.

    With the Pulumi Cloud REST API

    You can also use the Pulumi Cloud REST API to perform standard CRUD operations on your environments. See the Environments section of the REST API docs for details.

    Using secrets providers and OIDC

    In addition to static and interpolated values, environments can incorporate dynamically retrieved settings and secrets from many supported providers, including cloud providers via OpenID Connect (OIDC).

    The following example combines the aws-login and aws-secrets providers to obtain short-lived credentials from AWS to pull two secrets from AWS Secrets Manager (api-key and app-secret) into an environment:

    values:
      aws:
        login:
          fn::open::aws-login:
            oidc:
              roleArn: arn:aws:iam::01234567891011:role/some-role
              sessionName: some-session
    
        secrets:
          fn::open::aws-secrets:
            region: us-west-1
            login: ${aws.login}
            get:
              api-key:
                secretId: api-key
              app-secret:
                secretId: app-secret
    

    Opening an environment

    As mentioned, esc env get doesn’t resolve dynamic or secret values — for example, those managed with third-party services like AWS Secrets Manager. Instead, to gain access to these values, you must open the environment.

    To do so, use esc env open <environment-name>:

    $ esc env open myorg/test
    {
      "data": {
        "active": true,
        "nums": [
          1,
          2
        ]
      },
      "foo": "bar",
      "greeting": "Hello, World",
      "name": "World",
      "salutation": "Hello",
      "aws": {
        "login": {
          "accessKeyId": "AKIAIOSFODNN7EXAMPLE",
          "secretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLE",
          "sessionToken": "eyJpc3MiOiJPbmxpbmUgSldUIEJ1aWxkZXIiLCJpYXQiOEXAMPLE"
        },
        "secrets": {
          "api-key": "{\"keyName\":\"keyValue\"}",
          "app-secret": "{\"secretName\":\"secretValue\"}"
        }
      }
    }
    

    You can also use open to resolve the value of an individual key:

    $ esc env open myorg/test 'data.active'
    true
    
    $ esc env open myorg/test 'data.nums'
    [
      1,
      2
    ]
    
    $ esc env open myorg/test 'aws.secrets.app-secret' | jq -r 'fromjson | . .secretName'
    secretValue
    

    Projecting environment variables

    Pulumi ESC can automatically project the settings of a given environment as a set of environment variables. This projection does not happen by default, however; instead, you must define which settings to project, as well as how to name and format them.

    Environment variables are defined under the optional environmentVariables key, which can accept either static or interpolated values based on settings defined within the environment or imported from other environments:

    values:
      aws:
        login:
          fn::open::aws-login:
            oidc:
              roleArn: arn:aws:iam::01234567891011:role/some-role
              sessionName: some-session
    
      environmentVariables:
        AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
        AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
        AWS_SESSION_TOKEN: ${aws.login.sessionToken}
        MY_ENV_VAR: "true"
    

    To render these environmentVariables for use in the shell, use esc env open, passing either --format shell or --format dotenv:

    $ esc env open myorg/test --format shell
    export AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
    export AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLE"
    export AWS_SESSION_TOKEN="eyJpc3MiOiJPbmxpbmUgSldUIEJ1aWxkZXIiLCJpYXQiOEXAMPLE"
    export MY_ENV_VAR="true"
    
    $ esc env open myorg/test --format dotenv
    AWS_ACCESS_KEY_ID="AKIAIOSFODNN7EXAMPLE"
    AWS_SECRET_ACCESS_KEY="wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLE"
    AWS_SESSION_TOKEN="eyJpc3MiOiJPbmxpbmUgSldUIEJ1aWxkZXIiLCJpYXQiOEXAMPLE"
    MY_ENV_VAR="true"
    

    Export them into the current shell using eval or similar:

    $ eval $(esc env open myorg/test --format shell)
    $ echo $MY_ENV_VAR
    true
    

    Running commands with environment variables

    You can also run CLI commands directly, using environment variables obtained with Pulumi ESC — without having to export those variables into the shell first.

    To do this, use esc run <environment-name> <command>:

    $ esc run myorg/test aws s3 ls
    2023-10-10 16:09:19 my-s3-bucket
    

    If you need to pass one or more flags to the command, prefix the command with --:

    $ esc run myorg/test -- aws s3 ls s3://my-s3-bucket --recursive --summarize
    ...
    Total Objects: 5087
       Total Size: 2419123156
    

    For additional options and details, see esc run --help.

    Importing other environments

    Environments can also be composed from other environments.

    Different applications are often configured in similar ways and with common values — for example, an e-commerce site and order-management system both configured to use the same cloud account, database-connection string, and third-party API key. Managing the duplication of these values across multiple configuration files, however, can be difficult, especially when one of those values changes — e.g., when an API key is regenerated.

    To address these challenges, Pulumi ESC allows you to identify common or closely related configuration settings and define them only once, as individual environments, and then import those environments into other, more specialized environments as needed. Imports also allow you to expose certain environments without having to distribute any concrete values and to delegate responsibility for particular environments to other teams in your organization. Environments can import both static and dynamic values, including secrets, from any number of other environments.

    In the following example, two environments, aws-dev and stripe-dev, are used to compose a third environment, myapp-dev:

    # myorg/aws-dev
    values:
      aws:
        region: us-west-2
        login:
          fn::open::aws-login:
            static:
              accessKeyId:
                fn::secret: AKIAIOSFODNN7EXAMPLE
              secretAccessKey:
                fn::secret: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLE
    
    # myorg/stripe-dev
    values:
      stripe:
        apiURL: https://api.stripe.com
        apiKey:
          fn::secret: sk_XemWAl12i4x3hZhp4vBKDEXAMPLE
    

    The application-specific myapp-dev environment then imports these two environments and use their settings to compose new values:

    # myorg/myapp-dev
    imports:
      - aws-dev
      - stripe-dev
    
    values:
      greeting: Hello from the dev environment!
    
      environmentVariables:
        AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
        AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
        STRIPE_API_KEY: ${stripe.apiKey}
        STRIPE_API_URL: ${stripe.apiURL}
        GREETING: ${greeting}
    

    Finally, esc run renders myapp-dev’s environment variables for use on the command line:

    $ esc run myorg/myapp-dev -- bash -c 'echo $GREETING'
    Hello from the dev environment!
    
    $ esc run myorg/myapp-dev -- bash -c 'echo $STRIPE_API_URL'
    https://api.stripe.com
    
    $ esc run myorg/myapp-dev -- bash -c 'echo $STRIPE_API_KEY'
    [secret]
    
    $ esc run myorg/myapp-dev -- bash -c 'echo $AWS_SECRET_ACCESS_KEY'
    [secret]
    
    $ echo "'$GREETING'"
    ''
    

    Notice in the example that the environmentVariables were exposed to the bash command, but not to the surrounding shell, and that the values marked as secrets with fn::secret were protected from exposure.

    Using environments with Pulumi IaC

    With support for Pulumi ESC built into the Pulumi CLI, you can expose an environment’s settings and secrets to any or all of your Pulumi stacks, bypassing the need to define and maintain individual configuration settings or secrets “locally” in Pulumi config files. The optional pulumiConfig key enables this.

    The following example updates the myorg/myapp-dev environment by adding a pulumiConfig block. This block specifies the Pulumi configuration settings to expose to the Pulumi stack at runtime:

    # myorg/myapp-dev
    imports:
      - aws-dev
      - stripe-dev
    
    values:
      greeting: Hello from the dev environment!
    
      environmentVariables:
        AWS_ACCESS_KEY_ID: ${aws.login.accessKeyId}
        AWS_SECRET_ACCESS_KEY: ${aws.login.secretAccessKey}
        STRIPE_API_KEY: ${stripe.apiKey}
        STRIPE_API_URL: ${stripe.apiURL}
        GREETING: ${greeting}
    
      # Add a `pulumiConfig` block to expose these settings to your Pulumi stacks.
      pulumiConfig:
        aws:region: ${aws.region}
        stripeApiKey: ${stripe.apiKey}
        stripeApiURL: ${stripe.apiURL}
        greeting: ${greeting}
    

    Any stack belonging to the myorg organization can inherit these settings by adding the optional environment block to its stack-configuration file:

    # Pulumi.dev.yaml
    environment:
      - myapp-dev
    

    Values are accessible using the standard configuration API:

    // index.ts
    import * as pulumi from "@pulumi/pulumi";
    import * as aws from "@pulumi/aws";
    
    // Import the values using the standard Pulumi configuration API.
    const config = new pulumi.Config();
    const greeting = config.require("greeting");
    const stripeApiKey = config.requireSecret("stripeApiKey");
    const stripeApiURL= config.requireSecret("stripeApiURL");
    
    const callbackFunction = new aws.lambda.CallbackFunction("callback", {
        callback: async () => ({
            statusCode: 200,
            body: JSON.stringify({
                greeting,
    
                // Use them in your program as would any config value.
                stripeApiURL: process.env.STRIPE_API_URL,
             }),
        }),.
        environment: {
            variables: {
                STRIPE_API_URL: stripeApiURL,
            },
        },
    });
    
    const functionUrl = new aws.lambda.FunctionUrl("url", {
        functionName: callbackFunction.name,
        authorizationType: "NONE",
    });
    
    export const url = functionUrl.functionUrl;
    

    Stacks may only read from environments that belong to the same Pulumi organization.

    Precedence rules

    When multiple environment sources are combined and settings overlap, values are applied successively in the order in which they’re imported and defined.

    For example, in the following scenario, three environments define a key foo, each with a different value. The third environment, environment-c, imports environment-a and environment-b (importantly, in that order):

    # environment-a
    values:
      foo: bar
    
    # environment-b
    imports:
      - environment-a
    values:
      foo: baz
      pulumiConfig:
        foo: ${foo}
    
    # environment-c
    imports:
      - environment-a
      - environment-b
    values:
      foo: qux
      pulumiConfig:
        foo: ${foo}
    

    Notice how the value of foo is overwritten with each successive environment:

    $ esc env open environment-a foo
    "bar"
    
    $ esc env open environment-b foo
    "baz"
    
    $ esc env open environment-c foo
    "qux"
    
    $ pulumi preview
    Diagnostics:
      pulumi:pulumi:Stack (dev):
        { foo: 'qux' }
    

    Also notice that when the local definition of foo is removed from environment-c and its imports are reordered, the value of foo changes to reflect the value inherited from environment-a — i.e., the last-imported one:

    # environment-c
    imports:
      - environment-b
      - environment-a
    values:
      pulumiConfig:
        foo: ${foo}
    
    $ esc env open environment-c foo
    "bar"
    
    $ pulumi preview
    Diagnostics:
      pulumi:pulumi:Stack (dev):
        { foo: 'bar' }
    

    To unset a value inherited from another environment, overwrite it with null.

    Deleting an environment

    To remove an environment, use esc env rm [<org-name>/]<environment-name>:

    $ esc env rm myorg/test
    This will permanently remove the "myorg/test" environment!
    Please confirm that this is what you'd like to do by typing `myorg/test`: myorg/test
    Environment "myorg/test" has been removed!
    

    Environments cannot be renamed.

      Pulumi AI - What cloud infrastructure would you like to build? Generate Program