--- title: State & backends url: /docs/iac/concepts/state-and-backends/ --- Pulumi stores metadata about your infrastructure so that it can manage your cloud resources. This metadata is called _state_. Each [stack](/docs/concepts/stack/) has its own state, and state is how Pulumi knows when and how to create, read, delete, or update cloud resources. Pulumi stores state in a _backend_ of your choosing. A backend is an API and storage endpoint used by the CLI to coordinate updates, and read and write stack state whenever appropriate. Backend options include Pulumi Cloud, an easy-to-use, secure, and reliable hosted application with policies and safeguards to facilitate team collaboration, in addition to simple object storage in AWS S3, Microsoft Azure Blob Storage, Google Cloud Storage, any AWS S3 compatible server such as Minio or Ceph, or a local filesystem. The default experience is to use the hosted Pulumi Cloud, which takes care of the state and backend details for you. Conversely, when using cloud storage or a local filesystem as your backend, you gain control over where your state is located at the expense of having to handle security, state management, auditing, and other concerns Pulumi Cloud would otherwise handle for you. > Pulumi state does not include your cloud credentials. Credentials are kept local to your client — wherever the CLI runs — even when using the managed Pulumi Cloud backend. Pulumi _does_ store configuration and secrets, but encrypts those secrets using your chosen encryption provider. To learn more, see [Configuration and Secrets](/docs/concepts/secrets/). > **Note:** This page covers the technical details of state management and backend configuration. To understand the benefits and features of Pulumi Cloud versus DIY backends, see [Pulumi Cloud and Open Source Pulumi](/docs/iac/concepts/pulumi-cloud/). ## Deciding On a State Backend Pulumi supports two classes of state backends for storing your infrastructure state: - **Pulumi Cloud**: a managed cloud experience using the online or self-hosted Pulumi Cloud application - **DIY backend**: "Do it Yourself"- a manually managed object store, including AWS S3, Azure Blob Storage, Google Cloud Storage, any AWS S3 compatible server such as Minio or Ceph, a PostgreSQL database, or your local filesystem Pulumi's SDK works great with all backends, although some details differ between them. Pulumi Cloud, hosted at [`app.pulumi.com`](https://app.pulumi.com/signin), is the default backend, as it provides the best combination of usability, safety, and security for most users. Important features include: - Robust state management, with transactional checkpointing for fault tolerance and recovery - Concurrent state locking to prevent corrupting your infrastructure state in a team environment - Full deployment history for auditing and rollback purposes - Encrypted state in transit and at rest - Managed encryption and key management for secrets - Secure access to cloud resource metadata, with client-side authentication to your cloud provider - Team policies, including Policy as Code and Role Based Access Control (RBAC) The Pulumi Cloud backend requires no additional configuration after [installing the CLI](/docs/install/). Pulumi offers this backend hosted online free for individuals, with [advanced tiers](/pricing/) available for teams and enterprises (with [free trials](https://app.pulumi.com/site/trial)). It has successfully undergone multiple security audits including SOC2, pen-testing, and more. > To learn more about the Pulumi Cloud backend's design, including why it doesn't need your cloud credentials, see [Pulumi Cloud Architecture](#pulumi-cloud-architecture). If you are interested in hosting your own instance, see the [Self-Hosting User Guide](/docs/pulumi-cloud/self-hosted/). Pulumi also lets you manage state yourself using a DIY backend. Your state is stored in AWS S3, Azure Blob Storage, Google Cloud Storage, an S3-compatible server such as Minio or Ceph, a PostgreSQL database, or on your local filesystem. These DIY backends are all open source and free to use in any setting. DIY backends include several important built-in features: - **State locking**: Enabled by default to prevent concurrent state modifications - **History tracking**: Automatic checkpoint history for each stack - **Project-scoped stacks**: Stacks are [namespaced by project](#scoping) for backends created with Pulumi v3.61.0+ - **Secrets encryption**: Support for multiple encryption providers including passphrase, AWS KMS, Azure Key Vault, Google Cloud KMS, and HashiCorp Vault However, using a DIY backend requires you to manage operational aspects yourself, including: - Configuring secure access to your storage backend (IAM policies, network security) - Implementing backup and disaster recovery procedures - Monitoring and maintaining high availability - Managing team access and permissions To choose a DIY backend, use the `pulumi login` command [as documented below](#using-a-diy-backend). ## Logging into and out of State Backends The [`login` command](/docs/iac/cli/commands/pulumi_login) logs you into a backend: ```sh $ pulumi login ``` The [`logout` command](/docs/iac/cli/commands/pulumi_logout) logs you out of the current backend. ```sh $ pulumi logout ``` This will remove the credentials for the current backend from `~/.pulumi/credentials.json` and you will need to log in again before performing any subsequent stack or state operations. To remove credentials for all backends at once, use `pulumi logout --all`. To change backends, run `pulumi logout` followed by `pulumi login`. The basic form of `login` will use Pulumi Cloud by default. If you wish to log in to a specific backend, pass the backend-specific URL as the sole argument: ```sh $ pulumi login ``` Alternatively, there are 2 other options that help to avoid the need to type it every time: 1. Set the `PULUMI_BACKEND_URL` environment variable. 2. Set `backend` property in the project `Pulumi.yaml` config file as below: ```yaml .... backend: url: .... ``` For details on the various backend URL formats and options, please see the sections on using Pulumi Cloud and DIY backends. If you forget to log in, you will be automatically prompted to do so before you do anything that requires stacks or state. After logging in, your credentials are recorded in the `~/.pulumi/credentials.json` file, and all subsequent operations will use the chosen backend. From time to time, you will see a helpful URL to your update or stack pages. For example, after an update completes, you will see a link to that update's details. You can always go there to see a full history of updates. If you ever want to check what user is logged in, use the [`whoami` command](/docs/iac/cli/commands/pulumi_whoami). To additionally see what backend is currently being used, pass the `--verbose` (or `-v`) flag: ```bash $ pulumi whoami -v User: Backend URL: https://app.pulumi.com/ ``` ## Pulumi Cloud Backend Running `pulumi login` without any argument will log into the default Pulumi Cloud backend: ```sh $ pulumi login ``` This will display a prompt that asks for an [access token](/docs/pulumi-cloud/accounts#access-tokens): ``` Manage your Pulumi stacks by logging in. Run `pulumi login --help` for alternative login options. Enter your access token from https://app.pulumi.com/account/tokens or hit to log in using your browser: ``` To automatically generate and use a new access token, hit ``. This will open a web browser to interact with Pulumi Cloud and request a token. If this is your first time using Pulumi Cloud, you will be asked to authenticate using your chosen identity provider (GitHub, GitLab, Atlassian, SAML/SSO, or email). To view your access tokens, or create a new one manually, view the [Access Tokens](https://app.pulumi.com/account/tokens) page. You will see a list of past tokens, when they were last used, as well as the ability to revoke them. ![Pulumi.com Tokens Page](/images/docs/reference/state_tokens.png) To log into a self-hosted instance of Pulumi Cloud, pass its API URL to the `login` command: ```sh $ pulumi login https://pulumi.acmecorp.com ``` Everything works the same as with the standard Pulumi Cloud, except that Pulumi will target your private instance instead of the shared one hosted at `app.pulumi.com`. ### Pulumi Cloud Architecture Pulumi Cloud is comprised of two Internet-accessible endpoints—a web application at `app.pulumi.com` and a REST API at `api.pulumi.com`—with an assortment of cloud infrastructure to support its features. A simplified diagram of its architecture looks like this: ![Pulumi Cloud Architecture](/images/docs/reference/state_saas.png) Pulumi Cloud doesn't ever acquire your cloud credentials, and does not communicate with your cloud provider directly. Instead, the CLI itself coordinates with both Pulumi Cloud's API and your cloud provider's API directly. This ensures your IAM and key management does not need to change while adopting Pulumi. In particular, if you are running Pulumi deployments from [within a CI/CD environment](/docs/using-pulumi/continuous-delivery/), you can rely on existing mechanisms and security practices that your organization has already put in place. Pulumi Cloud is reliable, secure, and has undergone multiple audits, including SOC2 and professional pen-testing. Because of the client/server division of responsibilities — notably that the server doesn't have direct access to your cloud credentials, runtime data, or PII — Pulumi Cloud has been used in organizations with advanced compliance needs, including PCI, ISO 27001, HIPAA, and more. If you'd like to discuss any of these topics, please [contact us](/contact/). It is possible to host your own version of Pulumi Cloud in your private cloud environment. Pulumi offers versions that run natively on AWS, Azure, Google Cloud, Kubernetes, or simple virtual machine-based private and hybrid cloud environments. The architecture is very similar to the online version, but is privately hosted and does not depend on public access over the Internet: ![Pulumi Enterprise Architecture](/images/docs/reference/state_enterprise.png) To learn more about self-host options, see [Self-Hosted Pulumi Cloud](/docs/pulumi-cloud/self-hosted/) or [Contact Us](/pricing#contact). ## Using a DIY Backend The filesystem and cloud storage backends allow you to store state locally on your machine or remotely within a cloud object store. For DIY backends, state management including backup, sharing, and team access synchronization is custom and implemented manually. A basic file-based locking system is enabled by default for all DIY backends. > **Note**: Both Pulumi Cloud and DIY backends provide reliable state management. DIY backends include built-in state locking and history tracking. However, most users find Pulumi Cloud to be the easiest way to get started and scale. Pulumi Cloud handles operational concerns automatically, including backup and recovery, team collaboration, RBAC, and audit logging. It also provides a transactional API that offers stronger guarantees than blob storage protocols. With DIY backends, you will need to implement your own backup procedures and manage access control. For advanced state management concepts, see [Advanced State](#advanced-state). To use a DIY backend, specify a storage endpoint URL as `pulumi login`'s `` argument: `s3://`, `azblob://`, `gs://`, or `file://`. This will tell Pulumi to store state in AWS S3, Azure Blob Storage, Google Cloud Storage, or the local filesystem, respectively. Checkpoint files are stored in a relative`.pulumi` directory. For example, if you were using the Amazon S3 DIY backend, your checkpoint files would be stored at `s3://my-pulumi-state-bucket/.pulumi` where `my-pulumi-state-bucket` represents the name of your S3 bucket. Inside the `.pulumi` folder, we access the following subdirectories: 1. `meta.yaml`: This is the metadata file. It does not hold information about the stacks but rather information about the backend itself. 1. `stacks/`: Active state files for each stack (e.g. `dev.json` or `proj/dev.json` if the stack is scoped to a project). 1. `locks/`: Lock files for each stack if the stack is currently being operated on by a Pulumi operation (e.g. `dev/$lock.json` or `proj/dev/$lock.json` where `$lock` is a unique identifier for the lock). 1. `history/`: History for each stack (e.g. `dev/dev-$timestamp.history.json` or `proj/dev/dev-$timestamp.history.json` where `$timestamp` records the time the history file was created). The detailed format of the `` differs by backend and each has different options such as how to authenticate, as described below. ### Local Filesystem To use the filesystem backend to store your state files locally on your machine, pass the `--local` flag when logging in: ```sh $ pulumi login --local ``` You will see `Logged into as (file://~)` as a result where `` and `` are your configured machine and user names, respectively. All subsequent stack state will be stored as JSON files locally on your machine. The default directory for these JSON files is `~/.pulumi`. To store state files in an alternative location, specify a `file://` URL instead, where `` is the full path to the target directory where state files will be stored. For instance, to store state underneath `/app/data/.pulumi/` instead, run: ```sh $ pulumi login file:///app/data ``` > **Note:** If you use a relative path (e.g. `file://./einstein`), it will be relative to _the current working directory_. Notice that `pulumi login --local` is syntactic sugar for `pulumi login file://~`. ### AWS S3 To use the [AWS S3](https://aws.amazon.com/s3/) backend, pass the `s3://` as your ``: ```sh $ pulumi login s3:// ``` > **Note:** As of Pulumi CLI v3.33.1, instead of specifying the AWS Profile, add `awssdk=v2` along with the region and profile to the query string. The URL should be quoted to escape the shell operator `&`, and used as follows: ```sh pulumi login 's3://?region=us-east-1&awssdk=v2&profile=' ``` > **Note:** The `bucket-name` value can include multiple folders, such as `my-bucket/app/project1`. This is useful when storing multiple projects' state in the same bucket. To configure credentials and authorize access, please see the [AWS Session documentation](https://docs.aws.amazon.com/sdk-for-go/api/aws/session/). For additional configuration options, see [AWS Setup](/registry/packages/aws/installation-configuration/). If you're new to AWS S3, see [the AWS documentation](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html). This backend also supports [alternative object storage servers with AWS S3 compatible REST APIs](https://en.wikipedia.org/wiki/Amazon_S3#S3_API_and_competing_services), including [Minio](https://www.minio.io/), [Ceph](https://ceph.io/), or [SeaweedFS](https://github.com/chrislusf/seaweedfs). To use such a server, you may pass `endpoint`, `disableSSL`, and `s3ForcePathStyle` querystring parameters to your ``, as follows: ```sh $ pulumi login 's3://?endpoint=my.minio.local:8080&disableSSL=true&s3ForcePathStyle=true' ``` ### Azure Blob Storage To use the [Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/) backend, pass the `azblob://` as your ``: ```sh $ pulumi login azblob:// ``` Set the `AZURE_STORAGE_ACCOUNT` environment variable to specify which Azure storage account to use. For authentication, you may set `AZURE_STORAGE_KEY` (a storage account access key) or `AZURE_STORAGE_SAS_TOKEN` (a shared access signature token). If neither is provided, the backend authenticates using [Azure SDK for Go's `DefaultAzureCredential`](https://learn.microsoft.com/en-us/azure/developer/go/azure-sdk-authentication), which attempts a series of methods in order, including managed identity, workload identity federation, service principal credentials (`AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`), and the Azure CLI. If you're new to Azure Blob Storage, see [the Azure documentation](https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-cli). > **Note:** This backend authenticates using [Azure SDK for Go](https://learn.microsoft.com/en-us/azure/developer/go/azure-sdk-authentication), not the Pulumi Azure provider's authentication mechanism. The Azure provider's environment variables — such as `ARM_TENANT_ID`, `ARM_CLIENT_ID`, and `ARM_USE_OIDC` — are not supported. Use the Azure SDK's own environment variables (`AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`) for service principal authentication. > **Note:** As of Pulumi CLI v3.41.1, you can also specify the storage account directly in the backend URL after authenticating with `az login`: ```sh $ pulumi login azblob://?storage_account=account_name ``` > **Note:** The Azure account must have the [Storage Blob Data Contributor role](https://learn.microsoft.com/en-us/azure/role-based-access-control/built-in-roles#storage-blob-data-contributor) or an equivalent role with permissions to read, write, and delete blobs. ### Google Cloud Storage To use the [Google Cloud Storage](https://cloud.google.com/storage/) backend pass the `gs://` as your ``: ```sh $ pulumi login gs:// ``` To configure credentials for this backend, see [Application Default Credentials](https://cloud.google.com/docs/authentication/production). For additional configuration options, see [Google Cloud Setup](/registry/packages/gcp/installation-configuration/). If you're new to Google Cloud Storage, see [the Google Cloud documentation](https://cloud.google.com/storage/docs/quickstarts). ### PostgreSQL To use the [PostgreSQL](https://www.postgresql.org/) backend pass the `postgres://:@:/` as your ``: ```sh $ pulumi login postgres://:@:/ ``` > **Warning:** Avoid including credentials directly in commands. Consider using environment variables or other secure credential management methods. For additional configuration options, see the [README ↗](https://github.com/pulumi/pulumi/blob/master/pkg/backend/diy/postgres/README.md). ### Scoping Versions of Pulumi prior to v3.61.0 placed stacks in a global namespace in DIY backends. This meant that you couldn't share stack names (e.g. `dev`, `prod`, `staging`) across multiple projects in the same DIY backend. With Pulumi v3.61.0 and later, stacks created in new or empty DIY backends are scoped by project by default—same as the Pulumi Cloud backend. Existing DIY backends will continue to use the global namespace for stacks. You can upgrade an existing DIY backend to use project-scoped stacks using the `pulumi state upgrade` command. This command will upgrade all stacks in the backend to be scoped by project. > **Note:** `pulumi state upgrade` will make upgraded stacks inaccessible to older versions of Pulumi. This is a one-way operation. Once you have upgraded your backend, you cannot downgrade to the previous version. ## Migrating Between State Backends It is possible to start with one backend and then later migrate to another. This is common if you have began your project with Pulumi using a DIY backend but later decided to adopt Pulumi Cloud for easier use within your team. This section describes how to perform this operation, however, if you would like our assistance with a migration, [please get in touch](/contact/). The state for a stack includes information about its backend as well as other unique information such as its encryption provider. As such, moving a stack between backends isn't as simple as merely copying its state file. The [`pulumi stack rename` command](/docs/iac/cli/commands/pulumi_stack_rename) can be used for simple renames within the same backend; however, Pulumi also supports migrating stacks between backends using the `pulumi stack export` and `pulumi stack import` commands, which understand how to perform the necessary translations. As an example, imagine you'd like to migrate a stack named `my-app-production` from a DIY backend to the Pulumi Cloud backend. To perform the migration, run the following sequence commands: ```sh # switch to the backend/stack we want to export $ pulumi login --local $ pulumi stack select my-app-production # export the stack's state to a local file $ pulumi stack export --show-secrets --file my-app-production.stack.json # logout and login to the desired new backend $ pulumi logout $ pulumi login # default to Pulumi Cloud # create a new stack with the same name on pulumi.com $ pulumi stack init my-app-production # import the new existing state into pulumi.com $ pulumi stack import --file my-app-production.stack.json ``` After performing these steps, your stack will now be under the management of Pulumi Cloud. All subsequent operations should be performed using this new backend. > **Note:**: After migration, your stack's state will be managed by the Pulumi Cloud backend, but the stack will continue using the same secrets provider. You can separately [change the secrets provider](/docs/concepts/secrets#changing-the-secrets-provider-for-a-stack) for your stack if needed. ## Refreshing state Pulumi's state records what your infrastructure looked like after the last `pulumi up` or `pulumi refresh`. When you run `pulumi preview` or `pulumi up`, Pulumi compares this recorded state against the configuration declared in your program to determine which changes need to be made. It does not query each resource directly from your cloud provider on every run. This means that if someone modifies a resource _outside_ of Pulumi (by editing it in the cloud provider's console, applying a change with a provider CLI, or by some other out-of-band means), that change is not automatically reflected in Pulumi's state. The next `pulumi up` or `pulumi preview` will not account for those changes, and may produce unexpected results including overwriting them. ### Why Pulumi does not refresh automatically Pulumi intentionally does not refresh state before every operation, for several reasons: - **Performance**: Querying every resource's live state from the cloud provider can be slow, particularly for large stacks with hundreds or thousands of resources. Doing this implicitly on every `pulumi up` or `pulumi preview` would add significant latency for the typical case where no out-of-band changes have been made. - **Explicit control**: Pulumi treats your program as the source of truth for desired state. Automatically reconciling out-of-band changes before each operation could cause your program and your actual infrastructure to diverge silently, or could cause Pulumi to preserve changes you intended to overwrite. - **Predictability**: Keeping refresh explicit means you decide when and whether to incorporate external changes versus overwrite them, giving you confidence about what each `pulumi up` will do. ### Running `pulumi refresh` To synchronize Pulumi's recorded state with the actual state of your cloud resources, run: ```sh $ pulumi refresh ``` This command queries each resource in your stack from the cloud provider and updates Pulumi's state file to reflect any differences. If a resource has been deleted outside of Pulumi, Pulumi removes it from the state. If properties have changed, Pulumi updates the state to match. `pulumi refresh` updates only the state. It does not modify your Pulumi program or apply any changes to your infrastructure. If you want to preserve external changes going forward, you also need to update your program to reflect the new configuration; otherwise, the next `pulumi up` will attempt to revert those properties back to what your program declares. Run `pulumi refresh` when: - Resources have been modified or deleted outside of Pulumi, such as through the cloud console or another tool. - You are troubleshooting unexpected diffs during a `pulumi preview` and suspect the state may be stale. - You want to verify that Pulumi's recorded state accurately reflects reality before running a `pulumi destroy`. ### Refreshing as part of an update To refresh state and then apply your program's desired state in a single step, pass the `--refresh` flag to `pulumi up`: ```sh $ pulumi up --refresh ``` Similarly, to preview what an update would look like after first refreshing state: ```sh $ pulumi preview --refresh ``` ### Automated drift detection For teams that want to detect and remediate out-of-band changes on a schedule, Pulumi Cloud provides built-in [drift detection and remediation](/docs/deployments/deployments/drift/). With drift detection configured, Pulumi Cloud periodically runs `pulumi refresh` against your stacks and alerts you (or optionally remediates automatically) when the actual state of your infrastructure diverges from Pulumi's recorded state. To learn more, see [Drift detection](/docs/deployments/deployments/drift/). ## Advanced State Pulumi is designed to abstract state management away from you so that you can operate in terms of declarative infrastructure as code. In certain advanced cases, you may want or need to interact with state more directly, especially when using DIY backends. In those cases, the following sections may be helpful. ### Importing Existing Resources Pulumi supports importing resources that were already created outside of Pulumi, such as resources created using the cloud console, a cloud CLI or SDK, or even another infrastructure as code tool. Resource metadata is imported into your Pulumi state and source code is generated in your chosen language to match that state. To learn more about importing existing resources, see [Importing Infrastructure](/docs/using-pulumi/adopting-pulumi/import/). ### Checkpoints Pulumi state is usually stored in a transactional snapshot called a _checkpoint_. Pulumi records checkpoints early and often as it executes so that Pulumi can operate reliably, similar to how database transactions work. The basic functions of state allow Pulumi to diff your program's goal state against the last known update, recover from failure, and destroy resources accurately to clean up afterwards. The checkpoint format augments this with additional failure recovery capabilities in the face of partial failure. The Pulumi Cloud backend records every checkpoint through a transactional API, making it possible to recover from unusual failure scenarios such as network interruptions during updates. DIY backends also maintain checkpoint history (in the `.pulumi/history/` directory), but blob storage backends use a less transactional protocol that may have more difficulty recovering from partial failures. ### State Encryption State is stored in your target backend in the form of checkpoints. In the case of the Pulumi Cloud backend, all remote communication is done over TLS and data is encrypted at rest. ### Secrets A Pulumi "secret" can be used to store sensitive configuration values like database passwords and cloud tokens, and will always be handled safely. Pulumi understands the transitive usage of that secret in your state and will ensure everything it touches is encrypted, no matter which backend you've chosen. A secret can be created one of two ways: passing `--secret` to the `pulumi config set` command, or by [creating one programmatically](/docs/concepts/secrets#secrets). In both cases, the value is encrypted using your stack's chosen encryption provider. By default with Pulumi Cloud, a server-side HSM key is used, but you may customize the encryption provider if you'd like more control over keys, rotation, and so on. To learn more about available encryption providers and how to customize your stack's, see [Configuring Secrets Encryption](/docs/concepts/secrets#configuring-secrets-encryption). ### Exporting and Importing State The `pulumi stack export` and `pulumi stack import` commands can be used to export the latest or a specific version of a stack's state. This can be used to inspect or even manually edit the contents for advanced use cases. For more information on usage, [refer to the CLI documentation](/docs/iac/cli/commands/pulumi_stack/). ### Editing State Manually Although Pulumi was designed to shield you from manually needing to manage state, there are some circumstances where you will want or need to. This includes certain catastrophic failure scenarios, adding, deleting, renaming resources, and other advanced scenarios. The Pulumi state file uses a relatively easy to understand JSON format. The precise JSON format these state files use is not documented, but is defined in the [APIType source code](https://github.com/pulumi/pulumi/tree/master/sdk/go/common/apitype/). The [`state` command](/docs/iac/cli/commands/pulumi_state) also includes some helpful commands to edit your state.