Pulumi Release Notes: Colorized Stack Traces, Output Methods for JSON, OIDC integration, and much more!
Posted on
We have been busy shipping improvements in the last 2 months. Let’s walk through the release highlights across Pulumi engineering areas from January and February. If you want to learn more between release blogs, follow the CLI improvements in the pulumi/pulumi repo changelog and Pulumi Service features in the new features blogs.
Quick shortcuts to jump to your favorite area:
- Pulumi CLI
- Colorized stack traces for Node.js errors
- New
βshell
option for pulumi stack output pulumi new
in directories with source control- Support plugins hosted in GitLab
- Use access tokens to authenticate to Google Cloud backend
- Delete temporary files from plugin downloads
- “Yes, using Update Plans” prompt removed
- Pulumi Language SDKs
- Pulumi Automation API
- Cloud Providers and Packages
- Pulumi Service & Pulumi.com
Pulumi CLI
Colorized stack traces for Node.js errors
When an error happens in a Pulumi program, stack traces can be instrumental to understand the reasons behind the failure. Stack traces can often be large and contain frames unrelated to user code, which makes debugging harder.
We have now changed the CLI to display Node.js stack traces with colors, where the important bits are highlighted and less important parts faded out. The new display technique helps draw attention to relevant details and is similar to running vanilla node programs.
π Learn more in the Emit colorized stack traces pull request.
New –shell option for pulumi stack output
We added a new --shell
option to the pulumi stack output
command that writes the outputs for a stack as a shell script:
% pulumi stack output --shell
bucketName=mybucket-1234
websiteURL="http://mybucket-1234.example.com"
On Windows, it will produce a Powershell script:
% pulumi stack output --shell
$bucketName = 'mybucket-1234'
$websiteURL = 'http://mybucket-1234.example.com'
As a result, Pulumi users are now able to write shell scripts like the one below:
eval "$(pulumi stack output --shell)"
echo "Created bucket $bucketName"
# => Created bucket mybucket-1234
echo "Visit at $websiteURL"
# => Visit at http://mybucket-1234.example.com
π Learn more in the Add –shell option pull request.
pulumi new
in directories with source control
By default, pulumi new
won’t run unless the current directory is empty to avoid overriding any existing files. This behavior meant that the command failed in directories with initialized source control.
We shipped an improvement where Pulumi CLI will ignore .git, .hg, and .bzr files and directories when considering whether a directory is empty.
π Learn more in the Ignore VCS directories pull request.
Support plugins hosted in GitLab
Pulumi comes with a wealth of packages but users can also create and use custom providers and components. By default, Pulumi expects the binaries to be hosted on GitHub. On top of that, the CLI now supports GitLab as a first-class source of package downloads.
If your package binary is hosted on GitLab, you can now set its pluginDownloadURL
property (see Publish your package) to point to your GitLab project, e.g. to gitlab://gitlab.com/43429536
.
π Learn more in the Support custom plugins from other sources issue.
Use access tokens to authenticate to Google Cloud backend
When storing the state files in Google Cloud backend, you can now authenticate to the Google Cloud Storage using Google access tokens. If the environment variable GOOGLE_OAUTH_ACCESS_TOKEN
is set during a pulumi login gs://...
, Pulumi will utilize the provided token to login to the bucket, assuming this has not been pre-empted by another authentication type.
π Learn more in the Pulumi login to support google oauth access tokens for GCS buckets pull request.
Delete temporary files from plugin downloads
While a plugin is being installed, the CLI downloads the necessary archive and stores the archive in /tmp/pulumi-plugin-tar*
. The CLI is now more mindful of cleaning the temporary file after plugin installation completes to save disk space.
π Learn more in the Delete temporary files from plugin downloads pull request.
“Yes, using Update Plans” prompt removed
Update Plans is a preview feature to constrain the update only to the operations that were planned during preview. To get extra feedback from our users, we experimented with showing an extra option “yes, using Update Plans” at the update prompt. The experiment is now complete, so we removed the option.
Lean more in the Remove the “yes, using Update Plans” prompt issue.
Pulumi Language SDKs
Output methods to serialize and deserialize JSON
Building JSON from values and parsing strings into JSON are two of the most common operations in Pulumi programs. All programming languages have built-in libraries to do so. However, quite often JSON operations need to be combined with Pulumi Inputs and Outputs, which required writing a cumbersome combination of Apply
and serialization method calls.
To streamline this experience, Pulumi SDKs shipped helper serialization and deserialization functions that undertand the type Output<T>
natively. Here is a simple usage example in Python, where table.arn
is an output coming from another resource:
policy = iam.Policy(
"iam-policy",
policy= Output.json_dumps({
"Version": "2012-10-17",
"Statement": [{
"Action": [
"dynamodb:PutItem",
"dynamodb:GetItem"
],
"Effect": "Allow",
"Resource": [table.arn]
}]
}))
π Learn more in the Output methods issue that also tracks a number of futher additions to the toolset of helper methods.
Retrieve stack reference outputs as plain values
You can read outputs from one stack in another stack’s program using Stack References. In the past, the resulting values were always wrapped inside an Output<T>
container in order to preserve secret values (when needed).
With recent SDKs, we shipped an alternative method StackReference.getOutputDetails
. This method returns a plain object with two fields: value
and secretValue
. At most one of these fields is set, depending on whether the stack reference output is a secret or not.
If you know that the output is not a secret, you can write a program to use the plain value immediately, for example, to iterate though a list in a loop and create resources in each iteration.
π Learn more for each language:
StackReference.GetOutputDetails
for GoStackReference.get_output_details
for PythonStackReference.GetOutputDetailsAsync
for .NETStackReference.getOutputDetails
for Node.jsStackReference.outputDetailsAsync
for Java
Native arm64 binaries in Python SDK
Pulumi Python SDK has been upgraded to use the grpcio
library version 1.51.3, which comes with a wheel with native arm64 binaries (universal2
). The change eases the installation of the Pulumi Python SDK on Apple M1/M2 machines.
π Learn more in the Python grpc 1.51.3 contains universal2 binaries pull request.
Go version raised to 1.18
The minimum version of Go required by Pulumi was raised to 1.18. Raising this directive allows us to use language features like generics in the codbase. The change is applied to both pkg
and sdk
.
π Learn more in the go.mod: Raise to ‘go 1.18 pull request.
Delegate alias computation to the engine
Sometimes, two classes in Pulumi SDKs represent the same cloud resource. To enable migration from one to another, Pulumi sets aliases for those classes and declares them to be compatible.
In the past, each SDK was responsible for calculating all possible alias combinations for a given resource and its inherited parent. We recently added support in the engine for this computation along with an optimization to reduce complexity from a multiplicative factor to a linear one.
As a follow-up step, we updated our SDKs to only provide the base aliases, and allow the engine to do this computation. This enables smaller SDKs and faster runtime performance, especially for longer chains of parented resources. Other than that, the user experience hasn’t changed.
π Learn more in the Delegate alias computation from SDKs to Engine issue.
Pulumi Automation API
Adding tags to stacks through Automation API
You can now use Automation API to get, set, list, or remove tags on stacks programmatically. Here is a quick example in TypeScript:
const workspace = await LocalWorkspace.create({
projectSettings: projectSettings,
});
await workspace.createStack(stackName);
await workspace.setTag(stackName, "foo", "bar");
const tags = await workspace.listTags(stackName);
// tags will contain tags for `pulumi:project`, `pulumi:runtime`, and `foo` keys.
π Learn more in the Add support for adding tags to stacks through Automation API issue.
Support cloning from Azure DevOps in Automation API
We implemented a workaround to support Azure DevOps repositories when driving Pulumi from Automation API in Go. This is now also utilized in the Pulumi Kubernetes Operator.
π Learn more in the Support clones from Azure DevOps pull request.
Cloud Providers and Packages
Every week we keep shipping provider updates. Two highlights on our native providers:
- Azure Native shipped 20 new resources, including support for
networkcloud
andvoiceservices
(changelog) - AWS Native shipped 30 new resources in
apigatewayv2
,appflow
,applicationautoscaling
,cloudtrail
,connect
,directoryservice
,ec2
,fms
,gamecast
,gamelift
,kendraranking
,networkmanager
,omics
,organizations
,sagemaker
,secretsmanager
,simspaceweaver
, andsystemsmanagersap
modules (changelog)
Five new community packages were added to the Registry: Vultr, Zscaler Private Access (ZPA), Zscaler Internet Access (ZIA), Statuscake, and Nuage.
Pulumi Service & Pulumi.com
After launching Pulumi Deployments a few months ago and receiving a ton of community feedback (thank you!) we launched an integration with OpenID Connect (OIDC) to enable security first deployments and industry best practices.
OpenID Connect (OIDC) is an identity layer built on top of the OAuth 2.0 framework. It allows third-party applications to verify the identity of the end-user and to obtain basic user profile information. OIDC uses JSON web tokens (JWTs), which you can obtain using flows conforming to the OAuth 2.0 specifications.
Overview
The main benefits of using the OIDC integration for Pulumi Deployments are as follows:
Temporary credentials: You can move away from long term credentials within your organization to leverage temporary credentials that are generated every time a deployment runs. The Pulumi Service issues a new OIDC token that is limited to that deployment. This token is used to obtain cloud credentials that automatically expire, meaning you do not need to store sensitive credentials that are reused with each deployment.
Granular access control: The OIDC token generated by the Pulumi Service contains detailed information about the deployment with which it is associated. In AWS for example, you can configure our OIDC integration to only use temporary credentials if that token is associated with a particular Pulumi stack or Pulumi organization. This enables you to follow the principle of least privilege access and scope access down to only what is needed. For AWS, Azure, and GCP, Pulumi Deployments can automatically exchange its OIDC token for temporary credentials and make those credentials available to providers. For advanced scenarios or other cloud providers, the OIDC token can be manually exchanged for credentials by the user.
π Learn more on how to get started with Pulumi Deployments OIDC by reading the blog post.
If you havenβt yet tried out Pulumi Deployments
You should now see a βDeployβ tab under Stack Settings in the Pulumi Service console and have access to the Deployments REST API. View the documentation for help getting started and join our Slack (#pulumi-deployments channel) for questions and feedback.