Pulumi Blog: Infrastructure lifecycle management

DevSecOps Game-Changer: Security Automation That Delivers Business Results

Sara Huddleston — Thu, 21 Nov 2024 07:16:30 +0000

Organizations are under constant pressure to deliver new products and features faster than ever. But speed alone isn’t enough—businesses must also navigate the complex challenges of ensuring security and managing infrastructure costs effectively.

Enter DevSecOps - the strategic integration of security practices into the DevOps workflow. By automating security processes, organizations can achieve improved speed, scalability, and business impact, all while ensuring security remains a priority.

Tivity Health, a leading health and fitness solutions provider, has embraced this DevSecOps approach using Pulumi, a modern infrastructure as code (IaC) platform. During PulumiUP 2024, David Giambruno, Tivity Health’s VP of Engineering and DevOps, shared how, by leveraging Pulumi, he led the transformation that continuously drives remarkable results in speed, cost savings, and security.

On this article:

The Beginning: From Data Center to the Cloud

Tivity Health’s journey began with a strategic decision to transition from a traditional data center environment to a cloud-native architecture. Rather than opting for a “lift and shift” approach, they made the bold choice to go directly to a cloud-native model, embracing the principles of DevSecOps along the way.

The need to automate infrastructure deployment and management was fundamental. Giambruno explained, “If you can’t automate it, we don’t need it.” This philosophy drove the team to seek out a solution that would not only streamline their operations but also empower their developers to focus on building innovative products for their customers.

Embracing Pulumi: The Power of Automation, Productivity, and Security

Tivity Health’s search for the right tool led them to Pulumi, a unified platform for all the infrastructure needs that allows teams to use general programming languages, such as TypeScript, Python, Java, and Go, to define and manage their cloud infrastructure. Giambruno and his team immediately recognized Pulumi’s ability to deliver on their key requirements:

Automation: Pulumi’s infrastructure as code (IaC) approach enabled Tivity Health to automate the deployment and management of its cloud environments, reducing the time and effort required for these tasks.
Productivity: using Pulumi’s general-purpose programming languages allowed developers to define, deploy, and manage infrastructure within their existing tools.
Security: Pulumi’s security features allowed Tivity Health to integrate security practices into its infrastructure deployment processes, reducing the risk of security breaches and ensuring compliance.
Cost Optimization: Pulumi’s ability to automate the spin-up and teardown of cloud environments on demand has led to significant cost reductions for Tivity Health.

Giambruno describes Pulumi as a “three-dimensional” tool, offering a versatile set of capabilities that have been instrumental in transforming Tivity Health’s operations. “The ability to use those dimensions in lots of different ways to do the automation is what really makes a difference to the teams,” he says.

Driving Business Impact Through Security Automation

By adopting Pulumi and DevSecOps automation, Tivity Health realized significant business benefits:

Speed and Agility: By automating its infrastructure deployment and management processes, Tivity Health has dramatically reduced the time required to spin up new environments or change existing ones. “We run it through automation and boom, it’s out, it’s done,” Giambruno says. This newfound speed and agility have empowered Tivity Health’s developers to focus on building products and features rather than getting bogged down in infrastructure-related tasks.
Cost Optimization: Tivity Health’s cloud-native approach and Pulumi’s automation capabilities have resulted in significant cost savings. The company estimates that its annual cloud spend has decreased from $9.5 million in its data center days to just $2 million—a staggering 79% reduction. These cost savings have allowed Tivity Health to redirect resources towards more strategic initiatives that drive business growth.
Improved Security and Compliance: Tivity Health’s DevSecOps strategy, anchored by Pulumi, has strengthened its security posture and compliance efforts. By integrating security directly into its infrastructure workflows, Tivity Health has improved its security posture. Automation ensures that security measures are enforced consistently across their cloud environments, reducing risks and improving compliance.

Fostering DevOps Culture Through Cross-Functional Collaboration

Using Pulumi provided more than technical benefits. It also enabled better collaboration between the development, security, and operations teams. By providing a common language and framework for infrastructure management, Pulumi has helped break down silos and align these traditionally disparate groups towards a shared goal of delivering secure, high-quality products faster.

Lessons Learned: Navigating the Cultural Shift

Transitioning to DevSecOps and cloud-native practices required a cultural shift at Tivity Health. Giambrono acknowledges that this cultural shift was not without its challenges. He emphasizes the importance of addressing the human element of change, offering the following advice for organizations embarking on a similar journey:

Start with a proof of concepts: Giambruno recommends beginning with a small-scale proof of concepts to demonstrate the capabilities and benefits of the new technologies and processes to help alleviate fears and build confidence among team members.
Empower developers and make them feel safe: By involving developers in the process and ensuring they feel comfortable with the new tools and workflows, Tivity Health gained user buy-in and overall support for the new DevSecOps approach.
Secure buy-in from business and financial stakeholders: Address the concerns of business and financial stakeholders early on, such as the impact on costs and the ability to deliver tangible results. This is crucial for securing the necessary support and resources.
Celebrate successes and build momentum: Giambruno highlights the importance of celebrating the team’s achievements along the way, even when there are bumps on the road, as this helps build enthusiasm and keep the momentum going.

The Future of DevSecOps and Pulumi at Tivity Health

As Tivity Health continues to refine its DevSecOps strategy, Giambruno is optimistic about the future. He envisions a world where the company’s ability to deploy new products and features rapidly will give Tivity Health a significant competitive edge, allowing it to better serve its customers and drive business growth.

Looking ahead, Giambruno is particularly enthusiastic about the potential of Pulumi’s AI-powered capabilities, which he believes will further streamline and optimize the company’s infrastructure management processes. “I’m super looking forward to the tests we’re going to do, like when we acquire someone and then taking them in, ‘Borg-ing’ them into our automation and seeing how much we can take out of their operating cost as fast as possible,” he says.

Tivity Health’s journey with Pulumi is a powerful example of how the right tool, DevSecOps strategy, and automation can drive tangible business results. By focusing on automation, security, and collaboration, organizations can achieve faster, cheaper, and better cloud deployments—putting them on the path to long-term success in an increasingly competitive, cloud-native world.

To learn more about Pulumi and how it can transform your software development and infrastructure management:

Get started with Pulumi Tutorials
Attend an upcoming workshop
Try out the Pulumi Neo code assistant to accelerate your infrastructure as code journey

Unified and Programmatic Approach to Infrastructure Management at BMW Using Pulumi

Sara Huddleston — Thu, 03 Oct 2024 09:52:10 +0000

In the ever-evolving world of automotive technology, BMW has been at the forefront of innovation, seamlessly integrating software into the heart of their vehicles. As cars become increasingly complex, with a growing emphasis on connectivity, over-the-air upgrades, and brand-specific user experiences, the need for a robust and scalable software development approach has become paramount.

Enter the BMW Software Factory, a platform that aims to empower the company’s developers and provide them with a superior development experience. At the core of this initiative is the adoption of Pulumi, a modern infrastructure as code (IaC) solution that has transformed the way BMW manages its software ecosystem.

On this article:

The Challenges of a Sprawling Software Landscape

BMW’s software journey has been a testament to the exponential growth of automotive technology. What started with simple engine controllers has evolved into a complex network of electronic control units (ECUs) scattered throughout the vehicle. As the software footprint continues to expand, BMW recognized the need for a unified and efficient approach to software development and deployment.

The company’s initial efforts involved introducing a platform called “Code Craft,” which provided a comprehensive stack of services to support the software development lifecycle. This stack included a GitHub Enterprise application for source code management, a Gerrit system for Android-based developments, a continuous integration (CI) pipeline, artifact stores, build caching, and various other tools and services.

However, as the demand for software-driven features grew, the complexity of managing this sprawling ecosystem became increasingly challenging. BMW found itself grappling with the need to scale its infrastructure, navigate network limitations across multiple data centers, and adapt to the ever-changing landscape of cloud computing.

The Evolution of BMW’s Software Development Toolchain

BMW’s journey to streamline its software development process has been gradual and iterative. The company’s initial approach involved using Ansible for deployment and a custom-built deployment scripting solution for its OpenShift cluster.

As the complexity of the platform increased, BMW turned to Helm and Kubernetes to manage its containerized services. However, as the company ventured into the public cloud, the limitations of these tools became apparent. The team recognized the need for a more comprehensive and scalable solution to manage their infrastructure as code.

At this critical juncture, BMW discovered Pulumi. This modern IaC solution offered a unique advantage – the ability to leverage a full-fledged programming language, Python, to define and manage their infrastructure. This shift proved to be a game-changer, allowing BMW to leverage its expertise in Python and benefit from the rich ecosystem of libraries and tools available in the Python community.

Embracing Pulumi: Streamlining Infrastructure Management

BMW’s adoption of Pulumi was a strategic move that aimed to address the growing complexity of its software ecosystem. By transitioning from a patchwork of tools to a unified IaC solution, the company was able to streamline its infrastructure management and improve developer productivity.

Shared Modules: Promoting Reusability and Best Practices

One of BMW’s key initiatives was developing a shared modules library, which allowed the team to abstract the complexity of various infrastructure components, such as databases, and provide a consistent and user-friendly interface for their developers.

By leveraging Pulumi’s Python bindings, BMW was able to create reusable modules that encapsulated best practices and sensible defaults, making it easier for developers to provision and manage infrastructure resources. BMW used Pydantic, a data validation library, to define schema-based configurations for its infrastructure. These were then integrated into its IDEs, providing developers with auto-completion and validation support.
This approach not only accelerated the development process but also ensured that the infrastructure deployed across the organization adhered to consistent security and compliance standards.

Transformation Features: Protecting Legacy Services

As part of their software factory, BMW also faced the challenge of integrating legacy services that did not natively support modern authentication and authorization mechanisms, such as OpenID Connect (OIDC).
To address this, BMW leveraged Pulumi’s transformation features to seamlessly inject an OAuth2 proxy into their deployments, providing a secure and consistent way to protect these services without requiring extensive modifications to the underlying applications.
By encapsulating this functionality within a shared module, BMW was able to apply the OAuth2 proxy to multiple services, ensuring a consistent and secure access control layer across their software ecosystem.

Policy Enforcement: Ensuring Compliance and Security

One key benefit of Pulumi’s IaC approach is the ability to define and enforce policies across the organization, ensuring that infrastructure deployments adhere to security and compliance standards. BMW has leveraged Pulumi’s policy-as-code capabilities to implement mandatory checks, such as ensuring that all S3 buckets are encrypted at rest, preventing the deployment of non-compliant resources.

By integrating these policy checks into their deployment workflows, BMW has shifted security and compliance concerns to the left, addressing issues early in the development process and reducing the risk of costly post-deployment remediations.

The Benefits of Pulumi: Accelerating Development and Improving Maintainability

BMW’s adoption of Pulumi has yielded significant benefits, transforming the way the company approaches software development and infrastructure management.

Accelerated Development with Shared Modules

The implementation of shared modules has been a game-changer for BMW. It allows developers to leverage pre-built and tested infrastructure components without having to reinvent the wheel. This has resulted in a significant acceleration of the development process, as teams can focus on building their applications rather than grappling with the complexities of infrastructure provisioning.

Improved Maintainability and Consistency

By centralizing infrastructure management within the shared modules, BMW has ensured that best practices and security standards are consistently applied across the organization. This has not only improved the overall maintainability of the software ecosystem but has also reduced the risk of security and compliance violations.

Leveraging Python’s Ecosystem

BMW’s decision to leverage Pulumi’s Python bindings has been a strategic advantage, as the company was able to tap into the rich ecosystem of Python libraries and tools. This has enabled the team to seamlessly integrate Pulumi with their existing Python-based toolchain, including dependency management, testing frameworks, and code quality tools, further enhancing the development experience.

Streamlined Cloud Migration

As BMW continues to expand its use of public cloud services, Pulumi has played a crucial role in simplifying the migration process. By providing a consistent IaC approach across on-premises and cloud environments, Pulumi has enabled BMW to manage its infrastructure in a unified manner, reducing the complexity and overhead associated with multi-cloud deployments.

The Future of BMW’s Software Factory: Embracing the Cloud

Looking ahead, BMW’s Software Factory is poised to take the next step in its evolution, with plans to transition away from the self-hosted backend and embrace cloud-native services. This strategic move aims to further improve developer productivity and reduce the internal effort required to maintain the underlying infrastructure.

By leveraging the expertise and reliability of cloud service providers, BMW can focus on delivering innovative software features to their customers while the cloud providers handle the operational aspects of running the necessary services.

As BMW continues to push the boundaries of automotive software, Pulumi’s role in its Software Factory will only become more crucial. By providing a scalable and flexible IaC solution, Pulumi empowers BMW’s developers to innovate with confidence and be secure in the knowledge that their infrastructure is managed consistently and in alignment with the company’s security and compliance standards.

Conclusion: Unlocking the Future of Automotive Software

BMW’s journey with Pulumi in the Software Factory showcases the power of modern IaC solutions in navigating the complexities of the automotive software landscape. By embracing a unified and programmatic approach to infrastructure management, BMW has accelerated development, improved maintainability, and ensured compliance across its sprawling software ecosystem.

As the automotive industry continues to evolve, with cars becoming increasingly software-driven, the lessons learned by BMW can serve as a blueprint for other organizations looking to streamline their software development and deployment processes. By leveraging the capabilities of Pulumi and other cutting-edge technologies, the future of automotive software is poised to be more efficient, secure, and responsive to the ever-changing needs of both manufacturers and consumers.

To learn more about Pulumi and how it can transform your software development and infrastructure management:

Get started with Pulumi Tutorials
Attend an upcoming workshop
Try out the Pulumi Neo code assistant to accelerate your infrastructure as code journey

Announcing New Infrastructure Lifecycle Management Features in Pulumi Cloud

Joe Duffy — Wed, 24 Apr 2024 07:00:05 -0800

Today we announced a new wave of Infrastructure Lifecycle Management capabilities in Pulumi Cloud. These build upon and extend our rich Pulumi Deployments foundation for deployment orchestration and workflow. This is an exciting, customer-driven release, with new features that help with Day 2 operations and management.

These new features are:

Drift Detection and Remediation - Continuously detect and remediate drift
Time-to-Live Stacks - Clean up stale infrastructure automatically
Scheduled Deployments - Automate recurring workflows on a schedule
Free Deployments Minutes - Use all of the above for free, no matter your tier!

We built these features based on customer feedback, and our 2,500+ customers are excited:

“At Oleria, we understand the importance of addressing issues with privacy, security, and data integrity. Earning and maintaining our trust isn’t just a responsibility but a fundamental aspect of our mission,” said Jim Alkove, CEO of Oleria. “Pulumi also understands how these attributes affect cloud infrastructure. Our team is excited to use these new capabilities, giving us even more confidence as we enable our customers to securely manage access to decentralized SaaS applications, adaptively and intelligently.”

These advanced features are available today in the Enterprise and Business Critical editions of Pulumi Cloud. These features together enable an end-to-end experience for managing the lifecycle and consistency of your cloud infrastructure to ensure enterprises have the capabilities needed to operate at scale.

Infrastructure Lifecycle Management Capabilities - Excelling at Day 2 and Beyond

So, why infrastructure lifecycle management?

Infrastructure as Code (IaC) is table stakes for our customers. It helps to ensure Day 1, 2, and beyond cloud practices are robust, secure, and dependable. However, operational realities mean that sometimes teams need more flexibility than what strict IaC and Git-driven deployments offer.

For example, maybe you need to break glass and interact with the AWS console directly to open up SSH to an EC2 instance in order to debug a gnarly issue. Or maybe you need to scale up your Auto Scaling group (ASG) temporarily while a live-site incident of unexpected traffic peaks plays out. Or maybe someone added tags to your resource without knowing it was under the management of Pulumi. This can lead to drift, where your IaC configuration no longer matches reality, and reality no longer matches your IaC. This is where Drift Detection and Remediation comes in to automatically report on and reconcile it.

Alternatively, you may want to enable self-service infrastructure provisioning for your entire team, including developers and data scientists, who are not infrastructure experts. But what if they forget to wind things down when they’re done? You are left spending valuable money and resources managing infrastructure that’s no longer needed. Time-to-Live Stacks automatically reclaim such ephemeral infrastructure, enabling self-serve with confidence.

Next up, we often want to deploy on a Git commit, which Pulumi Deployments already handles. However, what if we want more flexibility around how and when deployments occur? For instance, perhaps you’re a retailer who doesn’t want to deploy on Black Friday. Or maybe you prefer to deploy at set times throughout the week rather than on every Git commit. This is where Scheduled Deployments comes in, to give us flexible cron-based scheduling of deployments.

Finally, we want everyone to be able to kick the tires with all of the above, no commitments needed. That’s why we’re also announcing today that all users – individuals, teams, and enterprises alike – get an allotment of free monthly deployment minutes to use all of this.

Get started now for free at app.pulumi.com. We can’t wait to hear what you think!

Happy cloud spelunking.

Pulumi Announces Drift Detection and Remediation: Continuously Monitor Your Infrastructure Changes

Meagan Cojocar — Wed, 24 Apr 2024 07:00:04 -0800

At Pulumi, we recognize the challenges platform teams face in maintaining the stability and compliance of their cloud infrastructures. One of the primary challenges is configuration drift, where the actual state of the infrastructure deviates from its intended state. This deviation can occur for various reasons, including manual adjustments made directly in the cloud provider’s console, unintended consequences of scripts, or unauthorized changes. Such drift can lead to significant problems including security vulnerabilities that open up potential breaches, compliance violations that can result in penalties, operational disruptions that affect user experience and business operations, and increased costs from unnecessary resource spend.

Today, we are excited to launch a new Drift Detection and Remediation feature in Pulumi Cloud to address these challenges. This new functionality is designed to automate the detection and correction of drift in your cloud environments, seamlessly integrating into your existing workflows, whether you use Pulumi Deployments or other CI/CD systems, and is supported for any resource you can manage with Pulumi, whether they are from Amazon Web Services, Microsoft Azure, Google Cloud Platform, or any of the 180+ supported providers. Drift Detection and Remediation is available today on the Enterprise and Business Critical editions of Pulumi Cloud.

Why Pulumi Cloud Drift Detection and Remediation?

Drift Detection is the process of identifying changes in the actual state of your cloud environment that deviate from the expected configuration stored in Pulumi Cloud. This can occur when an operator makes an out-of-band change directly in the cloud provider console, bypassing the usual code deployment processes.

Key benefits:

Enhanced Security: Stay ahead of potential security breaches caused by unauthorized changes to your infrastructure.
Compliance Assurance: Ensure that your infrastructure remains compliant with organizational policies and standards after out of band changes.
Time and Cost Efficiency: Reduce the time spent on tracking and rectifying configuration drift or managing a custom drift solution: leading to cost savings.
Risk Mitigation: Minimize the risk of errors and potential outages by promptly identifying and addressing accidental drifts in your environments.

How Drift Detection Works

Pulumi Cloud’s Drift Detection and Remediation operates continuously, on a schedule of your choosing, comparing the state of your resources with the expected configurations defined in your Pulumi setups. Any discrepancies triggered by modifications, deletions, or additions of resources are promptly reported in the new Drift tab in the Pulumi Cloud console. Alerts can be configured to be sent via webhooks, Slack, or Microsoft Teams when drift occurs, with detailed information about the drift’s nature and scope provided directly within the alerts.

For those using Pulumi Deployments, the system not only regularly detects but can also remedy the drift to align with your Pulumi program. This auto-remediation applies the last known infrastructure as code state, overwriting the drift. Auto-remediation can be set to occur by default or turned off to allow for manual intervention in the remediation of drift, depending on the criticality and nature of the stacks involved.

Under the hood, drift is detected by regularly running a new Pulumi operation we have added, pulumi refresh --preview-only. This operation collects the set of changes that exist between the cloud provider and the current desired state in Pulumi. These scheduled operations build on top of the new Scheduled Deployments functionality as is Time-to-Live Stacks, both of which were also announced today. If there is a delta between the cloud and the current desired state, it is presented in the Drift tab along with a “Drift detected” notification on the stack. If auto-remediation is configured, a pulumi update --refresh is run automatically following any detection of drift. All of this is automated by Pulumi Deployments for users who have configured Pulumi Deployments on their stack. For users who are not using Pulumi Deployments, they can configure their existing CI/CD system to run these same operations regularly, and the results will still be displayed as part of the Drift tab in Pulumi Cloud and you can receive notifications when drift is detected.

Getting Started with Drift Detection

To get started with Drift Detection in Pulumi, select which deployment route you will be using:

Pulumi Deployments: Pulumi programs are run on Pulumi-hosted compute, allowing you to set a schedule and let us handle the rest, including detecting drift, auto-remediation if you turn it on, history of each drift run and what was detected and notifications for drift events.
Existing CI/CD system: Run Drift Detection by regularly running pulumi refresh --preview-only to see what drift is detected in the Pulumi Cloud console and get notified on drift events.

Setting it Up in the Pulumi Cloud Console

In order to set up Drift Detection and Remediation in the Pulumi Cloud console, follow these steps:

Ensure Deployments Settings are configured on the stack see the docs
Navigate to the Stack > Settings > Schedules page
Select “Drift”
(Optional) Turn on auto-remediation if applicable
Set the schedule using a cron expression
Save the Schedule

And just like that, you have Drift Detection! To test what the output will look like you can use the Actions button in the top right on the Stack page to run an ad hoc Drift Detection operation or an Auto-remediate operation. Note: if you have never run a pulumi refresh on that stack before, you will want to run one before setting up Drift Detection and ensure a clean refresh state. More details on getting a clean refresh state in our documentation.

You will now see on the Drift tab, a summary of what resources have been updated or deleted, the properties that have changed, when the drift run happened and a link to the Deployment with further details.

When drift is detected, you have a couple of options:

Run remediation from Pulumi Cloud: If you want to overwrite the changes made in your cloud provider with the most recently specified desired state of your Pulumi program. This can be done via the “Remediate drift” option in the Actions drop down on your stack page.
Run refresh from Pulumi Cloud: If you want to accept the changes made in your cloud provider back into your Pulumi desired state. This can be done via the “Refresh” option in the Actions drop down on your stack page. You may also want to update your Pulumi program to align with the changes made in the cloud provider.

Setting it Up via the API

For those who prefer to automate and script their infrastructure tasks, Drift Detection and Remediation can be configured programmatically using simple HTTP requests. Here are the new endpoints we have added:

Create a Drift schedule
Get a Drift schedule
Update or delete a Drift schedule
Pause or resume a Drift schedule
List all schedules (includes raw Pulumi operations and Time-to-Live schedules)

Below is an example of setting up Drift Detection and Remediation on a stack, see the Pulumi Deployments REST API documentation for more details on how to set Drift Detection and Remediation up programmatically.

Create a Drift Detection and Remediation schedule:

curl -H "Accept: application/vnd.pulumi+json" \
 -H "Content-Type: application/json" \
 -H "Authorization: token $PULUMI_ACCESS_TOKEN" \
 --request POST \
 --data '{"scheduleCron":"0 0 * * *","autoRemediate":true}' \
 https://api.pulumi.com/api/stacks/{organization}/{project}/{stack}/deployments/drift/schedules

Setting it Up via the Pulumi Service Provider

The Pulumi Service Provider allows you to set up automated Drift Detection and Remediation in source control.

import * as pulumi from "@pulumi/pulumi";
import * as pulumiservice from "@pulumi/pulumiservice";

const organizationName = "my-org";
const projectName = "my-project";
const stackName = "production";

// Creating a DriftSchedule for automatically running drift detection
const driftDetectionSchedule = new pulumiservice.DriftSchedule("driftDetectionSchedule", {
 organization: organizationName,
 project: projectName,
 stack: stackName,
 scheduleCron: "0 0 * * *", // Run drift detection daily at midnight
 autoRemediate: true, // Automatically remediate any drift detected
});

export const driftScheduleId = driftDetectionSchedule.scheduleId;

import pulumi
import pulumi_pulumiservice as pulumiservice

organization_name = "my-org"
project_name = "my-project"
stack_name = "production"

# Create a drift detection schedule
drift_detection_schedule = pulumiservice.DriftSchedule("driftDetectionSchedule",
 organization=organization_name,
 project=project_name,
 stack=stack_name,
 schedule_cron="0 0 * * *", # Run drift detection daily at midnight
 auto_remediate=True) # Automatically remediate any drift detected

pulumi.export('driftScheduleId', drift_detection_schedule.schedule_id)

package main

import (
 "github.com/pulumi/pulumi-pulumiservice/sdk/go/pulumiservice"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 driftDetectionSchedule, err := pulumiservice.NewDriftSchedule(ctx, "driftDetectionSchedule", &pulumiservice.DriftScheduleArgs{
 Organization: pulumi.String("my-org"),
 Project: pulumi.String("my-project"),
 Stack: pulumi.String("production"),
 ScheduleCron: pulumi.String("0 0 * * *"), // Run drift detection daily at midnight
 AutoRemediate: pulumi.Bool(true), // Automatically remediate any drift detected
 })
 if err != nil {
 return err
 }

 ctx.Export("driftScheduleId", driftDetectionSchedule.ScheduleId)
 return nil
 })
}

using Pulumi;
using PulumiService = Pulumi.PulumiService;

class Program
{
 static Task<int> Main() => Deployment.RunAsync(() => {
 var driftDetectionSchedule = new PulumiService.DriftSchedule("driftDetectionSchedule", new PulumiService.DriftScheduleArgs
 {
 Organization = "my-org",
 Project = "my-project",
 Stack = "production",
 ScheduleCron = "0 0 * * *", // Run drift detection daily at midnight
 AutoRemediate = true, // Automatically remediate any drift detected
 });

 return new Dictionary<string, object?>
 {
 { "driftScheduleId", driftDetectionSchedule.ScheduleId }
 };
 });
}

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.pulumiservice.Webhook;
import com.pulumi.pulumiservice.WebhookArgs;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 private static void stack(Context ctx) {
 var driftDetectionSchedule = new Webhook("driftDetectionSchedule", WebhookArgs.builder()
 .organization("my-org")
 .project("my-project")
 .stack("production")
 .scheduleCron("0 0 * * *") // Run drift detection daily at midnight
 .autoRemediate(true) // Automatically remediate any drift detected
 .build());

 ctx.export("driftScheduleId", driftDetectionSchedule.scheduleId());
 }
}

name: drift-detection-setup
runtime: yaml
description: Setup of automated drift detection with Pulumi

resources:
 driftDetectionSchedule:
 type: pulumiservice:index:DriftSchedule
 properties:
 organization: my-org
 project: my-project
 stack: production
 scheduleCron: "0 0 * * *" # Run drift detection daily at midnight
 autoRemediate: true # Automatically remediate any drift detected

outputs:
 driftScheduleId: ${driftDetectionSchedule.scheduleId}

See the Pulumi Service Provider documentation for more details on how to manage Drift Detection and Remediation in source control.

Wrapping it up

We are thrilled to be releasing a top customer ask today and are looking forward to hearing about your experience using Drift Detection and Remediation.

Until next time, happy building!

Time-to-Live Stacks: Auto-Destroying Stacks

Meagan Cojocar — Wed, 24 Apr 2024 07:00:03 -0800

In the dynamic landscape of cloud-based software development, platform teams face the continuous challenge of balancing the need for innovation with the imperative to control cloud costs and security. Pulumi Cloud’s new Time-to-Live (TTL) Stacks feature directly addresses this challenge by empowering teams to manage infrastructure lifecycles automatically, mitigating the risks associated with stale infrastructure and resource costs ballooning.

Creating temporary environments for development, testing, or staging is a common practice in the lifecycle of software development. These environments are crucial for running experiments, testing new features, and ensuring that everything works smoothly before going live. However, these environments often don’t get decommissioned properly and linger, forgotten, consuming resources and adding unnecessary costs to your cloud bill.

Furthermore, managing these temporary stacks usually requires manual oversight, adding to the operational burden of your team. They need to track which environments are active, which are outdated, and manually shut them down to avoid security risks from idle and unmonitored resources. This process is not only time-consuming but also prone to human error, leading to either premature deletion of necessary resources or overlooked environments that pose security risks.

Simplifying Stack Management with Time-to-Live Stacks

The Time-to-Live Stacks feature in Pulumi Cloud is designed to solve these pain points by allowing teams to set a predefined lifespan on any stack directly in Pulumi Cloud. After the specified time, these stacks are automatically destroyed, and using the optional “delete after destroy” setting, they are also removed from Pulumi Cloud entirely. This setting plays a pivotal role by not just destroying the resources but also cleaning up the stack itself after all resources have been terminated, ensuring that no residual artifacts are left behind. This maintains a clean state in the cloud environment and further reduces costs by reducing clutter. Time-to-Live Stacks are available today on the Enterprise and Business Critical editions of Pulumi Cloud.

Key Benefits of TTL Stacks

Cost Control: Automatically terminate stacks that are no longer needed, helping to avoid overspending on cloud resources that are not in active use.
Developer Enablement: Provides developers the flexibility to quickly spin up and test application changes in sandbox environments without worrying about the cleanup.
Operational Efficiency: Reduces the administrative burden of manually tracking and decommissioning temporary or outdated stacks, allowing teams to focus on more strategic tasks.
Enhanced Security: Automatically removing unused or neglected stacks helps ensure compliance and reduces the potential attack surface, making your infrastructure safer.

Getting Started with Time-to-Live Stacks

Setting it up in the Pulumi Cloud Console

In order to set up Time-to-Live Stacks in the Pulumi Cloud console, follow these steps:

Ensure Deployments Settings are configured on the stack see the docs
Navigate to the Stack > Settings > Schedules page
Select “Time-to-Live”
(Optional) Turn on “Delete After Destroy” if applicable
Set the schedule using a cron expression
Save the Schedule

You will now see a schedule in the Schedules tab which highlights when the destroy will occur.

Setting it up via the API

For those who prefer to automate and script their infrastructure tasks, Time-to-Live schedules can be configured programmatically using simple HTTP requests.

Create a Time-to-Live schedule
Get a Time-to-Live schedule for a stack
Update or delete a Time-to-Live schedule
Pause or resume a Time-to-Live schedule
List all schedules in the organization (includes raw Pulumi operations and Drift schedules)

Below is an example of setting up Time-To-Live on a stack programmatically:

curl -H "Accept: application/vnd.pulumi+json" \
 -H "Content-Type: application/json" \
 -H "Authorization: token $PULUMI_ACCESS_TOKEN" \
 --request POST \
 --data '{"timestamp":"2024-12-31T23:59:59Z","deleteAfterDestroy":true}' \
 https://api.pulumi.com/api/stacks/{organization}/{project}/{stack}/deployments/ttl/schedules

Refer to the Pulumi Deployments REST API documentation for more details on how to use the REST API to manage Time-to-Live Stacks.

Setting it up via the Pulumi Service Provider

The Pulumi Service Provider allows you to set up and manage Time-to-Live Stacks in source control.

import * as pulumi from "@pulumi/pulumi";
import * as pulumiservice from "@pulumi/pulumiservice";

const organizationName = "my-org";
const projectName = "my-project";
const stackName = "temp-stack";

const ttlSchedule = new pulumiservice.TtlSchedule("ttlSchedule", {
 organization: organizationName,
 project: projectName,
 stack: stackName,
 timestamp: "2024-01-01T00:00:00Z" // Specify the ISO date/time for destruction
});

export const scheduleId = ttlSchedule.scheduleId;

import pulumi
import pulumi_pulumiservice as pulumiservice

organization_name = "my-org"
project_name = "my-project"
stack_name = "temp-stack"

# Create a TTL schedule for stack destruction
ttl_schedule = pulumiservice.TtlSchedule("ttlSchedule",
 organization=organization_name,
 project=project_name,
 stack=stack_name,
 timestamp="2024-01-01T00:00:00Z") # Specify the ISO date/time for destruction

pulumi.export('scheduleId', ttl_schedule.schedule_id)


package main

import (
 "github.com/pulumi/pulumi-pulumiservice/sdk/go/pulumiservice"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 ttlSchedule, err := pulumiservice.NewTtlSchedule(ctx, "ttlSchedule", &pulumiservice.TtlScheduleArgs{
 Organization: pulumi.String("my-org"),
 Project: pulumi.String("my-project"),
 Stack: pulumi.String("temp-stack"),
 Timestamp: pulumi.String("2024-01-01T00:00:00Z"), // Specify the ISO date/time for destruction
 })
 if err != nil {
 return err
 }

 ctx.Export("scheduleId", ttlSchedule.ScheduleId)
 return nil
 })
}

using Pulumi;
using PulumiService = Pulumi.PulumiService;

class Program
{
 static Task<int> Main() => Deployment.RunAsync(() => {
 var ttlSchedule = new PulumiService.TtlSchedule("ttlSchedule", new PulumiService.TtlScheduleArgs
 {
 Organization = "my-org",
 Project = "my-project",
 Stack = "temp-stack",
 Timestamp = "2024-01-01T00:00:00Z", // Specify the ISO date/time for destruction
 });

 return new Dictionary<string, object?>
 {
 { "scheduleID", ttlSchedule.ScheduleId }
 };
 });
}

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.pulumiservice.TtlSchedule;
import com.pulumi.pulumiservice.TtlScheduleArgs;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 private static void stack(Context ctx) {
 var ttlSchedule = new TtlSchedule("ttlSchedule", TtlScheduleArgs.builder()
 .organization("my-org")
 .project("my-project")
 .stack("temp-stack")
 .timestamp("2024-01-01T00:00:00Z") // Specify the ISO date/time for destruction
 .build());

 ctx.export("scheduleID", ttlSchedule.scheduleId());
 }
}

name: ttl-schedule-setup
runtime: yaml
description: Setup of TTL for automatic resource destruction with Pulumi

resources:
 ttlSchedule:
 type: pulumiservice:index:TtlSchedule
 properties:
 organization: my-org
 project: my-project
 stack: temp-stack
 timestamp: "2024-01-01T00:00:00Z" # Specify the ISO date/time for destruction

outputs:
 scheduleId: ${ttlSchedule.scheduleId}

See the Pulumi Service Provider documentation for more details on how to manage Time-to-Live Stacks in source control.

Wrapping it up

Time-to-Live Stacks has been one of the top requests from our Pulumi Cloud users, and we are thrilled to be making it available today. We look forward to further feedback on where Pulumi Cloud can best support your organizations’ infrastructure lifecycle management needs.

Happy hacking!

Pulumi Launches Scheduled Deployments: Automate Recurring Workflows

Meagan Cojocar — Wed, 24 Apr 2024 07:00:02 -0800

We are excited to announce Scheduled Deployments for Pulumi Cloud. This new feature, which builds upon the robust foundation of Pulumi Deployments, is designed to streamline cloud management by automating the deployment and deactivation of resources according to customizable schedules. Two new Pulumi features also announced today, Drift Detection and Remediation and Time-to-Live Stacks, are built on top of the new Scheduled Deployments functionality. These three new features are available today on the Enterprise and Business Critical editions of Pulumi Cloud.

Simplifying Cloud Operations Through Automation

Scheduled Deployments enable precise automation of cloud operations, allowing users to schedule any Pulumi operation—such as pulumi up, pulumi refresh, pulumi destroy, or pulumi preview—for any stack with Pulumi Deployments. This feature is not just about automating routine tasks; it’s about enhancing operational efficiency, reducing costs, and deploying resources exactly when needed.

Example Use Cases

Automating Deployments outside of Git: Pulumi Deployments handles Git based Deployments well, but we have heard from customers there are times you want to intentionally diverge from Git triggered Deployments, such as wanting to deploy only at set times ahead of a large usage event or deploy only on non-peak traffic times.
Cost Optimization: Automatically power down non-critical development or test stacks during off-hours to cut unnecessary cloud spend. These stacks can be reactivated on a schedule or on-demand, ensuring resources are available precisely when teams need them without incurring costs during idle times.
Product Launches: Configure infrastructure to deploy right before a major release with pulumi up, ensuring that all resources are aligned and active precisely for the event, eliminating the need for last-minute manual updates.
Scheduled Processing: Perfect for tasks that require computing resources at specific times, such as nightly batch processing or periodic machine learning model training. Schedule these operations to maximize resource utilization and shut down upon task completion to optimize costs.

How Scheduled Deployments Work

Users can define schedules for any Pulumi operation, on any stack, using cron expressions, which provide the flexibility to specify exact times for operations. This granular control over deployment timing allows for effective management of infrastructure tasks, accommodating even the most complex scheduling needs. Pulumi Deployments Schedules can be set up using the Pulumi Cloud console, our REST API or managed in source control using the Pulumi Service Provider.

Integration with Existing Features

Scheduled Deployments integrate seamlessly with existing Pulumi Deployments. The concurrency limits of Pulumi Deployments apply and if deployments on a stack are paused, the scheduled deployments will queue, waiting for the pause to lift before executing. This integration ensures that all scheduled activities respect the operational parameters set within Pulumi Cloud.

Getting Started with Scheduled Deployments

Setting it up in the Pulumi Cloud Console

We’ll walk through how to get up and running with Scheduled Deployments in the Pulumi Cloud console, REST API and Pulumi Service Provider. For more details on how to implement and maximize the benefits of Scheduled Deployments, please visit our documentation.

Setting up Scheduled Deployments via the Pulumi Cloud console is straightforward:

In order to set up Scheduled Deployments in the Pulumi Cloud console, follow these steps:

Ensure Deployments Settings are configured on the stack see the docs
Navigate to the Stack > Settings > Schedules page
Select “Raw operation”
Select from the drop-down the type of operation you would like to schedule
Set the schedule using a cron expression
Save the Schedule

Now you have a Scheduled Deployment!

The Stack > Settings > Schedules page will show all of the schedules configured for your stack, including Drift, TTL or Raw Operation schedules.

Setting it up in the API

For those who prefer to automate and script their infrastructure tasks, Time-to-Live schedules can be configured programmatically using simple HTTP requests.

Create a schedule
Get a schedule for a stack
Update or delete a schedule
Pause or resume a schedule
List all schedules in the organization (includes Drift and Time-to-Live schedules)

Below is an example of creating a Schedule on a stack programmatically:

curl \
 -H "Accept: application/vnd.pulumi+8" \
 -H "Content-Type: application/json" \
 -H "Authorization: token $PULUMI_ACCESS_TOKEN" \
 --request POST \
 --data '{ "scheduleCron":"0 0 * * *", "request": { "operation": "update" } }' \
 https://api.pulumi.com/api/stacks/{organization}/{project}/{stack}/deployments/schedules

Refer to the Pulumi Deployments REST API documentation for more details on how to use the REST API to manage Scheduled Deployments.

Setting it up in the Pulumi Service Provider

Pulumi Service Provider for Creating a Raw Schedule

The Pulumi Service provider allows you to set up a raw schedule for operations like updates or deployments based on a specific time or a recurring schedule inside of source control.

import * as pulumi from "@pulumi/pulumi";
import * as pulumiservice from "@pulumi/pulumiservice";

const organizationName = "my-org";
const projectName = "my-project";
const stackName = "prod-stack";

const rawSchedule = new pulumiservice.DeploymentSchedule("rawSchedule", {
 organization: organizationName,
 project: projectName,
 stack: stackName,
 scheduleCron: "0 0 * * *", // Run an update daily at midnight
 pulumiOperation: pulumiservice.PulumiOperation.update
});

export const scheduleId = rawSchedule.scheduleId;

import pulumi
import pulumi_pulumiservice as pulumiservice

organization_name = "my-org"
project_name = "my-project"
stack_name = "prod-stack"

# Create a raw deployment schedule
raw_schedule = pulumiservice.DeploymentSchedule("rawSchedule",
 organization=organization_name,
 project=project_name,
 stack=stack_name,
 schedule_cron="0 0 * * *", # Run an update daily at midnight
 pulumi_operation=pulumiservice.PulumiOperation.update)

pulumi.export('scheduleId', raw_schedule.schedule_id)

package main

import (
 "github.com/pulumi/pulumi-pulumiservice/sdk/go/pulumiservice"
 "github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 rawSchedule, err := pulumiservice.NewDeploymentSchedule(ctx, "rawSchedule", &pulumiservice.DeploymentScheduleArgs{
 Organization: pulumi.String("my-org"),
 Project: pulumi.String("my-project"),
 Stack: pulumi.String("prod-stack"),
 ScheduleCron: pulumi.String("0 0 * * *"), // Run an update daily at midnight
 PulumiOperation: pulumiservice.PulumiOperationUpdate,
 })
 if err != nil {
 return err
 }

 ctx.Export("scheduleId", rawSchedule.ScheduleID)
 return nil
 })
}

using Pulumi;
using PulumiService = Pulumi.PulumiService;

class Program
{
 static Task<int> Main() => Deployment.RunAsync(() => {
 var rawSchedule = new PulumiService.DeploymentSchedule("rawSchedule", new PulumiService.DeploymentScheduleArgs
 {
 Organization = "my-org",
 Project = "my-project",
 Stack = "prod-stack",
 ScheduleCron = "0 0 * * *", // Run an update daily at midnight
 PulumiOperation = PulumiService.PulumiOperation.Update,
 });

 return new Dictionary<string, object?>
 {
 { "scheduleId", rawSchedule.ScheduleId }
 };
 });
}

import com.pulumi.Context;
import com.pulumi.Pulumi;
import com.pulumi.pulumiservice.DeploymentSchedule;
import com.pulumi.pulumiservice.DeploymentScheduleArgs;

public class App {
 public static void main(String[] args) {
 Pulumi.run(App::stack);
 }

 private static to stack(Context ctx) {
 var rawSchedule = new DeploymentSchedule("rawSchedule", DeploymentScheduleArgs.builder()
 .organization("my-org")
 .project("my-project")
 .stack("prod-stack")
 .scheduleCron("0 0 * * *") // Run an update daily at midnight
 .pulumiOperation(com.pulumi.pulumiservice.PulumiOperation.update())
 .build());

 ctx.export("scheduleId", rawSchedule.name());
 }
}

name: raw-schedule-setup
runtime: yaml
description: Setup of a raw schedule for automatic operations with Pulumi

resources:
 rawSchedule:
 type: pulumiservice:index:DeploymentSchedule
 properties:
 organization: my-org
 project: my-project
 stack: prod-stack
 scheduleCron: "0 0 * * *" # Run an update daily at midnight
 pulumiOperation: update

outputs:
 scheduleId: ${rawSchedule.scheduleId}

See the Pulumi Service Provider documentation for more details on how to manage Scheduled Deployments in source control.

Wrapping it up

We are looking forward to unlocking this capability to enable an out of the box way to automate your reoccuring workflows.

Happy building!

New Pulumi Deployments Free Tier for Everyone: Automate Your Infrastructure Workflows

Meagan Cojocar — Wed, 24 Apr 2024 07:00:01 -0800

At Pulumi, we are committed to empowering developers and infrastructure teams with the tools they need to efficiently manage cloud resources at scale. As part of our ongoing efforts to enhance user experience and ease of use, we are excited to share some significant updates to Pulumi Deployments pricing, making it easier for all customers to experiment with and adopt the easiest way to go from code to cloud.

Starting today, all paid Pulumi Cloud organizations get 3,000 minutes per month of Pulumi Deployments included with their Pulumi Cloud subscription.

Paired with the launch of Drift Detection and Remediation, Time-to-Live Stacks and Scheduled Deployments - it has never been easier to kick the tires on the latest and greatest tools for infrastructure lifecycle management.

Introduction to Pulumi Deployments

Pulumi Deployments is a fully managed platform designed to empower teams to efficiently manage and scale their cloud infrastructure. Whether you are just getting started in the cloud and want the easiest path to set up secure CI/CD, or are a platform engineer building a self-service platform to support thousands of engineers, Pulumi Deployments provides you with the building blocks that enable you to scale your cloud footprint without scaling your headcount. It offers a fully managed, secure, and elastic compute environment for running infrastructure deployments. It simplifies configuration by centralizing the management of source code, cloud credentials, and other necessities at the stack level. Additionally, the platform supports a variety of deployment triggers such as REST API, Git Push to Deploy, and Click to Deploy, allowing users to customize and automate their cloud operations beyond the capabilities of traditional CI/CD systems. When a stack is managed with Pulumi Deployments, richer Pulumi Cloud functionality is supported, such as Review Stacks and Click to Deploy templates in the New Project Wizard.

Exciting New Features

Drift Detection

Stay on top of your infrastructure’s health with our new Drift Detection feature. Drift Detection helps you monitor and rectify discrepancies between your desired infrastructure state and the actual state, ensuring consistency and compliance.

TTL Stacks

Manage resources effectively with Time to Live (TTL) Stacks. This feature automatically deallocates resources after a set period, perfect for temporary or experimental deployments, reducing costs and manual cleanup efforts.

Scheduled Deployments

Optimize your deployment strategy with Scheduled Deployments. This feature allows you to plan and execute deployments at specific times, making it easier to manage updates during off-peak hours, minimizing impact on your services.

Pulumi Deployments for Everyone

To help ensure all Pulumi Cloud users can get started with Pulumi Deployments, we are introducing an updated allocation model for deployment minutes. Now, all customer tiers—Team, Enterprise, and Business Critical—will receive 3,000 free deployment minutes each month. This change allows our users to test and leverage Pulumi Deployments without upfront commitments or the need to engage in lengthy procurement processes. The table below shows the Pulumi Deployments dimensions (both pricing and limits) by tier, more details can be found on our pricing page.

Dimension	Individual	Team	Enterprise	Business Critical
Minutes Included	500/month	3000/month	3000/month	3000/month
Cost per minute		$0.01	Starts at $0.01	Starts at $0.01
Concurrency Limit	1	5	25	150
Self-hosted Deployment Runners	Available	Available	Available	Included

Looking Forward

As a core part of Pulumi Cloud, Pulumi Deployments is now easier to adopt in your organization. We expect to release even more rich features built on Pulumi Deployments, offloading the time consuming work to Pulumi Cloud with turn-key features to solve your existing pain points.

We can’t wait to hear any feedback on this new set of features and Pulumi Deployments as a whole!