“What’s in a name? That which we call a rose by any other name would smell as sweet.” William Shakespeare’s oft repeated quote was used to help Juliet explain that a “Montague” is worthy of love. Juliet may have underestimated the importance of a name, however, since things didn’t work out so well for everyone in Verona! Many customers have questions about “names” in Pulumi – and in an effort to make sure that things work out better for them than they did for Romeo, here’s a quick note on naming!
Most cloud infrastructure projects involve working with existing cloud resources — either building on top of existing resources or adopting existing resources under management with a new and more robust infrastructure provisioning solution.
Whether you are adopting resources that were deployed manually using your cloud provider’s console or CLI — or migrating existing infrastructure from tools like Terraform or CloudFormation — Pulumi makes it easy to adopt and manage your existing resources.
Testing your infrastructure using familiar tools like Node.js’s Mocha framework allows you to ensure configuration is correct before provisioning, and that the resulting infrastructure has certain desirable properties afterwards. This can enforce team standards, ensure security guidelines are being followed, and so much more. Because Pulumi uses general purpose languages, you can just embed tests alongside your infrastructure-as-code definitions themselves, using a familiar authoring style and reporting experience. In this post, we’ll explore the ins and outs of unit testing your infrastructure.
Guest Author: Chris Toomey, Solution Architect Lead @ Mapbox
With 8 billion+ connected IoT devices and 2 billion GPS-equipped smartphones already online, logistics businesses are tracking assets at every step in the supply chain. At this scale and complexity, it is imperative to have a flexible way to ingest, process, and act upon this data, without sacrificing security or best practices.
While some people coming to Pulumi are entirely new to Infrastructure as Code, increasingly teams are moving from other tools - whether cloud-specific in the form of CloudFormation or ARM Templates, or cloud-agnostic tools such as Terraform. In these organizations, new infrastructure provisioned with Pulumi must co-exist with existing resources provisioned with other tools, and often by different teams. For example, it’s common to see an application team deploying into a VPC owned and managed by a network operations team.
this kind of workflow
natively using the
type from the Pulumi SDK. Integration with the most popular
cloud-specific tools have been supported by Pulumi since the earliest
aws.cloudformation.getStack()function can be used to obtain the outputs from a CloudFormation Stack.
We recently added similar support for reading the outputs of a Terraform
state file - both from local
.tfstate files, and from all of the
remote state backends supported by Terraform. This is exposed via the
terraform.state.RemoteStateReference type in the
Today’s guest post is from Mikhail Shilkov, a Microsoft Azure MVP and early Pulumi user and contributor - enjoy!
Today I want to guide you through the process of developing Pulumi programs to leverage Azure Platform-as-a-Service (PaaS) services. My language of choice is TypeScript—a powerful and expressive typed language, which is very familiar to many Azure users.
Azure Platform as a Service
Azure consists of dozens of cloud services, from VMs to Kubernetes to Serverless. In my experience, a lot of customers choose Azure for its strong portfolio of PaaS-level services.
Azure App Service is a well-established managed compute offering to run web applications, RESTful APIs, or background workers. Azure SQL Database is a fully managed service to run relational databases with features like high availability and backups available out-of-the-box. Enriched by services like Azure DevOps for CI/CD and Application Insights for APM, PaaS is a powerful way to get the benefits of the cloud without the need to fully re-architect software solutions.
The power of relying on PaaS is evidenced by significant customer adoption. App Service is among the most popular compute services in Azure:
If you use automation (ARM, scripts, TF, …) to define and deploy Azure infrastructure, which services are your primary target? Vote & RT!
– Mikhail Shilkov (@MikhailShilkov) April 23, 2019
Nonetheless, PaaS services pose different challenges to application developers. In particular, the usage of multiple cloud services demands an investment in infrastructure automation. That’s where Pulumi comes to the rescue.
Update: Check out the Learning Machine Case Study where provisioning went from 3 weeks to 1 hour with Pulumi and AWS.
“The impact of serverless capabilities was also transformative for the Learning Machine business. Pulumi enabled a rapid shift to Amazon ECS, AWS Fargate and AWS Lambda — the net effect of which was a 67% reduction in AWS charges. This enabled the team to spend less time focused on maintaining existing infrastructure and more time deploying new applications on AWS and adding new customers.
Pulumi is the foundational technology that allowed us to transform our organization,” said Hughes. The entire DevOps process was streamlined and in addition to realizing better productivity and higher quality, the team has new insight into their SaaS offering that they never thought possible.”
The Pulumi ecosystem is continuously growing and today we’re excited to announce the F5 BIG-IP provider for Pulumi.
F5’s BIG-IP Local Traffic Managment (LTM) services provides advanced traffic management, acceleration, security, and analytics features to your applications. With the addition of our F5 BIG-IP Pulumi provider we are bringing Cloud Native Infrastructure as Code to F5 BIG-IP devices with real programming languages and a consistent programming model. This addresses a frequent use-case we’ve heard from our customers for both on-premises and Cloud workloads.
Let’s look at some examples to demonstrate what’s capable with this provider and the power and flexibility that Pulumi brings to working with your F5 BIG-IP systems.
Here at Pulumi, we love programming the cloud using infrastructure as code. From the project’s outset, we’ve been inspired by technologies like Terraform, AWS CloudFormation, and Helm, and in fact leverage the Terraform Providers ecosystem, to support a broad range of clouds, including AWS, Azure, and Google Cloud.
In this article, we will convert existing Terraform configuration to Pulumi TypeScript. By doing so, we’ll see how using general purpose programming languages can help you create simpler, more flexible infrastructure as code, with greater productivity and less repetition. The infrastructure we’ll be working with describes a load-balanced web server hosted by an AWS EC2 instance per availability zone with an option to allow SSH access. Of course, these same benefits would also accrue were we to target Azure, Google Cloud, or Kubernetes instead.
This guest post is from Simon Zelazny of Wallaroo Labs, and previously appeared on the Wallaroo Labs blog. Find out how Wallaroo powered their cluster provisioning with Pulumi, for data science on demand.
Oh no, more data!
Last month, we took a long-running pandas classifier and made it run faster by leveraging Wallaroo’s parallelization capabilities. This time around, we’d like to kick it up a notch and see if we can keep scaling out to meet higher demand. We’d also like to be as economical as possible: provision infrastructure as needed and de-provision it when we’re done processing.
If you don’t feel like reading the post linked above, here’s a short summary of the situation: there’s a batch job that you’re running every hour, on the hour. This job receives a CSV file and classifies each row of the file, using a Pandas-based algorithm. The run-time of the job is starting to near the one-hour mark, and there’s concern that the pipeline will break down once the input data grows past a particular point.
In the blog post, we show how to split up the input data into smaller dataframes, and distribute them among workers in an ad-hoc Wallaroo cluster, running on one physical machine. Parallelizing the work in this manner buys us a lot of time, and the batch job can continue processing increasing amounts of data.