Posts Tagged Infrastructure

Unit Testing Your Infrastructure with Node.js and Mocha

Unit Testing Your Infrastructure with Node.js and Mocha

Testing your infrastructure using familiar tools like Node.js’s Mocha framework allows you to ensure configuration is correct before provisioning, and that the resulting infrastructure has certain desirable properties afterwards. This can enforce team standards, ensure security guidelines are being followed, and so much more. Because Pulumi uses general purpose languages, you can just embed tests alongside your infrastructure-as-code definitions themselves, using a familiar authoring style and reporting experience. In this post, we’ll explore the ins and outs of unit testing your infrastructure.

Read more →

Mapbox IOT-as-code with Pulumi Crosswalk for AWS

Mapbox IOT-as-code with Pulumi Crosswalk for AWS

Guest Author: Chris Toomey, Solution Architect Lead @ Mapbox

With 8 billion+ connected IoT devices and 2 billion GPS-equipped smartphones already online, logistics businesses are tracking assets at every step in the supply chain. At this scale and complexity, it is imperative to have a flexible way to ingest, process, and act upon this data, without sacrificing security or best practices.

To meet this need, Mapbox has created an Asset Tracking Solution that uses Pulumi’s open source JavaScript libraries (AWS, AWSX) available with multi-language support with Pulumi Crosswalk for AWS. Pulumi Crosswalk for AWS is an open source framework that streamlines creation, deployment and management of AWS services with built-in AWS Best Practices and minimal lines of code in common programming languages.

In this blog, we will show snippets of the Javascript code that embraces the power of Pulumi to program AWS service APIs to create the Mapbox solution. To see the full architecture in action with a live bike race across America, please refer to this webinar recorded on June 13th 2019 and the Mapbox whitepaper. Also refer to this blog of the Race across America showcased live during the webinar tomorrow.

Read more →

Using Terraform Remote State with Pulumi

While some people coming to Pulumi are entirely new to Infrastructure as Code,¬†increasingly teams are moving from other tools - whether cloud-specific in the¬†form of CloudFormation or ARM Templates, or cloud-agnostic tools such as Terraform. In these organizations, new infrastructure provisioned¬†with Pulumi must co-exist with existing resources provisioned with other tools,¬†and often by different teams. For example, it’s common to see an application team deploying into a VPC owned and managed by a network operations team.

Pulumi supports this kind of workflow natively using the StackReference type from the Pulumi SDK. Integration with the most popular cloud-specific tools have been supported by Pulumi since the earliest days:

We recently added similar support for reading the outputs of a Terraform state file - both from local .tfstate files, and from all of the remote state backends supported by Terraform. This is exposed via the terraform.state.RemoteStateReference type in the @pulumi/terraform NPM package.

Read more →

Level up your Azure Platform as a Service applications with Pulumi

Level up your Azure Platform as a Service applications with Pulumi

Today’s guest post is from Mikhail Shilkov, a Microsoft Azure MVP and early Pulumi user and contributor - enjoy!

Today I want to guide you through the process of developing Pulumi programs to leverage Azure Platform-as-a-Service (PaaS) services. My language of choice is TypeScript—a powerful and expressive typed language, which is very familiar to many Azure users.

Azure Platform as a Service

Azure consists of dozens of cloud services, from VMs to Kubernetes to Serverless. In my experience, a lot of customers choose Azure for its strong portfolio of PaaS-level services.

Azure App Service is a well-established managed compute offering to run web applications, RESTful APIs, or background workers. Azure SQL Database is a fully managed service to run relational databases with features like high availability and backups available out-of-the-box. Enriched by services like Azure DevOps for CI/CD and Application Insights for APM, PaaS is a powerful way to get the benefits of the cloud without the need to fully re-architect software solutions.

The power of relying on PaaS is evidenced by significant customer adoption. App Service is among the most popular compute services in Azure:

If you use automation (ARM, scripts, TF, …) to define and deploy Azure infrastructure, which services are your primary target? Vote & RT!

– Mikhail Shilkov (@MikhailShilkov) April 23, 2019

Nonetheless, PaaS services pose different challenges to application developers. In particular, the usage of multiple cloud services demands an investment in infrastructure automation. That’s where Pulumi comes to the rescue.

Read more →

Get Started with Docker on AWS Fargate using Pulumi

Get Started with Docker on AWS Fargate using Pulumi

The Docker Getting Started tutorial¬†shows how to develop, build, and run a modern containerized application, from a single custom Docker container published to the Docker Hub, to a scaled out service with load balancing. But there are challenges: it requires you to program in YAML, run (or script) many CLI commands, and manage your own Swarm or Kubernetes cluster. There is an easier way. By using Pulumi’s infrastructure as code, we can build a custom Docker image, publish it to a private AWS container registry, and spin up an AWS Fargate load balanced service running that container, all in 28 lines of TypeScript code and a single pulumi up command.¬†The result¬†leverages¬†the best of what AWS has to offer, with the entire platform at your fingertips, with a single approach. In this article, we’ll see how.

Read more →

Managing F5 BIG-IP Systems with Pulumi

The Pulumi ecosystem is continuously growing and today we’re excited to announce the F5 BIG-IP provider for Pulumi.

F5’s BIG-IP Local Traffic Managment (LTM) services provides advanced traffic management, acceleration, security, and analytics features to your applications. With the addition of our F5 BIG-IP Pulumi provider we are bringing Cloud Native Infrastructure as Code to F5 BIG-IP devices with real programming languages and a consistent programming model. This addresses a frequent use-case we’ve heard from our customers for both on-premises and Cloud workloads.

Let’s look at some examples to demonstrate what’s capable with this provider and the power and flexibility that Pulumi brings to working with your F5 BIG-IP systems.

Read more →

From Terraform to Infrastructure as Software

From Terraform to Infrastructure as Software

Here at Pulumi, we love programming the cloud using infrastructure as code. From the project’s outset, we’ve been inspired by technologies like Terraform, AWS CloudFormation, and Helm, and in fact leverage the Terraform Providers ecosystem, to support a broad range of clouds, including AWS, Azure, and Google Cloud. Just recently, we extended this with first class support for Kubernetes. Pulumi delivers the same infrastructure as code workflows only using general purpose languages like JavaScript, TypeScript, Python, and Go, extending robust infrastructure provisioning with abstraction and reuse, highly productive tooling, and access to all the other things we already know and love about programming languages.

In this article, we will convert existing Terraform configuration to Pulumi TypeScript. By doing so, we’ll see how using general purpose programming languages can help you create simpler, more flexible infrastructure as code, with greater productivity and less repetition. The infrastructure we’ll be working with describes a load-balanced web server hosted by an AWS EC2 instance per availability zone with an option to allow SSH access. Of course, these same benefits would also accrue were we to target Azure, Google Cloud, or Kubernetes instead.

Read more →

Data science on demand: spinning up a Wallaroo cluster is easy with Pulumi

Data science on demand: spinning up a Wallaroo cluster is easy with Pulumi

This guest post is from Simon Zelazny of Wallaroo Labs, and previously appeared on the Wallaroo Labs blog. Find out how Wallaroo powered their cluster provisioning with Pulumi, for data science on demand.

Oh no, more data!

Last month, we took a long-running pandas classifier and made it run faster by leveraging Wallaroo’s parallelization capabilities. This time around, we’d like to kick it up a notch and see if we can keep scaling out to meet higher demand. We’d also like to be as economical as possible: provision infrastructure as needed and de-provision it when we’re done processing.

If you don’t feel like reading the post linked above, here’s a short summary of the situation: there’s a batch job that you’re running every hour, on the hour. This job receives a CSV file and classifies each row of the file, using a Pandas-based algorithm. The run-time of the job is starting to near the one-hour mark, and there’s concern that the pipeline will break down once the input data grows past a particular point.

In the blog post, we show how to split up the input data into smaller dataframes, and distribute them among workers in an ad-hoc Wallaroo cluster, running on one physical machine. Parallelizing the work in this manner buys us a lot of time, and the batch job can continue processing increasing amounts of data.

Read more →

Simple, Reproducible Kubernetes Deployments

Simple, Reproducible Kubernetes Deployments

Kubernetes is a powerful container orchestrator for cloud native applications that can run on any cloud – AWS, Azure, GCP – in addition to hybrid and on-premises environments. Its CLI, kubectl, offers basic built-in support for performing deployments, but intentionally stops short here. In particular, it doesn’t offer diffs and previews, the ability to know when a deployment has succeeded or failed, and why, and/or sophisticated deployment orchestration.

In this post, we’ll see how Pulumi, an open source cloud native development platform, can not only let you express Kubernetes programs in real programming languages, like¬†TypeScript, instead of endless YAML templates, but also how Pulumi delivers simple and reproducible, yet powerful, Kubernetes deployment workflows.

Read more →

Creating and Reusing Cloud Components using Package Managers

Creating and Reusing Cloud Components using Package Managers

Hello! A few weeks back I wrote a post on [serving static websites on AWS with Pulumi]/blog/serving-a-static-website-on-aws-with-pulumi/) detailing how to host a static website on AWS. Pulumi allowed me to wire four different AWS products together in only 200 lines of code. It would be a shame, however if I needed to copy and paste that code every time I wanted to to stand up a new website. Instead, we can package up, share, and reuse our code just like any other Node.js library. It just so happens that this one can be used to create cloud infrastructure.

Read more →