Atlassian: Increasing Developer Productivity

About Bitbucket Cloud

Bitbucket Cloud is a Git-based code hosting and collaboration tool designed for teams and is built by Atlassian. Bitbucket’s integrations with other Atlassian products like Jira allow the entire software team to work together on projects. With Bitbucket, teams have one place to collaborate on code from its concept, through automated testing, all the way to its deployment to the cloud.

Codifying Infrastructure for Cross-Region Deployments

Mike Corsaro is a Senior Software Engineer at Bitbucket. He works on the DevSpeed team, whose mission is to improve developer productivity at Bitbucket. His team helps about 100 Bitbucket developers located all over the world.

Speeding up the Sydney team

Mike was the founding member of the DevSpeed team. His first task was to add cross-regional support to the pipeline that deployed instances on AWS for Bitbucket developers. Developers have private instances of Bitbucket, which they can use to develop new features. The existing pipeline deployed only to the US West (Oregon) region, but it needed to also deploy to the Asia Pacific (Sydney) region.

Evaluating the Existing Tools

Mike first examined the existing toolset to see if it would solve his problem. He found that the team used an Infrastructure-as-Code tool, Terraform, that required users to learn its domain-specific language (DSL), which was not intuitive to Mike and most developers. A developer would go to the repo containing a YAML template written in the DSL, run the Bitbucket CI/CD pipeline, and the tool would spin up a self-contained version of Bitbucket in AWS.

Because of the DSL’s limitations, it would not be straightforward to codify infrastructure deployments for the Sydney team. Mike would need to learn the complicated DSL to complete his task, which would be unproductive and time-consuming. On top of that, he was relatively new to using AWS and the cloud.

He cited an example, saying, “We have three databases. With the old tool, that was 20 lines of code. And of course, coming from a C# background, I had no idea what any of it meant. It would take me hours to figure out how any of it worked and what each property name was and what it did. Another confusion I had was, defining a variable “name” in multiple places, and I had no idea which one was correct. So many things were unclear.”

Mike wanted a general and elegant solution that would let him quickly and easily manage infrastructure in AWS while using familiar languages and tools.

Pulumi to the Rescue

Mike says, “I knew about Pulumi because I followed Joe Duffy on Twitter. I come from the C# world, where he’s a well-known figure. I saw that he’d started a company called Pulumi and its mission really appealed to me.”

Pulumi allows developers to build, deploy, and manage cloud infrastructure and applications using the programming language of their choice, such as TypeScript, JavaScript, Python, Go, and .NET. That means they can use standard programming constructs and standard tools to author and manage infrastructure code.

Mike chose Python to build cloud infrastructure since Bitbucket developers already used Python. He said, “Because all the developers on Bitbucket write Python, they can use all the skills they already have from their day-to-day job to write infrastructure code. There are also safety features you’d expect from a standard programming language. For example, you know if you’re referencing something that isn’t declared or if you’re trying to use a property that doesn’t exist.”

Instead of discarding all the code written with the old tool, Mike used Pulumi’s conversion tool to assist with the transition. Mike says, “I knew that Pulumi provided an automatic conversion tool, and that helped a lot. I spent one day using the automatic conversion tool and then just spent some time making sure everything looked right. In two days, I was all set.”

As an example of how Pulumi made formerly complex tasks straightforward, Mike says, “With the old tool, spinning up our databases meant we had to have about 20 blocks of code and do a lot of copy and pasting. But with Pulumi, it’s Python. So you have a for statement and then you define the names of the databases you want. It’s five lines of code. If you want to add a new database, add one line, and you’re good to go.”

Pulumi also simplified cloud infrastructure management and provided clear visibility into all active resources. Mike said, “Before Pulumi, we had a manager for infrastructure state files that was a complete pain. We ran into weird issues and errors, and it was difficult to ask questions, such as ‘How many developer instances are running?’ Now, Pulumi manages all the state files for us and gives us that visibility.""

Mike also gave an example of how easy it was to deploy changes across all of their active instances with confidence. He says, “Subnets have a maximum number of IP addresses associated with them. We ran into an issue where there would be failures because addresses were selected randomly. I corrected the issue and could confidently deploy it by just writing a test that made sure the select logic worked. And because we’re using Python, I was able to just use the AWS Python API to figure out which subnets were full! That’s simply not possible using a DSL.”

Result

Using Pulumi, with its support for general-purpose languages like Python, Mike quickly and easily built a repeatable pipeline that deploys development environments for the Sydney team despite having limited knowledge of AWS. He also now understood how useful Pulumi could be in improving the developer experience at Bitbucket and how easy it was to use. He began to look for other ways to use Pulumi.

Improving Developer Productivity with Self-Service Cloud Infrastructure

Shortly afterward, Mike reviewed a survey which asked Bitbucket developers how they spent their time. He was surprised to learn that developers were spending 20% of their workweek maintaining their private Bitbucket development environments.

Again, part of the problem rested with the old tool, whose DSL was unfamiliar to most developers. Because it lacked standard programming capabilities, it made the infrastructure code verbose and difficult to understand. Mike knew that he could streamline the entire workflow with Pulumi and make infrastructure more accessible and simpler to use for developers who understand Python.

To make things simple and turnkey for developers, the DevSpeed team built a self-service dashboard that created and configured new Bitbucket instances. Developers enter a few pieces of information, such as what they want to call the instance, what region to use, and an SSH key. Then, a CI/CD pipeline using Bitbucket Pipelines and Bamboo calls Pulumi to deploy a complete developer environment.

Once the instance is ready, the developer simply SSH’s into it, and the instance it will look exactly like Bitbucket. The developer can make the changes they want and then create a pull request to merge their work into a main branch. The developer can also share the URL of the instance with teammates so that they can see the changes when they review the code.

Below is a diagram showing how a Bitbucket cloud instance is architected:

Using the Automation API

The Pulumi Automation API exposes the full power of infrastructure as code through a programmatic interface instead of CLI commands. It exposes Pulumi programs and stacks as strongly-typed and composable building blocks, enabling developers to automate deployments directly from their code at run-time. (A Pulumi program, which is written in your chosen programming language, describes how your cloud infrastructure should be composed.)

Mike and the rest of the DevSpeed team have found many creative ways to use the Automation API to make changes to their infrastructure easily. One example is that they wanted to update the address of their primary database across all developer environments, which were using a raw address from AWS. Instead, they wanted to update the addresses with something more readable. They used the Automation API to write Python code that would automatically select every running instance and make the change. Next, Mike’s team is looking to use Automation API to automatically deploy and run Bitbucket developer environments as a full integration test.

Results

Using Pulumi with its existing CI/CD process, the DevSpeed team built a self-service dashboard that enables any Bitbucket developer to quickly and easily provision a cloud-based development environment. Compared to the DSL-based tool they previously used, Pulumi significantly reduced the number of lines of code needed to model infrastructure and made the code much easier to read and understand. In terms of user satisfaction, the new dashboard meant that developers went from spending one day a week maintaining their environment to less than four hours per week. Mike’s team also receives significantly fewer infrastructure questions from developers on their Slack help channel.

Summary

Here is a summary of how Pulumi helped Bitbucket and the DevSpeed team.

• Allowed developers to use general purpose programming languages like Python and familiar software tools to build and manage infrastructure as code
• Enabled Bitbucket developers to easily provision approved cloud infrastructure using a self-service dashboard built with Pulumi
• Reduced the time Bitbucket developers spent maintaining their instances from 8 hours per week to fewer than four.
• Boosted productivity of developers in Sydney by extending cloud deployments into another AWS region using Infrastructure as Code in Python
• Reduced the size and complexity of codebase for managing infrastructure while increasing its clarity