Pulumi Blog: Organizational Patterns Series

Organizational Patterns: A Developer Portal

Matty Stratton — Thu, 30 Dec 2021 11:00:39 -0600

Using Pulumi is more than just writing code and components. In addition to common software development practices, there are also a number of success patterns related to how your company or team builds and deploys Pulumi programs to successfully build, deploy, and manage your infrastructure and applications. In this continuation of a series, I will explore one of these patterns - using the Pulumi Automation API to create a developer portal.

It is becoming more and more common to consider building out centralized web-based “portals” for developers (and other team members) to provision the infrastructure that their services and applications require. In many ways, the pattern of a developer portal takes portions of both the Single Infra Repo and Automation Team patterns covered in previous posts. The difference is that the developer portal is a single, centralized web application that can be used by multiple teams and organizations to provision infrastructure and applications, rather than writing the Pulumi programs themselves.

Depending upon the needs of the various teams, the developer portal could include options to deploy virtual machines into the cloud of their choice (or the organization’s standard cloud; perhaps the choice isn’t…a choice!), to provision cloud resources such as virtual networks, security groups, database instances, or even deployments to a kubernetes cluster.

Generally speaking, the developer portal is created and maintained by a central platform team. This is the team who knows the success patterns/standards that are employed by the organization, and can implement these patterns in the developer portal. The platform team is also responsible for maintaining the infrastructure that the developer portal requires.

One of the great advantages of providing a single portal for provisioning these resources is that all the “other stuff” that is often required (besides the resources from the hyperscaler) can be handled by the developer portal. For example, if the portal is used to provision a virtual machine, the code behind it can register the virtual machine with the company’s infrastructure management system.

A great example of what is possible in a portal can be seen with Komal Ali’s example, the Self Service Infrastructure Platyform. This is example code only, but it shows some of the capabilities of a developer portal.

In the example, the web portal is created using a combination of Python, Flask and Jinja templates (along with the Pulumi Automation API) to provide push-button access to deploy cloud resources according to the standards and patterns of the organization. The code snippets below illustrate some of the concepts (this code is not meant to be run, but rather to illustrate the concepts - the full example code can be found at github.com/komalali/self-service-platyform).

from flask import (Blueprint, current_app, request, flash,
 redirect, url_for, render_template)

import pulumi
import pulumi_aws as aws
import pulumi.automation as auto
import os
from pathlib import Path

bp = Blueprint("virtual_machines", __name__, url_prefix="/vms")
instance_types = ['c5.xlarge', 'p2.xlarge', 'p3.2xlarge']

In the initial snippet above, we are setting up what we need for the Flask application, but the key items to make note of are including the pulumi_aws and pulumi.automation packages, which will be used for the cloud specific resources and automation.


def create_pulumi_program(keydata: str, instance_type=str):
 ami = aws.ec2.get_ami(most_recent=True,
 owners=["amazon"],
 filters=[aws.GetAmiFilterArgs(name="name", values=["*amzn2-ami-minimal-hvm*"])])

 group = aws.ec2.SecurityGroup('web-secgrp',
 description='Enable SSH access',
 ingress=[aws.ec2.SecurityGroupIngressArgs(
 protocol='tcp',
 from_port=22,
 to_port=22,
 cidr_blocks=['0.0.0.0/0'],
 )])

Continuing on, we define the Pulumi program, and set up some of the standard settings that we consider our defaults and that are not selectable by the user.


 server = aws.ec2.Instance('dlami-server',
 instance_type=instance_type,
 vpc_security_group_ids=[group.id],
 key_name=keypair.id,
 ami=ami.id)

 pulumi.export('public_dns', server.public_dns)

This final snippet is what creates the actual instance, and it exports the public DNS of the instance so that it can be accessed by the user and presented in the portal.

Conclusion

This pattern has been implemented and deployed at many larger enterprises, where the platform team has the resources to dedicate to managing the portal as an actual product. This is a key part to remember - the developer portal is a product whose consumers are the developers and other teams in the organization, and it needs to be managed as such!

Don’t forget to check out the previous two posts in this series, Organizational Patterns - A Single Infra Repo and Organizational Patterns - An Automation Team".

Organizational Patterns: An Automation Team

Matty Stratton — Wed, 22 Dec 2021 11:49:08 -0600

I want to approach this pattern with a bit of caution - there are patterns and anti-patterns related to having a single central team focused on automation.

The anti-pattern is to have an “automation team” within your organization who is tasked with creating all infrastructure as code for the entire organization. This is a common pattern, but it is generally rife with issues.

Understanding how to build and deploy a service or application requires a certain amount of domain knowledge of that service. If you have a single set of people across your entire company who need to write Pulumi programs for each and every application or service, you are now expecting that small group to not only be experts in Pulumi, but also every bit of infrastructure and software applications (both third party and created internally) across your organization.

A more successful pattern is to have the automation team act as enablers and providers for the other groups within the company. This is a team of subject matter experts in using Pulumi, and not only do they have the knowledge to build and deploy infrastructure, but they are familiar with various patterns and good practices for how to build and deploy infrastructure.

There are different mechanisms for how this automation team can enable others. One way is to embed them temporarily within a product team to work alongside them to build their automation. Then, when the product team is done, the automation team member can go and help another squad.

Depending upon the size and skills of the company and this group, they may also be responsible for creating training materials for the other groups. These are usually based upon existing Pulumi training materials, but customized for the specific needs and culture of the company.

Reusable components

This central automation team also often provides reusable components that might be shared across multiple development teams. This helps the individual teams to not have to “reinvent the wheel” and to bring in these components for their own particular needs.

As a simple example, a component could be created (and published as a library to be consumed by product teams) that defines a Kubernetes deployment.

import * as deploy from "@mycorp/deploy";

const kuard = new deploy.Deployment("kuard", {
 image: "gcr.io/kuard-project/kuard:latest",
 replicas: 1,
});

What’s happened here in our (incomplete!) example is that the published package only exposes the parts that the team needs to change; the rest is provided by the automation team.

The application team don’t need to duplicate values (such as ports) because the abstractions cater for them. The application team also didn’t need to apply any metadata, health checks, and environment variables because it can be inferred through convention, increasing velocity across the organization. When values are provided by the application team, they override convention and they can still iterate and experiment, providing feedback to the automation team.

Final tips

One of the more important recommendations for this pattern is to avoid trying to create a “mandate” for all teams for adoption. The automation team provides capabilities, such as templated CI/CD pipelines, reusable components, and training materials, but it is still up to the individual teams to decide how to use them.

Conclusion

This is a pattern that works well at a larger organization, where it would be beyond the scope of a single group to be able to wrap their understanding around hundreds of services. However, it is a pattern that can be applied to smaller organizations as well!

Watch for the next post in this series, where we will dig into some other patterns! Or revisit the first post, Organizational Patterns - A Single Infra Repo.

Organizational Patterns: A Single Infra Repo

Matty Stratton — Fri, 17 Dec 2021 05:46:04 -0600

Using Pulumi is more than just writing code and components. In addition to common software development practices, there are also a number of success patterns related to how your company or team builds and deploys Pulumi programs to successfully build, deploy, and manage your infrastructure and applications. In this first post of a series, I will explore one of these patterns - the centralized platform infrastructure repository.

An emergent organizational pattern these days is that of a centralized “platform” team, which has various product and service teams (or squads) as internal customers. In this approach, the platform team takes responsibility for the tooling and infrastructure - if it’s not directly product related, it usually falls under the responsibility of the platform team. The platform team provides functionality and platforms to be consumed by the product teams.

For many of the examples I’ll be using to illustrate this pattern, I refer to a conversation I had recently with Jacob Foard, who is the Tech Lead for the Platform Team at GreenPark Sports. This pattern is used at GreenPark Sports, and he was very clear about the benefits of the pattern.

One of the key concepts to keep in mind is that when providing a platform, it is made up of more than just the compute and other resources provided by AWS, GCP, Azure, or even your own Kubernetes implementations. The platform also includes the infrastructure that is shared between the various teams, such as monitoring and observability tooling, version control/pipeline services, as well as secret and key management.

In this pattern, your main infrastructure repository is made up of directories for each product/service that your teams use, in addition to directories for each higher level shared service. Each of these directories is itself a Pulumi program. So it would look something like this:

├── bluth-apps
│   ├── apps
│   │   ├── apps.go
│   │   ├── bananastand.go
│   │   ├── suddenvalley.go
│   ├── main.go
│   ├── Pulumi.dev.yaml
│   └── Pulumi.prod.yaml
├── datadog
│   ├── main.go
│   └── Pulumi.prod.yaml
├── github
│   ├── main.go
│   └── Pulumi.prod.yaml
├── pkg
│   ├── datadog
│   ├── pagerduty
│   └── vault
└── .etc

In the above (fictional, but inspired by the GreenPark Sports pattern) example, the Bluth Company has two main services that are used in all of its environments (“Banana Stand” and “Sudden Valley”). The main apps.go file is the entry point that simply calls functions from each of the various apps to “set up” those apps, as well as the common infrastructure that an environment might require (networks, storage, etc). Note that the way you structure your code is up to you, and likely will vary depending upon the runtime for your particular Pulumi program, but this is the general idea.

Similarly, the github and datadog directories are Pulumi programs that are responsible for the “core” infrastructure for those services (perhaps creating roles, etc). The pkg directory is a directory that contains packages that are used by the other programs to implement that infrastructure. Again, the pkg convention is used by Go, but other runtimes will have a similar approach.

Examples

These examples are not complete runnable code, but used to illustrate the pattern. While these examples are using Go, they are written in a way that is compatible with any language that supports the Pulumi language.

main.go

package main

import (
 "github.com/bluthcompany/infra/bluth-apps/apps"
 "github.com/bluthcompany/infra/pkg/datadog"
 "github.com/bluthcompany/infra/pkg/github"
 "github.com/pulumi/pulumi/sdk/go/pulumi"
)

func main() {
 pulumi.Run(func(ctx *pulumi.Context) error {
 // Create the apps.
 apps.SetupApps(ctx, bananastand, suddenvalley)

 // Create the infrastructure.
 datadog.CreateInfrastructure(ctx)
 github.CreateInfrastructure(ctx)

 return nil
 })
}

apps.go

package apps
import (
 "github.com/pulumi/pulumi/sdk/go/pulumi"
)

func SetupApps(
 ctx *pulumi.Context,

 setupBananaStand(ctx)
 setupSuddenValley(ctx)

)

bananastand.go

package apps
// imports, etc
func setupBananaStand(ctx *pulumi.Context) {
 // Create the banana stand.
}

Setting up a new service in the platform repository

If a service/product team has a new service they want infrastructure for, they simply add a new myapp.go file to the apps directory for their service, and add it to the apps.go file to make sure it is called. This is then submitted as a pull request for the platform team to review.

One important part of this pattern is that the platform team does not want to be a “blocker” for the product and service teams. It’s key to make sure that you have more than one person able to review and merge these pull requests, and to add sufficient testing into your CI/CD pipeline for this infrastructure repository.

Variations on this pattern

In the fictional Bluth example, there is one Pulumi program that is used regardless of environment, and the different configurations are handled by the use of stacks. However, there are situations where you might have complex enough differences between your environments where the amount of conditionals you require in your code to handle this would make for very challenging maintenance and understanding of the code! This is the case with GreenPark Sports, so in their implementation, instead of a single bluth-apps directory at the root of the repo, you would instead have bluth-prod, bluth-dev, etc.

This approach does generate duplication of code, and it can provide challenges at scale, but it is up to you and your teams to determine the tradeoffs of the branching/conditional logic vs separate programs.

Conclusion

This pattern works well depending upon the makeup of your teams and services. It is a pattern that facilitates collaboration between teams and focuses on having a central platform team that enables product teams, rather than getting in their way.

Watch for the next posts in this series, where we will dig into some other patterns as well!