Container Management with Pulumi

Manage Clusters and Deploy Containers with Ease

Pulumi supports managing clusters and their associated infrastructure, whether it is Kubernetes, Amazon ECS, Azure ACI, or Google GKE. Build and deploy application containers to private registies, all in one programming model.

Any code, any cloud, any language.

// Deploy Nginx to AWS Fargate
import * as awsx from "@pulumi/awsx";

let web = new awsx.elb.ApplicationLoadBalancer(
    "net-lb", { external: true }).
    createListener("web", { port: 80, external: true });

let appService = new awsx.ecs.FargateService("nginx-svc", {
    taskDefinitionArgs: {
        container: {
            image: "nginx",
            portMappings: [ web ],
        },
    },
    desiredCount: 5,
});

export let url = web.endpoint.hostname;

What is Container Management?

Docker, and containers more generally, transformed the way that developers could build, ship, and run applications. As a standardized unit of delivery, containers have all of the libraries, code, and runtime knowledge they need to work, and so can be deployed and scaled into any environment.

For DevOps teams, container management is an important capability whether delivering containers into bespoke environments, or more likely through a managed cloud offering such as AWS Fargate, AWS ECS, and Microsoft ACI, or through Kubernetes - either on-premises or AWS EKS, Microsoft AKS, or Google GKE.

Deploying containers with Pulumi

Container management solutions are broadly split into two options: low-management solutions for ease of deployment, and Kubernetes-based solutions for complete control.

Low Cluster Management

AWS Fargate, and Microsoft ACI are examples of services that enable the deployment of containers without having to manage servers or clusters. This means teams can very easily build container applications and deploy them into production without worrying about infrastructure, but gaining the benefits of seamless scaling.

Kubernetes Management

While running Kubernetes on-premises is a viable, and well-used solution, the major cloud vendors also offer Kubernetes sevices: Amazon EKS, Microsoft AKS, and Google GKE.

Pulumi makes it simple to interact with any of the services available from the major cloud vendors, and with Kubernetes.

Deploy Nginx to AWS Fargate

In this example, Pulumi defines and uses a new Amazon ECS Fargate cluster, and creates a load balanced service running the standard Nginx image from the Docker Hub. The same experience is available on other clouds and Pulumi can pull from any container registry.

Get Started

// Deploy Nginx to AWS Fargate
import * as awsx from "@pulumi/awsx";

let web = new awsx.elb.ApplicationLoadBalancer(
    "net-lb", { external: true }).
    createListener("web", { port: 80, external: true });

let appService = new awsx.ecs.FargateService("nginx-svc", {
    taskDefinitionArgs: {
        container: {
            image: "nginx",
            portMappings: [ web ],
        },
    },
    desiredCount: 5,
});

export let url = web.endpoint.hostname;

Deploying with a custom build

This example uses a trivial Dockerfile that derives from the nginx base image and copies the ./www directory into the nginx HTML target so that it will be served up.

Get Started

// Using a custom build based on Nginx
import * as awsx from "@pulumi/awsx";

let web = new awsx.elb.ApplicationLoadBalancer(
    "net-lb", { external: true }).
    createListener("web", { port: 80, external: true });

const appService = new awsx.ecs.FargateService("nginx-svc", {
    taskDefinitionArgs: {
        container: {
            image: awsx.ecs.Image.fromPath("app-img", "./www");
            portMappings: [ web ],
        },
    },
    desiredCount: 5,
});

export const url = web.endpoint.hostname;

// Dockerfile
FROM nginx
COPY ./www /usr/share/nginx/html

Creating a Kubernetes cluster

Pulumi can provision Kubernetes clusters — in this example, an AWS EKS cluster — in addition to deploying application-level configuration, using a standard set of languages, abstractions, and tools.

Get Started

import * as awsx from "@pulumi/awsx";
import * as eks from "@pulumi/eks";

// Create a VPC for our cluster.
const vpc = new awsx.ec2.Vpc("vpc", {});

// Create the EKS cluster itself.
const cluster = new eks.Cluster("cluster", {
    vpcId: vpc.id,
    subnetIds: vpc.publicSubnetIds,
    instanceType: "t2.medium",
    desiredCapacity: 4,
    minSize: 3,
    maxSize: 5,
    storageClasses: "gp2",
    deployDashboard: true,
});

// Export the cluster's kubeconfig.
export const kubeconfig = cluster.kubeconfig;

Deploy containers to Microsoft ACI

The @pulumi/azure-native library provides fine-grained control of Azure resources. In this example, we deploy a simple linux container to Microsoft ACI, in the West US zone.

Get Started

import * as containerinstance from "@pulumi/azure-native/containerinstance";
import * as resources from "@pulumi/azure-native/resources";

const resourceGroup = new resources.ResourceGroup("resourcegroup", {
    location: "West US",
});

const imageName = "mcr.microsoft.com/azuredocs/aci-helloworld";
const containerGroup = new containerinstance.ContainerGroup("containerGroup", {
    resourceGroupName: resourceGroup.name,
    osType: "Linux",
    containers: [{
        name: "acilinuxpublicipcontainergroup",
        image: imageName,
        resources: {
            requests: {
                cpu: 1.0,
                memoryInGB: 1.5,
            },
        },
        ports: [{ port: 80 }],
    }],
    ipAddresses: [{
        ports: [{
          port: 80,
          protocol: "Tcp",
        }],
        type: "Public",
    }],
    restartPolicy: "always",
});

Invoke a long-running container as a task

This example shows a container used for executing a long-running task. Here, we use a container to perform a thumbnail extraction on a piece of video uploaded to an S3 bucket.

Get Started

import * as aws from "@pulumi/aws";
import * as awsx from "@pulumi/awsx";

// A bucket to store videos and thumbnails.
const videos = new aws.s3.Bucket("bucket");

// A task which runs a containerized FFMPEG job to extract a thumbnail image.
const ffmpegThumbnailTask = new awsx.ecs.FargateTaskDefinition("ffmpegThumbTask", {
    container: {
        image: awsx.ecs.Image.fromPath("ffmpegThumbTask", "./docker-ffmpeg-thumb"),
        memoryReservation: 512,
    },
});

// When a new video is uploaded, run the FFMPEG task on the video file.
videos.onObjectCreated("onNewVideo",
    new aws.lambda.CallbackFunction<aws.s3.BucketEvent, void>("onNewVideo", {
        // Specify appropriate policies so that this AWS lambda can run EC2 tasks.
        policies: [
            aws.iam.ManagedPolicy.AWSLambdaExecute,
            aws.iam.ManagedPolicy.AmazonECSFullAccess,
        ],
        callback: async bucketArgs => {
            for (const record of bucketArgs.Records) {
                const file = record.s3.object.key;
                const thumbnailFile = file.substring(0, file.indexOf('_')) + '.jpg';
                const framePos = file.substring(file.indexOf('_')+1, file.indexOf('.')).replace('-',':');
                await ffmpegThumbnailTask.run({
                    overrides: {
                        containerOverrides: [{
                            name: "container",
                            environment: [
                                { name: "S3_BUCKET", value: bucketName.get() },
                                { name: "INPUT_VIDEO", value: file },
                                { name: "TIME_OFFSET", value: framePos },
                                { name: "OUTPUT_FILE", value: thumbnailFile },
                            ],
                        }],
                    },
                });
            }
        },
    }), { filterSuffix: ".mp4" });

exports.bucketName = videos.bucket;