Simple Serverless programming with Google Cloud Functions and Pulumi

Pulumi lets you create, deploy, and manage Google Cloud applications and infrastructure in familiar languages like JavaScript, TypeScript, and Python, and without needing to learn new DSLs or YAML templating solutions. This means great productivity and getting to use tools you already know and love. Since serverless is all about focusing more on your application code, and less on infrastructure and configuration toil, we absolutely love Google Functions.

The Simplest Way to Serverless

In fact, serverless has never been simpler than it is when you combine Pulumi with Google Cloud Functions. Want to serve a simple HTTP API with no fixed costs? It’s just a few lines of code – and no, we’re not hiding any YAML here:

import * as gcp from "@pulumi/gcp";

let greeting = new gcp.cloudfunctions.HttpCallbackFunction("greeting", (req, res) => {
    // Change this code to fit your needs!
    res.send(`Greetings from ${req.body.name || 'Google Cloud Functions'}!`);
});

export let url = greeting.httpsTriggerUrl;

Thanks to Pulumi’s language heritage, we know how to take that function, serialize it, and publish it to a Google Bucket and Object, eliminating those tedious manual steps behind a single pulumi up CLI invocation.

Or perhaps a pubsub topic that runs some custom code on every message received. This sounds like a perfect use case for an event-driven style of programming, so why don’t we write it that way!

// Create a PubSub Topic
let requests = new gcp.pubsub.Topic("requests");
requests.onMessagePublished("newMessage", (data) => {
    // Print out a log message for every message on the Topic.
    // Change this code to fit your needs!
    console.log(Buffer.from(data.data, "base64").toString());
});

Because Pulumi knows how to manage infrastructure as code, it can provision and manage not only the Google Cloud Function, but also the pubsub topic resource itself, for each creation and management.

Or perhaps, we’d like a function that respond to any uploads of new objects to your storage bucket:

// Create a Storage Bucket
let bucket = new gcp.storage.Bucket("data");
bucket.onObjectFinalized("newobject", (data) => {
    // Print out a log message for every upload to the Bucket.
    // Change this code to fit your needs!
    console.log(`New file uploaded: ${data.name}`);
});

A Complete Google Functions Slack Bot

For an idea of how you might fit these together to make a real-world cloud application, let’s look a simple skeleton structure for a Slack Bot using Pulumi and Google Cloud Functions:

// secure config tokens to use to validate incoming messages as well as authenticate ourself to slack
const config = new pulumi.Config("mentionbot");
const slackToken = config.get("slackToken");
const verificationToken = config.get("verificationToken");

// A topic that we can enqueue slack events to so they can be processed in batch later on
const messageTopic = new gcp.pubsub.Topic("messages");

// Create an http endpoint that slack will use to push events to us.
const endpoint = new gcp.cloudfunctions.HttpCallbackFunction("bot", {
    callbackFactory: () => {
        const app = express();
        app.use(bodyParser.json());
        app.post("/events", (req, res) => {
            // Importantly: This is the code that will run in your serverless GCP cloud function!

            const body = req.body;

            // Process the body as appropriate. If it's something we need to respond to immediately
            // (like a verification request), then do so. Otherwise, add the message to our pubsub
            // topic to be processed later:
            const pubSub = new PubSub();
            const topic = pubSub.topic(messageTopic.name.get());
            topic.publish(Buffer.from(JSON.stringify(body)));

            // Quickly respond with success so that slack doesn't retry.
            res.status(200).end();
        });

        return app;
    }
});

messageTopic.onMessagePublished("processTopicMessage", async (data) => {
    // Actually handle the 'data' in the pubsub message.
    // Importantly: This is the code that will run in your serverless GCP cloud function!
});

// Give this url to slack to let them know where to post their events to.
export const url = endpoint.httpsTriggerUrl;

This is the real code for a complete SlackBot application running on GCP, from the cloud resources to the serverless code, all within a unified Pulumi application! Customizing this for your own use case is as simple as changing the code in the two JavaScript arrow-functions.

To see the complete project, take a look at our @mentionbot example. That example will listen for mentions of your name and will notify you of them in a channel of your choosing, giving you you a persistent timeline you can go back to look at to make sure you can find all these messages.

Although it’s a simple example, there are a lot of moving parts this takes care of that you would normally be responsible for. This includes:

1. Figuring out the shape (the input/output-types) for your Cloud Functions, and then creating an appropriate program that exports the right entrypoint that matches. In this case, because we’ve exposed the right abstractions (like HttpCallbackFunction and Topic.onMessagePublished), the arrow-functions you pass in will all have the right types, and your program will be typechecked by TypeScript.
2. Creating separate Cloud Functions for each serverless callback. One for listening and responding to the initial Slack events and the second for processing the messages in the Topic. Here, you can write a single Pulumi App where all the code can be placed how you like it (in this case in a single file).
3. Packaging each callback up in the appropriate structure Cloud Functions expects, including how to get all your dependencies in place.
4. Creating a Storage Bucket for all of your packaged programs to live in.
5. Uploading each packaged program to a Bucket Object in that bucket.
6. Configuring the appropriate triggers on your Cloud Functions stating how they should be triggered (for example, in response to an HTTP trigger or a Pub/Sub trigger).
7. Including the right information in the function so you can interact with your other cloud resources in the Pulumi App. Without this, you would need to find a way to include that data in each Cloud Function’s environment variables (or just hardcode them in ‘1’) so that your program can access the rest of your cloud infrastructure. In the above example, you can see how you can just reference your resources directly (like the PubSub Topic) directly from your Cloud Function callback. Pulumi makes sure this all works, and that the data you use is available in that Cloud Function when it finally is triggered.
8. Figuring out a safe and secure way to encode and access secrets for your Cloud Function. Here, we can use Pulumi’s Config Secrets to safely encrypt and manage secrets for your Cloud Function code.

Not to mention that by doing all of that, you can achieve continuous deployment Pulumi and Google Cloud Build.

Updating Your Google Functions Code

This is a lot to figure out to manage cloud applications and infrastructure. If you want to tweak things even slightly you might need to go make many manual changes and updates to ensure everything is properly updated. With Pulumi, all this complexity is handled with a single update! Just run pulumi up , and it figures out the rest.

Let’s just take a look at what Pulumi does when you tweak a few things in the program:

$ pulumi up
Updating (slackbot):
     Type                                       Name                        Status       Info
     pulumi:pulumi:Stack                        slackbot
 +-  ├─ gcp:pubsub:Topic                        messages                    replaced     [diff: +labels]
     │  └─ gcp:cloudfunctions:CallbackFunction  processTopicMessage
 +-  │     ├─ gcp:storage:BucketObject          processTopicMessage         replaced     [diff: ~name,source]
 ~   │     └─ gcp:cloudfunctions:Function       processTopicMessage         updated      [diff: ~eventTrigger,sourceArchiveObject]
     └─ gcp:cloudfunctions:CallbackFunction     mentionbot
 +-     ├─ gcp:storage:BucketObject             mentionbot                  replaced     [diff: ~name,source]
 ~      └─ gcp:cloudfunctions:Function          mentionbot                  updated      [diff: ~sourceArchiveObject]
 
Outputs:
    url: "https://***.cloudfunctions.net/mentionbot"
 
Resources:
    ~ 2 updated
    +-3 replaced
    2 changes. 5 unchanged
 
Duration: 52s

One command later, and your entire stack is updated properly!

Winding Down

The cloud provides tremendous potential, and we want to make it easy for developers to tap into that potential. Using Pulumi, it’s easy to blur the line between business logic and serverless resources, bringing the focus back to the problems you care about. One codebase that’s easy to create, update, and maintain.

To check things out, get started today:

• Get Started with Pulumi on GCP
• Deploy a Minimal Google Cloud Function Application

PS: If you’re interested in how Pulumi manages to take a JavaScript/TypeScript => function and somehow analyze and transform it into a form that Cloud Functions can use, please see our deep dive on this topic in: Lambdas as Lambdas: The magic of simple serverless Functions. We’re leveraging the same great programming language and analysis framework to power our GCP solution here.