How do I automate Pub/Sub messaging on GCS object change events?

In this guide, we will automate the process of sending Google Cloud Pub/Sub messages whenever there are changes to objects in a Google Cloud Storage (GCS) bucket. This involves creating a GCS bucket, a Pub/Sub topic, and configuring the bucket to send notifications to the Pub/Sub topic on object change events.

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

// Create a GCS bucket
const bucket = new gcp.storage.Bucket("my-bucket", {
    location: "US",
});

// Create a Pub/Sub topic
const topic = new gcp.pubsub.Topic("my-topic");

// Create a notification configuration to send events from the GCS bucket to the Pub/Sub topic
const bucketNotification = new gcp.storage.Notification("my-bucket-notification", {
    bucket: bucket.name,
    topic: topic.id,
    eventTypes: [
        "OBJECT_FINALIZE", // Trigger on object creation and updates
        "OBJECT_DELETE", // Trigger on object deletions
    ],
    payloadFormat: "JSON_API_V1",
});

// Export the bucket name and topic name
export const bucketName = bucket.name;
export const topicName = topic.name;

Key Points

• GCS Bucket: A storage bucket where objects will be stored.
• Pub/Sub Topic: A messaging service that allows you to send and receive messages between independent applications.
• Bucket Notification: Configuration that triggers Pub/Sub messages on specific events in the GCS bucket.

Summary

We created a GCS bucket and a Pub/Sub topic. Then, we configured the bucket to send notifications to the Pub/Sub topic whenever objects in the bucket are created, updated, or deleted. This setup allows you to automate the process of sending Pub/Sub messages based on changes to objects in your GCS bucket.