1. Fan-out SNS message processing to multiple SQS queues

    TypeScript

    To achieve fan-out SNS message processing to multiple SQS queues, we'll use AWS Simple Notification Service (SNS) Topics to distribute messages to multiple Amazon Simple Queue Service (SQS) queues. This pattern ensures that a message published to an SNS topic is delivered to multiple SQS queues subscribed to the topic.

    Here's how the process works:

    1. Create an SNS Topic.
    2. Create multiple SQS Queues.
    3. Subscribe each SQS Queue to the SNS Topic.
    4. Configure necessary permissions so the SNS Topic can send messages to the SQS Queues.

    Below is the Pulumi TypeScript program that sets up the resources:

    1. SNS Topic (aws.sns.Topic): This resource creates a new SNS topic that acts as the central publish point for messages.
    2. SQS Queues (aws.sqs.Queue): These resources create new SQS queues that will receive messages from the SNS topic.
    3. Topic Subscriptions (aws.sns.TopicSubscription): These resources create subscriptions to connect the SNS topic with the SQS queues.

    Let's implement this in code:

    import * as aws from "@pulumi/aws"; // Create an SNS topic const topic = new aws.sns.Topic("myTopic", {}); // Create two SQS queues const queue1 = new aws.sqs.Queue("myQueue1", {}); const queue2 = new aws.sqs.Queue("myQueue2", {}); // Subscribe SQS queues to the SNS topic const subscription1 = new aws.sns.TopicSubscription("mySubscription1", { topic: topic.arn, protocol: "sqs", endpoint: queue1.arn, rawMessageDelivery: true, }); const subscription2 = new aws.sns.TopicSubscription("mySubscription2", { topic: topic.arn, protocol: "sqs", endpoint: queue2.arn, rawMessageDelivery: true, }); // Export the names and ARNs of the SNS topic and SQS queues export const snsTopicName = topic.name; export const snsTopicArn = topic.arn; export const sqsQueue1Name = queue1.name; export const sqsQueue1Arn = queue1.arn; export const sqsQueue2Name = queue2.name; export const sqsQueue2Arn = queue2.arn;

    The program does the following:

    • Imports the AWS module from Pulumi's AWS package.
    • Creates an SNS topic named myTopic.
    • Creates two SQS queues named myQueue1 and myQueue2.
    • Creates subscriptions from each SQS queue to the SNS topic using the aws.sns.TopicSubscription resource, which links the topic ARN and the queue ARN.
    • Enables rawMessageDelivery to ensure that the message is delivered to the SQS queue as a raw string instead of as a JSON object with metadata.
    • Exports the names and ARNs of the created resources, which can be useful for referencing them outside the Pulumi program.

    Remember that you'll need the AWS Pulumi package installed in your project:

    npm install @pulumi/aws

    After setting this up, any message published to the SNS topic will be delivered to both SQS queues. This setup is used in fan-out architectures where you need to process the same message in multiple ways concurrently.

    Documentation:


GPT4o|Modeexplainbalancedcode