1. Answers
  2. Configuring AWS MSK with ELB

How do I configure AWS MSK with ELB using Pulumi?

In this guide, we will configure an Amazon Managed Streaming for Apache Kafka (MSK) cluster and an Elastic Load Balancer (ELB) using Pulumi. This setup will allow us to manage our Kafka cluster and distribute traffic efficiently using the ELB.

Key Points

  • We will create an MSK cluster.
  • We will create a Target Group for the ELB.
  • We will create an Application Load Balancer (ALB) and attach the Target Group to it.
  • We will configure the necessary security groups and VPC settings.
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Create a VPC
const vpc = new aws.ec2.Vpc("vpc", {
    cidrBlock: "10.0.0.0/16",
    enableDnsHostnames: true,
    enableDnsSupport: true,
    tags: {
        Name: "pulumi-vpc",
    },
});

// Create subnets
const subnetA = new aws.ec2.Subnet("subnetA", {
    vpcId: vpc.id,
    cidrBlock: "10.0.1.0/24",
    availabilityZone: "us-west-2a",
    tags: {
        Name: "pulumi-subnet-a",
    },
});

const subnetB = new aws.ec2.Subnet("subnetB", {
    vpcId: vpc.id,
    cidrBlock: "10.0.2.0/24",
    availabilityZone: "us-west-2b",
    tags: {
        Name: "pulumi-subnet-b",
    },
});

// Create a security group for MSK
const mskSecurityGroup = new aws.ec2.SecurityGroup("mskSecurityGroup", {
    vpcId: vpc.id,
    description: "Allow all inbound traffic",
    ingress: [
        { protocol: "-1", fromPort: 0, toPort: 0, cidrBlocks: ["0.0.0.0/0"] },
    ],
    egress: [
        { protocol: "-1", fromPort: 0, toPort: 0, cidrBlocks: ["0.0.0.0/0"] },
    ],
    tags: {
        Name: "msk-security-group",
    },
});

// Create an MSK cluster
const mskCluster = new aws.msk.Cluster("mskCluster", {
    clusterName: "pulumi-msk-cluster",
    kafkaVersion: "2.6.0",
    numberOfBrokerNodes: 2,
    brokerNodeGroupInfo: {
        instanceType: "kafka.m5.large",
        clientSubnets: [subnetA.id, subnetB.id],
        securityGroups: [mskSecurityGroup.id],
    },
    tags: {
        Name: "pulumi-msk-cluster",
    },
});

// Create a Target Group for the ALB
const targetGroup = new aws.lb.TargetGroup("targetGroup", {
    port: 9092,
    protocol: "TCP",
    vpcId: vpc.id,
    targetType: "instance",
    healthCheck: {
        protocol: "TCP",
        port: "9092",
    },
    tags: {
        Name: "pulumi-target-group",
    },
});

// Create an Application Load Balancer
const alb = new aws.lb.LoadBalancer("alb", {
    internal: false,
    securityGroups: [mskSecurityGroup.id],
    subnets: [subnetA.id, subnetB.id],
    loadBalancerType: "application",
    tags: {
        Name: "pulumi-alb",
    },
});

// Create a listener for the ALB
const listener = new aws.lb.Listener("listener", {
    loadBalancerArn: alb.arn,
    port: 80,
    defaultActions: [{
        type: "forward",
        targetGroupArn: targetGroup.arn,
    }],
    tags: {
        Name: "pulumi-listener",
    },
});

// Export the ALB DNS name
export const albDnsName = alb.dnsName;

Conclusion

In this guide, we configured an AWS MSK cluster with an Elastic Load Balancer (ELB) using Pulumi. We set up the necessary VPC, subnets, security groups, and created both the MSK cluster and the ALB. Finally, we linked the ALB to the MSK cluster using a target group and a listener, and exported the ALB DNS name.

Deploy this code

Want to deploy this code? Sign up for a free Pulumi account to deploy in a few clicks.

Sign up

New to Pulumi?

Want to deploy this code? Sign up with Pulumi to deploy in a few clicks.

Sign up