1. Answers
  2. Deploying Kafka Topics in Kubernetes

How do I deploy a Kubernetes kafka.strimzi.io kafkatopic with Pulumi?

This guide demonstrates how to deploy a Kafka topic in a Kubernetes cluster using Pulumi and the Strimzi operator. Strimzi simplifies running Apache Kafka on Kubernetes, and Pulumi makes it easy to manage your infrastructure as code.

Key Points:

  • Use the kubernetes.core.v1.Namespace resource to create a namespace.
  • Deploy the Strimzi Kafka operator using kubernetes.apiextensions.CustomResource.
  • Create a Kafka cluster using kubernetes.apiextensions.CustomResource.
  • Define a Kafka topic using kubernetes.apiextensions.CustomResource.

Below is the Pulumi program in TypeScript to achieve this:

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

// Create a namespace for Strimzi and Kafka resources
const kafkaNamespace = new k8s.core.v1.Namespace("kafka-namespace", {
    metadata: {
        name: "kafka",
    },
});

// Deploy the Strimzi Kafka operator
const strimziOperator = new k8s.apiextensions.CustomResource("strimzi-operator", {
    apiVersion: "apiextensions.k8s.io/v1",
    kind: "CustomResourceDefinition",
    metadata: {
        name: "strimzi-operator",
        namespace: kafkaNamespace.metadata.name,
    },
    spec: {
        group: "kafka.strimzi.io",
        versions: [{ name: "v1beta1", served: true, storage: true }],
        scope: "Namespaced",
        names: {
            plural: "kafkas",
            singular: "kafka",
            kind: "Kafka",
            shortNames: ["k"],
        },
    },
});

// Create a Kafka cluster
const kafkaCluster = new k8s.apiextensions.CustomResource("kafka-cluster", {
    apiVersion: "kafka.strimzi.io/v1beta1",
    kind: "Kafka",
    metadata: {
        name: "my-cluster",
        namespace: kafkaNamespace.metadata.name,
    },
    spec: {
        kafka: {
            version: "2.8.0",
            replicas: 3,
            listeners: [{ name: "plain", port: 9092, type: "internal" }],
            storage: {
                type: "ephemeral",
            },
        },
        zookeeper: {
            replicas: 3,
            storage: {
                type: "ephemeral",
            },
        },
    },
});

// Define a Kafka topic
const kafkaTopic = new k8s.apiextensions.CustomResource("kafka-topic", {
    apiVersion: "kafka.strimzi.io/v1beta1",
    kind: "KafkaTopic",
    metadata: {
        name: "my-topic",
        namespace: kafkaNamespace.metadata.name,
        labels: {
            strimziCluster: kafkaCluster.metadata.name,
        },
    },
    spec: {
        partitions: 3,
        replicas: 3,
    },
});

Summary

In this guide, we created a namespace for Kafka resources, deployed the Strimzi Kafka operator, created a Kafka cluster, and defined a Kafka topic. This setup allows you to manage Kafka topics within your Kubernetes cluster using Pulumi.

Deploy this code

Want to deploy this code? Sign up for a free Pulumi account to deploy in a few clicks.

Sign up

New to Pulumi?

Want to deploy this code? Sign up with Pulumi to deploy in a few clicks.

Sign up