Using kubernetes kustomize.toolkit.fluxcd.io with elasticsearch.k8s.elastic.co

Question

Pulumi · Accepted Answer

To use Kubernetes with the Kustomize Toolkit from FluxCD alongside Elastic's Elasticsearch operator, you would need to:

Bootstrap FluxCD in your Kubernetes Cluster: This initializes your cluster with the necessary components to pull configuration from a Git repository. This is done using the flux.FluxBootstrapGit resource which applies a series of Kubernetes manifests to install the Flux components.
Define a Kustomize Configuration: After bootstrapping Flux, you can define a Kustomization resource pointing to a directory within your synchronized Git repository, which includes the Elasticsearch operator and instance manifests. Use the kubernetes.kustomize.Directory resource to apply these manifests via Kustomize tooling.
Deploy Elasticsearch: You will need to include the Elasticsearch operator and instance manifests in your Git repository. The Elasticsearch operator (elasticsearch.k8s.elastic.co) is typically installed through its Helm chart or pre-made manifests, but it can also be included as part of your Kustomize overlay.

Below is a program written in TypeScript which you can use as a foundation. This program assumes that you have a Kubernetes cluster up and running and that you have a Git repository with the necessary Kustomize configurations for Elasticsearch:

import * as k8s from "@pulumi/kubernetes";
import * as flux from "@pulumi/flux";

// Initialize the Flux CD on your Kubernetes cluster
const fluxBootstrap = new flux.FluxBootstrapGit("flux-bootstrap", {
    // Specify the Git repository URL
    url: "<YOUR_GIT_REPO_URL>",
    // Define the branch to track changes
    branch: "main",
    // Set the path where to look for the manifest files in the repository
    path: "./clusters/my-cluster",
    // The namespace where Flux components should be installed
    namespace: "flux-system",
    // Define the synchronization interval
    interval: "1m",
    // The log level for Flux components
    logLevel: "info",
    // If you're using a private registry, you need to set it here
    registry: "ghcr.io",
    // Set to true to watch all namespaces (optional)
    watchAllNamespaces: true,
});

// Apply the Kustomize overlay for ElasticSearch located in your Git repository
const elasticsearchKustomization = new k8s.kustomize.Directory("elasticsearch-kustomization", {
    directory: "./kustomize/elasticsearch",
    // Optionally, apply transformations to customize the resources further
});

// Export the name of the namespace that was created
export const namespaceName = fluxBootstrap.status.apply(status => status.namespace);

Replace <YOUR_GIT_REPO_URL> with your actual Git repository URL and adjust the paths as needed for your setup.

Explanation:

flux.FluxBootstrapGit: This resource installs the Flux components into your Kubernetes cluster. It requires details about your Git repository, such as the URL, branch, and path. Flux will watch for changes in the specified path of the repository and synchronize those with your cluster.
k8s.kustomize.Directory: This Pulumi resource is used to apply a Kustomize configuration from a directory. This is especially useful for managing complex applications in Kubernetes. In our case, we're using it to apply the Kustomize configuration that includes the Elasticsearch operator and instance.

Note: The actual setup of your Elasticsearch operator and instance should be done in the referenced directory (./kustomize/elasticsearch), using standard Kustomize techniques.

Make sure your Git repository is structured in a way that reflects the paths given in the code sample, and that the Kustomize directory contains all necessary Kubernetes objects to deploy and configure Elasticsearch.

After deploying this Pulumi program, ensure that you monitor the FluxCD logs inside your Kubernetes cluster to track the synchronization process and handle any errors that might occur during the application of your Kustomize configuration.

Keep in mind that managing complex cluster configurations often requires a sound understanding of Kubernetes, Kustomize, and the specific operators you are using. The above program gives you a head start in automating the delivery of such configurations.