1. Deploy the elastic-filebeat helm chart on Google Kubernetes Engine (GKE)


    To deploy the Elastic Filebeat Helm chart on Google Kubernetes Engine (GKE), you will need to follow these steps:

    1. Set up a GKE Cluster: First, you'll set up a Kubernetes cluster in GKE where your Filebeat Helm chart will be deployed.
    2. Install and Configure Helm in your Pulumi Program: You'll need to install the Helm CLI tool and configure it to deploy charts to your GKE cluster.
    3. Deploy the Filebeat Helm Chart: Deploy the Filebeat chart from the Elastic Helm repository to your GKE cluster using Pulumi's Kubernetes provider.

    Below is a Pulumi program written in TypeScript that performs these steps. The program first creates a GKE cluster, then uses the Pulumi Kubernetes provider to deploy the Filebeat Helm chart to the cluster.

    Let's start by installing the necessary Pulumi packages for the GCP and Kubernetes providers:

    # Install the Pulumi GCP package $ npm install @pulumi/gcp # Install the Pulumi Kubernetes package $ npm install @pulumi/kubernetes

    Now, we'll create the Pulumi program that provisions the GKE cluster and deploys the Filebeat Helm chart to it:

    import * as gcp from "@pulumi/gcp"; import * as k8s from "@pulumi/kubernetes"; import * as pulumi from "@pulumi/pulumi"; // Create a GKE cluster. const cluster = new gcp.container.Cluster("my-gke-cluster", { initialNodeCount: 2, nodeConfig: { machineType: "n1-standard-1", oauthScopes: [ "https://www.googleapis.com/auth/compute", "https://www.googleapis.com/auth/devstorage.read_only", "https://www.googleapis.com/auth/logging.write", "https://www.googleapis.com/auth/monitoring" ], }, }); // Export the Cluster name export const clusterName = cluster.name; // Export the Kubeconfig to access the cluster using kubectl export const kubeconfig = pulumi. all([ cluster.name, cluster.endpoint, cluster.masterAuth ]). apply(([ name, endpoint, masterAuth ]) => { const context = `${gcp.config.project}_${gcp.config.zone}_${name}`; return `apiVersion: v1 clusters: - cluster: certificate-authority-data: ${masterAuth.clusterCaCertificate} server: https://${endpoint} name: ${context} contexts: - context: cluster: ${context} user: ${context} name: ${context} current-context: ${context} kind: Config preferences: {} users: - name: ${context} user: auth-provider: config: cmd-args: config config-helper --format=json cmd-path: gcloud expiry-key: '{.credential.token_expiry}' token-key: '{.credential.access_token}' name: gcp `; }); // Create a Kubernetes provider instance that uses our cluster from above. const k8sProvider = new k8s.Provider("my-k8s-provider", { kubeconfig: kubeconfig, }); // Deploy Filebeat Helm chart to the GKE cluster using the Helm Provider. const filebeatChart = new k8s.helm.v3.Chart("filebeat", { chart: "filebeat", version: "7.14.0", // specify the version of the chart you wish to deploy fetchOpts:{ repo: "https://helm.elastic.co", }, // Set the necessary Filebeat configuration values here. values: { // For example, specify which Elasticsearch instance to use: // elasticsearchHosts: "http://your-elastic-instance:9200" }, }, { provider: k8sProvider }); // Export the Filebeat Chart status export const filebeatStatus = filebeatChart.status;

    In the program above:

    • We use @pulumi/gcp to create a new GKE cluster.
    • We construct the kubeconfig needed to communicate with the GKE cluster.
    • We create a Pulumi Kubernetes provider that uses this kubeconfig.
    • We instantiate a new Helm Chart resource, specifying the filebeat Helm chart to be deployed from the Elastic Helm chart repository.
    • We export the status of the Filebeat deployment and the kubeconfig, which you can later use with kubectl to interact with your GKE cluster.

    To run this Pulumi program:

    1. Save the code to a file with a .ts extension, for instance, index.ts.
    2. Ensure you are authenticated with GCP and have set up the necessary permissions.
    3. Run pulumi up to preview and deploy the changes.

    After deployment, Filebeat will start shipping logs to the specified output, which is typically an Elasticsearch instance.

    Please replace placeholder values like your-elastic-instance with the actual values for your ElasticSearch URL and adjust any configurations in the values section according to your requirements. Make sure to specify the version of the Filebeat Helm chart that you want to deploy; here it is set to 7.14.0 as an example.