How do I create interactive log analysis dashboards with Grafana and Loki?

In this guide, we will create interactive log analysis dashboards using Grafana and Loki on Kubernetes. Grafana is a popular open-source platform for monitoring and observability, while Loki is a log aggregation system designed to work with Grafana. We will deploy both Grafana and Loki on a Kubernetes cluster using Pulumi.

Steps:

Provision a Kubernetes cluster: We will assume you already have a Kubernetes cluster running.
Deploy Loki: We will deploy Loki as a log aggregation system.
Deploy Grafana: We will deploy Grafana to visualize logs collected by Loki.
Configure Grafana to use Loki: We will configure Grafana to use Loki as a data source.

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

// Define the Loki deployment
const lokiDeployment = new k8s.apps.v1.Deployment("loki-deployment", {
    spec: {
        selector: { matchLabels: { app: "loki" } },
        replicas: 1,
        template: {
            metadata: { labels: { app: "loki" } },
            spec: {
                containers: [{
                    name: "loki",
                    image: "grafana/loki:2.4.1",
                    ports: [{ containerPort: 3100 }],
                    args: [
                        "-config.file=/etc/loki/local-config.yaml"
                    ],
                    volumeMounts: [{
                        name: "config",
                        mountPath: "/etc/loki"
                    }]
                }],
                volumes: [{
                    name: "config",
                    configMap: {
                        name: "loki-config"
                    }
                }]
            }
        }
    }
});

// Define the Loki service
const lokiService = new k8s.core.v1.Service("loki-service", {
    spec: {
        selector: { app: "loki" },
        ports: [{ port: 3100, targetPort: 3100 }]
    }
});

// Define the Loki config map
const lokiConfigMap = new k8s.core.v1.ConfigMap("loki-config", {
    metadata: { name: "loki-config" },
    data: {
        "local-config.yaml": `
auth_enabled: false

server:
  http_listen_port: 3100

ingester:
  lifecycler:
    address: 127.0.0.1
    ring:
      kvstore:
        store: inmemory
    final_sleep: 0s
  chunk_idle_period: 5m
  chunk_retain_period: 30s
  max_transfer_retries: 0

schema_config:
  configs:
  - from: 2020-10-24
    store: boltdb-shipper
    object_store: filesystem
    schema: v11
    index:
      prefix: index_
      period: 168h

storage_config:
  boltdb_shipper:
    active_index_directory: /loki/index
    cache_location: /loki/index_cache
    shared_store: filesystem
  filesystem:
    directory: /loki/chunks

limits_config:
  enforce_metric_name: false
  reject_old_samples: true
  reject_old_samples_max_age: 168h

chunk_store_config:
  max_look_back_period: 0s

table_manager:
  retention_deletes_enabled: false
  retention_period: 0s
`
    }
});

// Define the Grafana deployment
const grafanaDeployment = new k8s.apps.v1.Deployment("grafana-deployment", {
    spec: {
        selector: { matchLabels: { app: "grafana" } },
        replicas: 1,
        template: {
            metadata: { labels: { app: "grafana" } },
            spec: {
                containers: [{
                    name: "grafana",
                    image: "grafana/grafana:8.3.3",
                    ports: [{ containerPort: 3000 }],
                    env: [{
                        name: "GF_SECURITY_ADMIN_PASSWORD",
                        value: "admin"
                    }]
                }]
            }
        }
    }
});

// Define the Grafana service
const grafanaService = new k8s.core.v1.Service("grafana-service", {
    spec: {
        selector: { app: "grafana" },
        ports: [{ port: 80, targetPort: 3000 }]
    }
});

// Export the URLs to access Grafana and Loki
export const grafanaUrl = grafanaService.status.loadBalancer.ingress[0].hostname;
export const lokiUrl = lokiService.status.loadBalancer.ingress[0].hostname;

Key Points:

Loki Deployment: We deploy Loki using a Kubernetes Deployment resource, specifying the necessary container image and configuration.
Loki Service: We expose Loki using a Kubernetes Service resource.
Grafana Deployment: We deploy Grafana using a Kubernetes Deployment resource, setting the admin password via environment variables.
Grafana Service: We expose Grafana using a Kubernetes Service resource.
Configuration: We use a ConfigMap to store Loki’s configuration.

Summary:

In this guide, we deployed Grafana and Loki on a Kubernetes cluster using Pulumi. We configured Loki to aggregate logs and Grafana to visualize them. The services were exposed to allow access to Grafana and Loki. This setup provides an interactive log analysis dashboard using Grafana and Loki.

Deploy this code

Want to deploy this code? Sign up for a free Pulumi account to deploy in a few clicks.

New to Pulumi?

Want to deploy this code? Sign up with Pulumi to deploy in a few clicks.