1. Answers
  2. Analyzing AWS ECS Container Logs with Splunk's Docker Logging Driver

How do I analyze AWS ECS container logs with Splunk's Docker logging driver?

In this guide, we will configure an Amazon ECS service to forward its container logs to Splunk using Docker’s logging driver. We’ll create an ECS cluster, task definition, and service, and configure Docker logging to send logs to Splunk.

Step-by-Step Setup

  1. Define the ECS Cluster: This is where your container instances will run.
  2. Create the Task Definition: This specifies the Docker containers to run, along with their configurations.
  3. Set Up the ECS Service: This maintains the specified number of task instances.
  4. Configure the Docker Logging Driver: Set the logging parameters to forward logs to Splunk.

Here’s an example of how to set this up:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const example = new aws.ecs.Cluster("example", {name: "example-cluster"});
const exampleTaskDefinition = new aws.ecs.TaskDefinition("example", {
    family: "example-task",
    networkMode: "awsvpc",
    requiresCompatibilities: ["FARGATE"],
    cpu: "256",
    memory: "512",
    containerDefinitions: JSON.stringify([{
        name: "example-app",
        image: "nginx:latest",
        essential: true,
        logConfiguration: {
            logDriver: "splunk",
            options: {
                "splunk-token": "your-splunk-token",
                "splunk-url": "https://splunk-server:8088",
                "splunk-index": "main",
                "splunk-insecureskipverify": "true",
            },
        },
    }]),
});
const exampleVpc = new aws.ec2.Vpc("example", {cidrBlock: "10.0.0.0/16"});
const exampleSubnet = new aws.ec2.Subnet("example", {
    vpcId: exampleVpc.id,
    cidrBlock: "10.0.1.0/24",
});
const exampleSecurityGroup = new aws.ec2.SecurityGroup("example", {
    vpcId: exampleVpc.id,
    ingress: [{
        fromPort: 80,
        toPort: 80,
        protocol: "tcp",
        cidrBlocks: ["0.0.0.0/0"],
    }],
    egress: [{
        fromPort: 0,
        toPort: 0,
        protocol: "-1",
        cidrBlocks: ["0.0.0.0/0"],
    }],
});
const exampleService = new aws.ecs.Service("example", {
    name: "example-service",
    cluster: example.id,
    taskDefinition: exampleTaskDefinition.arn,
    desiredCount: 1,
    launchType: "FARGATE",
    networkConfiguration: {
        subnets: [exampleSubnet.id],
        securityGroups: [exampleSecurityGroup.id],
    },
});
export const clusterName = example.name;
export const serviceName = exampleService.name;

This setup creates an ECS cluster, a Fargate task definition with a specified container running Nginx, and configures Docker logging to send container logs to a Splunk instance. The ECS service ensures that the task runs in the cluster. We also set up a VPC, subnet, and security group for networking.

In conclusion, we configured AWS ECS to use Docker’s Splunk logging driver, enabling you to analyze container logs in Splunk for better observability and debugging.

Deploy this code

Want to deploy this code? Sign up for a free Pulumi account to deploy in a few clicks.

Sign up

New to Pulumi?

Want to deploy this code? Sign up with Pulumi to deploy in a few clicks.

Sign up