Using Datadog to collect logs from ECS containers via YAML

Question

Pulumi · Accepted Answer

To enable log collection from ECS containers using Datadog, you'll need to ensure that you have the Datadog Agent running on your ECS instances. This agent will forward logs to Datadog, where they can be aggregated and analyzed.

Here are the general steps we'll follow:

1. Ensure that the Datadog Agent is included in your ECS container definitions.
2. The Agent containers must have the necessary permissions and settings to retrieve logs from other containers on the same instance.
3. Provide a Datadog Agent configuration file (usually `datadog.yaml`) to enable log collection and specify other settings.

In the Pulumi program below, we'll create the necessary infrastructure on AWS ECS. However, consider that the specific details of your application and your environment may require adjustments to this code.

We'll define a new ECS Task Definition with the Datadog Agent container configured to collect logs. We'll assume that you've already set up an ECS cluster and a Datadog account with your API key available.

Now, let's write a Pulumi program in TypeScript to achieve this:

```typescript
import * as pulumi from '@pulumi/pulumi';
import * as aws from '@pulumi/aws';
// Assuming there is existing ECS Cluster with a name 'ecs-cluster'.
const ecsClusterName = 'ecs-cluster';

// Define the ECS Task Definition for the Datadog Agent
const datadogAgentTaskDefinition = new aws.ecs.TaskDefinition('datadog-agent', {
    family: 'datadog-agent',
    cpu: '256', // Define the CPU and Memory based on the requirements for your agent
    memory: '512',
    networkMode: 'bridge',
    requiresCompatibilities: ['EC2'], // Ensure that this is compatible with your ECS launch type
    executionRoleArn: aws.iam.Role.get('ecsExecutionRole', 'ecs-execution-role').arn,
    containerDefinitions: pulumi.output([{
        name: 'datadog-agent',
        image: 'datadog/agent:latest', // Use the latest Datadog Agent image
        cpu: 256,
        memory: 512,
        essential: true,
        environment: [{
            name: 'DD_API_KEY',
            value: 'your-datadog-api-key-here' // Replace with your actual DD_API_KEY
        },
        {
            name: 'ECS_FARGATE',
            value: 'true' // Set to true if using Fargate, otherwise remove
        }],
        mountPoints: [{
            sourceVolume: 'docker_sock',
            containerPath: '/var/run/docker.sock',
            readOnly: false,
        },
        {
            sourceVolume: 'proc',
            containerPath: '/host/proc',
            readOnly: true,
        },
        {
            sourceVolume: 'cgroup',
            containerPath: '/host/sys/fs/cgroup',
            readOnly: true,
        }],
        logConfiguration: {
            logDriver: 'awslogs',
            options: {
                awslogs-group: '/ecs/datadog-agent',
                awslogs-region: 'us-east-1', // replace with your region
                awslogs-stream-prefix: 'ecs',
            },
        },
    }]).apply(JSON.stringify),
});

// Define the mount points for the Datadog Agent container definition
const dockerSockVolume = new aws.ecs.Volume('dockerSockVolume', {
    name: 'docker_sock',
    hostPath: {
        path: '/var/run/docker.sock',
    },
});

const procVolume = new aws.ecs.Volume('procVolume', {
    name: 'proc',
    hostPath: {
        path: '/proc',
    },
});

const cgroupVolume = new aws.ecs.Volume('cgroupVolume', {
    name: 'cgroup',
    hostPath: {
        path: '/sys/fs/cgroup',
    },
});
```

In the above program, replace `your-datadog-api-key-here` with your actual Datadog API key. The `containerDefinitions` is a JSON string that configures the Datadog Agent container, including setting up the appropriate volume mounts and log configuration.

This configuration specifies that the Datadog Agent container should be connected to the Docker socket and have access to the `/proc` and `/sys/fs/cgroup` directories from the host. These are necessary for the Agent to collect telemetry from other containers.

Ensure you review the [Datadog Agent Docker documentation](https://www.pulumi.com/registry/packages/datadog/api-docs/servicedefinitionyaml/) for any additional configuration that may be relevant to your situation.

Lastly, deploy this program using the Pulumi CLI by running `pulumi up` in the directory containing your Pulumi code. This command will create the AWS resources defined in the program. You can check the status of the deployment and, upon completion, see the outputs that include the resource identifiers and any other exported properties.