What is the method for collecting metrics from Amazon EKS automatically using Amazon Managed Service for Prometheus in TypeScript

Question

Pulumi · Accepted Answer

To collect metrics from an Amazon Elastic Kubernetes Service (EKS) cluster using Amazon Managed Service for Prometheus (AMP), you would typically follow these steps:

1. **Create an AMP Workspace:** AMP workspaces are isolated environments to store your Prometheus monitoring data. 
2. **Deploy Prometheus:** You would then set up Prometheus to run in your EKS cluster. Prometheus is an open-source monitoring system that collects metrics from specified targets by scraping metrics HTTP endpoints on these targets.
3. **Configure Prometheus to send metrics to AMP:** You'll need to configure Prometheus to remote-write metrics to your AMP workspace.

We'll walk through a Pulumi TypeScript program to automate the creation of an AMP workspace. Then, you would configure Prometheus to send those metrics to the AMP workspace. It includes setting up the necessary IAM policies and roles for Prometheus to write data to AMP.

Here's the TypeScript example that uses Pulumi to:

- Create an AMP workspace.
- Set up an IAM role and policy allowing Prometheus to remote-write metrics to the AMP workspace.

Before running this program, ensure you have installed the AWS Pulumi provider and you are authenticated with your AWS account. For your Prometheus deployment on EKS to use AMP, you will need to configure the Prometheus server with the right settings to use the AMP workspace, which can be done after this Pulumi program sets up the initial infrastructure.

```typescript
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Create an AMP workspace
const workspace = new aws.amp.Workspace("myWorkspace", {});

// IAM policy for Prometheus to write to AMP workspace
const prometheusPolicy = new aws.iam.Policy("prometheusPolicy", {
    description: "IAM policy for Prometheus to write to AMP workspace",
    policy: pulumi.interpolate`{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Action": [
                    "aps:RemoteWrite",
                    "aps:QueryMetrics",
                    "aps:GetSeries",
                    "aps:GetLabels",
                    "aps:GetMetricMetadata"
                ],
                "Effect": "Allow",
                "Resource": "${workspace.arn}"
            }
        ]
    }`,
});

// IAM role for Prometheus with the above policy attached
const prometheusRole = new aws.iam.Role("prometheusRole", {
    assumeRolePolicy: `{
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Principal": {
                    "Service": "eks.amazonaws.com"
                },
                "Action": "sts:AssumeRole"
            }
        ]
    }`,
});

// Attach the policy to the role
const rolePolicyAttachment = new aws.iam.RolePolicyAttachment("rolePolicyAttachment", {
    role: prometheusRole.name,
    policyArn: prometheusPolicy.arn,
});

// Export the workspace ID for use in Prometheus configuration
export const workspaceId = workspace.id;

```

This program will create the necessary AWS resources for integrating Prometheus with AMP, and it will export the workspace ID, which you should use to configure Prometheus in your EKS cluster.

After you've run this Pulumi program successfully, you'll need to update your Prometheus configuration to use the AMP workspace for remote-write. Please refer to [Amazon Managed Service for Prometheus documentation](https://docs.aws.amazon.com/prometheus/latest/userguide/AMP-onboard-ingest-metrics.html) for further instructions on how to configure the Prometheus server.

To deploy Prometheus with these configurations inside your EKS cluster, you can use Helm charts, Kubernetes Operator, or any other preferred method. This will not be covered in this program, but please ensure that Prometheus is appropriately configured to send metrics to the newly created AMP workspace.