How do I use Splunk with Amazon EC2?

Introduction

In this guide, we will walk you through the process of integrating Splunk with your Amazon EC2 instances. Splunk is a widely used platform for searching, monitoring, and analyzing machine-generated data. By sending logs from your EC2 instances to Splunk, you can gain valuable insights and enhance your infrastructure’s overall monitoring and alerting capabilities.

We’ll cover the following steps:

1. Setting up the EC2 instances.
2. Installing the Splunk Universal Forwarder on the instances.
3. Configuring the Splunk Universal Forwarder to send data to your Splunk instance.

Key Resources

• AWS EC2 Instance: These are the virtual machines that will run your applications and generate logs.
• Splunk Universal Forwarder: This lightweight version of Splunk collects and forwards log data to your Splunk server.

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const splunkEc2 = new aws.ec2.Instance("splunk_ec2", {
    ami: "ami-0c55b159cbfafe1f0",
    instanceType: aws.ec2.InstanceType.T2_Micro,
    tags: {
        Name: "Splunk-EC2-Instance",
    },
    userData: `#!/bin/bash
yum update -y
yum install -y wget
cd /opt
wget -O splunkforwarder-8.2.1-be11b2c94a98-Linux-x86_64.rpm 'https://www.splunk.com/page/download_track?file=8.2.1/linux/splunkforwarder-8.2.1-be11b2c94a98-Linux-x86_64.rpm&ac=&wget=true&name=wget&platform=default&architecture=default&version=default&product=splunk&typed=releases'
rpm -i splunkforwarder-8.2.1-be11b2c94a98-Linux-x86_64.rpm
/opt/splunkforwarder/bin/splunk start --accept-license
/opt/splunkforwarder/bin/splunk enable boot-start
/opt/splunkforwarder/bin/splunk add forward-server <splunk-server-ip>:9997 -auth admin:changeme
/opt/splunkforwarder/bin/splunk add monitor /var/log
`,
});
export const instanceId = splunkEc2.id;
export const publicIp = splunkEc2.publicIp;

Key Points

• Provider Configuration: The configuration sets up the AWS provider to interface with your specified region.
• EC2 Instance: An instance is created with a specified Amazon Machine Image (AMI), type, and tags.
• User Data Script: This script runs on instance launch to:
  • Install necessary updates and tools.
  • Download and install the Splunk Universal Forwarder.
  • Configure the Splunk Forwarder to start automatically, forward logs to a Splunk server, and monitor /var/log directory.

Summary

In this guide, we created an Amazon EC2 instance and installed Splunk Universal Forwarder to forward log data to a Splunk server, enabling enhanced monitoring and logging capabilities for your infrastructure.