Storing Application Logs in a MinIO S3 Bucket

Introduction

In this guide, we will set up a MinIO S3 bucket to store application logs using Pulumi. MinIO is a high-performance, S3-compatible object storage system. We will use Pulumi to provision the necessary infrastructure on AWS.

Step-by-Step Explanation

Step 1: Set Up Pulumi Project

First, ensure you have the Pulumi CLI installed and configured. Create a new Pulumi project using TypeScript:

pulumi new aws-typescript

Step 2: Install Dependencies

Install the required Pulumi packages:

npm install @pulumi/pulumi @pulumi/aws @pulumi/awsx

Step 3: Configure AWS Provider

Ensure your AWS credentials are configured. You can set them up using the AWS CLI:

aws configure

Step 4: Create an S3 Bucket

Create an S3 bucket to store the application logs:

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

const bucket = new aws.s3.Bucket("logBucket", {
    acl: "private",
});

Step 5: Configure Bucket Policy

Set up a bucket policy to allow MinIO to write logs to the bucket:

const bucketPolicy = new aws.s3.BucketPolicy("bucketPolicy", {
    bucket: bucket.bucket,
    policy: bucket.bucket.apply(bucketName => JSON.stringify({
        Version: "2012-10-17",
        Statement: [
            {
                Effect: "Allow",
                Principal: "*",
                Action: [
                    "s3:PutObject",
                    "s3:PutObjectAcl"
                ],
                Resource: [
                    `arn:aws:s3:::${bucketName}/*`
                ]
            }
        ]
    }))
});

Step 6: Deploy the Infrastructure

Deploy the stack to create the S3 bucket and apply the policy:

pulumi up

Step 7: Configure MinIO

Set up MinIO to use the S3 bucket for log storage. This involves configuring the MinIO server with the appropriate AWS credentials and bucket information. Refer to the MinIO documentation for detailed steps on this configuration.

Conclusion

By following these steps, you have successfully set up a MinIO S3 bucket to store application logs using Pulumi. This setup ensures that your logs are securely stored in an S3-compatible storage system, leveraging the scalability and reliability of AWS S3.

Full Code Example

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// Create an S3 bucket to store the application logs
const bucket = new aws.s3.Bucket("logBucket", {
    acl: "private",
});

// Set up a bucket policy to allow MinIO to write logs to the bucket
const bucketPolicy = new aws.s3.BucketPolicy("bucketPolicy", {
    bucket: bucket.bucket,
    policy: bucket.bucket.apply(bucketName => JSON.stringify({
        Version: "2012-10-17",
        Statement: [
            {
                Effect: "Allow",
                Principal: "*",
                Action: [
                    "s3:PutObject",
                    "s3:PutObjectAcl"
                ],
                Resource: [
                    \`arn:aws:s3:::\${bucketName}/*\`
                ]
            }
        ]
    }))
});