How do I create serverless data processing endpoints with AWS Lambda and API Gateway?

In this guide, we’ll create serverless data processing endpoints using AWS Lambda and API Gateway. These technologies allow you to run code in response to HTTP requests without provisioning or managing servers.

Explanation

We’ll define:

An AWS Lambda function to handle the data processing.
An API Gateway to provide HTTP endpoints that trigger the Lambda function.

Let’s dive into the details:

AWS Lambda Function: This is where your processing logic will reside. We’ll write a simple Lambda function in Python.
API Gateway: This creates a RESTful API with resources and methods. It connects the HTTP requests to the Lambda function.

The Program

import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";

// IAM role that grants the Lambda function necessary permissions
const lambdaExecRole = new aws.iam.Role("lambda_exec_role", {
    name: "lambda_exec_role",
    assumeRolePolicy: JSON.stringify({
        Version: "2012-10-17",
        Statement: [{
            Action: "sts:AssumeRole",
            Effect: "Allow",
            Sid: "",
            Principal: {
                Service: "lambda.amazonaws.com",
            },
        }],
    }),
});
// IAM role policy attachment for basic execution
const lambdaPolicyAttach = new aws.iam.RolePolicyAttachment("lambda_policy_attach", {
    role: lambdaExecRole.name,
    policyArn: "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole",
});
// Lambda function resource
const dataProcessingLambda = new aws.lambda.Function("data_processing_lambda", {
    name: "dataProcessingFunction",
    role: lambdaExecRole.arn,
    runtime: aws.lambda.Runtime.Python3d8,
    handler: "lambda_function.lambda_handler",
    code: `import json

def lambda_handler(event, context):
    body = {
        "message": "Data processed successfully!",
        "input": event
    }

    return {
        "statusCode": 200,
        "body": json.dumps(body)
    }
`,
});
// API Gateway REST API
const api = new aws.apigateway.RestApi("api", {
    name: "DataProcessingAPI",
    description: "API Gateway to trigger Lambda function",
});
// API Gateway resource
const resource = new aws.apigateway.Resource("resource", {
    restApi: api.id,
    parentId: api.rootResourceId,
    pathPart: "process",
});
// API Gateway method
const method = new aws.apigateway.Method("method", {
    restApi: api.id,
    resourceId: resource.id,
    httpMethod: "POST",
    authorization: "NONE",
});
// API Gateway integration with Lambda function
const integration = new aws.apigateway.Integration("integration", {
    restApi: api.id,
    resourceId: resource.id,
    httpMethod: method.httpMethod,
    type: "AWS_PROXY",
    integrationHttpMethod: "POST",
    uri: dataProcessingLambda.invokeArn,
});
// API Gateway deployment
const deployment = new aws.apigateway.Deployment("deployment", {
    restApi: api.id,
    stageName: "v1",
}, {
    dependsOn: [integration],
});
export const apiUrl = deployment.invokeUrl;

Key Points

IAM Role: Allows Lambda to interact with AWS services.
Lambda Function: Contains code to handle data processing.
API Gateway: Provides a method (POST) to trigger the Lambda function.
Integration & Deployment: Links API Gateway to the Lambda function and deploys the API.

Summary

In this example, we created a simple serverless application using AWS Lambda and API Gateway. The Lambda function processes incoming HTTP requests via the API Gateway endpoint. This setup is scalable and cost-effective for building APIs without managing servers.

Deploy this code

Want to deploy this code? Sign up for a free Pulumi account to deploy in a few clicks.

New to Pulumi?

Want to deploy this code? Sign up with Pulumi to deploy in a few clicks.