---
title: "ETL pipeline with Amazon Redshift and AWS Glue"
description: "An ETL pipeline with Amazon Redshift and AWS Glue"
url: "https://www.pulumi.com/dev/examples/aws-ts-redshift-glue-etl/"
image: "https://www.pulumi.com/assets/og/dev/examples/aws-ts-redshift-glue-etl.png"
---

# ETL pipeline with Amazon Redshift and AWS Glue

An ETL pipeline with Amazon Redshift and AWS Glue

- Source on GitHub: https://github.com/pulumi/examples/tree/master/aws-ts-redshift-glue-etl
- Deploy with Pulumi: https://app.pulumi.com/new?template=https%3A%2F%2Fgithub.com%2Fpulumi%2Fexamples%2Ftree%2Fmaster%2Faws-ts-redshift-glue-etl

## Get started with this example

This example lives in the [pulumi/examples](https://github.com/pulumi/examples/tree/master/aws-ts-redshift-glue-etl) repo. Pull down just this directory to follow along:

```bash
git clone --filter=blob:none --sparse https://github.com/pulumi/examples pulumi-examples
git -C pulumi-examples sparse-checkout set aws-ts-redshift-glue-etl
cd pulumi-examples/aws-ts-redshift-glue-etl
```

This example creates an ETL pipeline using Amazon Redshift and AWS Glue. The pipeline extracts data from an S3 bucket with a Glue crawler, transforms it with a Python script wrapped in a Glue job, and loads it into a Redshift database deployed in a VPC.

## Prerequisites

1. [Install Pulumi](https://www.pulumi.com/docs/get-started/install/).
1. [Install Node.js](https://www.pulumi.com/docs/intro/languages/javascript/).
1. Configure your [AWS credentials](https://www.pulumi.com/docs/intro/cloud-providers/aws/setup/).

### Deploying the App

1. Clone this repo, change to this directory, then create a new [stack](https://www.pulumi.com/docs/intro/concepts/stack/) for the project:

    ```bash
    pulumi stack init
    ```

1. Specify an AWS region to deploy into:

    ```bash
    pulumi config set aws:region us-west-2
    ```


1. Install Node dependencies and run Pulumi:

    ```bash
    npm install
    pulumi up
    ```

1. In a few moments, the Redshift cluster and Glue components will be up and running and the S3 bucket name emitted as a Pulumi [stack output](https://www.pulumi.com/docs/intro/concepts/stack/#outputs).

    ```bash
    ...
    Outputs:
        dataBucketName: "events-56e424a"
    ```

1. Upload the included sample data file to S3 to verify the automation works as expected:

    ```bash
    aws s3 cp events-1.txt s3://$(pulumi stack output dataBucketName)
    ```

1. When you're ready, destroy your stack and remove it:

    ```bash
    pulumi destroy --yes
    pulumi stack rm --yes
    ```