---
title: "ChatGPT clone with Pulumi ESC managed secrets"
description: "Let's make a ChatGPT clone, using Pulumi! This tutorial will walk you through creating a web app that interacts with the OpenAI API. You'll use the Pulumi CLI to manage the configuration and secrets needed to access the OpenAI API. We'll also show how to use the ESC SDK to dynamically update your configuration and secrets."
url: "https://www.pulumi.com/dev/tutorials/esc-managed-chatapp/"
image: "https://www.pulumi.com/assets/og/dev/tutorials/esc-managed-chatapp.png"
---
# ChatGPT clone with Pulumi ESC managed secrets
Let's make a ChatGPT clone, using Pulumi! This tutorial will walk you through creating a web app that interacts with the OpenAI API. You'll use the Pulumi CLI to manage the configuration and secrets needed to access the OpenAI API. We'll also show how to use the ESC SDK to dynamically update your configuration and secrets.
## What you'll learn
- How to build an OpenAI-based ChatGPT clone
- How to create an environment with the Pulumi CLI
- How to manage configuration and secrets with the Pulumi CLI
- How to project your ESC configuration as environment variables
- How to dynamically refresh configuration and secrets without restart
## Prerequisites
- Familiarity with Python
- Familiarity with OpenAI API (optional)
- Python 3.7 or newer
- A [Pulumi Cloud account](https://app.pulumi.com/signup) and an [access token](https://www.pulumi.com/docs/administration/access-identity/access-tokens/)
- An OpenAI account w/ some credits
- The [Pulumi CLI](https://www.pulumi.com/docs/install/)
- A Bash-like UNIX shell environment
## What are we building
This tutorial will walk you through making an OpenAI-based ChatGPT clone using Python.
The app will be implemented as a [Flask](https://flask.palletsprojects.com/en/3.0.x/) web app, which makes calls to the [`completions`](https://platform.openai.com/docs/guides/chat-completions) endpoint of the [OpenAI API](https://platform.openai.com/docs/overview).
_Figure: Architecture of the chat app_
The app has two important variables to manage: the OpenAI API Key, and the choice of model that the OpenAI backend uses.
## Three ways of managing secrets and configuration
After we get a basic version of the app running, we'll modify the app two times, to show three different ways of managing our secrets and configurations; unmanaged, managed static configuration, and managed dynamic configuration.
The first version of our app will be _unmanaged_ and will have hardcoded settings for the OpenAI model we're using, reading it from the default environment variable `OPENAI_API_KEY` for the API key.
The second version will be _statically managed_ using Pulumi ESC, providing a custom set of environment variables to the app at runtime.
The third version will be _dynamically managed_ via the Pulumi ESC SDK to reload the configuration, so that there's no downtime required to reconfigure the web app.
_Figure: Three different versions of the code, using different configuration methods_
## Creating the ChatApp
> **Note**
>
> **Before you start:** Make sure you have an [OpenAI account](https://platform.openai.com/docs/quickstart) with enough credits to test your app.
### Step 1: Create a OpenAI project, and service account API key
First we'll need to create an OpenAI project called `chatapp`.
_Figure: OpenAI Create Project dialog_
Next, create a service account API key. You can follow the [OpenAI Quickstart](https://platform.openai.com/docs/quickstart) docs for details.
_Figure: OpenAI Create API Key dialog._
Next, copy the key to your clipboard, and use the following command to set the `OPENAI_API_KEY` environment variable:
```bash
$ export OPENAI_API_KEY=`pbpaste`
```
> **Caution**
>
> **Shell environments may differ:** This tutorial is assuming a Bash-like shell in a MacOS environment where the `pbcopy` and `pbpaste` tools are available by default.
>
> You may need to install a similar tool for your environment to avoid pasting the API key text in plain-text into your shell (and by proxy, into your shell command history).
>
> Two such utilities are `xclip` and `xsel`, which are available in the default repositories of most Linux distributions.
### Step 2: Install dependencies with pip.
Now lets setup our working environment. Create a new directory called `chatapp`.
```bash
$ mkdir chatapp
$ cd chatapp
```
Next create a Python virtual environment:
```bash
$ python -m venv .venv
```
And then install the `flask` and `openai` packages
```bash
pip install flask openai
```
### Step 3: Create the webapp
Finally, lets create the code files for our chat app. We'll need two files `chatapp.py`, the backend Flask app, and `templates/chatapp.html`, an HTML template that will render the UI. You can copy and paste these from here, into your favorite code editor and save.
**Code Listing:** `chatapp.py`
```python
import os
from flask import Flask, render_template, request, jsonify
from openai import OpenAI
app = Flask(__name__)
def get_openai_response(api_key, model, prompt):
client = OpenAI(api_key=api_key)
try:
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
except Exception as e:
return f"An error occurred: {str(e)}"
@app.route('/')
def home():
return render_template('chatapp.html')
@app.route('/chat', methods=['POST'])
def chat():
model = "gpt-3.5-turbo"
api_key = os.getenv("OPENAI_API_KEY")
prompt = request.json['message']
response = get_openai_response(api_key, model, prompt)
return jsonify({'response': response, 'model': model})
if __name__ == '__main__':
app.run(debug=True)
```
**Code Listing:** `templates/chatapp.html`
```html
OpenAI Chat App
OpenAI Chat App
```
The Flask app sets up two routes:
- `/`: serves the UI template contained in `templates/chatapp.html`
- `/chat`: uses the function `get_openai_response` to query the OpenAI API
It will read the key from the `OPENAI_API_KEY` environment variable and chose the OpenAI model based on the hardcoded value `gpt-3.5-turbo` which is defined in the `chat` route.
Finally, let's run the app and make sure it works.
```bash
$ python chatapp.py
```
You should see your app running on [http://127.0.0.1:5000](http://127.0.0.1:5000). Go ahead and try it out:
_Figure: The chat app web UI_
## ChatApp V2: Statically managed configuration and secrets using Pulumi ESC
Well that was fun! We now have a [ChatGPT](https://openai.com/chatgpt/) clone. However, there are some operational issues that we need to address. How do we manage the API key more securely? Right now we're just copying and pasting into shell environment variables. That's not great. Since we can only copy it once from OpenAI's console, if we somehow lose that environment variable (e.g. close the shell window), we will have to go generate an entirely new key.
Also, what if we want to change which model we're using? We'd have to change the value hardcoded into the app... every time!
What if we want to share the API key with others, or run this in some environment other than our developer's laptop?
Lets get these settings under management. Pulumi ESC to the rescue!
> **Note**
>
> **Before you start:** Make sure you have the [Pulumi CLI installed](https://www.pulumi.com/docs/iac/download-install/) and a [Pulumi Cloud account](https://app.pulumi.com/signup).
### Step 1: Create an ESC environment for our app
First, the `pulumi` CLI needs to be logged in to your Pulumi Cloud account.
```bash
$ pulumi login
```
Next, lets create an ESC environment called `chatapp`:
```bash
$ pulumi env init default/chatapp
```
You can verify that the `chatapp` environment exists by listing your available environments:
```bash
$ pulumi env ls
default/chatapp
```
> **Caution**
>
> **Project Naming:** If your project name is something other than `default`, make sure these commands, and others the throughout this tutorial, are modified to use the name of your project.
Now let's create the configuration settings. First we'll create a configuration value called `model` which has the [ID of the model](https://platform.openai.com/docs/models) we want to use in the OpenAI API:
```bash
$ pulumi env set default/chatapp model gpt-3.5-turbo
```
Then we'll create a secret called `api_key` that holds the OpenAI API Key. Notice the `--secrets` flag which indicates that this is a secret that we want to be encrypted, both at rest, and in transit. We're sourcing the value for this from the existing `OPENAI_API_KEY` environment variable (to make sure we don't end up with a plain-text version of this in our shell's command history).
```bash
$ pulumi env set default/chatapp api_key $OPENAI_API_KEY --secret
```
Finally, let's verify the ESC environment we just created. We can view the environment using the `pulumi env get` subcommand.
```bash
$ pulumi env get default/chatapp
```
You should see output that looks something like this:
```bash
Value
{
"api_key": "[secret]",
"model": "gpt-4o-mini"
}
Definition
values:
model: gpt-4o-mini
api_key:
fn::secret:
ciphertext: ######################
##################################
##################################
```
You can see that `api_key` is a secret and so it won't show us the value. In the definition display, it shows us the encrypted version of the key, but not the actual key value. This will be decrypted at runtime, waiting until the last possible moment to pass the actual value to the code that needs it.
### Step 2: Update the app to use Pulumi ESC settings
The next step is to get these ESC environment values accessible to our app. For this we're going to extract these configuration values and project them as two environment variables named `MODEL` and `API_KEY`.
To do this, we will [edit the YAML definition](https://www.pulumi.com/docs/esc/concepts/environments/#managing-environments) of our ESC environment. The `pulumi env edit` subcommand will load the YAML into your default text editor. Editing and saving the file will update it in Pulumi ESC's backend storage.
> **Note**
>
> **YAML Schema:** A full description of what Pulumi ESC YAML supports is available in the [ESC YAML Syntax Reference](https://www.pulumi.com/docs/esc/reference/).
```bash
$ pulumi env edit default/chatapp
```
To this YAML file we'll add an `environmentVariables:` section, which defines the names of the environment variables we want to set in our program's shell environment, and which ESC configuration values those map to.
```yaml
values:
model: gpt-3.5-turbo
api_key:
fn::secret:
ciphertext: ######################
##################################
##################################
environmentVariables:
MODEL: ${model}
API_KEY: ${api_key}
```
Now we need to modify `chatapp.py` to use the new ESC-provided environment variables in the `/chat` route:
**Code Listing:** Modify Flask `/chat` route in `chatapp.py` _(version 2)_
```python
@app.route('/chat', methods=['POST'])
def chat():
# Set configuration from ESC provided environment variables
model = os.getenv('MODEL')
api_key = os.getenv('API_KEY')
prompt = request.json['message']
response, model = get_openai_response(api_key, model, prompt)
return jsonify({'response': response, 'model': model})
```
Now let's run the app, with ESC providing the variables, to make sure it works.
```bash
$ pulumi env run default/chatapp -- python chatapp.py
```
As before, check that the app is running at [http://127.0.0.1:5000](http://127.0.0.1:5000) and able to get responses from the OpenAI API.
This command `pulumi env run -- ` will launch the provided command in a subshell, which will have the environment variables that we defined in the `environmentVariables` section of our ESC environment's YAML. Note that we're no longer relying on the `OPENAI_API_KEY` variable for our secret, rather, we're pull that from the Pulumi ESC environment. Similarly, we can now change our model from the ESC environment `model` setting, which means we can easily update the configuration without having to modify the code.
## Dynamically retrieving ESC configuration at runtime
The last step worked great to get our configuration and secrets under management,
but there still some downsides. What happens if the configuration or API key needs to change? Right now the app sets the configuration when it starts and always uses the same values the entire time it's running. The app would need to be restarted to get new settings. This would cause downtime for our service, just to update the key, or change the model we're using.
We'd rather not take downtime, so let's update the app's code to use the ESC SDK to dynamically fetch settings from ESC every time the `/chat` endpoint is used.
### Step 1: Install the Pulumi ESC SDK
First we need to add the `pulumi-esc-sdk` and `pyyaml` packages:
```bash
$ pip install pulumi-esc-sdk pyyaml
```
### Step 2: Update the ESC environment settings
In order to use the Pulumi ESC SDK from within our Python code, we will need to create an ESC client, and call the `open_and_read_environment` method. The client will need a [Pulumi Cloud access token](https://www.pulumi.com/docs/pulumi-cloud/access-management/access-tokens/), and some configuration values defining the organization, project, and environment name to pull our settings from.
First, edit your ESC environment to contain the following YAML code:
```bash
$ pulumi env edit default/chatapp
```
**Code Listing:** YAML representation of `default/chatapp` environment
```yaml
values:
model: gpt-3.5-turbo
api_key:
fn::secret:
ciphertext: ########################
##################################
##################################
esc_org: ${context.pulumi.organization.login}
esc_project: "default"
esc_environment: ${context.currentEnvironment.name}
environmentVariables:
ESC_ORG: ${esc_org}
ESC_PROJECT: ${esc_project}
ESC_ENVIRONMENT: ${esc_environment}
---
```
Here we have added three new ESC values `esc_org`, `esc_project`, and `esc_environment`, with the organization and environment pulled from the [ESC execution context](https://www.pulumi.com/docs/esc/concepts/interpolations-and-references/#context) directly. These are projected into the `ESC_ORG`, `ESC_PROJECT`, and `ESC_ENVIRONMENT` environment variables respectively.
> **Caution**
>
> **Project Naming:** The project name is not able to be derived from context, so we have to use a static value here. If your project name is something other than `default`, make sure to set this value to the name of your project.
We've also removed the environment variable projections for `API_KEY` and `MODEL` since we won't be accessing them that way anymore.
Next we will need to ensure that the `PULUMI_ACCESS_TOKEN` variable exists in our shell. If you're an active Pulumi user or have been going through other tutorials, you may already have this set in your shell. Let's check for that using Bash's `env` command:
```bash
$ env | grep PULUMI_ACCESS_TOKEN
PULUMI_ACCESS_TOKEN=#########
```
If this returns nothing, you'll need to [create an access token](https://www.pulumi.com/docs/pulumi-cloud/access-management/access-tokens/):
_Figure: Create an access token in Pulumi Cloud_
Copy that value to the clipboard, and paste it securely into your shell:
```bash
$ export PULUMI_ACCESS_TOKEN=`pbpaste`
```
### Step 3: Modify `chatapp.py` code to fetch settings dynamically
Finally, let's modify the `chatapp.py` to use these new values and pull the settings dynamically.
```python
import os
import pulumi_esc_sdk as esc
from flask import Flask, render_template, request, jsonify
from openai import OpenAI
app = Flask(__name__)
# Set up the ESC client to manage secrets
escOrg = os.getenv("ESC_ORG")
escProjectName = os.getenv("ESC_PROJECT")
escEnvironment = os.getenv("ESC_ENVIRONMENT")
def get_openai_response(api_key, model, prompt):
client = OpenAI(api_key=api_key)
try:
response = client.chat.completions.create(
model=model,
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
except Exception as e:
return f"An error occurred: {str(e)}"
@app.route('/')
def home():
return render_template('chatapp.html')
@app.route('/chat', methods=['POST'])
def chat():
# Fetch most recent configuration from ESC
esc_client = esc.esc_client.default_client()
_, values, _ = esc_client.open_and_read_environment(escOrg, escProjectName, escEnvironment)
# Access the configuration values and set the local variables
model = values['model']
api_key = values['api_key']
prompt = request.json['message']
response = get_openai_response(api_key, model, prompt)
return jsonify({'response': response, 'model': model})
if __name__ == '__main__':
app.run(debug=True)
```
This last set of changes does a few things:
- imports the Pulumi ESC SDK
- pulls the client configuration values from the ESC provided environment variables and sets up the `esc.Configuration` object
- creates an ESC client and modifies the `/chat` endpoint to read from the ESC environment every time it is called
This will ensure that the app always gets the most recent values, without the need to restart!
With those changes in place, lets run the app and make sure it works:
```bash
$ pulumi env run default/chatapp -- python chatapp.py
```
As before, check that the app is running at [http://127.0.0.1:5000](http://127.0.0.1:5000) and able to get responses from the OpenAI API.
### Step 4: Manage the settings while the app is running
Let's leave the chat app running and open up another shell window.
In order to test the dynamic nature of the configuration we'll modify the ESC environment at runtime and look for the changes to appear in our running app. We'll change the `model` setting and use the more efficient and up-to-date model `gpt-4o-mini`.
```bash
$ pulumi env set default/chatapp model gpt-4o-mini
```
Try querying the chat app and you should see that the model has been updated. This time, no restart required!
_Figure: The chatapp web UI is now using the gpt-4o-mini model_
## Conclusion
Pulumi ESC and the ESC SDK can be used in many ways to manage your configurations. But this just the tip of the iceberg!
Check out some of our other tutorials to see how you can version your ESC environments and use environment tags to update entire sets of configuration values and secrets with a simple drag and drop interface in the Pulumi Cloud console.
You can also do even more complicated things like manage the configuration of Kubernetes, and apps running within a Kubernetes cluster at the same time, with a single tool.
### More ESC tutorials:
- [Secure Kubernetes Application Secrets with Pulumi ESC](https://www.pulumi.com/dev/tutorials/secure-kubernetes-app-secrets-with-esc/)