Pulumi Blog: AI Slack Bot Series

Uploading documents to your AI Slackbot in real-time using S3, SQS and Pulumi on AWS

Tyler Mulligan — Mon, 03 Jun 2024 17:21:02 +0100

In the introductory blog post, we learned to Create an AI Slack Bot to Chat with Your Data Using Embedchain, Pulumi on AWS, and continued with Adding data to Pinecone using S3, Embedchain, and Pulumi on AWS for an AI Slack bot.

For reference, here’s what our architecture looked like at the end of the second blog post.

To follow along, clone the project, git clone https://github.com/catmeme/arti.git or view it on GitHub.

S3 Events and SQS

AWS provides a few ways to integrate with AWS S3 Events, for this project, we’re going to focus on the latest and most straight-forward, S3 Event Notifications. Depending on your use case, integrating with EventBridge may make more sense.

While we can integrate S3 events directly with Lambda, what happens when we add 100’s of files at the same time? Potentially, hundreds of Lambdas could be invoked simultaneously, which could become an issue depending on other factors. We’ll solve for this by Using Lambda with Amazon SQS, and reserved concurrency.

Our Pulumi program is updated to add a Queue and integrate it with our new Lambda SQS handler (policies left out for brevity, you can view them in the deploy/pulumi/__main__.py in the project).

# Add a queue
sqs_queue = aws.sqs.Queue(...)

# Add a docker image runtime for our Lambda SQS handler
docker_image_lambda_sqs_handler = docker.Image(...)

# Add Lambda SQS handler
app_sqs_lambda = aws.lambda_.Function(
 resource_name=f"{app_name}-sqs-lambda",
 environment=lambda_environment, # We've added the sqs_queue.url here as well
 image_uri=docker_image_lambda_sqs_handler.repo_digest,
 opts=pulumi.ResourceOptions(depends_on=[docker_image_lambda_sqs_handler]),
 **common_lambda_options,
)

# Add event source mapping
event_source_mapping = aws.lambda_.EventSourceMapping(
 f"{app_name}-lambda-sqs-event-source-mapping",
 event_source_arn=sqs_queue.arn,
 function_name=app_sqs_lambda.name,
 enabled=True,
 batch_size=10,
)

Add the application code responsible for processing S3 events coming from SQS.

def handler(event, context):
 for record in event.get("Records", []):
 s3_event = json.loads(record.get("body", "{}"))
 for s3_record in s3_event.get("Records", []):
 object_key = s3_record.get("s3", {}).get("object", {}).get("key")
 if object_key:
 logger.info("Processed %s in bucket %s.", object_key, app_bucket_name)
 load_data(asset_location=f"{app_bucket_name}/{object_key}")

 return {"statusCode": 200, "body": json.dumps("Successfully processed S3 event(s) from SQS messages.")}

We also have a scheduled event that we set up for ingestion in the last post, which we’re going to disable in favor of real-time events.

That’s pretty much it on the infrastructure side, here’s the updated diagram.

Slack to S3 file upload bridge

So what do we do with this new functionality? How do we expose it to the user? The most obvious approach is to reuse our existing Slack integration.

We’ll start by creating a Slack file upload bridge to S3, the bot watches for file_shared events on Slack and re-uploads files to our S3 bucket.

Using Slack Bolt, we add a new function handle_file_upload with the @app.event("file_shared") decorator, binding it to said event. When that event occurs, it’s processed, and the details of our file are retrieved, a new payload is constructed using requests, and the file gets uploaded to S3, this is our bridge.

@app.event("file_shared")
def handle_file_uploads(event, say, client, logger):
 file_id = event['file']['id']
 channel_id = event['channel_id']

 # Fetch file details using the Slack API
 file_info = client.files_info(token=slack_bot_token, file=file_id)['file']
 file_url = file_info['url_private_download']

 # Download file content from Slack
 headers = {'Authorization': f"Bearer {slack_bot_token}"}
 response = requests.get(file_url, headers=headers, stream=True)

 if response.status_code == 200:
 # Upload the file to S3
 s3_client.upload_fileobj(
 Fileobj=response.raw,
 Bucket=os.getenv("APP_BUCKET_NAME"),
 Key=file_info['name']
 )

 # Fetch recent messages to find the correct timestamp
 messages = client.conversations_history(channel=channel_id, limit=10)['messages']
 for msg in messages:
 if msg.get('files') and any(f['id'] == file_id for f in msg.get('files', [])):
 # Found the message with the correct file
 message_ts = msg['ts']
 client.reactions_add(channel=channel_id, name="white_check_mark", timestamp=message_ts)
 # Respond in a thread to the message containing the file
 say(text=f"File `{file_info['name']}` uploaded to S3 successfully! :tada:", thread_ts=message_ts)
 return

 logger.error("No valid message found for reaction.")

Keep in mind, we’re not doing any validation here yet. We’re blindly copying files from Slack to S3 and having our data loader process them. It’s probably not a good idea to deploy this bot to a public Slack channel without more moderation/validation in place.

Here’s what it looks like when we drag/drop a file into Slack now. On the right, a file was shared with the bot in private message. The bot acknowledged this upload with the checkmark reaction, the job is placed on the queue. Shortly after, on the left, the SQS Lambda handler has finished processing the uploaded file. We’re now able to use the bot to talk about the uploaded files.

Slack interactivity

In minutes, we were able to move from a scheduled task for processing files in a bucket, to real-time file processing integrated with Slack. We have our /arti command we can use to interrogate the documents we just uploaded, but we’re going to have a little more fun.

Thanks to Slack Block Kit Builder, we can build a form that exposes all of the configuration options for the query method in Embedchain’s app, providing the user with more options for interacting with the RAG model.

Our code to utilize the JSON we’ve crafted in the builder looks something like (we’ll call this /smarti):

@app.command("/smarti")
def open_smarti_modal(ack, body, client):
 ack()
 print("Opening SMARTI modal...")
 modal_file_path = Path(config.project_root) / 'config' / 'slack_modals' / 'chat.json'

 with open(modal_file_path, 'r') as json_file:
 modal_view = json.load(json_file)

 client.views_open(
 trigger_id=body["trigger_id"],
 view=modal_view
 )

And the code for handling that submission looks something like:

def handle_smarti_submission(client, logger, body, view):
 values = view["state"]["values"]
 question = values["question"]["input"]["value"]
 model = values["model"]["input"]["selected_option"]["value"]
 temperature = float(values["temperature"]["input"]["value"])
 max_tokens = int(values["max_tokens"]["input"]["value"])
 top_p = float(values["top_p"]["input"]["value"])
 prompt = values["prompt"]["input"]["value"]
 system_prompt = values["system_prompt"]["input"]["value"]
 citation_options = values["citations"]["input"].get("selected_options", [])
 citations = any(option['value'] == 'true' for option in citation_options)

 ai_response = ask_ai(
 question=question,
 model=model,
 temperature=temperature,
 max_tokens=max_tokens,
 top_p=top_p,
 prompt=prompt,
 system_prompt=system_prompt,
 citations=citations
 )
 ...

Submitting that form results in a response such as below. The bot honored our prompts, talking like Shakespeare, a thread is created, and citations were noted, showing where the data was retrieved from.

Lastly, if we want interact with the bot more publicly, then we can use the app_mention event.

@app.event("app_mention")
def handle_app_mention(event, say, client):
 text = event['text']
 bot_user_id = client.auth_test()["user_id"]

 # Strip the mention of the bot to isolate the command and arguments
 command_text = text.split(f'<@{bot_user_id}>')[1].strip() if f'<@{bot_user_id}>' in text else text

 # Parse the command and arguments
 parts = command_text.split(maxsplit=1)
 if not parts:
 say("I need a command to do something useful!")
 return

 # Get the command and the remainder of the message as arguments
 command = parts[0].lower()
 args = parts[1] if len(parts) > 1 else ""

 if command == "ask":
 if args:
 response = ask_ai(question=args)
 say(response)
 else:
 say("Please provide a question to ask.")
 elif command == "list":
 list_data_sources(say)
 else:
 say(f"Sorry, I don't recognize the command `{command}`.")

Resulting in an interaction such as

Of course, this same methodology can be used on any of the events exposed by Slack.

To learn more about extending this bot, see Slack Bolt’s documentation.

Conclusion

In this blog post, we took our existing Slack bot which has a data loading process that loads documents from an S3 bucket into a vector database on a schedule, and we wired it up to behave in real time.

To expose this functionality to the end user, we created a bridge between file uploads to Slack and S3, allowing drag/drop capabilities of files in conversational chat.

Furthering our interactions with our application, we leveraged Slack’s Block Kit to create a custom modal, and interact with events.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!

Adding data to Pinecone using S3, Embedchain and Pulumi on AWS for an AI Slack bot

Tyler Mulligan — Wed, 10 Apr 2024 17:00:00 +0100

In the introductory blog post, we learned to Create an AI Slack Bot to Chat with Your Data Using Embedchain, Pulumi on AWS. However, we made a few concessions in the application logic to illustrate the broader picture of what we were able to achieve combining these three technologies. Now that we have a solid foundation for deploying our Slack bot and querying our data, lets begin moving from proof-of-concept to production-ready, iteratively.

For reference, here’s what our architecture looked like at the end of the first iteration.

To follow along, clone the project, git clone https://github.com/catmeme/arti.git

Configuration

In our proof-of-concept, we defined our Embedchain config in our Python code. Best practices dictate strict separation of config from code, so the first thing we must address is getting the embedchain configuration out of our code.

A config/config.yaml made from our dict, is much easier to read, maintain and share.

app:
 config:
 id: arti
 collect_metrics: False
 log_level: INFO
llm:
 provider: openai
 config:
 model: 'gpt-3.5-turbo-1106'
 temperature: 0.5
 max_tokens: 1000
 top_p: 1
embedder:
 provider: openai
chunker:
 chunk_size: 2000
 chunk_overlap: 0
 length_function: len
vectordb:
 provider: chroma
 config:
 collection_name: arti-ai
 dir: /tmp/db
 allow_reset: True

Our goal is to replace our big inline dict, and read a YAML file instead, following this configuration specification.

def ask_ai(question):
 """Ask the AI a question."""
 print(f"Processing question: {question}")

 app_config = {
 # hardcoded configuration we're moving out to YAML
 }

 loader = DirectoryLoader(config={"recursive": True})
 app = App.from_config(config=app_config)
 app.add("assets", loader=loader)

 return app.query(question)

To do that, though, we don’t want to implement it directly as we already have a config module, src/arti_ai/config.py. Let’s define a new method to read that YAML file, and pass it to our application at startup.

class Config:
 def __init__(self, config_file="config/config.yaml", secrets_manager_client=None):
 self.secrets_manager_client = secrets_manager_client or boto3.client("secretsmanager")
 self.project_root = Path(__file__).resolve().parents[2]
 self.config_file = self.project_root / config_file
 load_dotenv()

 def get_embedchain_config(self):
 try:
 with open(self.config_file, 'r') as file:
 return yaml.safe_load(file)
 except FileNotFoundError as e:
 raise FileNotFoundError(f"Configuration file not found: {self.config_file}") from e
 except yaml.YAMLError as e:
 raise yaml.YAMLError(f"Error parsing YAML configuration: {self.config_file}") from e

Our app.py function simplifies, becoming more readable and flexible, we’ve even fixed a bug, making our application more portable with config.project_root, now arti can be run from any directory.

def ask_ai(question: str):
 """Ask the AI a question."""
 print(f"Processing question: {question}")

 from embedchain import App
 from embedchain.loaders.directory_loader import DirectoryLoader

 loader = DirectoryLoader(config={"recursive": True})
 app = App.from_config(config=config.get_embedchain_config()) # inline dict has become this
 app.add(f"{config.project_root}/assets", loader=loader)

 return app.query(question)

Now that we’ve separated out our configuration, we can have multiple scenarios preconfigured. In the repository, there are a few examples, including a private ai configuration, so that embedchain can be run 100% locally on your machine using gpt4all. The configuration difference boils down to updating the LLM and Embedder to open-source models.

llm:
 provider: gpt4all
 config:
 model: 'orca-mini-3b-gguf2-q4_0.gguf'
 max_tokens: 1000
 top_p: 1
embedder:
 provider: huggingface
 config:
 model: 'sentence-transformers/all-MiniLM-L6-v2'

Now that we have a dynamic approach to loading configuration from YAML, lets move on to improving the application architecture, which was not built to scale beyond relatively small data sets.

Application architecture

Knowing your limits

The current application architecture uses a single process with the expectation that a response is generated within 29 seconds, before the API Gateway timeout.

Slack also has an expectation, that we must acknowledge (or ack) a request within 3 seconds.

Currently, when the application receives a request, the API Gateway Lambda integration starts the application. The lifecycle of this single process approach is diagrammed below.

If we break this into two processes, Data Processing, and API Handling, we give our API handler more dedicated time within that 29 seconds. Simultaneously, we overcome another limitation, large amounts of data being loaded will take more than 29 seconds, by breaking out data processing into its own process, we can now have up to 15 minutes to load data using Lambda.

We’re going to use the same docker image for both Lambdas by adding an entrypoint script that can take an argument to select the handler, essentially deploying the same application twice as different Lambda Functions.

Our entrypoint.sh

#!/usr/bin/env sh
set -e

. /app/venv/bin/activate

if [ -n "$LAMBDA_HANDLER" ]; then
 /app/venv/bin/python -m awslambdaric $LAMBDA_HANDLER
else
 arti "$@"
fi

Here’s what the two process lifecycles look like separated

We have, however, introduced a new issue, and that is persistence. Our single process approach always loads the data, and warm requests will have the data cached, so it was okay that the Chroma database was ephemeral, it could always be recreated on the fly.

When our corpus is many gigabytes or terabytes, losing and recreating those embeddings for the vector database is a costly endeavor, both in time, and very likely monetarily.

How do we solve for that?

Multi-tier architecture and data persistence

By putting our vector database into another tier, we don’t couple its lifecycle with our application’s. It’s pretty cool that we can have an ephemeral vector database in Lambda for demonstration purposes, but it’s not an optimal configuration for production.

One option, is to deploy Chroma ourselves using a service such as EC2 or ECS, for example, but that incurs costs. For the ease of this demonstration, we’re going to use Pinecone, which Pulumi conveniently has a provider for. Though, there are many alternative options.

Navigate to Pinecone’s homepage, and click “Get started.” After confirming your account, login to Pinecone, and click “API Keys” in the sidebar. Create a new API key for Arti.

After configuring our Pinecone API key in AWS Secrets Manager, we are able to define our Pinecone index in our __main__.py Pulumi program.

import pulumi
import pinecone_pulumi as pinecone

pinecone_index = pinecone.PineconeIndex(
 f"{app_name}-pinecone-index",
 name="main-index",
 dimension=1536,
 metric=pinecone.IndexMetric.COSINE,
 spec=pinecone.PineconeSpecArgs(
 pod=pinecone.PineconePodSpecArgs(
 environment="gcp-starter",
 pod_type="starter",
 replicas=1,
 )
 ),
)

pulumi.export("output", {"value": pinecone_index.host})

Handling data load events

In the proof-of-concept, we are abusing docker to distribute our data sources, this isn’t optimal for a few reasons, the size of the docker image increases, which also increases build times. Do our documents really belong in the container, what about privacy, triggering builds, lead time to get new data added, etc.?

The obvious next place to put our documents is AWS S3 (Simple Storage Service). We don’t want to regress our existing implementation, which works locally as well as deployed to Lambda, so we abstract the concept of loading a path of assets, whether it’s a local path or S3. In the presence of an S3 bucket name, assume we intend to use that loader, otherwise use the directory loader.

def get_loader_and_asset_root():
 """Select and configures the appropriate loader based on AWS S3 bucket presence."""
 app_bucket_name = os.getenv("APP_BUCKET_NAME")
 root_path = os.getenv("ASSETS_ROOT_PATH", "assets")
 asset_location = root_path

 # pylint: disable=import-outside-toplevel
 if app_bucket_name:
 from arti_ai.s3_loader import S3BucketLoader # type: ignore

 loader = S3BucketLoader()
 asset_location = f"{app_bucket_name}/{root_path}"
 else:
 from embedchain.loaders.directory_loader import DirectoryLoader # type: ignore

 loader = DirectoryLoader(config={"recursive": True})

 # pylint: enable=import-outside-toplevel

 return loader, asset_location


def load_data():
 """Load data from data sources."""
 print("Loading data")

 # pylint: disable=import-outside-toplevel
 from embedchain import App

 # pylint: enable=import-outside-toplevel

 app = App.from_config(config=config.get_embedchain_config())

 loader, asset_location = get_loader_and_asset_root()
 response = app.add(asset_location, loader=loader)

 return response

With our data loading logic figured out, we’ll pass the APP_BUCKET_NAME in our Pulumi program to our Lambda Function, so it can retrieve the data.

app_bucket = aws.s3.Bucket(
 f"{app_name}-bucket",
 bucket=f"{app_name}-bucket",
 # ...
)

lambda_environment = pulumi.Output.all(
 app_bucket.id,
).apply(
 lambda args: {
 "variables": {
 # ...
 "APP_BUCKET_NAME": args[0],
 # ...
 }
 }
)

app_event_lambda = aws.lambda_.Function(
 f"{app_name}-lambda",
 architectures=["x86_64"],
 environment=lambda_environment,
 image_uri=docker_image_lambda_event_handler.repo_digest,
 # ...
)

If you’re new to Pulumi, you might be wondering what’s going on with the lambda_environment, why isn’t app_bucket.id referenced directly, and what is pulumi.Output.all? Simply put, this tells Pulumi to wait until the bucket is created before trying to use its name in a string we’re building. This is an important core feature of Pulumi that largely happens by nesting Outputs. To learn more, see Inputs & Outputs.

Triggering the data loader

When deployed, we’re going to need something that doesn’t require a local machine. One method for loading data is to use a scheduled task. AWS EventBridge makes that task simple, our Pulumi resource declarations are below. We define a rule/schedule (pulled from stack configuration), add our target (the event handler Lambda), and setup permissions to invoke the Lambda function.

event_rule_schedule = aws.cloudwatch.EventRule(
 f"{app_name}-event-rule-schedule",
 schedule_expression=data_load_event_rule_schedule,
 state=data_load_event_rule_state,
)

event_rule_target = aws.cloudwatch.EventTarget(
 f"{app_name}-event-target-schedule",
 rule=event_rule_schedule.name,
 arn=app_event_lambda.arn,
)

event_rule_target_permission = aws.lambda_.Permission(
 f"{app_name}-lambda-event-target-permissions-schedule",
 action="lambda:InvokeFunction",
 function=app_event_lambda.name,
 principal="events.amazonaws.com",
 source_arn=event_rule_schedule.arn,
)

Our event lambda handler looks something like

import json

from arti_ai.app import load_data


def handler(event, context):
 print("Received event: " + json.dumps(event, indent=2))
 print("Context AWS request ID: " + json.dumps(context, indent=2))

 load_data()

 return {"statusCode": 200, "body": json.dumps("Event processed successfully!")}

Locally, we can use our command-line interface (CLI), defined in __main__.py

if args.command == "ask":
 handle_ask(args.question)
elif args.command == "load":
 load_data()
elif args.command == "reset":
 reset_data()
else:
 parser.print_help()

Any of our YAML configuration files should work with the following commands:

arti ask "what can you tell me about fruits and vegetables?"
arti load
arti reset
arti -h

You could manage your live Pinecone data this way, whether you should is highly dependent on the use case.

Wrapping up

We reflected on the project that was presented in the original post, and reviewed short-comings with the proof-of-concept, working through improvements that touched upon architecture, application design, and infrastructure. Specifically, we have moved application configuration out of code into YAML, decoupled data-loading from querying, broken out our vector database to another tier, and integrated our application with AWS EventBridge to trigger scheduled tasks.

Our delivery on the concept was expedited by utilizing services that aligned with our improvements, and Pulumi helped us achieve our architecture changes using familiar patterns in just a few lines of code.

In the next post in this series, we’ll continue to make iterative improvements to both our applications and infrastructure, and start getting deeper into RAG.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!

Build an AI Slack Bot on AWS Using Embedchain & Pulumi

Tyler Mulligan — Mon, 18 Mar 2024 17:21:02 +0100

The integration of artificial intelligence (AI) to improve user experiences is gaining popularity in today’s world. One fascinating application of AI is the creation of chatbots, which can engage users in conversation and provide helpful information or services.

In this blog post, we’ll explore the process of building an AI-powered Slack bot using Embedchain, a Retrieval-Augmented Generation (RAG) framework powered by LangChain. Additionally, we’ll deploy our bot on AWS using Pulumi, a modern infrastructure as code (IaC) platform.

What are LLMs and RAG chatbots?

Large Language Models (LLMs) are sophisticated AI models trained on vast amounts of text data, enabling them to generate human-like text responses. Retrieval-Augmented Generated (RAG) is a cutting-edge approach to chatbot development that combines the capabilities of LLMs with information retrieval techniques. RAG chatbots can generate responses based on both the input query and relevant information retrieved from a knowledge base, resulting in more contextually relevant and informative interactions.

What’s an Embedchain?

Embedchain is a RAG framework powered by LangChain. It simplifies the process of creating RAG applications by providing high-level abstractions that reduce the complexity of AI technologies. With Embedchain, developers can focus on defining conversational flows and configuring the bot’s behavior using YAML configuration files, rather than dealing with low-level implementation details.

Embedchain supports configuration for various data sources, LLMs, vector databases, embedding models, and evaluation.

Creating a Slack bot

Start by cloning the project:

git clone https://github.com/catmeme/arti.git
cd arti

The files within the src/arti_ai directory can be summarized as:

The fourth file isn’t strictly necessary, but it illustrates a design pattern. The main application, app.py, is imported by slack_app but also __main__.py. This makes it easy to add additional user interfaces, such as APIs, or other chat services with minimal code.

The AI application

Let’s start with “the brain”, our Embedchain application. Below is a simple example (using explicit configuration for clarity). This code expects that OPENAI_API_KEY is populated in your environment. export OPENAI_API_KEY=<your key> to experiment with the code below.

from embedchain import App

app_config = {
 "app": {
 "config": {
 "collect_metrics": False,
 "log_level": "INFO",
 }
 },
 "llm": {
 "provider": "openai",
 "config": {
 "model": "gpt-3.5-turbo-1106",
 "temperature": 0.5,
 "max_tokens": 1000,
 "top_p": 1,
 },
 },
 "embedder": {"provider": "openai"},
 "chunker": {"chunk_size": 2000, "chunk_overlap": 0, "length_function": "len"},
 "vectordb": {
 "provider": "chroma", # we're writing to a vectordb in memory in lambda
 "config": {"collection_name": "arti-ai", "dir": "/tmp/db", "allow_reset": True},
 },
}

app = App.from_config(config=app_config)
app.add("https://www.pulumi.com/docs/", data_type="docs_site")
app.query("What is Pulumi?")

See how easy it was for us to configure our application, an LLM, an embedding model, chunker and vector database? Then we added a datasource, which is abstracting away more complex logic for us. The simple line app.add("https://www.pulumi.com/docs/", "docs_site") ends up crawling the website, chunking the documents, generating embeddings and inserting them into our vector database.

The app_config reference is the Embedchain configuration schema.

The Slack application

Now that our brain is built, we need a way for our users to interact with it. Slack Bolt is an official library for building Slack bots.

Follow their instructions to create a new Slack app.

OAuth & permissions

chat:write
channels:read
commands
im:read
im:write
users:read
users:write

Event subscriptions

message.channels
message.im

Code breakdown

The example below demonstrates two commands. knock knock is a basic ping/pong test that you expect to respond immediately. It’s not doing any complex logic, so that’s easy. What’s going on with this /arti command then? Slack requires your bot respond in 3 seconds or less. In our serverless architecture utilizing AWS API Gateway and Lambda, we’re operating on thin margins when we have cold starts. This enables us to respond quickly to Slack, but start our longer running process.

It is worth noting that in this architecture we’re still operating under 29 seconds for API Gateway’s response timeout. This can be overcome in a number of ways, but that’s out of scope for this demonstration.

from slack_bolt import App

from arti_ai.app import ask_ai, config

slack_bot_token, slack_bot_signing_secret = config.get_slack_credentials()
app = App(process_before_response=True, token=slack_bot_token, signing_secret=slack_bot_signing_secret)


def process_request(respond, body):
 """Process the request and respond to the user."""
 question = body["text"]
 response = ask_ai(question)
 respond(f"Q: _{question}_ A: {response}")


def respond_to_slack_within_3_seconds(ack):
 """Respond to the user within 3 seconds."""
 ack("Thinking...")


app.command("/arti")(ack=respond_to_slack_within_3_seconds, lazy=[process_request])


@app.message("knock knock")
def ask_who(_message, say):
 """Ask who's there."""
 say("_Who's there?_")

Configuration

The configuration module in this application normalizes retrieval of secrets from environment variables or AWS Secrets Manager. You may have noticed the Embedchain application configuration is hardcoded into the app.py. This isn’t considered best practice, and may be a fun first step in making this project yours. Otherwise, stay tuned for future posts in this series.

Deploying on AWS with Pulumi

Once our bot is ready, we’ll deploy it on AWS using Pulumi. Pulumi allows us to define our cloud infrastructure using familiar programming languages like TypeScript or Python, making it easy to manage and scale our deployment.

It’s important to note that in this selected architecture, we’re working within a few limitations. Lambda’s only writable directory is /tmp and it’s limited to 500MB, and we need to use a Docker container to overcome the 250MB filesize limitation of our application, which we’ll be bundling our data with.

The Pulumi program is broken up into the following sections:

Config: Get configuration from stack configuration, environment, and set up resource tagging
Networking: Set up VPC, VPC endpoints, security groups
S3: Bucket for later use
Application: Docker, Lambda, policies
API: API Gateway
Output: Stack outputs

Resources are contained within the VPC, policies and security groups exercise principle of least privilege, and logging is enabled. lambda_api_handler.py is responsible for handling API Gateway requests and imports the Slack app.

Prerequisites

Create the following keys in Secrets Manager. These names are configurable in deploy/pulumi/Pulumi.<stack>.yaml. Slack and Pinecone are only necessary if they are configured for use.

Secret Name	Schema
/catmeme/cloudplatform/sandbox/arti/access-token/openai	`{ "apiKey": "" }`
/catmeme/cloudplatform/sandbox/arti/access-token/slack	`{ "apiKey": "", "signingSecret": "" }`

Makefile

Makefiles offer a powerful, flexible, and efficient way to manage project tasks, contributing to smoother development processes, better team collaboration, and more reliable product builds. See the project README.md for more information.

cp -R /path/to/my/assets ./assets
cp sample.env .env
python3 -m venv venv
. venv/bin/activate
make develop

Ensure you have a matching AWS profile name to the one in the stack. The Makefile assumes <environment>-deployment. This script will attempt to copy the deploy/pulumi/Pulumi.catmeme.arti.sandbox.us-east-1.yaml as a base for configuration of the stack you deploy.

make deploy DEPLOY_ENVIRONMENT=dev

At this point, you should be able to use /arti to interact with your data via Slack.

Conclusion

In this blog post, we’ve explored the process of building an AI-powered Slack bot using Embedchain and deploying it on AWS with Pulumi. By harnessing the capabilities of LLMs and RAG chatbots, we can create intelligent conversational agents that enhance user interactions and provide valuable insights. With Embedchain’s high-level abstractions and Pulumi’s infrastructure as code platform, developers can streamline the development and deployment of AI-powered applications, unlocking new possibilities for innovation in the digital space.

As always, we welcome your feedback and contributions in the Pulumi Community Slack, GitHub repository, and Pulumi Community Discussions.

New to Pulumi? Signing up is easy and free. Get started today!

Happy AI building!