Event-Driven Email Notifications for Model Training Completion

Question

Pulumi · Accepted Answer

To set up event-driven email notifications for model training completion, we'll use AWS services, specifically AWS Lambda and Amazon Simple Notification Service (SNS). The general flow is as follows:

1. **AWS Lambda Function**: It will be triggered at the completion of your model training. This can be an invocation from another AWS service (e.g., AWS SageMaker, if you're using it for training the model) or a custom event that you define. The function will publish a message to an SNS topic.

2. **Amazon SNS Topic**: SNS is a pub/sub messaging service that facilitates message delivery. You will create an SNS topic to which the Lambda function will send a message once the model training is complete.

3. **SNS Email Subscription**: You will subscribe an email endpoint (recipient's email address) to the SNS topic. After subscribing, SNS will send an email to the subscriber(s) each time a message is published to the topic.

Here's a Pulumi program in Python that sets up these resources. Make sure you replace the `email_address` variable value with the recipient's email address.

Note: Before running this program, ensure that your AWS credentials are configured either via the AWS CLI or environment variables, and you've installed the needed Pulumi AWS SDK.

```python
import pulumi
import pulumi_aws as aws

# Replace with the email address you want to notify
email_address = "notify@example.com"

# Create an SNS topic for sending notifications
sns_topic = aws.sns.Topic('model-training-completion-topic')

# Subscribe an email address to the SNS topic
email_subscription = aws.sns.TopicSubscription('model-training-completion-subscription',
    topic=sns_topic.arn,
    protocol='email',
    endpoint=email_address
)

# Create an IAM Role for Lambda with SNS Publish permissions
lambda_role = aws.iam.Role('lambda-exec-role',
    assume_role_policy="""{
        "Version": "2012-10-17",
        "Statement": [{
            "Action": "sts:AssumeRole",
            "Principal": {
                "Service": "lambda.amazonaws.com"
            },
            "Effect": "Allow",
            "Sid": ""
        }]
    }"""
)

sns_publish_policy = aws.iam.RolePolicy('sns-publish-policy',
    role=lambda_role.id,
    policy=sns_topic.arn.apply(lambda arn: f"""{{
        "Version": "2012-10-17",
        "Statement": {{
            "Action": "sns:Publish",
            "Resource": "{arn}",
            "Effect": "Allow"
        }}
    }}""")
)

# Note: You must provide a zip file containing your Lambda function code and specify its path below.
lambda_function = aws.lambda_.Function('model-training-completion-function',
    code=pulumi.FileArchive('./lambda_function.zip'),
    role=lambda_role.arn,
    handler='lambda_function.handler', # Update this to the correct handler value
    runtime='python3.8'
)

# Export the SNS topic ARN and Lambda function name
pulumi.export('sns_topic_arn', sns_topic.arn)
pulumi.export('lambda_function_name', lambda_function.name)
```

Here's what each part of the Pulumi program does:

- **Email Subscription**: The `aws.sns.TopicSubscription` class creates a subscription to your SNS topic with the defined email address.
  
- **IAM Role**: The `aws.iam.Role` class defines an IAM role for Lambda with a trust relationship that allows Lambda to assume the role. We need this role to give Lambda permissions to interact with other AWS services.

- **IAM Role Policy**: The policy `aws.iam.RolePolicy` defines the permissions that the Lambda function has. In this case, it has permission to publish messages to the SNS topic.

- **Lambda Function**: The `aws.lambda_.Function` class creates the Lambda function with the specified runtime and handler. You need to provide the zip file containing your Lambda function code, which is triggered upon completion of the model training and publishes to the SNS topic.

Lastly, we export the SNS topic ARN and Lambda function name using `pulumi.export` for easy reference from the Pulumi CLI or for use in additional Pulumi code stacks.

You will need to confirm the SNS subscription through an email that AWS sends to the subscribed email address before it can receive notifications. Also, replace `'./lambda_function.zip'` with the actual path to the zip file containing your Lambda function code and `'lambda_function.handler'` with your function's handler.