Integration Testing for AI Services with GitHub Actions

Question

Pulumi · Accepted Answer

Integration testing is a critical step in the CI/CD pipeline to ensure that different parts of your application work together as expected. For AI services, integration tests can validate that the machine learning models and associated data processing pipelines are working correctly in conjunction with your application's business logic.

When using GitHub Actions for CI/CD, you can set up workflows to run integration tests automatically on various events such as a push or pull request. For the integration testing of AI services, these tests can be set to run in environments where the AI models are hosted.

Below is a Python program using Pulumi to create necessary GitHub secrets for a repository, which can then be utilized in GitHub Actions workflows to interact with cloud services or other protected resources that your integration testing suite may need to access.

The Pulumi `github` package is used here to interact with your GitHub repository. Specifically, the `github.ActionsSecret` class is used to create secrets within your GitHub repo, allowing your GitHub Actions workflows to authenticate with and access other services:

```python
import pulumi
import pulumi_github as github

# Replace these with your actual GitHub repository name and secret values
github_repository_name = "my-ai-app"
aws_access_key_id_secret_value = "YOUR_AWS_ACCESS_KEY_ID"
aws_secret_access_key_secret_value = "YOUR_AWS_SECRET_ACCESS_KEY"

# Create a GitHub Actions Secret for AWS_ACCESS_KEY_ID
aws_access_key_id_secret = github.ActionsSecret("AWS_ACCESS_KEY_ID",
    repository=github_repository_name,
    plaintext_value=aws_access_key_id_secret_value)

# Create a GitHub Actions Secret for AWS_SECRET_ACCESS_KEY
aws_secret_access_key_secret = github.ActionsSecret("AWS_SECRET_ACCESS_KEY",
    repository=github_repository_name,
    plaintext_value=aws_secret_access_key_secret_value)

# Export the secret names to be used in the GitHub Actions workflow
pulumi.export("aws_access_key_id_secret_name", aws_access_key_id_secret.name)
pulumi.export("aws_secret_access_key_secret_name", aws_secret_access_key_secret.name)
```

This program sets up two secrets – `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`. These are commonly needed for AWS SDKs to authenticate against AWS services for operations like deploying an AI model to an AWS service such as Amazon SageMaker, retrieving data from an Amazon S3 bucket, or any AWS service you might be interacting with as part of your integration test.

You can then reference these secrets in your `.github/workflows/integration_test.yml` GitHub Actions workflow file like so:

```yaml
name: Integration Test

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout code
      uses: actions/checkout@v2

- name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: '3.8'

- name: Install dependencies
      run: |
        pip install -r requirements.txt
        
    - name: Run integration tests
      env:
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      run: |
        python -m unittest discover tests
```

This workflow file outlines a simple integration test setup. It checks out the code from your repository, sets up the Python environment, installs dependencies, and then runs tests using the secrets you provisioned with Pulumi.

To make this program operational, replace the placeholder values for the repository name and secret values with the actual values you intend to secure. After running this Pulumi program, your GitHub repository will have the secrets needed to carry out integration testing that interacts with other services safely.