Secure S3 Data Access for AI Services with OIDC Federated Identities

Question

Pulumi · Accepted Answer

To secure S3 data access for AI services with OIDC federated identities, we'll need to perform a few tasks:

Create an AWS Identity and Access Management (IAM) role that trusts the OIDC identity provider.
Define a policy that specifies the permissions for the S3 bucket, ensuring it allows access for the AI services.
Associate the policy with the IAM role.
Configure the AI service to assume the IAM role using the OIDC identity for authentication.

We'll do this using Pulumi to create and manage the AWS infrastructure needed for such a setup.

First, we would start by defining an IAM role using the AssumableRoleWithOIDC class, which creates a role that can be assumed by an OpenID Connect (OIDC) provider. This role will be assumed by the AI services for accessing the S3 bucket.

Next, we'll create an S3 bucket with a policy attached to it that allows access to this bucket for the specific IAM role. The policy will be scoped to the necessary S3 actions (like s3:GetObject, s3:PutObject, s3:ListBucket, etc) depending on what operations the AI services need to perform.

Finally, we'll configure an OIDC identity provider using AWS IAM that establishes our trust relationship between your OIDC compatible identity provider (like Okta, Auth0, etc.) and AWS.

Here's how you would use Pulumi in Python to set these up:

import pulumi
import pulumi_aws as aws

# Replace these variables with your own information
# The OIDC Provider URL
oidc_provider_url = "https://oidc-idp-url.com"
# The OIDC Provider ARN
oidc_provider_arn = "arn:aws:iam::<account-id>:oidc-provider/<provider-name>"
# The Audience (Client ID) for your OIDC identity provider
oidc_audience = "my-audience"

# Create an IAM role which can be assumed by an identity provider
s3_ai_service_role = aws.iam.Role("s3AIServiceRole",
    assume_role_policy=pulumi.Output.from_input({
        "Version": "2012-10-17",
        "Statement": [{
            "Effect": "Allow",
            "Principal": {
                "Federated": oidc_provider_arn
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringEquals": {
                    f"{oidc_provider_url}:aud": oidc_audience
                }
            }
        }]
    })
)

# Create a policy for accessing the S3 bucket
s3_access_policy = aws.iam.Policy("s3AccessPolicy",
    policy=pulumi.Output.from_input({
        "Version": "2012-10-17",
        "Statement": [{
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
                # ...add other actions the AI service needs
            ],
            "Resource": [
                # Assuming 'my-bucket-name' is your Bucket name,
                # modify as necessary to reference your S3 bucket and objects.
                "arn:aws:s3:::my-bucket-name/*"
            ]
        }]
    })
)

# Attach the policy to the role
policy_attachment = aws.iam.RolePolicyAttachment("s3AccessPolicyAttachment",
    role=s3_ai_service_role.name,
    policy_arn=s3_access_policy.arn
)

# Create the S3 bucket
ai_data_bucket = aws.s3.Bucket("aiDataBucket",
    acl="private"
    # You may add other configuration for the S3 bucket as needed
)

# Export the names of the resources we created
pulumi.export("role_name", s3_ai_service_role.name)
pulumi.export("policy_arn", s3_access_policy.arn)
pulumi.export("bucket_name", ai_data_bucket.id)

What we have done in the Pulumi program above is defined an AWS IAM Role that trusts an OIDC identity provider by specifying the provider URL and expected audience. We then created an IAM policy that outlines the required permissions for our S3 bucket and tied this policy to our role. Lastly, we provisioned an S3 bucket.

This will allow your AI services to use a token provided by your OIDC identity provider to assume the IAM role we created, and in doing so, gain the permissions outlined in the associated policy to interact with the S3 bucket.

Ensure that your AI services are configured to provide the OIDC token when making requests to AWS, and have set the proper audience in the token to match what's set in the IAM role condition.

Please double-check the documentation for specific configuration on the identity provider side, as this can differ between services like Okta, Auth0, and others. This Pulumi program must be accompanied by appropriate configuration of your identity provider and your AI services to leverage the federated OIDC identities.