1. Dynamic Secrets for AI Pipeline Services Using Vault JWT


    Dynamic secrets are an advanced feature often used in applications requiring increased security and short-lived access credentials. In the context of AI pipeline services, dynamic secrets allow for the secure, on-the-fly generation of access keys and tokens that services within the pipeline can use to authenticate with one another and various cloud resources.

    For this purpose, we will use the HashiCorp Vault JWT (JSON Web Token) backend. This backend generates JWT tokens dynamically and could be used to authenticate with services that support JWT authentication.

    To use dynamic secrets with Vault, you need to have:

    1. Vault server up and running.
    2. Vault client configured with proper access to communicate with the server.
    3. The JWT authentication backend must be enabled and configured in Vault.

    The following program demonstrates how to set up a Vault JWT authentication backend using Pulumi in Python. The program also defines a role within that backend. When someone logs into Vault with a valid JWT token and requests a secret under this role, Vault will validate the JWT token using the configured method. Upon successful validation, it will return a secret as per the role configuration.

    Note that this program assumes that you have a Vault server already deployed and running.

    import pulumi from pulumi_vault import jwt # Enable the JWT auth backend in Vault auth_backend = jwt.AuthBackend("auth-backend", path="jwt", # The path where the auth method is accessible description="JWT backend used for dynamic secrets", type="jwt") # Create a role in JWT auth backend. # This role will define the policies and claims required in JWT tokens to authenticate. role = jwt.AuthBackendRole("auth-role", backend=auth_backend.path, role_name="example-role", user_claim="sub", # The claim to use to uniquely identify the user. bound_audiences=["example-audience"], # Expected audience in the authentication token token_policies=["example-policy"], # List of policies to assign to the tokens token_ttl="1h", # The TTL of the tokens generated token_max_ttl="4h") # The max TTL of the tokens generated # Export the path to the JWT auth backend and the name of the role created pulumi.export("jwt_auth_backend_path", auth_backend.path) pulumi.export("jwt_auth_role_name", role.role_name)

    In the code above, we define a JWT authentication backend in Vault at a custom path "jwt" (the default path is normally "jwt" unless you choose to customize it). We then define a role, "example-role", with specific claims and policies. These policies are Vault policies that users authenticating with this JWT backend would be granted.

    The user_claim parameter is what Vault will use to uniquely identify the user or machine presenting the token, which is typically a claim like "user_email" or "sub" (subject) in JWT tokens. The bound_audiences is an important security feature that ensures the presented JWT token is intended to be used with the Vault server.

    Remember that this configuration has to be aligned with the JWT tokens your services are using. The tokens should include the 'sub' claim and the audience matching 'example-audience'.

    Please ensure that your Pulumi stack is configured with the necessary Vault provider settings, and bear in mind that interacting with Vault through Pulumi will require appropriate permissions.

    To understand how to connect to your Vault server and authenticate using the Pulumi Vault Provider, you can refer to the documentation, which includes how to set up your Vault access credentials as environment variables or within the Pulumi configuration system.