Securing Machine Learning API keys with HashiCorp Vault Transit

Question

Pulumi · Accepted Answer

If you're managing Machine Learning (ML) API keys, you'll want to make sure they are securely stored and managed. One way to achieve this is by using the HashiCorp Vault Transit secrets engine. This engine handles cryptographic functions on data in-transit. Vault Transit can encrypt and decrypt data, manage keys, and enforce access controls.

Below, I'll provide you with a Pulumi Python program that demonstrates how to create a transit secrets engine in Vault, and set up an encryption key that can be used for your ML API keys. This transit backend can then be used to encrypt your ML API keys before storing them elsewhere, or you may store the encrypted value directly in Vault, depending on your requirements.

Here's the high-level approach of the program:

1. Initialize a Vault transit secret backend.
2. Create a named encryption key within the transit backend.
3. Use the created named encryption key to encrypt an example plaintext string representing an ML API key.
4. Optionally, decrypt the ciphertext to verify it's working as expected.

Please replace the `"my-ml-api-key"` in the program with the actual key you obtain from your ML service provider. This key is only used as a placeholder to illustrate the encryption process.

```python
import pulumi
import pulumi_vault as vault

# Initialize a Vault transit secret backend.
transit_backend = vault.TransitSecretBackend("transit",
    path="transit",  # You can customize the mount path for the transit secrets engine.
    description="Transit backend for encrypting ML API keys"
)

# Create a named encryption key within the transit backend.
# The name 'ml-api-key' represent the encryption key name and can be any name that is descriptive to you.
ml_api_key = vault.TransitSecretBackendKey("ml-api-key",
    backend=transit_backend.path,  # Specify the mount path of the transit backend.
    name="ml-api-key",  # The name of the encryption key to create.
    type="aes256-gcm96",  # Specify the encryption algorithm, aes256-gcm96 is a recommended choice.
)

# Use the created named encryption key to encrypt an example plaintext string.
# For better security, you would typically fetch this value from secure input or configuration.
plaintext_ml_api_key = "my-ml-api-key"
ciphertext = ml_api_key.encrypt(plaintext_ml_api_key)

# pulumi.export to expose the ciphertext outside of Pulumi for reference or use in other systems.
pulumi.export("ciphertext", ciphertext.ciphertext)

# Decrypt the ciphertext to verify it's working as expected.
# This step is optional and typically would be done where you need to use the API key.
decrypted = ml_api_key.decrypt(ciphertext.ciphertext)

# Ensure the decrypted value is the same as the original plaintext (only for test/validation purposes).
pulumi.export("decrypted_plaintext", decrypted.plaintext)

# Remember that encryption and decryption actions should be handled with care,
# and you do not want to expose sensitive keys or decrypted values unnecessarily.
```

In this program, we used Vault's `TransitSecretBackend` and `TransitSecretBackendKey` ([documentation](https://www.pulumi.com/registry/packages/vault/api-docs/transit/secretbackendkey/)) to configure encryption for our hypothetical ML API key. We encrypt the plaintext API key and also provided an example of decryption, though in practice you would limit decryption operations to when you actually need to use the API key.

Remember to secure access to Vault and limit the visibility of sensitive plaintext and ciphertext within your workflows. Managing encryption keys properly is crucial for overall security. Use the Pulumi `export` feature judiciously, avoiding exposure of sensitive data.