Securing In-cluster Inference Services with TLS Encryption

Question

Pulumi · Accepted Answer

To secure in-cluster inference services with TLS encryption, you'll typically need to perform the following tasks:

1. Create a TLS certificate and a corresponding private key.
2. Store the certificate and key in a Kubernetes secret, or use a managed service like KMS on cloud providers to manage them securely.
3. Configure the inference service to use the TLS certificate for encrypted communication.

In the program below, we're going to use Pulumi to achieve this within a Kubernetes cluster. The steps are as follows:

- Use the `tls` Pulumi package to create a self-signed TLS certificate and private key.
- Define a Kubernetes secret to store the TLS certificate and key.
- Adjust the Service and optionally an Ingress (if you use one) to make use of the TLS Secret.

Here's a program that demonstrates these steps:

```python
import pulumi
import pulumi_tls as tls
import pulumi_kubernetes as kubernetes

# Create a self-signed certificate for the inference service
cert = tls.PrivateKey("private-key",
    algorithm="RSA",
    rsa_bits=2048
)

# Create a self-signed TLS certificate using the private key
tls_cert = tls.SelfSignedCert("tls-cert",
    key_algorithm="RSA",
    private_key_pem=cert.private_key_pem,
    subjects=[tls.SelfSignedCertSubjectArgs(
        organization="Acme Co",
    )],
    is_ca_certificate=True,
    allowed_uses=[
        "key_encipherment",
        "digital_signature",
        "server_auth",
    ],
    validity_period_hours=8760,
    dns_names=["inference-service.example.com"]
)

# Create a Kubernetes secret to store the TLS certificate and key.
# This secret will be used by the inference service and potentially an Ingress.
tls_secret = kubernetes.core.v1.Secret("tls-secret",
    metadata=kubernetes.meta.v1.ObjectMetaArgs(
        name="inference-tls"
    ),
    type="kubernetes.io/tls",
    string_data={
        "tls.crt": tls_cert.cert_pem,
        "tls.key": cert.private_key_pem
    }
)

# The Service definition for your inference service should reference the TLS secret
# for TLS termination. Here's a generic example of such a service. You will need
# to adjust the `ports` and `selector` to match your actual inference service.
inference_service = kubernetes.core.v1.Service("inference-service",
    metadata=kubernetes.meta.v1.ObjectMetaArgs(
        name="inference-service",
    ),
    spec=kubernetes.core.v1.ServiceSpecArgs(
        ports=[kubernetes.core.v1.ServicePortArgs(
            port=443,
            target_port=8443
        )],
        selector={
            "app": "inference-service"
        },
        type="ClusterIP"
    )
)

# Optionally, you might have an Ingress that makes the inference service available
# from outside the cluster. The Ingress would also use the TLS secret to secure
# the incoming traffic.

# Finally, we will export the URL of the service for easy access
pulumi.export("inference_service_url", pulumi.Output.concat("https://inference-service.example.com"))
```

This program does the following:

- It creates a 2048-bit RSA private key.
- It then uses that key to create a self-signed TLS certificate. The domain name `inference-service.example.com` is used as part of the example, but in a real-world scenario, this should match the actual DNS name that points to your inference service.
- A Kubernetes secret `tls-secret` is then created to store the certificate and the private key. This secret will be referenced by other Kubernetes resources that need to perform TLS termination.
- A Kubernetes `Service` resource is created. In a real-world use case, you would configure this to match the actual ports and selectors corresponding to your inference service pods.
- Optionally, you could add an Ingress resource to expose the service outside the Kubernetes cluster, using the same way to reference the secret for TLS.
- The URL for the service is then exported, albeit it would only be accessible within the cluster.

Please adjust the `dns_names` in the TLS certificate, and `selector` and `ports` in the Service definition to match your service's actual configuration. Make sure the DNS name is resolvable to the service or Ingress behind which the inference service is running.

Remember to replace the placeholder DNS name with the actual one you have registered. Also, for production systems, you might want to obtain your TLS certificate from a trusted certificate authority (CA) instead of using a self-signed certificate.