Kubernetes Ingress Control for AI Service Orchestration

Question

Pulumi · Accepted Answer

In a Kubernetes cluster, an Ingress is an API object that manages external access to the services in a cluster, typically HTTP. Ingress can provide load balancing, SSL termination, and name-based virtual hosting. Ingress Controllers are responsible for fulfilling the Ingress by acting as the entry point for the traffic and routing it to the appropriate services.

To set up Ingress Control for an AI Service Orchestration, we'll need to create several Kubernetes resources:

1. An **Ingress Controller**: this is a pod that's responsible for managing the Ingress resources within the cluster by acting as a reverse proxy and load balancer.
2. One or more **Ingress Resources**: these resources define how to route traffic that comes into the cluster to the various services.
3. **Services**: these represent the AI services running in the cluster that you want to expose via the Ingress.

Assuming you have your Kubernetes cluster up and running, the following program will guide you through setting up an Ingress Controller and creating an Ingress Resource to manage traffic to your AI service.

First, we install the Ingress Controller. This example uses the NGINX Ingress Controller, which is one of the most popular Ingress controllers. We rely on the `pulumi_kubernetes` package to deploy this controller to your cluster.

Next, we define two AI services we wish to orchestrate using Kubernetes. These are dummy services represented as `Service` objects in Kubernetes. For each of these services, we create a corresponding `Deployment` and `Service` definition in Kubernetes.

Finally, we define an `Ingress` resource, specifying the rules that route traffic to these AI services based on the incoming request's host or path.

Below is the Python Pulumi program:

```python
import pulumi
import pulumi_kubernetes as k8s

# Defining the NGINX Ingress Controller using its Helm chart. The installation can be
# customized by setting the appropriate values. Here we're using default values.
nginx_ingress_controller = k8s.helm.v3.Chart(
    "nginx-ingress-controller",
    k8s.helm.v3.ChartOpts(
        chart="ingress-nginx",
        version="3.7.1",
        fetch_opts=k8s.helm.v3.FetchOpts(
            repo="https://kubernetes.github.io/ingress-nginx"
        ),
    ),
)

# Define the first dummy AI service deployment.
ai_service_1_deployment = k8s.apps.v1.Deployment(
    "ai-service-1",
    metadata=k8s.meta.v1.ObjectMetaArgs(name="ai-service-1"),
    spec=k8s.apps.v1.DeploymentSpecArgs(
        replicas=1,
        selector=k8s.meta.v1.LabelSelectorArgs(match_labels={"app": "ai-service-1"}),
        template=k8s.core.v1.PodTemplateSpecArgs(
            metadata=k8s.meta.v1.ObjectMetaArgs(labels={"app": "ai-service-1"}),
            spec=k8s.core.v1.PodSpecArgs(
                containers=[k8s.core.v1.ContainerArgs(
                    name="ai-service-1",
                    image="my-ai-service:latest",  # Replace with your AI service image
                )]
            ),
        ),
    ),
)

# Define Service for the above deployment.
ai_service_1_svc = k8s.core.v1.Service(
    "ai-service-1",
    metadata=k8s.meta.v1.ObjectMetaArgs(
        name="ai-service-1",
    ),
    spec=k8s.core.v1.ServiceSpecArgs(
        selector={"app": "ai-service-1"},
        ports=[k8s.core.v1.ServicePortArgs(
            port=80,
            target_port=pulumi.IntOrString.from_int(80),
        )],
        type="ClusterIP",
    ),
)

# Define the second dummy AI service deployment.
ai_service_2_deployment = k8s.apps.v1.Deployment(
    "ai-service-2",
    metadata=k8s.meta.v1.ObjectMetaArgs(name="ai-service-2"),
    spec=k8s.apps.v1.DeploymentSpecArgs(
        replicas=1,
        selector=k8s.meta.v1.LabelSelectorArgs(match_labels={"app": "ai-service-2"}),
        template=k8s.core.v1.PodTemplateSpecArgs(
            metadata=k8s.meta.v1.ObjectMetaArgs(labels={"app": "ai-service-2"}),
            spec=k8s.core.v1.PodSpecArgs(
                containers=[k8s.core.v1.ContainerArgs(
                    name="ai-service-2",
                    image="my-ai-service:latest",  # Replace with your AI service image
                )]
            ),
        ),
    ),
)

# Define Service for the second AI service deployment.
ai_service_2_svc = k8s.core.v1.Service(
    "ai-service-2",
    metadata=k8s.meta.v1.ObjectMetaArgs(
        name="ai-service-2",
    ),
    spec=k8s.core.v1.ServiceSpecArgs(
        selector={"app": "ai-service-2"},
        ports=[k8s.core.v1.ServicePortArgs(
            port=80,
            target_port=pulumi.IntOrString.from_int(80),
        )],
        type="ClusterIP",
    ),
)

# Define Ingress resource to route traffic based on the request's host or path.
ai_ingress = k8s.networking.v1.Ingress(
    "ai-ingress",
    metadata=k8s.meta.v1.ObjectMetaArgs(
        name="ai-ingress",
        annotations={
            "kubernetes.io/ingress.class": "nginx",
            # Add any other necessary annotations.
        },
    ),
    spec=k8s.networking.v1.IngressSpecArgs(
        rules=[
            k8s.networking.v1.IngressRuleArgs(
                host="ai-service-1.mydomain.com",
                http=k8s.networking.v1.HTTPIngressRuleValueArgs(
                    paths=[
                        k8s.networking.v1.HTTPIngressPathArgs(
                            path="/",
                            pathType="Prefix",
                            backend=k8s.networking.v1.IngressBackendArgs(
                                service=k8s.networking.v1.IngressServiceBackendArgs(
                                    name="ai-service-1",
                                    port=k8s.networking.v1.ServiceBackendPortArgs(
                                        number=80
                                    ),
                                ),
                            ),
                        ),
                    ],
                ),
            ),
            k8s.networking.v1.IngressRuleArgs(
                host="ai-service-2.mydomain.com",
                http=k8s.networking.v1.HTTPIngressRuleValueArgs(
                    paths=[
                        k8s.networking.v1.HTTPIngressPathArgs(
                            path="/",
                            pathType="Prefix",
                            backend=k8s.networking.v1.IngressBackendArgs(
                                service=k8s.networking.v1.IngressServiceBackendArgs(
                                    name="ai-service-2",
                                    port=k8s.networking.v1.ServiceBackendPortArgs(
                                        number=80
                                    ),
                                ),
                            ),
                        ),
                    ],
                ),
            ),
        ],
    ),
)

# Export the Ingress endpoint so we can access it easily.
pulumi.export('ingress_url', ai_ingress.metadata.apply(lambda x: x.name + ".mydomain.com"))
```

This program demonstrates how to create and configure a Kubernetes Ingress to manage the traffic across two AI services running on Kubernetes.

Here are the different components explained:

- `nginx_ingress_controller`: The Helm chart installs NGINX as an Ingress controller. This program uses default values for the installation.
- `ai_service_1_deployment`: This creates a new deployment resource for the first AI service in Kubernetes. Replace `my-ai-service:latest` with the Docker image that you would like to deploy.
- `ai_service_1_svc`: This creates a new Kubernetes Service resource that points to the `ai-service-1` deployment. It is set to run on port 80 and is only accessible within the cluster (because it is of type `ClusterIP`).
- `ai_service_2_deployment` and `ai_service_2_svc`: These create a similar deployment and service for the second AI service as done for the first service.
- `ai_ingress`: This resource creates an Ingress that defines rules for routing traffic to the AI services. You will need to use valid domain names (replacing `ai-service-1.mydomain.com` and `ai-service-2.mydomain.com` with your domains) and ensure that your cluster has access to these domains.

Remember to replace `my-ai-service:latest` with your actual service images. You will need to adjust the domain names in the `ai_ingress` resource to match your actual domain and ensure that your DNS is configured to route traffic to the IP address of your Ingress controller.