Harbor as a Container Registry for Kubernetes-based AI Applications

Question

Pulumi · Accepted Answer

Harbor is an open-source cloud native registry that stores, signs, and scans content. It's commonly used in Kubernetes environments for managing container images across different stages of the development lifecycle. It's particularly useful when you want to ensure that your images are secure and compliant before being deployed into Kubernetes clusters.

To set up Harbor as a container registry for Kubernetes-based AI applications using Pulumi, you'll typically need to install and configure Harbor on a Kubernetes cluster. Pulumi can help automate the deployment of Harbor with the appropriate configuration, storage, and other cloud resources needed to support it.

Below is a Pulumi program that sets up Harbor on a Kubernetes cluster. The program assumes you have a pre-existing Kubernetes cluster and `pulumi_kubernetes` provider is configured accordingly. If you need to create a Kubernetes cluster, you can also use Pulumi to do so, but that is outside the scope of this program.

Let's walk through the Pulumi program I'm providing:

1. **Harbor components**: We first import the necessary Pulumi packages. We'll use the Harbor package (`pulumi_harbor`) to create Harbor-specific resources like projects and users in the Harbor registry. There isn't a Pulumi component for deploying the Harbor application itself to Kubernetes. Therefore, you would typically use a Helm chart for deploying Harbor.

2. **Kubernetes integration**: We create a Kubernetes namespace for Harbor and deploy the Harbor Helm chart into that namespace. The Helm chart method is standard for deploying applications in Kubernetes and Pulumi can manage Helm charts as part of your infrastructure code.

3. **Configuration**: We configure Harbor's service to be of type `LoadBalancer` so that it's accessible externally. You'll need to specify details in the `values` like the external URL, persistence configurations, etc.

4. **Persistence**: To persist Harbor data, we create a persistent volume claim (PVC). We configure it with ample storage to ensure that the container registry has enough space for your AI application images.

5. **Exporting URL**: At the end of the program, we export the Harbor application URL which you would use to access the Harbor dashboard.

Here is the Pulumi program that you can use as a starting point for integrating Harbor into your Kubernetes-based AI applications:

```python
import pulumi
import pulumi_kubernetes as k8s
from pulumi_kubernetes.helm.v3 import Chart, ChartOpts

# Configure the Harbor chart's values
# You would typically fetch these configuration values from your environment.
harbor_values = {
    'expose': {
        'type': 'LoadBalancer',
        'tls': {
            'enabled': True,
        },
        'externalURL': 'https://harbor.mycompany.com',
    },
    'persistence': {
        'enabled': True,
        'size': '10Gi',
    },
    'database': {
        'password': 'changeit',  # Replace with your database password
    },
    'harborAdminPassword': 'changeit',  # Replace with a strong admin password
    # ... include other necessary Harbor configuration values here
}

# Create a Kubernetes namespace for Harbor
harbor_namespace = k8s.core.v1.Namespace(
    'harbor-ai-applications-ns',
    metadata={'name': 'harbor-ai-applications'}
)

# Deploy the Harbor Helm chart in the created namespace
harbor_chart = Chart(
    'harbor',
    ChartOpts(
        chart='harbor',
        version='1.6.0',  # Use the version of Harbor chart that suits your needs
        fetch_opts={'repo': 'https://helm.goharbor.io'},
        namespace=harbor_namespace.metadata['name'],
        values=harbor_values,
    ),
)

# Export the Harbor URL to access the dashboard
harbor_application_url = harbor_chart.getResourceProperty('v1/Service', 'harbor/harbor-harbor-portal', 'status').apply(
    lambda status: status['load_balancer']['ingress'][0]['ip']
)

pulumi.export('harbor_application_url', harbor_application_url)
```

To run this Pulumi program, you would need to have Pulumi installed and configured for your Kubernetes cluster.

This program starts by setting up configurations to deploy Harbor using its Helm chart. It specifies values for service exposure, persistence options, and administrative credentials. These values are crucial for setting up Harbor according to your specific environment and security requirements.

A unique namespace is created for Harbor to ensure that it does not interfere with other applications deployed to the same Kubernetes cluster.

Next, the Harbor Helm chart is deployed to the Kubernetes cluster within the created namespace. The version of the Harbor Helm chart is specified; make sure to use the version that suits your environment or is considered stable at the time you deploy.

Finally, the external IP address of the Harbor service is extracted from its load balancer status and exported. This IP is how users and CI/CD systems would interact with Harbor to push or pull images.

Please note that the passwords and critical data in the `harbor_values` configuration should be secured. It's recommended to use Pulumi's secrets management or replace them with your existing secrets management solution.

In addition, depending on your cloud provider or Kubernetes setup, the way services are exposed and how LoadBalancers are set up may differ. You should adjust the configuration to match your cloud provider's requirements and best practices.