High-Performance Compute Instances for LLMs via InstanceTemplate

Question

Pulumi · Accepted Answer

To create high-performance compute instances on Google Cloud Platform (GCP) specifically tailored for running Large Language Models (LLMs) like GPT-3 or BERT, we would typically need a configuration with high CPU, memory, and possibly GPU support. Google Cloud Instances with attached GPUs would be a great fit for this purpose, as GPUs can significantly speed up machine learning computations.

We'll use Pulumi and GCP's Compute Engine to set up an Instance Template. The Instance Template will define the properties for the VMs that we want to run. This template can then be used to create individual VM instances or managed instance groups with consistent settings across all VMs.

An InstanceTemplate in GCP allows you to create a template for the instances you want to build, specifying machine types, disk configurations, network settings, and more. The importance of using an InstanceTemplate is that it helps ensure consistency across instances, and when you need to scale up for high performance, you can do so quickly and efficiently by creating more instances from the same template.

Let's begin with a sample Pulumi program in Python to create an InstanceTemplate designed for high-performance computing:

```python
import pulumi
import pulumi_gcp as gcp

# Define the machine type and GPU settings
# This example uses an `n1-standard-32` machine type with 32 vCPUs and 120 GB of memory.
# Adjust the machine type as needed for your specific LLM workload.
# Add a GPU (like the NVIDIA Tesla V100) to the instance template for accelerated computing.
machine_type = "n1-standard-32"
gpu_type = "nvidia-tesla-v100"
gpu_count = 1

# Create the instance template
# The template includes the machine type, disk settings, network settings, and enables GPU.
high_perf_compute_template = gcp.compute.InstanceTemplate("high-perf-compute-template",
    name="high-perf-compute-template",
    description="Template for high-performance compute instances for LLMs",
    region="us-central1",  # Choose the region that best suits your needs or provides lowest latency
    machine_type=machine_type,
    disks=[{
        "boot": True,
        "autoDelete": True,
        "deviceName": "boot",
        "type": "PERSISTENT",
        "initializeParams": {
            "image": # specify your preferred image, for example:
                     "projects/deeplearning-platform-release/global/images/family/tf-latest-gpu",
            "size": 50,  # disk size in GB, adjust as needed
            "type": "pd-standard"  # or `pd-ssd` for SSD-based storage
        },
    }],
    guest_accelerators=[{
        "type": gpu_type,
        "count": gpu_count
    }],
    network_interfaces=[{
        "network": "default",
        "accessConfigs": [{
            "natIp": None,
            "networkTier": "PREMIUM",
        }],
    }],
    scheduling={
        "automaticRestart": True,
        "onHostMaintenance": "TERMINATE",
        "preemptible": False
    },
    service_account={
        "email": "default",  # Use the default service account or specify another
        "scopes": ["https://www.googleapis.com/auth/cloud-platform"]
    },
)

# Export the selfLink of the created template which can be used to instantiate VMs
pulumi.export("template_selfLink", high_perf_compute_template.self_link)
```

Here's what each part of the program is doing:

- We start by defining the `machine_type` to use. For high performance, it's common to select a machine type with many vCPUs and a high memory capacity. This example uses `n1-standard-32`, which has 32 vCPUs and 120 GB of memory.
- We also define the `gpu_type` as `nvidia-tesla-v100` and set the `gpu_count` to `1`. This indicates that we want each instance created from this template to have one NVIDIA Tesla V100 GPU.
- Next, we create the `InstanceTemplate` using `gcp.compute.InstanceTemplate`. We specify the disk settings, network interfaces, and scheduling details. The image selected for the disk initialization is chosen based on your requirements; in this case, I've used a TensorFlow image which would come pre-installed with the deep learning frameworks supporting GPU acceleration.
- We also define a service account with the necessary access scope. This allows instances created from the template to interact with other GCP services.
- Finally, we export the selfLink of the InstanceTemplate so that it can be referenced later when creating VM instances or instance groups.

After running the above program with Pulumi, you'll have an instance template that you can use to create as many VMs as needed for your workloads with Large Language Models. The consistency and ease of scaling offered by using an instance template are valuable when working with such high-performance needs.