Ensuring consistent egress IP for services across GKE nodes

Question

Pulumi · Accepted Answer

To ensure consistent egress IP for services across Google Kubernetes Engine (GKE) nodes, you'll need to set up a network with a NAT gateway that will direct all outbound traffic through it. This way, regardless of which node the traffic originates from, it will appear to external services as coming from a single IP address.

The process involves the following steps:

Create a GKE cluster: You need a Kubernetes cluster where your services are running.
Configure Cloud NAT: Cloud NAT (Network Address Translation) allows VM instances without external IP addresses and private clusters to connect to the internet in a controlled and efficient manner. All the egress traffic from nodes is routed through the NAT gateway ensuring a consistent IP.
Configure GKE Node Pool: To ensure that all nodes in the cluster are part of the same network and are configured to use Cloud NAT, you will configure the node pool with the proper settings.
Configure Firewall Rules: To appropriately filter incoming and outgoing traffic and ensure that your services can communicate with external addresses, you must create firewall rules.

In Pulumi, you would use the gcp and kubernetes providers to accomplish this. Here's how you can create such an infrastructure using Pulumi with TypeScript:

import * as gcp from "@pulumi/gcp";
import * as k8s from "@pulumi/kubernetes";

// Create a GKE cluster
const cluster = new gcp.container.Cluster("my-cluster", {
  // ... configure your GKE cluster
});

// Create a Cloud Router as this is needed for Cloud NAT
const router = new gcp.compute.Router("my-router", {
  network: "<YOUR-VPC-NETWORK>",
  region: "<REGION>",
});

// Create the Cloud NAT configuration for your project
const nat = new gcp.compute.Address("my-nat", {
  name: "my-nat",
  region: "<REGION>",
});

const cloudNat = new gcp.compute.RouterNat("my-router-nat", {
  router: router.name,
  region: "<REGION>",
  natIpAllocateOption: "MANUAL_ONLY",
  natIps: [nat.address],
  sourceSubnetworkIpRangesToNat: "ALL_SUBNETWORKS_ALL_IP_RANGES",
});

// Then link NAT to the node pool
const nodePool = new gcp.container.NodePool("my-node-pool", {
  cluster: cluster.name,
  nodeConfig: {
    // ... other necessary configurations for your nodes
  },
  // Make sure to set this to a region that supports Cloud NAT
  region: "<REGION>",
});

const provider = new k8s.Provider("k8s-provider", {
  kubeconfig: cluster.kubeconfig,
});

// (Optional) Here you might define your Kubernetes deployments or services
// as needed which will automatically use the egress IP established via Cloud NAT.

// Export the cluster's kubeconfig and NAT IP to access them outside Pulumi
export const kubeconfig = cluster.name.apply(name => cluster.kubeconfig);
export const natIp = nat.address;

In this program:

Replace <YOUR-VPC-NETWORK> with the name of your VPC network and <REGION> with your region, e.g., us-central1.
The GKE cluster is created without specifying details in this example. You need to fill in the ... configure your GKE cluster part based on your requirements.
A Google Cloud Router is created, which is necessary for setting up Cloud NAT.
The Cloud NAT is configured with the option MANUAL_ONLY for natIpAllocateOption, and the dedicated NAT IP is associated. This is a crucial step for ensuring consistent egress IP.
A GKE node pool is specified with the region which supports Cloud NAT.
The Kubernetes provider is set up with the cluster's kubeconfig, allowing you to deploy Kubernetes resources.
Lastly, the program exports the kubeconfig and the NAT IP. The kubeconfig is necessary for interacting with the cluster via kubectl, and the NAT IP can be used to know the egress IP address of your services.

Remember to ensure that your Pulumi stack is configured with the appropriate GCP project and credentials.

This code provides the basic infrastructure to ensure all outbound traffic from all the nodes in the GKE cluster will have a consistent egress IP. Additional configuration may be required depending on your application and network requirements.