Pulumi Blog: Aks

How Elkjøp Nordic enables self-service infrastructure for developers

George Huang — Fri, 01 Jul 2022 00:00:00 +0000

At PulumiUP 2022, Tomas Jansson, software architect at Elkjøp Nordic, gave a presentation on how to enable developers to self-service infrastructure by using Pulumi’s Automation API.

Elkjøp Nordic is the leading consumer electronics retailer in the Nordics. The company sells consumer electronics, mobile phones, computers, white goods, domestic appliances, and services linked to these products both directly to consumers and to businesses. It is an omnichannel retailer and serves customers both online and through more than 400 stores. Elkjøp has retail outlets in Norway, Sweden, Finland and Denmark, and franchise operations in Greenland and Faroe Islands.

To support the company’s modernization strategy, Tomas developed a self-service infrastructure platform that would enable any developer to easily provision cloud infrastructure that uses approved resources and security practices. The goal was to empower teams at Elkjøp to increase their development velocity by using the cloud, while maintaining guardrails that uphold security and compliance.

Watch the full presentation below or read on to learn the highlights of his presentation:

The Problem: Balancing Agility and Governance

To achieve his goals, Tomas knew he had to solve the problem of balancing freedom with control. On the one hand, he wanted to foster the creativity of his developers and encourage innovation. He wanted his team to feel they were trusted and had the room to create the best solutions they could. On the other hand, he knew he had to maintain some control. Elkjøp needed to keep its applications secure. It also needed good governance so that teams can keep track of who is making changes. Tomas also wanted to limit the impact of deploying untested code. Code that was still being developed needed to be confined to the developer’s own environment. Finding the right balance between independence and control was paramount. Tomas believed this would result in great software and in developers who would enjoy their work.

The Solution: A Self-Service Infrastructure Platform

Tomas’ solution was to build a self-service infrastructure platform. Self-service infrastructure makes it possible for developers to easily access the infrastructure they need while limiting access to resources they don’t need. In effect, self-service infrastructure is a walled garden. Within the garden, developers have all the freedom they need. However, nothing they do affects what’s outside the wall. Code that causes unintended changes only affects them and no one else.

Tomas built an application that provides a “walled garden” of self-service infrastructure. Developers can create the environments they need, including an Azure resource group and a Kubernetes namespace, all through a graphical user interface. Developers can define whether the environment is for development, test or production. They can also assign values, such as which team members should have admin privileges to the associated GitHub repositories or who can contribute. From there, the platform handles provisioning and configuring all of these resources.

Here is an example that shows how a developer can create a repository simply by selecting a few parameters from a drop-down list.

Tomas determined that the best way to implement self-service infrastructure was with Pulumi and its Automation API. By using Pulumi, he could use standard programming languages to define infrastructure while using familiar software tools like IDEs and test frameworks. Pulumi also natively supports Azure and Kubernetes with 100% coverage of each provider’s APIs and same-day access to new features and updates. Finally, the Pulumi Automation API would enable him to program infrastructure automation directly into his application.

The Pulumi Automation API is a programmatic interface for running Pulumi programs without the Pulumi CLI. Automation API encapsulates the functionality of the CLI (pulumi up, pulumi preview, pulumi destroy, pulumi stack init, etc.) but with more flexibility since infrastructure operations can be controlled programmatically.

How the self-service infrastructure platform was built

Here is the workflow for the Elkjøp self-service infrastructure platform.

The self-service platform is a web application that runs the Pulumi Automation API on the server side. The application uses Pulumi to provision and configure every environment resource such as Azure resource groups, Kubernetes namespaces, service principals, and Kubernetes service accounts. Each environment comes with a Git repository that’s used by developers to manage their deployments. The Git repositories are the locus of each environment because they store everything needed to deploy changes such as infrastructure code and environment credentials.

Using the Pulumi Automation API, the application creates an Azure resource group, a team in both Azure AD and GitHub, and a new GitHub repository. The application then grants administrative access to the GitHub repo to members of the AD group. In addition to creating a Git repository for each environment, the application uses stack references for each Azure environment to inject the associated credentials into the GitHub Secret Store. These credentials are scoped to each GitHub environment so that developers can only work in their own environment. If a team has multiple environments, each environment has its own credentials so developers can’t, for example, deploy from a test environment to a production environment. Secrets are managed through the Pulumi Service.

Every new repository comes with Pulumi programs that developers can use to deploy their application. GitHub Actions initiate the deployment. When an action triggers, Pulumi reads the credentials that were generated by the self-service application, allowing Pulumi to deploy new Kubernetes or Azure resources to the environment.

Watch the demo

To illustrate how Elkjøp’s self-service platform works, Tomas presented a simplified demo application that’s based on Elkjøp’s production platform. The code for the application is available at https://github.com/mastoj/pulumi-automation-demo.

The demo is a TypeScript application that includes an API that provisions Azure and GitHub resources. Pulumi Automation API code is used to deploy new stacks by calling handler code, which is a Pulumi program that defines resources and outputs. The handler code can create Azure resource groups or GitHub repositories, and stack references are used so that outputs generated by the Azure resources can be stored in GitHub (e.g., resource group credentials).

Ready to dive into how this works in detail? Watch the demo in the video below starting around 10:20.

Next steps

After watching the demo of a self-service infrastructure platform, you can try it out for yourself by checking out the demo code. You can also learn more about Pulumi Automation API in our documentation.

If you’re new to Pulumi, then follow our getting started guide.

Top 5 Things an Azure Developer Needs to Know: Kubernetes Infrastructure

Sophia Parafina — Thu, 12 Aug 2021 00:00:00 +0000

History lesson time! In 2011, microservices debuted as an architectural style suited for the cloud. In 2013, Docker simplified building containers. Combining containers and microservices sparked a change in how applications were built and distributed in the cloud. As performance, scaling, and reliability became an increasing concern, container orchestration platforms became widely available. Kubernetes became the dominant container orchestration through community and corporate support, and some have suggested it was inevitable. Every major cloud service provider, including Azure, offers a version of Kubernetes.

Kubernetes streamlines container deployment and management, making applications scale and accessible. This article demonstrates configuring and deploying Kubernetes with Azure.

A Kubernetes Review

If you’re not familiar with Kubernetes concepts and terminology, the Getting Started with Kubernetes series can help get you up to speed.

Azure Kubernetes Service

Azure Kubernetes Service (AKS) is a hosted Kubernetes service. Azure manages the Kubernetes master nodes, and you are responsible for managing the agent or worker nodes. You only pay for the worker nodes in your cluster that make up your application.

Kubernetes nodes are the worker machines that can be either physical or virtual. AKS nodes use Azure virtual machines (VMs); and you can add storage, upgrade cluster components, or even run multiple node pools with mixed operating systems.

You can create an AKS cluster with

the Azure CLI,
the Azure portal,
PowerShell, and
templates, such as Azure Resource Manager (ARM) templates.

AKS features

Configuring and deploying Kubernetes can be complex. AKS provides many features to simplify the process, including the following features:

Cluster node and pod scaling adjust the number of nodes or pods as resource demand changes. Pods can scale horizontally, and the cluster can also autoscale.
Kubernetes is a fast-moving open source project, and AKS supports multiple versions. You can upgrade the cluster nodes without interrupting the application.
If you perform machine learning, AKS can create GPU-enabled node pools for compute-intensive, graphics-intensive, and visualization workloads.
AKS supports access control through Kubernetes RBAC and Azure Active Directory.
Integrated logging and monitoring with Azure Monitor for Containers collects memory and processor metrics from containers, nodes, and controllers.
AKS lets you mount static or dynamic storage volumes for persistent data. Storage volumes use Azure Disks for single pod access or Azure Files for concurrent pod access.
Virtual networking for pods in the cluster and other nodes let you use a peered virtual network. The network also provides ingress for applications with HTTP application routing.
Integration with development tools for Helm and a Visual Studio Code extension for Kubernetes improves the developer experience.
Support for Docker images, including storage with Azure Container Registry, makes deployment easier.

Create an AKS Cluster with the Azure Portal

We’ll use the Azure Portal to illustrate the steps to configure and deploy an AKS Cluster.

Step 1: Create a Kubernetes Service.

Open the Azure Portal and select Create Resource.
Select Containers > Kubernetes Service.
Select Create.

Step 2: Create an AKS cluster.

For this example, we will configure several options in the Basic window but use defaults for most options.

In Subscription, select an Azure Subscription.
Create or select a Resource Group.
In Cluster details, set the Kubernetes cluster name.
Change the Region if needed.

Use the default values for Primary node pool, and select Next: node pools.

Step 3: Configure node pools.

A node pool is a logical grouping of nodes. Nodes in a pool can have different virtual machines, different Kubernetes versions, and other attributes. You can use different node pools for different purposes, such as grouping workflows, e.g., a node pool for production and one for dev or test.

For this example, use the default values for node pools.

Step 4: Configure authentication.

You can authenticate, authorize, secure, and control access to Kubernetes clusters with

Kubernetes role-based access control (Kubernetes RBAC) to grant access to service accounts or
Azure Active Directory integration that provides a multi-tenant, cloud-based directory and identity management service.

To configure Authentication for this example, set the Authentication method to System-assigned managed identity. You can set Role-based access control (RBAC) to enabled, but it’s unnecessary for the example. Use the default value for the Node pool OS disk encryption type.

Step 5: Configure networking

AKS can use either kubenet or Azure CNI networking. Kubenet is the default configuration for AKS cluster creation.

With kubenet, Azure creates and configures the virtual network. However, only the nodes receive a routable IP address and pods use a NAT to communicate with resources outside the AKS cluster. This approach reduces the number of IP addresses you need to reserve in your network space for pods to use.

Azure CNIassigns an IP address to every pod, which makes them directly accessible. The IP addresses must be unique and planned in advance. Each node must be configured for the maximum number of pods to reserve IP addresses per node. Without accounting for the number of pods, the network can run out of IP addresses to allocate or necessitate the need to rebuild clusters with a larger subnet.

For this example, we’ll use kubenet for it’s simplicity, but in production Azure CNI maybe the better choice for applications.

Step 6: Complete the deployment

Select Review + create to deploy. Deployment takes several minutes to complete. On completion, you can verify the Deployment details or select Connect to cluster.

Step 7: Connect to the cluster

You can manage your cluster with kubectl, a command line tool for managing Kubernetes. You can install kubectl on linux, macOS, and Windows.

Copy and paste the commands to connect and authenticate to your cluster using the Azure CLI.

After connecting and authenticating to your cluster, you can use kubectl to query your cluster.

$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
aks-agentpool-19694923-vmss000000 Ready agent 9m35s v1.20.7
aks-agentpool-19694923-vmss000001 Ready agent 9m39s v1.20.7
aks-agentpool-19694923-vmss000002 Ready agent 9m47s v1.20.7
$ kubectl get all
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 11m

Your AKS cluster is deployed and running.

Deploying an AKS cluster with code

Now that we have reviewed the process for creating an AKS cluster using the Azure Portal, we can repeat the process using code.

Step 1: Create a Resource group and a Service principal for the cluster. Note that we used a System-assigned managed identity in the Azure Portal example.

 // Create an Azure Resource Group
var resourceGroup = new ResourceGroup("azure-cs-aks");
// Create an AD service principal
var adApp = new Application("aks", new ApplicationArgs
{
DisplayName = "aks"
});
var adSp = new ServicePrincipal("aksSp", new ServicePrincipalArgs
{
ApplicationId = adApp.ApplicationId
});
// Generate random password
var password = new RandomPassword("password", new RandomPasswordArgs
{
Length = 20,
Special = true
});
// Create the Service Principal Password
var adSpPassword = new ServicePrincipalPassword("aksSpPassword", new ServicePrincipalPasswordArgs
{
ServicePrincipalId = adSp.Id,
Value = password.Result,
EndDate = "2099-01-01T00:00:00Z"
});
// Generate an SSH key
var sshKey = new PrivateKey("ssh-key", new PrivateKeyArgs
{
Algorithm = "RSA",
RsaBits = 4096
});

 // Create an Azure Resource Group
resourceGroup, err := resources.NewResourceGroup(ctx, "azure-go-aks", nil)
if err != nil {
return err
}
// Create an AD service principal.
adApp, err := azuread.NewApplication(ctx, "aks", &azuread.ApplicationArgs{
DisplayName: pulumi.String("aks"),
})
if err != nil {
return err
}
adSp, err := azuread.NewServicePrincipal(ctx, "aksSp", &azuread.ServicePrincipalArgs{
ApplicationId: adApp.ApplicationId,
})
if err != nil {
return err
}
// Generate a random password.
password, err := random.NewRandomPassword(ctx, "password", &random.RandomPasswordArgs{
Length: pulumi.Int(20),
Special: pulumi.Bool(true),
})
if err != nil {
return err
}
// Create the Service Principal Password.
adSpPassword, err := azuread.NewServicePrincipalPassword(ctx, "aksSpPassword", &azuread.ServicePrincipalPasswordArgs{
ServicePrincipalId: adSp.ID(),
Value: password.Result,
EndDate: pulumi.String("2099-01-01T00:00:00Z"),
})
if err != nil {
return err
}
// Generate an SSH key.
sshArgs := tls.PrivateKeyArgs{
Algorithm: pulumi.String("RSA"),
RsaBits: pulumi.Int(4096),
}
sshKey, err := tls.NewPrivateKey(ctx, "ssh-key", &sshArgs)
if err != nil {
return err
}

 # Create new resource group
resource_group = resources.ResourceGroup("azure-native-py-aks")
# Create an AD service principal
ad_app = azuread.Application("aks", display_name="aks")
ad_sp = azuread.ServicePrincipal("aksSp", application_id=ad_app.application_id)
# Generate random password
password = random.RandomPassword("password", length=20, special=True)
# Create the Service Principal Password
ad_sp_password = azuread.ServicePrincipalPassword("aksSpPassword",
service_principal_id=ad_sp.id,
value=password.result,
end_date="2099-01-01T00:00:00Z")
# Generate an SSH key
ssh_key = tls.PrivateKey("ssh-key", algorithm="RSA", rsa_bits=4096)

 // Create an Azure Resource Group
const resourceGroup = new resources.ResourceGroup("azure-go-aks");
// Create an AD service principal
const adApp = new azuread.Application("aks", {
displayName: "aks",
});
const adSp = new azuread.ServicePrincipal("aksSp", {
applicationId: adApp.applicationId,
});
// Generate random password
const password = new random.RandomPassword("password", {
length: 20,
special: true,
});
// Create the Service Principal Password
const adSpPassword = new azuread.ServicePrincipalPassword("aksSpPassword", {
servicePrincipalId: adSp.id,
value: password.result,
endDate: "2099-01-01T00:00:00Z",
});
// Generate an SSH key
const sshKey = new tls.PrivateKey("ssh-key", {
algorithm: "RSA",
rsaBits: 4096,
});

Step 2: Configure the AKS cluster

We set the configuration options in ManagedClusterAgentPoolProfileArgs:

Count: Number of virtual machines
MaxPods: Maximum number of pods that can run on a node.
Mode: Sets the type of pool node, which can be system or user
Name: Name of the pool node
OsType: Specifies the OS type
Type: Choose between a VirtualMachineScaleSet or an AvailabilitySet
VmSize: The virtual machine size used by the cluster.

The remaining parameters set the Kubernetes version, enables RBAC, and configures the Linux profile for the ContainerService.

 var cluster = new ManagedCluster("my-aks", new ManagedClusterArgs
{
ResourceGroupName = resourceGroup.Name,
AgentPoolProfiles =
{
new ManagedClusterAgentPoolProfileArgs
{
Count = 3,
MaxPods = 110,
Mode = "System",
Name = "agentpool",
OsDiskSizeGB = 30,
OsType = "Linux",
Type = "VirtualMachineScaleSets",
VmSize = "Standard_DS2_v2",
}
},
DnsPrefix = "AzureNativeprovider",
EnableRBAC = true,
KubernetesVersion = "1.18.14",
LinuxProfile = new ContainerServiceLinuxProfileArgs
{
AdminUsername = "testuser",
Ssh = new ContainerServiceSshConfigurationArgs
{
PublicKeys =
{
new ContainerServiceSshPublicKeyArgs
{
KeyData = sshKey.PublicKeyOpenssh,
}
}
}
},
NodeResourceGroup = $"MC_azure-cs_my_aks",
ServicePrincipalProfile = new ManagedClusterServicePrincipalProfileArgs
{
ClientId = adApp.ApplicationId,
Secret = adSpPassword.Value
}
});

 // Create the Azure Kubernetes Service cluster.
cluster, err := containerservice.NewManagedCluster(ctx, "go-aks", &containerservice.ManagedClusterArgs{
ResourceGroupName: resourceGroup.Name,
AgentPoolProfiles: containerservice.ManagedClusterAgentPoolProfileArray{
&containerservice.ManagedClusterAgentPoolProfileArgs{
Name: pulumi.String("agentpool"),
Mode: pulumi.String("System"),
OsDiskSizeGB: pulumi.Int(30),
Count: pulumi.Int(3),
VmSize: pulumi.String("Standard_DS2_v2"),
OsType: pulumi.String("Linux"),
},
},
LinuxProfile: &containerservice.ContainerServiceLinuxProfileArgs{
AdminUsername: pulumi.String("testuser"),
Ssh: containerservice.ContainerServiceSshConfigurationArgs{
PublicKeys: containerservice.ContainerServiceSshPublicKeyArray{
containerservice.ContainerServiceSshPublicKeyArgs{
KeyData: sshKey.PublicKeyOpenssh,
},
},
},
},
DnsPrefix: resourceGroup.Name,
ServicePrincipalProfile: &containerservice.ManagedClusterServicePrincipalProfileArgs{
ClientId: adApp.ApplicationId,
Secret: adSpPassword.Value,
},
KubernetesVersion: pulumi.String("1.18.14"),
})
if err != nil {
return err
}

 # Create cluster
managed_cluster_name = config.get("managedClusterName")
if managed_cluster_name is None:
managed_cluster_name = "azure-native-aks"
managed_cluster = containerservice.ManagedCluster(
managed_cluster_name,
resource_group_name=resource_group.name,
agent_pool_profiles=[{
"count": 3,
"max_pods": 110,
"mode": "System",
"name": "agentpool",
"node_labels": {},
"os_disk_size_gb": 30,
"os_type": "Linux",
"type": "VirtualMachineScaleSets",
"vm_size": "Standard_DS2_v2",
}],
enable_rbac=True,
kubernetes_version="1.18.14",
linux_profile={
"admin_username": "testuser",
"ssh": {
"public_keys": [{
"key_data": ssh_key.public_key_openssh,
}],
},
},
dns_prefix=resource_group.name,
node_resource_group=f"MC_azure-native-go_{managed_cluster_name}_westus",
service_principal_profile={
"client_id": ad_app.application_id,
"secret": ad_sp_password.value
})

 const cluster = new containerservice.ManagedCluster(managedClusterName, {
resourceGroupName: resourceGroup.name,
agentPoolProfiles: [{
count: 3,
maxPods: 110,
mode: "System",
name: "agentpool",
nodeLabels: {},
osDiskSizeGB: 30,
osType: "Linux",
type: "VirtualMachineScaleSets",
vmSize: "Standard_DS2_v2",
}],
dnsPrefix: resourceGroup.name,
enableRBAC: true,
kubernetesVersion: "1.18.14",
linuxProfile: {
adminUsername: "testuser",
ssh: {
publicKeys: [{
keyData: sshKey.publicKeyOpenssh,
}],
},
},
nodeResourceGroup: `MC_azure-go_${managedClusterName}`,
servicePrincipalProfile: {
clientId: adApp.applicationId,
secret: adSpPassword.value,
},
});

Step 3: Export the kubeconfig file

A kubeconfig file organizes information about clusters and allows kubectl connect to the cluster.

...
// Export the KubeConfig
this.KubeConfig = Output.Tuple(resourceGroup.Name, cluster.Name).Apply(names =>
GetKubeConfig(names.Item1, names.Item2));
}
[Output]
public Output<string> KubeConfig { get; set; }
private static async Task<string> GetKubeConfig(string resourceGroupName, string clusterName)
{
var credentials = await ListManagedClusterUserCredentials.InvokeAsync(new ListManagedClusterUserCredentialsArgs
{
ResourceGroupName = resourceGroupName,
ResourceName = clusterName
});
var encoded = credentials.Kubeconfigs[0].Value;
var data = Convert.FromBase64String(encoded);
return Encoding.UTF8.GetString(data);
}

 ctx.Export("kubeconfig", pulumi.All(cluster.Name, resourceGroup.Name, resourceGroup.ID()).ApplyT(func(args interface{}) (string, error) {
clusterName := args.([]interface{})[0].(string)
resourceGroupName := args.([]interface{})[1].(string)
creds, err := containerservice.ListManagedClusterUserCredentials(ctx, &containerservice.ListManagedClusterUserCredentialsArgs{
ResourceGroupName: resourceGroupName,
ResourceName: clusterName,
})
if err != nil {
return "", err
}
encoded := creds.Kubeconfigs[0].Value
kubeconfig, err := base64.StdEncoding.DecodeString(encoded)
if err != nil {
return "", err
}
return string(kubeconfig), nil
}))

 # Export kubeconfig
encoded = creds.kubeconfigs[0].value
kubeconfig = encoded.apply(
lambda enc: base64.b64decode(enc).decode())
pulumi.export("kubeconfig", kubeconfig)

 const creds = pulumi.all([cluster.name, resourceGroup.name]).apply(([clusterName, rgName]) => {
return containerservice.listManagedClusterUserCredentials({
resourceGroupName: rgName,
resourceName: clusterName,
});
});
const encoded = creds.kubeconfigs[0].value;
export const kubeconfig = encoded.apply(enc => Buffer.from(enc, "base64").toString());

Step 4: Connect to the AKS cluster

We can use the kubeconfig file connect to the cluster. It’s common practice to copy the kubeconfig file to ~/.kube/config, which is the directory that kubectl looks for the file. However, you can use any directory with the --kubeconfig flag, e.g.,

$ kubectl --kubeconfig /path/to/kubeconfig_file get pods

As this example shows, deploying an AKS cluster requires setting parameters. This action can be done in the portal or with code. The advantage of code is that clusters can be created on demand without having to use the Azure portal. This setup is convenient when you have different environments, such as a dev/test environment and a production environment.

The complete code for deploying AKS is available on GitHub for Typescript, Python, Go, and C#.

Summary

Azure Kubernetes Service lets you deploy a Kubernetes cluster quickly and efficiently. An AKS deployment configures the worker nodes since the master nodes are provided by Azure. The Azure Portal provides a simplified interface for configuration, but if you want fine-grain control over the deployment, infrastructure as code is an option. This is particularly true for production deployments where the configuration requires tuning for an application deployment.

In the next article in this series, we’ll deploy an application on an AKS cluster using different methods.

Top 5 Things an Azure Developer Needs to Know: Introduction

Sophia Parafina — Mon, 02 Aug 2021 00:00:00 +0000

The Azure cloud platform includes over 200 products and cloud services. Wherever you are in your Microsoft cloud engineering journey, you should be familiar with these top 5 cloud tasks that are essential building blocks commonly used to deploy applications and infrastructure to the Azure cloud.

In this series of articles, we’ll go in-depth on virtual machines, Azure Functions, static websites, building an Azure Kubernetes Service cluster and deploying applications on AKS, and DevOps with Azure App Service.

Azure - a comprehensive cloud platform

With Azure, you can develop, test, deploy and manage your applications in an integrated cloud computing environment. With the Azure Portal, you can manage the cloud infrastructure for your applications. Azure gives you the choice of a portal for managing services or managing them programmatically using service APIs and templates.

With over 200 services, knowing which one to use can be confusing. Deciding where to host your application is the first step. This series of articles covers the most commonly used Azure services such as virtual machines, serverless functions, static websites, Kubernetes, and Azure App Service. In addition, we’ll cover how to deploy applications using Azure DevOps.

In this article, we’ll match use cases to cloud services to help you navigate Azure.

Azure Virtual Machines

A common cloud migration path is to use virtual machines if you have existing infrastructure or on-premise applications. It is especially true if you want complete control over the infrastructure and application environment.

Azure provides Infrastructure as a Service (IaaS) for deploying your applications on either Linux or Windows VMs. IaaS includes all necessary support, such as Azure Virtual Network and storage. As with any IaaS solution, you are responsible for the installation of software, configuration, and maintenance of your application and any maintenance the VM might require, such as operating system patches.

If your application doesn’t fit into a Platform as a Service (PaaS) model, consider using virtual machines to provide the fine-grained control required for your deployment.

Serverless

At the opposite end of cloud infrastructure is serverless computing. Serverless eliminates the need to manage infrastructure and lets you focus on writing and deploying code. Azure automatically provisions, scales, and manages the infrastructure. You can write code that runs in response to events or on a schedule.

Azure Functions lets you focus on the application’s business logic, and the code executes when called by a webhook, an HTTP or REST request, a schedule, or an event. Serverless is particularly appealing because you can use the development language of your choice, ranging from PHP, Python, Node.js, or .NET languages. Azure functions scale as needed, and consumption-based pricing means that you’re charged only when the code runs.

If you don’t need a completely hosted project and only pay for what you use, choose Azure Functions triggered by web-based events, a schedule, or other Azure services.

Static websites

Static websites are quickly becoming the defacto way to publish content. Built from libraries and frameworks that don’t use server side rendering like React, Angular, or Vue, they include HTML, CSS, image assets, and Javascript in the application. Instead of serving assets from a server or servers scaled horizontally, the static assets are distributed through a Content Delivery Network (CDN). This reduces latency, resulting in faster websites and increased customer satisfaction.

Azure provides two ways to deploy static websites. Azure lets you deploy a static web application with Azure App Service and the recently released Azure Static Web Apps. Azure App Service deploys websites by uploading the website to Azure storage configured to serve web content. Azure App Service automatically builds and deploys full stack web apps to Azure from a code repository, which follows the Jamstack model for developing and deploying websites.

We’ll look at both methods and discuss the pros and cons of both methods.

Kubernetes

Kubernetes is the leading container orchestration service. Azure Kubernetes Service (AKS) is a hosted service that can deploy, scale, and manage containerized applications in a cluster. AKS fulfills many different use cases, for example:

If you have an existing application, think n-tier applications, you can containerize the components and “lift and shift” your application to run in Kubernetes.
You can deploy microservices on AKS to add horizontal scaling, load balancing, secrets management, and self-healing.
Data scientists are training machine learning models on AKS using TensorFlow and Kubeflow.
You can even use AKS as part of a build pipeline by pairing with a CI, such as Jenkins, to build containers for your application.

AKS supports many use cases by providing provisioning, scaling, and resource upgrades without requiring expertise in container orchestration. With AKS, the Kubernetes master nodes are managed by Azure, and you are only charged for the worker nodes in your cluster.

We’ll do two separate deep dives into deploying AKS clusters and deploying applications in AKS in this series.

Deploy AKS

Azure provides many ways to deploy an AKS cluster, including:

When you deploy an AKS Cluster, the worker nodes and Kubernetes master are configured with a basic configuration. You can additionally configure networking, storage, monitoring, and policies. We’ll cover configuring AKS in this series.

Deploy Applications

Deploying applications on AKS is equally flexible, and you can deploy using:

kubectl and YAML manifests
Helm, an open-source Kubernetes packaging tool
Azure Pipelines Infrastructure as Code

We’ll examine these options and do a deep dive on application deployment in a follow-up article.

DevOps integration

Before Continuous integration (CI) and Continuous Deployment (CD) became prevalent, applications were manually built, compiled, and maintained in a large codebase. CI/CD automation lets you build quickly, test, and deploy services. Both application fixes and new features are shipped quickly to your customers, and you can rapidly respond to changing customer demands. For example, if demand increases, you can quickly add resources to your application or add a serverless function that increases and decreases resources as needed.

If you’re deploying applications on Azure Functions, virtual machines, AKS, or provisioning infrastructure, the build process should be automated. Azure offers a range of services such as Azure Container Registry, AKS, and Azure Cosmos DB that can integrate with popular CI/CD pipelines like Azure DevOps, Jenkins, GitHub, and TeamCity.

If you’re interested in using Azure DevOps to automate infrastructure and application delivery, stay tuned!

Let’s Get Started!

So here are the top 5 things essential to know as an Azure developer:

configuring and deploying virtual machines
developing and deploying Azure Functions
deploying static websites on Azure
configuring Azure Kubernetes Service and deploying containerized applications
using Azure DevOps to automate the building and deployment of infrastructure and applications

First up and next in this series of articles is deploying Azure virtual machines with an application.

Multicloud Kubernetes: Running Apps Across EKS, AKS, and GKE

Mike Metral — Wed, 14 Aug 2019 00:00:00 +0000

Multi-cloud Kubernetes means running clusters on more than one cloud provider, typically some mix of Amazon EKS, Azure AKS, and Google GKE, and managing them through one consistent workflow instead of three separate ones. Teams take this on to avoid vendor lock-in, meet data-residency requirements, or let each application team run on the cloud that fits its workload.

The catch is that EKS, AKS, and GKE each provision and configure clusters differently, so treating them as one target complicates cluster provisioning, application deployment, and CI/CD. If you also need to run the same application across providers for testing or resilience, details like load balancer outputs and cluster connection settings become one more thing to keep in sync by hand.

This post shows how to provision Kubernetes clusters on EKS, AKS, and GKE with Pulumi, and how to deploy the same application to all three from one codebase. We’ll use TypeScript for the full walkthrough, with a Python equivalent for the AWS cluster to show the pattern carries across languages.

![Architecture diagram showing an application deployed identically across EKS, AKS, and GKE clusters](multicloud.png)

Why run Kubernetes across multiple clouds?

A few situations make multi-cloud Kubernetes worth the extra coordination:

Avoiding lock-in. Workloads that run identically on EKS, AKS, and GKE keep you free to negotiate on price or move a workload if a provider’s roadmap or pricing changes.
Meeting compliance and data-residency requirements. Some customers or regulations require workloads to run on a specific provider or in a specific region that only one cloud serves well.
Matching teams to the cloud they already use. In a company built through acquisition, or one where different teams standardized on different clouds, multi-cloud Kubernetes lets each team keep its cloud without forking the deployment pipeline.
Testing across environments. Running the same app on EKS, AKS, and GKE surfaces provider-specific bugs, such as differences in default storage classes or load balancer behavior, before they reach customers.

What are the challenges of multi-cloud Kubernetes?

Multi-cloud Kubernetes is a real tradeoff, not a free upgrade. Cluster provisioning APIs, IAM models, default networking, and managed add-ons differ across EKS, AKS, and GKE, so a setup that is one API call on one provider can be several resources on another. Application manifests are mostly portable, but anything that touches a provider’s native load balancer, storage class, or identity system needs provider-specific configuration. Running clusters on three providers also means three sets of the provider’s own operational quirks, upgrade cadences, and support channels to track. Most teams find multi-cloud Kubernetes worth it only once the reason for doing it (compliance, team autonomy, or genuine lock-in risk) is concrete, rather than adopting it as a default posture.

How do you provision Kubernetes clusters across EKS, AKS, and GKE with Pulumi?

Pulumi provisions each provider’s managed Kubernetes clusters using ordinary TypeScript, Python, Go, C#, Java, or YAML, so cluster definitions get the same code review, testing, and reuse as the rest of your infrastructure.

Provision an EKS cluster with the @pulumi/eks package, which wraps the underlying VPC, IAM, and node group resources behind a single component:

import * as eks from "@pulumi/eks";

const eksCluster = new eks.Cluster("eks-cluster", {
 instanceType: "t3.medium",
 desiredCapacity: 2,
 minSize: 1,
 maxSize: 3,
});

export const eksKubeconfig = eksCluster.kubeconfig;

Provision an AKS cluster with the azure-native provider, which maps directly onto the Azure Resource Manager API:

import * as resources from "@pulumi/azure-native/resources";
import * as containerservice from "@pulumi/azure-native/containerservice";

const resourceGroup = new resources.ResourceGroup("aks-rg");

const aksCluster = new containerservice.ManagedCluster("aks-cluster", {
 resourceGroupName: resourceGroup.name,
 agentPoolProfiles: [{
 name: "agentpool",
 count: 2,
 vmSize: "Standard_DS2_v2",
 mode: "System",
 }],
 dnsPrefix: "multicloudaks",
 identity: { type: "SystemAssigned" },
});

const aksCreds = containerservice.listManagedClusterUserCredentialsOutput({
 resourceGroupName: resourceGroup.name,
 resourceName: aksCluster.name,
});

export const aksKubeconfig = aksCreds.kubeconfigs[0].value.apply(
 v => Buffer.from(v, "base64").toString());

Provision a GKE cluster with the gcp provider:

import * as pulumi from "@pulumi/pulumi";
import * as gcp from "@pulumi/gcp";

const gkeCluster = new gcp.container.Cluster("gke-cluster", {
 initialNodeCount: 2,
 nodeConfig: {
 machineType: "e2-medium",
 oauthScopes: [
 "https://www.googleapis.com/auth/cloud-platform",
 ],
 },
});

export const gkeKubeconfig = pulumi
 .all([gkeCluster.name, gkeCluster.endpoint, gkeCluster.masterAuth])
 .apply(([name, endpoint, auth]) => `apiVersion: v1
kind: Config
clusters:
- name: ${name}
 cluster:
 certificate-authority-data: ${auth.clusterCaCertificate}
 server: https://${endpoint}
contexts:
- name: ${name}
 context:
 cluster: ${name}
 user: ${name}
current-context: ${name}
users:
- name: ${name}
 user:
 exec:
 apiVersion: client.authentication.k8s.io/v1beta1
 command: gke-gcloud-auth-plugin
 installHint: Install gke-gcloud-auth-plugin to authenticate to GKE
 provideClusterInfo: true
`);

The same EKS cluster in Python looks like this, using pulumi_eks:

import pulumi
import pulumi_eks as eks

cluster = eks.Cluster("eks-cluster",
 instance_type="t3.medium",
 desired_capacity=2,
 min_size=1,
 max_size=3)

pulumi.export("kubeconfig", cluster.kubeconfig)

Each block above is a normal Pulumi resource, so it gets the same pulumi preview, unit tests, and code review as any other change in the stack, rather than living in a separate cluster-provisioning tool.

How do you deploy the same application to multiple Kubernetes clusters?

Once the clusters exist, the kubernetes provider deploys workloads to any of them using the same resource definitions, by pointing a separate provider instance at each cluster’s kubeconfig:

import * as pulumi from "@pulumi/pulumi";
import * as k8s from "@pulumi/kubernetes";

interface ClusterTarget {
 name: string;
 kubeconfig: pulumi.Output<string>;
}

const clusters: ClusterTarget[] = [
 { name: "eks", kubeconfig: eksCluster.kubeconfig },
 { name: "aks", kubeconfig: aksKubeconfig },
 { name: "gke", kubeconfig: gkeKubeconfig },
];

for (const cluster of clusters) {
 const provider = new k8s.Provider(`${cluster.name}-provider`, {
 kubeconfig: cluster.kubeconfig,
 });

 new k8s.apps.v1.Deployment(`${cluster.name}-kuard`, {
 spec: {
 replicas: 1,
 selector: { matchLabels: { app: "kuard" } },
 template: {
 metadata: { labels: { app: "kuard" } },
 spec: {
 containers: [{
 name: "kuard",
 image: "gcr.io/kuar-demo/kuard-amd64:blue",
 ports: [{ containerPort: 8080 }],
 }],
 },
 },
 },
 }, { provider });
}

Because the loop body is ordinary code, adding a fourth provider, or a local cluster reachable through a kubeconfig file (Docker Desktop, kind, or a self-managed cluster), means adding one more entry to the clusters array rather than writing a new deployment pipeline. The full, runnable version of this example, including cluster provisioning for all three providers, is in the kubernetes-ts-multicloud example.

Which tools manage multi-cloud Kubernetes fleets?

Provisioning the clusters is one job; keeping workloads scheduled and consistent across them once they exist is another. These are complementary layers, not competing choices:

Layer	What it does	Example tools	Best for
Fleet and control-plane federation	Coordinates workload placement and policy across clusters that already exist, from one control plane	Karmada, Google Anthos, Rancher, Gardener	Teams that already run multiple clusters and want centralized scheduling and policy enforcement
Kubernetes-native infrastructure as code	Manages cloud infrastructure through Kubernetes custom resources and controllers	Crossplane	Teams standardized on GitOps and the Kubernetes API as the control plane for infrastructure
General-purpose infrastructure as code	Provisions the clusters themselves, and every supporting resource around them (networking, IAM, node pools), in the same codebase and language as the rest of your infrastructure	Pulumi	Teams that want the clusters, the workloads, and the surrounding cloud resources under one review and testing process

A fleet tool like Karmada doesn’t create the EKS, AKS, and GKE clusters it schedules onto; something has to provision those first. Pulumi is a good fit for that provisioning step, and it can sit alongside a fleet-management tool rather than replacing it.

Where to go next

This post covers provisioning and deploying across EKS, AKS, and GKE directly. If your goal is closer to sharing a golden-path Kubernetes setup across teams through a reusable component and the Automation API, see Multicloud with Kubernetes and Pulumi, which walks through building a customizable, self-service Kubernetes provisioning experience for other teams to consume.

To get started with Kubernetes on Pulumi:

Is multi-cloud Kubernetes worth the added complexity?

It depends on why you’re considering it. If the driver is a concrete requirement, such as a customer contract that specifies a provider, a compliance rule tied to a region only one cloud serves, or a team that already standardized on a different cloud after an acquisition, the coordination cost is worth paying. If the driver is “just in case we need to switch providers someday,” the ongoing cost of testing and operating three cluster types usually outweighs a lock-in risk that may never materialize. Start with the workloads that actually need to run on more than one cloud, rather than replicating everything by default.

Can I use the same Kubernetes manifests across EKS, AKS, and GKE?

Mostly. Core Kubernetes resources, such as Deployments, Services of type ClusterIP, and ConfigMaps, behave the same way regardless of provider, because they’re implemented by Kubernetes itself rather than the cloud. Anything that touches provider-specific infrastructure differs: a Service of type LoadBalancer provisions an AWS load balancer (a Network Load Balancer with the AWS Load Balancer Controller or EKS Auto Mode, a Classic Load Balancer otherwise), an Azure Load Balancer, or a Google Cloud Load Balancer depending on the cluster, each with its own annotations for things like internal-only access or SSL termination. Persistent volume storage classes also differ by default (gp3 on EKS, managed-csi on AKS, standard-rwo on GKE). Plan for a small provider-specific configuration layer around an otherwise-shared manifest, rather than expecting one manifest to be entirely provider-agnostic.

Do I still need a tool like Karmada if I’m provisioning clusters with Pulumi?

Only if you need active workload scheduling and policy enforcement across clusters that are already running. Pulumi provisions the clusters and can deploy the same workload definitions to each one, which covers most teams running a handful of clusters with a known, mostly static placement. A fleet-management tool like Karmada earns its keep once you’re actively rebalancing workloads between clusters based on capacity or failures, or enforcing cluster-wide policy from a single control plane across a larger fleet. The two aren’t mutually exclusive: Pulumi can provision the clusters that Karmada then manages.