Pulumi Blog: Cloud deployment

Your Perfect Infrastructure May Not Be So Perfect

Simen A. W. Olsen — Mon, 16 Dec 2024 10:43:07 +0000

Guest Article: Simen A. W. Olsen from Bjerk, is here to share his lessons learned on why designing the perfect architecture for your future needs might be a mistake

I remember standing in front of our engineering team in 2018, proudly presenting what I believed was the future-proof architectural design for our new distributed system. The diagrams were immaculate, the technology choices were cutting-edge, and the scalability patterns were ready for any possible future scenario.

I was basically the Leonardo da Vinci of system design… if Leonardo had been really into Kubernetes and had a concerning addiction to coffee. But six months later, that “future-proof” architecture had become a constraint rather than an enabler, and my masterpiece was looking more like a finger painting done by a caffeinated raccoon.

This experience taught me something crucial: trying to build the perfect system that anticipates every future need is often worse than creating a system designed to change quickly. It’s like trying to predict what your kid will want to be when they grow up and pre-buying all the necessary equipment. Congrats, you now own a space suit, a stethoscope, and a dragon costume — and they decided to become a software engineer anyway.

The Over-Planning

Many teams fall into a common trap: they try to design systems that anticipate every possible future requirement. This happens even in agile teams, where we convince ourselves we need to “get the architecture right” before we can start iterating. You know, because nothing says “agile” like spending three months in a room drawing boxes and arrows while muttering “microservices” under your breath like it’s a magic spell.

In 2008, Netflix faced a choice: build the perfect data center that could handle all their anticipated future needs, or move to the cloud with a simpler architecture that could evolve. They chose the latter, focusing on making their system easy to change rather than trying to make it perfect. Smart move — unlike those my past self made who probably would’ve insisted on building a data center capable of streaming to Mars, just in case Elon asked nicely.

You might also like:

The Core Principles of Change-Ready Architecture

Through both failures and successes, I’ve identified three principles that define truly adaptable architecture:

Embrace simplicity. I think it makes sense to start with the simplest architecture that could possibly work for your current needs. Complexity should be earned, not presumed. If your architecture diagram looks like a plate of spaghetti that’s been hit by lightning, you might be doing it wrong.
Make change cheap. Instead of trying to avoid change, make it inexpensive. This means investing in automated testing, continuous deployment, and monitoring. When change is cheap, you don’t need to fear it.
Learn through action. Rather than trying to predict the future, build mechanisms that help you learn quickly about real needs. It includes feature toggles (Protip: Try Unleash.), A/B testing, and robust monitoring of how your system is actually being used. You know, actual data, not just what that one loud guy in planning insists will definitely happen.

The rise of AI and machine learning systems has made one thing clear: we can’t predict how our systems will need to evolve. The most successful teams aren’t those that try to build the perfect AI architecture upfront, but those that can rapidly experiment and adapt their systems based on real-world feedback.

The biggest pushback I hear is, “But what if we need to scale?” or “What about future requirements?” These fears often drive teams to over-architect their solutions. But here’s the reality: the cost of changing a simple system is usually lower than the cost of maintaining an over-engineered one. The key is understanding that good architecture isn’t about predicting the future — it’s about making future changes as painless as possible.

Conclusion

That over-engineered system I was so proud of in 2018? Its most significant flaw wasn’t in what it got wrong about the future — it was that it tried too hard to be right about the future in the first place. It’s like bringing a fully packed suitcase to a first date. Today, I know that the best architecture isn’t one that anticipates every need, but one that makes it easy to respond to needs as they emerge.

The next time you’re tempted to design for every possible future scenario, remember: the goal isn’t to build a perfect system, but to build one that’s perfectly easy to change. And if someone tells you they’ve designed the perfect future-proof architecture, they’re either lying, or they’ve discovered time travel — and in that case, they should be sharing lottery numbers, not system designs.

You might also like:

Replicating Data to Support Multi-Region Applications

Adora Nwodo — Wed, 27 Dec 2023 11:03:10 +0000

In the previous article, we covered multi-region scaling, its importance, and how you can use Pulumi stacks to represent multiple regions and environments. The big takeaway from that article is that scaling your application across multiple regions is an important architectural decision to enable scalability and availability, but it comes with its own set of considerations. One of these considerations is around the data that your application needs or uses. Data replication is typically necessary in multi-region architectures because if you have multiple application instances worldwide making calls to just one database instance, the latency of your application will be high. After all, it will take a long time for that database instance to perform data operations and send back a result to the user through the application. Some requests might have repeated timeouts. In this article, we will cover data replication in multi-region applications and how it plays a pivotal role in distributed systems.

Why is data replication important?

In almost every instance of using multi-region scaling with an application, data replication becomes imperative for a few different reasons:

Latency reduction: Replicating data closer to end users minimizes data access latency, providing a seamless and responsive user experience.
Availability: When data is distributed across different locations, applications may be able to remain available even when there are regional outages or disruptions. There are some architectural/design decisions around this that we’ll discuss in the next section, but generally speaking a globally distributed approach creates redundancy. This means that if one region experiences an outage, application users can still access data from alternative locations.
Load balancing: Data replication enables effective load balancing. To minimize overload on a single server or region, traffic can be distributed among numerous replicas, optimizing resource utilization and boosting overall system performance.

Data Replication and the CAP Theorem

Any discussion of data replication among multiple regions invariably needs to address the CAP theorem. In a nutshell, the CAP theorem (also known as Brewer’s theorem) states that a distributed data store can only provide two of three guarantees: consistency, availability, and partition tolerance. Generally, these three guarantees are defined in this way:

Consistency – all clients always have the same view of the data: Choosing strong consistency assures that all clients, regardless of their geographical location in the distributed system, see the same data. However, the pursuit of uniformity frequently results in increased latency, particularly when dealing with low bandwidth and/or high latency networks.
Availability – all clients can always read and write the data: Prioritizing high availability means that the system responds to client read and write requests even in the event of network partitions. Accepting eventual consistency and accepting some transient divergence among replicas may be required to achieve availability.
Partition Tolerance – the system will continue to work despite partitions: Partition tolerance is non-negotiable in cloud computing, when systems span numerous regions and components are distributed. It ensures that the system continues to function even in the face of physical partitions or network outages, but doing so will require strategic compromises in terms of consistency or availability.

To reiterate, application and infrastructure architects have no choice but to tolerate network partitioning, as networks outside their control or influence—either the cloud provider’s network or the internet or both—typically fall in the replication path. Therefore, application and infrastructure architects have to choose the behavior of the data store when (not if) network partitioning occurs:

Does the data store cancel the operation, preserving consistency but affecting availability?
Does the data store proceed with the operation, prioritizing availability at the cost of consistency?

As mentioned earlier, though, more than just network partitioning comes into play here. Network bandwidth affects the balance between preserving consistency or optimizing for availability in the context of multi-region applications. Network configurations also come into play; what impact will a larger MTU (Maximum Transmission Unit) have on data replication? Decisions here are full of trade-offs. For example, prioritizing replication speed to ensure data consistency across regions may lead to increased latency, affecting the system’s ability to meet stringent availability requirements. An extension to the CAP theorem, known as PACELC, captures the trade-off between latency and consistency even when network partitions aren’t present.

Therefore, the challenge lies in finding the right balance. Striking a compromise between data consistency and system performance (in the form of availability) is a delicate task requiring a careful analysis of the principles of the CAP theorem, guiding system architects in making informed choices that align with the specific needs and priorities of their distributed applications.

Using Pulumi to configure data replication

Now that you’ve seen the impact of CAP theorem-related decisions on distributed data stores and the applications that use them, let’s take a look at a specific example. Let’s say you are building a distributed system and you want to have a central database with multiple replicas across the different availability zones (or regions) where your application will be deployed.

Cloud providers such as Azure, AWS, and Google Cloud all have databases that offer multi-region capabilities (AWS has DynamoDB, Azure has Cosmos DB, and Google Cloud has Cloud Spanner). Thanks to Pulumi’s integration with a variety of cloud providers, including (but not limited to) Azure, AWS, and Google Cloud, you can use Pulumi to deploy instances of these multi-region databases in your preferred programming language. This allows you to take advantage of these offerings and still reap the benefits of using infrastructure as code with Pulumi.

Let’s look at an example of using Pulumi to define an Azure Cosmos DB account with multi-region replication:

import * as pulumi from "@pulumi/pulumi";
import * as azure from "@pulumi/azure-native";

const resourceGroup = new azure.resources.ResourceGroup("myResourceGroup");

const cosmosDbAccount = new azure.documentdb.DatabaseAccount("myCosmosDbAccount", {
 resourceGroupName: resourceGroup.name,
 location: resourceGroup.location,
 locations: [
 { locationName: "East US" },
 { locationName: "West US" },
 // Add additional Azure regions for replication
 ],
 databaseAccountOfferType: "Standard",
 consistencyPolicy: {
 defaultConsistencyLevel: "Session",
 },
});

In this example, the locations parameter is used to specify the Azure regions where Cosmos DB replicas will be deployed, enabling global data distribution. Aligning these locations with the locations where your application code is deployed allows you to minimize latency due to reading from the database.

Additionally, you can also enable multi-region writes. This is typically useful for write-active scenarios, ensuring that write operations can be directed to any region with a writable replica, and can reduce latency associated with writes to the database. This reduction in write latency often translates directly into improved performance for a multi-region application.

const cosmosDbAccount = new azure.documentdb.DatabaseAccount("myCosmosDbAccount", {
 resourceGroupName: resourceGroup.name,
 … // more code
 enableMultipleWriteLocations: true,
});

Consistency

You will need to choose a consistency level that aligns with your application requirements, such as “Session,” “Bounded staleness,” “Eventual,” “Strong,” or “Consistent prefix.” This page has more details on the different consistency levels, but let’s look at a couple specific configurations.

Recall that strong consistency ensures that all nodes or replicas in the system view the same data simultaneously, regardless of which node they are accessing. In other words, after performing a write operation, any subsequent read action from any node must return the most recent write value. This consistency mode guarantees the linear ordering of processes, and the system behaves like a single, coherent entity. Strong consistency prioritizes consistency over availability in the event of network partition. It also hinders performance as the write must be acknowledged by multiple nodes.

You can prioritize strong consistency by configuring the Azure Cosmos DB account to use the “Strong” consistency level. This ensures that all read and write operations observe the most recent version of the data across all replicas.

const cosmosDbAccount = new azure.documentdb.DatabaseAccount("myCosmosDbAccount", {
 resourceGroupName: resourceGroup.name,
 consistencyPolicy: {
 defaultConsistencyLevel: "Strong",
 },
});

Cosmos DB also supports tunable consistency. Tunable consistency means that the consistency can be changed for each read and write request. Tunable consistency levels can be used in instances when strict strong consistency is not required. When configured this way, the application now assumes the responsibility for determining the consistency.

This next code snippet shows how to configure the Cosmos DB account with a “Bounded Staleness” consistency level, which allows you to balance consistency and latency based on application requirements.

const cosmosDBAccount = new azure.documentdb.DatabaseAccount("myCosmosDB", {
 resourceGroupName: resourceGroup.name,
 consistencyPolicy: {
 // Leverage tunable consistency levels for specific scenarios
 defaultConsistencyLevel: "BoundedStaleness",
 maxStalenessPrefix: 100,
 },
});

Now that the responsibility for assigning consistency is with the client, the code snippet below shows how to make API calls with tunable consistency for both reads and writes using Azure Cosmos DB SDK:

import { CosmosClient } from "@azure/cosmos";

const endpointUri = "Your_Cosmos_DB_Endpoint_URI";
const primaryKey = "Your_Cosmos_DB_Primary_Key";
const databaseId = "Your_Database_Id";
const containerId = "Your_Container_Id";

const cosmosClient = new CosmosClient({ endpoint: endpointUri, key: primaryKey });
const database = cosmosClient.database(databaseId);
const container = database.container(containerId);

async function writeDocumentWithTunableConsistency() {
 const document = { id: "1", PartitionKey: "MyPartitionKey", Message: "Hello, Cosmos DB!" };

 const requestOptions = {
 consistencyLevel: "BoundedStaleness",
 maxStalenessPrefix: 100,
 };

 await container.items.create(document, requestOptions);

 console.log("Document written with tunable consistency.");
}

async function readDocumentWithTunableConsistency() {
 const documentId = "Your_Document_Id";

 const requestOptions = {
 consistencyLevel: "Eventual",
 };

 const { resource } = await container.item(documentId, "Your_Partition_Key").read(requestOptions);

 console.log(`Document read with tunable consistency. Message: ${resource.Message}`);
}

(async () => {
 await writeDocumentWithTunableConsistency();
 await readDocumentWithTunableConsistency();
})();

Availability

In the context of the CAP and PACELC theorems, availability refers to the guarantee that every request will receive a non-error response—but without the guarantee that it contains the most recent write. As we’ve discussed throughout this article, when dealing with distributed data stores this means navigating the trade-offs between consistency and availability. If you opt for solid consistency then you ensure uniform data views but you reduce availability during partitions. If you opt to prioritize availability, then you have no choice but to deal with consistency issues.

This tradeoff becomes especially relevant when uninterrupted system availability is necessary, such as high-traffic applications or systems with stringent service level agreements (SLAs).

It’s also important to understand that availability is not the same as fault tolerance, especially in a CAP theorem context. Going back to our example of using multiple Cosmos DB replicas to support a multi-region application, Cosmos DB (as shown earlier) absolutely supports placing replicas in different availability zones or different regions. This enables cross-region replication, ensuring that the data exists in multiple locations. In the event one of these locations fails or becomes unreachable or inoperable, the application can tolerate this fault (meaning the application isn’t necessarily completely down and there is little or no data loss) but availability may still be impacted. For example, choosing to prioritize consistency over availability means that when this fault occurs the database may block, restrict, or delay operations until the fault is resolved.

So how does one tune availability with a distributed data store? Consistency and availability are two sides of the same coin; adjusting consistency affects availability. With Cosmos DB, you tune availability by adjusting the consistency settings on the database instance. If you opt for strict consistency, then availability is impacted; if you opt for a less strict consistency level, then availability is not impacted as much (or not at all).

Here’s how to configure CosmosDB to prioritize availability:

const cosmosDbAccount = new azure.documentdb.DatabaseAccount("myCosmosDbAccount", {
 resourceGroupName: resourceGroup.name,
 location: resourceGroup.location,
 locations: [
 { locationName: "East US" },
 { locationName: "West US" },
 // Add additional Azure regions for replication
 ],
 consistencyPolicy: {
 defaultConsistencyLevel: "Eventual",
 },
 enableAutomaticFailover: true,
});

When prioritizing availability, you might choose to use a default consistency level of “Eventual,” as shown in the example above. Eventual consistency prioritizes high availability by enabling updates to propagate gradually across replicas. While it has decreased read latency, it accepts temporary inconsistencies that resolve over time. As you might expect, your application will need to be prepared to deal with these temporary inconsistencies.

A more balanced option is called “BoundedStaleness.” This choice introduces staleness in read operations, allowing for better availability during network partitions. “BoundedStaleness” provides a controlled staleness window (maxStalenessPrefix), enabling developers to fine-tune the balance between consistency and availability. Bounded Staleness offers a sort of middle ground, allowing users to define a controlled maximum lag for read operations and therefore ensuring a specified level of consistency while maintaining availability during network partitions. This model suits scenarios where a compromise between consistency and availability is necessary, and the application can tolerate a defined level of data staleness.

Your application’s requirements influence the decision between these options.

By also enabling enableAutomaticFailover, the code ensures that Cosmos DB automatically performs failover in the event of a regional or node failure. Automatic failover improves availability by redirecting traffic to healthy instances and minimizing downtime during unexpected disruptions.

Partition tolerance

Partition tolerance isn’t a tunable/configurable parameter. Application and infrastructure architects need to accept that network partitions will occur, and instead need to configure the data store to behave in an expected fashion (either prioritizing consistency at the expense of availability, or prioritizing availability at the cost of consistency).

The CAP theorem posits that, during network partitions, a system must make a trade-off between consistency and availability. In Cosmos DB, tweaking the consistency level influences data presentation to clients. Opting for solid consistency ensures uniform data views but may reduce availability during partitions. Conversely, prioritizing availability over consistency allows for continuous operation, even with potential data discrepancies. High consistency levels may result in increased latency and reduced availability during partitions, while prioritizing availability ensures responsiveness, even in partitioned scenarios.

Understanding these trade-offs is crucial for designing resilient systems. It allows for informed decisions about the system’s behavior during network partitions, ultimately contributing to effectively managing partition tolerance in distributed systems and databases.

Conclusion

Data replication is important for ensuring the resilience and performance of stateful distributed services. This article not only emphasized the significance of data replication in distributed systems, but it also demonstrated how Pulumi, in collaboration with Azure, enables developers to architect and manage resilient, performant, and globally distributed infrastructures for their stateful applications. As cloud-native architectures expand, the importance of infrastructure as code (IaC) in ensuring optimal data replication outcomes becomes increasingly important, and Pulumi can be used to provision the infrastructure required.

Did you learn something new? We’d like to know. If you’d like to try this out for yourself, you can get started with Pulumi Cloud here. You can also join our community to be a part of the continuous conversation, and we can’t wait to see what you build!

Scaling Applications Across Multiple Regions

Adora Nwodo — Wed, 06 Dec 2023 20:24:40 +0000

As a team building a distributed cloud service that will be used by different people around the world, you will need strategies to cater to users globally, ensuring uninterrupted service even in the face of disruptions. Have you ever wondered how businesses operate and deliver high-level performance across different regions? Well, that is the result of scaling applications across multiple regions — in a bid to ensure constant access and flexibility, organizations distribute their applications and databases across various geographic regions so that if one part faces issues, the service remains available elsewhere.

In this article, we will explore the significance of multi-region scaling and show using Pulumi for multi-region deployment.

Understanding multi-region scaling

When distributed systems adopt (or implement) multi-region scaling, they position themselves for high availability because the redundancy minimizes downtime and ensures that the service stays up and running. If one region experiences issues, such as server downtime or network problems, the application remains accessible to users in other regions. This also leads to better performance in their systems because a fast and responsive experience for users across the globe is achieved.

It is easier to reach a global audience when you have scaled geographically. This means that if you have customers in Europe and the US and you’re a multi-regional cloud service, by distributing your traffic across different instances, you can improve your customers’ experience by serving users worldwide and also catering to diverse regions and time zones. Specific data protection and privacy regulations are also met, as multi-region scaling allows systems to adhere to these regulations by keeping data within defined boundaries.

Although multi-region deployments are needed for scaling, availability, and performance, there could be downsides associated with them if the design process for being multi-region isn’t well thought out. Some of the challenges that could be encountered are:

Complexity: Each region may have unique requirements, and ensuring consistency in configuration can be challenging. Troubleshooting across regions can add complexity to the infrastructure setup.
Cost: Expanding to multiple regions often increases infrastructure costs. Businesses need to balance the cost of maintaining multiple resources with the benefits of high availability and performance.
Deployment consistency: Inconsistencies in resource configuration can lead to performance issues and errors, impacting the user experience.
Security compliance: Meeting security and compliance standards in different regions may require additional effort. Each jurisdiction may have unique data protection and privacy regulations, and this can be a challenge.

The Pulumi solution

A stack is a distinct deployment target within a Pulumi project. Stacks enable you to organize, version, and manage your infrastructure code effectively. They are flexible enough to do more than merely plan deployments across different stages of development. You may better control infrastructure orchestration with them as an adaptable tool for managing deployments in a variety of circumstances. Each stack represents a separate instance of your infrastructure, allowing you to deploy and manage your resources in different environments or configurations. To learn more about stacks, you can read this.

Consider how Pulumi Stacks can be used effectively in the following ways:

Deploying to different environments: You can use stacks to deploy into different environments, like development, staging, and production. Stacks allow separate configurations to be maintained throughout these contexts.
Deploying to different regions: You can use stacks to effortlessly manage deployments in multiple regions (e.g. westus2, westeurope, northeurope etc.). This enables you to scale your infrastructure across geographical locations while maintaining a centralized and consistent approach to resource allocation. These deployments could be in the same environment, but different regions (e.g. production westeurope, and production westus2), and you can use stacks to handle the different permutations.
Deploying to different cloud provider accounts: Stacks can also be used to automate deployments across several cloud provider accounts. Even when working with different cloud platforms, your deployment process can be unified and consistent.

Each of these stacks have their own configuration, but there are also some configuration values that are shared across stacks. To obey the DRY (Don’t Repeat Yourself) programming rule, you can pair stacks with Pulumi ESC (Environments, Secrets, and Configuration). This pairing results in a flexible deployment strategy. The combination empowers users to segregate configurations for each environment, manage secrets securely, and maintain a centralized store for configuration data. This ensures a clean separation of concerns, contributing to improved security and maintainability in multi-environment deployments. Pulumi ESC is a pivotal aspect of managing stacks, especially as the number of stacks grows. The idea is to store configuration settings separately from your code, ensuring that each stack has configuration values tailored to its specific needs.

In the next section, we’ll show how to deploy applications across different regions using Pulumi stacks and Pulumi ESC.

Define regions and stacks

Let’s look at an example of deploying to three distinct regions — production westus2, production westeurope, and staging southeastasia — using stacks. In this scenario, we will use three stacks: prodwu2, prodwe, and stagingsea. Each of these stacks will have a stack configuration file associated with it, named Pulumi.<stack>.yaml. In this example, we are using stacks both to deploy into different environments (production and staging), but also to deploy into different regions (West US 2, West Europe, and Southeast Asia).

You can create your stacks by running:

pulumi stack init prodwu2
pulumi stack init prodwe
pulumi stack init stagingsea

After creating your stacks, you can update your Pulumi configuration YAML files with their corresponding values (typically done with the pulumi config set command).

Pulumi.prodwu2.yaml:

config:
 multi-region:appServiceTier: PremiumV2
 multi-region:appServiceSize: P1V2
 multi-region:environment: production
 multi-region:location: westus2

Pulumi.prodwe.yaml:

config:
 multi-region:appServiceTier: PremiumV3
 multi-region:appServiceSize: P1V3
 multi-region:environment: production
 multi-region:location: westeurope

Pulumi.stagingsea.yaml:

config:
 multi-region:appServiceTier: Standard
 multi-region:appServiceSize: S1
 multi-region:environment: staging
 multi-region:location: southeastasia

Leveraging Pulumi ESC

For configuration values that are shared across stacks, Pulumi ESC allows you to create them once and reference them in multiple stacks. In the referenced codebase, there are other values e.g. resourceNamePrefix, storageKind, storageSkuName, and tenantId. Since these values are the same across all our stacks, we can leverage ESC so that we can declare them once and reuse them everywhere.

Pulumi ESC is a feature of Pulumi Cloud and in order to use ESC, you should be using Pulumi Cloud as your backend.

To leverage Pulumi ESC, you first need to initialize a new environment like this:

pulumi env init shared-multi-region

You can use the pulumi env command, as shown.

After which, you can set all the value for the configurations you need. Remember to add it as part of the pulumiConfig object, so that your Pulumi stack understands and can pick it up.

pulumi env set shared-multi-region pulumiConfig.resourceNamePrefix mrapp
pulumi env set shared-multi-region pulumiConfig.storageKind "Standard_LRS"
pulumi env set shared-multi-region pulumiConfig.storageSkuName "Standard_LRS"
pulumi env set shared-multi-region pulumiConfig.tenantId "00000000-0000-0000-0000-000000000000" # Your tenantId

After successfully adding your config values to your new environment, you can reference it across all your stacks like this:

config:
 multi-region:appServiceTier: Standard
 multi-region:appServiceSize: S1
 multi-region:environment: staging
 multi-region:location: southeastasia

environment:
 imports:
 - shared-multi-region

Here, shared-multi-region has more configurations and you can add references to them in your stack without copying and pasting multiple lines.

Update Pulumi code

Once you have set up the Pulumi ESC environment and linked it to your Pulumi stack configuration, you also need to update your Pulumi code to use the configuration values instead of any hard-coded values. This is shown in the code snippet below using the config.require method:

import * as pulumi from "@pulumi/pulumi";
import * as azure from "@pulumi/azure";

// Define the configuration
const stackName: string = pulumi.getStack();
const config = new pulumi.Config();

const appServiceTier = config.require("appServiceTier");
const appServiceSize = config.require("appServiceSize");
const resourceNamePrefix = config.require("resourceNamePrefix"); // gotten from ESC
const storageKind = config.require("storageKind"); // gotten from ESC

// ... more code

// Create an Azure Resource Group
const resourceGroup = new azure.resources.ResourceGroup(resourceGroupName, {
 resourceGroupName: resourceGroupName,
 location: location,
 tags
});

// Create an Azure Storage Account
const storageAccount = new azure.storage.StorageAccount(storageAccountName, {
 accountName: storageAccountName,
 resourceGroupName: resourceGroup.name,
 kind: storageKind,
 location: location,
 sku: {
 name: storageSkuName
 },
 tags
});

// ... more code

Deploy stacks

Your Pulumi stacks should be ready to deploy now—you’ve updated the code to use configuration values that either come from the stack or from Pulumi ESC, and you’ve done both the per-stack configurations and the Pulumi ESC configuration values. Now it’s just a matter of running pulumi up:

pulumi up --stack prodwu2
pulumi up --stack prodwe
pulumi up --stack stagingsea

Now, you have seen how to use stacks to represent regions and environments. With this approach, the dynamism happens through your configuration and your code doesn’t have to change per region.

To see a full application with more detail, click here.

Considerations for high availability

Once you’ve configured your multi-region deployments, you should consider strategies for high availability to ensure that applications remain operational and accessible even in the face of failures.

Placing workloads across multiple regions

Effectively distributing workloads across various geographic regions is a key strategy for maintaining high availability. You can employ one of the following methods to distribute traffic across multiple regions:

Active/Passive with hot standby: In this approach, one region actively handles traffic while the other remains on hot standby, meaning the services in the secondary region are always running.
Active/Passive with cold standby: Similar to hot standby, in this case, the services in the secondary region aren’t allocated until needed for failover. This minimizes costs but might increase recovery time.
Active/Active: Both regions are active, and requests are load-balanced between them. If one region becomes unavailable, it is taken out of rotation.

In the context of infrastructure as code using Pulumi, these strategies directly translate to the management of multiple stacks. In active/hot standby and active/active configurations, there will typically be two Pulumi stacks, each with its allocated set of resources. This allows for the independent management and deployment of resources in each region.

On the other hand, in an active/cold standby scenario, while there might be a Pulumi stack defined for the cold standby site, it won’t have any resources initially allocated. Part of the failover process would involve running a pulumi up specifically for the cold standby region, triggering the allocation of resources as needed. This dynamic resource allocation during failover helps optimize costs, as resources are provisioned only when necessary, contributing to a more efficient use of cloud resources.

Replicating data

Implementing appropriate data replication solutions is an important factor in enhancing system resilience. This involves creating duplicates of our databases and storing them in several locations to ensure that our data remains consistent and accessible. Such replication is especially important for sustaining smooth operations if one region has disruptions or malfunctions.

Exploring the details of data replication exposes the most difficult aspects of being multi-region with high availability. Finding the correct balance between Recovery Point Objective (RPO) and Recovery Time Objective (RTO) is an important factor. RPO refers to how frequently we store our data, but RTO refers to how soon we can get things back on track if there is a problem. Finding the correct balance entails considering how frequently we update our data and how long it takes to recover from unanticipated errors.

Services such as AWS Aurora Global Databases or Azure Cosmos DB can be used to simplify this complex operation. These services are intended to enable multi-region database replication as simple as possible, while also maintaining high availability across distributed systems.

Configuring load balancing, failover, and disaster recovery

Configuring load balancing, failover, and disaster recovery is needed to achieve high availability in a system. These strategies help distribute traffic efficiently, ensure continuous operation, and mitigate the impact of potential failures or disasters.

Load balancing: Choose a way to balance the work of your servers—it could be by using a cloud service like Azure Traffic Managers. Then, decide how you want to divide the job among your servers, like taking turns or picking the one with the least number of tasks to process. Make sure you set up health checks to keep an eye on how well your servers are doing. If you’re working in different parts of the world, use global balancing to send people to the closest or healthiest place for your data.
Failover: Think about things that might go wrong, like servers crashing or network problems. If something breaks, set up a plan to automatically switch to a healthy backup. Monitor your servers to catch anomalies quickly. Make sure you have failover infrastructure ready to take over if there is ever an outage. Also, make sure the tool that shares the work (the load balancer) doesn’t break itself. If something goes wrong, set up a way for your system to keep working, even if it’s not at full power. This is known as graceful degradation.
Graceful degradation: Graceful degradation is a concept that prepares systems for potential failures. When faced with unanticipated challenges such as server crashes or network outages, a system designed with graceful degradation adjusts its performance to maintain key functionality. Consider a web application that is significantly reliant on external APIs for real-time data. If one of these APIs experiences a temporary glitch, a system that embraces graceful degradation may seamlessly switch to presenting cached data or simplifying functionality, ensuring users maintain access to core functionalities even when service outages occur. This strategy is also applicable to cloud-based services, where a system intelligently scales down non-essential features under high loads in order to prioritize key activities, proving the robustness of systems designed using gentle degradation principles.
Disaster recovery: Be ready for the worst by copying your important data to different places. Have clear steps on how to bring back your data and programs if everything goes wrong. Spread your tools and services in different areas to reduce problems if one place has a big issue. Make sure your system can easily switch to another area if a big problem happens. Plan for disasters by making step-by-step guides on what to do. Test your disaster plans often to be sure they work well and to find any areas that need fixing.

Security considerations

It is one thing to multi-scale, it’s another responsibility to secure the infrastructure. Securing a multi-regional infrastructure introduces challenges. In multi-region setups, data privacy can be tricky due to different regulations, like GDPR in Europe, or The California Consumer Privacy Act (CCPA). Using differential privacy and data residency controls are important when handling data according to each region’s rules. Pulumi policy as code also enables the proactive design and enforcement of compliance-ready policies. Ensuring compliance in industries like healthcare and finance adds complexity, requiring a tailored approach based on specific regulations.

Expanding infrastructure in multi-regional setups makes them more vulnerable to cyber threats. To tackle this, it’s important to use good security practices like micro-segmentation, intrusion detection systems, and regular penetration testing. Being proactive in incident response planning, controlling access effectively, and employing strong encryption methods are also important for a secure cloud infrastructure.

Conclusion

As we navigate the complexities of multi-region deployments, Pulumi can help in simplifying infrastructure management. It is important to note that embracing the challenges of multi-region scaling is a strategic move for technology projects navigating the global landscape. With Pulumi stacks and ESC configurations, you can seamlessly manage deployments and configurations, ensuring adaptability and security.

Did you learn something new? We’d like to know. If you’d like to explore further or discuss your specific use case, you can schedule a consultation with a solutions engineer. You can also join our community to be a part of the continuous conversation, and we can’t wait to see what you build!