Pulumi Blog: Application scalability

Your Perfect Infrastructure May Not Be So Perfect

Simen A. W. Olsen — Mon, 16 Dec 2024 10:43:07 +0000

Guest Article: Simen A. W. Olsen from Bjerk, is here to share his lessons learned on why designing the perfect architecture for your future needs might be a mistake

I remember standing in front of our engineering team in 2018, proudly presenting what I believed was the future-proof architectural design for our new distributed system. The diagrams were immaculate, the technology choices were cutting-edge, and the scalability patterns were ready for any possible future scenario.

I was basically the Leonardo da Vinci of system design… if Leonardo had been really into Kubernetes and had a concerning addiction to coffee. But six months later, that “future-proof” architecture had become a constraint rather than an enabler, and my masterpiece was looking more like a finger painting done by a caffeinated raccoon.

This experience taught me something crucial: trying to build the perfect system that anticipates every future need is often worse than creating a system designed to change quickly. It’s like trying to predict what your kid will want to be when they grow up and pre-buying all the necessary equipment. Congrats, you now own a space suit, a stethoscope, and a dragon costume — and they decided to become a software engineer anyway.

The Over-Planning

Many teams fall into a common trap: they try to design systems that anticipate every possible future requirement. This happens even in agile teams, where we convince ourselves we need to “get the architecture right” before we can start iterating. You know, because nothing says “agile” like spending three months in a room drawing boxes and arrows while muttering “microservices” under your breath like it’s a magic spell.

In 2008, Netflix faced a choice: build the perfect data center that could handle all their anticipated future needs, or move to the cloud with a simpler architecture that could evolve. They chose the latter, focusing on making their system easy to change rather than trying to make it perfect. Smart move — unlike those my past self made who probably would’ve insisted on building a data center capable of streaming to Mars, just in case Elon asked nicely.

You might also like:

The Core Principles of Change-Ready Architecture

Through both failures and successes, I’ve identified three principles that define truly adaptable architecture:

Embrace simplicity. I think it makes sense to start with the simplest architecture that could possibly work for your current needs. Complexity should be earned, not presumed. If your architecture diagram looks like a plate of spaghetti that’s been hit by lightning, you might be doing it wrong.
Make change cheap. Instead of trying to avoid change, make it inexpensive. This means investing in automated testing, continuous deployment, and monitoring. When change is cheap, you don’t need to fear it.
Learn through action. Rather than trying to predict the future, build mechanisms that help you learn quickly about real needs. It includes feature toggles (Protip: Try Unleash.), A/B testing, and robust monitoring of how your system is actually being used. You know, actual data, not just what that one loud guy in planning insists will definitely happen.

The rise of AI and machine learning systems has made one thing clear: we can’t predict how our systems will need to evolve. The most successful teams aren’t those that try to build the perfect AI architecture upfront, but those that can rapidly experiment and adapt their systems based on real-world feedback.

The biggest pushback I hear is, “But what if we need to scale?” or “What about future requirements?” These fears often drive teams to over-architect their solutions. But here’s the reality: the cost of changing a simple system is usually lower than the cost of maintaining an over-engineered one. The key is understanding that good architecture isn’t about predicting the future — it’s about making future changes as painless as possible.

Conclusion

That over-engineered system I was so proud of in 2018? Its most significant flaw wasn’t in what it got wrong about the future — it was that it tried too hard to be right about the future in the first place. It’s like bringing a fully packed suitcase to a first date. Today, I know that the best architecture isn’t one that anticipates every need, but one that makes it easy to respond to needs as they emerge.

The next time you’re tempted to design for every possible future scenario, remember: the goal isn’t to build a perfect system, but to build one that’s perfectly easy to change. And if someone tells you they’ve designed the perfect future-proof architecture, they’re either lying, or they’ve discovered time travel — and in that case, they should be sharing lottery numbers, not system designs.

You might also like:

Scaling Applications Across Multiple Regions

Adora Nwodo — Wed, 06 Dec 2023 20:24:40 +0000

As a team building a distributed cloud service that will be used by different people around the world, you will need strategies to cater to users globally, ensuring uninterrupted service even in the face of disruptions. Have you ever wondered how businesses operate and deliver high-level performance across different regions? Well, that is the result of scaling applications across multiple regions — in a bid to ensure constant access and flexibility, organizations distribute their applications and databases across various geographic regions so that if one part faces issues, the service remains available elsewhere.

In this article, we will explore the significance of multi-region scaling and show using Pulumi for multi-region deployment.

Understanding multi-region scaling

When distributed systems adopt (or implement) multi-region scaling, they position themselves for high availability because the redundancy minimizes downtime and ensures that the service stays up and running. If one region experiences issues, such as server downtime or network problems, the application remains accessible to users in other regions. This also leads to better performance in their systems because a fast and responsive experience for users across the globe is achieved.

It is easier to reach a global audience when you have scaled geographically. This means that if you have customers in Europe and the US and you’re a multi-regional cloud service, by distributing your traffic across different instances, you can improve your customers’ experience by serving users worldwide and also catering to diverse regions and time zones. Specific data protection and privacy regulations are also met, as multi-region scaling allows systems to adhere to these regulations by keeping data within defined boundaries.

Although multi-region deployments are needed for scaling, availability, and performance, there could be downsides associated with them if the design process for being multi-region isn’t well thought out. Some of the challenges that could be encountered are:

Complexity: Each region may have unique requirements, and ensuring consistency in configuration can be challenging. Troubleshooting across regions can add complexity to the infrastructure setup.
Cost: Expanding to multiple regions often increases infrastructure costs. Businesses need to balance the cost of maintaining multiple resources with the benefits of high availability and performance.
Deployment consistency: Inconsistencies in resource configuration can lead to performance issues and errors, impacting the user experience.
Security compliance: Meeting security and compliance standards in different regions may require additional effort. Each jurisdiction may have unique data protection and privacy regulations, and this can be a challenge.

The Pulumi solution

A stack is a distinct deployment target within a Pulumi project. Stacks enable you to organize, version, and manage your infrastructure code effectively. They are flexible enough to do more than merely plan deployments across different stages of development. You may better control infrastructure orchestration with them as an adaptable tool for managing deployments in a variety of circumstances. Each stack represents a separate instance of your infrastructure, allowing you to deploy and manage your resources in different environments or configurations. To learn more about stacks, you can read this.

Consider how Pulumi Stacks can be used effectively in the following ways:

Deploying to different environments: You can use stacks to deploy into different environments, like development, staging, and production. Stacks allow separate configurations to be maintained throughout these contexts.
Deploying to different regions: You can use stacks to effortlessly manage deployments in multiple regions (e.g. westus2, westeurope, northeurope etc.). This enables you to scale your infrastructure across geographical locations while maintaining a centralized and consistent approach to resource allocation. These deployments could be in the same environment, but different regions (e.g. production westeurope, and production westus2), and you can use stacks to handle the different permutations.
Deploying to different cloud provider accounts: Stacks can also be used to automate deployments across several cloud provider accounts. Even when working with different cloud platforms, your deployment process can be unified and consistent.

Each of these stacks have their own configuration, but there are also some configuration values that are shared across stacks. To obey the DRY (Don’t Repeat Yourself) programming rule, you can pair stacks with Pulumi ESC (Environments, Secrets, and Configuration). This pairing results in a flexible deployment strategy. The combination empowers users to segregate configurations for each environment, manage secrets securely, and maintain a centralized store for configuration data. This ensures a clean separation of concerns, contributing to improved security and maintainability in multi-environment deployments. Pulumi ESC is a pivotal aspect of managing stacks, especially as the number of stacks grows. The idea is to store configuration settings separately from your code, ensuring that each stack has configuration values tailored to its specific needs.

In the next section, we’ll show how to deploy applications across different regions using Pulumi stacks and Pulumi ESC.

Define regions and stacks

Let’s look at an example of deploying to three distinct regions — production westus2, production westeurope, and staging southeastasia — using stacks. In this scenario, we will use three stacks: prodwu2, prodwe, and stagingsea. Each of these stacks will have a stack configuration file associated with it, named Pulumi.<stack>.yaml. In this example, we are using stacks both to deploy into different environments (production and staging), but also to deploy into different regions (West US 2, West Europe, and Southeast Asia).

You can create your stacks by running:

pulumi stack init prodwu2
pulumi stack init prodwe
pulumi stack init stagingsea

After creating your stacks, you can update your Pulumi configuration YAML files with their corresponding values (typically done with the pulumi config set command).

Pulumi.prodwu2.yaml:

config:
 multi-region:appServiceTier: PremiumV2
 multi-region:appServiceSize: P1V2
 multi-region:environment: production
 multi-region:location: westus2

Pulumi.prodwe.yaml:

config:
 multi-region:appServiceTier: PremiumV3
 multi-region:appServiceSize: P1V3
 multi-region:environment: production
 multi-region:location: westeurope

Pulumi.stagingsea.yaml:

config:
 multi-region:appServiceTier: Standard
 multi-region:appServiceSize: S1
 multi-region:environment: staging
 multi-region:location: southeastasia

Leveraging Pulumi ESC

For configuration values that are shared across stacks, Pulumi ESC allows you to create them once and reference them in multiple stacks. In the referenced codebase, there are other values e.g. resourceNamePrefix, storageKind, storageSkuName, and tenantId. Since these values are the same across all our stacks, we can leverage ESC so that we can declare them once and reuse them everywhere.

Pulumi ESC is a feature of Pulumi Cloud and in order to use ESC, you should be using Pulumi Cloud as your backend.

To leverage Pulumi ESC, you first need to initialize a new environment like this:

pulumi env init shared-multi-region

You can use the pulumi env command, as shown.

After which, you can set all the value for the configurations you need. Remember to add it as part of the pulumiConfig object, so that your Pulumi stack understands and can pick it up.

pulumi env set shared-multi-region pulumiConfig.resourceNamePrefix mrapp
pulumi env set shared-multi-region pulumiConfig.storageKind "Standard_LRS"
pulumi env set shared-multi-region pulumiConfig.storageSkuName "Standard_LRS"
pulumi env set shared-multi-region pulumiConfig.tenantId "00000000-0000-0000-0000-000000000000" # Your tenantId

After successfully adding your config values to your new environment, you can reference it across all your stacks like this:

config:
 multi-region:appServiceTier: Standard
 multi-region:appServiceSize: S1
 multi-region:environment: staging
 multi-region:location: southeastasia

environment:
 imports:
 - shared-multi-region

Here, shared-multi-region has more configurations and you can add references to them in your stack without copying and pasting multiple lines.

Update Pulumi code

Once you have set up the Pulumi ESC environment and linked it to your Pulumi stack configuration, you also need to update your Pulumi code to use the configuration values instead of any hard-coded values. This is shown in the code snippet below using the config.require method:

import * as pulumi from "@pulumi/pulumi";
import * as azure from "@pulumi/azure";

// Define the configuration
const stackName: string = pulumi.getStack();
const config = new pulumi.Config();

const appServiceTier = config.require("appServiceTier");
const appServiceSize = config.require("appServiceSize");
const resourceNamePrefix = config.require("resourceNamePrefix"); // gotten from ESC
const storageKind = config.require("storageKind"); // gotten from ESC

// ... more code

// Create an Azure Resource Group
const resourceGroup = new azure.resources.ResourceGroup(resourceGroupName, {
 resourceGroupName: resourceGroupName,
 location: location,
 tags
});

// Create an Azure Storage Account
const storageAccount = new azure.storage.StorageAccount(storageAccountName, {
 accountName: storageAccountName,
 resourceGroupName: resourceGroup.name,
 kind: storageKind,
 location: location,
 sku: {
 name: storageSkuName
 },
 tags
});

// ... more code

Deploy stacks

Your Pulumi stacks should be ready to deploy now—you’ve updated the code to use configuration values that either come from the stack or from Pulumi ESC, and you’ve done both the per-stack configurations and the Pulumi ESC configuration values. Now it’s just a matter of running pulumi up:

pulumi up --stack prodwu2
pulumi up --stack prodwe
pulumi up --stack stagingsea

Now, you have seen how to use stacks to represent regions and environments. With this approach, the dynamism happens through your configuration and your code doesn’t have to change per region.

To see a full application with more detail, click here.

Considerations for high availability

Once you’ve configured your multi-region deployments, you should consider strategies for high availability to ensure that applications remain operational and accessible even in the face of failures.

Placing workloads across multiple regions

Effectively distributing workloads across various geographic regions is a key strategy for maintaining high availability. You can employ one of the following methods to distribute traffic across multiple regions:

Active/Passive with hot standby: In this approach, one region actively handles traffic while the other remains on hot standby, meaning the services in the secondary region are always running.
Active/Passive with cold standby: Similar to hot standby, in this case, the services in the secondary region aren’t allocated until needed for failover. This minimizes costs but might increase recovery time.
Active/Active: Both regions are active, and requests are load-balanced between them. If one region becomes unavailable, it is taken out of rotation.

In the context of infrastructure as code using Pulumi, these strategies directly translate to the management of multiple stacks. In active/hot standby and active/active configurations, there will typically be two Pulumi stacks, each with its allocated set of resources. This allows for the independent management and deployment of resources in each region.

On the other hand, in an active/cold standby scenario, while there might be a Pulumi stack defined for the cold standby site, it won’t have any resources initially allocated. Part of the failover process would involve running a pulumi up specifically for the cold standby region, triggering the allocation of resources as needed. This dynamic resource allocation during failover helps optimize costs, as resources are provisioned only when necessary, contributing to a more efficient use of cloud resources.

Replicating data

Implementing appropriate data replication solutions is an important factor in enhancing system resilience. This involves creating duplicates of our databases and storing them in several locations to ensure that our data remains consistent and accessible. Such replication is especially important for sustaining smooth operations if one region has disruptions or malfunctions.

Exploring the details of data replication exposes the most difficult aspects of being multi-region with high availability. Finding the correct balance between Recovery Point Objective (RPO) and Recovery Time Objective (RTO) is an important factor. RPO refers to how frequently we store our data, but RTO refers to how soon we can get things back on track if there is a problem. Finding the correct balance entails considering how frequently we update our data and how long it takes to recover from unanticipated errors.

Services such as AWS Aurora Global Databases or Azure Cosmos DB can be used to simplify this complex operation. These services are intended to enable multi-region database replication as simple as possible, while also maintaining high availability across distributed systems.

Configuring load balancing, failover, and disaster recovery

Configuring load balancing, failover, and disaster recovery is needed to achieve high availability in a system. These strategies help distribute traffic efficiently, ensure continuous operation, and mitigate the impact of potential failures or disasters.

Load balancing: Choose a way to balance the work of your servers—it could be by using a cloud service like Azure Traffic Managers. Then, decide how you want to divide the job among your servers, like taking turns or picking the one with the least number of tasks to process. Make sure you set up health checks to keep an eye on how well your servers are doing. If you’re working in different parts of the world, use global balancing to send people to the closest or healthiest place for your data.
Failover: Think about things that might go wrong, like servers crashing or network problems. If something breaks, set up a plan to automatically switch to a healthy backup. Monitor your servers to catch anomalies quickly. Make sure you have failover infrastructure ready to take over if there is ever an outage. Also, make sure the tool that shares the work (the load balancer) doesn’t break itself. If something goes wrong, set up a way for your system to keep working, even if it’s not at full power. This is known as graceful degradation.
Graceful degradation: Graceful degradation is a concept that prepares systems for potential failures. When faced with unanticipated challenges such as server crashes or network outages, a system designed with graceful degradation adjusts its performance to maintain key functionality. Consider a web application that is significantly reliant on external APIs for real-time data. If one of these APIs experiences a temporary glitch, a system that embraces graceful degradation may seamlessly switch to presenting cached data or simplifying functionality, ensuring users maintain access to core functionalities even when service outages occur. This strategy is also applicable to cloud-based services, where a system intelligently scales down non-essential features under high loads in order to prioritize key activities, proving the robustness of systems designed using gentle degradation principles.
Disaster recovery: Be ready for the worst by copying your important data to different places. Have clear steps on how to bring back your data and programs if everything goes wrong. Spread your tools and services in different areas to reduce problems if one place has a big issue. Make sure your system can easily switch to another area if a big problem happens. Plan for disasters by making step-by-step guides on what to do. Test your disaster plans often to be sure they work well and to find any areas that need fixing.

Security considerations

It is one thing to multi-scale, it’s another responsibility to secure the infrastructure. Securing a multi-regional infrastructure introduces challenges. In multi-region setups, data privacy can be tricky due to different regulations, like GDPR in Europe, or The California Consumer Privacy Act (CCPA). Using differential privacy and data residency controls are important when handling data according to each region’s rules. Pulumi policy as code also enables the proactive design and enforcement of compliance-ready policies. Ensuring compliance in industries like healthcare and finance adds complexity, requiring a tailored approach based on specific regulations.

Expanding infrastructure in multi-regional setups makes them more vulnerable to cyber threats. To tackle this, it’s important to use good security practices like micro-segmentation, intrusion detection systems, and regular penetration testing. Being proactive in incident response planning, controlling access effectively, and employing strong encryption methods are also important for a secure cloud infrastructure.

Conclusion

As we navigate the complexities of multi-region deployments, Pulumi can help in simplifying infrastructure management. It is important to note that embracing the challenges of multi-region scaling is a strategic move for technology projects navigating the global landscape. With Pulumi stacks and ESC configurations, you can seamlessly manage deployments and configurations, ensuring adaptability and security.

Did you learn something new? We’d like to know. If you’d like to explore further or discuss your specific use case, you can schedule a consultation with a solutions engineer. You can also join our community to be a part of the continuous conversation, and we can’t wait to see what you build!