Posts Tagged aws

It’s fun to think about how much data there is swirling around in the global datasphere these days. However you choose to measure it (and there are various ways), it’s a quantity so massive — hundreds of zettabytes, by some estimates — that it’s kind of a hard thing to quite get your head around.

If you could convert all the world’s data into droplets of water, for instance, at one megabyte per drop, you’d have enough 1MB drops to fill two more Lake Washingtons. If you could store all that data on 3.5" floppies, you’d need more than a hundred quadrillion floppies to capture it all — enough to cover the planet entirely (with much room for overlap) or to pave a nice bridge for yourself from your front porch well into interstellar space. If you could pull all that data into an HD movie, and you sat down to start watching that movie 2.5 million years ago (with your favorite saber-toothed friend, say), you’d still be watching the same movie today.

Read more →

As AWS Lambda has matured as a serverless platform, there are two key ways the service has evolved:

1. New capabilities that extend the platform to support new use cases like Lambda Container support, Lambda URLs and attribute-based access control support.
2. Performance enhancements that enable Lambda functions to be more responsive and cost-effective such as Tiered compilation, and Graviton2 support are just a few examples of the investments AWS made in this space.

With each advancement, the reasons to deploy a full VM or cluster to support your application get fewer and the time to deliver value in the cloud gets shorter.

Read more →

We introduced Crosswalk for AWS three years ago as a library of components on top of the core AWS platform to make it easier to get from zero to production on AWS, easier to adopt AWS best practices by default, and easier to evolve your AWS infrastructure as your application needs mature. Since then, we’ve added many new capabilities, expanded the portfolio of libraries, and made these libraries available to all Pulumi languages. We’ve also seen thousands of Pulumi customers, including more than 25% of all Pulumi AWS users, adopting one or more of the Crosswalk for AWS components to aid in delivering their AWS-based applications and services.

Read more →

Self-service infrastructure is the holy grail of DevOps. When platform engineering teams can empower application teams to provision their own infrastructure without needing to understand the details of configuring networking, storage, and compute resources, IT organizations can drastically increase their ability to deliver on organizational goals. The first step in this process is to codify infrastructure best practices using platforms like Pulumi and the next step is to make these best practices available in a workflow that fits into the application team’s software development tools and process.

Read more →

Denis Willett is a software engineer at the North Carolina Institute of Climate Studies who works on the NOAA Open Data Dissemination Program. His work focuses on leveraging cloud technologies for the development of data processing and machine learning pipelines. Denis did his PhD in Entomology and Nematology at University of Florida and his undergraduate and masters work in Earth Systems at Stanford University. You can read his full bio here.

NOAA Open Data Dissemination (NODD) makes environmental data freely and publicly accessible across Amazon Web Services (AWS), Microsoft Azure (Azure), and Google Cloud Platform (GCP). These data include near real-time satellite imagery, weather models, radar feeds, drought information, ocean databases, and a suite of climate data records among many others. This program supports more than 220 datasets and over 24PB of open data. Since its inception, the program has been growing rapidly, almost doubling in size over the past year.

Read more →

Building on top of the cloud can be frustrating at times. You will likely run into resources that complicated to create, others that are very tedious to create, and worst of all resources that are complicated and tedious to create. As cloud engineers ourselves, we feel the same pain as our users and strive to build abstractions that make cloud engineering a more productive and pleasant experience.

Recently we released our Community AWS IAM Package to help deliver on the promise of making the cloud easier to use for every operator, engineer, and user. This package is based on the Terraform AWS IAM Module, so it allows our users to take advantage of battle-tested abstractions. The package also helps transitioning Terraform users by maintaining similar resource names and inputs so they can focus on taking advantage of features of their programming language of choice (TypeScript, JavaScript, Python, Go, .NET, and YAML).

Read more →

Creating a website on AWS with an S3 bucket is a fairly straightforward task. You just need to create an S3 bucket, configure it to be a website, and add your content, right?

Unfortunately getting your content into a bucket is only a part of the story. To get your website ready to handle traffic, you will want to associate a domain name and likely want to use a CDN, like Cloudfront, to help with performance. The AWS Static Website Package makes it easy for you to associate a domain and stand up a CDN with only a few arguments.

Read more →

In this blog post, we’re going to use some Angular framework components to assemble a static website and then use Pulumi and its AWS Static Website component to deploy it to AWS. The website is for a café called the Pulumi Café. It will contain two pages, one an About page and the other a Menu page, as well as some navigational pieces.

To follow this example, you need to have both Angular and Pulumi installed. (Here’s a link to the Pulumi installation instructions.) You’ll also need an AWS account.

Read more →

I love data. I mean, I really love data. Data gives you the ability to understand the world around you and, to a certain degree, project what the future could look like. At Pulumi we use data every day to help make smarter product and business decisions.

Though one hurdle we encountered was not only the sheer volume of data we have but also the large disparity of systems storing that data. Like many companies before us, we chose to build a data warehouse, specifically AWS Redshift, to store all of our data.

Read more →

Pulumi makes it easy to flexibly deploy your cloud infrastructure using code. Usually deployments encompass a single slack and a single region in your cloud of choice. If you need to go multi-region, that usually means creating a stack per-region, which Pulumi’s configuration system makes easy. A stack per region isn’t required, though! Sometimes we want a single stack to span regions for performance, scalability, resilience, or just hard requirements. In these cases, Pulumi can seamlessly orchestrate deployments to, or even across, multiple regions, accounts, or clusters. In this article, we’ll see this in action by provisioning an AWS RDS primary database into one region and a read replica in an entirely different region – all from a single Pulumi program, stack, and pulumi up incantation.

Read more →