Posts Tagged python

Deploy AI Models on Amazon SageMaker using Pulumi Python IaC

Deploy AI Models on Amazon SageMaker using Pulumi Python IaC

Running models from Hugging Face on Amazon SageMaker is a popular deployment option for AI/ML services. While the SageMaker console allows for provisioning these cloud resources, this deployment pattern is labor intensive to document and vulnerable to human errors when reproducing as a regular operations practice. Infrastructure as Code (IaC) offers a reliable and easy to duplicate deployment practice. By developing this IaC with Pulumi, practitioners can choose to write their infrastructure code in Python and seamlessly develop both AI application code and IaC code in the same language.

Read more →

The Real AI Challenge is Cloud, not Code!

The Real AI Challenge is Cloud, not Code!

The AI industry is stealing the show as tech’s goldrush of the ’20s. Just looking at ChatGPT’s record setting user growth, and rapid 3rd party integration by top brands, it is not surprising the hype suggests this is the beginning of a major digital transformation. However, using AI/ML in your own products has some major challenges and obstacles. Below is a diagram of the end to end workflow of building and using an AI model: preparing the data, training a model, fine-tuning a model, hosting and running a model, building a backend service to serve the model, and building the user interface that interacts with the model.

Read more →

Advanced AWS Networking, Part 2

Advanced AWS Networking, Part 2

In this blog series, you will learn how to create a hub-and-spoke network architecture in AWS with centralized egress and traffic inspection. In this second installment, we’ll show you how to create spoke VPCs to run your workloads, verify centralized egress is working, and then add centralized traffic inspection using Pulumi, the infrastructure as code tool that enables you to manage infrastructure with real programming languages!

Read more →

Advanced AWS Networking, Part 1

Advanced AWS Networking, Part 1

In this blog series you will learn how to create a hub-and-spoke network architecture in AWS with centralized egress and traffic inspection. In this first installment, we’ll talk about the benefits of this architecture and begin to lay out some of its parts in Python with Pulumi, the infrastructure as code tool that enables you to manage infrastructure with real programming languages!

Read more →

Managing NOAA Open Data across Multiple Clouds with Pulumi

Managing NOAA Open Data across Multiple Clouds with Pulumi

Denis Willett is a software engineer at the North Carolina Institute of Climate Studies who works on the NOAA Open Data Dissemination Program. His work focuses on leveraging cloud technologies for the development of data processing and machine learning pipelines. Denis did his PhD in Entomology and Nematology at University of Florida and his undergraduate and masters work in Earth Systems at Stanford University. You can read his full bio here.

NOAA Open Data Dissemination (NODD) makes environmental data freely and publicly accessible across Amazon Web Services (AWS), Microsoft Azure (Azure), and Google Cloud Platform (GCP). These data include near real-time satellite imagery, weather models, radar feeds, drought information, ocean databases, and a suite of climate data records among many others. This program supports more than 220 datasets and over 24PB of open data. Since its inception, the program has been growing rapidly, almost doubling in size over the past year.

Read more →

Data Science in the Cloud

Data Science in the Cloud

Data science has advanced because tools like Jupyter Notebook hide complexity by running high level code for the specific problem they are trying to solve. Increasing the level of abstraction lets a data scientist be more productive by reducing the effort to try multiple approaches to near zero, which encourages experimentation and better results.

Data scientists typically work locally, but they often store data for analyses and models in the cloud. There are clear advantages to using cloud resources for these tasks:

  • Data scientists generally don’t want to manage their storage and databases.
  • They need to be able to store large data sets cheaply.
  • They need large capacity swings available on-demand.

SDKs like AWS’ Python library, boto3, can create resources, but they still require domain expertise to manage and properly architect a solution. The Pulumi Automation API improves on raw SDKs by providing high-level abstractions for creating and managing cloud services, letting data scientists concentrate on analyses and models without being well-versed in cloud APIs.

Read more →

Zero Downtime InfluxDB Migration with Pulumi and Aiven

Zero Downtime InfluxDB Migration with Pulumi and Aiven

In this article, I’ll show how Pulumi can be used with Aiven’s services to create infrastructure that can be migrated from cloud to cloud with no downtime.

This tutorial will use Python, Pulumi, Grafana, and an AWS Lambda function to simulate recording temperature data in an InfluxDB database.

Register for Multicloud OSS Database Deployments With Zero Downtime - Pulumi and Aiven and learn how to build robust, multi-cloud applications using the language, open source database, and cloud of your choice.

Read more →

Automate Your Infrastructure with Automation API and Python

Automate Your Infrastructure with Automation API and Python

General-purpose languages enable Infrastructure as Software – bringing tested toolchains and best practices to building infrastructure, e.g., languages, IDEs, testing, debugging, componentization, packaging, and versioning. Available in public preview, Pulumi’s Automation API is a robust programmatic layer on top of Pulumi’s infrastructure engine. It exposes Pulumi programs and stacks as strongly-typed and composable building blocks. Automation API allows you to embed the Pulumi engine inside your software projects so you can build software automation around entire infrastructure provisioning processes that normally require humans to operate.

Today, we are excited to announce Python support for this powerful feature, opening up a world of possibilities for Python developers.

Read more →

Create Amazon EKS clusters in your favorite language

Create Amazon EKS clusters in your favorite language

Pulumi’s infrastructure as code tooling combines the programming languages and tools you already know with the full power of cloud infrastructure. But until now, some Pulumi components for cloud infrastructure, like our popular EKS package for Amazon’s Elastic Kubernetes Service, were only available in a subset of the languages supported by Pulumi.

Now, you can use the EKS package–previously only available for TypeScript–in all four Pulumi languages: TypeScript, Python, .NET, and Go. Regardless of the language you choose, you can manage EKS clusters with Pulumi, starting with the v0.22.0 release. Check out our Modern Infrastructure Wednesday video to see it in action:

Read more →

Announcing Cross-Language Enum Support

Announcing Cross-Language Enum Support

Here at Pulumi, we believe in leveraging the best features of programming languages to create a delightful development experience for our users. Today, we continue our contributions in this area by announcing cross-language support for enum types in our provider SDKs, available in all Pulumi languages - Python, TypeScript, .NET and Go.

Read more →