Posts Tagged data-and-analytics

Resource Search - AI Assist is Generally Available

Resource Search - AI Assist is Generally Available

Pulumi Cloud Resource Search AI assist functionality is now generally available to all organizations! In addition we have shipped some improvements to the feature to make it easier to use and more discoverable: a toggle on the search bar, suggested queries and an “I’m Feeling Lucky” button to generate a random query for you.

Read more →

Pulumi Insights: Intelligence for Cloud Infrastructure

Pulumi Insights: Intelligence for Cloud Infrastructure

We’ve seen incredible acceleration of cloud adoption over the past 5 years. Pulumi’s flagship open source IaC solution gives engineers great tools to scale up their cloud infrastructure using the same programming languages and tools they already know and love. As a result, thousands of companies of every size and scale have adopted Pulumi as a lynchpin of their cloud infrastructure strategy.

Today we’re excited to announce Pulumi Insights, the next major productivity enhancement for infrastructure as code. Pulumi Insights provides intelligence, search, and analytics over any infrastructure, in any cloud across your organization, leveraging the latest advances in generative AI and Large Language Models (LLMs). Whether you have an AWS VPC, a Kubernetes CRD, or a DataDog alarm definition, Pulumi Insights enables you to intelligently find and interact with all of your resources from within the Pulumi Cloud.

Read more →

Data Science in the Cloud

Data Science in the Cloud

Data science has advanced because tools like Jupyter Notebook hide complexity by running high level code for the specific problem they are trying to solve. Increasing the level of abstraction lets a data scientist be more productive by reducing the effort to try multiple approaches to near zero, which encourages experimentation and better results.

Data scientists typically work locally, but they often store data for analyses and models in the cloud. There are clear advantages to using cloud resources for these tasks:

  • Data scientists generally don’t want to manage their storage and databases.
  • They need to be able to store large data sets cheaply.
  • They need large capacity swings available on-demand.

SDKs like AWS’ Python library, boto3, can create resources, but they still require domain expertise to manage and properly architect a solution. The Pulumi Automation API improves on raw SDKs by providing high-level abstractions for creating and managing cloud services, letting data scientists concentrate on analyses and models without being well-versed in cloud APIs.

Read more →

AWS Serverless Analytics

AWS Serverless Analytics

Whether it’s an IoT installation, a website, or a mobile app, modern software systems generate a trove of usage and performance data. While it can be daunting to collect and manage, surfacing data empowers the business to make informed product investments. In this article, we’ll explore the following: An overview of the traditional Redshift analytics stack on AWS, the use cases it excels at, and where it falls apart. An alternative architecture utilizing serverless and streaming.

Read more →

Create Secure Jupyter Notebooks on Kubernetes using Pulumi

Create Secure Jupyter Notebooks on Kubernetes using Pulumi

In this post, we will work through an example that shows how to use Pulumi to create Jupyter Notebooks on Kubernetes. Having worked on Kubernetes since 2015, a couple of critical benefits jump out that may resonate with you as well:

  • You write everything in code - TypeScript in our example here.
  • You need not initialize Tiller or Helm to work with existing Helm charts like nginx-ingress-controller that we use here.
  • The security patterns in Helm and Tiller are no longer concerns, rather you get to focus on the RBAC of the actual service which is Jupyter-notebook in this example.
  • You accomplish more with less YAML and iteratively work towards your use cases.

Read more →