Pulumi Blog: Developer portals

The Guide to Platform Engineering: 7 Steps to Get It Right

Sara Huddleston — Tue, 22 Oct 2024 07:47:03 +0000

In today’s fast-paced digital landscape, organizations are increasingly adopting platform engineering to optimize their software delivery and operations. Gartner predicts that by 2026, 80% of large software engineering organizations will have platform engineering teams to provide reusable services, components, and tools for application delivery. Additionally, by 2027, 80% of large enterprises will leverage platform engineering to scale DevOps initiatives in hybrid cloud environments effectively.

This shift is driven by the rise of cloud adoption, where many enterprises face the challenge of uncoordinated application teams deploying workloads in different ways across various cloud platforms. This siloed approach often results in a lack of standardization, security risks, and operational inefficiencies.

Platform engineering offers a strategic solution to these issues. This guide provides the essential steps to successfully implement platform engineering, from laying the foundation to scaling internal developer platforms (IDPs) for future growth.

On this article:

Step 1: Securing Executive Buy-in

The first step in your platform engineering journey is securing executive buy-in. This high-level support is essential, as platform engineering teams must create an organization-wide strategy that integrates internal developer portals (IDPs) and self-service capabilities. Leadership needs to understand how platform engineering can address inefficiencies caused by siloed application teams and uncoordinated cloud deployments.

To do this, present a clear roadmap with measurable outcomes, such as improved delivery speed and security. Use metrics like DORA (Deployment Frequency, Lead Time, Mean Time to Recovery) to demonstrate how platform engineering enhances security, standardization, and efficiency. Outline the required resources (headcount, budget, tooling) and align the project with business objectives.

Step 2: Staffing the Platform Engineering Team

Once you have executive backing, the next step is to assemble your platform engineering team. This team will develop and maintain your internal developer platform (IDP), ensuring it offers the organization secure, scalable, and self-service solutions.

Successful platform engineering teams are characterized by a strong customer focus, empathy for the needs of application teams, and the ability to act as the “glue” that binds the organization together. Key roles and responsibilities within the platform team include:

Customer-facing roles: Serve as the primary point of contact for application teams, understanding their needs and advocating for their requirements within the platform.
Infrastructure expertise: Possess deep knowledge of the underlying infrastructure, whether on-premises or in the cloud, to ensure the platform is reliable and scalable.
DevOps and automation skills: Leverage infrastructure as code (IaC) tools and techniques to automate the provisioning and management of the platform.
FinOps and cost optimization expertise: Understand the organization’s financial systems and processes to ensure the platform is cost-effective and aligned with budgetary constraints.
Software development capabilities: Develop the platform’s core components, including self-service interfaces and reusable infrastructure modules, using best practices in software engineering.

By assembling a team with this diverse set of skills and a customer-centric mindset, you’ll be well-positioned to build a platform that truly meets the needs of your application teams.

Step 3: Defining the Platform Mandate

Next, clearly define the platform team’s mandate to set expectations. This document should outline the platform’s purpose, the services it provides, and how it supports internal developer portals (IDPs) and self-service capabilities.

The platform mandate should serve as a central reference point for the platform team and its customers, ensuring everyone is aligned on the team’s mission and its value. Key elements to include in the platform mandate include:

Introduction: A high-level overview of the platform team’s purpose and the organizational outcomes it aims to achieve.
Supported systems: A list of the infrastructure, services, and tools that the platform team is responsible for managing and supporting.
Support model: Details on how application teams can access platform services, including any self-service capabilities and the team’s response times for different types of requests.
Team structure: An overview of the platform team’s members and their areas of expertise, making it easy for application teams to identify the right point of contact.

A well-defined mandate aligns the team and its customers with the platform’s value and purpose.

Step 4: Implementing the Pre-Production Pipeline

With a mandate in place, begin building the core capabilities, starting with a pre-production pipeline. This pipeline standardizes how application code moves from development to deployment, emphasizing security and consistency.

Key components of the pre-production pipeline include:

Version control: Implement a centralized version control system to manage application code and infrastructure as code (IaC) configurations.
Continuous integration (CI): Automate the build and testing of application artifacts, such as container images or deployment packages.
Artifact management: Provide a secure and versioned repository for storing and distributing the application artifacts produced by the CI process.
Static analysis: Integrate tools that can scan application code and dependencies for security vulnerabilities and other issues.
Application delivery: Automate the deployment of application artifacts to pre-production environments, such as staging or testing.

By establishing a standardized pre-production pipeline, you’ll ensure that all application teams follow a consistent, secure, and efficient process for getting their code into a production-ready state.

Step 5: Embracing Infrastructure as Code

Leveraging infrastructure as code (IaC) is crucial for managing the platform’s infrastructure. It enables consistent, scalable, and secure operations by automating the provisioning of resources, monitoring, and enforcing security policies.

Key areas where IaC can be applied within the platform include:

Foundational infrastructure: Provision and manage the underlying cloud resources, such as virtual networks, storage, and compute, that form the platform’s foundation.
Runtime platforms: Deploy and configure the runtime environments where application workloads will be executed, such as Kubernetes clusters or serverless functions.
Observability and monitoring: Set up the logging, metrics, and alerting systems that provide visibility into the platform’s health and performance.
Security and compliance: Implement security controls, such as secrets management and access policies, to ensure the platform meets regulatory and organizational requirements.
Pipelines and automation: Use IaC to define and version-control the platform’s own deployment and management pipelines, ensuring consistency and repeatability.

With infrastructure as code, the platform engineering team can ensure reliable, scalable, and secure infrastructure across the organization.

Step 6: Implementing Policy as Code

Alongside your IaC efforts, it’s important to set up a robust policy-as-code framework within your platform. Policy-as-code allows you to enforce security, compliance, and operational policies programmatically. This provides governance at scale, ensuring all teams follow the same rules.

Policy as code can be applied in two key ways:

Preventative controls: Implement policies that proactively validate and reject non-compliant infrastructure changes before they are provisioned, providing fast feedback to application teams.
Detective controls: Establish policies that continuously monitor the deployed infrastructure, triggering alerts or remediation actions when deviations from the desired state are detected.

By combining IaC and policy as code with self-service provisioning, you maintain security and compliance while giving teams autonomy.

Step 7: Evolving Towards Self-Service

As your platform matures, the ultimate goal is to empower your application teams with self-service capabilities, allowing them to provision the infrastructure and resources they need quickly and autonomously. This self-service model can take various forms, from pre-defined infrastructure modules to fully automated deployment pipelines.

When implementing self-service capabilities, keep the following platform engineering best practices in mind:

Start small and iterate: Don’t try to boil the ocean. Begin with a few well-defined use cases, such as CI/CD pipelines or database provisioning, and gradually expand the self-service offerings as your platform team gains experience and confidence.
Focus on user experience: Ensure that the self-service interfaces, whether web-based or command-line, are intuitive and easy to use. Gather feedback from application teams and continuously refine the user experience.
Maintain stability and reliability: Even with self-service, the platform team is responsible for ensuring the underlying infrastructure and services remain stable and reliable. Implement robust monitoring, alerting, and incident response processes to maintain platform health.
Embrace a product mindset: Treat the platform as a product, with the application teams as your customers. Continuously gather feedback, measure key performance indicators, and iterate on the platform’s capabilities to meet evolving needs.

By gradually transitioning towards a self-service model, you’ll empower your application teams to move faster while maintaining the necessary guardrails and controls to ensure the platform’s long-term success.

Conclusion: Embracing the Platform Engineering Journey

Building a successful platform engineering strategy involves following a clear roadmap. But remember, implementing the platform engineering strategy is a journey, not a destination. By following the steps outlined in this guide, you’ll be well on your way to building a robust, secure, scalable, and customer-centric internal developer platform (IDP) that empowers your organization to deliver software more efficiently and effectively.

With the right approach and mindset, your platform engineering efforts will become a strategic enabler for your organization’s digital transformation.

Build secure, scalable platforms with confidence—register for the Platform Engineering Course

Frequently Asked Questions

What is Platform Engineering?

There are many definitions for platform engineering, one of which is designing, building, and maintaining internal platforms that streamline software delivery by providing reusable tools, services, and workflows for developers. It focuses on scalability, efficiency, and self-service capabilities for internal teams.

For a more in-depth explanation of real-world use cases, see What is platform engineering.

Who are the Platform Customers?

The platform customers are the end users, typically internal teams (e.g., application developers and DevOps engineers), who use the platform’s tools and services to build, deploy, and manage software.

What is an Internal Developer Platform (IDP)?

A centralized platform that offers developers the tools and environments they need to build, test, and deploy applications. It abstracts away the complexity of underlying infrastructure, allowing teams to focus on coding and delivery.

What is a Platform Team?

The platform engineering team is a dedicated group of engineers responsible for managing the internal platform. They ensure the platform meets the needs of the organization, providing reliability, security, and scalability.

What is a Self-Service Infrastructure?

A platform feature that allows development teams to provision, configure, and manage infrastructure resources on demand without relying on other teams, fostering agility.

What is Infrastructure as Code (IaC)?

The process of managing and provisioning infrastructure through code, allowing infrastructure to be versioned, automated, and managed consistently across environments.

For a more in-depth explanation, see the What is Infrastructure as Code? page.

What is Policy as Code (PaC)?

Policy as Code (PaC) defines security, compliance, and operational policies in code to automate their enforcement across infrastructure and application deployments.

What is Developer Experience (DevEx)?

The overall experience developers have with using the platform, tools, and workflows provided by platform engineering. Optimizing DevEx ensures that developers can work more efficiently and effectively.

For a more in-depth explanation, read Beyond Productivity: Developer Experience is Business Critical.

Org-only Template Gallery and Setting Default Repositories: Streamline Your Development

Arun Loganathan — Fri, 19 Apr 2024 00:00:00 +0000

We’re constantly evolving the New Project Wizard to make it the preferred choice for platform teams to empower their internal teams to deploy resources quickly and in adherence to organizational standards. In January, we launched the Developer Portal Gallery to boost development velocity and reduce operational friction. Since then, we have seen strong adoption from our customers and received several new requests. We are excited to share the latest improvements we’ve made.

Primer on New Project Wizard and Developer Portal Gallery

The New Project Wizard

The New Project Wizard is a core component of Pulumi for Developer Portal that guides users through a structured process to deploy infrastructure resources through Pulumi. It allows users to select from pre-defined templates or generate new programs through Pulumi AI, configure, and deploy them without leaving the browser. This wizard significantly reduced the time and effort required to get projects off the ground, enabling developers to concentrate more on their development tasks and less on initial setup. Platform teams using the New Project Wizard do not have to custom-build and maintain complex tooling.

Overview of the Developer Portal Gallery

The Developer Portal Gallery allows platform teams to curate and distribute approved project templates within their organization. This feature supports compliance and standardization by ensuring that all projects adhere to organizational policies and best practices right from the start. The gallery not only facilitates the discovery and use of these templates but also makes it easy for developers to start new projects with templates that are pre-vetted and ready to deploy.

Using the Developer Portal Gallery can enhance collaboration across teams by standardizing the tools and processes used for creating and deploying projects. This alignment helps in maintaining a high level of governance as projects scale and ensures consistency across the development lifecycle. Platform teams can oversee template usage and configurations, making sure that each project conforms to the required security, architecture, and performance standards.

What’s New?

Here’s a breakdown of the latest enhancements:

Org-only Template Sources

Previously, the Developer Portal Gallery included both the Organization’s custom templates as well as Pulumi-authored templates. With this update, you can now add or remove the Pulumi-authored templates, ensuring that users only deploy organizationally approved templates. This simplifies template discovery and selection, reinforcing governance and compliance across development projects.

Default Repository Locations

We heard from customers they want their developers to not even have to worry about what repo their Pulumi code lives in. Starting today, admins can set default repository locations for each template source within the gallery. This eliminates the need for developers to select a repository, automatically using predetermined locations and streamlining their deployment process.

Auto-generated Project Names

Another improvement is the auto-generation of project names. Previously, users had to specify a project name, requiring additional input and sometimes leading to naming conflicts. Now, we automatically generate unique project names, further reducing friction and simplifying the project creation process.

Wrapping up

We’re continuously striving to make your experience with Pulumi Cloud as seamless and efficient as possible. Your feedback, big or small, is valuable to us, please feel free to share it in the Pulumi Cloud Requests repository.

Happy building!

Why Developer Experience (DevEx) is Business Critical

Sara Huddleston — Tue, 20 Feb 2024 20:47:44 +0000

“Developer experience is hard to sell,” said Cleve Littlefield, Engineering Manager at Pulumi, during a casual meeting. With experience as both an end-user developer and a lead in self-service platform implementation, Cleve’s observation stuck with me.

Though I have expertise in leading implementations and upgrades for internal platforms, none were specifically for developers. However, experience remains vital across departments, addressing tools, processes, systems, and best practices, aiming to reduce cognitive load, increase productivity, enhance collaboration, boost communication and much more.

Intriguingly, engineering teams may perceive its value differently. Therefore, we will dive into the concept of developer experience, aka DevEx, which, in truth, should translate into a competitive advantage.

In this DevEx article

What is developer experience?

Developer experience (DevEx), also referred to as DevX or DX, encompasses systems, technology, processes, and culture that collectively impact the effectiveness of software development. It looks at all components of a developer’s ecosystem—from environment to workflows to tools—and evaluates how they contribute to developer productivity, satisfaction, and operational impact.

Developer experience revolves around the ease or difficulty of executing essential tasks. A positive developer experience means those tasks are relatively easy to perform, translating into higher performance.

The quality of the experience is equally vital for developers building and managing internal software as it is for those involved in developing customer-facing software products.

Is developer experience important?

It is not just important. It’s business critical. If your company is in the business of creating and selling software or relies on critical internal software, developers are vital internal stakeholders to your business foundation.

Greg Mondello, director of product at GitHub, said, “In most contexts, software development capacity is the limiting factor for innovation. Therefore, improvements to the effectiveness of software development are inherently valuable […] companies with better DevEx outperform their competitors, regardless of vertical.”

Platform Engineering Forrester Opportunity Snapshot - DevEx impacts overall business performance. Credit: Humanitec

As per McKinsey’s report, an enhanced developer experience (DevEx) can yield significant advantages for organizations, including enhanced talent hiring and retention, improved security, and increased developer productivity- this makes DevEx important for all companies, not just tech.

A positive developer experience can yield numerous business advantages, which can include:

Improvement in collaboration across departments
Higher attraction and retention top talent
Increase developer productivity and developer velocity
Reduction in operational costs
Accelerate time-to-market
Support innovation
Elevate customer attraction and retention
Increase bottom line through revenue growth

To this end, let’s look into what goes into developer experience and how to achieve it.

The GitHub developer experience (DevEx) formula

GitHub’s DevEx formula takes into consideration the following:

Productivity: The speed and simplicity of implementing changes to a codebase
Impact: The seamless transition from idea to production without obstacles
Satisfaction: The environment, workflows, and tools influence developer happiness

Optimizing DevEx involves fostering a collaborative environment where developers can be most productive, impactful, and satisfied.

DevEx formula. Credit: GitHub

Noah Gift, the founder of Pragmatic AI Labs and a professor at Duke University specializing in machine learning, MLOps, AI, data science, and cloud architecture, noted that with the right platform, there could be:

A 75% increase in productivity
A sustained 22% productivity increase three years later
An 80% reduction in onboarding time

Merely having skilled developers is insufficient. Developers also require the appropriate tools and processes to excel in their work. Top-tier developers anticipate the availability of such resources, influencing the hiring and retention of talent.

What does a great developer experience look like?

Engineering teams that have good developer experiences are more productive and efficient. Great DevEx is often enabled by a platform where the end-users are the developers.

Developer-first readily available internal developer platform, infrastructure, and development tooling, with well-organized and explicit documentation, including tutorials, demo environments, and curated learning paths.
A unified hub for all developer requirements eliminates the need for developers to navigate through numerous tools, saving valuable time.
Simplified workflows emerge when developers can swiftly select app patterns, such as API microservices or front-end journeys, and deploy them within minutes. This includes all-encompassing elements like environments, pre-integrated DevSecOps pipelines, monitoring, and fully automated change and release procedures.
Centralized tracking of software, API, and infrastructure status and versions promotes asset transparency. A real-time view supports adherence to architectural standards, security controls, and patching status, with bots offering automatic suggestions for code enhancements related to issues.
Integrated analytics and KPIs such as developer velocity, tech debt, error rate, mean time to recovery, and infrastructure cost can be automatically pulled by standard organizational tools, including backlog, pipeline, and monitoring tools.

Survey by GitHub on a typical experience for developers. Credit: GitHub

Without the proper tools, progress may slow down the development process and hinder developer productivity. It becomes crucial to implement appropriate measures to avoid disruptions that could impact everyone involved, including the developers themselves. Although not often mentioned, DevOps and Platform Engineering also aim to enhance the developer experience and achieve the associated benefits.

How does DevEx intersect with DevOps?

DevOps is about developers and operations working together and sharing values, assumptions, and responsibility for the software they build and maintain.

According to Vilas Veeraraghavan, VP of Engineering at Truckstop, “Developer Experience (DevEx) is the investment you make to enable solid DevOps practices, reduce developer burnout, and improve retention. The need for this investment is more pronounced as companies grow and the tech stack gets more diverse.”

Watch Jowanza Joseph, Head of Engineering at StreamFT, below as he discusses how the right tool empowered developers, improved DevEx, reduced bottlenecks, increased productivity, and helped DevOps be a culture, a practice, not a person.

Let’s address a strong indicator of success or failure - culture.

Culture is not something you are—it’s something you do. Software companies often focus on DevOps practices and culture, but all would benefit from adopting a DevOpsAll since the entire company should strive for the same thing. The diversity of perspectives, ideas, and insights can spark creativity and innovation and even ensure product-market alignment.

Organizational culture is a predictor of outcome success

Why it matters? Organizational culture encompasses everyone, from the development team and operations to product management teams, sales, marketing, and more.

According to George Spafford, Senior Director Analyst at Gartner, “Infrastructure & Operations leaders embarking on DevOps initiatives and struggling to address organizational change and the value they will provide to the larger enterprise […] people-related factors tend to be the greatest challenges — not technology.”

According to a Harvard study encompassing 200 companies over 11 years, companies that focused on shaping their culture outperformed their competitors:

Revenue growth was 4.1 times higher
The stock price was 12.2 times higher
Net income was 756% vs. 1%
Return on investment (ROI) was 15 times higher

The driving force behind this exceptional financial performance is that engaged employees are prone to increased productivity, fostering superior innovation and more effective problem-solving.

Slide from Mark’s New York Atlassian Community Group presentation. Credit: Mark Cruth, principal modern work coach/advocate at Atlassian

Based on decades of research on employee engagement, Gallup has established that engaged employees consistently yield superior business outcomes compared to their counterparts. This holds true across various industries, company sizes, and nationalities and remains consistent in prosperous and challenging economic conditions.

If it is not yet clear, then I will spell it out. Developer experience impacts employee engagement. A better developer experience can boost the engagement of the engineers and developers in the organization, and more engaged people are more likely to go the extra mile.

How developer experience supports better business outcomes. Credit: Swarmia

If organizations invest in cultural values centered on human-first factors, then even the choices made in tools, processes, and systems will also be with the end-user in mind.

How do you implement a great developer experience?

There isn’t a bulletproof checklist, but there are several aspects to consider, and you can see the outline below. First, you need to view the developers as customers.

You will need to understand how developers work, the developer workflow, what the onboarding process looks like for a new developer, the developer teams’ pain points, what developers love and hate about their day-to-day work, where they are spending most of their time, where they wish they spent most of their time, and so much more, that boils down to communication, and asking the right questions, and deep diving into analytics.

Cultivate a DevOps culture

Shared responsibility: Encourage a culture of shared responsibility where developers, operations, and other stakeholders work collaboratively
Continuous learning: Promote a culture of continuous learning and improvement to adapt to evolving technologies and methodologies
Open communication: Foster open communication channels to enhance collaboration and transparency among team members
Company-wide collaboration tools: It’s essential to encourage communication among developers and cross-functional teams to stimulate creativity, spark innovation, and enhance productivity and the speed of decision-making. Cross-functional collaboration contributes to successful business outcomes

Define clear objectives

Alignment with business goals: Ensure DevEx objectives align with business goals to drive meaningful outcomes
User-centric approach: Prioritize user needs and feedback, shaping the DevEx strategy around the actual experiences of developers

Leverage developer-first tools

Internal developer portals (IDPs): IDPs enable developers to quickly provision approved infrastructure, boosting productivity with pre-configured architectures, automated testing, and deployment adhering to organizational standards
CI pipelines: usually, pipelines can be made faster and more unified. If they have a microservice architecture, they are likely duplicating pipeline code for every service
Software development kits (SDKs): SDKs are integral components of a positive DevEx and provide developers with tools that facilitate crafting applications tailored to specific platforms or frameworks. Offering easy-to-access, well-designed, functional SDKs can make a meaningful difference in helping engineers quickly prototype ideas and refine their creations
Infrastructure as code (IaC): Developers like to write code, particularly in their preferred programming language, provision infrastructure, ensuring consistency and reproducibility
Monitoring and observability: Implement effective monitoring and observability tools (e.g., Prometheus, New Relic, Grafana) to gain insights into application performance and proactively address issues
Collaborative documentation platforms: Use collaborative documentation platforms to document code, processes, and best practices, ensuring knowledge sharing and reducing onboarding friction

Platform Engineering Forrester Opportunity Snapshot - Which of the following steps organizations are taking to improve DevEx? Credit: Humanitec

Prioritize developer-friendly workflows

Fast feedback loops: Minimize feedback loops by integrating quick feedback mechanisms, allowing developers to identify and resolve issues early in the development cycle
Self-service environments: Provide developers with self-service environments to facilitate experimentation, testing, and debugging without unnecessary dependencies on other teams
Version control best practices: Enforce version control best practices to maintain a clean and organized codebase, allowing for easier collaboration and tracking of changes

Continuous integration of feedback

Developer surveys: Conduct regular surveys to gather feedback on the DevEx, ensuring continuous improvement based on real user experiences
Rapid iteration: Encourage a culture of rapid iteration, where feedback is quickly incorporated into the development process to enhance the overall developer experience

Measure and optimize

Key performance indicators (KPIs): Define and track key metrics such as deployment frequency, lead time, and mean time to recovery to measure the effectiveness of the DevEx strategy
Iterative optimization: Continuously iterate and optimize DevEx based on performance metrics and user feedback

Implementing a Developer Experience requires a holistic approach that blends cultural aspects with powerful DevOps and platform engineering tools. Watch Daniel Tao, Head of Engineering at Atlassian, and Sven Peters, developer evangelist, discuss how to build a great developer experience through culture and an integrated approach to developer tools.

What DevEx is not

Now that DevEx has been defined, let’s address some common misconceptions about developer experience.

DevEx is not the same as User Experience (UX)

Developer experience may sound similar to user experience (for developers), but it’s different, although user experience is a component. For example, a platform engineer building an internal developer platform must view the developers as their customers and create a good user experience by making it intuitive and easy to use.

DevEx is not about enabling lazy developers

Part of the DevEx strategy is to retain talent. Companies that invest in DevEx don’t do it to pamper developers. The true reason behind it is that they want their developers to stay “in the zone,” also known as “deep work.” When in this state, they are immersed in writing code and solving problems they enjoy working on, and their creativity is higher, it sparks innovative ideas, and the quality and speed of their output are also higher.

DevEx is not about implementing AI and cool tools

Technology impacts developer experience, a robust generative AI tool can help create code faster, and internal developer platforms can help reduce cognitive load on developers. But again, tooling is only one aspect of it. Tools can’t fix culture or replace processes. Many DevEx considerations should be considered to ensure that when the development teams adopt a tool, it addresses their pain points and enables developers.

DevEx is not a one-person decision

DevEx should be a joint effort from the entire team. One needs to be aware of the issues to plan to fix them, so open communication and team collaboration are vital to developing and implementing a DevEx strategy, intersecting DevOps and platform engineering initiatives. Many large organizations already have DevEx teams or professionals with a bird’s-eye view of what happens within the engineering team. They are internal advocates for software engineers. They consider the developer’s journey from being new developers and their ‘Time to First Contribution’ to day-to-day activities. They work with engineers to learn which problems are a priority and identify misconceptions and bottlenecks that may slow the development process.

Conclusion

Developer experience (DevEx) is essential for creating maximum output while increasing developer satisfaction. The appropriate technology tools, workflows, and a supportive development community can help foster progress and innovation while promoting high-quality software engineering results that can become a competitive advantage and ultimately benefit the business’s bottom line.

By dedicating attention to improving the DevEx, companies will also be able to bring more talent aboard, reduce their “Time to First Contribution,” enable developers to provide higher output contributions and improve talent retention. It’s time to start focusing on superior developer experiences.

Frequently asked questions

What does business critical mean?

Business critical means essential for a business to operate and be successful. It refers to business processes, systems, or activities that are considered essential for proper functioning, success, or survival. If these elements are disrupted or compromised, the impact can significantly affect its operations, financial health, or overall business objectives. Business-critical components are often prioritized for attention, protection, and resource allocation to ensure the organization’s continued effectiveness and stability, and to remain competitive.

What is the role of a Developer Experience Engineer (DXE)?

Developer Experience Engineers are internal advocates indispensable to making life easier for developers by making workflows more efficient, implementing the same processes across the development team, and assisting in choosing tools to lighten their load. Some companies do not necessarily have engineers with that job title, and it’s instead seen as part of DevOps or Platform Engineering initiatives. Other times, the internal developer advocate natural rises from the developer community to help form an integral part of the developer team’s unity. They also champion communication and work with cross-function coworkers and feel safe means of providing constructive feedback.

How do we cultivate a positive developer community?

The importance of DevEx’s human element is now in focus. A productive and effective developer community could be seen as a flourishing habitat, strengthening cooperation and maximizing productivity through collaboration. Such an atmosphere also sustains morale among the developers, resulting in job satisfaction and contentment.

Here are some steps to build a developer community: open communication needs to become standard, successes should consistently be recognized, and avenues for growth need to exist. Diversity and acceptance are significant. Task expectations should be attainable goals and set high enough not to compromise quality but not realistic so they can be achieved without burnout and unbalanced life routines.

How does technology impact developer experience?

Technological advancements continue to change software development and impact the Developer experience. AI, ML, TDD, and internal developer platforms are some of these technologies that help reduce developers’ cognitive load. They automate complex tasks for more accuracy and enable intelligent decision-making. Besides this, generative AI has come into play in creating higher-quality code faster. At the same time, GitOps is a management approach based on one source of truth - Git, which makes collaboration smoother & boosts productivity by allowing automation. All these things undoubtedly improve the DX substantially for any developer out there!

Developer Portal Gallery: Org Templates, Pulumi Templates and AI Generated Templates

Meagan Cojocar — Thu, 25 Jan 2024 00:00:00 +0000

Late last year we announced Pulumi for Developer Portals: a suite of features designed to empower organizations to construct robust internal developer portals. Since launching the level of adoption and customer interest has led us to make further improvements to enhance developer productivity and collaboration in the authoring experience. We are excited to announce the latest enhancement in Pulumi Cloud: the introduction of a template gallery in New Project Wizard, making creating cloud infrastructure easier than ever.

See it in action

What the Pulumi Cloud New Project Wizard?

The Pulumi Cloud New Project Wizard is the fastest way to get a Pulumi program up and running. The New Project Wizard provides a more turnkey, out-of-the-box alternative to the Pulumi Backstage Plugin or custom-built integrations.

By using the wizard, developers can generate Pulumi programs from your Organization Templates, commit and push code to GitHub, and trigger an initial deployment – all in a few clicks and without leaving the browser.

Try it out by selecting the ‘New Project’ option on the left-nav bar inside your Pulumi organization.

What is a template gallery?

The Pulumi Cloud template gallery is a curated collection of templates designed to help Platform Teams provision and distribute templates that get their developers up and running. Whether they’re setting up a simple server or orchestrating a complex microservices architecture, the template gallery can support a wide array of starting points that are both time-efficient and best practice-oriented. The idea is simple yet powerful – provide developers with the resources they need to get up and running swiftly, without having to reinvent the wheel each time.

The new template gallery in Pulumi Cloud contains three types of templates:

Organization Templates: Pulumi Cloud Enterprise and Business Critical editions support registering your own custom templates for your organization which will be made available in the New Project Wizard. This enables platform teams to standardize infrastructure creation and distribute those best practices where developers are already spending their time.
Pulumi Templates: All Pulumi Cloud organizations get a handful of Pulumi templates included in the gallery, templates Pulumi has authored to cover common architecture patterns.
AI Generated Templates: Starting today, you can leverage Pulumi AI to genera†e the templates you need from the template gallery. Pulumi AI is a purpose-built AI assistant that can create cloud infrastructure using Pulumi. If your developers are looking for a template that is not yet created, Pulumi AI can dramatically reduce the time it takes to build their Pulumi program.

Wrapping up

We look forward to hearing about your experience trying out the new project wizard template gallery! As always, please raise any feedback- big or small- in the Pulumi Cloud Requests repository.

Happy building!

Build The Best Developer Portal with Pulumi Templates

Meagan Cojocar — Thu, 12 Oct 2023 00:00:00 +0000

At Pulumi, we work with organizations that range from a few platform team members to entire departments for managing infrastructure. Many organizations, like Mercedes-Benz, have built internal developer platforms on top of Pulumi to enable developers to self-serve infrastructure templates, and partners like AWS Proton and Port have built integrations with Pulumi to enable self-service scenarios. We are thrilled to have announced yesterday the launch of our new suite of tools to build internal developer portals with Pulumi, a result of working directly with our customers to understand their problems and how Pulumi can solve it.

Pulumi launched Pulumi Templates last year, out of the box templates based for common architectures, such as serverless templates, container service templates and Kubernetes cluster templates. The introduction of these templates allows Pulumi customers to go from 0 to 1 much quicker, but as they scale their IaC complexity a need for custom templates arises. To address this, we are expending into managing custom templates as a self-service provisioning experience.

Introduction to Developer Portals with Pulumi

Many of our customers are building their own developer portals on top of Pulumi Automation API and Pulumi Deployments, but we also wanted to offer options for teams that cannot (or do not want to) build a custom solution. The suite of tools for building internal developer portals include:

New Project Wizard: Showcased in the GIF above, Pulumi Cloud now enables developers to provision infrastructure entirely via point-and-click in Pulumi Cloud with the new New Project Wizard experience. This offers a great out-of-the box developer portal experience for any team using Pulumi Cloud.
Organization Templates: Pulumi Cloud now supports registering your own custom templates for your organization which will be made available in the New Project Wizard. This enables platform teams to standardize infrastructure creation and distribute those best practices where developers are already spending their time.
Pulumi Backstage Plugin: For teams using Backstage, they can now integrate Pulumi directly within their Backstage instance, via support for a new Pulumi plugin as well as new Pulumi scaffolder actions.
pulumi new support for private templates: By default, pulumi new provides a number of templates provided by Pulumi, but now your organization can also use private templates.

Key Benefits

Ease of Deployment: With this new functionality, you can quickly find and deploy the services you need, reducing the time and complexity of managing your cloud infrastructure. As a developer, if you are looking for a platform team sanctioned way to deploy a Kubernetes cluster, instead of chasing it down you can head to Pulumi Cloud, find the golden template and deploy it.

Enhanced Distribution: As a platform team member, share templates and configurations with your developers to ensure consistency and collaboration across your organization. It’s as simple as adding the template to a repo and selecting that repo in Pulumi Cloud. Now all of your developers have access to it when they go to create new infrastructure.

Combination with Other Features: The new template experience seamlessly integrates with your existing Pulumi workflows, providing a unified experience across your infrastructure management. Increase your developer productivity by using this suite of features in combination with Pulumi ESC. Attach an Environment to your template to have it come with configuration and secret values. Once the templates are deployed they will show in the Pulumi Cloud console use Policies over it, set up Review Stacks on the new stack to get ephemeral environments tied to PRs, find the resources that were deployed with Resource Search, and all the other Pulumi Cloud features.

Extensibility: You can leverage the out of the box New Project Wizard Experience or build developer portals with Pulumi yourself using any combination of: the Pulumi Backstage Plugin, pulumi new public and private templates, Automation API, Pulumi Deployments, or the Pulumi Cloud REST API.

Getting Started

New Project Wizard

The New Project Wizard allows anyone in your organization to pick a template they want to install, create the repo or directory for it, and walk through configuring the deployment of that template. We’ve added support for configuring Pulumi Deployments, meaning any template can be deployed without needing the Pulumi CLI locally, or any other CI/CD configuration. Follow these steps to be up and running with your infrastructure in a few clicks:

You can either 1/ use the UI or 2/ use Deploy with Pulumi buttons. If using the UI, navigate to the “New Project” tab. Select “Use a template” if you’d like to create a fully featured project, or select “Use a starter” if you want to create a bare-bones project with only the minimal necessary boilerplate file structure and Pulumi code. If you are using Deploy with Pulumi buttons, you can embed a button on your templates inside GitHub to enable point-and-click deployment of those templates.
In order to use templates you will need to authorize Pulumi with GitHub so that it can clone private repositories as template sources and commit new code for your projects. Click the “Authorize GitHub” button and accept the required permissions if you would like to use templates.
On the next screen select your deployment method. You may need to install the Pulumi GitHub app if you haven’t already to use Pulumi Deployments.
You’ll now be prompted to enter some information about the project you’re about to create: the project and stack name, the configuration details specific to that template, the repo to use when committing your new project code, and optionally the Environment you want to use. Environments enable you to use a bundle of pre-configured secrets and configuration values without needing to re-specify all of them during project creation.
After you’ve configured your project settings, you will be able to configure the behavior of Deployments, including when to trigger new Deployments and environment variables to include with your updates.

And just like that you have created a new project from a template that is deployed, without the use of the Pulumi CLI.

Organization Templates

Pulumi now allows customers on the Pulumi Cloud Enterprise and Business Critical editions to define Organization Templates to help get projects off the ground faster. A Pulumi organization admin can specify an Organization Template Source (or Sources) which will then be populated in the New Project Wizard experience described above.

Navigate to the “Integrations” tab to define your Organization Templates Source. Enter sources as github.com/<owner>/<repo>/<optional subdirectory>. A source can be a directory containing either a Pulumi.yaml, or other subdirectories with their own Pulumi.yaml files. A single repository can provide multiple templates from various subdirectories. For example, these are both valid sources: github.com/pulumi/templates (all public Pulumi templates) and github.com/pulumi/templates/aws-typescript (a specific public template).

The Pulumi.yaml file can optionally contain a template section, which typically includes a config section for specifying required config values for the project. Each config value can have a description and a default value which can be overridden. Config values can be marked as secret, which ensures values in templated projects will be stored with secure encryption.

name: my-aws-project
runtime: nodejs
description: My AWS project description
template:
 config:
 aws:region:
 description: The AWS region to deploy into
 default: us-west-2
 myAccessToken:
 description: My access token
 secret: true

The above snippet includes an aws:region configuration value with a default value of us-west-2, as well as a myAccessToken secret without a default value.

Pulumi & Backstage

We’ve seen many developer portal technologies rapidly growing in popularity over the last few years. In particular, we’ve seen Pulumi users adopting Backstage and as a result we built the Pulumi Backstage Plugin to address the needs of organizations using Backstage and Pulumi together. There are now two ways to use Pulumi from within Backstage, you can leverage both or just one or the other:

Pulumi Scaffolder Backend Module
Pulumi Backstage Plugin

Pulumi Scaffolder Backend Module: The Pulumi Scaffolder Backend Module extends the scaffolding with two new actions: Pulumi New and Pulumi Up. The Pulumi New action is a custom action that allows you to create a new Pulumi project from a template using the pulumi new Pulumi CLI command. The Pulumi Up Action is a custom action that allows you to run the pulumi up command, which creates or updates resources in a stack.

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
 name: kubernetes-template
 title: Kubernetes Cluster
 description: |
 A template for creating a new Kubernetes Cluster.
 tags:
 - pulumi
 - kubernetes
spec:
 steps:
 - id: pulumi-new-component
 name: Cookie cut the component Pulumi project
 action: pulumi:new
 input:
 name: "${{ parameters.component_id }}-infrastructure"
 description: ${{ parameters.description | dump }}
 organization: ediri
 stack: ${{ parameters.stack }}
 template: "https://github.com/my-silly-organisation/microservice-civo/tree/main/infrastructure-${{ parameters.cloud }}-${{ parameters.language }}"
 config:
 "node:node_count": "${{ parameters.nodeCount }}"
 folder: .

With a few clicks, developers can create new software projects provisioned with best practices, with code managed in the right repositories, and with the right configurations, all from a self-service UI.

Pulumi Backstage Plugin: Backstage Plugins are extensions or add-ons to the core Backstage application that allow you to integrate additional tools, services, and functionalities into the platform, enhancing its capabilities and customizing it to meet the specific needs of your development environment.

The Pulumi Backstage Plugin extends Backstage by displaying essential details like the Pulumi stack, organization, project name, and description, as well as providing an activity view with aggregated information from the Pulumi Cloud. See a screenshot of the new Pulumi tab within Backstage below:

Check out the Pulumi Backstage Plugin in the Backstage Plugin directory or the Roadie Pulumi Backstage Plugin guide to get started today.

Wrapping it up

The launch of this new functionality for internal developer portals marks a significant step forward in our commitment to providing powerful, turn-key tools for platform teams. We believe these features will enhance your experience with Pulumi, whether you’re a seasoned cloud expert leading a team of platform engineers or just getting started on your IaC journey.

As always, we welcome your feedback in the Pulumi Cloud Requests repository and are here to support you every step of the way on your platform engineering journey.

Pulumi for Platform Teams: New Features for Developer Portals, Policy and Deployments

Luke Hoban — Wed, 11 Oct 2023 04:00:00 -0700

Over the last two years, we’ve seen a huge surge in adoption of Pulumi by Platform Teams – centralized teams within a business responsible for building out core cloud infrastructure and providing tools to the rest of the organization to maximize the productivity, cost efficacy, compliance and velocity of application and service delivery throughout the organization. These teams use Pulumi to manage their own cloud infrastructure complexity, to offer best practices components to their organizations, to enforce organizational policy, and to drive infrastructure delivery automation.

Today, we’re excited to announce a broad range of new features that extend Pulumi Cloud’s support for Platform Teams, across the entire lifecycle of cloud delivery.

Developer Portals: Every platform team we work with is looking to expose reusable cloud infrastructure components and templates via a developer portal of some kind. Many are building their own custom developer portals on top of Pulumi Automation API and Pulumi Deployments, but we also wanted to offer options for teams that cannot (or do not want to) build a custom solution. Today, we are releasing three new options for Platform Teams:

🆕 Organization Templates and New Project Wizard: Pulumi Cloud now supports registering your own custom templates for your organization, and enabling developers in your organization to provision them entirely via point-and-click in Pulumi Cloud with the new New Project Wizard. This offers a great out-of-the box developer portal experience for any team using Pulumi Cloud.
🆕 Pulumi Backstage Plugin: For teams using Backstage, they can now integrate Pulumi directly within their Backstage instance, via support for a new Pulumi plugin as well as new Pulumi scaffolder actions.
🆕 pulumi new support for private templates: By default, pulumi new provides a number of templates provided by Pulumi, but it can also use your own private templates.

Policy and Compliance: Platform teams need robust and flexible solutions for establishing policy and compliance guardrails for their cloud infrastructure. Pulumi’s existing CrossGuard policy-as-code framework provides the richest option for defining and enforcing policy across all infrastructure in an organization, using the full capabilities of your favorite languages. Today, we are extending that support with two important new features for platform teams:

🆕 Compliance Ready Policies: New pre-built policy packs for for enforcing common security and compliance policies (PCI DSS, ISO 27001 and CIS) across a broad range of cloud providers (AWS, Azure, Google Cloud and Kubernetes), and dozens of services within each platform.
🆕 Remediation Policies: Pulumi Policy as Code now supports authoring policies which don’t just warn or error, but actually change the infrastructure directly to remediate compliance gaps. Remediation policies can be configured and applied across any subset of stacks within an organization.

Deployments and Delivery: Managing the delivery of cloud infrastructure changes across a wide variety of environments is an essential component of a cloud platform, and we released Pulumi Deployments in preview last year to provide a rich solution for infrastructure delivery directly within Pulumi Cloud. Today, we are making that support Generally Available (GA) along with a few key enhancements:

🆕 Deployments GA: Pulumi Deployments is now generally available, with pricing and support available across all Pulumi Cloud editions.
🆕 Deployments Support for GitHub Enterprise: Pulumi Deployments now supports integration with GitHub Enterprise as well as GitHub.com.
🆕 Org-Wide Deployments Overview Page: A new page where you can see all deployments across the organization, giving you a single pane of glass to manage company wide deployments.
🔜 Deployments Customer Managed Agents: Customers will soon be able to run deployments inside agents they run within their own infrastructure, for cases where this is required for unique compliance or security reasons.

Together with Pulumi ESC (Environments, Secrets and Configuration) which we announced yesterday, and the core Pulumi IaC infrastructure as code platform, these new features provide a rich collection of building blocks and out of the box experiences for empowering platform teams via Pulumi Cloud. At every layer of the Platform Engineering stack, Pulumi and Pulumi Cloud offer key technologies to accelerate your Platform Engineering projects.

Organization Templates and New Project Wizard

We’ve introduced a new wizard experience to create Pulumi projects into Pulumi Cloud, making it easier than ever to get started deploying new infrastructure with Pulumi.

The new project wizard allows anyone in your organization to pick a template they want to install, and walk through configuring the deployment of that template. We’ve added support for configuring Pulumi Deployments automatically, so that the template can be deployed without needing the Pulumi CLI locally, or any other CI/CD configuration. Just a few clicks and you have deployed your infrastructure.

We’ve also added integration with the newly released Pulumi ESC so that you can set up environments (and the configuration and credentials needed to deploy to them) that users within your organization can deploy into, determine who can use which environments using RBAC, and then allow those users to deploy templates directly into the environments they can access – no need to copy secrets, credentials or configuration around!

We’ve also added the ability for organizations to provide their own custom set of Organization Templates — templates specific to their organization, which can live in either public or private Git repositories. The Organization Template Source provided will populate the templates that are displayed to members in the New Project wizard experience.

Together, Organization Templates and the new New Project Wizard offer a foundational developer portal experience directly within the Pulumi Cloud console. Users can come to Pulumi Cloud, select one of the pre-defined templates set up by their Platform Team, and then in a few clicks have it deployed into a predefined cloud environment.

The New Project wizard is available now for all Pulumi users, and Organization Templates are available to Pulumi Enterprise and Business Critical users. Check out the documentation to get started.

Backstage Plugin

We’ve seen many developer portal technologies rapidly growing in popularity over the last few years. In particular, we’ve seen many Pulumi users adopting Backstage. Last week we launched a new Pulumi Backstage Plugin to address the needs of organizations using Backstage and Pulumi together.

The new Pulumi tab gives you direct access to all Pulumi stack activity associated with your backstage projects that include Pulumi stacks.

The Pulumi Backstage Plugin also supports two new scaffolding actions, pulumi:new and pulumi:up, which can be used to template out new Pulumi projects and to trigger updates to Pulumi stacks, fully integrated into the Backstage scaffolding system.

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
 name: kubernetes-template
 title: Kubernetes Cluster
 description: |
 A template for creating a new Kubernetes Cluster.
 tags:
 - pulumi
 - kubernetes
spec:
 steps:
 - id: pulumi-new-component
 name: Cookie cut the component Pulumi project
 action: pulumi:new
 input:
 name: "${{ parameters.component_id }}-infrastructure"
 description: ${{ parameters.description | dump }}
 organization: ediri
 stack: ${{ parameters.stack }}
 template: "https://github.com/my-silly-organisation/microservice-civo/tree/main/infrastructure-${{ parameters.cloud }}-${{ parameters.language }}"
 config:
 "node:node_count": "${{ parameters.nodeCount }}"
 folder: .

Check out the Pulumi Backstage Plugin in the Backstage Plugin directory or the Roadie Pulumi Backstage Plugin guide to get started today.

Compliance Ready Policies

We first released our CrossGuard policy-as-code framework nearly 4 years ago, and have seen many large organizations leverage its rich expressiveness, flexibility and early prevention features to enforce key compliance criteria for their organization. Support in Pulumi Cloud for automatically enforcing these policies across all Pulumi updates within an organization allows platform teams to put compliance guardrails in place for users within their organization.

However, traditionally each organization has needed to build their own policies, resulting in some activation energy needed to start benefiting from CrossGuard. As we’ve worked with many customers to build out their own internal compliance policies using CrossGuard, we’ve identified some common compliance standards and policies that we can offer directly as part of CrossGuard.

Today, we are making a large collection of new policies available for CrossGuard users. These policies support many common compliance standards (PCI DSS, ISO 27001 and CIS) across a broad range of cloud providers (AWS, Azure, Google Cloud and Kubernetes), and dozens of services within each platform.

These policies are offered as a library, from which Platform Teams can enable the subset they want to apply. For example:

import { PolicyPack } from "@pulumi/policy";
import { policyManager } from "@pulumi/compliance-policy-manager";

new PolicyPack("aws-iso27001-compliance-ready-policies-typescript", {
 policies:[
 ...policyManager.selectPolicies({
 vendors: ["aws"],
 services: ["ec2", "eks"],
 severities: ["critical", "high"],
 frameworks: ["iso27001"]
 }, "advisory"),
 ],
});

The new Policy Pack wizard also makes it easier than ever to create a new policy pack from the compliance-ready policies, and enable it within your organization.

Check out the documentation and open source repository to get started with Compliance-Ready Policies today.

Remediation Policies

Existing CrossGuard policies have always had the ability to be set to Advisory to emit a warning when they fail, or to Mandatory to prevent the Pulumi deployment from completing when they fail. This enables strict prevention of non-compliant infrastructure deployments before they even get into production.

However, in many cases, it’s clear what change needs to be made to the infrastructure to make it compliant. Today, we’ve launched new support in CrossGuard for Remediation Policies, a new kind of policy which actually fixes up the infrastructure to be compliant instead of just warning or erroring!

This is a magical experience for both policy authors and infrastructure engineers.

const s3NoPublicRead: ResourceValidationPolicy = {
 name: "s3-no-public-read",
 description: "Prohibits setting the publicRead or publicReadWrite permission on AWS S3 buckets.",
 enforcementLevel: "remediate",
 remediateResource: remediateResourceOfType(aws.s3.Bucket, (bucket, args) => {
 if (bucket.acl === "public-read" || bucket.acl === "public-read-write") {
 // Modify the ACL and return the new bucket state to use instead.
 bucket.acl = "private";
 return bucket;
 }
 }),
};

def s3_no_public_read_remediator(args: ResourceValidationArgs):
 if args.resource_type == "aws:s3/bucket:Bucket" and "acl" in args.props:
 acl = args.props["acl"]
 if acl == "public-read" or acl == "public-read-write":
 # Modify the ACL and return the new bucket state to use instead.
 args.props["acl"] = "private
 return args.props

s3_no_public_read = ResourceValidationPolicy(
 name="s3-no-public-read",
 description="Prohibits setting the publicRead or publicReadWrite permission on AWS S3 buckets.",
 enforcement_level=EnforcementLevel.REMEDIATE,
 remediate=s3_no_public_read_remediator,
)

Check out the documentation to get started adding remediation policies to your policy-as-code configuration today.

Deployments GA

In the last year since we launched Pulumi Deployments in preview, we’ve seen incredible growth in usage and scale of Pulumi Deployments across the Pulumi user base. Customers use Pulumi Deployments for a single built-for-purpose CI/CD solution for their Pulumi infrastructure delivery, but also leverage rich API-driven deployments, Review Stacks, Click-to-Deploy within the Pulumi Cloud Console, and much more.

As part of General Availability, we are announcing pricing for deployments at $0.01 per deployment minute. The new pricing will be effective November 1st. Individual Edition users get 500 minutes free per month, and Team Edition users get 3,000 minutes free per month. Full details of the pricing can be found on the Pricing Page.

In addition to pricing, we’re making available several new features in the Deployments platform.

The new Deployments pane in the Pulumi Cloud console provides a unified view across all deployments in your organization. This includes in progress and queued deployments so that you can easily access any currently in flight deployments, as well as a full history of all deployments performed across all stacks you have access to within your organization.

And finally, we’re extending Pulumi Deployments to support additional key enterprise needs. We recently added support for connecting Pulumi Deployments to GitHub Enterprise, and we will be delivering support for self-hosted Pulumi Deployments agents which can be run within your own infrastructure later this month. You can fill out this form to register for the private preview of self-hosted agents today.

Conclusion

Pulumi Cloud offers a rich and broad platform to support any platform team today. Across the entire lifecycle of cloud infrastructure and application delivery, Pulumi Cloud supports platform teams in building cloud platforms for their organizations which empower developers via rich IaC components and developer portals, enable seamless delivery through Deployments and Automation API, provide robust compliance guardrails through Compliance-Ready Policies and Remediation Policies, and securely manage secrets and configuration across a variety of environments with Pulumi ESC.

It’s never been easier to build your internal developer platform with the help of Pulumi Cloud.

Pulumi Backstage Plugin: Developer Portal Integration

Engin Diri — Fri, 06 Oct 2023 15:00:48 -0400

Backstage is an open source framework for building developer portals, created at Spotify and donated to the CNCF. It allows engineers to create their own development portal internally. Backstage makes it easy for teams to create a unified portal for their infrastructure tooling, software templates, services, documentation and plugins for external tools.

What are Backstage plugins?

Backstage plugins typically include functions from another application or platform and place them in context in Backstage. This adds functionality within the development environment and helps centralize critical developer functions. Backstage provides a marketplace where developers can review plugins and implement them into their own applications.

Pulumi Joins the Stage

Today we are announcing a new Pulumi Plugin for Backstage. Using the Pulumi Backstage Plugin, you can create a new Pulumi project from an organization template stored in Pulumi, you can run previews and updates, and you can run and view Deployment.

How to get started with the Pulumi Backstage Integration

The Pulumi Backstage integration has two independent components:

Pulumi Scaffolder Backend Module
Pulumi Backstage Plugin

What is Scaffolding in Backstage?

The Backstage Scaffolder is a core part of the Backstage developer portal. Using standardized templates, developers can create new components (services, libraries, websites, or other software). With a few clicks, developers can create new software projects provisioned with best practices, with code managed in the right repositories, and with the right configurations, all from a self-service UI.

Pulumi Extends Backstage Scaffolding

The Pulumi Scaffolder Backend Module extends the scaffolding with two new actions: pulumi:new and pulumi:up

With the new Pulumi Scaffolder Backend Module, organizations can standardize infrastructure creation through their own custom templates. This enables a consistent approach for every new project while minimizing manual setup and configuration by developers and platform teams.

The next component of the Pulumi Backstage integration is the Pulumi Plugin. Backstage Plugins are extensions or add-ons to the core Backstage application that allow you to integrate additional tools, services, and functionalities into the platform, enhancing its capabilities and customizing it to meet the specific needs of your development environment.

The Pulumi plugin extends Backstage by displaying essential details like the Pulumi stack, organization, project name, and description, as well as providing an activity view with aggregated information from the Pulumi Cloud.

Connecting Pulumi Cloud with Backstage with the Pulumi Backstage plugin

The Pulumi plugin extends Backstage by displaying essential details like the Pulumi stack, organization, project name, and description, as well as providing an activity view for the entity with aggregating information from the Pulumi Cloud.

Utilizing the Pulumi Backstage integration, including its plugin and scaffolder, brings forth many benefits that streamline infrastructure management, enhance developer experience, and foster a well-orchestrated DevOps culture. See the key benefits below.

Key Benefits of the Pulumi Backstage Plugin

By integrating with Backstage, Pulumi is simplifying a common scenario that many of our users encounter: empowering developers to self-service new infrastructure while enforcing common standards and best prctices. A few benefits of this integration are:

Consolidated View: Obtain a singular view of your infrastructure and application status, merging Pulumi’s infrastructure-as-code (IaC) capabilities with the Backstage developer portal.

Simplified Navigation: Manage both applications and infrastructure from a single pane of glass - even for complex multi-cloud deployments.

Transparent Communication: Facilitate seamless communication across development, operations, and infrastructure teams by providing a shared view and management platform.

Knowledge Sharing: Enable teams to share infrastructure code, configurations, and status transparently, fostering collective knowledge and expertise.

IaC Adoption: Leverage Pulumi’s infrastructure-as-code capabilities to define, deploy, and manage infrastructure efficiently.

Automated Scaffolding: Utilize the Backstage scaffolder to automate the creation and management of Pulumi projects and stacks.

Self-Service Operations: Empower developers with the ability to provision, configure, and manage infrastructure independently, reducing dependencies on operations teams.

Accelerated Development: Enhance the speed and agility of development cycles by providing developers with the tools to manage their infrastructure needs autonomously.

Standardized Workflows: Employ consistent and standardized workflows for infrastructure provisioning and management across all development teams.

Best Practices: Promote the usage of best practices in infrastructure management and application deployment through pre-defined templates and configurations.

How to Get the Pulumi Backstage Plugin

There are two ways to get the Pulumi Backstage Plugin: First, if you are hosting Backstage yourself, you can find the plugin in our GitHub repo.

Secondly, if you are using the Roadie managed Backstage service, you can follow these instructions to use the Pulumi plugin with Roadie.

You might also like:

Conclusion

The Pulumi Backstage Plugin brings simplicity and power to developers by combining Pulumi’s infrastructure management with the user-friendly Backstage developer portal. It makes managing apps and infrastructure easy, all from one place, boosting teamwork and making developers’ lives simpler. With straightforward tools and automated setups, developers can work faster and smarter, managing their projects with fewer hurdles and more clarity.

In short, it’s not just a plugin; it’s a step towards smoother, more integrated development workflows for everyone involved.