Integrating DevOps and Security in Platform Engineering

Posted on

Sara Huddleston Sara Huddleston

Platform engineering has become essential for mid-to-large organizations, moving beyond a DevOps trend. Gartner predicts that by 2026, 80% of software companies will have internal platform services to streamline development. The goal is to empower developers with self-service tools while maintaining security, compliance, and reliability through DevSecOps practices.

At PulumiUP Europe 2024, experts shared insights on aligning DevOps with security to build scalable, secure platforms:

  • Jess Mink, Sr. Director of Platform Engineering at Honeycomb
  • Kief Morris, Global Head of Infrastructure Engineering at ThoughtWorks
  • Lindsay Jack, VP of Engineering & Security at Snyk
  • Nariman Aga-Tagiyev, Application Security Architect at WiseFrog Security
  • Komal Ali, Engineering Manager at Pulumi

The panel discussed key strategies, challenges, and pillars of successful platform engineering.

In this article:

The Core Pillars of Platform Engineering

Platform engineering teams comprise multiple professionals with many responsibilities and focus areas. According to our panel of experts, the core pillars of platform engineering include:

  • Developer Experience (DevEx): Provide developers with the tools, frameworks, and abstractions they need to be productive and proactive without getting stuck in infrastructure or operational concerns.
  • Reliability and Scalability: Ensure that the platform and infrastructure can support the organization’s needs, with the ability to scale up or down as required.
  • Security and Compliance: Embed robust, accessible security and compliance frameworks into the development lifecycle while making it easy for developers to adhere to these policies.
  • Automation and Tooling: Leverage Infrastructure as Code (IaC) and automation to enforce standardized processes and consistency and reduce cognitive load and manual effort.
  • Observability and Monitoring: Provide visibility into the platform’s health and performance, delivering actionable insights that allow teams to identify and resolve issues quickly.

These pillars work together to create a platform that empowers developers to innovate and deliver value to the organization and customers while maintaining the necessary controls and safeguards.

As Jess Mink, Director of Platform Engineering at Honeycomb, explains:

“The goal of platform engineering is to help the company run smoother and faster and unlock things people didn’t know were possible […] We tend to focus on tools and software, but it’s really about people, processes, and tools. If you consider this, platforms are responsible for social and technical support across the organization. A common pitfall is building tools no one uses because you didn’t meet people where they are.”

Aligning DevOps and Security for Secure Platform Engineering

Integrating security into platform engineering ensures it becomes a proactive part of the development lifecycle and ensures that “DevSecOps” is not an afterthought but a core consideration.

Key best practices shared by the panel include:

Shift Left Security

One of the fundamental principles of DevSecOps is to “shift left” - that is, to integrate security earlier in the development process rather than waiting until the end. This can involve activities such as:

  • Incorporating security requirements and threat modeling into the initial design phase
  • Automating security scans and tests as part of the continuous integration (CI) pipeline
  • Providing developers with secure coding guidelines and tools to identify and remediate vulnerabilities

By addressing security concerns upfront, organizations can reduce the time and cost of remediating issues later in the development lifecycle.

Embrace Automation and Standardization

Consistency is key. Platform engineering teams should leverage automation and built-in safeguards and security processes. This may include:

  • Automating the provisioning of secure infrastructure and application environments
  • Implementing Infrastructure as Code (IaC) to define and manage infrastructure in a declarative, version-controlled manner
  • Standardizing security controls, policies, and configurations across the platform

By automating these tasks, platform engineering teams can reduce the risk of human error, improve visibility and auditability, and free up developers to focus on building features rather than managing infrastructure.

Prioritize Observability and Monitoring

Effective security requires visibility into the health and performance of the platform. Platform engineering teams should invest in robust observability and monitoring solutions to:

By integrating security-focused monitoring and alerting into the platform, organizations can quickly identify and mitigate threats while also providing developers with the necessary context to understand and address security-related issues.

Foster a Culture of Collaboration

Platform engineering is often referred to as being the practical application of DevOps practices. Integrating security practices often requires a cultural shift towards collaboration and shared responsibility between all teams in development, operations, and security, thus the name DevSecOps. Platform engineering teams can facilitate this by:

  • Treating Platforms as Products
  • Involving security stakeholders in the design and planning of platform initiatives
  • Providing security training and education to developers to empower them to make informed decisions
  • Establishing communication channels and feedback loops between teams

As Kief Morris, Global Head of Infrastructure Engineering at ThoughtWorks, explains:

“There is a new way of thinking that is trying to avoid that “build it, and they will come mentality,” which leads to building it and nobody using it. One of the trends we are seeing is product thinking […]—using techniques like creating user personas of different types of users in the organization, conducting research to understand their needs, and talking with them.”

Breaking silos and fostering communication helps organizations build secure, scalable platforms that support the needs of developers, platform engineers, architects, and security teams.

Challenges of Integrating Security in Platform Engineering

While the benefits of integrating DevOps and security in platform engineering are clear, the journey has expected challenges. Our panel of experts highlighted several key obstacles that organizations may face:

Balancing Autonomy and Control

The primary goal is to empower developers to be more productive and proactive. However, this autonomy needs to be balanced with necessary security controls and governance measures.

Jess Mink points out the importance of achieving harmony between developer autonomy and operational control, stating:

“It’s a delicate balance - you want to make things easy for developers, but you also need to maintain the right level of control and security. It’s about finding the right abstractions and interfaces that give developers the freedom they need while still ensuring the platform remains secure and compliant.”

Driving Adoption and Changing Mindsets

Integrating security into the platform engineering workflow can often be met with resistance from developers accustomed to moving quickly and may view security as an obstacle to their productivity.

As Nariman, a Software Security Architect, notes:

“The challenge is not just about the tools or the technology - it’s about changing the mindset and getting people to understand the importance of security. You need to find ways to motivate developers and make them feel like they’re part of the solution rather than just imposing more rules and processes.”

Effective communication, education, and a focus on the business value of security are key to driving adoption and fostering a culture of shared responsibility.

Adapting to Evolving Needs and Technologies

As organizations grow and their technology stacks evolve, the demands on the platform engineering team can shift rapidly. Keeping up with these changes, while maintaining a secure and reliable platform, can be a significant challenge.

Lindsay Jack, VP of Engineering for the Platform Division at Snyk, explains:

“You might have a platform team that’s really good at a certain set of technologies, but then the organization starts moving in a new direction, and suddenly those skills don’t match up anymore. It’s about being agile and adaptable and making sure you have the right mix of skills and expertise to support the organization’s evolving needs.”

Fostering internal mobility, continuous learning, and a flexible, modular platform architecture can help platform engineering teams navigate these changes more effectively.

Measuring Success in Secure Platform Engineering

Measuring the success of a platform engineering initiative can be complex as it involves balancing a range of technical, operational, and business-oriented metrics but also considers human factors. According to our panel, some key metrics to include:

  • Developer Experience: Metrics such as developer satisfaction surveys, time-to-onboard new developers, and the number of self-service platform capabilities can provide insights into the effectiveness of the platform in supporting developer productivity and autonomy.
  • Reliability and Scalability: Monitoring service-level objectives (SLOs), incident response times, and the ability to handle increased traffic or user demands can help assess the platform’s reliability and scalability.
  • Security and Compliance: Tracking the number of security incidents, the ratio of security issues found during threat modeling versus post-deployment, and the adoption of security best practices can indicate the platform’s security posture.
  • Automation and Tooling: Metrics like the percentage of infrastructure provisioned through code, the frequency of platform updates, and the time saved through automation can demonstrate the platform’s operational efficiency.
  • Observability and Monitoring: Measuring the effectiveness of observability tools, the time to detect and resolve issues, and the quality of incident reports can provide insights into the platform’s overall health and performance.

As Jess Mink emphasizes, it’s important not just to collect these metrics but to use them to drive meaningful action and improvement:

“We look at all of those [metric categories] every quarter and write summaries that go up to the executive level. This creates visibility and a shared understanding of problems so that there’s room for movement and change in the right ways.”

The Future of Secure Platform Engineering

Software development and infrastructure management are evolving, and the role of platform engineering will only become more critical to support it. By integrating DevOps and security practices, platform engineering teams can create scalable, secure platforms that empower developers to be more productive and innovate, delivering business value.

Learn how Pulumi customers build secure, scalable platforms and empower their development teams:

Discover platform engineering best practices in The Guide to Platform Engineering: 7 Steps to Get It Right.

Build secure, scalable platforms with confidence—get started with the Platform Engineering Workshop Series & Course

Share this post

Platform Engineering Workshop Series - Register Now