Pulumi Blog: Secrets management

Secrets Management Tools: The Complete 2025 Guide

Asaf Ashirov — Thu, 24 Jul 2025 00:00:00 +0000

Every modern application depends on secrets to function: database passwords, API keys, certificates, and configuration values that enable secure communication between services. But here’s the challenge: as your infrastructure grows, managing these secrets becomes exponentially more complex.

The numbers tell a stark story. According to the Akeyless State of Secrets Management Report, 96% of organizations struggle with secrets sprawl such as credentials scattered across code repositories, configuration files, and deployment scripts. The consequences are severe: Verizon’s 2025 Data Breach Investigations Report found that 88% of data breaches involved compromised credentials, with IBM’s research showing the average breach now costs organizations $4.88 million.

The path forward requires more than just storing secrets securely, it demands intelligent orchestration across your entire infrastructure. But with dozens of solutions available, from traditional enterprise vaults to modern orchestration platforms, finding the right approach can feel overwhelming.

This guide cuts through the complexity. We’ll examine the leading secrets management tools across multiple categories, helping you understand not just what they do, but when and why to use them. Whether you’re architecting your startup’s first production environment or modernizing enterprise legacy systems, you’ll find actionable insights to make the right choice for your specific needs.

Secrets Management Tools Overview

The Evolution Beyond Simple Secret Storage

What started as simple password vaults has evolved into something far more sophisticated. Traditional secret storage addressed only part of the challenge: keeping credentials secure. But modern infrastructure demands orchestration across complex, distributed environments where applications need dozens of different secrets available across multiple cloud providers and deployment stages.

The fundamental shift happened when teams realized that managing secrets effectively requires treating them as infrastructure components, not just security artifacts. Your applications don’t just need database passwords stored securely—they need those credentials dynamically generated, automatically rotated, and consistently available across development, staging, and production environments without manual intervention.

Today’s leading platforms function as intelligent orchestrators rather than simple storage systems. They aggregate secrets from multiple existing sources, provide configuration-as-code capabilities that enable version control and reproducible deployments, and generate dynamic credentials with automatic expiration. This orchestration approach means you can work with your existing secret stores while gaining centralized management and modern capabilities like hierarchical organization that reduces duplication across environments.

The best solutions recognize that secrets management is fundamentally an infrastructure orchestration problem. They provide unified access patterns that work seamlessly across multi-cloud environments, comprehensive audit logging for compliance requirements, and fine-grained access controls that adapt to your organizational structure all while maintaining compatibility with the tools and workflows your teams already use.

Most Popular Secrets Management Tools

Secrets Orchestration Platforms

This represents the newest evolution in secrets management: platforms that connect and coordinate multiple secret sources rather than forcing you to abandon your existing infrastructure. Instead of requiring wholesale migration to yet another secret store, these intelligent brokers provide unified interfaces that work with your current tools while adding modern capabilities like dynamic credential generation and configuration-as-code workflows.

Pulumi ESC (Environments, Secrets, and Configuration)

Pulumi ESC takes a fundamentally different approach to secrets management. Rather than forcing you to migrate from your existing secret stores, it orchestrates them connecting HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, Google Secret Manager, and over 20 other providers through a unified interface. This means you can modernize your secrets workflow without abandoning your current infrastructure investments.

The platform’s strength lies in its configuration-as-code approach. You define hierarchical YAML environments that cascade from base configurations through development, staging, and production. This eliminates the manual configuration copying that leads to secrets sprawl: define common settings once, inherit them everywhere they’re needed, while maintaining appropriate security boundaries between environments.

Enterprise-Grade Security and Compliance: ESC provides comprehensive audit logging with detailed access trails, including user identity, timestamp, access method, and resource details. The platform’s audit capabilities support compliance requirements for frameworks like SOC 2, GDPR, and industry-specific regulations by providing immutable audit logs and real-time monitoring of secret access patterns.

Performance and Scalability: ESC delivers enterprise-scale performance with low-latency secret retrieval, high availability across multiple regions, and the ability to handle thousands of concurrent requests. The architecture is designed to scale automatically with organizational growth while maintaining consistent sub-second response times for secret operations.

Dynamic credential generation sets Pulumi ESC apart from traditional storage-focused tools. Instead of storing long-lived AWS access keys or Azure service principal secrets, it automatically provisions short-lived OIDC tokens on-demand. This dynamic approach eliminates entire categories of security risk by ensuring credentials expire automatically and can’t be misused over extended periods.

Advanced Key Management: ESC provides advanced key management capabilities with support for industry-standard encryption algorithms, automated key rotation, and secure key derivation. The platform offers envelope encryption and fine-grained access controls that ensure encryption keys are managed securely and in accordance with regulatory requirements.

The platform maintains a zero vendor lock-in philosophy through its open-source engine and comprehensive multi-language SDK support for TypeScript, Python, Go, and .NET. Teams can integrate ESC into existing workflows through familiar CLI tooling while maintaining full visibility into orchestration logic through transparent, auditable code.

Cost-Effective Enterprise Solution: Pulumi ESC offers significant cost advantages through its unified platform approach, reducing the need for multiple point solutions and simplifying operational overhead. ESC’s integration with existing infrastructure workflows eliminates the need for separate tools and reduces training costs, while its automated rotation and compliance features reduce manual administrative tasks and associated labor costs. For organizations with strict data residency requirements, Pulumi Service offers self-hosting options that provide additional cost predictability and security control.

Pulumi ESC offers a generous free tier with unlimited environments, making it accessible for teams to experiment with configuration-as-code principles. Usage-based pricing for advanced features scales naturally with organizational growth, making it particularly attractive for multi-cloud environments and teams prioritizing systematic approaches to eliminating secrets sprawl.

Doppler

Doppler focuses on developer experience with an intuitive interface and extensive integrations for modern development workflows. The platform provides branch-based environment management and real-time secret synchronization across connected services, with plans starting at $3 per user per month.

Infisical

Infisical combines open-source transparency with modern secrets management, offering both self-hosted and cloud options. The platform includes built-in secrets scanning capabilities and comprehensive SDK support, with cloud service starting at $8 per user per month.

Enterprise Secrets Vaults

Traditional enterprise vaults focus on maximum security and compliance, offering comprehensive secret storage with advanced features designed for large organizations with sophisticated security requirements.

HashiCorp Vault

HashiCorp Vault remains the established standard for enterprise secrets management, offering unmatched flexibility and a comprehensive feature set for complex environments. The platform provides dynamic secrets generation with automatic expiration for over 50 different systems, enabling organizations to move away from static, long-lived credentials toward more secure, just-in-time access patterns.

Vault’s architecture supports over 100 authentication methods and secret engines, providing flexibility to integrate with virtually any existing infrastructure or identity system. Multi-region clustering with strong consistency guarantees ensures high availability and data integrity across geographically distributed deployments. Extensive audit logging and compliance reporting capabilities help organizations meet regulatory requirements while maintaining operational visibility.

The platform’s cloud-agnostic deployment options support both Kubernetes and traditional VM-based infrastructure, making it suitable for organizations with diverse technical environments. Vault’s open-source core provides transparency and community-driven development, while commercial enterprise features starting at $2 per node per hour add advanced management, replication, and support capabilities.

HashiCorp Vault excels in organizations requiring maximum flexibility and customization, complex multi-cloud environments, and teams with strong DevOps expertise capable of managing sophisticated infrastructure platforms.

CyberArk Conjur

CyberArk Conjur focuses on enterprise security with advanced compliance features and integration with broader privileged access management systems. The platform provides enterprise-grade role-based access control with comprehensive audit capabilities, enabling organizations to implement fine-grained security policies across their entire infrastructure.

Modern versions of Conjur incorporate AI-powered anomaly detection and threat analysis, helping security teams identify unusual access patterns that may indicate compromise or policy violations. Automated compliance reporting supports SOC 2, PCI-DSS, and other regulatory frameworks, reducing the operational overhead of maintaining compliance documentation.

Conjur’s integration with CyberArk’s broader privileged access management suite provides a comprehensive approach to credential security that extends beyond simple secret storage to include session monitoring, just-in-time access provisioning, and threat detection. The platform’s advanced policy engine enables fine-grained access controls that can adapt to changing organizational and regulatory requirements.

With enterprise licensing based on scale and deployment requirements, CyberArk Conjur targets regulated industries, enterprises with strict compliance requirements, and organizations prioritizing maximum security over operational simplicity.

Akeyless

Akeyless offers a modern, cloud-native approach to enterprise secrets management without the operational complexity of traditional self-hosted solutions. The platform’s SaaS-first architecture eliminates infrastructure management overhead while providing enterprise-grade capabilities including just-in-time access and automated credential rotation.

The platform implements zero-knowledge encryption with client-side key management, ensuring that even Akeyless cannot access customer secrets. This approach addresses concerns about cloud-based secret storage while maintaining the operational benefits of a managed service. A comprehensive API and integration ecosystem enables connection with existing tools and workflows.

Akeyless provides a generous free tier supporting up to 5 clients, with usage-based pricing that scales naturally with organizational growth. This pricing model, combined with the elimination of operational overhead, makes Akeyless attractive for organizations seeking enterprise features without the complexity of self-managed infrastructure, particularly cloud-first teams prioritizing rapid deployment and scaling.

Cloud-Native Secrets Managers

Cloud-native solutions provide tight integration with specific cloud platforms, offering optimized experiences for organizations committed to particular cloud ecosystems. These platforms leverage native cloud services and identity systems to provide seamless integration with existing infrastructure.

AWS Secrets Manager

Amazon’s native secrets management service provides deep integration with the AWS ecosystem and automatic rotation capabilities specifically designed for AWS services. The platform offers native integration with over 50 AWS services including RDS, Lambda, ECS, and EKS, enabling seamless credential management across the entire AWS service portfolio.

Automatic rotation capabilities for supported databases and AWS services eliminate the operational overhead of manual credential management while improving security through regular rotation cycles. Cross-region replication with automatic failover ensures high availability for critical secrets, while fine-grained IAM permissions with resource-based policies provide precise access control aligned with AWS security best practices.

VPC endpoint support enables private network access without internet routing, addressing security requirements for air-gapped or highly secure environments. The service’s pricing model of $0.40 per secret per month plus $0.05 per 10,000 API calls provides predictable costs that scale with usage.

AWS Secrets Manager excels for AWS-native organizations, applications requiring automatic credential rotation, and teams leveraging extensive AWS service portfolios where tight integration provides operational efficiency and security benefits.

Azure Key Vault

Microsoft’s comprehensive platform manages secrets, keys, and certificates within the Azure ecosystem while providing strong compliance features particularly valued by government and enterprise customers. The platform integrates seamlessly with Azure services and Active Directory authentication, leveraging existing identity infrastructure to provide consistent access control across the Azure ecosystem.

FIPS 140-2 Level 2 validated Hardware Security Modules provide the highest levels of cryptographic security, making Azure Key Vault suitable for government contractors and organizations with stringent security requirements. Managed HSM support extends these capabilities for scenarios requiring maximum security control and audit capabilities.

Certificate lifecycle management with automatic renewal reduces operational overhead while ensuring consistent security posture across web applications and services. Virtual network integration and private endpoint support enable secure access patterns that align with enterprise network security architectures.

Azure Key Vault’s standard tier pricing of $0.03 per 10,000 operations provides cost-effective secret management for Azure-centric organizations, government contractors requiring FIPS compliance, and enterprises with Microsoft-heavy infrastructure environments.

Google Secret Manager

Google Cloud’s scalable secrets management service is optimized for high-volume, global deployments with strong integration across GCP services. The platform provides automatic global replication across all GCP regions, ensuring low-latency access to secrets regardless of geographic distribution.

IAM integration supports fine-grained, condition-based permissions that can adapt to complex organizational structures and security requirements. Secret versioning with automatic rollback capabilities provides operational safety for configuration changes, while integration with Cloud Build, GKE, Cloud Run, and App Engine enables seamless secret management across Google Cloud’s entire service portfolio.

The platform’s high-performance API with global edge caching minimizes latency for high-frequency secret retrieval operations, making it suitable for applications with demanding performance requirements. Pricing of $0.06 per secret version per month plus $0.03 per 10,000 API calls provides cost-effective scaling for high-volume usage patterns.

Google Secret Manager serves Google Cloud-native organizations, applications requiring massive scale, and global deployments where low latency and high availability are critical operational requirements.

Developer-Focused Tools

These tools prioritize user experience and ease of adoption, often serving as excellent entry points for teams beginning their secrets management journey. They emphasize intuitive interfaces and efficient workflows that reduce barriers to adoption.

1Password Secrets Automation

1Password has successfully expanded beyond personal password management to offer comprehensive business secrets management while maintaining their renowned user experience. The platform implements zero-knowledge architecture with client-side encryption, ensuring that even 1Password cannot access customer secrets while providing the usability that made their consumer products successful.

The intuitive interface remains accessible to both technical and non-technical users, addressing the common challenge of secrets management tools that require specialized expertise. Comprehensive SDK and CLI support enables developer workflows while maintaining the simplicity that drives organization-wide adoption. Integration with popular CI/CD platforms and development tools provides the automation capabilities modern development teams require.

Advanced sharing capabilities include granular, time-limited permissions that enable secure collaboration while maintaining audit trails and access control. This combination of security and usability makes 1Password particularly effective in mixed environments where both technical and business users need access to secrets.

At $8 per user per month with volume discounts available, 1Password Secrets Automation targets teams prioritizing usability over advanced features, mixed technical/non-technical environments, and organizations already invested in the 1Password ecosystem.

Bitwarden Secrets Manager

Bitwarden leverages their password management expertise to provide cost-effective secrets management specifically designed for development teams. The platform includes machine account support for automated CI/CD workflows, addressing the common requirement for non-human access to secrets in modern development processes.

CLI and SDK integration covers major programming languages while maintaining the straightforward approach that characterizes Bitwarden’s products. Comprehensive event logging and audit trails provide the visibility required for security and compliance, while two-person integrity controls add an extra layer of protection for sensitive operations.

The platform’s competitive per-user pricing of $3 per month with transparent costs makes it particularly attractive for small to medium teams, organizations seeking simple and cost-effective solutions, and existing Bitwarden users looking to extend their investment into development workflows.

Application Security & Scanning

These tools focus on preventing secrets leakage by detecting credentials in code repositories, CI/CD pipelines, and other development artifacts. Rather than managing secrets after they’re created, these platforms prevent security incidents by identifying and remediating credential exposure.

GitGuardian

GitGuardian provides comprehensive secrets detection and remediation across the entire software development lifecycle with high accuracy and low false positives. The platform offers real-time scanning of commits, pull requests, and issues across all major Git platforms, ensuring that credential exposure is identified and addressed before code reaches production environments.

Advanced machine learning algorithms detect over 350 secret types while minimizing false positives that can lead to alert fatigue. Developer-friendly remediation workflows provide guided resolution steps that help development teams address issues quickly without disrupting their normal workflows. Historical repository scanning capabilities enable comprehensive security assessment of existing codebases.

Integration with GitHub, GitLab, Bitbucket, and Azure DevOps ensures coverage across diverse development environments. The platform’s free tier supports small teams, while enterprise plans starting at $18 per developer per month provide advanced features and support for larger organizations.

GitGuardian excels in organizations with large codebases, teams implementing DevSecOps practices, and compliance-focused environments where preventing credential exposure is critical to maintaining security posture.

TruffleHog

TruffleHog offers powerful open-source secrets detection with both community and enterprise versions for comprehensive credential scanning. The platform uses high-performance scanning algorithms that combine advanced entropy analysis with pattern matching to identify credentials across diverse file types and encoding formats.

Support for over 700 credential types includes custom patterns that can adapt to organization-specific secret formats. CI/CD integration with GitHub Actions, GitLab CI, and Jenkins enables automated scanning as part of existing development workflows. Historical git repository scanning provides commit-level analysis that can identify when and how credentials entered the codebase.

The active open-source community ensures regular updates and improvements while providing transparency into detection algorithms. Enterprise features and support options are available for organizations requiring commercial backing and advanced capabilities.

TruffleHog serves open-source advocates, security-conscious development teams, and organizations with strong technical security expertise capable of implementing and maintaining open-source security tools.

Specialized & Integration Tools

These tools address specific use cases or provide bridge functionality between secrets managers and deployment platforms, filling gaps in comprehensive secrets management architectures.

External Secrets Operator

The External Secrets Operator provides Kubernetes-native integration with external secrets management systems, enabling GitOps workflows with secure secret handling. The platform uses Kubernetes Custom Resource Definitions to provide native resource management that integrates seamlessly with existing Kubernetes operational patterns.

Multi-provider support includes Vault, AWS, Azure, GCP, and numerous other secret sources, enabling organizations to maintain consistent Kubernetes secret management regardless of their underlying secret storage choices. Automatic secret synchronization with configurable refresh intervals ensures that Kubernetes secrets remain current with external sources while minimizing API load.

Template-based secret transformation and formatting capabilities enable adaptation of external secret formats to Kubernetes requirements. As an active CNCF community project with growing enterprise adoption, the External Secrets Operator benefits from diverse contributions and broad compatibility testing.

This free and open-source tool excels in Kubernetes-heavy environments, teams adopting GitOps practices, and organizations requiring cloud-agnostic secret synchronization across diverse infrastructure platforms.

Berglas

Google’s Berglas provides optimized secrets injection for serverless and container environments, particularly Google Cloud Run and similar platforms. The platform’s serverless-optimized secret injection minimizes cold start impact, addressing one of the key performance challenges in serverless architectures.

Deep integration with Google Cloud services and IAM leverages existing cloud infrastructure to provide secure, efficient secret access. Container-native deployment patterns use init container approaches that work seamlessly with existing containerized application architectures. Minimal runtime overhead and memory footprint ensure that secret management doesn’t impact application performance.

As an open-source project backed by Google’s engineering team, Berglas benefits from direct integration with Google Cloud services and ongoing development aligned with Google Cloud platform evolution. The free and open-source nature makes it accessible for teams of all sizes.

Berglas serves Google Cloud serverless deployments, container-native applications, and teams prioritizing minimal overhead in performance-sensitive environments.

Confidant

Lyft’s open-source Confidant offers a production-proven approach to secrets management with focus on developer experience and AWS optimization. Battle-tested at Lyft’s scale with millions of requests daily, Confidant provides real-world validation of its architecture and performance characteristics.

AWS-optimized architecture leverages KMS and DynamoDB to provide secure, scalable secret storage using proven AWS services. Developer-centric design emphasizes simple API and CLI interfaces that reduce friction in day-to-day development workflows. The blind credentials feature provides enhanced security by ensuring that even system administrators cannot access certain types of sensitive information.

IAM integration with fine-grained permissions leverages AWS’s native access control systems to provide consistent security policies across the entire infrastructure. As a free and open-source solution, Confidant enables organizations to implement enterprise-grade secret management without licensing costs.

Confidant excels in AWS-heavy environments, teams with strong technical capabilities, and organizations seeking proven open-source solutions with demonstrated scale and reliability.

Chamber

Segment’s Chamber provides elegant secrets management using AWS Parameter Store, optimizing for simplicity and cost-effectiveness. The platform leverages existing AWS infrastructure to minimize complexity while providing essential secret management capabilities.

Cost-effective operations use Parameter Store’s generous free tier to provide secret storage without additional infrastructure or licensing costs. Simple deployment requires minimal operational overhead, making it suitable for teams with limited DevOps resources. Version control integration supports infrastructure as code practices by enabling secret management through familiar development workflows.

Namespace organization with hierarchical parameter structure provides logical organization that scales with application complexity while maintaining simplicity of operation. As a free and open-source tool with only AWS Parameter Store costs, Chamber provides extremely cost-effective secret management.

Chamber serves AWS-native deployments seeking simplicity, teams with cost constraints, and organizations preferring minimal tooling complexity over advanced feature sets.

Top Features to Look for in Secrets Management Tools

Secrets Orchestration and Multi-Source Integration

The most advanced secrets management platforms function as intelligent orchestrators, connecting multiple secret sources and providing unified access patterns. This capability is crucial for organizations with heterogeneous environments where secrets are distributed across multiple systems. Rather than forcing migration to a single secret store, orchestration platforms work with existing infrastructure investments while providing centralized management and consistent access patterns.

Effective orchestration platforms can aggregate secrets from existing enterprise vaults like HashiCorp Vault and CyberArk, cloud-native services including AWS Secrets Manager, Azure Key Vault, and Google Secret Manager, legacy systems with database configuration and file-based secrets, and third-party services like 1Password, Bitwarden, and specialized tools. Pulumi ESC exemplifies this orchestration approach, providing a unified interface across over 20 secret sources while maintaining compatibility with existing infrastructure investments.

This orchestration capability reduces migration risk and preserves existing investments while enabling gradual adoption of modern secret management practices. Organizations can maintain their current secret stores while gaining the benefits of centralized management, consistent access patterns, and advanced features like dynamic credential generation.

Configuration as Code and Environment Management

Modern secrets management extends beyond simple credential storage to comprehensive configuration management. The best platforms support configuration-as-code principles that bring software engineering discipline to environment management and secret distribution.

Version control integration with full change tracking and rollback capabilities ensures that configuration changes undergo the same scrutiny as application code. Hierarchical organization reduces duplication through inheritance and composition, enabling teams to define common configurations once and apply them consistently across multiple environments. Reproducible deployments ensure consistency across environments while code review processes apply software engineering practices to configuration changes. Automated testing capabilities validate configuration changes before deployment, preventing configuration errors that could lead to security vulnerabilities or application failures.

This approach transforms secrets from scattered, manual processes into systematic, auditable infrastructure that follows established software engineering best practices. Teams can apply familiar development workflows to secret management, improving both security and operational efficiency.

Dynamic Credential Generation

Static secrets pose inherent security risks due to their long-lived nature and potential for unauthorized sharing. Dynamic credential generation addresses these challenges by creating credentials on-demand with automatic expiration, significantly reducing the attack surface associated with credential compromise.

Effective dynamic credential systems provide short-lived tokens with automatic expiration and renewal, eliminating the need for manual rotation while ensuring credentials cannot be misused over extended periods. Just-in-time access generates credentials only when needed, reducing the window of opportunity for credential misuse. OIDC integration leverages identity providers for cloud access, enabling seamless authentication without storing long-lived cloud credentials.

Database credentials with automatic rotation for major database systems ensure that application access remains current while removing the operational burden of manual credential management. API keys with configurable lifespans and scope limitations provide fine-grained control over third-party service access. This dynamic approach significantly improves security posture while reducing operational overhead.

Multi-Platform Integration and SDK Support

Your secrets management platform should integrate seamlessly with your existing development and deployment infrastructure, providing native support for the tools and workflows your teams already use. Effective integration reduces friction in adoption while ensuring that secret management becomes a natural part of existing processes rather than an additional burden.

Development integration should include CI/CD platforms like GitHub Actions, GitLab CI, Jenkins, and Azure DevOps, enabling automated secret management as part of deployment workflows. Container orchestration platforms including Kubernetes, Docker Swarm, and Nomad should provide native secret injection capabilities. Infrastructure as Code tools like Terraform, Pulumi, and CloudFormation should support secret management through familiar provisioning workflows. Local development integration should include environment synchronization and IDE integration to provide consistent experiences across development and production environments.

Runtime integration capabilities should extend to application frameworks like Spring Boot, Django, and Express.js, enabling developers to access secrets through familiar programming patterns. Cloud platforms including AWS Lambda, Azure Functions, and Google Cloud Run should provide optimized secret injection with minimal performance impact. Monitoring tools, service meshes, and other infrastructure components should integrate seamlessly to provide comprehensive observability and management capabilities.

Comprehensive Audit and Compliance Capabilities

Security and compliance requirements demand detailed visibility into secret access and management activities. Effective audit capabilities provide the transparency required for security monitoring, compliance reporting, and incident investigation while enabling organizations to demonstrate adherence to regulatory requirements.

Complete audit trails with detailed attribution, timestamps, and context enable security teams to understand exactly who accessed which secrets when and from where. Change tracking for all secret modifications, rotations, and access grants provides visibility into the complete lifecycle of credential management. Compliance reporting supporting SOC 2, ISO 27001, PCI-DSS, and other regulatory frameworks automates the documentation required for compliance audits.

Anomaly detection capabilities identify unusual access patterns that may indicate compromise or policy violations, enabling proactive security response. Integration with SIEM systems provides centralized security monitoring and alerting, ensuring that secret management activities are included in overall security operations. These capabilities transform secret management from a potential compliance liability into a security asset that enhances overall organizational security posture.

Performance and Scalability

Consider both current requirements and future growth when evaluating secrets management platforms. API performance with low latency for high-frequency secret retrieval ensures that secret management doesn’t become a bottleneck in application performance. Geographic distribution supporting global deployments with regional access provides consistent performance regardless of deployment location.

High availability with automatic failover and disaster recovery ensures that secret management doesn’t become a single point of failure in critical applications. Horizontal scaling capabilities handle increased load without performance degradation as organizations grow. Intelligent caching strategies optimize performance while maintaining security, reducing API load while ensuring secrets remain current.

Effective performance and scalability ensure that secret management enhances rather than hinders application performance, enabling organizations to scale their security practices alongside their infrastructure growth.

Try secrets orchestration

Pulumi ESC connects your existing secret stores behind a single interface, adding configuration as code and dynamic, short-lived credentials.

Get started

Key Considerations for Choosing Secrets Management Tools

Orchestration vs. Replacement Strategy

The fundamental choice between orchestration and replacement approaches significantly impacts both implementation complexity and long-term flexibility. Orchestration platforms like Pulumi ESC work with existing infrastructure, providing unified interfaces without requiring complete migration. This approach offers reduced migration risk with gradual adoption, preserved investments in existing secret stores and processes, flexibility to use best-of-breed solutions for different use cases, and simplified operations through centralized management of distributed secrets.

Replacement approaches require migration of all secrets to a single platform. While potentially simpler architecturally, this approach involves higher migration costs and extended transition periods, vendor lock-in that makes future changes more difficult, all-or-nothing adoption requiring complete workflow changes, and potential loss of functionality from specialized existing systems.

Organizations with heterogeneous environments, multiple cloud providers, or significant existing investments in secret management infrastructure typically benefit from orchestration approaches that preserve flexibility while providing centralized management capabilities.

Cloud Strategy Alignment

Your choice of secrets management platform should align with your overall cloud strategy and deployment patterns. Organizations with multi-cloud or hybrid strategies benefit from cloud-agnostic solutions that provide consistent experiences across platforms. Pulumi ESC, HashiCorp Vault, and other orchestration platforms excel in these environments by providing unified interfaces regardless of underlying infrastructure.

Organizations committed to specific cloud platforms can leverage native services like AWS Secrets Manager, Azure Key Vault, and Google Secret Manager for optimal integration and cost efficiency within their chosen ecosystem. These platforms provide deep integration with cloud-native services while optimizing costs through native billing and resource management.

Migration flexibility becomes important as organizational requirements evolve. Orchestration platforms provide natural migration paths between different underlying secret stores as requirements change, while cloud-native solutions may require more significant changes if cloud strategy evolves. Consider solutions that maintain flexibility for future changes while meeting current operational requirements.

Team Structure and Expertise

The technical complexity and operational requirements of different secrets management platforms vary significantly, making team structure and expertise critical factors in platform selection. High-complexity tools like HashiCorp Vault and CyberArk Conjur suit organizations with dedicated security or platform teams capable of managing sophisticated infrastructure. These platforms offer maximum flexibility and features but require significant expertise to implement and operate effectively.

Medium-complexity tools like Pulumi ESC and Doppler balance advanced features with operational simplicity, making them suitable for organizations with strong technical teams that prefer managed services or simplified operations. These platforms provide enterprise-grade capabilities without the operational overhead of self-managed infrastructure.

Low-complexity tools like 1Password and Bitwarden prioritize ease of use over advanced capabilities, making them suitable for organizations with limited technical resources or mixed teams where non-technical users require access to secrets. These platforms excel in environments where adoption and usability take priority over advanced features.

Organizational structure also influences platform choice. Centralized teams can manage sophisticated platforms and enforce consistent policies across the organization, while distributed teams benefit from self-service capabilities and intuitive interfaces that enable independent operation. Mixed environments require tools accessible to both technical and non-technical users, emphasizing usability and clear documentation.

Security and Compliance Requirements

Regulatory and security requirements significantly influence platform selection, with different industries and use cases demanding specific capabilities and certifications. Government and defense organizations may require FIPS 140-2 Level 2 validation provided by platforms like Azure Key Vault and CyberArk, while financial services often need comprehensive audit trails and segregation of duties capabilities.

Healthcare organizations require HIPAA-compliant secret handling and access controls, while European operations must consider GDPR implications for secret storage and processing. Understanding these requirements early in the selection process helps narrow platform choices and avoid costly compliance gaps.

Security model preferences also vary significantly between organizations. Zero-knowledge encryption models where service providers cannot access secrets appeal to security-conscious organizations, while Hardware Security Module support provides the highest levels of cryptographic security for sensitive applications. Network isolation capabilities become important for air-gapped or highly secure environments, while multi-factor authentication integration with existing identity systems ensures consistent security policies.

Cost Optimization

Understanding the total cost of ownership for different secrets management approaches requires consideration of both direct and indirect costs. Per-user pricing models suit teams with predictable user counts but can become expensive as organizations scale. Usage-based pricing scales naturally with actual consumption but requires careful monitoring to avoid unexpected costs. Per-secret pricing works well for applications with many users but few secrets, while enterprise licensing may provide cost-effective solutions for large deployments.

Total cost of ownership extends beyond licensing to include operational overhead for self-hosted versus managed solutions, training and certification requirements for specialized platforms, integration costs including development time and ongoing maintenance, and migration expenses from existing solutions. Managed services may appear more expensive initially but often provide better total cost of ownership when operational overhead is considered.

Consider both current costs and future scaling requirements when evaluating pricing models. Platforms that provide cost-effective entry points while supporting growth can provide better long-term value than solutions optimized only for current requirements.

The Future of Secrets Management: Orchestration and Configuration as Code

The secrets management landscape is undergoing a fundamental shift from simple storage systems to sophisticated orchestration platforms. This evolution reflects the reality of modern infrastructure—applications span multiple clouds, integrate with dozens of services, and require dynamic adaptation to changing requirements.

From Storage to Orchestration

Traditional secret management models rely on centralized vaults where all secrets are stored in a single system, requiring applications to understand specific retrieval mechanisms and access patterns. This approach often creates vendor lock-in and forces organizations to migrate from existing secret stores, disrupting established workflows and creating migration risk.

The emerging orchestration model uses intelligent brokers that aggregate secrets from multiple sources—existing vaults, cloud services, legacy systems—presenting them through unified, consistent interfaces. This orchestration approach provides investment protection by working with existing secret stores, gradual migration without disruptive all-or-nothing transitions, best-of-breed integration leveraging specialized solutions where appropriate, and operational simplification through centralized management of distributed secrets.

Orchestration platforms enable organizations to maintain their current infrastructure investments while gaining the benefits of centralized management, consistent access patterns, and modern capabilities like dynamic credential generation. This approach reduces both risk and cost while providing a clear path toward modern secret management practices.

Configuration as Code Revolution

The convergence of secrets and configuration management represents the next major evolution in infrastructure management. Modern applications require more than isolated credentials—they need comprehensive environment configuration including secrets, feature flags, service endpoints, and deployment parameters. Managing these elements separately creates complexity and increases the risk of configuration drift between environments.

Configuration-as-code principles bring software engineering discipline to environment management by providing version control with complete change history and rollback capabilities, code review processes that ensure changes undergo appropriate scrutiny, automated testing that validates configuration changes before deployment, and reproducible environments that eliminate configuration drift between stages.

Pulumi ESC pioneered this approach, demonstrating how configuration-as-code can systematically address secrets sprawl while improving security, reliability, and developer productivity. This convergence transforms environment management from manual, error-prone processes into systematic, auditable infrastructure that follows established software engineering best practices.

Dynamic Credentials and Zero-Trust Architecture

The shift toward dynamic, short-lived credentials aligns with zero-trust security principles that assume compromise and limit blast radius through minimal access grants. Rather than managing long-lived secrets that require careful rotation and access control, modern platforms generate credentials on-demand with automatic expiration.

This approach provides reduced attack surface through minimal credential lifetime, simplified rotation with automatic credential lifecycle management, enhanced audit trails with precise attribution and timing, and improved compliance through automated policy enforcement. Dynamic credentials eliminate many of the operational challenges associated with traditional secret management while significantly improving security posture.

Organizations adopting zero-trust architectures benefit from secret management platforms that support dynamic credential generation, fine-grained access controls, and comprehensive audit capabilities. These platforms become integral components of zero-trust implementations rather than separate security tools.

AI and Machine Learning Integration

Advanced platforms are incorporating AI and machine learning capabilities to enhance security and operational efficiency. Anomaly detection algorithms identify unusual access patterns that may indicate compromise or policy violations, enabling proactive security response before incidents escalate. Intelligent rotation systems optimize credential lifecycles based on usage patterns and risk assessment, balancing security requirements with operational efficiency.

Automated compliance capabilities ensure policies are consistently applied across all environments while predictive security features identify potential vulnerabilities before they’re exploited. These AI-powered capabilities transform secret management from reactive operations into proactive security systems that enhance overall organizational security posture.

Choosing the Right Path Forward

The secrets management landscape has fundamentally shifted from simple storage to intelligent orchestration. Your choice today will determine not just your current security posture, but your ability to scale and adapt as your infrastructure grows more complex.

The decision framework is clearer than you might expect. If you’re managing multi-cloud environments or have significant existing secret store investments, orchestration platforms like Pulumi ESC provide the most strategic value. They let you modernize your approach without disrupting current workflows, while adding configuration-as-code capabilities and hierarchical environment management that systematically address secrets sprawl.

Organizations committed to maximum flexibility and customization will find HashiCorp Vault’s extensibility unmatched, though it requires substantial technical expertise to operate effectively. Cloud-native teams can leverage AWS Secrets Manager, Azure Key Vault, or Google Secret Manager for seamless integration within their chosen ecosystem, optimizing both cost and operational complexity.

Teams prioritizing rapid adoption and developer experience should consider Doppler, Infisical, or 1Password Secrets Automation. These platforms excel at driving organization-wide adoption through intuitive interfaces while providing essential capabilities without overwhelming complexity.

Here’s the critical insight: secrets management isn’t just a security decision—it directly impacts your development velocity and operational efficiency. Platforms that embrace orchestration, configuration-as-code, and dynamic credentials enable faster development cycles while reducing the operational overhead that comes from managing scattered, manual processes.

The cost of maintaining ad-hoc secrets management extends far beyond security incidents. Developer productivity losses from hunting down credentials, operational complexity from manual rotation processes, and the constant risk of configuration drift between environments create technical debt that compounds over time. The investment in systematic solutions pays dividends immediately through reduced operational overhead and improved security posture.

The industry’s evolution toward zero-trust architectures and configuration-as-code practices isn’t slowing down. Organizations that adopt systematic approaches today position themselves to handle tomorrow’s infrastructure complexity with confidence, while those that delay face increasingly difficult migration challenges as their systems grow more distributed and interdependent.

Ready to revolutionize your secrets and configuration management? Explore Pulumi ESC and discover how secrets orchestration can transform your development workflows while eliminating sprawl across your entire infrastructure.

Why Every Cloud Engineer Needs Pulumi ESC for Secrets Management

Sara Huddleston — Mon, 17 Mar 2025 07:28:56 +0000

Managing secrets is one of the most critical responsibilities in cloud engineering. Secrets like API keys, database credentials, and encryption tokens are the backbone of secure and seamless cloud operations. Yet they are so often an afterthought. They get replicated across cloud-specific secrets managers and stuffed in GitHub secrets, compromising security for the sake of simplicity. ¿Por que no los dos? Why can’t secrets management be secure and simple?

Enter Pulumi ESC (Environments, Secrets, and Configuration)—a breakthrough in taming secrets sprawl and streamlining configuration management across infrastructure. Let’s explore why Pulumi ESC is a necessity for cloud engineers, helping make secrets management secure while keeping it simple.

The Challenge of Secrets Management in Modern Cloud Environments

Secrets management has changed significantly over the past decade. Gone are the days when secrets could be hardcoded, manually maintained in static configuration files, or left floating around in plaintext on dev machines. Nowadays, cloud environments demand a higher level of sophistication:

Distributed Systems: Even in multi-cloud and hybrid setups, secrets must be accessible across varied platforms without exposing vulnerabilities.
Dynamic Infrastructure: Kubernetes, serverless architectures, and ephemeral environments must have secrets that adapt dynamically.
Security Risks: Hardcoded or poorly managed secrets can lead to catastrophic breaches, costing companies millions in data recovery and compliance penalties.
Operational Burdens: Manual secrets management is error-prone, inefficient, unscalable, and needs to align with DevOps and DevSecOps best practices.

Pulumi ESC addresses these issues head-on, redefining how cloud engineers manage secrets efficiently, securely, and at scale.

Jk Jensen, Team Lead at Mysten Labs, said:

“Pulumi ESC has been a lifesaver for us. It’s nice to throw everything behind an ESC environment and eliminate one-off granting/IAM permissions and other issues related to static credentials. It gives us peace of mind knowing that we can grant permissions quickly and revoke easily limiting blast radius for any access.”

What is Pulumi ESC?

Pulumi ESC simplifies how organizations manage secrets and configurations. It is designed to secure sensitive configurations across modern cloud environments and supports seamless integration, enabling engineers to:

Access, share, and manage secrets, passwords, API keys, and configuration settings like network and deployment options.
Synchronize secrets and configuration from any secrets manager to any app, tool, or CI/CD platform.
Access secrets securely through CLI, API, Kubernetes operator, the Pulumi Cloud UI, and in-code with Typescript/Javascript, Python, and Go SDKs.
Connect to cloud providers and secrets stores via OIDC to generate dynamic, short-lived, auto-expiring credentials.
Set role-based access controls (RBAC), making securing secrets and configurations easy by assigning permissions to users based on their roles.

Whether integrated with Pulumi’s Infrastructure as Code (IaC) platform or used as a standalone service, Pulumi ESC enables cloud engineers to streamline secrets management with centralized control.

Key Features of Pulumi ESC

1. Seamless Integration with External Platforms

Pulumi ESC integrates with popular secrets providers, including AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, 1Password, and HashiCorp Vault, making it adaptable for multi-cloud and hybrid cloud architectures.

2. Dynamic Secrets Synchronization with ESO

Partnering with the External Secrets Operator, Pulumi ESC synchronizes secrets securely into Kubernetes clusters. This eliminates hardcoding secrets into manifests or relying on unsecured manual processes.

3. Safely Roll Back to a Previous Version

Pulumi ESC Versioning gives you unprecedented control over your secrets and configuration. Every change is captured in an immutable revision history, allowing you to audit modifications, compare versions, and safely roll back.

4. Secure by Design

Pulumi ESC follows a “secure by default” model, employing encryption, access control, and detailed audit trails. Engineers can meet compliance regulations effortlessly while gaining full visibility into secret access patterns.

5. Automated Rotation and Expiry

Pulumi ESC minimizes security risks by automating the rotation of secrets. This feature aligns secrets management with CI/CD processes for cloud engineers focused on DevOps, ensuring credentials remain valid only when needed.

6. Configuration-as-Code, Automation, and Integration Everywhere

Pulumi ESC embraces an “as-code” approach, enabling configuration and secrets management using TypeScript, JavaScript, Go, Python, or YAML. The ’esc’ CLI and API support automation in CI/CD environments, reducing credential duplication and ensuring a single source of truth.

7. Dev Tools Integrations

Pulumi ESC’s metadata and support for popular configuration formats enable seamless integration with tools like Direnv, Docker, and GitHub, allowing easy management of environment variables, secrets, and configurations.

8. Infrastructure Tools Integrations

Pulumi ESC extends its capabilities beyond Pulumi IaC by integrating with other infrastructure tools such as Cloudflare, Terraform, and OpenTofu. These integrations enable seamless provisioning of cloud credentials and input variables directly from ESC environments.

Why Cloud Engineers Need Pulumi ESC

Enhanced Security and Compliance

Cloud engineers often have to balance security and operations. Pulumi ESC centralizes secrets in a secure vault, ensuring encrypted storage, access control, and audit visibility. It also complies with industry standards like SOC 2, PCI-DSS, and MTCS.

Streamlined Multi-Cloud Management

Managing secrets across AWS, Azure, and GCP can quickly become chaotic. Pulumi ESC’s cross-platform integration simplifies this process, ensuring engineers can manage and sync secrets from a single interface.

Simplified Kubernetes Secrets

Kubernetes’ default secrets offer limited security and scalability. Pulumi ESC with ESO and Secrets Store CSI Driver overcomes these limitations by securely synchronizing and managing secrets within Kubernetes clusters. This prevents unauthorized access and provides automated updates.

Zero Downtime Through Automation

Manual secrets management often leads to errors such as expired credentials or outdated tokens. Pulumi ESC automates the entire lifecycle of secrets—creation, rotation, replication, and expiry—guaranteeing uninterrupted services.

Developer-Friendly Workflows

Tools should make engineers’ lives easier, not harder. Pulumi ESC’s CLI, SDKs, and API provide intuitive ways to integrate into existing workflows. For cloud engineers leveraging Infrastructure as Code with Pulumi, managing secrets alongside the stack becomes effortless.

Thomas Meckel, Senior Cloud Architect at Ophios GmbH, explains:

“Pulumi’s built-in encryption of secrets sets it apart as a leader in secure infrastructure management. Unlike other tools that leave sensitive data exposed or require complex configurations, Pulumi ensures secrets are encrypted by default, leveraging flexible options like Azure Key Vault or AWS KMS. This eliminates the risk of plaintext exposure and simplifies secure deployments. For any organization serious about cloud security, Pulumi’s approach to secrets management is a critical differentiator, combining ease of use with robust protection against breaches.”

Kubernetes Secrets Management

For secrets management in Kubernetes environments, Pulumi ESC becomes even more powerful when paired with External Secrets Operator (ESO) or the Secrets Store CSI Driver. Here’s how it works:

Using Pulumi ESC with External Secrets Operator

External Secrets Operator is an open-source Kubernetes operator that syncs secrets from external providers (like Pulumi ESC) into Kubernetes as native secrets.

Centralized Storage: Store secrets securely in Pulumi ESC and synchronize them with Kubernetes across multiple clusters.
Dynamic Updates: Whenever a secret is updated in ESC, ESO automatically replicates the changes into Kubernetes, eliminating manual intervention.
Granular Management: Define which secrets are synchronized and to which namespaces, ensuring tight access controls and minimizing risk.
Automated Secret Rotation: By leveraging ESC’s rotation capabilities, ESO ensures Kubernetes receives refreshed credentials without requiring downtime.

Learn how to manage Kubernetes secrets with Pulumi ESC and External Secrets Operator: The Perfect Solution for Today’s Cloud-Native Secret Management.

Using Pulumi ESC with Secrets Store CSI Driver

The Secret Store CSI Driver mounts secrets, certificates, and keys from external stores into Kubernetes pods as volumes.

Automated Secrets Injection: Secrets Store CSI Driver enables Pulumi ESC to automatically inject secrets into Kubernetes pods as mounted volumes or environment variables. This reduces the manual overhead of managing secrets directly in Kubernetes.
Secure and Dynamic Secret Access: By leveraging Pulumi ESC, Kubernetes applications can securely fetch secrets from external providers, ensuring dynamic access without exposing credentials in pod specifications.
Streamlined Operations: This integration simplifies the process of synchronizing secrets from Pulumi ESC with Kubernetes-native constructs. The automation reduces errors and boosts efficiency.

Learn how to Master Kubernetes Secrets with Pulumi ESC + Secrets Store CSI Driver.

The Future of Secrets Management with Pulumi ESC

Pulumi ESC is more than just a secrets management tool — it’s the foundation of secure, scalable, and agile cloud operations. For cloud engineers grappling with multi-cloud complexity, Kubernetes deployments, and compliance concerns, it provides a streamlined, automated approach that enhances efficiency and security.

With rising cyber threats and stricter data regulations, adopting tools like Pulumi ESC is no longer optional. It’s a competitive advantage for cloud engineers and organizations aiming to lead in today’s cloud-native world.

Next Steps to Secure Your Secrets

Secrets management doesn’t have to be a headache! With Pulumi ESC, you can safeguard your infrastructure, ensure compliance, and eliminate manual errors—all while enhancing security and agility.

Get started with Pulumi ESC today and experience the future of secrets management. Take control of your cloud environment—securely and effortlessly.

How Secrets Sprawl is Slowing You Down—And What to Do About It.

Engin Diri — Mon, 28 Oct 2024 12:40:10 +0200

Only a few things are certain in the lives of developers and DevOps engineers: taxes, yearly performance reviews, and secret sprawl. While the first two are inevitable, the last one is something that can be managed.

As we keep adding new cloud resources and releasing new applications, the number of secrets we need to manage keeps growing: passwords, API keys, certificates, and more. And as if this isn’t enough, we need to manage secrets across different systems and environments with different teams that need to access them; we end up with duplicates.

Let’s have a look at a typical environment and see what teams use: Everything container-related may use Docker secrets or Kubernetes secrets. There is also a high chance that they might use cloud-specific secrets managers like Azure Key Vault, AWS Secrets Manager, or Google Secret Manager. In addition to these, there is a high chance that they might use HashiCorp Vault or some enterprise-grade secrets manager like CyberArk.

There are still a lot of organizations that store their secrets (like the API key for a cloud service) in environment variables to configure their applications on virtual machines, containers, or serverless functions. And as if that weren’t risky enough, some developers still hardcode sensitive information in their application code. Or use plain unencrypted text files such as .env files and share them inside the team. To be fair, this is not something that is done on purpose, and most teams realize the risks of these practices. But the problem is that they have not found a better way to manage their secrets that won’t slow them down and impact their productivity.

As the IT infrastructure grows, it becomes increasingly challenging for organizations to secure, organize, and distribute their secrets. This is where secrets sprawl comes into play. Secrets sprawl is the uncontrolled and unmanaged distribution of secrets across different systems and environments.

Modern architectures are adding to the problem of secret sprawl as modern applications are more distributed and consist of smaller building blocks with a growing number of different technologies.

While teams are ensuring that each of their applications has everything it needs to run, it’s also easy for them to lose track of the secrets they are using. Everything can be fine until they need to rotate a secret or revoke a compromised one. Then they need to go through all the systems and environments to find and update the secret. Tracking down all the places where a secret is used can be a time-consuming and error-prone process with a high risk of missing some places.

So what are some of the pain points of secret sprawl, and how can we overcome them to regain our productivity?

Pain Points of Secrets Sprawl

With secrets sprawled all across our teams (development and operations) and infrastructure, we face several points:

It is challenging to keep track of all the secrets and where they are used. When we need, for example, to find a specific secret, we need to go through different secret managers, code, and, in the worst case, the local files of one of the developers.

Do we need to rotate a secret? Now we need to go through all the systems and environments to find and update every duplicated secret.
Security risk is another pain point to mention. When a secret is spread throughout the organization, it’s hard to manage access control and enforce specific access policies.

Security best practices also recommend that secrets should be rotated regularly. But with secrets sprawled all over the place, it makes it a lot of work to rotate them. Time we could spend on more important tasks like developing new features.
Secrets sprawl makes it extremely challenging to distribute secrets. Having a secret, for example, in a cloud provider’s secret manager is cool and great, but what if we need to use it in a Kubernetes cluster? What is the process for that?

Also, having secrets in multiple places means there are different owners for them. These secrets are now some kind of gatekeeping by the team that owns them. This makes it hard for other teams to access them when they need them. And even if they get access, they will then be mostly shared in plain text as the other team does not have access to the secret manager itself.

As we can see, secret sprawl is a real problem that can slow down our teams and impact our productivity. But it also affects how fast an organization can scale its development and operations teams.

With an “organic” way of managing secrets and no real transparency, it’s hard to keep up with the pace of the business, and secret sprawl becomes a major bottleneck. Additionally, that growth means a whole bunch of new secrets that need to be managed.

Rinse and repeat.

But behold, there is a solution to this problem. And it’s called centralized secrets management.

Centralized Secrets Management against Secrets Sprawl

The cure for secrets sprawl is centralized secrets management. When you centralize your secrets in one place, you know where they are and who has access to them.

This makes it easier to manage access control and enforce specific access policies. It also makes it easier to rotate secrets as you only need to do it in one place.

Have a look at an example scaling of an organization or team:

When the company is small, the team decides to use Kubernetes to deploy their applications, and they store their secrets in Kubernetes secrets. This first step is already a centralization of secrets as it tightens access to the secrets and makes it easier to audit them. Maintaining an encrypted secrets store and centralizing the secrets in one place is a good start and ensures the organization is following a consistent threat protection policy.

As the company grows, one DevOps engineer is responsible for multiple Kubernetes clusters. In these cases, individual users can’t enforce their own application-wide policies, meaning secrets are better managed and controlled.

But what if the company grows even more and the team decides to use multiple cloud providers? There are many reasons for this, like costs, different service offerings, and better disaster recovery when using multiple cloud providers.

Sticking to the built-in secrets managers of the cloud providers is not a good idea as it will lead to secrets sprawl.

You might also like:

Why Pulumi ESC is the right choice for centralized multi-cloud secrets management

Pulumi ESC is the right choice for centralized multi-cloud capable secrets management as it reduces secrets sprawl by centralizing secrets storage with a lot of integrations for a wide range of platforms.

Pulumi ESC delivers a seamless integration so you can still use your existing secrets managers like Azure Key Vault, AWS Secrets Manager, Google Secret Manager, or HashiCorp Vault. Secrets are automatically synced to Pulumi ESC, saving developers a lot of time so they can focus on their main tasks like developing and building new products and features.

Pulumi ESC is a completely managed service that again saves time and resources for the organization as they don’t need to worry about the infrastructure, maintenance, and support of their self-hosted secrets manager.

Wrapping Up

Every organization faces the risk of secrets sprawl as they grow and add more cloud resources and applications, which they may not even be aware of.

Multi-cloud scenarios make it even more challenging to manage secrets across different systems and environments.

As teams have to juggle between delivering and building new products and features, managing secrets in a secure and efficient way becomes a major bottleneck on productivity and a major security risk. A ticking time bomb that can explode at any time.

Secrets managers can help you reduce secrets sprawl by centralizing secrets storage and providing a seamless way to use, change, and distribute secrets across different systems and environments. Independent of the size of your organization, a startup or a large enterprise, it’s never too late to start managing your secrets in a secure and efficient way.

Pulumi ESC is on the way to becoming the best choice to solve the problem of secrets sprawl. Making both developers and security teams happy by providing a secure and efficient way to manage secrets across different systems and environments.

Start using Pulumi ESC and ESO today by creating an account on the Pulumi Cloud Console and begin managing your secrets in a secure and efficient way.

Create an Account

Pulumi ESC and External Secrets Operator: The Perfect Solution for Today's Cloud-Native Secret Management

Engin Diri — Fri, 04 Oct 2024 00:00:00 +0000

Managing secrets in a cloud-native environment can be challenging, but it is crucial for ensuring the security and integrity of any application or infrastructure. We encounter a lot of different types of secrets, from API keys, database passwords, and certificates to tokens and passwords. These secrets need to be stored securely and accessed by different services in a secure way without exposing any sensitive information to unauthorized users.

Here is where Pulumi ESC and External Secrets Operator come into play by providing a secure and efficient solution for cloud-native secret management.

Pulumi ESC (Environments, Secrets, and Configuration)

Pulumi ESC is our new secrets management & orchestration service for environments, secrets, and configurations. It can be used as part of your Pulumi IaC (Infrastructure as Code) projects using pulumiConfig to expose the stored values to your Pulumi stacks.

But it can also be used as a standalone service to manage secrets and configurations. Pulumi ESC provides support for applications through its dedicated SDKs built for different programming languages. It can also be managed from the command line with the pulumi env commands in the Pulumi CLI, which can inject secrets and configuration as a context for your CLI tools (e.g. terraform).

For more information on Pulumi ESC, you can check out the official documentation.

External Secrets Operator (ESO)

The Open Source project External Secrets Operator (ESO) is a Kubernetes operator that helps manage secrets in a highly secure and scalable way. ESO synchronizes secrets stored in external secret management systems like Pulumi ESC, HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, and more into Kubernetes secrets. This approach allows us to control secret sprawl and centralize the management of secrets in a single place while providing a secure and controlled way to access them.

Why Kubernetes secrets are not enough?

Kubernetes secrets are a great way to store sensitive information in a Kubernetes cluster. Kubernetes secrets are created and stored as a Kubernetes resource object and then used by pods or other resources inside the cluster.

Here is an example of a Kubernetes secret manifest file:

apiVersion: v1
kind: Secret
metadata:
 name: mysecret
type: Opaque
data:
 username: YWRtaW4= # admin
 password: YWRtaW4= # admin

or using kubectl:

kubectl create secret generic mysecret --from-literal=username=admin --from-literal=password=admin

Kubernetes secrets have several limitations. Here are the top ones:

They are stored by default in etcd in base64 encoded format, which is not secure.
They are created either manually by using kubectl commands or by using a manifest file, making them hard to manage at scale.
Hard to manage and synchronize secrets across different environments and clusters.
There is no default way to rotate secrets automatically.

ESO shines by tackling all the challenges listed and providing a way to manage secrets in a more secure and efficient manner. Benefits include:

Mitigate security risks as secrets are stored in a secure external secret management system like Pulumi ESC, which provides more robust security features like encryption, access control, and auditing.
Eliminate manual management of secrets by synchronizing them from external secret management systems into the Kubernetes cluster, reducing the need for manual intervention and minimizing the risk of human error.
ESO provides a unified interface, making it easy to manage secrets across clusters and multi-cloud environments.
ESO integrates with external secret management systems which ensure that access to secrets is tracked and an audit trail is maintained. This is crucial for security compliance and adhering to regulatory standards.
Automate secrets rotation when the external secret management system has support it.

External Secrets Operator architecture

The central part of the ESO architecture is the extension of the Kubernetes API with a new Custom Resource Definition (CRD), which defines where the secrets are stored and how they are synchronized into Kubernetes secrets. The ESO controller then retrieves the secrets from the external secret management system and creates or updates the Kubernetes secrets. Every time a secret is updated in the external secret management system, the ESO controller will synchronize the secret in the Kubernetes cluster and update the secret accordingly.

Core resources of ESO

(Cluster)SecretStore: Defines the external secret management system where the secrets are stored. It contains the necessary information to connect to the external secret management system, such as the endpoint, authentication credentials, and other configuration options. The SecretStore is namespaced while the ClusterSecretStore is cluster wide, which allows you cross-namespaced access to the secrets.
(Cluster)ExternalSecret: Defines the secret that needs to be synchronized into the Kubernetes cluster. It contains the reference to the SecretStore and the secret key and how the data should be transformed and saved as a Kubernetes secret.
PushSecret: With the PushSecret resource, you can push secrets from the Kubernetes cluster to the external secret management system. The Pulumi ESC provider for ESO supports this feature, which allows you to push secrets from the cluster into Pulumi ESC. This feature is useful when you create secrets through other means and want to synchronize them into Pulumi ESC to manage them centrally.

How to Use Pulumi ESC and ESO Together

To demonstrate how to use Pulumi ESC and ESO together, we will store some secrets and configuration values in Pulumi ESC, then synchronize them into a Kubernetes cluster using ESO.

To follow along, you’ll need the following prerequisites:

A Kubernetes cluster (I will be using a local KinD cluster, but you can use any Kubernetes cluster)
Pulumi CLI installed. You can use the pulumi env commands to manage secrets and configurations.
kubectl CLI installed for some debugging

Step 1: Install ESO and Link to Pulumi ESC

We will use Pulumi with the pulumi-kubernetes provider to deploy ESO into the Kubernetes cluster. The power of Pulumi ESC integration comes in two ways: First, to supply the Pulumi access token to ESO via Pulumi IaC, allowing the new ESO instance to access Pulumi ESC later. Second, to provide configuration and secrets from Pulumi ESC, via ESO, to the PodInfo application within the Kubernetes cluster.

pulumi new kubernetes-typescript --name eso-pulumi-esc

You should see the following output:

This command will walk you through creating a new Pulumi project.

Enter a value or leave blank to accept the (default), and press <ENTER>.
Press ^C at any time to quit.

Project description (A minimal Kubernetes TypeScript Pulumi program):
Created project 'eso-pulumi-esc'

Please enter your desired stack name.
To create a stack in an organization, use the format <org-name>/<stack-name> (e.g. `acmecorp/dev`).
Stack name (dev):
Created stack 'dev'

The package manager to use for installing dependencies npm
Installing dependencies...


added 310 packages, and audited 311 packages in 12s

44 packages are looking for funding
 run `npm fund` for details

found 0 vulnerabilities
Finished installing dependencies

Your new project is ready to go! ✨

To perform an initial deployment, run `pulumi up`

Head to Pulumi Cloud Console and create a new Pulumi ESC project with the name eso-esc-demo:

And the environment eso-esc-dev:

In the editor add the following yaml into the Environment definition:

values:
 pulumiConfig:
 pulumi-pat:
 fn::secret: <your-pulumi-pat>

Don’t know how to create a PAT? Check out the official Pulumi documentation.

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/eso-esc-demo/eso-esc-dev

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/eso-esc-demo/eso-esc-dev

Now, we need to link the Pulumi ESC project to the Pulumi IaC project. To do this, we need to add the following to your Pulumi.dev.yaml:

environment:
- eso-esc-demo/eso-esc-dev

Now that all the wiring is done, we can deploy the ESO instance into the Kubernetes cluster. Open the index.ts file and add the following code:

import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

const config = new pulumi.Config();

// Create a Kubernetes namespace
const ns = new k8s.core.v1.Namespace("external-secrets", {
 metadata: {
 name: "external-secrets",
 },
});

// Deploy a Helm release into the namespace
const externalSecrets = new k8s.helm.v3.Release("external-secrets", {
 chart: "external-secrets",
 version: "0.10.4", // Specify the version of the chart
 namespace: ns.metadata.name,
 repositoryOpts: {
 repo: "https://charts.external-secrets.io",
 },
});

// Deploy a secret into the namespace
const mySecret = new k8s.core.v1.Secret("my-secret", {
 metadata: {
 namespace: ns.metadata.name,
 name: "pulumi-access-token",
 },
 stringData: {
 "PULUMI_ACCESS_TOKEN": config.require("pulumi-pat"),
 },
 type: "Opaque",
});

// Export the namespace name
export const namespaceName = ns.metadata.name;

// Export the Helm release name
export const helmReleaseName = externalSecrets.name;

Deploy the stack by running:

pulumi up

And you should see that the secret was created in the Kubernetes cluster and the ESO instance was deployed successfully.

kubectl get secret -n external-secrets pulumi-access-token -o jsonpath='{.data.PULUMI_ACCESS_TOKEN}' | base64 -d

Step 2: Create an External Secret

Now, we will create a secret in the Pulumi ESC project and synchronize it into the Kubernetes cluster by creating an ESO reference to the external secret.

Create a new ESC environment called esp-esc-app in the eso-esc-demo ESC project.

Add the following YAML into the Environment definition:

values:
 app:
 hello: world
 hello-secret:
 fn::secret: world

If you prefer to use the Pulumi CLI, you can create the environment by running:

pulumi env init <your-org>/eso-esc-demo/esp-esc-app

And set the configuration by running the env edit command and copy the above YAML into the editor:

pulumi env edit <your-org>/eso-esc-demo/esp-esc-app

Either way, you should see following environment configuration in the Pulumi Cloud Console:

We can now create ClusterExternalSecret in the index.ts and reference the ESC secret.

import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

// cut for brevity

const crd = new k8s.apiextensions.CustomResource("cluster-secret-store", {
 apiVersion: "external-secrets.io/v1beta1",
 kind: "ClusterSecretStore",
 metadata: {
 name: "secret-store",
 },
 spec: {
 provider: {
 pulumi: {
 organization: pulumi.getOrganization(),
 project: "eso-esc-demo",
 environment: "esp-esc-app",
 accessToken: {
 secretRef: {
 name: mySecret.metadata.name,
 key: "PULUMI_ACCESS_TOKEN",
 namespace: mySecret.metadata.namespace,
 },
 },
 },
 },
 },
});

We can check that the secret was successfully synchronized by running:

kubectl get clustersecretstore secret-store -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}'

Step 3: Deploy an Application and Reference the Secret

Now, we can deploy an application that references the secret from the Kubernetes cluster. I am going to deploy the app called PodInfo, which is an application that exposes the pod information as an API. Add the following code to the index.ts:

import * as k8s from "@pulumi/kubernetes";
import * as pulumi from "@pulumi/pulumi";

// cut for brevity
const podInfo = new k8s.helm.v3.Release("podinfo", {
 chart: "podinfo",
 version: "6.7.0",
 namespace: "podinfo",
 createNamespace: true,
 repositoryOpts: {
 repo: "https://stefanprodan.github.io/podinfo",
 },
 values: {
 extraEnvs: [
 {
 name: "FROM_ESC_VIA_ESO",
 valueFrom: {
 secretKeyRef: {
 name: "esc-secret-store",
 key: "hello-secret",
 }
 }
 },
 {
 name: "FROM_ESC_VIA_ESO_2",
 valueFrom: {
 secretKeyRef: {
 name: "esc-secret-store",
 key: "hello",
 }
 }
 }
 ]
 },
});

const externalSecretPodInfo = new k8s.apiextensions.CustomResource("external-secret-podinfo", {
 apiVersion: "external-secrets.io/v1beta1",
 kind: "ExternalSecret",
 metadata: {
 name: "esc-secret-store",
 namespace: podInfo.namespace,
 },
 spec: {
 dataFrom: [
 {
 extract: {
 conversionStrategy: "Default",
 key: "app",
 }
 }
 ],
 refreshInterval: "1h",
 secretStoreRef: {
 kind: clusterSecretStore.kind,
 name: clusterSecretStore.metadata.name,
 }
 },
});

This will deploy the PodInfo application and create an ExternalSecret resource that references the secret from the Pulumi ESC project. We mount the secret into the pod as an environment variable by using the extraEnvs field in the Helm chart.

In a real-world application, you might use alternate ways to reference the secret, like mounting it as a volume or using fromEnv field in the Helm chart. It depends on your application and how you want to consume the secret.

After deploying the stack, you should see that the PodInfo application is running. To access the application, we can port-forward the service by running:

kubectl port-forward svc/podinfo 9898:9898 -n podinfo

Our PodInfo demo application provides a large selection of different endpoints to interact with. We are interested in the /env endpoint to check if the secret was successfully mounted into the pod. Let’s try that:

curl -s localhost:9898/env | grep FROM
 "FROM_ESC_VIA_ESO=world",
 "FROM_ESC_VIA_ESO_2=world",

And here we see secret values from the ESC project, proxied by ESO, and passed to the PodInfo application as environment variables. That means the secret was successfully mounted into the pod and the application is running as expected.

Step 4: Clean Up

After you are done with the demo, you can clean up the resources by running:

pulumi destroy

Conclusion

The combination of Pulumi ESC and External Secrets Operator provides a secure and efficient way to manage secrets in a cloud-native environment, improving security, efficiency, and compliance when consuming secrets in your Kubernetes cluster. By using Pulumi ESC, you can manage secrets and configurations in a centralized way, and with ESO, you now have a streamlined process to synchronize secrets into your Kubernetes cluster.

Start using Pulumi ESC and ESO today by creating an account on the Pulumi Cloud Console and begin managing your secrets in a secure and efficient way.

Create an Account

Pulumi Blog: Secrets management

Secrets Management Tools: The Complete 2025 Guide

Secrets Management Tools Overview

Secrets Orchestration Platforms

Enterprise Secrets Vaults

Cloud-Native Secrets Managers

Developer-Focused Tools

Application Security & Scanning

Specialized & Integration Tools

The Evolution Beyond Simple Secret Storage

Most Popular Secrets Management Tools

Secrets Orchestration Platforms

Pulumi ESC (Environments, Secrets, and Configuration)

Doppler

Infisical

Enterprise Secrets Vaults

HashiCorp Vault

CyberArk Conjur

Akeyless

Cloud-Native Secrets Managers

AWS Secrets Manager

Azure Key Vault

Google Secret Manager

Developer-Focused Tools

1Password Secrets Automation

Bitwarden Secrets Manager

Application Security & Scanning

GitGuardian

TruffleHog

Specialized & Integration Tools

External Secrets Operator

Berglas

Confidant

Chamber

Top Features to Look for in Secrets Management Tools

Secrets Orchestration and Multi-Source Integration

Configuration as Code and Environment Management

Dynamic Credential Generation

Multi-Platform Integration and SDK Support

Comprehensive Audit and Compliance Capabilities

Performance and Scalability

Key Considerations for Choosing Secrets Management Tools

Orchestration vs. Replacement Strategy

Cloud Strategy Alignment

Team Structure and Expertise

Security and Compliance Requirements

Cost Optimization

The Future of Secrets Management: Orchestration and Configuration as Code

From Storage to Orchestration

Configuration as Code Revolution

Dynamic Credentials and Zero-Trust Architecture

AI and Machine Learning Integration

Choosing the Right Path Forward

Why Every Cloud Engineer Needs Pulumi ESC for Secrets Management

In this article:

The Challenge of Secrets Management in Modern Cloud Environments

What is Pulumi ESC?

Key Features of Pulumi ESC

1. Seamless Integration with External Platforms

2. Dynamic Secrets Synchronization with ESO

3. Safely Roll Back to a Previous Version

4. Secure by Design

5. Automated Rotation and Expiry

6. Configuration-as-Code, Automation, and Integration Everywhere

7. Dev Tools Integrations

8. Infrastructure Tools Integrations

Why Cloud Engineers Need Pulumi ESC

Enhanced Security and Compliance

Streamlined Multi-Cloud Management

Simplified Kubernetes Secrets

Zero Downtime Through Automation

Developer-Friendly Workflows

Kubernetes Secrets Management

Using Pulumi ESC with External Secrets Operator

Using Pulumi ESC with Secrets Store CSI Driver

The Future of Secrets Management with Pulumi ESC

Next Steps to Secure Your Secrets

How Secrets Sprawl is Slowing You Down—And What to Do About It.

Pain Points of Secrets Sprawl

Centralized Secrets Management against Secrets Sprawl