AI in DevOps | AI Talks for DevOps Overview

Posted on Monday, Nov 13, 2023

The integration of artificial intelligence (AI) with DevOps signals a new era in software development. DevOps possesses unique characteristics and needs that make it exceptionally compatible with AI augmentation. Given that code fundamentally relies on language, and large language models (LLMs) serve as the core of GPT functionality, these models are particularly well-suited for tasks such as code generation. This article unwraps the topics addressed during our “AI: Friends or Foe | AI Talks for DevOps” event in San Francisco.

Watch the discussion between Luke Hoban, Pulumi’s CTO and cofounder of TypeScript, and Nana from TechWorld of Nana, moderated by community engineer Adora Nwodo.

On this AI in DevOps article:

How will AI Impact the Future of DevOps

In the Fireside chat, Luke Hoban expressed his enthusiasm for incorporating AI into DevOps, emphasizing its potential utilization by both DevOps professionals and the development team. In contrast, Nana Janashia, TechWorld with Nana, adopted a more cautious stance, drawing attention to potential risks, particularly regarding reliability.

What is DevOps in AI

When considering the adoption of AI and AI engineering, such as prompt engineering and other specific disciplines, AI engineers rely heavily on the cloud. Many AI-based systems are built around cloud-native tools, practices, and managed services.

The extensive use of cloud-native primitives means that AI is fundamentally a cloud adoption story. Consequently, established DevOps and infrastructure as code tooling and best practices play an important role in AI engineering, in what is being done, and in leveraging its use.

What is AI in DevOps

Integrating AI in DevOps entails employing machine learning (ML) and other artificial intelligence technologies to automate and optimize various software development and delivery aspects, including tasks such as automating testing and deployment processes and refining resource management.

The incorporation of AI in DevOps can result in:

Enhanced speed
Improved resource management
Increased reliability throughout the software development lifecycle

Consequently, organizations may experience swifter deployments, reduced errors, and heightened overall productivity. By substituting manual processes with automation and AI-powered solutions, DevOps teams can elevate product quality and more efficiently oversee their systems.

How AI is Currently Used in DevOps

Luke Hoban at the San Francisco Pulumi Group meetup discussing the usage of AI, during the AI Talks for DevOps

DevOps significantly impacts all stages of the development and operations lifecycle. It has helped break silos and optimize to increase developer velocity and productivity, creating a positive impact on the ops side.

AI assists DevOps teams to refine their operations by detecting inefficiencies and triggering warnings as soon as issues appear. This technology also enables real-time observation of systems and applications, permitting prompt actions from operations personnel and minimizing downtime.

AI-driven automation can also help the software development process with code generation. The operations teams from both departments quickly pinpoint any issues in their workflows while speeding up decision-making processes.

In the long term, the AI may use cloud credentials to create and autonomously automate deployment and management and perform DevOps tasks on behalf of the DevOps or platform engineering team. In the near term, it is used to assist humans, from DevOps professionals to developers.

“In the near term, AI usage in DevOps is about how do you keep the human in the loop, how do you use it as a learning tool, as a code generation tool, and for auditing to provide actionable notifications, all of which are still very human-dependent. There is so much more it can be used for. Currently, it’s a tool serving humans. But it is fascinating to imagine it in the future when it can be trusted to be an autonomous agent.” - Luke Hoban, CTO of Pulumi and co-creator of TypeScript

So, how does this technology bring about these advancements? Two main ways are by streamlining development tasks and improving monitoring and security protocols.

Streamlining Development Workflows

AI has the potential to significantly change the software development life cycle by automating routine processes, using resources efficiently, and offering solutions for improved code. DevOps teams can leverage AI-driven tools to increase effectiveness when managing their assets while also improving code quality, leading to faster project deliveries.

For instance, AI can identify any slowdowns during progress and recommend improvements, analyze root causes, and help make judicious decisions regarding resource allocation, ultimately resulting in efficient delivery of software products.

Enhancing Monitoring and Security

The usage of AI significantly boosts DevOps monitoring and security. With the assistance of AI tools, operations teams can use machine learning algorithms to examine a massive amount of data to protect their applications from potential risks and enhance reliability. These algorithmic solutions provide valuable advice for development personnel, which helps strengthen protection measures while optimizing performance simultaneously.

How Can a DevOps Team Take Advantage of Artificial Intelligence

A DevOps team can leverage AI in several ways to enhance efficiency, automation, and overall productivity. Here are some key strategies for a DevOps team to leverage AI:

Code Generation and Learning

AI can be a powerful tool to make the development process much smoother. It can help engineers focus on what matters instead of searching for documentation on websites, developer forums, or search engines for answers to help solve specific problems. The perfect code snippet is often hard to find or may not even exist.

Generative AI, powered by LLMs, can help developers and DevOps engineers quickly discover, learn, and, for example, use new cloud infrastructure APIs.

Pulumi AI response to Run metabase in Azure, using TypeScript

You can see it in practice using Pulumi AI. You can use natural language to ask for a specific cloud infrastructure goal and obtain a corresponding Pulumi program in your preferred programming language. Moreover, Pulumi AI facilitates the iterative refinement of your cloud infrastructure, incorporating new features, enhancements to security and performance, resolution of correctness issues, and clarification of requirements. Try Pulumi AI.

Intelligent Resource Management

Platform and developer teams are spinning up resources across many cloud accounts within multiple cloud providers. It can be almost impossible to aggregate and search your infrastructure across these accounts since every cloud and every account is (by design!) a walled garden.

Often, to answer key questions about their business, such as “Which team has the largest cloud footprint?”, teams need to combine data across various clouds, cloud accounts, and SaaS tools. This prevents organizations from getting quick answers to operational questions and barriers to building aggregations to derive insights. AI can provide actionable insights over infrastructure with natural language processing (NLP) queries.

“I can get developers using IaC immediately with Pulumi Deployments and its GitHub integration, while Pulumi Insights [Resource Search] makes it really easy to find idle developer environments that need to be shut down, which reduces our cloud costs” - Santosh Dornal, Head of Software Test & DevOps, Alkira.

When using Pulumi Cloud, you can leverage AI Assist. You can provide a plain English (Spanish, French, or Japanese) request to express queries where you might not know the exact syntax, type tokens, or package names. AI Assist makes it easier by providing answers to queries like:

show me all s3 buckets not tagged in production
show azure and azure native security groups
show all AWS VPCs

Remediation tool: Continuous and Automatic Compliance

Defining and managing security rules, criteria, and conditions are key to enabling security at scale. Policy as Code, or Compliance as Code, has the ability to verify and spot problems before deploying the infrastructure. When policies are written with code, you can apply software development practices such as testing, automated deployment, and version control.

AI can be valuable not just in anomaly detection but also in remediation, which doesn’t just warn or error but can change the infrastructure directly to remediate compliance gaps. With Pulumi, Remediation policies can be configured and applied across any subset of stacks within an organization.

Consider the unfortunate incident at GitLab, marked by a disastrous sequence of events resulting in the loss of segments of production databases and substantial user data.

In this scenario, AI could have swiftly pinpointed the root cause and proposed a rapid solution, potentially mitigating the extent of damage before it escalated.

While the benefits of writing policies as code are evident for DevOps engineers, the organizational benefits are even more significant.

Enhance Collaboration and Communication

Nowadays, many professionals with DevOps in their titles are held accountable for the multiple aspects of Dev + Ops. At its core, DevOps is a cultural movement that removes barriers among teams and promotes collaboration and shared responsibility between dev and ops.

DevOps teams have seen great benefits from AI-driven tools, particularly in improving transparency, encouraging feedback, providing quick answers to common queries, and automating routine tasks. Many major companies, like Microsoft, use AI to build efficient workflows that make better decisions, leading to increased business outcomes. This includes smoother application delivery, a prime function of DevOps initiatives.

Improved Product Delivery

Companies incorporating AI into their DevOps strategy reap the rewards of accelerated product delivery. This has been made possible through improved workflows, automated testing, and strengthened security measures, leading to greater efficiency in software development by allowing development and operations teams more scope for dealing with complex tasks using machine learning models. They can make better decisions, resulting in faster product delivery and enhanced business outcomes.

By streamlining existing systems within this context, customers can benefit from superior products delivered faster than before while companies provide and reinforce their business value.

Planning and Cost Optimization

Implementing AI algorithms or AI tools that analyze usage patterns can assist in predicting future resource needs and optimize cloud infrastructure costs. This also ties in with intelligent resource management mentioned before, which can enhance cost efficiency, optimize resource allocation, and prevent unnecessary expenses.

Top 5 Skills for DevOps Engineers in the AI Era

Adora Nwodo with Nana discussing skills that DevOps engineers will need to learn to make AI work

DevOps engineers must open new skills, mainly in cloud services, data, and AI/ML.

Programming Languages: Although DevOps don’t usually need to know general programming languages in-depth, they should know what is most used by application teams. It’s a nice-to-have to find errors in CI/CD pipelines and fix them themselves as needed, but it is a must-have if the goal is to work MLOps.
Infrastructure as Code: Automation is important for DevOps and critical for MLOps. IaC, using familiar languages, has been used since the early days of MLOps and is a key component to solving the cloud orchestration challenge present in AI/ML. Dive into more detail in The Real AI Challenge is Cloud, not Code!
Continuous Integration and Delivery: CI/CD is mandatory knowledge, and it must be understood in detail to predict data preparation code, AI/ML code, CI/CD code, and infrastructure-related code.
Custom containers: As a DevOps professional, you are expected to understand Docker and/or Kubernetes concepts and best practices.
Orchestration: This is the holy grail of Ops and your key to unlocking AI.

How to Overcome AI for DevOps Challenges

The integration of AI in DevOps brings forth several challenges that organizations must navigate to leverage the full potential of these technologies. Here are some key challenges and potential strategies to overcome them:

1. Skills and Expertise

Challenge: AI & ML expertise is often scarce within DevOps teams, leading to a skill gap in implementing and managing AI-driven tools.

Overcoming Strategy: Organizations will need to hire professionals with AI and ML expertise, or existing team members will need training in machine learning, becoming proficient in MLOps combined to DevOps principles. Collaboration between DevOps and data science teams can also bridge the knowledge gap.

2. Integration Complexity

Challenge: Integrating AI seamlessly into existing DevOps workflows can be complex, given the diverse technologies and processes involved.

Overcoming Strategy: Prioritize a phased integration approach, gradually introducing AI capabilities into specific stages of the DevOps pipeline. This allows teams to adapt without disrupting established processes.

3. Scalability

Challenge: As AI applications within DevOps expand, ensuring scalability becomes crucial to accommodate growing workloads.

Overcoming Strategy: Design AI solutions with scalability in mind, utilizing cloud resources and containerization. Regularly assess performance and adjust infrastructure to meet evolving demands.

4. Data Quality and Availability

Challenge: AI models rely heavily on high-quality and readily available data. Inconsistent or inadequate data can lead to inaccurate predictions or suboptimal performance.

Overcoming Strategy: Prioritize data quality and establish robust data pipelines. Conduct regular audits to ensure data accuracy and implement strategies for handling missing or incomplete data.

5. Cultural Resistance

Challenge: Resistance to change or skepticism about the benefits of AI within DevOps and in software development can impede successful integration.

Overcoming Strategy: Foster a culture of innovation and continuous improvement. Provide clear communication about the benefits of AI, involve team members in the decision-making process, and showcase success stories to build confidence.

Navigating these challenges requires a thoughtful and strategic approach, focusing on collaboration, skill development, and the gradual incorporation of AI into existing DevOps practices. Regular evaluation and adaptation are essential to ensure a successful and sustainable integration of AI in the DevOps environment.

Conclusion

AI in DevOps can transform software development by making workflows more efficient, monitoring and security stronger, and overall effectiveness increased. Organizations that embrace AI/ML aim for better collaboration between teams, automating and intelligently managing infrastructure, reducing time-to-market, and increasing developer productivity and cost savings due to streamlining processes within their development lifecycle. AI and ML knowledge will become necessary skills expected to be applied to enhance and benefit the development, security, and operation teams as a whole. Follow the Pulumi Python + AI/ML series to upskill and get ahead:

Frequently Asked Questions

Can AI replace DevOps?

DevOps has not been taken over by AI yet. Rather, its use has made a difference in code quality and accelerated certain processes. Improvements to the quality of coding have been seen as one of the main benefits brought on through integration with artificial intelligence.

What is an example of AI in DevOps?

AI can be utilized to make DevOps processes more efficient by creating test cases and pipelines, assessing infrastructure configurations for security concerns, and deploying applications. This technology includes machine learning algorithms, natural language processing systems, computer vision capabilities, chatbot programs, and virtual assistants. All are designed with one goal in mind – optimization of operations within development teams.

What is the future of DevOps?

Operations teams collaborating with developers to create quality products quickly and effectively is the goal for DevOps moving forward. The vision of this process involves both sides working together, resulting in an improved system and increased efficiency.