Courtesy Generative AI with Garima Bajpai at AIE 2024

Hello everyone, and welcome to the artificially Intelligent Enterprise Online Conference. It's an initiative by Techstrong Group. I would start my talk with setting the stage and the key takeaways from my, uh, talk today.

Obviously, DevOps professionals have come a long way, uh, fostering the change in the service delivery with time. Their role has been evolving in the DevOps ecosystem. In this talk, we discuss the key impact of generative AI on the key DevOps roles.

With my talk, I also wanted to highlight some areas of impact and how DevOps professionals should be prepared themselves to leverage the indulgence of generative AI in their respective roles. And further, um, I will also provide on how organizations can leverage generative AI capability to shift the approach for fulfilling short-term skill gap to a long-term flexible workforce model. And concluding my talk, I aim to provide guidance on how to stay ahead in the curve and maximize the potential of generative AI and apply it into DevOps roles through a task-based framework.

I hope that you will still be interested to see what I bring on the table today. So obviously, I start my talk with, um, generative AI assisted software development teams and how the outlook for future looks like. 13 million USD in 2022, but it has exponentially risen.

And the demand, uh, for generative AI products could add up to around $280 billion for new software revenue. 3 trillion market share by 2032 as per Bloomberg. And I've also, um, associated on the left-hand side, some of the key areas you can watch out for, starting from coding, ai, workload infrastructure software.

And also in terms of some verticals like drug discovery software or cybersecurity or education related, even gaming industry could benefit out of it, right? So the market is characterized by a strong competition, of course, because, uh, the stakes are too high with a few major worldwide competitors owning the significant share of the market. So, uh, if you have heard about Code Academy or Code AI or Deep Court tab nine, open AI and with corporations, Microsoft and Google, I think, uh, mostly, um, these companies are driving these initiatives.

And of course, um, let's dig deeper into some history as well, like how, uh, this whole movement got exponential, um, integration. And of course, um, if, uh, you look at chart GPT, it was publicly released on November 30th, 2022, largely as a technology demonstration. Two months later, it had attracted us as estimated a hundred million active users making it the fastest growing consumer application in the, uh, history.

Uh, rolling back to March 16th, 2023, Microsoft announced an integration of chat GPD fall into its office suite. March 21st, 2023, Google releases Bard, uh, a competitive product, an AI chat bot based on Lambda family of LLMs. And April 13th, 2023.

Amazon is not behind. It announces Bedrock, a fully managed, uh, integrated managed service, uh, that makes models and having an API based integration with multiple providers, um, in addition to Amazon's own Titan, LLM. So as, as you can see, um, there is a lot of action and a lot of in interest and investment happening, uh, from generative AI perspective.

So in today's talk, what I would do is I would have broken down the stock into three parts. The first part would be generative AI capabilities for DevOps and how it be as DevOps practitioners professionals can leverage, uh, a generative AI capability. Part two of this talk would be how we develop this, uh, with emerging technology and what can be our next steps in the process.

So part three of this, uh, talk would also hit the continuous innovation button in using next generation business model, conceptualizing open source innovation and what entails, uh, as an operating model for future companies when it comes to software lifecycle management. So if you're still around and you're interested in my talk, I would like to introduce myself. Um, I'm the founder for the Dev Canada DevOps Community of Practice, which has several chapters, all of our Montreal Edmonton Atlantic provinces.

Um, I'm also the producers for Summits Canada, chair for the Ambassador Program at Continuous Delivery Foundation. I was the winner for the DevOps Executive of the Year Award by DevOps dozen group. Uh, uh, it's a text on initiative DevOps in 2023, and I've also published several blogs, articles, and books.

One of my upcoming book is CICD Design Pattern, uh, which we are aiming for for all this year. My call to action is leadership and, uh, practices and communities and how we work together. So, fast forward to part one, generative AI capabilities for DevOps.

Now, adding generative AI capability to every job role can be challenging, but also it comes with a certain business value. So let's, uh, before digging deeper into this conversation, let's build some common vocabulary. The key difference between traditional AI and generative ai, it's ability to create new content.

So according to Gartner, what, uh, generative AI can mean is it can learn from existing artifacts to generate new realistic artifacts at scale. And that reflects the characteristics of the training data, but not repeat repeated. It can produce the idea of novel content like images, videos, news, speech, text, software code and whatnot.

So what generative AI can do for DevOps practitioners, how it got hyped, let's explore further. So obviously, if you are, you, you are follow the trends. And I, I quote here, garner says that more than 50% of software engineering leaders role will explicitly require oversight of generative AI by 2025.

So clearly, it's not only a productivity tool for developers. Organizations are looking at the broader potential, how to interrupt organizational structures with this kind of capability beyond the rise of new level of technical expertise such as AI specialist and machine learning engineers. What is the potential which we can leverage?

Can we have this tool as, or this capability as you know, moving the legacy environment and doing transformation for legacy pieces of the puzzle to a more next generation, uh, you know, software. So let's start to rethink, uh, around those broader possibilities as well. And, uh, again, for, from a role perspective, identifying that the skills can be combined to create new positions or eliminate redundancies.

So how, how the future looks like. Now, it can be challenging, interesting, overwhelming, uh, to most of the people, right? Because it has first of all, uh, exponential possibilities to disrupt.

And the second part of this is it is fast growing. It is like, it's not like innovation cycles from the past. It is continuing and it's happening now.

So from an early assessment perspective of gen AI adoption, uh, what's happening today is the initiatives are focused on efficiency. More or less gen AI capabilities are being explored for that rather than any transformation initiatives, like what I said, like for banking, for example, can we move the legacy part of the code and applications? We generate AI to a more, uh, next generation make the stuff software.

So that possibility, I think, uh, has not yet started to be explored, but that's, that has a potential word to build tune and run large scale AI models. Right now, it's mostly occurring in the cloud. And because of the reasons that GPU TPUs required are more expensive and they're scars, also the market is dominate by a few tech giants.

So if you are following up, there are some startups which are backed by significant investments. And, uh, you know, OpenAI is an example of that. To build applications on top of foundational models, practitioners need a base to store and, um, access the foundational models.

And secondly, they need specialized ops tooling, technology and practices. So obviously there are also some pre prerequisites from, uh, you know, integration of general gen AI perspective. Now, what these first wave of gen AI application are and how software practitioners are using it, right?

If you are a DevOps practitioner or professional, and you are looking at the next steps, I think there are several of the shelf tool AI tools, which are out there, like hub copilot, core dm, Amazon coreper, tab nine, ADA are like all developer focused and mostly assisting and enhancing the individual work. Gene AI capabilities are now generally available across all the tools. So if you look at like, or if you use like, for example, JIRA software or Confluence, um, or service management tools, all these are leveraging combination of inhi in-house and open AI RGPT four and similar kind of, you know, integrations, security tools are another, uh, big area for securing gen ai, uh, developed code and also exploring more possibilities.

So for example, Snyk has a generative AI half that creates code fixes as well as the rigid and regulated symbolic AI half will with the best in class knowledge base and tailored rules that tailor and double check what generative AI half is producing. So obviously there are multiple, um, use cases when it comes to security tools. So obviously, uh, this provides you a good foundation of what, uh, we would be, you know, talking about when we move on to the part two of this, uh, talk developing with emerging technologies and generative ai.

And we will mostly talk about like the DevOps practitioners and professionals here. So why do you think that we should hit the refresh button for 2024 as DevOps practitioners? And obviously I've worked across many geographies, um, many, uh, leadership profile in the DevOps space, and what I've seen, and of course this is not something news to all the DevOps practitioners that, uh, it's very hard to believe that we only started a decade ago with DevOps, with exponential growth in tools.

And, uh, with most of the practices and concepts which contains delivery is bringing to life are more like, uh, continually, uh, getting innovated, adding new features. It's also a, uh, adding to complexity. So it's, uh, obviously rapid adoption of all these, uh, tools and applications bring new risks over operational overheads and cognitive load to the practitioners.

Cost is another factor with more and more services and application moving to cloud financial practices and engineering practices are getting tightly integrated. And lastly, skill shortage. There are many aspects of learning many people are getting behind.

So there's a lack of time for self self development. So this has triggered, uh, the hit the refresh button for 2024 for DevOps practitioners. So, uh, one of the key areas and the key, you know, initiatives, which I see across the DevOps, you know, adoption is how do we integrate gen AI tools and applications?

And obviously, uh, to make it more tangible, what I, uh, wanted to do through this talk is discuss a few key roles and see how these roles are leveraging gen AI capabilities in their day-to-day work. So obviously, um, not reinventing the wheel, and of course, I, I'm not a like, uh, advertising PagerDuty here, but I thought that PagerDuty is one. Um, uh, there was one article which was very good, that at outline six main roles for modern organizations when, uh, it, uh, it comes to like DevOps thinking and the DevOps ways of working and onboarding to DevOps culture.

So I thought that this is a good kind of, you know, article to kind of start with and discuss all these roles. So obviously I give you some like pointers to what these roles do, and then again, uh, how they are integrating their gen AI capabilities. So if you are into DevOps space, you would know DevOps evangelists.

They're all over the place responsible for delivering the change, collaborating with dev and ops teams, and working towards optimizing the flow feedback and fostering ex experimentation. We'll see in the next slides how this role will be impacted by gene ai. The next, um, role I will talk about is code, really release manager and of course, uh, creating the cadence between Del delivery and deployment of code.

And this role needs deep, deep technical expertise to run and maintain the releases. The next role is automation Architect responsible for designing and implementing automated pipelines, reducing the manual toil. Uh, another important role experience assurance experts.

So obviously a lot of discussion around observability, for example, responsible for user experience, enabling feedback loops and superior experience in terms of, you know, how your features are performing on the field and improving the end product usability as well. Software developers and testers, heart of everything, what we do in the software space responsible for designing, developing testing, uh, and, uh, of the software product. And prime responsibility is writing code, testing and bug fixing.

The sixth role is security and compliance engineer, of course responsible for the overall security posture. So now, um, now we have discussed some of the key roles. I would also like to discuss how these roles are getting disrupted or have getting enhanced by the, uh, gen AI capability.

So automated code generation, for example, and we have discussed about GitHub copilot where, you know, there's a substantial rise in the AI augmented code generation, uh, capability and GitHub copilot leads. Uh, uh, the integration with a lot of, you know, um, code predictive code generation, like, um, features support multiple languages, and, uh, is also a great way to train and re-skill the staff. So obviously this is one of the tools which we, you can look out for tab nine, another one of my favorite, another tool that stands out with deep learning capabilities and support 20 plus programming languages.

Uh, software developers, uh, will also have more user friendly tools to support the journey and probably slow down the race of hiring developers. However, the core job of the practitioners would be to focus on the business value driven development rather than writing too much of code and obviously generate, uh, this also, you know, indulgence is also generating new roles. So generative AI based moderators, for example, it's a new job role in making these moderators can help you exporting and reusing its existing code rather than trying to reinvent the wheel.

So if you are, uh, in the space, you are a software developer, software tester, you can see some synergies here, AI based release management. So obviously release management is getting more and more complex, but, uh, obviously generative AI comes to the rescue. Here.

Companies have started to introduce predictive decision making, threat insights, and many more features to orchestrate releases. There are also some more like, uh, cadence with, uh, applications which are generating AI based, uh, you know, generating release notes, for example, for boosting the productivity. Um, I also, I feel that there are more integrations with generative ai, uh, in this space, uh, needed where, you know, you can do real time releases and integration with several channels, special edition software, classic upgrades, and of course, software release and addressing different, uh, segments of customers with managed subscription or open source versions, et cetera.

So obviously there is a lot more potential, uh, which an AI capability and tools and applications. And if you're a release manager, you can watch out for those, you know, uh, key advanced features and developments happening in this space. Security and compliance engineer, of course, I will quote some examples.

So one of the recent announcements that Jfr the DevSecOps company is integrated in generative AI capability into x-ray and Artifactory are, it's automatically detecting security f laws and license violations, et cetera. Another example, uh, is when, um, there was a survey done, uh, for IT professionals conducted by sny, and they found that artificial intelligence to write code is also creating a security paradox. And this is, again, another area where, you know, security and compliance engineers can, can see, um, growth opportunity or enhanced kind of, you know, um, capability of integrating, um, let's say security practices into generative ai.

Um, because it's a double-edged sword, it can be used to enhance the portfolio of bad actors also. So how the compliance and security engineers can, uh, you know, be better positioned to not only, um, uh, identify the code, which is generated by generative AI tools and applications, but also simulating, let's say, malware attacks and identifying vulnerabilities and also auditing and assessing how generative AI outcome is performing in specific, you know, uh, specific, um, for specific requirements. So obviously there are next generation cyber capabilities, which are in making automation architect, of course.

Uh, this is, um, this is, this role will be, uh, I, I think will leverage a lot of capabilities from gen AI tools, uh, right starting right from auto-generated tools, which can be more sustainable, less rework and delays. As an automation architect, generative AI can be another tool in the tool set with more efficient and productive, uh, you know, capabilities in making, um, automation, uh, architects can, you know, look at it. Uh, as you know, a tool for decomposing complex task also have, uh, the possibility for automating reusable code, uh, which is generated in any language.

And also providing self-service options for people or teams to automate their redundant tasks through AI coding assistance. So now, uh, in practicality what it mean, uh, for experts as well as for leadership. So obviously many companies view as generative ai, um, capabilities as part of gold, where most of the work, which requires like, let's say for building templates or infrastructure provisioning or log analysis or reuse or refactoring, the code can be done with the help of these tools.

So obviously it'll help, uh, companies with short term fulfilling skill gap software skill gap is everywhere, as we know, and, uh, the actions companies will take is more than rescaling. So they're investing in these tools, not only to fulfill the gap quickly, but also repurpose skill staff to more relevant jobs, midterm, uh, talent argumentation. So obviously I talked about like a role disruption is most likely to happen, and I've given you some examples of that.

So obviously, uh, in the midterm, you will see more and more of these injection and integration points and how human and machine collaborative models, uh, will come up, like, uh, will be explored and ensuring that the performance of the new work models are, uh, enhancing the overall productivity of the system long term. I think, uh, what would happen in a, a widespread and massive onboarding of generative AI capabilities in long term will require pragmatic approach for, uh, flexibility modeling your workforce. And o obviously, um, there are many possibilities, but the journey, um, for each organization and individual will be unique.

So if you see here, I'm quoting this from an article which I found very interesting, design, AI driven software engineering teams. So eventually maybe companies will start to explore, um, like the AI development team, the data management team, the AI integration and testing team, and AI ethics and compliance team as something which is like a shared service, shared AI service. I as a phase one approach.

So AI development team will help develop, customize, and maintain AI models, especially alms for various software development tasks. Data management team, for example, will manage the data and require training and refining AI models. AI integration and testing team will integrate AI solutions into the existing processes.

AI ethics and compliance would ensure the ethical development and deployment of AI systems going to the next step. I think, uh, what, uh, could happen is introducing the AI agents. So once, um, these tools and applications start to mature, an AI agent backed by generic or specialized large language model, um, can act as like a software development agent or a product management agent or, uh, operation support agent or research agent.

And the initial investment in developing and training and integrating AI system for software development, which is we, uh, we foresee it substantial. So obviously there is a phased, uh, approach to all this. And how do we continuously manage, update, and train our AI systems to keep it more effective, also will incur significant costs.

So we'll have to watch out how the operating model shift would happen and how soon it can happen. But we all have to be prepared for that challenge. And eventually, I think, um, it's also a very, very interesting, uh, article, the rise of AI Engineer, engineer.

So I feel that, uh, a software engineering becomes more and more accessible through generative ai. Uh, a new profession, which is the AI engineering engineer is emerging, and how that will be integrated in the ecosystem can be an interesting, uh, you know, outlook. They will sit, uh, uh, at a higher level, uh, where they will, they can train and create models and bring the glue to the software, which is being created by the AI function at the hand of the users.

So obviously, um, this is a lot more futuristic view of, uh, how the system or the operating model could look like. But in general, what will happen is we'll have to watch out for the challenges and the opportunities in this space. And I'll talk about a little bit more about, uh, how the challenges that, how to retain code quality, for example, influenced and developed by and with generative ai.

Um, inexperience programmers, for example, use the tool more than experience ones, and it could make more work for software development team in the longer run. So obviously, um, we need to be consciously supporting this, uh, integration and onboarding. Integrating AI coding assistant into the workflow has mixed impact on core maintainability and readability also.

And as per one of the reports in the overflow developer survey, less than 3% of the developers are highly trust accuracy of AI output. So obviously there is more work to be done in that space. There is another, uh, aspect of it is generative AI models from third party providers.

So obviously training models is expensive and time consuming, so there is work in progress of making smaller models more efficient. So that is another area of, you know, uh, uh, exploration, higher level of data security. And privacy can be another area where, you know, more development needs to happen.

Um, there is more work in the leg and compliance of data, how the data is being used, who owns the IPR, uh, the creative output and how, and who manages the output. And lastly, how, how services we developed and what GTM models would we adopted in the wrong. And all these are like things in making, and, you know, it'll influence how we develop our business models, uh, accordingly.

What I also feel that, um, some of the here and now challenges like build or why buy, like incrementally trained open source models or in your organization or own the retrained model, and you must maintain many more models. So it's like fine tuning of the own, uh, an, uh, open source model can also incur a lot of cost. Um, another, um, way to look at it is testing and validation of generative AI in order of magnitude more than complex and, uh, software projects.

So obviously there are some challenges in that space. So, um, our organizations are looking at, uh, a simplified answer, build or buy. And this is again, uh, something which we have to watch out for diversifying the supply chain is another, uh, area.

Um, where, you know, the foundation models are AI models are trained on a vast quantity of usually unlevel data. And some of these models are open sourced while others are proprietary. So companies have, uh, only began recently on large scale productization of these models.

And I, I think this is also a space to watch out for significant entry barriers means that large technology companies, which are already possessing substantial computation resources will have edge on everything what we do here. Concluding my talk today, I think with the advancement in technology companies will, uh, be forced to design new ways of working, to keep pace with the changing dynamics. And it's purely on us to be one step ahead.

We the DevOps people and the practitioners, it's an opportunity to pivot ahead in the curve and why not start with your own task-based framework, task based framework will allow you to assess the productivity and enhance it with new tools and capabilities from time to time, and helping you to be ahead in the curve. I have also linked some of the articles and some of the, the, the good blogs and materials, which you can also read through. Uh, if you're interested to kind of follow through the stock and in case you want to connect with me, um, here are my credentials and, uh, you can talk to me about generative AI space, how DevOps practitioners will be impacted, and in general, how the leadership challenges are kind of, you know, uh, coming up and evolving and how, how we can support the team and the, the, the overall evolution process.

With that, I come to the end of my topic. Uh, thank you for listening today and have a good day.