The Unfolding Tale of AI and Cloud Native at Cloud Native Now 2024

Welcome, and thank you for joining us this afternoon. Uh, this is Mustafa coming to live from Chicago. I help energy leaders by this is outcomes using Lab Native, and today I'll be sharing with you the unfolding story of Cloud native and ai.

Let's get a story. We will start off, uh, by learning more about, uh, the intersection of cloud native and ai. It's, uh, a unique, uh, relationship.

Uh, the, I believe they are better together, and we'll see why. And we'll dive deep into the unsung unsung hero behind all what we're seeing, uh, with the AI revolution today with some stories from the field and how ai M ml is actually shaping cloud data itself by b, making us as cloud native engineers and professionals, uh, more productive. And, uh, we'll, we'll, uh, we'll, uh, we'll recap and answer some of your questions.

You probably have botted this in the animal and channel or National Geographic, uh, species, uh, interacting with each other in a very special, uh, relationship. Uh, more of like a, a give and take. You have, we have seen the, the shark fish, uh, swimming along the sharks and what's called the crocodile bird, uh, Pickens food from in the crocodile teeth or getting any harm, and even in ourselves as humans.

And the, the microbacteria in our guts keeps us, uh, healthy, help us with the, the digestion. And actually it fits, uh, uh, our mood. Those all those are kind of special relationships.

We see them anywhere, everywhere, including in business and technology in business. Yeah. The relationship between wholesale and retailers in technologies, technologies contribute and take and give to each other.

Uh, I believe cloud native and AI are very, very similar. Uh, we're not gonna argue now who's, who's the priest and who's the bird or the, the bird with the crocodile. But there certainly, this is, uh, this type of unique relationship is, is everywhere.

We'll go to the, into the plan line of cloud data by how we got, we got here. We, we started with the muni js feature in the Unix systems. 0.

It's a way to be able to isolate cer certain processes depending on the, the, uh, the networking as well as the computation power. And, uh, after that came very lots of great projects, including Borg from Google, but that was proprietary, mainly related to what managing, uh, Google infrastructure and the, the application used by millions of peoples, including Gmail, Google documents, and like, and we have seen that 12 sector, uh, uh, uh, that principles coming from Rocco. And we started to look at infrastructure code, ICD by appliance and then with, uh, with, uh, with, uh, seeing the containers, uh, revolution and, uh, democratization of, of, uh, containers with Docker, Kubernetes, the foundation of the CHF with 200 must projects in the ecosystem, plus all comm commercial solutions.

The AI timeline is, uh, slightly older. Of course, AI is, is, is not a new concept in computer science. Started at the fifties or forties, uh, by Lan Turing and introducing the concept of, uh, o of a machine learning and ai.

And then we have seen IBM Watson take over or defeating and the game of chess and, uh, the explosion of different type of, of, of, uh, of models, language models, uh, at GBT one called the Way to work GPT-4. And, uh, it has been a fantastic experience and lots of people are trying to figure out how can I make sense of AI and what is next, uh, for me? Of course, the technology is gonna continue to evolve at the cloud native side as well as ai, but how can we make this, uh, useful for me, my business and my organization of it Throughout the, what we have been hearing from many organizations, larger, small, in the field, not only AI models are very hard and expensive to build, to build because of the competition power we need the very specialized skills and AI and me and the engineers supporting the infrastructure behind it, but they never make it to the real world.

Uh, according to together, 50% of those don't make. You can have a very specialized, uh, AI model, for example, in financial services. Uh, and that's slightly better than just a generic model trained on data on, for example, on Wikipedia, because this is gonna be related, very related to what you're doing, your business, any proprietary information related to products and services and hence is gonna serve you better.

But unfortunately, we'll not be able to take those models, uh, to the real world and get them to production. This import related to operational security and privacy, privacy challenges, I believe Cloud native, uh, has played a major role in, in, in making the, the ai, uh, revolution we're leaving now happen and can empower you to get started and get the best out of the, you already probably everyone in this call invested somehow shape or form in containers Kubernetes by building your old platform, go multi-cloud multi clusters. How can we, uh, capitalize those investments to accelerate the AI and ml, uh, Germany new organization per team?

This case, a study, for example, came from OpenAI in 2080, and it published, as you see in the CNCF, uh, cases studies, uh, website. And it highlights how, uh, Kubernetes and containers allow the team there to be able to scan and ski, uh, uh, accelerate experiments, uh, with later or no efforts to manage those AI and then workloads at scale. Another example that just came a few months ago, this is from, uh, IBM, uh, Watson X, which is an, uh, AI assistant can help you with customer care relations in a website or, uh, as a customer having, uh, difficulty pushing something online, they can ask questions.

It helps them reduce the training time from minute and a half to, uh, the average training time to around 15 or 30 seconds, and allow them to increase the number of trainings they can do per month to millions of ML model trainings and help the dev as well reduce the hardware operational courses, uh, related to those m ai and MAL models. As we talked earlier, those are very, very expensive to run, but if you run, have a good handling infrastructure and using some cloud native properly and to manage and fine tune, for example, you go and purchase GPUs, those are very expensive. How can I fine tune my operations?

For example, in, in Kubernetes, we'll talk about Kubernetes operators. Those can help you get maximize the, or utilize those GPUs at CPU for your particular AI and workloads instead of having, uh, or even if you consume GPUs, uh, via cloud services. I'm gonna talk about, about, uh, three cloud native project or tools that can help you accelerate your AI and well journey.

There are vast number of, of tech of, of tech, of tools and technologies, open source, proprietary and commercial. These days, I just focus on those three because I've seen a lot of them adopting them in the field, and they can help the, the, the, the common thing about all of that, they can help you accelerate your journey. Quick hint, uh, ML is a unified open source model serving framework to build, uh, scalable AI applications with Python.

This is an inspired by the into box. You probably have seen this before of having a meal that got everything you need it just one box. Yeah, the carbs, yeah, the protein, a little bit of a dessert.

So very similarly, it's more of like end to end. You can do serving optimization, uh, model packaging and production deployment. It plays nicely with the cloud providers, the major cloud providers we have today.

A great way to get started for your GI ml, uh, workloads. QQ flow, uh, especially helpful if you are just getting started. And you wanna run your AI MN workloads at a scale in Kubernetes, um, you can think of it as end-to-end MLOps and Kubernetes.

As I mentioned earlier, half of a a, uh, AI models don't make it to production. MLOps is the, in a nutshell, the idea of using dip ops and what will unit throughout the last seven, 10 years at building application infrastructure to make those m AI ops lifecycle easier and faster. And the Q flow can help you do this using, uh, Kubernetes.

If you are already invested in Kubernetes, you run Kubernetes workloads, you might be able to adopt, uh, cube flow and scale your BI in then workloads. That way you can focus more in building your applications and models that we talked about earlier. For example, specialized model for your financial services, banking or agriculture or, uh, uh, IRIS based, whatever the, the, the field might be, rather than just using, uh, ready AI models might be helpful to you just to get a strategy with ai, but it's not very specific to your particular use case products or services or customer needs.

Cooper, uh, is another project. It's mainly a Kubernetes operator operator that enables you to run Ray applications at the steel Kubernetes. And to elaborate more, you might ask, you might ask, what is Ray Kubernetes operator?

Uh, Ray is just an open source project to scale AI applications. You can run them anywhere you want on-Prem Cloud Kubernetes, but you, uh, we've seen this before. For example, if you go and run databases in Kubernetes, it might become a difficult, uh, operation to manage backup and restore, uh, uh, recovery upgrades and the like.

So we have that concept in Kubernetes called operator, which which usually takes away all the heavy lifting out of the picture, so you can focus on bending your models and applications. And with that, we're able to do, uh, we help you run, if you have, build three applications for your ai, uh, for, uh, for your ai, and then wanna run those in Kubernetes, you core will help you by do that using Kubernetes operators. Alright, let's look on the other side.

So this is how we can get a start to be already invested in cloud data. Very familiar with it. We'll try to make sense of ai.

And the other side, AI is, uh, is actually influencing cloud native as well. Cloud native, we have lots of, of, uh, of Kubernetes, the ML files and GitHub and others. And we can train models on, on, on those.

That's a code. That's code, right? That's first course.

So it can be trained on and we can deliver, uh, value and make sure to, uh, we can troubleshoot applications faster. Maybe we can, uh, run a, uh, clusters and scale them faster than, than today and make your cloud native connect easier than before. It, it, we know it's challenging.

There are some clients, but, uh, with, with extra work, you can optimize your, for example, cost re uh, troubleshoot problems and reduce that complexity. I will introduce some, some of the projects today. There are plenty, and they're coming up, uh, very quickly, but there are two in Pacific.

I have had experience with them to empower your cloud native journey. We talked for example, about, uh, the Kubernetes operators. If you have purchased GPUs, for example, Nvidia or Intel or cloud providers, they provide you with some Kubernetes operators to be able to utilize those GPUs to the maximum, uh, maximum results.

Two tools in particular for, for those of us that use, you know, cloud native technologies in a daily basis. Kubernetes g uh, GPT, which uses shared GPT behind the scenes by integrating with its API to help you, uh, explain and understand Kubernetes in plain English. The other project is a co-pilot, and I know you have seen co-pilot everywhere of Microsoft Co-pilot and gi, the GitHub co-pilot.

This is very similar, more of an assistant. They help you when you are running, uh, Kubernetes commands. If you wanna analyze something in a, in a part or if you wanna troubleshoot, uh, issues.

This is a, a quick, uh, uh, a screencast of what, uh, Kubernetes GPT can do for you. Uh, you, you can think of it again as an assistant, your, your running your Kubernetes commands, uh, and you can understand some of it in, in plain English. You don't necessarily have to understand all those complex e ml files to be able to explain or troubleshoot that issue.

Copilot, uh, runs very similar to what you do in, in, uh, in Q-C-C-T-L, uh, but with extra features that allow you to, uh, monitor, troubleshoot, and, uh, do some security, uh, scanning and, uh, of, of your images and, and containers and more, more of those tools are, are, are available today. I'm, I'm common almost every week. This is just the tip of the iceberg with those such tools.

You don't necessarily need to have to download plenty of, of different tools to do infrastructure as code monitoring and lobbying and the like. You can use such tools, even if you don't have the much experience with, with Kubernetes containers or may be intimidating at the very beginning. Those can help you throughout your cloud native journey and make it faster and streamline the process.

I'm hoping after this, uh, uh, this session, you will consider cloud native AI to accelerate your AI and mal journey. Especially if you already have made investments in Cloud Native over the years. It can help you streamline your journey given all the great tools and projects we use and love it.

The CNCF landscape, including containers, Kubernetes, helm, and the like. I'm gonna leave you with some, uh, learning resources. Um, the cloud native AI white paper from the CNCF, that's a great way to start in your journey.

You understand, very similar to, to what I discussed, the history of cloud native as well as, uh, ai and where, where is the intersection between both, how can you use, uh, cloud native technologies and those investments you made in containers, Kubernetes, multi cluster, multi-cloud to get started your AI journey. And we all know AI is a very, very big topic and we cannot cover it in a session or two or or hundred. There are ethical aspects of ai.

There is a technology aspects of ai I highly recommend to consider this book for, uh, Mustafa Soleman. Uh, he's the former founder of a company called Deep, uh, deep, uh, mine that was acquired by Google. And currently he's sitting as the CEO of Microsoft ai.

You will get a, a good understanding of where we are with ai, where are we going, and the other aspects and things we have to think about when we build AI applications and, uh, implement, uh, solutions related to AI and our organizations and, uh, teams. The key key takeaway three takeaways from, from the talk today, I would like you to, to look and, and consider cloud b ai, especially if you are just getting started with AI and then workloads your position and you already invested in cloud. I want you to consider innovating and feeling fast and get feedback in your experiments on model and AI ML workloads, all the tools I mentioned earlier.

Uh, AI or Cloud native are supposed to get you there and make you able to innovate and fail fast to get the feedback, do more experiments, get them to production, learn from those mistakes and make it better. And, and, and, and the cycle goes on. We're all new to this and trying to figure things out.

The more we learn from each other, including success, failure, stories that matter, we can harness this technology. I want to especially turn the tech strong team for organizing this a as well as the sponsors and most importantly you for taking the time to join us today. Thank you, and please feel free to reach out should you have any questions or could be of any help.

Thank you again.