The Alliance Harnessing AI with Ted Stuart

This is Techron tv. Hey guys, thanks for the throw. We're here with Ted Stewart, who's president of Mission Cloud, and we're talking about a new strategic collaboration agreement they have signed with AWS and what is driving workloads into the cloud these days.

'cause a lot of conversation about what belongs, what doesn't, and hybrid and serverless and you name it. We got it. Hey Ted, thanks for being on the show.

Thanks for having me, Mike. Excited to be here. There are a lot of agreements between companies, so what makes this one somewhat different, unique and, and what attracted you to partnering up with AWS in the first place?

Yeah, so what makes this one a little bit different is it is, uh, a hundred percent focused on, uh, gen ai. So doing things like, uh, funding more, more POCs for customers, uh, providing more things like sandbox credits for us to develop new, uh, productized offerings on AWS providing more market development funds, that sort of thing. What is unique about Gen AI and what are you hearing from customers about their ability to actually operationalize this at this point?

Yeah, so it's super exciting. Um, so we've come across a kind of a number of different use cases. Um, so one of them, uh, is dubbing in translation and we're finding quite a large, uh, group of customers need that particular use case.

So one great example of that is a company called Magellan tv, and they're a, uh, a documentary company. They have about 3000 titles all in English. And they saw an opportunity that if they could translate those documentaries into different languages, they could take their library global and not, and not, we're talking about not just translating the, you know, the subtext, but translating the actual voiceover.

And so we did a project with them where, um, we tested several different models and um, and hap found for this particular use case, the AWS's Titan model happened to perform the best. And it turns out that, um, translating and, you know, changing the voiceovers to documentaries is actually much more complex than it sounds because you have things like you have slang, you have idioms, um, you have social sensitivities, you know, et cetera. So there's a lot that goes into it.

Um, so we're able to do a POC for them and we've now put that into production for them. And so they've taken their cost from translating videos, kind of the, the old way with humans doing it, uh, from $20 a minute down to doing it with the AI model that we built for them down to $1 a minute. So pretty spectacular cost savings.

Um, we're seeing similar things across, um, uh, IIDP, which is int inte intelligent document processing. And if you think about customers that are in like the medical records business or legal eDiscovery or interestingly capital markets that are trying to follow, you know, thousands of relatively small companies. And so we're able to take large volumes of data and whether you're doing legal eDiscovery or or drug discovery, what have you, process all of that information and then bubble up the relevant summarized information to analysts who then take the next step with it.

So whether it's, you know, dubbing, IDP, you know, chatbots is kind of the one you always hear about, but that's a fairly simple use case. We're seeing a lot of use cases around call centers as an example, but lots of really interesting use cases that we're seeing across a variety of customers and industries. Did you guys have an existing relationship with AWS before this agreement or is this an extension of that or how does that work?

Yes, we've been with AWS for many, many years. Um, and we had, so we've been a premier consulting partner for, I don't know, at least five, six years now. Um, we're also an ISV accelerate partner for AWS, so we're 100% focused on AWS and we have been for a long time.

Um, so we have, uh, we had originally in place and we still have a strategic collaboration agreement that covers a lot of different things like, um, industry plays or, um, going into new geographical markets, things of that nature. Um, this particular agreement was, is very specific to AI and to sort of accelerate our, our efforts in this area in particular. What is your sense of how will this play out over time?

Is, are customers gonna wind up using differing classes of AI models for different use cases and they need somebody to help them figure out what the right mix of that is, for lack of a better phrase, right? Sizing the AI models? Yeah, exactly.

And just to be clear, even though we are a an AWS focused partner, um, we are model agnostic. So whatever the customer's particular use case is, uh, typically we will, um, try out different models and see what works best. And so as an example, in the Magellan TV example that I gave a few minutes ago, uh, we tested, uh, chat GBT, we test tested Claude, we tested Titan, and you just never know for the particular use case, like what's gonna give you the best results.

So we use the open source models, we use the large models that everybody's familiar with, and in a lot of cases also you end up using multiple models, uh, because you might have one model that's really good at translating, but then you need another model that sort of is better at testing for cultural sensitivity and things like that. But we're model agnostic. We take the use case and we run it through the different models and then we, we work alongside the customer to pick which one is is best for their particular situation.

As this kind of evolves, do you think people will swap out AI models as, 'cause it every day I wake up and there's a new one and there's different capabilities. So are they gonna pretty much for the use case lock into that particular AI model for the long haul? And I guess I'm trying to figure out how disposable do we think these models are gonna get?

I think the, the models will be somewhat interchangeable for sure. And if somebody comes out with a better model, I think it'll be relatively easy for customers to swap out and go to that, that newest model. We also see the immersions of small language models and they're large and small now, and I figure someday they'll probably be medium and they'll come in t-shirt sizes, but um, or do customers need advice as to when to use what type of model?

Because they're not all the same these days. They're not all LLMs per se. Exactly.

Yeah. Customers actually need a lot of advice in that area, in particular, Mike, because, um, what we've seen, you know, having done, uh, quite a number of POCs with, with customers and so forth, and we've also had customers that have done POCs on their own and then come to us, um, we think it's, IM, so you can do POCs, um, using large language models as an example, relatively cost effectively. But then when you try to take that and actually put it into production, it it's, it tends to often not be the right architecture, the right setup.

So what we try to do with customers early on is to say, okay, let's assume that the POC goes well, and let's assume that we're gonna want to take the next step with this particular project. How do we architect it from the beginning that it's scalable and it's cost effective? And that's exactly to your point.

Uh, we may wanna choose a smaller model that is maybe more industry specific and certainly more cost effective. We definitely have to have an eye towards let's you know, first of all, define the success criteria, define the KPIs, what does success look like? And assuming we're going to be successful, let's make sure we a, you know, we, we architect something that can actually scale and be cost effective and provide the ROI.

We also try to do a lot of upfront work with our customers for them to understand the ROI the like, does it make sense for us to do this? Is it going to pay for itself? And so a lot of that is in which model you choose and how you set up the overall architecture.

On the other side of that, are customers kind of defaulting to GPUs or are they looking at other types of processors to run some of these AI models? Um, so they're definitely doing both. I would say GPUs obviously are the predominant, you know, choice in the marketplace right now.

The challenge is, you know, though is that they're very expensive. So we are seeing customers start to play around with like AWS's proprietary chips as an example. Um, the, the upside of that is that it's very, very cost effective compared to some of the more expensive GPU chips.

Sort of the, the other side of it though is that there's a fair amount of engineering work that needs to be done with the software development kit, the neuron software, the neuron SDK, to make a particular application run effectively on that chip. So there, there's, there's definitely trade-offs and that's part of looking at the, you know, what are the long-term costs gonna be and where does it make sense to put the dollars? The customers have a good understanding of the cost because I mean, on the face of it, it's kinda like, well, you know, there's outputs and inputs and you're paying for those and they seem relatively cheap, but then when you start using the model, it's suddenly you've got tons and tons of inputs and outputs and it gets expensive quickly.

It gets very expensive quickly, especially when you start talking about things like, you know, tokens up, tokens down or images per hour, you know, all of these different metrics that drive incremental costs. So I would say that most customers don't understand that aspect of it and 'cause it's quite complicated, it's not easy to understand. And that's something where we really do a lot of work with our customers and it's, it's always evolving as well.

So it's, it's constantly changing and we see a lot, we see that because we're literally working with hundreds of customers, you know, using different technologies. So we can bring that expertise and, you know, help them understand the cost. But I would think at this point in the game, very few companies really understand at a detailed level, you know, how, how to, what these costs are gonna be and, and how to, how they can manage, you know, manage them.

How do you think this is gonna play out long term? Are folks going to essentially draw a line in the sand and just add AI to their new applications? Or will they go back in and retrofit their legacy applications to add AI capabilities and trying to modernize them?

Uh, I, you know, I think both will will happen. Um, so much of what you can do with AI comes down to the state of your data. So in a lot of cases what we're finding is that we need to help customers with an overall, you know, data strategy, potentially data cleansing, data consolidation, you know, those sorts of things, you know, before they can fully leverage ai.

So I think a lot of it is gonna have to do with, you know, where is the data and, and that will, that will drive the decision. But to the extent that you can add AI to an existing application and all the data is in a good place and a in good shape, um, that is certainly a great use case, obviously if you're building something from, but if it's not the case, then maybe building something from the ground up is, is better. But I think you'll definitely see both and I, I don't have enough insight to say, uh, who the longer term winner is gonna be on that front.

Mm-Hmm. Is AI kind of finally forcing us to address those data management issues? I feel like they've been kind of floating around in it for decades and we never really tightened up our data management.

So is this the year we kinda do that on our way to operationalize AI and maybe that's what we're spending 2024 doing? I think so Mike, I mean, I think that because the reality is if you kind of have to do it at this point, because if you don't do it and your competitor does it, you're just gonna be at a severe disadvantage. And so I do think it's a bit of a forcing function to force companies to look at their data and especially when they see a use case that's very specific to, you know, their industry and you know what their competitors are doing.

Uh, you certainly wanna, you know, if you can, you know, move quickly and, and, and, you know, grab the first mover advantage if you can, as opposed to catching up to, you can also be completely disrupted by a startup that comes out of the blue that has, you know, ai. So for all of those reasons, I think, yeah, companies are gonna have an increased sense of urgency around, uh, their data and how they manage their data. Well, I can't help but think the, the people that have the data might be better off than those that don't.

So it's gonna be harder for startups to disrupt 'cause where are they gonna get the data from? Yeah, that's exactly correct. I mean, having data is a, is a, is a big part of this for sure.

We see data science teams, we see DevOps teams, there's data engineers, there's security people rolling around. How do we kind of marshal that into something that feels like, you know, the proverbial village that's required to go build these things versus I think today we did a lot of things with data science teams that never quite made it all the way to production. So is the way we need to manage it changing?

I think so. Um, what sort of interesting is, um, if you look on, what's kind of crazy to me, Mike, is if you look at cloud infrastructure now, it's almost becoming, uh, a little bit legacy. You know, it's been around for quite a, quite a while now and I think the, the resource needs are well understood and in, and in some cases there's a lot of resources that are very cross-functional and can, you know, do a lot of different things with within cloud infrastructure.

Uh, from my point of view in the AI space, it's, it's, it's much more specialized at this point. So, you know, you need for certain projects, you need, you know, ML engineers or you need a gen AI engineer, or in some cases you may need a data scientist, but there are very few folks out there who, you know, can go cut across all of the different technologies and really understand all of it. So typically what we're seeing in our projects is based on the requirement of the project, we have to assemble the right team.

And it's much more specialized, I would say, than what we see on the cloud side. Certainly there are specializations within cloud. I'm not trying to say that there's not, but it is more specialized on the, on the AI side.

So for every project that we're doing, we're sizing it up and putting the right mix of engineers in there. The other interesting thing as an example is, uh, most gen AI projects require application development on the front end. You know, typically there's some sort of a user interface that has to be built.

So with our, our AI teams that we have in place now, we've embedded with them, uh, application development engineers that can do that front end engineering work. So it's quite interesting. Do you think in the age of AI more organizations are gonna be willing to work with external services providers?

'cause there's always been a little bit of tension between the two, but is that getting better for everybody? Yeah, I think on the AI front, folks are much more open to that because candidly the, the, the skill sets are just, you know, they're very hard to find. And so I think our experience is that folks are much more willing to bring in external resources from the outside.

And at mission we had a quite a large, what we used to call a, a data analytics machine learning practice before the whole, you know, iPhone moment hit. So we were doing a lot of com computer visioning projects, a lot of natural language processing projects, all the stuff that is now called Gen ai. We were already doing those projects.

We, we have a lot of customers that are live and in production. And so just knowing from our perspective, you know, how difficult it is to find and retain those resources, I, I don't think, um, other than some very large interesting companies, um, you know, it's gonna be hard for your sort of average company to go out and find and attract that talent. And certainly the way that we've always worked with customers is, um, you know, you can either outsource a project to us and we'll define the, the, the, the scope and the requirements and all of that and, and build it for you.

Um, but also it, I think in a lot of cases, customers, they, you know, they have part of the team, but they need to fill in some, some resources on that team that they don't have. So we also see a lot of projects where we're collaborating very closely and building the right team together that brings, you know, brings to bear all of the resources that are needed for a particular project. But they're very open to working with us on AI in particular because they just, very few companies have what they need.

And also I would say very few companies just have the experience, you know, test playing around with all these different, you know, language models and, and foundational models and all the services that are out there. And to keep up with that, it's just, it's, it's evolving. It's such a rapid pace.

Um, I, I think customers need the, the outside help more so than in other areas. So is that one thing you wish that customers would have resolved or be aligned on before they engage with you? It seems like a lot of folks are experimenting with things, but when it comes to ai, is there something you kind of wish that they were a little more mature about before they started down the path?

Uh, that's a great question. Um, you know, I would just sort of acknowledge where you're at and you know, it's okay. So we work with companies that, you know, they'll come to us and maybe they're getting pressure from their board or investors to do something with AI and they just, they, they don't know where to start.

And so we're happy to do, you know, working backwards sessions and brainstorming sessions and come up with some, you know, potential use cases and then prioritize those use cases by ROI or whatever the right criteria might be. You know, or we've also, you know, worked with very advanced cus you know, customers like in the AWS startup space in particular where they are an AI company, that's what they do and they've gone out with a small team and they've built an MVP and now that MVP is working and now, now they need to take that MVP and make it scalable and, you know, so very sophisticated in their knowledge on the AI side, have a very specific use case but need help now scaling up and commercializing that application. So we, you know, our customers, I would say are sort of, you know, all over the map in terms of where they're at in their AI journey.

I think the, the most important thing they can do is just acknowledge it and then say, Hey, you know, we're either early on and we just need some, some help getting started or we're more sophisticated and we can bring the right resources to bear. Um, but you know, we, we work with customers of all flavors, so I don't think there's really a, a right answer. Some customers come with a, a very specific use case and they just need help building it.

Others come, come, come to us with a, Hey, we've got a few ideas, can you help us figure this out? And all of that's okay. All right folks, you heard it here.

You don't have to be perfect to do ai. You just have to be willing to get started. Right, exactly.

Ted, thanks for being on the show. Thanks for having me, Mike. This was great.

Thank you. All right. And back to you guys in the studio.