Shaping the Future of Data Team Infrastructure – Erik Bernhardsson, Modal Labs

This is Textron tv. Hey everyone. Welcome back here to techron tv.

Our next, uh, guest is his first time here on Techron tv. It's a new company we haven't covered here before. Say hello to Eric Bern Hartson.

Eric is the founder and c e o of Modal Labs. That's M O D A L Labs. Hey, Eric, welcome to Tech Drunk tv, man.

Great to have you on. Thanks. Thanks.

It's great to be here. Eric, we wanna hear all about Modal Labs, but before we jump into Modal, let's hear a little bit about Eric. You know, tell us kinda your story.

Yeah, I, I, I grew up in Sweden. I, uh, when I was young, I did a lot of programming competitions. Uh, a lot of my friends from back then who did that ended up starting at this obscure, unknown music streaming startup called Spotify back then.

Mm-hmm. I was just based outta Stockholm, so I ended up joining two, uh, ended up spending seven years at Spotify, some point moving to New York. In particular, I built a music recommendation system at Spotify.

Open sourced a few things, uh, then left Spotify, uh, joined a, a now infamous company called Better, where I was the C T O for six years. Very cool. I say a crazy rollercoaster.

And then in 2000, uh, 21 I started working on with Snap Modal. So Eric, you know what, I've been in tech 30 something years and serial entrepreneur myself, no one decides, Hey, I just want to go start a company. You've gotta feel it like deep in your guts, you know?

And what, what was it that kind of drove you to found to start what, you know, what is now modal Modal Labs? Yeah. I mean, I, I think first of all, I'm kind of a startup degenerate.

I always worked at startups my whole life, and I, I just loved mm-hmm. Sort of, you know, getting a bunch of smart people in a room and just like coming up with ideas. But in particular, why I ended up working on Modal was that most of my career was always very focused on data, right?

Like the, you know, whether it's like large scale numerical methods to data analytics, business intelligence, machine learning, ai, uh, a lot of that was always what I did. Uh, so I started thinking about like, the, the space in late 2020 realized, there's so many different startups, there's so many different products, but still, like data teams are kind of struggling to just ship like basic stuff into production and, and building things and scaling things out. And, and, and, and then entire data stack seems like, you know, massive area where I, I really think that there, there's a set of tools that could make data teams a lot more productive.

It could enable them to build all these apps much quicker. Uh, so I started thinking about what that would look like in late 2020. I realized in order to do that, I had to go pretty deep down and like do a lot of the, like, the, the infrastructure late first, I decided that's where I wanted to start.

So that was sort of the, the genesis of, you know, what's now what, what is now modal, at least like where we are today. Very cool. Excellent.

com Alright, I just wanna make sure in case we forget we got it out there. Um, so Eric, three years kind of in the making, why don't you, you know, we can't go day by day, but why don't you give us sort of the condensed version of, of, you know, over the last three years, what a little bit of modals journey here and now. Yeah, totally.

'cause it's actually shifted a little bit. Like, I, I still think, you know, we believe in the same vision. Like the, the vision was always, we wanted to make data teams more productive.

Uh, and we wanted to build a whole suite of tools that kind of rethink a lot of the data stack. And, and in order to do that, we had to go and, and focus on the infrastructure. We started thinking about it from the point of view.

It's hard to run code in the cloud. And so like, what should that experience look like? And, and, and in particular, we, we wanted to build something, or it was just me at that point.

So I wanted to build something that basically takes code and makes it run in the cloud with the same experience that you have from running code locally, right? Like, like almost like this magical experience of like, it feels like it's local, uh, but, but it's actually in the cloud. Uh, and so started thinking about like, what that would entail.

You don't, you know, like a lot of traditional tools, like when you like, want to run something in the cloud, you have to build a container, push the container, then go and trigger some cloud console download log. You have this like very annoying feedback loop, uh, and realized in order to do that, we basically had to build our own containers engine. We had to build our own file system and a number of things in order to make that loop, like that feedback cycle, like very fast to have the developer experience we wanted.

Uh, so it's pretty quickly, uh, this turned into a, a, a pretty hard engineering problem. And we spent the first year or two basically kind of rethinking, like, we can't use Docker, we can't use Kubernetes. So like, we have to go, we're gonna have to rebuild a lot of these things in order to deliver on that user experience.

And building this sort of, I think of it as almost like a W ss Lambda, but like more targeted towards like data needs. Um, so in the process, supporting GPUs, supporting long running jobs with a lot of resources and really like abstract, abstracting all the infrastructure away from customers. Like everything runs in our cloud in in a sense that like we are the cloud provider.

Like we charge people per usage. Um, and, and we handle all the billing. Uh, and, and, and when you use model, you don't have to think about any configuration, any infrastructure.

It just like, takes the code and runs it in the cloud. I think, I think what, what what's like, what, what's evolved the most over the, the, the three years we've worked on it is, is, is more around use cases. We, we built this kind of, almost like naively in hindsight thinking we're just gonna build a replacement for Kubernetes, or, or, or, or something that data that's so good that, you know, you you, you just wanna run all your code and model.

It turns out it's actually quite hard to, to convince enterprise companies to, to just switch something over. However, about a year ago, uh, uh, we started seeing all these new companies going after more like AI use cases, like focusing on G P U and, and, and, and running a lot of, you know, a lot of these like very, uh, I mean, you, you've seen all that, you know, stable effusion, dream booth control, like all these like very cool applications based on sort of next generation, particular generated ai. Mm-hmm.

And, and so, so we started seeing very, a lot of traction in that space, uh, about a year ago. And, and that's now effectively like been driving a lot of our, uh, usage is, is really sort of capturing that, that audience, uh, lot of companies who run Gen AI at large scale is now using modal, uh, and, you know, running things like very large scale, you know, thousands of GPUs, uh, stable diffusion, like generating images, but also to some extent video, music, audio, audio transcription, uh, speech synthesis, just a lot of those types of things to some extent language models too. We do have other exciting use cases too.

So we have a couple companies using us for protein folding and, and other types of Really That's very cool. Yeah, like, so, so video processing, we, we have a bunch of people using us for things like web scraping or, or code generation, code execution. So, so there's like, actually there is a wide range of use cases, but I, but I would say like a lot of our core focus right now, right now is really like delivering, you know, that this magical experience when you wanna build gen AI applications, It's a good time to be in that, in that space right now.

Um, you know what we should mention, you guys, you raised the seed round, it was back in 22 or something, right? Yep. Yeah.

And you recently announced, uh, an official series a, a pretty healthy series A. Yep. Yep, that's right.

Yeah. Super excited about that. Uh, we didn't raise some details.

Yeah, yeah. We, we raised our seed round from Amplify. Uh, we raised our, a round from RedPoint.

I, I mm-hmm. Spent a lot of time with both, in particular Amplify, but also RedPoint, uh, I mean, they're, they're like, they, they love data, they love infrastructure and dev tools. Like, you know, you sort of, everyone you talk to almost over there is, is a former engineer now VC joined the dark side.

Mm-hmm. Uh, so, so this sort of get it, uh, which, you know, I, I think is very exciting. Um, and, uh, yeah, I mean, you know, we're, we're, we're not a profitable company.

Like we, we, we need money to grow. Like we're building some very hardcore technology and it's Long time. Yeah, Yeah, exactly.

Yeah. I mean, that's the point of a startup. Like that's why, you know, venture capital exists.

So, so, so we're raising this money now, um, with, with the sort of, you know, in, in this situation where we are, we're starting to see a lot of demand. Uh, we think it makes a lot of sense to hire more people and sort of accelerate the product development roadmap to, to gain more market share and capture more customers. Sure.

So, so, so I, I think it was a good time to, to raise, you know, an, an additional absolutely. Capital injection. Look, anytime you could raise money is a good time to raise money.

Um, so congratulations to you on that. Thank You. Let me ask a question.

Do you see sort of generative AI as the generative AI applications as the engine that drives modals growth? Or do you see it maybe going full circle where, hey, that that may get you there, but you may eventually come back to your original kind of vision, you know, in terms of Kubernetes replacement and stuff like that? I, I definitely think the latter, right?

Like, I, I always started this company with the vision that I wanted to kind of rethink big parts of the data stack and, and starting with the sort of Kubernetes layer of like running code, right? And, and we did find a very strong niche in Gen ai, but, but gen AI didn't really exist when we started. So like, I, I always like, thought about this, like, what would've, what would we have done if that hadn't happened?

I think we would've found something else. Like, as I mentioned, we do have use cases in biotech and other use. So, so I, I think right now, like the, I'm obviously very happy that gen ai, like has been driving so much demand.

Um, I see a it as like, okay, we have this, now this position in gen ai, we have this web into the market. We have a lot of like, really, you know, large scale customers. For me now, the next important thing is to kind of widen that, you know, that that u that set of use cases, like trying to move into other areas, like other adjacencies, we're starting to look at more like enterprise customers too, starting to get a couple of exciting enterprise customers, ramp and scale business are using us.

Uh, so, so there's a lot of different ways we now want to expand, like from the position that we now have, the position with it, which is generating a meaningful amount of revenue. Uh, but I think, you know, who knows, like maybe in, in two years we'll be tired of gen AI and something else is cool. And in that case, like, I don't want to be, you know, I don't wanna tie myself entirely to that trend.

I think it's very cool. You know, I, I most likely it's here to stay. Uh, but I think it's very important to me to also establish ourselves in other, in other types of use cases.

All technologies never die, right? They just, they get, they get subsumed into whatever the next cool thing is. Yeah.

Right? And that's, yeah, kind of the way it goes. Let, let, let, so look, we got people out here in the audience, they're developers, they're DevOps, other kind of folks.

Eric, like, how do people come to saying, Hey, you know what, this is a great use case for this company Modal I heard about, you know what I mean? How do they engage? How do they, how do, how do they match up to you?

I, I think our typical customer right now, the, the one we do best with is a customer where their focus is to build machine learning or AI models. Like they have, you know, some custom model or some use case that they're, you know, that they're really excited about and they built and then, and, but they're not infrastructure engineers and they don't want deal with the cloud. They don't want deal with, you know, the configuration, all the ops, like all of, you know, the, the DevOps stuff they wanna focus on like the application, right?

Uh, and, and, and in a way it's like, even if they know it, that's not what they wanna spend time on, right? Like I, I have, you know, I've used a w s for 15 years and I love it for what it enables me to do, but like still after 15 years of a w s like, I'm banging my head against like the console, like every three days trying to figure something out, like something basic. So, so in a way I'm building a tool even as like a person who knows a fair amount about DevOps.

I'm building a tool that I always wanted to have. Uh, but I, but I would say like the, the main use case that we're seen right now is, is really people who are building some custom machine learning or some AI cool stuff. Like they have some, some model particularly for inference, like, and they want to deploy that model, and they don't want to ha think about scaling.

They don't want to think about resource management, provisioning, like configuration. They just want to take that model and just like, get it running in the cloud. And, you know, they want the ability to call that model and get, you know, predictions back.

And, and, and that's the level of like how they think about infrastructure and, and that, and, and we handle all the rest. It's kind of like automating their infrastructure needs. That's Right.

Yeah. And, and a long-term vision that I have is that a lot of developers, I think, you know, in, in, in three to five, maybe 10 years, won't actually have to deal with the clouds directly. Like a w s, uh, G C P, Azure and, and, and other clouds.

They're actually kind of a low level abstraction. They're pretty hard to work with. And, and I would say the same thing with Kubernetes.

And so like, a vision I have is like, give it a few years, and I think you're increasingly seeing this, a lot of engineers don't want to or shouldn't have to interface with these things directly. Like they should use higher level abstractions that, that focuses more on the ability for them to build applications. And we focus on that for data teams.

Like data teams need better abstractions. I, I'll tell you a funny story. I went to DockerCon maybe eight years ago, something like that, seven, eight years ago in Austin and everybody there, it was when Kuber, Kubernetes one oh wasn't out yet.

880, or something like that, you know, a pre official release. And everybody was talking about Kubernetes and every booth was what, you know, had their Kubernetes story came back here to the office and I told my team, look, I saw this thing down there. Kubernetes, it's hard as s**t, man.

I don't understand whoever's going to use it. This isn't going anywhere, you know? And which is why I'm still working and not retired, I guess.

But I never, you know, to me Kubernetes was always too hard, Right? It is, it is very hard. Uh, i, I, I sort of see why it's like still found a niche because it, because it is quite useful, at least for backend developer.

Uh, but when, when, when you, like, what, what I think also happens at a lot of companies is then, you know, the ops team then tries to push Kubernetes on the data team. And that's where I think there's an even bigger mismatch between what the data team wants to do and what the, their, their internal platform team offers. And, and so I, I, I think that, that the chasm is larger and data teams, but even for backend teams, I think like Kubernetes is like, you know, long term.

I'm not so sure it is pretty Well, but you know what, look, you can't argue with success. It certainly has become, you know, a defacto standard. Yeah.

Where on the other hand, I remember the first time I saw Solomon hikes from, uh, from Dock Docker talk, and man developers were eating that up, right? Developers loved the idea of, of, you know, what Docker had done with containers and containers weren't, wasn't a new concept, right? We, we've had containers before.

There was a docker and stuff like that, but it was, it was a very different, you know, like people just got that, you know, and they, and they, no, no pun intended. They swarmed to it, right? And, uh, it would, from so that I understood Kubernetes never quite did.

But Anyway, even containers. Even containers, I would just say, uh, it, it, like, I, I think that the, the core idea is very good. Uh, but, but, but something like Docker, like I would actually argue like the developer experience was never that good.

And, and I think that's very clear in the sense that like a lot of data teams and a lot of backup teams actually don't use talker, right? Because it it, or they don't use it as the like normal development workflow, right? Like they use it maybe like when they deploy code and then it's this like annoying thing that you have to deal with, but like, it has still hasn't like won over like, you know, the core development feedback loop.

Like people don't run code in Docker while they develop the code, which I, I think is kind of lost opportunity. Uh, 'cause the developed experience is kind of, It certainly adds another step to it. Um, Exactly.

Yeah. Um, What about platform engineers and platform engineering? How does this play into all that?

You guys hear anything from modal on that? I, I, I think, I think my, my feeling is like whatever the platform engineers are are doing today, that, you know, the, the goal is always to, to make that obsolete in the next five, 10 years. Uh, it, it doesn't necessarily mean like, you know, I wanna, you know, make, make all the platform engineers unemployed.

Uh, but, but I think, you know, a lot of platform engineering, especially like when you look at like big companies, it's like just taking a platform that already kind of exists and then like tailoring it to their unique use cases. Uh, and that's the type of work where like, I feel like, well, if the underlying platform was the good to start with, like why do you need to like tailor it? So I I, I'm almost like a little bit skeptical.

Like, I, I think a lot of what happens that, you know, if, if, if you go look at an average company and look at like the platform team and what they're doing, my feeling is always like there's gonna be a startup or a product that does exactly that. And so in five years, like whatever you're doing now, like you're gonna have to find something else through. That's pretty much how tech is.

But you know what, I wouldn't be surprised if we still have platform engineers, but what they do changes and that, that's kind of how it is, right? I Think so too. Obstructions, move up the stack and like you, you find higher levels of things to, to build platforms on top of.

And, and that's always like, I think the case. Absolutely. Um, Eric, we're about outta time.

com, M O D A L. com and hey man, this fascinating stuff. Keep us posted, come back, you know for sure.

Keep, look what's going on. Eric Bharon, founder, c e o at Modal Labs here on Tech Drunk tv. We're gonna take a break.

We'll be back in a minute.