DBOS’s Mike Stonebraker on Replacing Linux and Kubernetes

This is Textron tv. Hey guys, thanks for the throw. We're here with Michael Stoneberg, who is now leading an outfit called Deboss, and we're talking about the convergence of databases and operating systems, and the cloud and functions and all kinds of fun stuff, but they just raised some additional funding, and we're gonna dive into what all that means.

Michael, welcome to show. Thank you, Michael. Glad to be here.

So what is it you guys are trying to do? Because, well, we've had databases and operating systems, you know, tangentially related all these years, but what's changing and how come we need to rethink all this stuff? Okay, uh, the, well, this, this, uh, deboss project started in 2020 as the research project at MIT and Stanford.

And we were motivated by two, two points. The first was, uh, Linux is very elderly. Uh, I, and I first used Unix on a PD DP 1140, uh, one processor.

Uh, this was in 1974, uh, 48 K, not m or g of main memory, and 20 gigabyte, 20 megabytes of disc. Uh, one of the environments in which, uh, DeVos runs is the MIT Supercloud, which is 32,000 processors, several terabytes of main memory and many terabytes of secondary storage. So that means the resources that the operating system has to manage has gone up by about six orders of magnitude, uh, you know, in the last 40 ish years.

So, uh, without me saying another word that makes managing operating system state a database problem, so, uh, and Linux, of course, knows nothing about databases and Linux, uh, is elderly. It's having a hard time making forward progress fast. So, for example, there's no multi-node version of Linux.

You've gotta run something like Kubernetes. You get a fairly complicated stack. So, uh, basically, uh, it's time to send Linux to the home for retired software.

Uh, and so that's what we are attempting to do. That's number one. Uh, and so we have a commercial version of a replacement for Linux, and to, no one's surprised if the managing state is a big problem.

You wanna run a database in the kernel, and you wanna basically, uh, run everything on top of a database. So the way to think about deboss is you're familiar with running an operating system on top, uh, underneath a database system. That's the way everything runs these days.

We do exactly the opposite, which we run the operating system on top of, uh, the database. And so, operating system services are written in sql, and this puppy is fast enough, and it has much, it has great properties. Uh, so for example, uh, if the database is the only thing running, uh, then one of the things that the database does is logs events.

That's what databases do. So we capture all events and we log them into transactionally, into a data warehouse. So if you'd like to back up the operating system 10 minutes, uh, just go ahead and do it.

So we, we support time travel in the operating system, and if the, uh, and if the database is fast enough to manage operating system state, it's fast enough to run your application state. So you should put your application state in the same database, and then you time travel everything. So if there's a ransomware attack nine minutes ago, you just back up everything, 10 minutes, single step around the, the intrusion and keep going.

So it has wonderful security properties. So that's, that's number one. We, you know, uh, deboss is an attempt to displace Linux and Kubernetes, uh, and a bunch of other, you know, like if you're running a transactional file system, well, that comes built in because databases are transactional.

So it basically simplifies the operating system stack a whole bunch. So, and that's just plain goodness. mm-Hmm.

Uh, and it's much more secure. But then the other thing that we do is, uh, in terms of how do you write applications for this operating system? Well, we could implement po, uh, which is sort of yesterday standard.

And in the modern, uh, cloud oriented software, the service oriented world, hardly anybody cares about ics. So we have a new programming environment, uh, which is, uh, transactional. It's, it's basically a TypeScript environment in which everything goes into the database, uh, and it's software as a service.

So you structure your application as a collection of modules that are connected in the graph. Each module is transactional. Each module is durable.

Uh, so this creates a very nice programming environment, uh, that's database oriented, transaction oriented, and, uh, and in my opinion, a a great, a great way to worry about, um, not doing your applications. So we have two things. We have a new operating system, and we have a new programming environment, uh, that, both of which we're very proud of.

Much to unpack there. Was it not possible to think about just upgrading or modernizing Linux in a way that would add a database capability to it? Or do we need to replace the entire stack?

Well, in my, in my opinion, uh, in my opinion, Linux ought to do that. Uh, but that still leaves it, uh, with mountains of code that are, that's a leaky boat, security wise. Uh, and, uh, that would, that would allow you to, since, since if you upgrade Linux with a database, then, uh, the database is multi-node, uh, and you've now got a bunch of engineering to make Linux multi-node.

So it's, it's considerable amount of work. But in my opinion, uh, my opinion, all future operating systems should have a database at the bottom. And there's a, a good reason for that.

I can give you two quick vignettes. Uh, the first one, which was occurred in 2020, which was when we were just about to start Deboss, and I listened to a talk by Mattea Zaria, who was the founder of Databricks, of course. And Mattea said, uh, on a routine day, Databricks, which manages spark instances in the cloud, uh, Databricks manages a million, uh, spark subtasks.

And he said, scheduling a million things using conventional technology. Just, you know, is, is a non-starter. So, uh, Databricks and a bunch of other big, uh, cloud properties, uh, managed, uh, state and managed scheduling out of the database.

So Matay, uh, Databricks is already using some of our ideas. And of course, uh, Matay started whining about, uh, Postgres performance, which is what they were using for scheduling. So I said, we can do better than that.

So, first, first, uh, thing is that big properties are realizing that you can't do operating system scheduling. You need to do scheduling, scalable scheduling. Uh, the second thing is, you know, Uber, uh, the guys that, uh, drive you around in the city, uh, they have a programming environment where, uh, they have a collection of schemas.

And you, the way you share data is you add columns to this, to this schema. So everything, all sharing goes through the database, which is exactly what we're advocating. So forward, forward thinking companies are already, you know, using some of our ideas.

We just, uh, make it all simple and easy. In future, will I also need a, a separate database still for my application? Or is that something I'm gonna invoke directly from the new operating system?

Uh, the way deboss works is that we store operating system data in a thing called Foundation db, which is open source. Uh, our investors said, you've gotta have an open source offering. And so we do.

And Foundation DB is not a relational database system, so we thought users would balk at putting their data in it. So we allow you to put your data in any Postgres compliant DBMS, pick your favorite poison. Uh, and from my point of view, all of the big, uh, cloud properties are standardizing on, uh, Postgres wire compatible DBMS.

So run, run, uh, cockroach, run citus, run, gigabyte run, RDS run, Postgres, run whatever you want. Uh, and it's in the kernel. So the answer is your favorite database system should go in the kernel.

Uh, and, uh, from our point of view, we're happy to be user database agnostic. Uh, and if, if, uh, if it comes to, if it comes to it, we will write a database system at some point if that's turns out to be a, a good business decision. Architecturally, have we kind of painted ourselves into a corner because we have Linux and Kubernetes and all these things and, and stitching them together, and, um, managing them adds a level of complexity.

And we invest in observability and monitoring tools in the whole stack seems to have gotten fairly weighty over the years. So, you know, at what point do we kind of, um, have to do something? 'cause the current pressure of the stack is just too much.

Uh, that's another reason to move to Debo. So all that stuff goes away. So if I'm managing the environment and all that information is in the database, then am I just launching a, uh, a SQL query to surface that?

Or, because today I have to kind of use various querying tools, and a lot of people don't know exactly how to make that, uh, work because, well, they're all written in these kind of proprietary formats. Exactly. So you could give my pitch, excuse me.

So all operating system state is in the database. So if you wanna know anything about what's happening, you just run a SQL query. So if you wanna know how many users are chewing up space, only counting, uh, uh, files that are bigger than a hundred gigabytes, that's just a SQL query.

'cause, uh, remember the way deboss works is we have, we have a message system. It, it's written in sql. So there is a message table with a sender, a receiver, and a payload.

And to send a message, you do a SQL insert into this table. Uh, and that table is partition. So it ends up, uh, the, the bites in your message end up at the home node of the receiver who reads a message by doing a SQL query.

So that's all you have to do. And you can, since the message system is a table, anybody can query it in sql. Uh, and if you wanna, if you wanna say how, uh, if you have a suspected bad actor and you wanna say, tell me everybody who's sent the bad actor a message or received a message from the bad actor in the last two hours just to sequel query, whereas today, uh, it's essentially impossible to find that information.

You talked about the cybersecurity implications of all of this. Um, do you think that they'll be involved in this decision making process or this transition, or are they gonna exert a lot of influence on that? Or is it more like a, a, a downstream benefit that they'll appreciate one day, but today they're not that involved?

Well, here's my point of view, which is everyone is moving everything they can to the cloud as quickly as they can. And so one way to deal with the cloud is to do a lift and shift. Uh, and if you do that, then the successor in your job will inherit all the problems you currently have.

So, in my opinion, you have a once in a generation opportunity to make your software system better. Uh, and so adventuresome enterprises, I think, are willing to refactor, rewrite, uh, put green, uh, do green fields in a better way. So our point of view is we are appealing, uh, primarily to say deboss is only, uh, runs on the cloud.

Uh, we will offer an on-prem solution if we have to at some point, but the cloud is where things are going. Uh, and on the cloud, you are highly encouraged to run software as a service. Uh, all you have to do is look at Amazon pricing, uh, to figure that out.

And so if you're gonna move to a software ser as a service environment on the cloud, uh, it's just natural to run our stuff, uh, rather, and if you're gonna refactor what you're currently running, uh, you might as well refactor it into us rather than into something else. So, uh, we, we are, we are planning on, uh, marketing to adventure adventuresome enterprises as they move to the cloud. Also, the three letter agencies, uh, love our security story.

Mm-Hmm. Uh, and also, uh, financial services folks, uh, love us for a different reason, uh, which is, are you familiar with the term Once and only once? Um, it's been a while, but yeah, I seem to remember hearing that a a while back.

So just for example, if, if I want to, uh, give you a hundred dollars, uh, you, your finances are presumably in a different system than mine. Uh, and this application was motivated by a large, uh, northeast regional bank who we talked to at, at some length, and are plan, they are planning to be an early, uh, deboss user. So the way they, the way they do it is they, they debit my account, then they send a message to your system, they increment your system, they, uh, get a return message back to my system, and then you want to commit the transaction.

And only then, uh, and of course, your system and my system, uh, in the banking world are unlikely to implement xa, meaning they're unlikely to implement distributed transactions, which is what you need to do once and only once, which is this, this whole, this whole saga either, you know, uh, runs to completion or it looks like it never happened. So because we run the message system, uh, we can do once and only once, uh, banking transactions, and that's very attractive to this particular regional bank. Uh, they figure that, uh, somewhere between a third and a half of their application code is making sure that this stuff works.

Uh, and so distributed transactions are not for the faint of heart, and if you do them an application code, they're brittle. There's a lot of code they tend to screw up. So this regional bank would love to get rid of all that.

Uh, and so they, they are incented to move to a new environment by the ability to, to do once and only once, uh, in a much cleaner way. Hmm. I'm suddenly having memories of misadventures with two-phase commits and all that fun stuff.

Right? Exactly. All right.

So today, everywhere you turn around, there's these massive investments in ai. Um, and, and a lot of that is in the realm of AI ops. Um, are we investing in AI as it relates to IT operations to compensate for the complexity, but maybe we should be going the other way and just reducing the complexity, Complexity, uh, to the ex.

Uh, I think, uh, you said it perfectly about 10 minutes ago, which is the current operating environment that people are trying to manage is ridiculously complicated. Uh, you need a Linux administrator, you need a Kubernetes administrator, you've gotta make sure that, uh, your Linux settings and your Kubernetes system, uh, settings don't conflict. Uh, you may be running a transactional file system, you may be running a separate high availability system.

Uh, you're probably investing in two or three, uh, security oriented monitoring products. Getting all that stuff to work correctly is daunting. So we promise a much simpler life that we'll make.

Uh, it'll make operations a great deal simpler. Uh, and, uh, from my point of view, that's totally separate from chat, GPT and other large language models, which are basically decision support like things, uh, and we are not particularly focused on decision support since they are, they are often don't care about transactions, don't care about high availability, don't care about recovery and so forth. All right, folks, you heard it here.

The principle is still the same, right? Keep it simple, stupid, and you have a happier life. Hey Michael, thanks for being on the show.

Oh, thanks for your time, Michael. All in.