Taylor Thomas and Bailey Hayes, Cosmonic | DevOps Experience 2022

Hey everyone, and Welcome to our emerging challenges in distributed devops panel. I'm Taylor Thomas. I am a director of engineering at cosmonic and I just a quick introduction about who I am, but what I kind of background come from I'm a recovering infrastructure engineer.

I do a lot of back in engineering right now a lot of interests a lot of go before I was a core contributor to helm for a very long time. I did a lot of cut like writing of the code of Helm 3. I've done a lot of things with kubernetes built kubernetes platforms at 3D companies and really kind of jumped on the the containers and kubernetes bandwagon way back at the beginning.

So that's a little bit about me and I'll just hand it off the baby so she can introduce herself as well. Hi, my name is Bailey Hayes. I'm a director of Open Source at cosmonic.

I have recently joined cosmonic before that. I I wore many different hats. I think I'm probably coming more from the dev side than Taylor working on distributed applications mostly written and go but I I am a recovering spring boot developers.

So there is that I guess we both were all recovering in our own ways. Right? So I'm also part of the team because I have been a blossom champion from the very beginning and started working with it's even precursor.

So earliest 2012 so been in the space for a while and really really love talking about it. And one thing that Taylor and I both have been thinking a lot about is around software delivery, especially when it comes to Wasim or webassembly. And this is where I usually pass off to you Taylor Thomas as our common joke is pedal has vindals.

Yeah, so that came about because I'm often talking about another project that I am a co-creator. I've called bindle that is a basically a new way of storing artifacts and it's really just meant to be This idea of what we call a silverware drawer for software because most the time when we put things together like they're not necessarily the exact same thing, but they belong together. And so that's what bindil is is.

It is really just meant to be like object storage for a whole a whole whole new way of doing things specifically in webassembly. And so we call that aggregate object storage and the ideas that you store these closely related things together like you do in the Silver Bridge or you mean your forks and your spoon sometimes other times you just need a fortnight. Sometimes you need this fancy soup spoons and all those things are the kinds of things that we we work with they'll see.

Yeah, we've been in this space for a little bit like I created this couple years ago and part of what is called Sig registry with Bailey. It's part of the by code Alliance. So we're working towards to finding how we want to store webassembly artifacts and how they can be basically recomposed together depending on your needs.

So it's kind of an interesting space. We're in just something else that we're kind of both working on right now. Yeah, so you're probably wondering I mean, I guess everybody has stood up a Docker registry.

So when you when you kind of think about that in terms of quantum, it already works right with with oci. There's there's several different ways of making that work. But but that's already available.

What we're kind of talking about is rethinking the software supply chain from the very beginning and how do we understand all the capabilities that it wasn't modules allowed to do from the very beginning so that say, you know, the threat level of the given module way before you ever run it, you know that it needs access to sockets, you know, it's gonna make requests over http. Or maybe it doesn't maybe it doesn't do any of those things and it's just super safe and it's just a single unit of compute being able to understand those types of aspects of a given program opens up the door for a ton of two cool new opportunities. But I think maybe we should take a step back and talk about you know, hey, this is devops everywhere.

Why are we personally motivated to work with devops and I'm just gonna throw that straight to Taylor. Yeah, so this is a subject. I'd rather passionate about having been part of the Part of the the whole devops explosion that came around with kubernetes.

And I I always do this as kind of a story of what happened here. We kind of have had this pendulum swing back and forth in if you go back to like the late 90s and early thousands you had this thing that we're developers would create their applications packaged it up into the trash bag and throw their trash right over the wall. And then the Ops Team would just be responsible for it.

That was a common refraine. I've actually had been on teams even later than early 2000. So where they were they were still doing that like someone just throw the application over the wall, and then they just had to handle it and it was kind of the old style of doing it was all kind of the Ops teams deal to run that and then we came around to kubernetes and Docker and everything swung like completely the opposite direction and you ended up with this idea of well now as a developer, I have to know what cluster I'm running on where my cluster is which crds I have installed what databases I expect to have all of this stuff that you had to know about and then the Ops people had to worry about anywhere devops people as they were called by that point had to go and saying well I need to know exactly what this person's running and I need to know what their memory requirements are.

I have to know what dependencies they have and it's just a big mess because now everybody has to know everybody else's job. So everybody does it poorly the section that isn't theirs And so that's where we've kind of like swung back and forth on this like pendulum of devonops and we're talking a lot about how this works in this distributed like devops and what things gonna do we're gonna really focus around like webassembly and how this can kind of bring it back to the middle to allow developers to focus on the things that matter to them. In platform Engineers srees devops people to focus on the things that matter to them.

Rather than either siling them entirely or having everybody everything and everybody be everybody's business. And so we're trying to do that with what we've seen in webassem. We're going to talk a little bit about that today as we mentioned a few things.

Sure, so as a developer, I know that it's basically my responsibility have a good idea how to build my application and create that final release artifact. Let's say it's a Docker container. Maybe I also have to build it for multiple different platforms since that's maybe the requirement of where we distribute.

So I package all those things up. I know, you know General ideas about how the system works what it's gonna run and what conditions it'll run under but I would say that from a general perspective as a developer. I'm gonna be like Hey Taylor, how do I do cross region?

How is this supposed to work cross Cloud you help me now, right? And then what do you say Taylor? Yeah, that's that's a classically when we were talking about this as we were talking about how the kind of conversation we wanted to have.

We're thinking like well, Like no matter where you run something like devops SRE those kind of things are required no matter where you're running in but like in today's Paradigm, no one can actually do those cross cloud or or even cross region things very, well. There's plenty of people who have data centers that they still want to use for various reasons, and I personally used to always think cross Cloud was a pipe dream because I would ask people and I'd say well what what are you doing with this? Like why do you need cross cloud?

Because we want cross Cloud not like but why and they no one could ever really give me a good answer so I kind of dismissed it out of hand, but then I ran into Banks and then you start having these weird regulatory requirements and things to make sure you don't have specific types of downtime and other things and you start to see where heterogeneous environments made up of all sorts of different things could be brought to it are a big deal. And right now if a developer comes to someone working on an SRT SRT team it says here you go. You're you're in trouble.

You're like, oh no, like how am I supposed to do this or there's this there's a huge lift to actually make that work and in the current Paradigm, that doesn't work very well and The problem too. And I mean, I'm sure you have commentary on this too Bailey that when we get to like the way we do this distributed stuff right now with kubernetes. There's just it's just so difficult to do a cross Cloud.

There's no way to and and that's the thing. It's a hard problem to solve but it also like kubernetes is kind of the way people do distributed nowadays. I mean, there's other things out there we ourselves use Nomad quite a bit but there's too much of the shared knowledge that we've talked about before between the two teams of like Dev versus the devops side and kubernetes should have been exclusively the domain platform teams and somehow along the line everyone got involved with it.

And so now like I mentioned before devs have to worry about where these things run like if they're reach and supported which things are installed and I mean, I'm sure you've seen some similar stuff before the different platform work and and things you've done Oh, yeah, totally, you know, I was gonna say come on Taylor don't does my sea cat because I I did have to you know, learn how to swim maybe tread water when things are happening and prod or Dev and making sure that like, you know, hey, you know, where do my logs go is it did my Sidecar get attached. Does it sending up the right stuff to Prometheus all that kind of stuff that if I if I waited for a very overloaded platform team, I'd be waiting very long time that I could fix myself and in just a few minutes and there's a lot of things that I could do from the developer side to make sure that I'm exposing the right things. So if I'm constantly living and breathing, you know, all the OpenTelemetry aspects of my application that's really important and that is something though that the platform to you should provide but it's where I got to meet in the middle.

So doing all of that is really important, but who's not easy right? And and there's there's just I don't know about you but I just feel like there's Such even Arcane knowledge to how to do things and when somebody doesn't have like a kubernetes Alias to keep cuddle Alias. I say keep cuddle.

I just want to go ahead and say that out there. I want things to cuddle if if I can't type K Tab and there's not like autocompletion. You know, it's just like what what are you living your life?

But if you try to to take a new death, that's just getting started in the career and expect them to be productive in this new world that we've built. It's very hard and just trying to explain to them why we're doing all of this you're like, you know. We might be causing a lot of pain for ourselves here.

And after I've onboarded and of it interns and early early doves. I've definitely been thinking there's got to be a better way. Yeah, that's that's something I think we've all experienced at this point.

Just you you just see this especially if you start to bring these people on board. I did a lot of this one. I worked previously at Microsoft like working with customers.

and getting people to grasp it with such a large amount of effort and so The thing we've just kind of pointed out here is like really. the the way we do distributed right now kind of breaks down when you get to cross Cloud cross region those kind of things and also anything that can truly make a distributed not just across multiple nodes, but across like any type of device and with kubernetes. The other thing you have is like anytime you do something interesting at all.

I think we've run into this we kind of have like Your own crds your own controllers your own scheduler extensions your own, you know, like you might have a specific way. You run your data queries and house and all that all has to be shoved into this box that people have somehow designated as the only way to distributed programming and so I have found it really interesting that like We kind of actually kind of actually not only did we create a world that's hard to bring new people into and how to like there's so much shared knowledge that's needed. But we also created something that is actually just very difficult to bring current systems and current ways of modeling things too, especially in a distributed way because sometimes the things you're trying to bring are all ready distributed in there in their own special way.

They they run across multiple areas or things with the work differently than the way we're trying to run them or connect them to yeah, not only that but a lot of people think that because they move things to the cloud it's going to be easier or cheaper and and those are really can be myths. Yes, it's more scalable. It's more reliable you get all these other properties, but if you lift and shift what you currently have which I experienced at one of my previous roles.

It was very expensive and that is why we actually had a mission why why am I was recovering spring boot app developer is that I had to go from finding a way to go from many different spring food applications that use reflection and magic and they were sitting at like I want to say 750 megabytes was about the average for a single microservice and convert 300 ish of those to go and that got down like 17ish. Megabytes is the was the typical size there that that, you know vastly reduce the cost but If you were just comparing that to say our bare metal version of the same thing. Way bigger way way bigger.

I don't know about you. Have you have you dealt with some of that? Yeah, that's something that we we've run into quite a bit.

So as part of our work at cosmonic and we've been talking to people about like their clusters and what they're using and honestly the numbers we get and just from my own experience having run kubernetes question things is utilization of a clusters between 40 to 60 percent. And so even if we take kubernetes out of the equation, it's gonna be anything similar that's based on Docker like just you you're not gonna use all that you have to have all that capacity there to make it distributed and you and just do to the amount I think about it as well. Like if you're doing something in Java you've probably I mean Java is obviously the worst offender here but think about anything even stuff like dotnet and even go to an extent where you have a runtime that's compiled into the language.

Now, it goes obviously way down the totem pole on like how bulky it is because it's not that big there's a lot of cool work that's gone into the go run time, but when you actually look these things that have been done like If you if you sit and stop and just think about it for a second you are. Running an application. I mean, this is something that's just like a simple API.

If you're at a bank, you're like writing an API that might calculate an interest rate or something like that. It could be a little service or you could be adding to a service. And so this little service that maybe if you boil it down to it's Core Business logic is coupled hundred maybe a few thousand lines at the worst.

You're running in a bare metal machine somewhere like it has to hit silicon at some point. And then on top of that you have a virtual machine. And on top of that you have kubernetes and inside of that you have Docker and inside of that you have the whole run time for every single little micro service or thing.

You deploy like that is a crap ton of stuff you are bringing along with you and that's like one of those big problems about running. These things distributed is that it becomes very hard to actually like run like between just the cross region you start throwing in like cross-platform and other stuff that you start running into and it's just difficult. Like it's it's heavy it's it's more you're bringing much more along with it than you actually need to and that causes a lot of like, even if you are running it capacity if you're running a capacity with one of these things, you're still probably over utilizing by quite a large amount because instead of having one thing on these Java because it's the heaviest here.

You could be fully like up to 90% utilization on on your cluster. But more I would say about half of that is just repetitive jvms being spun up like at that point. You've lost a lot of the benefit and even like when you go down to things like go which is much more lightly.

You're still bringing a runtime along for every single one of those and it's you start to see there's a lot of inefficiencies there that really matter once you get to scale and to all that but I mean, this is where like I mean both of us have experiencing containers and I'm just curious like Daily, I know you have plenty of experience with these kind of container things in the past. Like what are you going to run into as you've tried to do these things? Yeah, some of the things that my platform team asked me to do was to pack all of our services into one pod or or like, you know, maybe three pots when we when we previously had 300 and you know, I I had pushback there because it's like hey when these things first and they Auto scale they actually have very different scaling properties.

So when I'm thinking about this from a distributed aspect, I I expect things to be very different but then there's like the yeah, but you know, this is requesting this amount of memory and then you know, you get into the whole Java argument of well, yes, but this is how much I need to start it with with the runtime, but when it's actually processing requests, it's a lot higher so this is why and this is what the max limit is for a reasonable run. And so those those are things that's that's again on both sides of that fence of that devonopsy fence. But other things that I've been asked to do is like constantly find different ways to rebuild my final release artifact.

So working at an isv. We had a very typical enterprise-year class which is hey, you know, we've we have the set of blessed face images. You know, they might be based on red hat uvi, which is fantastic.

Maybe you just got a contract and you need to support a different architecture. Like maybe power 9 from IBM or probably most people who are listening right now. They've they've gone and rebuilt their stuff for arm that's really common.

But you know at least where I was working out we had dependencies very easily a decade old. Nobody was working on getting those working on new architectures and the work of just turning the crank on hundreds of different projects is super expensive especially projects. That really are not changing Beyond like hey, we got a cve here like let's let's get these things out because we need to and unless so on like actual features or fixes.

So just that, you know the like screw if you can imagine turning your crank and just hearing a year is definitely how I describe many weeks. While working at a large Enterprises trying to do microservice work. And that's that's one of the reasons why I joined our team, but how about you Taylor you get anything you want to add on that point?

I know it's just darker really came like we've talked about a lot of the benefits of doctor like yeah, it really Dockers so easy to just like get something into it and then run there's benefits there, but we kind of Deceived ourselves. We said oh doctor can run anywhere and it it can't like I'm just gonna be blunt and honest there. It can't like yes a lot of really smart people.

Some of whom I even met like made containers work on Windows, but they work fundamentally. You cannot run a Windows container in Linux. And then you can't even run a Linux container that's built for arm like you were saying or like any of the other types of processors for a Linux x86 processor.

It just it doesn't work. It's not cross-platform and We just pretended it was and so that's one of those things. I've just ran into constantly because when you find a requirement for that need or you try to do stuff like move that towards the edge and you start getting more bespoke processors or things have to run really tiny it just you just run into to issues.

And so honestly, I think this is this frustration and Bailey you could speak for yourself in agreement or not. But I think this kind of is what led both of us to webassembly and to awesome Cloud which is our open source project that we help maintain and so like those are those are the things that really let us there because I mean it there's just a lot of frustration there. We've really improved where we came like we don't have to deploy things on bare metal anymore, which is our lug into a Mainframe like we don't have to do that and that's great.

But we've kind of just like brought the baggage along with us and it's caused some of this like pain that we've talked about especially running distributed applications. And so that's where I think both of us kind of came to webassembly was from that frustration that we saw with getting these things for real and and big environments. Yeah, I guess you know, I'm realizing that we didn't explain.

What was them or webassembly is we're just like it's awesome. It's a technology that we both work on. So I'll throw out my version of what is wasm and wasm also known as webassembly is a portable compilation Target.

so that is a DOT Blossom that any Wasim enabled runtime can run. And so one example wasm runtime is your browser and Firefox. It's spider lightning and chrome.

It's V8 basically all the major browser vendors support it and have since 2019 was when it was a recommended web standard, but many had support way before that. I actually ship my first was a map in 2015. So it's been around in the web space and because it's been around in the web space and web was really that first platform that was targeted a lot of Interesting and unique properties were required of this new bytecode that many others didn't have before.

And to download really fast it even supports actually streaming compilation. So as it's being downloaded it can be actively compiled. It also has to be secure if you think about taking arbitrary code and running in your browser from any random website on the web.

You've got to be able to make that so that it's safely sandbox. And it also needs to be fast because if you go on a web page and it's slow and sluggish you navigate straight off always. So the combination of all of these is very valuable.

I forgot to mention the cross-platform aspect. So we were just riffing on a containers and how it it needs to you need to create a container for each host architecture that you're targeting say. It's arm or Windows or Linux x86 and with lots of them.

You don't need to do that. It's a portable bytecode format that you can distribute. It's it's really the runtime.

So you have to have a runtime that runs on those different systems, but there is a plethora of option as far as runtime. So, you know one doesn't run Docker in the web right now, but with wazum you can for example, that's one place on many different Edge type like whammer is one another really great one was Edge that Target iot devices. So there's just so many that are that are out there that we can choose from gosh.

I feel like you know, I can talk about lazim all day. It's my favorite thing. Is there anything that I should have said about it as the hello world Taylor?

No, I I think you captured I think you capture the hello world pretty well. I just want to kind of point out what are the features of webassembly that I I really really enjoy. It's kind of like the one of its most not killer features, but one of the most important things it brings.

That a lot of people don't notice the first time they learn about it. Is this plethora of runtimes? This isn't like what happened at the beginning of Docker where you have?

Oh, does it Docker do you use cryo or do you use rocket or do like that's like who's gonna win? There isn't a who's gonna win question here. And I find that very very refreshing.

Number one and number two very good for for the technology. The whole idea is that you can run on any one of these things we say there were some for like smaller devices. There's some that work on they're more like general purpose and you can choose how things run and what you run it with at runtime.

So you're not only choosing optimizations at runtime, but you're also choosing the runtime at runtime. And so those are all really cool features that allow for some really awesome tweaking and optimization that you don't normally see with other Technologies. And really just the rest of it.

I think I think you covered really well it just It's much smaller. It's much more compact. It runs everywhere.

Those are just Features for me as someone who's done a lot of this crop cross platform support and many different things. So much nicer than what I used to have to do. So, I think I think the what the 101 level this is what webassembly is.

I think we covered pretty well. So and and that I think you highlighted why it's so great for when you talk about distributed applications because nowadays The Edge is you it's it's your device. It's your laptop.

It it goes with you wherever you go in the car. So having runtimes having software that can go from exactly where you are on the edge to the cloud. And at any scale that that's really what we're targeting.

So how do we make it so that we're not creating these different bespoke solutions across all of these different platforms and making it so that we have something that kind of ties these together in a distributed way. I I think it's really just that's right. There is your if you want to come with one thing just know about webassembly giving you those features, but the whole panel that the whole discussion that we're supposed to be having around this panel is around the distributed nature of things.

And I I wanted to kind of mention a little bit about that because you already mentioned this idea of the edge and to be clear the edge is not just one use of the engine. It's a very good use of the edge just like what you see with things like a flare or other CDN type stuff where it's literally Doing like very like little amounts of processing or like your HTML and JavaScript and serving it from something. That's a very very close to the user.

But you could also think of this as data closeness you can think of this as which type of device because the edge is like Bailey said your laptop or your phone like that's the edge. It's kind of become a buzzword and I understand why but it is. It's just very very Broad and what it means but we really mean like distributed.

Means from the cloud to the edge at any scale. That's what we mean by distributed and we like to say that webassembly can run anywhere from a light bulb to a super computer and anything in between. And based on what we've done so far that holds fairly true.

You can run it pretty much anywhere. And so because we have that like what one of these one of these things just thinking about all the different devices you we can connect. So anyway, that's what I think of like distributed.

So and that's something that's really important to mention is kind of the follow ones what we've been talking about with awesome. I have a story I want to share so that gives you the idea of what we mean when we're talking about distributed in the edge. So we were at keepcon gosh, I guess it was two weeks ago.

Maybe there's even three weeks times flying in Detroit and it was sort of like the middle of the week. We've been we'd already had lazam day. We'd already had several talks and panels and all kinds of stuff that we had to do and then we had boots Beauty yet again.

I think this was Wednesday or Thursday, so I'm walking in it's like right on the dot 10 AM when we're supposed to start running our booth and I catch tailor and then Dan who's our infrastructure lead kind of like Just over a device there. It was a steam deck and just sort of like giggling and I was like, okay, what are y'all doing? Like we're supposed, you know act like we're you know Booth Booth attendees now and I look over and Taylor tilts his steam deck and it's got our our logo on it and ASCII art the console was up.

And then what they showed me just like completely blew me away because they were like, look, I've got a running on I have an actor deployed and running on my steam deck. That is also connected to gcpa AWS Azure even Oracle Cloud all at once which you know that we all were like a little Giddy and a lot of people came by and they're like, what are you doing? We're like well They apparently had this discussion at a bar last night and thought they would just try it and one command later.

It is working and that is pretty pretty cool. Do you want to describe what that feature is? Yeah, so this is a feature that we call a super constellations that cosmonic.

It's also enabled by by Watson cut we mentioned was Some Cloud a couple times. I'm just gonna bring it up here that it's a cncf owned project. We are the main and principal maintainers of it, but it is it is an open entirely open source project.

We work off of no core and cosmonic is the hosted version of and you can do this with either one of them with cosmotic. We made it super easy. You install our command line tool and you literally run one command.

I do not have to install any other software on my steam deck now steam deck is a little bit more fully featured than what some like people would consider like an iot device, but even then like I didn't do anything else and I just attached it and Just having to be able to like have things in three different clouds and stuff connected from like a conference floor with terrible conference. Wi-Fi that string up requests back out to people on the Internet is just a little bit mind-blowing to actually see in action. And so we absolutely loved that that feature and just that that idea is that they they help solve this idea of everything can be wildly distributed across any type or system you do you could bring your own servers you could bring something from a data center.

You can bring devices that have sensors on them. You can bring your laptop. It doesn't mean matter.

And that's why those those kind of things are so cool because we really wanted to have that developers. Just don't care about those kind of details. For us like this kind of has a lot of ramifications for How We Do software development like developers don't need to care about these details and they shouldn't need to re-architect right now.

And it's always been this way, but I think it's even with the speed of how fat like Cloud native development has generally been associated with like a lot faster cycles of development. And so you run in this more often like you write your application, it's running and then it's successful people like well now we have to make it so it scales across three different things. And so then you have to rewrite it again and you have to change your code and how you're doing and that's not really like developers are good and generally enjoy writing new features or creating new things.

They don't have they don't want to have to go like figure out like which flag to turn on on this Library so that they can connect to this thing. That's just I mean some people do that. It's Different Strokes for different folks, we know that overall like most developers that that I've talked to and I think a lot of us have talked to that's kind of the thing.

They don't have to worry about it. You have to re-architect your application every time. You have to rewrite your application every time you have to re-architect it.

And so that's no longer a thing here with with what we've set up using Wasim cloud is that you can write your code and instead of it have instead of you needing to know exactly where it's going to run or how it's gonna connect to you're able to then just completely rejecting. We'll talk a little bit more about that in a little bit I think but really this is a huge impact on software development. This is what we were mentioning at the beginning with the whole pendulum swinging from one side to the others.

It brings us back to the middle. You're not having like developers no longer have to carry care about things like where am I running? How many am I running?

What kind of thing like what kind of things am I going to be running across you don't have to worry about that as much there's still a little there's always a little details we can there's always gotchas in software development. But for most people like that's not something you really have to worry about that often and we try to extract that away. Yeah for me the thing I saw was wow.

Okay, so I have the exact same as a module running on a steam deck as I do on an x86 machine in a different cloud. Running, you know also in AWS Azure an AWS. We're actually using super cheap arm devices.

I've also available to use it now running off of my M1 Mac and to me it's the same thing and it has the exact same behavior characteristics, which is like performance memory footprint all of those things. That's extremely predictable. So as a developer that's just gold right as far as what I can design an architect for a distributed application so often that's like the thing that always bites me in the end when I'm thinking about working on some of these things at scale but one of the other things that we haven't dug in much yet is also the the way that you can write your software and write your applications.

That's a little bit different from what maybe a lot of people are used to and that's where with whasm Cloud again open source cncf sandbox project. It eliminates non-functional requirements out of your code. So that's another thing that we didn't we didn't hit on much when we were talking about what goes into just your pod right and let's say in my pod.

I'm running screen Boot and I've got a some type of runtime, you know, maybe it's Java. And I'm also running my own HP server because that's what most of these microservices do and guess which piece of software always has the cve's it's it turns out it's it's the thing that communicates over the network or it's the thing that is looking at certificates like nine at a time nine times out of 10 when you go and look at the cpes that you have to rev like, you know, Mission critical like we need to get this out in the next 24 hours it's there and it has nothing to do with my application logic. I actually I could care less which HP server you want to deploy my stuff on I just here's my credit application.

This is my API. Like I want to give you that information and I want somebody else or you know, even if it's me I want somebody to pick the best thing and and run with that and that is what Lawson Club does. It's it's a feature with capability providers and at runtime we Link in the things that you need.

So the thing that is actually in my wazon code like the logic there, I just say I'm Coding to a contract I expect HP requests and I expect to be able to maybe serve a Content but that's it. And at runtime you could put in an HTTP server that was written go that was it written and Russ it was served out. I don't know who knows where all of that is eliminated from the concerns that I have to worry about as a developer.

Which which is extremely powerful. It makes me happy, but I think it makes tailors of the world happy, too. Oh, it definitely makes tailors of the world Happy.

Let's just say that it really like just think about how big of a deal that is. Imagine a log for J incident happens in I use a lot for Jake not because like it was a bug that most people know about if I mean, I have written plenty of bugs and many of them have been security bugs. So I cannot there's no judgment here, but that was just a big one that affected everybody.

And so what happened? And once again, we talked to a million people about this. People had to go back to their containers and they had to say okay.

We have all these different like all these dependencies in here one of them's locked for J. So they had to go rebuild their Java stuff like makes like update the dependency, then they had to rebuild all their containers and then redeploy all their AppSec. Like 10,000 of them at least some of these big companies and they had to redeploy all of them and this dependency had nothing to do with their day-to-day business and their day-to-day like money-making that they needed to get done with this stuff.

They were writing and instead like they had to spend all this time and effort to update that. And when we pull out these nominal requirements, you're just can hot swap it. This is something that where my co-worker Brooks and I actually gave a talk at kubecon you where we did this live on stage like the introduced a big bug into one of these these providers and showed that we could just rotate it out without the developer ever having changed their code and it can just pretty much be hot swapped at runtime and that's just such a cool characteristic to have because now you just update the dependencies.

And once again, this is bringing that pendulum back to the middle. the Developers Can do the code that they need to do that the devops or SRE side can handle all the compliance and platform work and all things that they need to do without having to know every single thing that the other person's doing. We use this to our advantage because cosmonic is actually like built.

I'll talk a little bit more about this but it's built on top of Watson cloud. Like we've written most of the actual products that you use is written using webassembly. And one of the things we did at the beginning was we started with a key value store just to store a lot of our data for the application.

And as we went along we're like, okay, there's some stuff in here that secret where we might be adding more secret stuff in here. So we need to put it somewhere. In normal application development if I was doing this with like a normal microservice to container, I'd have to go.

Okay. Well, we're going to use Vault because pretty much everybody uses all for this. It's great tool.

So now I have to get the Vault client and I have to configure The Vault client. I have to do all these things and I would have literally had to rewrite my code to take advantage of that. Instead, we wrote a provider that satisfied one of these contracts have the key value contract.

We slotted it in we had to make one or two line changes because we're trying to segment our data in a few of them. But for the most part we didn't even have to change your code. We just swapped it out.

And that's that is a completely Paradigm changing way of how you can do it because you're not restricted to anything we offer you either. And so that's I think like it's I we harp on this all the time and we repeat some of these stories Ad nauseam as you talk about it, but it it's it's real like this isn't just made up. You don't have to like obey a ton of special rules here you just to code against these capabilities and then get what you want at runtime based on whatever the platform wants to give you rather than making the something to do.

At the first decision of writing a code. So basically imagine fixing that dependency. Maybe the one that had logged for J shell vulnerability.

Maybe you had stress somewhere, but that gets fixed in one place compiled released and then at runtime fixed everywhere else. That's pretty powerful. You know, we can almost start saying that we're in eco-friendly company because we're providing this type of capability but there's actually one other big reason why blasm Cloud approaches this problem in this way and that's because plasm itself is constrained by allowing certain things in and out of the sandbox and the MVP of lazam and the current set of Open Standards is pretty restrictive of what is allowed today.

And there is the way to extend it is basically saying, hey, you can export this function and when you when you get a handle to those function, you can call it and it lets you do, you know say pass a request around then it's on the hosts side to fulfill that whatever whatever that need is and so in the case of HTTP, for example the wise of module itself the code in it isn't the thing that's making the HP request. It's it's not To the network or any way shape or form. It is really the host that the host runtime that that is running.

That was a module that does that work and with Wasim cloud with these contracts it lets it so that even though wasm isn't fully featured. It's not like a general purpose programming language that you can just do everything you expect open sockets and networking and threads all that type of stuff. Although there are standards for all of those underway before way before even those standards become available.

Allow. Some Club makes it accessible because you just need that contract and then on the plasm cloud side, we're able to fulfill that with many different capability providers. Yeah, and that's really important to call out to people here is that like Wasim is still kind of new?

I mean it's been going for a while. But what we're using it for on the server side is still very new and it has rough edges and I mentioned it before we built cosmonic on top of Wasim clouds. So you can build real things with webassembly right now using Watson cloud and that was the whole goal and we still follow all these standards all these cool standards that Bailey just mentioned of being able to do all the kind of the basic behaviors are going to be folded straight into awesome Cloud, but right now you're still able to do something real with it.

And the other thing too is this this kind of sounds like magic and honestly the first time you see it working it does kind of look like magic but underneath the hood all this is fairly well protected like wazam itself is sandboxed and wazam Cloud follows a whole idea of defensive depth. We want to this is another problem that's really hard across the distributed system is how do you secure those things and people turn to mTLS and a couple other different types of communication? Things that go along here, but what we do is everything that's on this the network that we connect together is signed every invocation between every single component of the system is signed.

The things that join in are signed. Each of these pieces of code is signed and everything has to be valid for it to actually work and most of this is actually fairly transparent to the user. So we've managed to give a massive defense and depth here too that complements.

What was them already gives us with it sandbox model and so all these things are just kind of the extra features that wasn't Cloud brings on top. But the reason we talk about it is because once people hear about wazon, they want to know what they can do with it. And Blossom cloud is something you can do right now.

And if you wanted to like literally do a one click I start doing something with it. That's what cosmonic is for because we have hosted awesome Cloud for you. So those are kind of the this is the reason we really wanted to talk about and get to because we thought a long time about how we want to do this in a distributed way and we we've created something that is really worth people's time and takes away a lot of those rough edges without crippling or forcing you to work within a specific box all the time.

Or language even that is something else that I forgot to mention and webassembly 101 just about any language can compata Wasim there is a set like things that are aesthetically typed. So something like rust and c++, those compile really well to yasm and are super well supported. There's other tools like tiny go that works for go.

So there are many and many of them are up and coming but it that is something else that happens with platforms and platform teams or cross companies is that they they bless a certain language a bless a certain dependency. And what we're hoping to do is open the door to you let you do a lot of these different things but in a way that satisfies all of your security constraints and honestly right now wasm is feeling a lot like it did in 2013 and 2015. I mean, it's it's really exciting.

I felt like walking around kubecon. Everybody was was hyping it and really I would say that there's really no better time to jump on and start playing around. I really recommend going in our slack.

We've got different wild Club lab so you can try things out. I would love to see you in our community. Yeah, we'd love to see you there.

And also hopefully this discussion has been useful to you. Feel free to hit us up on on the slack things ask us any questions and we'll be there to answer them and and help out with whatever you want to know about weather something. So thank you so much for joining us everyone today and hopefully you enjoy the rest of the conference as well.