Sudeep Goswami on Traefik Labs: Managing Microservices and Cloud-Native Infrastructure

Hey, everyone. We're back here, live on the floor of CubeCon and it's, well, it's Wednesday and it's, I'm going to guess it's about two o'clock, right? And we've had an amazing day and a half so far of some great conversations.

You probably see behind us, people moving in and out. The, the show floor, as I think I mentioned, is, is really big this year, like CubeCon style, not too much on top of each other, but a little bit more, um, concentrated than like, let's say during COVID when stuff was really spread out. I want to introduce you to our next guest, his name Isse.

Goswani Go. Did I get that right? Go.

Swami SDI is the CEO of a company called Traffic Labs, but it's spelled a little funny. We're gonna get into it, but first let's find out more about Saddi. Welcome.

How are you, man? Yeah, Hi, Alan. Great to be here.

Great to have you. So, Saddi, before we talk about Traffic Labs and CubeCon and all this, let's talk a little bit about you. As I said, you're the CEO here, but you know, you weren't born the CEO of traffic.

Give us a little bit of your journey. Yeah, so engineer by trade, um, used to implement technology before I got on the other side and started to build technology and then sell technology. So really kind of dealt with the challenges of day two ops and what users have to do after they buy a per certain piece of technology and all the trials and tribulations they have to go through, and being able to implement that at scale.

Yep. So I'm gonna assume that being that you said that traffic deals with sort of a day two ops problem. Is that what you would think?

Yes. We definitely focus on day two ops for microservices and cloud native infrastructure in general. Got it.

Let's talk about how you went from being the, how you came to be the CEO at Traffic labs. Right? That that's a, there's a lot of people out there, of course, who aspire to be CEOs and having been a CEO now a few times I don't.

But, um, how did you know, when did you, let's say, take the track and say, you know what, I want to be a c, the CEO? Well, I, I'm a problem solver at Heart Uhhuh. For me, it's all about solving problems, you know, whether it's a technical problem or a business problem.

So I joined the company as a CRO. Okay. And then, uh, we had a conversation internally and try to match my skill sets to what the company needed and the problems that needed to be solved.

And, uh, you know, the rest is the Rest history as they say. Yeah. Cool.

Let's talk about the history of traffic, if you don't mind. Sure. You know, again, probably a Traffic Labs company that maybe a lot of our folks haven't heard of.

As I mentioned, it's spelled a little different. Why don't we start there? How do you spell it?

Yeah, traffic's spelled T-R-A-E-F-I-K. io and the traffics claim to fame, uh, it's really around the open source project when traffic started almost 10 years ago now. Mm-hmm.

Uh, where our founder and now the CTO put out, basically, he's a problem solver as well. You know, he found a problem deploying microservices, was found it very painful. So he came up with his own solution, put it out there on GitHub, and overnight took off.

It just took off. So is the open source project called Traffic as well? Yes.

That is called, that is called Traffic Now or Traffic. Some people know it as traffic proxy. I'm sorry, say that again.

Traffic Proxy. Traffic Proxy. Yeah.

Absolutely. And, uh, you know, that was in 2016. 4 billion with a b downloads.

Wow. It's got over almost 60,000 GitHub stars. Got a very vibrant community of active contributors both inside and outside the company.

I love It. It's, it's one of the best projects and, you know, people who come to our booth here, uh, they know the brand. Uh, the, the brand is the gopher, uh, that's the, the brand, you know, The Masco if you will, Mascot.

Exactly. So people are always coming to our booth for the Gopher the go Shirts. Got it.

Um, it's funny, I'm old enough to remember when Gopher meant something else in the internet right before there was a web, we had wide area search Gopher and stuff like that. That's right. You remember.

That's right. Yep. Um, wanted, so is I, I just wanted from like a book of housekeeping.

Is traffic A-C-N-C-F project or No, just traffic class. Traffic is not A-C-N-C-F project. It is an open source project and it's got an open core model.

Mm-hmm. And then so it has the open source, uh, binary itself, and then there's a secondary binary for all the enterprise features. So we built on top of the open source as the foundation.

Got it. And, and that, you know, the open core, the open, uh, core model is one of course that's very familiar to everyone here in the, uh, in the, uh, open source community. Right.

It's, it's evolved. Um, so with open source, I'm assuming you have like premium modules or premium modules that ride on top of the open core, open source one that give you enhanced, you know, capabilities. Let's talk about, you know, what comes in the, in the pure open source one.

Mm-hmm. And then what some of the add-on, uh, modules are and what they do. Yeah, sure.

I'll talk about what comes in the add-on modules, but I also want to talk about the upgrade process because we made that extremely user friendly and seamless. Mm-hmm. So let's start with the capabilities in the open source.

People can deploy that as a reverse proxy, as a load balancer, and as a router, it auto discovers all of the microservices that you're running. So you don't have to do that manual work. Wow.

And that's the core value proposition of the open source that people love That. Yeah. They don't have to deal with kind of fi finding a microservice what microservice it automatically does.

That creates the routes. So traffic starts flowing to it. Okay.

That's the foundation. It's, uh, That's in the open source. That's in the open source.

People love that. And a lot of people for them that is enough. And then we can give them enterprise support on top of that.

So they have the peace of mind for when they do need to call us and get support. Got It. Now moving to the, uh, the paid add-ons, Premium or premium, All the additional things which comes with our advanced, uh, or the another binary, so to speak, people use that.

One of the most common use cases for that is security from an authentication and authorization standpoint. So when their microservices need that front door where you can authenticate and authorize users and traffic coming in, that's when you need what's called an API gateway. Okay.

And that is the most, uh, common use case for our paid offering. What we have done is built on top of that. And that API gateway.

So when you say it's an API gateway, it's an, it's a API built or that rides on top of the open source core that allows other people to plug in? Correct. Okay.

It's can, so we have our own API gateway, but there are plugins available on the open source, uh, binary Yeah. That communities can contribute to. So you can pull off of that.

Yeah. You can add functionality or you would go with our paid offering that we have written at Traffic Labs and we support. So, and that's the most natural path people take.

Got It. Is the API gateway. And then we have created extensions of that API gateway into dev, uh, developing an AI gateway and an gateway.

So that be sort of an MCP. Okay. So that's an MCP gateway.

Yeah. An AI gateway is separate and an MCP gateway is separate. Now they have Different, so is the AI like a A two A, uh, was a two A or something Different?

Ai. Think of the AI gateway as, uh, uh, LLM router. Okay.

So, you know, it does many more things. So at its core, it's a routing to different LLM endpoints while and before it's routing. We also provide a way to, uh, enrich the traffic or to guardrail the traffic.

So not every query that you send to an LLM should be sent there. Maybe your corporate policies don't allow certain types of content. Got it.

So we integrated with NVIDIA's safety name guardrails. So before you route the traffic to an LLM, you take it through a set of safety guardrails, then comes some kind of caching layer. So if you keep asking the same question over and over again, you don't need to consume these expensive tokens.

So only then do you route all of that. Right. Is offered as part of the AI gateway.

Got it. MCP gateways, uh, it requires that as the foundation, but it's not enough. MCP gateway needs more than that because it's a new protocol.

Right. And what that requires is for your agents to be able to talk to the MCP resources, and then the MCP gateway provides that governance layer. So it allows which tools under the MCP umbrella can you talk to?

How do you, uh, talk to those tools? What operations can you do under those tools? So it's an extra layer of complexity that needs to be governed properly.

That the API gateway definitely is not gonna give you outta the box. And AI gateway is not enough. So you need essentially what we call the triple gate pattern for MCP, you need the AI conversations protected, you need the API conversations protected, and you need the MCP conversations protected.

And all three of those are premium. And, and I, I could and All these are premium. Give us an example of some of the other premium models, The other premium modules.

Yeah. And which is really around API lifecycle management as everything is becoming an API. So your LLM endpoints are being exposed as a, uh, a APIs, even your MCP endpoints become exposed as APIs.

You need, you have an API management, uh, management problem, or you have an API proliferation problem. So what you need is a full lifecycle management of that. That means your APIs need to be versioned controlled, they need to be sunsetted, they need to be deprecated.

You need to provide rate limits and quotas. So, you know, there's a layer of defense. This is why you have a true sort of gateway that controls all of that.

Right. Rather than just, uh, you know, uh, an API to api. So, I mean, just the Correct, Yeah.

Uh, a direct connection, if you will. Plus you also need to share those APIs with your community of developers. They could be internal, they could be external.

So this is where developer portal is needed. So all those things come under the umbrella of API Lifecycle manage. Got It.

Got It. Now, the beauty of this going from open source to all these extra paid capabilities, it takes 30 seconds to do the upgrade. And it's an in place upgrade mm-hmm.

That you would do. It preserves all the configuration that you do with the open source. And after 30 seconds you Have a new, it's Top of that and you have explore you.

So it's like a mesh layer that sits on top of the pure open, uh, stack, if you will. Um, it's like, I'm, I'm wondering, so it's not like, so people host this and run this themselves. It's not, you know, 'cause that's another model.

Right. An open source where I, I'll take my open source and I'll run it as a SaaS so you don't have to worry about. Right.

We will, We provide a self-hosted model, A Self-hosted, so people usually deploy this on public cloud. So because we're Kubernetes native mm-hmm. They can deploy it on any of the public Kubernetes distro.

So A-K-S-E-K-S-G-K-O-K Got it. LK. But they can also deploy this on-prem in a completely air gapped and offline environment.

Okay. So it doesn't even need connectivity at That point? No, it does not.

And people can deploy it at the edge. We are a natively integrated in the K three s with rancher. Mm-hmm.

They can deploy this in, you know, bigger environment, you name it. Like it works seamlessly everywhere. Excellent.

And so does it require, it requires Kube though. Um, the, the Kubernetes native that if you're deploying in Kubernetes, yes, it's gonna require Kubernetes, but because we started in 2016 when Cornes was not even around, we have had into native integration with many different alternative ways of deploying microservices, starting with just bare metal Linux. Okay.

And then came HashiCorp Nomad. Yep. We can be deployed just as a Docker container or we could be deployed in Kubernetes.

Okay. Got It. And we can even be deployed in Windows, if you like.

Listen to that. Oh, you're the first one to mention any deployment of Windows. People Don't talk about the W No, they don't.

They don't. They don't. It's the year of the Linux desktop.

But anyway, um, did we mention the website? io. Say It again for me.

io. Dot io. io and download the open source Yes.

Package and install it yourself. Yeah. It's on, it's on GitHub.

You can download it through a helm chart. Uh, there are many ways to install it. Absolutely.

And then once you're up and running, putting in the, uh, the, the third, the, the premium modules, as you said, it's a 32nd, it's A helm chart update. Takes 30 seconds and you're good to go. Excellent.

I can't, I what, well, let me ask you, what has the show been for you this year? It's great. It's great.

We announced, actually some, some great stuff this Week. What you announced. So, uh, there's a, you know, with everything we just discussed, there's a fragmentation problem happening now in the industry.

It's a big realization. People are having that virtual machines are not going away. No.

So there's gonna be a coexistence of virtual machines and containers, and now serverless workloads as well. Mm-hmm. So, and But they're not mutually exclusive.

They're not mutually exclusive, but the way people manage them are very independently and differently managed. The way you manage a VM stack and the way you expose your services running inside a VM is very different than the way you expose and manage a Kubernetes. Yeah.

So what we have launched is a unified layer, uh, which we call the application layer intelligence that connects the VM environment and the Kubernetes environment, and also the serverless environment. We launched support for, You can manage all three. So you can manage all three in a very cohesive way.

And it's through a single application layer, which is a layer seven, uh, construct. Mm-hmm. And you can seamlessly redirect traffic because what's happening is, as monoliths are being decomposed into microservices, your backend might be sitting in a vm, but your front end is going into Kubernetes.

Absolutely. Right. But it's still, still the same application.

Yep. So at the application layer, when a traffic comes in, you want to have that intelligence so you can seamlessly direct traffic. It goes left or right, or somewhere else.

I mean, with containers, they literally could be distributed anywhere. Right. It, uh, it's not just Right.

And we did the integration with Nutanix as a, as a reference architecture. Oh, very cool. 'cause Nutanix as a platform does a great job of providing a virtualized environment and the Kubernetes environment, you know, so they have their A HV with flow networking for virtual machines.

We have done the integration with them. And what that allows users to do is to auto discover all the virtual machines that are running so then they can write policies against it. We are already working with them in the kuber, in their Kubernetes distro, and we can all auto discover services running there.

And now with K native, with serverless, they are the perfect reference architecture for this solution. Mm-hmm. Excellent.

Any other announcements here? Um, just that we on the gateway, API, you know, which is on everybody's mind, uh, here, uh, we continue to lead support on that. 4.

Uh, our founder and CTO and other team members, they're part of the c you know, the community actively working with the Gateway, API community in advancing it forward. So, you know, that's one of the other big announcements that we continue to open source is important for us, it's very strategic for us, and we continue to invest in that. Love it.

Alright. I think we're about outta time here. I think we hit most everything.

Is there anything we left Out? I think so. Nope.

Just, uh, you know, you know where to find us. io. Very cool.

Thank you very much. Hey, thank you Alan. Enjoy the rest of CubeCon.

Hey, we're going to take a break. We've got, well, we probably have another two hours or more of coverage here at CubeCon today. We'll be back again tomorrow.

But let's take a quick, uh, break. We'll be back in a moment.