Closing the Testing Communication Gap – Harpreet Singh and Kohsuke Kawaguchi, Launchable

This is Textron TV. Hey everyone, welcome back to Tech strong TV. I am really happy to have two of my friends join us on this next session.

They're the co-ceos at launchable and I I believe the website is still launchable Inc. com, if that's correct, forgive me if I think correct but let me introduce you first of all to my friend KK goes. Okay coming Gucci and My friend harpreet Singh together.

They've launched lunchable and it oh, it's probably been probably two years now, right? Yeah. I didn't go to Theory.

I think that almost three. Yeah. Wow.

Well wouldn't in covid time you lose track. But anyway, how free KK welcome. Thank you for coming on.

Yeah. Thanks for having us. Thanks.

So neither one of you really need an introduction our audience, but I'm gonna make you do it anyway, if it's okay KK, why don't you go first and then we'll come to harpreet. Yeah, so I am yeah, I'm probably known that the guy who created Jenkins I spent with almost a decade that cloud these they're very passionate about the developer productivity and sometimes really sick. I'm still up that's lawful.

Actually, I pray harpreet I go back a long way with KK as well. I was VP of products at cloudbees. Then I was at it last seen for a short while with the atlassian bitbucket as the as the head of product there and we both got together and said, hey we need to inflict more pain on ourselves.

We do another startup and so here we are. Absolutely. interviewing you both when we now when you announce launchable, right, so I feel like I've been here since the birth if you will of of this baby and you know, there was a very specific.

kind of problem that you guys saw with lunchable. You didn't just decide to go do a startup there was as often happens with Founders. You saw there was a gap.

There was a problem. There was an issue something that had to be done better and it was around the way we do testing and the way we set up what tests to run went to run him how to run and what we do with the results that whole area. For as smart as we are with devops and cicd and all of these things it was it was lagging if you will.

Yeah, right. Yeah, they wanted the words in your mouth good guitar. Yeah, so it's a little bit about that.

So, you know, I so this is kind of at the conversions point of Serial line of thoughts that little independently happens. So the one is you know, when I was developing Jenkins it became really that's that is the project going on for 10 years. So it's a master a lot of tests.

And then at the point the software also got bigger. So every time you're making a small change and you're always making a small change even with the big spot there and be softer a small change in we are running all the test every single breaking time and I have to wait for a whole hour. And then that started to feel like really slowing me down and I was talking to the other software development teams around the world because of my role and it turned out that I believe on I'll back to your side if my test is done in an hour.

They're talking about like a kind of hour that multiple days that back in the day when you're only releasing like everyone's a month. Yes painful, but he was okay you're trying to deploy like every say in six hours or like every hour you obviously can't have the test cycle that's taking 24 hours the clearly that's the good I see a friction there and then so that's those ideas got combined the idealize. Okay, so we were driving lots of automations and you think he's played a tea party.

That's a lot of software that open things are going through that Journey. A lot of them are suffering is like best problem in that work though that you're going very overboard and they are not really be able And recordings. I haven't yet become a cross anymore products that will feel security is adequate.

And so they're always want to harm more tests, but they also need to release faster. So that creates them in the Rock and Hard Place and some are not surprisingly because they feel like it's such an unsexy things. They don't know they don't talk about things.

They always felt like it's like, oh like it's because our team stocks. No, like everybody have that problem. And then I Facebook came out with this paper in which state it turned out that they have the problem like a hundred eggs, you know, and then as they apply the machine learning 50 years, if you're making a small change, what are the meaningful set of 10 tests that we can run the glorified the verified Authority?

So okay. I I have this problem myself. I'm hearing a lot of people who are having that problem.

I'm even trying to solve them and it's cutting its company. So already that point it's solution so clearly well that's there's a wide enough problem in the world. So that's why thank you launchable.

So and then something like it grew from there that well. So it turns out that the mostly what we're doing in this object development is actually not smart. He talked about in a divorce is became smaller.

It's actually mostly Brute Force, right? So there's so many opportunities that the data could improve the process and then that thinking for example, you can see it in the manufacturing process sales and marketing and other department a surprisingly missing points of train engine. So I always thought that you know, one of the engine leadership that or the ability for us to articulate our organization behaving as a whole building efficiency or is quite limited.

So that's also another thing that they can say, it's that that's the space so here There's a picture I want to put in your mind is you know, we've done that, you know, every ci/cd company is talking about this infinity loop of continuous delivery. And it looks like once you connect all of that it's a very seamless process like things really move fast and you're pushing things to production and so on so forth. So one the data there is not being used.

So that's that's one thing but the picture actually is that it's really it's not a smooth infinity loop. It's it's actually imagine our tire that's letting off air and it's just it's all crumpled up. It's it's bloated on the testing side.

So that's what happens. So people connect and go through the ci/cd process and often they're on this journey for months if not years and then they look back and they say like, okay we did all of this to actually go fast like how fast are we it's not quantifiable and it is not really fast because really the the is bottleneck in there is your tests are taking you a lot of time and and none of the data that is coming out from that exhaust is being used to figure out like can we do the smartly and so that sort of Falls our insight? excellent Let's fast forward to today guys.

right without you know rehashing the last almost three years. Where are we today in lunchable? You think we're you know progress?

What what what what's you know exciting you now? What what are you doing now that because the problem still there obviously, right? This is a this is entrenched.

Yeah. So one of the things that you recently work on is what we call these if notifications, it came out the world reading this, you know, often okay our own practicing of the book and process, you know, it's you know, the people like the this the oh that the test that they review is a program in the code, but they, you know, most of the time today they only get exposed like a one beat to me who made some awesome fail in the pur request and then when they fail when it passes that's fine. Like if that's the passing there's nothing more to think about that's good.

But I mean it fails there's this next step or the working. It's okay. So what is a field how they feel and that like a derives?

Set of actionable for the developers. So there we did this this software image that we provide that instant feedback when a similar tested lover you we tell you where you get those real-time notification in case like and then you show you the relevant setting information that can help you get deduction more quickly. And in this tight Loop there this like a height Loop that the developer goes through every single day, which is like they have a task cut and they make a code change and they push it up and the CIA system runs them and then you correct that mistake until you're happy and then that gets moved on to the next change gets moved up the next days and you walk on it's so that is the part of the academic or like a life cycle your focused on and you know, that that is all free product that anybody could sign up and use we just need stuff or like what month ago something like that.

Yeah. So that's something you're very excited about at the moment. But first take a step back on our three-year Journey right?

When we first talk when we kicked off our vision that we talked to you as we'll build a machine learning platform that consumes data from your tests and your code and start doing something useful. So we launched our first product which was predictive and then we wanted to build applications on top of it using that data. So we built our Premier product was predictive test selections which cuts down these run times of tests that we were talking about.

Right so you can cut down like up to you know, between 40 to 80% is where people are cutting down their test times. So imagine selenium and I just wrote a case study. So it's on top of my mind where team actually cut down their execution time until test by 90% right and then As we were doing this we found that people actually didn't have a good idea about their tests like, you know, so we started talking to organizations.

I was just talking to somebody who thought their tests are actually running once a day and they sent the data over to us and we found no no, you're actually running this once a month. Like there's like a massive discrepancy between what you think is going on versus what is actually going reality. Yeah, right.

So we we took that product and we put that into what we call test insights and then as you're doing this, you know, this conversation kept coming about that devs are sort of getting these notifications very slowly like they are going through CI tools and so on and so forth and I remember a stripe report which said like, you know, death spend about 35 hours of a 40 hour work week dealing with bad code, right? That's your life. And in that context with what course the case saying if the feedback is coming to you slowly.

It's like friction, you know this there's always a speed bump you can never really go. Person consequently, we didn't you know build that product for test notifications and we feel like it's it's a, you know good enough value for people to just put that in their pipeline, you know, start moving a little faster and and that that's that's where we are today. Kind of Harvey you mentioned that case study.

Through the old marketing part of her. Is that available? com and it's on our on the top and customers if you click there.

It's called radical reduction selenium test execution times and there are a few more there where people are using us, you know in all sorts of places. electronic it's funny. You know, I as you both know I've been involved in the devops space now nine maybe 10 years.

com. The you know, the the Scuttle boat was well devops is gonna kill testing right QA people. You better find a new job because we're going to just automated and it's you know QA function.

It's gonna go away. Of course, nothing could be further than the truth. But what amazes me is something like selenium for instance.

It still is dominant as it was and as it is, I mean it was dominant then it's as more dominant now in many ways. is that in and of itself indicative of We're not making progress. or you know was just selenium a unicorn or whatever.

Well, I mean this so then you clearly solves the hard problem which is the I mean one of these like a fundamental challenging in QA is the next observation and manipulation of the Target World of their testing is always hard like a brother being the primary one. So that is a legitimate part problem that they tackle and so well and then there's you know, always in the tech safety numbers and it's open source and whatnot. So, yeah, it's it's incredibly sticky technology and while you can make I'm sure you can make improve incremental improvements, but it's kind of hard the yeah, so I don't think it's like a lack of sign of a progress It's just You know, like I think of it as like almost like I don't know Unicode, you know, I'm old enough that when the Unicode came along it was like a flashy new things and today nobody thinks about that guy Unicode but it's still very much there the foundation of what we do is just building more stuff on top of it.

And then so we tend to talk about the stuff that's like a new year building on top of it. But you know underneath you all got the same token balls here. I mean to me it's akin to Jenkins right?

There are a lot of companies a lot of companies out here who say we built a better Mouse Trot. We have a better sea ICD talk, right Jenkins is old yet. You know, what's the biggest cicd2 still in the marketplace?

Probably Jenkins by by much, right? So it isn't that let's talk about this closing the Gap though, especially in the communications and so forth. That you guys are focusing in on here.

right What's the ultimate aim of it? First of all, right, so I mean obviously you are let's face. You understand why we do it why you'd want this time, right?

So I guess so I Optimist is a long goal. I think they are out there goes like every I don't know how there's them particle go disappear and then the universe dies a slow this but the I think so like the kind of key Milestone or like an old store that I'm thinking about is like, you know, we how can we live or like a law level? Decision making or information gathering delegated back to the machines.

In ways that example maybe that doctors work with nurses like when you go see a doctor, you know, you don't actually get to see a doctor right away. Like you almost gets really by the north season, they measure you and whatnot gets the old information and like in the ready and then the doctor like a strong Stone scene spent five minutes with you and he moves on because like their time is precious. So they divide up the labor and I feel like a lot of what the software developers do today.

It's still like those act togethering, you know, the bubble up these like they're even formation from the ticket and yes, baby or some code changes and whatnot. And then like they connect that then the more into extra more happens. So some of these like a law level work closer to the date and like a facts on the ground that is automatable.

I feel and it's getting harder because the scale of the development. It's Are the gold commissions getting bigger and whatnot so, okay occasionally, we call that like a executive admins for every developer. You know, like that's the kind of Or what?

I think the one Milestone that's still pretty far in terms of the distance. It's not probably the ultimate destinations, but I think it's important milestone. Are pretty.

Oh, no, he nailed I think it and so I'll sort of lay down the breadcrumbs from the hair to the executive assistant right when we thought about even this problem. It's it's you know, when you get into the nitty gritties and their depth of it and you go like on nobody saw that that's that's really surprising that just seems like we should be solving it. And so when KK and me were sort of doing Jenkins and cloudbees that you know, our our ethos is like let's talk to some customers see what the problems are and then tackle the ones that are burning the most and that resonates with me as a product manager.

So what we've done today is solve one of the lowest hanging fruits out there. We're hoping that devs sort of views that and find it useful. We'll find what the next set of challenges on this journey for this Executive Admin to an as developer looks like and we'll pick that and we'll solve that right the data on you know, is there in the platform we can consume that and we can hopefully show that to you in good ways, you know, you connect to us and now we have your data we can say, ah, you know, you thought you're running these like once, you know once a multiple times a day.

No, you know, that's something else is happening. Maybe your tests are failing too much and and that sort of drives different behaviors for you, right? Art is data driven and that has been like our vision 101 from you know day one.

It's like how can we use this data and and get devs to use that data and you know be smarter about like what challenges they pick because their times are really really valuable. Excellent guys speaking of time. We're about out of it here.

I apologize, but we coming up on top of the hour. I mentioned in the beginning of today. com is the website.

Harpreet, you would mentioned that some of these case studies and and use cases available right there. Right people should go check download and check those out guys. It's always a pleasure to have you both on always and you come back soon and keep us posted.

Okay? Yeah. So there's a pleasure to be here.

My pleasure our preaching coaching KK here from lunchable link on text on TV. We're gonna take a break and we'll be right back.