Shifting Application Testing – QUART, Marcus Thomas

This is Textron TV. Hey guys. Thanks for the throw.

We're here with Marcus Thomas who is the president of Court. They are a consulting firm that specializes in test Automation and we're gonna talk about what's going on with test Automation and devops because perhaps things aren't going as well as we would all like Marcus. How you doing?

Welcome the show. How you doing? I'm doing fine yourself.

What fundamentally is the problem? We've been talking about automating tests for as long as I can remembering it most of the tests that people put together are still manual and they're kind of somewhat semi-integrated with their ci/cd platforms with mixed results. So what's the fundamental issue?

And what do we need to fix? I think the fundamental issue is this we talk a lot about shift Left Right shift left is it's been you know, it's been a new mantra that's been you know that people would speak about for it's been speaking about for the least the last 10 15 years. And I think we've gotten to the point where we think we've mastered it.

Right? We um, and because we think we've mastered it. I think we've gotten a complacent with that word shift left, but we actually haven't really truly shipped the shift left, right?

We haven't done that proficiently enough and what I mean by that is this when you go into when you start a test when you go into starting to test automation for project what happens the last thing that happens is test automation. The last thing that application is developed right in the Sprint. Let's say you're talking about the developers the creating the user story and acceptance criteria, you know with this scrum team and then it's handed to the testers to test it and then it's handed to the automated tester to automate it.

Right so it's still very much a shift, right? Concept this in a shortest set test cycle. So what happens now is that software development teams have gotten good at delivering software fast, right?

We've kind of you know master that now and the customers are now hungry, right? They're now hungry for features that used to take Years and we're giving it to them and if you don't scale agile, we're giving it to me three months, right but the automation has not kept pace right? We're still using the same methodology.

We're still talking about the same thing selenium, right? We're still talking about page object model, you know, we're now talking about gherkin right with which one Gerkin is trying to accomplish that when you're right you accept this criteria, you should be able to take that accept this guy to you and literally run the desperate with it. So if that's the case when we talking about shifting further left towards developers and letting them write the tests and they're more automated fashion or what exactly is the solution.

Yeah, excellent question. So what happens talking about what I've been into trying to introduce to this industry is shift down, right? We need to shift down and what I mean by that is is that the product owner Should be as at once they responsible but should have focused on test automation at the time that the features acceptance criteria is getting written and the product owner.

That's not gonna be Technical and they don't have to be involved in actually building out the actual automated test, but they should actually make it take the accountability of saying hey. Here is the acceptance criteria and how we're going to automate the test cases for this and what I mean by that is this there are three fundamental things that you need in order to automate a test. Right you need to have from a technical standpoint.

You need to be able to understand the objects that you're going to interact with in an application. You need to understand what the requirement is for that application and then you need to have a framework that can actually build that out. So that discussion it needs to happen from a product owner standpoint.

The product owner needs to take responsibility. So what we've been doing in my company is we have a shift down criteria that we add to every single user story and what the product owners doing is. He's starting at discussion.

Hey scrum team developers automated. What do you need to be able to automate this acceptance criteria here? So the test automated engineer, it's gonna look at the development and say hey did you think about Adding a unique locator to this object.

I see here from your screen lock up or from the application. You know, I'm gonna use the login example because that's the simplest form you have this username password field here. Do you have something that's gonna uniquely identified as using username text box.

Hey, I noticed that you're using it drop down here, but it doesn't quite operate like a drop down. Is there a reason why we're doing that because that may interfere with how we're automating this test case so that discussion is happening at the beginning of planning, right you you're talking about this and it kind of just the whole Focus not just from what the user wants at the end, but how to use is going to get it, you know, and how quality how clean of a product you want to get because we all know that if you want to get a product fast and you want to get it clean you want to need to be able to automate that test so shift down is the thing that I think is going to help shift that mindset. It's in the product owners get that or they capable of thinking that through entirely or do you have to kind of hold their hands to make that all work?

Yeah, good question. Um, so I think they understand that automation is needed right for sure. They don't necessarily understand how is going to be automated and what is included.

So what we're asking our product owner to do is raise the question the question that you want to raise is how we're gonna automate this acceptance criteria how you gonna build the test cases? And then once that question is raised now in the test automation Engineers going to ask the developer in the same scrum meeting. Hey, are you adding unique locators?

So that's what the product owners doing is just driving the conversation and making sure that the discussion it's had in that Sprint planning session the more that happens the power and the product owner would now have that pattern. He'll understand. Hey, I noticed in this particular acceptance quite to you here, you know, we talking about changing fields.

Did you do you understand? Did you actually think about you know those locators? So I think the product owner would learn to be able to understand that that level of detail over time, but at the beginning of the process, it's not about the product owner being technical enough to answer those questions.

It's just about them being able to ask the question and having a scrum team answer it. Who should be running the test these days because you hear a lot more about developers or running the test, but some folks would wonder you know, is that equivalent of the foxes watching the chicken and the hen house because you know, the developers are going to go. Yep.

I passed that that's great in a way we go with no one's there to kind of verify. So can we trust developers to run the test or do we need to still have some sort of second party validation? Oh, absolutely, you hit it right on the nail man.

And this has been my biggest Mantra right? I just did another speaking of games in my topic was this. Automating tests should be as effortlessly as creating and manual test case, right and what I mean by that is my highlight highlight was automate like a tester.

Right because right now what we see in the industry, the industry is going into different directions, right the first Direction one of the directions I should say is it's going in the coldest Direction where we're training, you know, we're coming out with these automation tools and we're telling a tester. All you have to do is just click the record button and play it back and the script would do all of this magic for you and it will run and with and the industry is starting to see obviously that That is not completely true the second direction we're going in with the event of you know, cucumber and selenium is that we're going in the hundred percent cold Direction like you suggested like some teams believe that they have to have developers hire another development team that does the testing right? So you got two development teams essentially one is doing your testing and one is now doing your Automation and what's happening is we're going back to the pre-wide 2K days where the developers were going most of the testing right?

So I'm a big big component and big fan of having the tester whether they have development skills or not having the tester being in control of automating your tests and maintaining their tests, right? So we are Building Solutions around that concept where there's a hybrid between the coldest UI because I think there's a component of that that is still very much needed when you can quickly. Build automated tests without writing one single line of code.

And then we have a framework. It's an open fast starter framework that you can actually use and to F add custom code for you know, say 20 25% of the functionality and most applications need some level of coding. This is why those completely coolest tools sometimes you're not work because they're trying to fit, you know functionality into that cold is too that doesn't quite fit.

So our approach is to use a hybrid what we have a coolest UI that's integrated with an actual open fast framework that allows you to do that and a tester whether they have programming skills or not can still be able to automate the test case maintain it and writing and that's what we that's what we're pushing in the industry. We hear a lot about Ai and testing these days. How real is that?

And or how much of that is kind of Wishful Thinking in the machines. Not quite ready there write those tests in any kind of way that scales or is Meaningful. Yeah.

Yeah, it's AI. I'm still not sure wish Direction AI is going to take his in if we're going in because it gets like I said, it's industry is kind of split down the middle right now. And but what I hope AI would be able to do without at least what I'm going to Advocate and push for in a solutions that I that I feel the tester needs to be controlled over how that AI is implemented and I'll give you an example.

Let's say for example, you have an object, you know winning application because a lot of people don't understand it testers mindset is what needs to be remain in control of tester. It's not necessarily, you know, it's all about test case design. No matter what solution you build what product you use that tester need to understand how to design test cases those boundary conditions, right the equivalent partitioning right the state transition, you know, all of those things need to be considered when you're building manual test cases, so I don't think an AI can necessarily replace that right because that tester is is capable of thinking outside of the box and AI is within a box obviously, right?

So now here's where I think AI can actually benefit. Let's say we have a text box and in that requirement, we know that that text box can handle 10 characters and it only can handle a alpha string right with a tester should be able to say hey, this is what I want to test script to do for this particular text box and that's what should drive the AI and not necessarily have the AI drive, you know the testing It seems like we still want to put testing at the back end of that development process and then we run up against the deadline and then we kind of skip a lot of the testing and then we ship it into production and then we're surprised when there's a million bugs to be fixed. So it's the whole process somewhat broken and do we need to really think through you know, what tests need to be run when exactly during the pre-production process and the development process and we just never seem to have gotten their heads wrapped around what the exact flow should be.

Yeah, and this goes back to the shift down that is another component of shift down the other conversation that happens at the beginning of writing that acceptance criteria is how are we going to link the tests that we need to automate and need to execute when you developer the minute you say get push and you push that code what tests are gonna be kicked off and not We're not gonna wait until the test hit the test environment right? We're also not gonna get into the code and help you unit. Test either right?

But we do want you to have access to these function. I caught on functional unit tests because they believe they hate it a unit level but they're still functional right and we want you to have access to these functional you to test so that minute yet you push your code right in your local environment. There should be a suite of test executed against that that push and now we're catching everything now, we've truly shifted that right because you're not waiting until you deliver.

Flowing to the test environment and throwing it over the wall to the testers you get that immediate fade back and we talk a lot about shift left and trying to do that but to your point it doesn't happen because test Automation and testing is still in afterthought even within you know, the agile process. And some of this just comes down to context because if I wrote code a week ago and we're testing it now, I'm not gonna remember what the issue of the day was but if I get the tests in your real time do when I actually pushed it. I'm gonna remember what it was and what the issue was and I can act on it faster in the more meaningful way because otherwise I've moved on some other project by the time you get our test results back to me.

Exactly exactly we have these so one of the things we do is we have what we call our smart components we take and the the future, you know with the components and that tells us a functional area the application that it's going to be impacted and we pass those tags along due to continue through the pipeline and that's how we make sure that you know, we're not rely on memory. All too often developers to be frank kind of look at testers as the enemy. So how do we turn them into allies and kind of think about this more as somebody who's gonna, you know prevent you from having an issue later on.

You know, it's funny. Um, you know, I I teach training as well and I try to you know, one of the things I teach my students is that you know, one thing that developers like is a well written but hey bugs, but if you're gonna write it write it, well right give them all the information that they need, you know to be able to you know, reproduce that book. I think with what's going to get developers on the tester side is helping them do the one thing they don't like to do and that's tests unit testing.

It is the biggest thing that they dread right just because it takes them away from the fun. They take them away from actually developing right so we can find a way to help the developers do their unit testing. You know, I think that's gonna be the bridge, you know, that that is going to help you know, he's a lot of those contention points.

All right, folks. You heard it here. Well written test and a little empathy goes a long way.

Hey Marcus. Thanks for being on the show. Thank you.

All right back as in the studio.