Scott Adams – A Small (But Vital) Step In Our DevOps Journey – Rolls-Royce UK SMRs

When you're in a very large enterprise or a regulated industry, transformation can be hard . In this session. Scott Adams from Rolls Royce explains how his team, which works on nuclear reactors, was still able to take incremental steps toward their DevOps transformation.

Let's see how. K. SMR program.

So the UK SMR program, is a Rolls-Royce consortium led program, and we're designing a small modular reactor for electricity generation on proven technology. So why why am I here? So I'm currently leading the performance methods and the program and which is the thermal hydraulics aspects of the the whole nuclear reactor.

And when you kind of break it down into its parts its the data, it's mostly off the shelf software, in-house codes and models. And all aspects of this can use live modern software techniques. So we took the plunge and here to talk you through a small step into the realms of DevOps.

So essentially, we're a really small analytical team and performance. We develop, test, deploy and utilize all of our Methods. And it's it's all to do with providing data, not only through design.

So we're still in design phase of this nuclear reactor, but once we're there, we want to justify it as well. So we want to do a whole bunch of analysis to make sure that it's as safe as possible. And in reality, we've got five nuclear engineers requiring one analytical method with very little software based experience.

So kind of a glance at our project and our method utilizes common nuclear codes, so we've kind of broken all of our models down into aspects. So on the right, you'll see steam generators and pressurizes the RPV. So we've got a model for each of those bits.

And each of the models are wrapped in Python so that we can put what data we need into them. At one time All of those models are kind of stitched together so that we get a whole plant model. So all of this was kind of originally developed, done on a really siloed Internet isolated aging network that had the version of Python that it was built on it.

6. And because we had kind of little software experiments, we we went with the requirements capture technique that we know, which kind of pushed us down the waterfall type approach that I'm sure many of you can relate to. So that ended up with this going from kind of a start to a first release in about eight months.

So Because of this approach, we had kind of limited customer engagement. So we took all of our requirements right at the beginning and hardly spoke to them for about six months afterwards. And since then, we've been doing periodic reviews and and updates of our method.

And it's mainly to kind of keep up the design changes. So there's various teams dotted around making quite major changes in some cases to the design. And we need to change our models to be able to capture that.

And due to the last few years and the fact that we're kind of still in design, it means that we've actually historically had quite a high turnover in engineers on our relatively small team, which hasn't helped with our optic processes either. So why was it that we kind of really needed to change and I mean, this list could be 100 times longer but I kind of went through the fact that we didn't really capture all of our customer requirements because the waterfall technique, you end up getting a snapshot of your customer requirements right at the beginning. And so we've got all of our requirements upfront and then that just didn't change for the rest of time.

And to be frank as well, I don't think our development cycles was really up to scratch for the amount of engineers that we had. So we're trying to kind of push the boundaries so that this this method and approach is really novel in the nuclear industry. And it's quite, quite a big task to upkeep as well.

And with five engineers, that's quite, quite a challenge. We had a lot longer development cycles than we wanted. And because we're we're the same team, I mean, developers, the same people that use our method.

, Um, we had zero configuration control and so. Well, unless you claim chmond r+x in it, when we were originally on our linux platform, but we've migrated over to a standalone Windows engineering pieces. And since then we may have had the tools, but we definitely didn't have the the mindset to properly configuration control.

IDEs was a big problem. So nearly all of our development was tax based. So we have Python we all of our models with based on the old punch cards.

So they were all text based inputs and so our developments were all done using Notepad++, plus or even worse text emphasis. And so that that kind of peer reviews and technical in long winded because you didn't necessarily know what changed. The design was getting increasingly more complicated.

And we were struggling to kind of keep up as is, and we were starting to consider kind of other interactions with other areas. I think one of the main reasons that we needed to change was it was just a really paper heavy kind of process. So we had a five step process for the plan, design, develop, test and deploy.

And at stage three. You'd already broken your solution up into parts. And For each of these parts You need the memo to describe what your new design was.

And then when you were testing it, you need the separate test plan to ensure that it worked standalone and that needs an author and they need to approve it to make sure the test you're performing were right. And needed someone to ensure that you'd done your tests. And then more often than not, the system owner wanted oversight.

So for just that small part, you needed 4 people to look at it. And we were already in a five person team. So if you did it properly, more than 50 percent of your team was getting involved in every minor update.

And then to make things worse, when we deployed, we pulled all the things together into a colossal document with integration testing evidence which just. Was more time spent writing and less time actually testing. So I'm going to put little snippets of advice in my talk, and I think one major one is don't go waiting for the catalyst to implement the solution.

We waited for ours and we waited too long. But don't wait for yours. So our catalyst was Covid19, so overnight we went from working on our engineering and engineering pieces quite happily to having basically zero capability whatsoever, our developments came to a standstill and so our access.

We had some standard laptops that we could use that they kept crashing. Essentially, every time you open up for work they wouldn't be able to run the codes that we need. And so the key requirements for kind of action plan that I put forward was we needed to be able to enable the networking.

We needed to work with what we had. We have no additional budget for this and we needed a quick turnaround, as quick as possible. So essentially, my solution was that I was going to use the IT provisions that we had in the office, and thankfully we've had 10 of these little mini PCs lying around in data training activities.

I was going to repurpose them using Ubuntu, one of them the head host, and use Docker to host a whole bunch of servers that we need. So I ended up using MySQL for our data, GitLab for our lifecycle management, and puppet so I could push some infrastructure to the client node. And why these tools?

So I'm kind of a little bit nerdy, but I run a few servers at home. I've got Plex up and running to my my boys can and watch as much Frozen as they want on their tablets. So I use Docker for that.

I also do a little bit of coding at home I'd use PyCharm and it's quite a nice interface. And given that our solution was predominantly Python, it seemed like the best way to go. Id used MySQL a little bit before in the past, so it seemed like a sensible tool to use and the ones I wasn't too sure of was our lifecycle management.

So because of the the community edition version of GitLab and the fact that we could we could host it by ourselves, it seemed like the obvious choice. And Puppet seemed quite nice given it uses Yamal I've previously used before. And we wanted to go cloud computing as well as this kind of a stepping stone in that direction as well.

So I would suggest that anyone wanted to implement changes do your biggest changes first. I mean, with DevOps, there's normally a big mindset shift that needs to happen and just unlock the potential quick. Get your quick wins, it will be big help.

So I wanted to show our current DevOps score, I mean, it looks relatively low, but I'm really proud of it and kind of shows that what I've implemented is starting to click and starting to work. So people are happy putting issues on the board and supplying comments for peer reviews. Merge requests are being used, which means people are branching, its great.

So where are we're going to go next? I think we just need to push further and further, further down this DevOps train. So I want to utilize this CD for all of our testing requirements, get those automated.

Build method into a container so we can get a Docker image that we can deploy. Eventually go cloud-based, join everybody else using Python through it. What I would say is, you know, what's best is kind of no one fits all solution to DevOps.

So just give yourselves a little bit of time to explore and try and unlock the potential for you. And if you're not entirely happy, just giving someone free reign to go and explore, link it to in-flight tasks. So our setup has been incredible.

So it's been much better than I envisioned. And we've got remote working capabilities, which is amazing, but it's increased our capability to more than we had in the office. We've got a framework to build on our peer reviews of so much simpler now.

And with configuration control Finally, we're starting to automate these really repetitive tasks, which is really good to see. So I'm really pleased with where we are at the minute. And kind of as a closing remark, what I would say is just go just yourself, regardless of your position, if you if you feel any part DevOps would give you a benefit and then make those changes.

Um, if it really is a benefit, the justification will be simple. So I really hope some of this story has resonated with some people, and thank you for listening.