Darren House – Secure by Design Principles in Cloud-Native DevOps – DevOps Experience 2024

Hi, my name is Darren House. I'm the C two O of Nextmark Cloud Company, and I'm really glad to be here to talk to you about the Secure by Design principles and a data and DevOps. Um, so we're just gonna dive in.

Um, so I'm gonna be covering the role of architecture, of system secure by design and default talking about how serverless designs can align with the concepts of bookends. And then I wanna share some real world examples about the importance of the architectural perspective. But at the same time, I really want you to consider a couple things while we're talking about this.

The first thing is I want you to consider that architecture plays a crucial role in enabling secure by design principles, especially when we're talking about cloud. That needs to be a serverless shift in the traditional three tier model. And this becomes a very important part of how we look at architecture today.

And last, I want you to really consider that secure by default and by design architectures using serverless technologies, they really can minimize the cognitive load and firefighting for all teams across the board. So first, let's talk about a couple definitions, right? So I mentioned architectural perspectives.

So these basically take the whole of business view to try and align technology and design and implementation along with business outcomes. And that's really what I'm talking about when I say the architectural perspective is taking that whole of business perspective. Now, I'm also gonna be mentioning Secure By Design a lot.

So what is Secure by Design? Well, it was created by this organization called cisa, the Cybersecurity and Infrastructure Security Agency, which is a part of the Department of Homeland Security. And I really just want you to think about the Secure by Design as SSA's extension to Shift left, but from a broader industry-wide industry-wide product and application perspective.

Now, there's a lot of principles that are involved in the Secure By Design, uh, framework, but I'm gonna focus on three of them today. So the first one is that end users really should not be responsible for Product Secure. The second one is that we as an industry, we have to get better at reducing vulnerabilities and we have the ability, we just have to do it.

And number three is we should be able to improve product and application resilience to that cyber threats. So I really encourage you to check Outa Secure By Desire website for yourself, read their white paper, it's really good information. Um, but moving on, I also talked about the term cognitive load.

I'm pretty sure everybody's heard that term. It's just basically the mental effort that takes to process and perform tasks. And it's something that's growing in a lot of the domains like security and DevOps and multi-cloud and you name it.

There's just a lot of growth and, and this cognitive load that we all have to deal with. And oftentimes we have to synthesize and make really good choices about technology. Oftentimes there's really limited information and that just, I exacerbates that problem.

But altogether, I just want you to understand that the architectural perspective using cloud native serverless capabilities can simplify systems secure by design goals. And it does that by abstracting infrastructure management. Uh, it does it by automatically applying security best practices, which are often considered secure by design or secure by default, and also can help reduce the attack surface and also can reduce, um, or minimize product and application EXPLO exposure to vulnerabilities.

So these are big deals. So let's talk about, uh, the concept of bookends. And with serverless, I find this concept really helps to simplify these conversations.

So it doesn't matter if you're developers, architects, infrastructure or security ops teams, they all have the ability to start aligning on balancing cost and risk using minimum and maximums, right? And those are the bookends I'm talking about. So for example, if you have to design a new web app or a product, there's gonna be a lot of complexity, especially trying to figure out the specifics, uh, performance requirements.

Um, and these can impact the technical design and the trade-offs that you have to make for each component and each tier of that three tier web application. And because it's gonna be a new application of product, oftentimes you're gonna have no data. You have no baseline, so you don't know what the applications gonna require.

So you have to start inferring things, but you really don't know. So the ability of using serverless and being able to do bookends with minimum and maximum parameters, it really can help you manage your cost and risk. Uh, on the serverless side, it can also simplify this.

So you can make assumptions and even hedge your bests, hedge your bets on the risk and balancing performance and resiliency, while also eliminating unexpected costs. It also can help you make faster decisions and reduce a lot of that operational risk by focusing on those minimum and maximums and thresholds that you met, that you measure. So before we dive into the details of serverless and bookends, we need to do a quick refresher about the traditional three tier web application architecture.

That's where this is displaying here. It's the presentation peer tier. You're typically running on engine X or Apache and they're running on lytics or S boxes.

You're handling inbound outbound requests, you're handling, uh, how those requests are handled. You're managing security a little bit for that front end, um, ensuring just basically the smooth operation of that front end of the application. Then you have the application tier where all the business logic is, uh, handled and processed.

And this typically will use Java based servers like Apache, Tomcat, JBoss or dotnet servers like IAS or using no JS for example. And then those will usually communicate with that data tier as well that stores and manages that business critical data that's in there using relational databases like SQL or no SQL databases like MongoDB and Cassandra or file and object storage solutions. Um, so this is just a high level view of that three tier architecture 'cause we're gonna compare that to the serverless architecture and talk about the values that you can get from serverless that you don't get from this traditional architecture.

And one of the things, things I really need you to understand is that with this traditional three tier architecture, there are gonna be these common design configuration scaling issues that you just can't avoid. What are those gonna be? Operating system security of operating system hardening the patching for the operating system and for the application.

The whole vulnerabil vulnerability management program that you have to manage all that, the agency you have to deploy and manage the compliance requirements that you have for configurations, um, the scalability and backups, all these things that you have to manage. These all increase that code and load that require to deploy a lot of apps and products. Now, even if you standardize on the hardware and operating systems and applications or between the tiers, you're still gonna have to deal with a lot of these complexities that way.

We look at it from the serverless side, right? The cloud native serverless side, this is a three tier model. And here we're using AWS environment as an example to describe this.

Um, and I need you to understand here that you still have the same functions in that you perform on each tier, but now you're dealing with the actual cloud services that deliver a lot more secure by defined design and default values than you do from the traditional architectures. So, and the presentation tier, you have requests that are gonna be first handled by CloudFront and S3, which can steal 'em massively with uh, no or minimal configurations. And that it reaches down to the app tier for dynamic requests to get routed from CloudFront to API gateway, which will then, uh, manage those app routes to different random functions that perform application tasks.

And typically those will result in reading and writing to and from data stores like over world service, uh, V two or Dynamo db or S3 buckets. And there's a lot of benefits over this, the traditional design, and I'm gonna dive into the details specifically for each layer on the next couple slides. But I just wanna give you a high level view of the things you can offload from your copywriter plate, as I like to call it, all operating systems.

Security hardening and patching gets offloaded because the serverless capabilities are managed by the cloud provider itself. That is a huge benefit alone. All those operating systems, security and hardening and patching requirements, there's a big majority of the vulnerability management tasks that you can offload.

Now, yeah, you still have to manage the runtime and dependency vulnerabilities, but that's a small subset of that larger vulnerability management, uh, tasks that you have. There's no more installing, uh, or managing agents. And all this becomes a significant, uh, cognitive, uh, reduction in your cognitive load due to these practices.

In addition, uh, compliance frameworks, there's a lot of work that you have to do for client compliance frameworks that you can offload because you're loop using serverless services. And we'll dive into that much, much more, uh, in the next couple slides. So let's go ahead and dive in for the, uh, presentation tier, right?

So in this one we're talking about CloudFront and Amazon S3 buckets. And I'm gonna be focusing on a couple of the security design and default features that are inflated with serverless that will come either at zero or very minimal configurations. And I want you to understand that this diagram, it's, uh, category is using the Mitre attack framework.

And Mitre attack is basically a globally accessible knowledge base for adversarial tactics and techniques that are based on our real world observation. So this gives you that indication of what you get out of the box that is secure by default and secure by design based off of industry observations of what attackers do. Um, so the first one here is SSLT LS encryption.

This is a simple check box that mitigates that technique, 1557, the man in the middle, where with CloudFront it's easy that all sessions be use H-C-T-P-S connections of the latest TLS protocols and it prevents ea dropping, it prevents man in the middle attacks and it can prevents sensitive information from being intercepted. The next one down is, uh, field level encryption. This can be configured in CloudFront so that you can encrypt your fields before they get forwarded to the origin.

Now this mitigates the technique 1213 and it does it by adding this extra layer of security, uh, for sensitive data that is protected, even if that origin gets compromised because it gets encrypted at the cloud front, uh, layer. So when it's on your origin, it's already encrypted. Uh, the third one is, uh, custom HT TP header requests.

You can use that to send your origins. And this allows you to validate requests and mitigate, uh, 1590, um, by preventing direct access to the origin. And it also helps you obscure a lot of information about the backend infrastructure.

So on the right hand side, we have a little bit of information about the compliance pieces. And it's important to understand that just by using these services when done right using best practices, you could typically transfer up to 70 to 80% of your compliance efforts to the cloud provider. And this is because that provider is already has attestations for these, uh, common frameworks like PCI and like HIPAA and soc CIS and FedRAMP and, and more.

But using serverless means you get to inherit what the cloud provider's already done. So that gets off your plate and that was a huge benefit for you. So let's go to the application tier and again, we're gonna be focusing on a couple of the secure by design and default, uh, features included with serverless, uh, that have either zero or minimal configurations and focus on my attack framework to provide information about that.

So with API gateway, you can do rate limiting for request quotas and you can do throttling of limits and that that can help mitigate, uh, T 1499 that's focused on denial of service and distributed denial of service tax. These are easily done with check boxes for the most part. API gateway can offer, uh, input validation, uh, and request filterings to mitigate 1190.

Uh, that can validate requests and reject malformed, but potential malicious requests. Another big benefit for you that, uh, provides secure by design, secure by default, Lambda provides automatic patching of runtime environments against known vulnerabilities. And that mitigates 1190 and this is a huge transference of your risk and work on your plate.

And the question I have to ask you is, well, why would you keep this on your to-do list when the cloud service provider has already perfected the art of this? Get rid of it so you don't have to worry about it and focus on value in your organization and your applications. And again, compliance when you use these things, when done right, you can transfer 70 to 80% of the compliance efforts to the cloud provider.

So let's look at the data tier. Uh, we're talking about uh, Aurora, uh, serverless version two. Uh, this natively provides encryption at rest, mitigating mitigating T 15, uh, 52.

And this can protect you from any credentials or other sensitive data that might be stored your databases. And not only does it encrypt that database, but it also encrypts backups and snapshots using A-W-S-K-M-S. And like Lambda Aurora also provides automatic automated patching, which will take care of uh, T 1190 that will reduce the exposure to non vulnerabilities.

And these can potentially eliminate classes of vulnerabilities from your workload. Again, a huge benefit from the vulnerability perspective and S3 as well at natively supports encryption at rest. Um, so you can uh, basically do a checkbox to require encryption for your S3 buckets and it makes it simple and it aligns with those secure by design and secure by default principles.

So let's take a little pause and consider some of the small ways that you can reduce firefighting and cognitive load with a serverless shift in a three tier architecture. Now this table that I'm showing here, it's a very small part and a much larger secure by design picture of eliminating classes of vulnerabilities like unpatched software. So lemme ask you a question.

How many tools and systems do you have in place to manage and deal with vulnerability management in your environment? I'm assuming it's a lot. And each one of those becomes more and more complex and that adds to that cognitive blow.

And let me ask you another related question. This one, unfortunately, I'm betting that most of you're gonna answer yes, but have you ever had to jump through a bunch of hoops to patch a critical vulnerability on a resource that wasn't directly accessible from the internet? I'm sure you have, right?

And this can be an all hands on deck effort because of critical vulnerabilities and especially if there's an active threat for it, if there's somebody exploring that vulnerability, it becomes a huge effort. All hands on deck. People have to try and get all, collect all the resources and do the mitigations to get rid of that uh, critical vulnerability.

Even though the risk might be low because it's not directly accessible from the internet, doesn't matter, it must be a priority. Now, can you imagine not having to worry about that? I think that matters and that's part of what serverless can do for you.

Now let's look at it from a little bit of a different perspective and that's the compliance perspective. There are 318 controls in the NIST 853 rev five moderate uh, framework. Limiting your lift of compliance to 20 to 30% means that you're only responsible for 65 to 95 controls out of those 318.

Now SO'S gonna be responsible for all 318, but you can transfer that risk and that cost and that effort to the cloud service provider because they've already done it and then you inherit those controls from them, right? This possibly removes 223 to 253 controls from your list or something in your organization's list from an organizational perspective looking at the architecture that matters and that can have a big impact on your organization. Um, now the results may vary, uh, but compliance offload really is a real value and it depends on the how secure by design, by default you use those uh, capabilities.

So now I'm gonna get to the part that's really fun to me. This is how we apply the architectural perspective of secure by design using serverless technologies and the traditional three tier model. And so this slide on the next slide, it's gonna describe this three tier architecture that can dynamically scale resources throughout all three tiers.

And it'll do it based off of utilization. It'll be doing it by managing thresholds, using minimum and maximum ranges to provide bookends for costs for performance and for several resilience. Now this uses lambda concurrency, it uses target tracking and application auto-scaling and also uses RDS aurora serverless B two auto scaling.

And it uses these to ensure that the application and the database tiers that they can scale with the demand from the presentation tier while maintaining performance cost efficiency and resiliency as the application workload can fluctuate due to normal conditions or it can fluctuate due to a cyber attack and you can scale an an out for that so that your customers don't realize that there's an attack going up. Now a quick definition, uh, 'cause I just brought this up, Lambda provision concurrency, if you're not familiar with this, that's where AWS will pre initialize these execution environments for lambda functions to reduce latency. And by having that, you know, have that coldstar latency so you add better performance for your applications.

So it's a really, uh, it's a good uh, feature for you to use. Now remember back when I was talking about the bookends and the uh, slide earlier I discussed the cost of risks of new applications, uh, through the bookends of minimum of maximum. Well this is a scenario where I designed a new SaaS application to handle up to 150 Concur requests during peak hours.

Now to offset my costs, I decided to ensure low latency for the first 100 of those requests at any given time. But I also wanted to focus on maintaining about a 70% utilization for the provision concurrence usage to gimme some flexibility, cost and scaling. And that's what this configuration that you're seeing is doing.

First it's registering a lambda function in application auto scaling, using provision concurrency as a scaling dimension. And then it creates a target tracking policy to maintain a 70% utilization of that provision concurrency. So this means that if my current utilization was at 35%, which is half of that target, then the result would be that the system would scale down to 50 provisional concurrent uh, functions, which would reduce my cost because I'm not using them.

I'd read them. So those drop down. Now all of a sudden if my current utilization jumps up to 95%, that's okay.

Lambo will want to scale it to 1 29 because that's how the math works out. But it's capped at a hundred because you don't wanna go over that cost. You can modify that at any time if your cost, uh, parameters change or anything else changes.

But here you can capture costs. And so any requests over a hundred would end up requirements called lambic cold starts, which might not do a little bit of a additional latency, but the application still performs. But you focused on the the first 100 and you can balance that out.

Um, also at the bottom of this policy, just so you know, uh, I also added a 62nd cool down period for the scale out actions and then a five minute cool out period for the scale out actions to prevent fracturing, which you definitely wanna try and avoid. So this is a very simple, easy way to do a minimum of maximums with thresholds to simplify your architecture and have it scale as you need for uh, whether it's just normal, uh, dynamics of the application or for cyber resilience. Now this is great for the application tier, but what about the database tier?

Let's go take a look. So you can align your database tier, uh, along with the application tier. And in this scenario you can see I just updated my minimum and maximum a CU.

So that's your work as units and that's a mix of CPU memory and networking throughput. 5 to 16, that's the range, the min and max for my read instances for uh, read requests, right? And then I registered a scalable target so that I can automatically spin up new, um, I'm sorry, the first one was for right instances.

And then I can set the read rep replica for uh, spinning up new re replicas to handle the increased DDD reload. And so I can assign the scaling policy that will track the average CP utilization and to target a 75% utilization rate. So that means that it helps me connect my application tier and my database tier, simplifying that app design and also simplifying the troubleshooting process that uses predefined but adjustable metrics for target tracking and auto scale.

And then application auto scaling will try to maintain an average CP utilization of 75% across all my READ replicas. 5 to 16 a CU range so that I can have that flexibility in managing performance, cost and risk. And like I said, I can modify it at any time if I need to.

So now in addition to the CPU utilization metric, you can also use database connections metrics as well. So you have multiple metrics you can use on that. And this is also supported in DynamoDB and ElastiCache as well.

So these become very powerful tools to provide at minimum and maximum ranges to simplify how you design your architecture and it can also simplify your cognitive load and reduce your troubleshooting because the application, the whole stack of that application can scale up and down based off of utilization rates. And that's a great benefit for you. Now I wanna move on to a, uh, a story I guess I'm gonna tell you.

And this is something that had a huge impact on me. Um, 'cause I witnessed something that I've never seen before in my life and this really changed my perspective on the power of serverless, uh, primarily but also the power of managed services with serverless at the cloud environment. And so here we're talking about finra.

FINRA is the financial industry regulatory authority. It's a non-profit organization that regulates broker deals, uh, broker dealers and the securities industry. And one of the big parts of their mission is ingesting tons and tons of data to analyze market data, looking for illegal activity, uh, fraud and abuse.

So the article that you see on the top left, this is just one of many articles from this time that we're calling out how their cloud strategy, which was defined from an architectural perspective of defining how they design applications and infrastructure, right? And this perspective, it guided them to eliminate any non-differentiating commodity from their architecture so that they could focus on value. That was what they tried to do and that led them to focusing on Lambda S3 and EMR Elastic MapReduce, uh, managed architectures for some applications that were used to be able to check for fraud in, in markets.

And this design and this perspective got them through some massive volatility in 2017 to 2018. And so let me give you a couple stats here as well. So in 2018, RA held 28 petabytes of data.

And just so you can understand that, that is enough to play 40,000 years of music without repeating once. That's a lot of data that they collect. Now in 2017 they average 35 billion records that they, uh, process daily.

By February of 2018 it jumped to 53 billion. So that's almost double. Now on October 10th it went to 107 billion records, and on October 11th it went to 135 billion.

This is a massive increase. These are big numbers and I was an AWS architect during this time and I was doing some work at Fenra and I happened to be in their offices during this period in October of 2018. And that's why I'm saying this had a profound effect on me.

'cause it any traditional organization or tradit traditionally design application, when you're utilization doubles in a triples in a short period of time, I'm telling you it'll be a hair on fire exercise just to keep the systems running, much less trying to, hoping that you're not gonna lose data, right? It's a big deal. And I've been to some of these fires and they are not fun and they take a lot of time and they exhaust you.

Now this would've been a multi-week effort. One to try and not lose any records. Two, to try and figure out how to get the system not to crash and then try and get it, figure out how it can catch up with all the, the processing needs to do without losing data that's coming into the pipeline.

'cause they're still getting every day more data coming in. It's a big deal and it's hard to do. But because of their architectural perspective, ephemera, this was just another day, you know, the only reason that they really knew something was going on is because they had dashboards that they were measuring the volume and the data that was coming in.

Um, so that's how they knew. And as they realized what was happening, I remember that I took a look outside of the cubicles and down the hallways I walked around the offices and it was crazy 'cause everything was so wrong. And my past experience with issues, issues like this, it's not normal.

There's people running around trying to solve problems. There's RA voices raised, you're trying to, you get focused on trying to figure out these problems and it has an impact on them. But this, it was surreal.

They were just focusing on their daily activities. And so I came to a realization that day about the power of cloud, the power of Salus, and how architecture can truly have an impact on cognitive load, on the business outcomes, on firefighting, on all these things. And so my hope is that I could start helping and guiding other people to get to this place, right?

And the place that we need to get to is this, where you can close that loop on firefighting because your architecture is designed to handle those fluctuations. You know, today this includes taking an architectural perspective, uh, using secure, bright and design principles. It also takes, uh, using serverless technologies designed with that concept of book ends in mind so that you can reduce that cognitive load and the firefighting for your teams.

And you can do this because the cloud providers have already built in security and flexibility and observability and performance and resilience and all these things into their services so that you can get to a place where you're focused on the future while you manage the operational metrics and pushing buttons and turning scalable knobs. Um, you're not spending massive amounts of time troubleshooting or redeploying or on all hands to solve issues. You use the cloud provider's best practices, you focus on security, flexibility, observability, performance of resilience.

So hopefully you can be like this dude who's chilling in front of a green screen focused on the future. So I really appreciate your time. I hope this was useful and informational to you.

And, um, thank you very much.