The Hacker Mind Podcast: Incident Response in the Cloud

Incident response in the cloud. How is it different, and why do we need to pay more attention to it today, before something major happens tomorrow?

James Campbell, CEO of Cado Security, shares his experience with traditional incident response, and how the cloud, with its elastic structure, able to spin up and spin down instances, is changing incident response. 

It’s 3am and the call comes in. There’s been a major data breach, and you’re booked on the next night flight out, at 6am. As you hustle to the airport, the team already onsite is collecting the log files and the backups as needed so that when you arrive you can begin the investigation. 

There are the servers you control. There are servers that you don’t. Routers. Laptops. And other hardware. 

But what happens when an incident happens to an organization that’s entirely in the cloud, where developers can spin up and spin down new instances ? When log files are not all that detailed.  How can investigators go back and figure out what happened and when?  In a moment, I’ll introduce you to someone who is thinking about that problem and actively working to resolve it.

[Music]

Welcome to The Hacker Mind, an original podcast from ForAllSecure. It’s about challenging our expectations about people who hack for a living.  I’m Robert Vamosi and in this episode I’m exploring incident response in the cloud. How is it different, and why do we need to pay more attention to it today, before something major happens tomorrow.

[MUSIC]

VAMOSI: If you haven’t been paying attention, cloud security is critical right now. It simply doesn’t work to say that you can take your existing security and port it into the cloud. For one thing, we’re talking about microservices and containers that can be spun up and spun down as needed. Which creates some interesting challenges such as how do you respond to an incident when all the data can be cleared out of the cloud? So I decided to ask an expert.

CAMPBELL:  James Campbell, CEO and co founder of Cado security. Cado Security is a cloud security company that provides a platform to our customers to allow them to investigate and respond to incidents in the cloud.

VAMOSI: Perhaps we should first define what is considered incident response?

CAMPBELL:  Incident Response is the, I guess the NSA is always close to the topic of, you know, responding to, you know, suspicious or malicious activity that you've detected. On a computer system or network. So, you what you want to do is be able to understand, okay, I have a suspicious detection, you know, is this something I should be concerned about? And if so, you know, what, what is causing this alert? What's the root cause and how do I know, mitigate it and then prevent it in future?

VAMOSI: is this something that a company would sign up for in advance or is it at that moment when they go oh, crap, something's going on. We better call it incident response

CAMPBELL: A bit of both actually say the gold prepaid companies say the kind of more mature ones in the security domain, usually highly regulated industries. They will tend to have their own response team or their own response plans that are kind of tried and tested and so when they detect something that is suspicious, they'll have a plan or procedure in place to, you know, act accordingly. Determine what the actual detection is, is it something they care about and then this is how we mitigate and move on from that. In some cases, you'll have some companies where it's it's a bit of a you know, an aha moment I was gonna swear that I shouldn't let a you know in one of those moments, and, you know, if it's something that's a bit bigger than they can necessarily handle and they need some extra expertise, they may call upon, you know, like a service like a Mandiant, PwC, IR, etc, to an external consulting firm to come in and help investigate in more detail.

VAMOSI: Given that the incident response is in the cloud, I’m imagining that you can be anywhere in the world or is that not true? Do you have a go to on premise, right?

CAMPBELL:  It can be anywhere in the world. That's the unique thing about clouds that's a different challenge. It represents I guess, or a different way of doing things, not necessarily a different challenge. In fact, it can actually be a Savior as well, as, you know, detriment. So, you know, if you think back in my old days, right, you know, when cloud wasn't really a thing, and everything was on premise, you know, I would literally get a call from a customer. You know, next thing you know, 30 minutes later I'm grabbing a bag, I literally had a bag ready to go by the way. I literally grabbed a bag into the airport on the fly and, you know, told my girlfriend I'm not entirely sure when I'm gonna be home. I'll see you in a few days. And then you go and help the customer on site because the data is on site and you need to actually kind of get hands on to the point we used to do imaging in big data centers and stuff and it take hours because terabytes of data and you'd have people sleeping in the data center, like which is crazy. Cloud has actually really changed that because now you can do it all remote things happen in minutes rather than kind of hours and hours on end. And it's kind of really changed the game there. But I think the industry is still catching up. It is now different to what it used to be.

VAMOSI: Even so, I’m imagining that on site is still necessary. You can have hybrid clouds, you can have private clouds.

CAMPBELL: Yes, absolutely. There's, you know, there's more and more kinds of cloud native companies coming out in an organization. So where do all their operations mean? They rely on all their operations with cloud technologies and cloud based technologies. That said, they tend to be the newer organizations so the kind of organizations that have been around for a while will tend to always have some level of on premise, you know, computing environment. So you know, on all the data centers and stuff like that, but it is shifting, so a lot of the kind of data is so important data workloads, you know, all the kind of number crunching, so to say where you need a lot of computing is starting to shift to the cloud. I think where, you know, where I guess the last move will be is, you know, I'm writing right now as I speak to you, I'm on a laptop, you know, everybody's gonna have a laptop everybody's gonna have like a local physical device to connect to the cloud environment and, And largely that's going to remain kind of on premise, so to say for a little while yet, but definitely a lot of the kind of critical data is starting to move to the cloud and at a pretty rapid pace as well. I've seen over my last 16 odd years in cybersecurity.

VAMOSI experience in the past, going on site and doing traditional digital forensics, How's it different, doing forensics in the cloud?

CAMPBELL: The cloud is a real enabler. There's definitely some tricky things in the cloud. Maybe we can touch base on that separately on some of the pitfalls of cloud, but I guess I'll talk about some of the positive points at least at the moment. And some of those are, you know, one, your ability to remotely access the data you need and quickly. The downside to that is it can be quite complicated on how cloud works and the type of, you know, kind of infrastructure and technologies that are available. So this is where that kind of gray area lies and we can kind of dive into that later I guess. But if you know what you're doing it is you can get access to data relatively quickly. Whereas physically on prem you are literally flying someone out. You know, I did a job once where we had a customer and involved compromises at different servers right in and, you know, we literally had a whole team just to do data collection. And it took us a month to collect that data, like a month like meanwhile, there's an ABD group running around the network causing havoc. But you know, that just shouldn't be the case anymore. You know, because what we had to do is actually go out physically obtain the data, bring it back to a central location, copy the data across to another drive, process it one by one with a whole bunch of tools and open source solutions, etc, etc. And then eventually, you come out with a timeline of the events of what's happened. And the great thing about cloud is, well, now I can just grab all that data within minutes. Instead of having to fly people out all over the place. I can use the cloud to process that data in parallel. So you know, so I can spin up more resources because I need more resources to process the data. I don't have to do it sequentially. I can do it all at once. And then all of a sudden, you have a picture and you no matter of moments rather a matter of months, which is that's game changing at the end of the day.

VAMOSI: it seems to me that there might be more transient data harder to capture in the cloud.

CAMPBELL: Win. Yeah, absolutely. So you know, if you think about some of the some of the really kind of utilized technologies that are in the cloud, if you move to the cloud, you know, the one thing you're going to try and do is save money. That's one of the reasons why music that is right is you know, instead of having a metal box sitting there, wherever your way of doing resources, you know, with cloud, you can grow and recycle your resources. So you've got auto scaling groups, or like virtual machines and kind of easy to use this as an example, your containerization as well instead of list environments, where again, you're only kind of using the resources you need. This is great. This is how you should use it. This is how big enterprises save millions of dollars a year. And they're basically their IT spend and they can put that towards other things which is great. But the challenges that represent right, you know, I've got some customers, they'll have containers, which are serving their customers. So not sure if the visitors are necessarily familiar, but yeah, the kind of computing kind of virtual machines in the cloud, but even have a smaller instance than that kind of a very small computing workload kind of instance called containers, where essentially you do kind of very short lived kind of services for external customers or internal customers. And anyway, so you have these kind of short-lived systems which live in the cloud. And a lot of my customers use their systems every last 15 minutes. So that's great. Like, you know, that's great use of resources that you only need the downside to that is and this is where the industry is starting to slowly mature to this is right okay, I had a detection at lunchtime while I was at lunch. Something suspicious happened in my container in this kind of micro system. But by the time I've come back from lunch, the system's gone, right? It's been recycled, it's deleted. So how do I work out? How do I investigate? How do I even get the data in the first place? For a system that only leaves for 15 minutes? Like, these are the kinds of challenges which people are starting to face now with cloud environments, you know, the use of ephemeral infrastructure or scaling infrastructure. You shouldn't be embracing these sorts of technologies, but it starts to bring in a new risk, which you didn't have before, necessarily in the on premise world.

VAMOSI double back on that, you can have containers spin up and spin down. But the data goes, the logs go everything goes

CAMPBELL: exactly like there's usually some levels of logging but the detailing is so high level that it's really hard to ascertain exactly what's happened. So you might see there is a suspicious login as an example. And, you know, you might, if you have some sort of detection technology in place, you might even see there was a file created, but you're not going to know what was in the file. Right? You won't know if they dumped out a database and it was in the file and it was exfiltrated. Now if you're in particular, you know, part of like a regulated industry, yeah. What are you going to do when an auditor or a regulator comes to you and says, Oh, hey, we noticed you had detected a container and it was suspicious. Well, what was it? And then in most cases, people just can say, well, we don't know. We don't we didn't have the ability to grab that file and investigate a time because it disappears. And so you know, this is definitely one of those challenges people are facing and you know, one of the groups that we track at the moment for hacking group, they compromise. They do like crypto mining and containers and stuff. So seems relatively benign, but one thing a lot of people don't realize is that they have a detection for crypto mining and they'll just destroy the system. But one thing this group does with their core team TNT, by the way, is they actually still have some Cloud credentials. Which are potentially sitting within that locker system within that container. And those credentials, actually potentially have access to the underlying cloud environment to service systems and things like that. But you wouldn't know they stole it. You wouldn't know that they potentially gained access to those, you know, credentials. Unless you investigated in some form of like, traditional insert response or, you know, forensic like manner. And so a lot of customers out there tend to not understand how they're getting compromised through other methods because of game credentials, and they haven't been able to investigate and see the fact that credentials are stolen, so yeah, so it's quite interesting. You know, the dynamic is changing a little bit and, you know, the adoption of these new technologies has caused that.

[MUSIC]

VAMOSI: So there are cases where, where legitimately, an event has occurred, but after the fact it's hard to pull all the strings and find out what that event was.

CAMPBELL: Yeah, exactly. Right. Yeah. Exactly. Right. So it's very, very, very hard to piece together a puzzle when, you know, the puzzle has disappeared. The pieces I should say disappeared.

VAMOSI: But you have examples of successful data forensics in the cloud, do you not?

CAMPBELL: Yeah, absolutely. And I think the, you know, this is, you know, this is part of what we're about at Cato, I guess is is trying to educate the industry that, hey, actually, you can investigate, you can do forensics in the cloud. And what you need to do is use cloud native technologies to do that. You can't use your old ways of doing things as you did on premise because it just doesn't translate, you know, they're like different sized Lego bricks, right? They just don't fit together. You know, people try to jam them together, but it doesn't fit. It looks like a Lego brick, but it doesn't work. And say, you know, people need to understand, you know, what is different about the cloud, how are they using the cloud, you know, and what are the gaps they need to feel now technology is how you build those gaps. You know, because one of the other problems is that clouds are complicated, right? Like now I'm a security expert, he has to know all that on premise stuff, you know, data centers, firewalls, etc, etc. EDR is SDRs for every solution you can think of, and now we're moving. Now I have an organization that's moving to AWS just as an example. And hey, we're also in his year as well, because we have a license for Microsoft. So we're in two different clouds. And we're using two AWS, as an example has three different container technologies. It has lambda which is serverless, you know, functions, you know, basically discrete tasks that you can do as a one off without even starting your system in the first place. So it's kind of like, what is this kind of duty? What is this like magic? That is so different on premise, well, like how is the security guy or girl I should say, you know, how should you know for them to learn how every different technology works in the cloud? That's nearly impossible. And for each one of those technologies, there are different ways and different levels, you can gain access to data, and you have to learn that too. And to do all of that manually would just be an impossible task. And this is where, you know, this is where you should be using Cloud to do a little automation. And so automation is the key here. And this is how you solve a lot of those problems of complexity. And how does this technology work over another technology? The idea is to automate that out of your life. Because as an analyst, all you care about is the system x y Zed is compromised. I need a system x y Zed I shouldn't need to care that's a letter. That's a container. That's a, you know, virtual machine running in a year in Japan. I shouldn't need to necessarily worry about that. I just need access to data to investigate it to see if it's a real threat and how do I mitigate it as soon as possible?

VAMOSI: So you alluded to the gray area of where the data actually is. Yes. Would you like to expand on that?

CAMPBELL: Well, yeah, that's a good question. They, I guess this is kind of going a little bit back on the previous point is for each of the different technologies, you know, the data is represented differently. So as an example, with AWS, there are three different ways you can spin up a container. And for each one of those ways, there's actually different data sets you would need to investigate in order to understand if there is something you should worry about, something suspicious or malicious happening, what's the recourse etc. And to varying degrees, you know, is the detail of that data so as an example, you know, in AWS, you can run what's called a Kubernetes cluster and on that particular environment, you can actually get quite a lot of data, you can actually get like a copy of the container and what's called the node which runs the container and a lot of log files around it. Because generally, you have access to the infrastructure, more or less and so you can actually do a relatively detailed investigation that says, there's another service called fargate in AWS and AWS, like absolutely amazing technology. They run the infrastructure for you. You don't need to run the infrastructure. But what does that mean in that kind of shared model and that shared responsibility model is you don't have underlying access to the infrastructure in the say, the amount of data that is available to to actually investigate something's happened is a lot like is very limited, basically. So your understanding of you know, the difference between those two different container technologies, you know, it's very complicated. Process and, and there can be big gaps in your ability to actually investigate like, and work out what the root cause of something is and how to mitigate that.

VAMOSI So that's interesting. If it's managed by the cloud, you have less access to the data than if it's fully managed by the organization.

CAMPBELL: Yeah, that's right. And, you know, this is all part of what I call the shared responsibility model. And this is something which I think, you know, a lot of customers, you know, really struggle to grasp with as well. So I think with people moving to the cloud, they will kind of automatically assume they release some level of responsibility from a security perspective. And while a part of that is true to the physical infrastructure that, you know, these kind of cloud systems are running on largely you're actually responsible for your own security and so there's a little bit of a misconception there now, through automation through patching and things like that. These sorts of things are much easier in the cloud. So technically, you should be in a better place. Once you understand how the cloud works. That is, you should be in a better place for the cloud to be more secure than your on premise environment. But you know, you need to understand like, what Where does your responsibility from a security perspective, like compared to your cloud environment and the particular cloud services they are offering so and there's often a bit of a gap there there's often a bit of misinformation you know, it's not necessarily AWS are zero as of Google's responsibility to investigate anything for you at all, or even provide you the data. So you know, it is your responsibility to make sure you're capturing the right data, you have the right visibility and have the ability to investigate.

VAMOSI: So give me an idea of some of the incidents that you're investigating. Are these like commercial companies that are being attacked? Are these like nation state secrets, the whole range?

CAMPBELL: It's a bit of a range really, I think, the I'll come back to the commercial side, I guess the nation state side is getting a bit more of a light on it these days, obviously, particularly in light of the Ukraine war, etc. And, and, you know, Ukrainians have actually embraced the cloud which is great, you know, to move a lot of their on premise stuff into the cloud to make it easier to essentially secure and be more resilient, so to say, against particular attacks. That said, you know, because the industry is not, yeah, the industry is still maturing. On how the cloud works, right? And say, when they wait, one of the great things about cloud is that you can rapidly adopt technology and so this causes that shadow ID problem where you're spinning up new services, you have techies, you know, DevOps guys you've got your coders, programmers, etc. Like your r&d team. They're all spinning up services faster than you can secure them. And this causes problems so in light on one side, you're like, oh, Cloud is amazing. I can adopt new technologies really quickly. But on the flip side, it's like oh, crap, but security can't catch up. Say, you know, so now you have this problem where you got a bit of a wild west and really have systems and services that are spinning up and a lot. What happens there is that those people that are spinning up, unnecessarily securing them in the right way and so attackers, particularly kind of nation state side taking advantage of things like Miss configurations, you know, poor passwords, you know, open s3, buckets, etc, etc. Not understanding how you know, the firewall policies Working Cloud that can be very complicated on its own, and doesn't work necessarily the same as on premise and so these sorts of things cause, you know, basically glaring holes open in the cloud environments.

VAMOSI: So, on the one hand there’s the setting up and maintenance of the cloud environment for use. And there’s the other side, the side where the attacker is now starting to understand and use the cloud … to attack.

CAMPBELL: One of the things I know the US government's been worried about lately, and they've been a bit more vocal, which is great, is that they've seen nation states using Cloud resources to perform an attack. So basically, you know, using it as a command and control or less to perform attacks on other companies. Because when you're on AWS or Google you kind of blend in with the noise a lot because a lot of cloud services out there so much easier for them to blend the noise much harder for the US to respond to something that's in a legitimate cloud. environment as well. So they are taking advantage of that perspective, do we have visibility of everything they're doing and are definitely not and part of that is because people don't necessarily understand the cloud themselves. They're not actually looking. They're not actually detecting this activity. You know, it's still very mature in this space. It is growing rapidly and it's getting better. But you know, there are a lot of companies or customers out there as an example, don't even think you can do anything about a container as an example in AWS. So we had a customer once and they said, Oh, yeah, our security vendor said, You can't do anything about containers. So we just never did anything that I can data about when it comes to detecting anything suspicious, there's nothing it's like no actually can there's lots of stuff you can do. You know, it's just, it's just a little bit more complicated than, you know, what you probably traditionally you know, would have thought so. So people are just not looking necessarily.

VAMOSI: So I’ve often found that when there’s a jump in a report of new activity, part of me wonders how much of that is enhanced detection -- we now know what to look for -- vs an actual rise in activity.

CAMPBELL: I think CrowdStrike released a report recently, and they said there was like a two fold increase in cloud attacks, and I think 300% Increasing kind of, you know, hackers ability to understand cloud and utilize cloud and, and I, you know, I think partially that is yes, they're definitely learning more but the other thing as well as that we're just starting to detect more so, you know, that stuff could have already been there. And I guess the commercial space of that kind of space to reflect on that and how a beat the state space could be because, you know, if you could have a look at a group like Team TNT and which I mentioned before, you know, they just do math scans of IP ranges of AWS or Google of always year and they look for misconfigured services, and they've compromised 10s of 1000s. And in fact, they even had, I'm not sure if it's up right now but actually have an online tracker which is available to the public, that shows you how many systems are compromised, and there's literally 10s of 1000s of them.

VAMOSI: So now you have attackers who can spin up an instance and then spin it back down. Again, this complicates the forensics later if law enforcement, for example, wants to prove that a particular group launched the attack. And this isn’t hyperbole -- James has see it himself.

CAMPBELL:  And, you know, just a funny joke. What happened to us before we were doing a presentation for a sans conference, and so we're putting together material on compromising containers and how you can do that and how you can use that to laterally move across your cloud state to basically use that initial access to gain access to wider systems. And you know, we put up a honeypot basically so we put up our own system online, we made it purposely vulnerable for the purpose of the demonstration. And then the funny thing is within 30 minutes the team TNT guys, it's Ghana and compromised it within that time frame. And, and it was actually really, it was quite funny, but it was really annoying for us because we were trying to create a demo and they ended up actually compromising it and we're like Ah come on. But that just shows you how weak it happens in the cloud like it takes literally 3030 minutes if you've got a vulnerable open service like that you it's been scan I bet you someone has had a go at it. And I bet you the chances are it's probably been compromised. But most of the customers we've come across you know, just don't have the visibility they need to understand this even happened in the first place, which is that scary.

VAMOSI Is it possible to aggregate some of these glaring holes like you alluded to misconfigurations which I hear often is that perhaps one of the bigger buckets that we can throw these compromises into or are there other areas that are a problem?

CAMPBELL: Definitely misconfigurations is probably the biggest bucket, so to say. The other thing we've seen as well as you still get it's probably more reserved for like your nation state style kind of resource activities is, is you still get kind of you know, cloud engineers or engineers in general. being targeted by nation states stealing credentials and noodle utilizing those credentials to gain access to environments. And so, you know, there used to be on prem because we just went in cloud, but now those credentials include access to cloud, cloud data and cloud environments as well. So definitely a bit of, you know, more of the same answers via phishing emails, steal credentials, gain access to the data in the cloud. And so that kind of, you know, what you'd say the traditional approach of targeting is still occurring, but it's still leading to cloud compromised as well

[Music]

VAMOSI: A moment ago we alluded to the Ukrainian situation, and I think 10 years ago, Estonia was attacked and crippled, and this was not a cloud attack. This was a more traditional attack. Are governments putting their resources online?

CAMPBELL: Yeah, well, interestingly, Ukraine didn't use the cloud and actually had its public knowledge. They had legislation or law that said, you couldn't use Cloud assets. And, you know, I think that's all of the old Soviet era kind of mindset, so to say, of Western technologies and, and you know, but it's great, they've really embraced it now. And it's, you know, solving a lot of problems for them, which is fantastic. And you've seen, you know, a big shift, especially in the US government and moving to cloud as well as lots of, I think the UK Government as well as really started shifting to cloud to and so there's started being a big shift from government to cloud. It might not be the, you know, the super secret data source that, you know, they put it in the cloud, but, you know, from a general running of the government and operational capability, it makes a lot of sense for them to use the cloud and a couple of good bonuses added that come out of it. Is that once you, you know, if you do a good job of it, you know, you can make your cloud environment super secure, right? It's much easier to maintain like patching and have visibility of your assets as well. So actually understanding what you own in the first place is one of the big problems if on premise environments in the cloud, way easier, way easier to manage your assets. Which is fantastic. So you kind of understand where your data is a lot better and you understand how to protect it. You can also do kind of big global goals as well. So as an example, we were discussing misconfiguration and one of the one of the easy misconfiguration pitfalls people fall for is like, you know, your zero blob storage or your AWS s3 bucket with all your data and it's open to the internet, and someone found the link and say, like, something as simple as that is, you can actually set global controls to say do not allow open public, publicly accessible s3 buckets as an example. So you can set these sort of controls at a kind of global level on your kind of cloud accounts. And so you can actually make management of your estate a lot easier. But I guess the thing that we need to do to get there is is you know, is do that upskill piece, understand how the cloud works and how you're going to utilize the cloud. And then the other bit is just that resilience piece is you know, you can spin up resources. If you need more resources, you can change infrastructure at a click of a button. And you know, you're talking about the Estonia attack, you know, there's lots of denial service attacks and things like that, as part of that. You know, in this case, for cloud to take down the whole cloud environment, you will need a hell of a DDOS say, you know, so you'd be able to manage those a lot better. 

VAMOSI: IF that sounds like a stretch, James actually did that for a client. 

CAMPBELL:  I did a big denial of service with the government. once. I won't mention which department but we were kind of prepared for a potential denial service. And it was, you know, they had a physical data center in a physical data environment. And we actually had to set up two physical links to switch between in order to, you know, basically route traffic. That was important in a way you know, so it wasn't disrupted by a denial service. And say, you know, to do that, that required like weeks of effort is, you know, crazy. We have to be very well prepared, but when it comes to the cloud, you can make those changes or decisions or route traffic. Very quickly, very easily compared to on premises environments. So it brings a lot of resiliency to the picture as well, particularly for public facing government services.

VAMOSI:  So this brings into the picture the cloud providers themselves, such as the commercial choices of AWS, Google, and Microsoft, but also the private providers that maintain their own cloud infrastructures.

CAMPBELL: Again, guess the, one of the one of the interesting things we you know, one of the topics that come up particularly since the Ukrainian war as well, you know, I think goes about saying, you know, cyber is definitely solidified itself as one of the domains when it comes to you know, you know, an act of war. And so, you know, whether it be to gather intelligence, disrupt or degrade infrastructure and communications, etc. And I guess one of the things which is interesting, and I guess this goes back to a little bit to the shared responsibility model, and cloud is one of the topics is around cyber and during wartime you know, where does the responsibility lie for security is that with the government is that with the cloud providers is that with the, you know, the the actual industry body itself that are in the cloud, environmental the account you know, where does that respond

VAMOSI: So this raises the question where the responsibility lies in a you know a government that’s under attack.

CAMPBELL: Yeah, yeah, I guess. So. There's a bit of a conversation. So there's a conversation going on at the moment in industry and also in government where, you know, I think with the Ukraine war, it's really kind of shined a light on it in the sense of, you know, where does the responsibility lie when it comes to nation state based attacks, you know, and particularly if you have cloud infrastructure adds an extra level here. So, you know, is it the government's responsibility to make sure nation states are not attacking? Is it the industry's responsibility to make sure their own infrastructure is secure? Is it the cloud service providers responsibility or even a managed service providers responsibility to make sure that they're reporting you know, any kind of potential activity they're seeing either to customers and or the government as well? And so, there's this kind of three way responsibility model or most of you know, who is responsible or is it shared? And whether those lines are kind of where those lines are drawn? And so I think that's, there's a bit of a discussion that's going on at the moment. I don't necessarily have the answers, but it definitely is a shared model. You know, everybody has a part to play. And I think the one thing we're probably missing at the moment is a bit of guidance on or a bit of discussion, which leads to guidance, I should say, and leads to a framework maybe of, you know, what's the responsibility of the government was the responsibility of providers, so your service providers, cloud providers, SAS providers, etc. And you know, what's the responsibility of you as an organization, particularly when you're coming up against more advanced threats, such as nation state activity, right?

VAMOSI would this get into international treaties or discussion or because you're crossing borders at this point?

CAMPBELL: I know right. I guess it's probably why we haven't done it yet. It's so hard. It's such a hard thing to do and to get something agreed cross border level would be a hard thing to do, I guess, you know, in the first instance, you would probably need to start with like minded governments, or or even local, like your national level first, and then look at see how you can branch out I think if he tried to do it at a global level, we just end up spinning our wheels a little bit. But I think if we, you know, if we collaborated with all the key service providers or the key cloud providers, government and industry all together, then you know, at least for the kind of regulated industries then yeah, that's a good starting point and kind of go from there, but it's an interesting one, because there's a lot of shifting of blame, you know, like, it's, it's not my environment, or oh, it's state sponsored. So therefore, you're sophisticated. Let me tell you a lot of state sponsored attacks are not necessarily sophisticated. A lot of the criminal ones actually tend to be a bit more sophisticated. Not to say there aren't sophisticated ones, but more often than not, you know, why? Why use you know, your hottest, you know, technology to compromise someone if you don't need it, right. And so, you know, that would just be giving away your hand and they don't need to use the latest and greatest things. And so, yeah, so I think, you know, from an international standpoint, we're way off. I think so, but, you know, let's start nationally first, then there may be like-minded governments that can link up on that front. And really, it's about open dialogue between industry and the government. I think at the end of the day,

VAMOSI: Is there any government currently setting a good practice?

CAMPBELL: I think the US government is being very shouty, like in this kind of domain right now. And I think it's nothing that's great, I think is causing a lot of discussion. A lot of people were having a bit of food before. I mean, the government is, you know, right to say, you know, industry needs to do more. You know, they know, the US government can't necessarily solve all the problems. They can't just have a silver bullet or a magic button which you know, automatically defends everybody that's not realistic. And, you know, that's, that's, you know, bad security practice, just rely on that. So, you know, so I think, you know, I think it's great that they're putting a little bit of pressure on industry to improve things. You know, let me give you a practical example of my past right so UK, in the UK before GDPR came along. So that's the data protection regulation that was pushed out across Europe for those that are unaware of it. We had the Information Commissioner's Office in the UK and they were responsible for you know, for PII breaches, so that's breaches of personally identifiable information. And the maximum fine you could get for a company before GDPR came out was 500,000 pounds. Now, you know, when you've got a company making you know, billions write you know, a fine of half a million pounds is not necessarily enough. And in all honesty, I talked to one company and one said, " Well, okay, so you want us to invest in you know, 10 million let's just use a random figure but it was, you know, something like this 10 million pounds right into cybersecurity. But if we don't, and something does happen, I only get a fine of 500,000 pounds, like where's the investment trade off? Other than reputational risk and um, you know, obviously that's, that's one that's hard to quantify and to value. But, you know, now that those big fines are coming across the government has been involved, and you know, it's putting more onus on the industry to secure themselves. I've seen a huge spike in security posture across industries. It's still not there yet, but like, I've seen a huge improvement across security because that regulation came out and so, you know, I think these are, you know, with the US government kind of standing up and shouting about it a little bit more. I think we know we need to put the few things in writing. You know, as far as kind of regulation is concerned, or at least industries of concern or national significance. You know, that will help change the mindset of security because people need to see security as an investment not necessarily something that's just taking away from you know, the person to say so, we need to see it as an investment rather than just to spend.

VAMOSI in your recommendation is to start locally, within a country and then perhaps within the EU and branch out larger and larger.

CAMPBELL: Yeah, exactly. Yes. Yes. That's the reason why we bought too much. It's gonna get too complicated and take too many years. And then, you know, I think as part of that conversation, the big providers need to be involved like your Microsoft's, your Amazon's, your Googles, etc. The big players need to be involved in that conversation. And, you know, they have more visibility and honesty than most governments and you know, they have you know, they can make a big change themselves in getting involved in those conversations. So I think that would be good to see.

VAMOSI: And you also have the related issue of where the data physically lives. So having server farms in say, a questionable country as opposed to a you know, a friendly country

CAMPBELL:  yet, absolutely. So, you know, I think, I don't know how many regions there are in AWS off top of my head, but there's a lot and they also have their own instance for China as well. So for data regions in China because the Chinese government has some extra rules and regulations around, you know, how to handle data in China. And so yeah, it does add an extra level of complexity there. But you're right as well, because if you think of how many data regions are still managed by one company that is all you know, all accessed through the same portal, so to say, so, yeah, it's a tricky one, I think. I think it's great that you can quite easily set up different data regions to handle data appropriately. But yeah, you're right at the end of the day, it's, you have these big companies that do have access still to all those data regions in kind of a few clicks of a button, so to say so where it physically lay as be Yeah, it's, it's important, but it's, you know, you do have to be careful when it comes to cloud and making sure that you don't cross those boundaries or borders accidentally, because quite easy to do.

VAMOSI  What are some best practices?

CAMPBELL: Yeah, so really it's about understanding how you're going to use Cloud, what's in your cloud, and you know, where, where your gaps loans. So you need to think about the end to end when it comes to security and your cloud. So it's not just you've got your prevention, you've got your detection, you know, but then it's like okay, I've detected something. What do I do next, like how to actually investigate what happened, like, Is this something I should care about? How do I stand up in front of an auditor and put my hand on my heart and say, We did enough to say this was not a big deal, or we did enough to say this was a big deal, but we mitigated it. And how do you kind of put your hand on your heart and say that and so that's one of the bits that I think a lot of customers are missing is kind of they're doing that prevention too. So doing the detection piece, but haven't moved on to the investigation and the response phase, so and so there's a big kind of gap there. And I think part of that is just the you know, the maturity of the market and understanding how cloud works and the technologies there. It has been improving, but it's happening a lot slower than people are adopting cloud, so the gap or the divide, so to say, between your ability to actually understand what's happening in your cloud environment. is actually getting bigger, not smaller. So that's something that I asked people to think about is, you know, how do I do an end to end thing it's not just about prevention detection, something will get through I guarantee you that like handling or buy a beer if it doesn't, but something will get through eventually and you know, you need to know how do you how do you investigate that and make sure you're responding and accordingly based off what's happened. Great. Well, thank you for your time, James. I appreciate it. I know you're just starting your day in Australia. So the jetlag is definitely set in. I can tell you that but it's a beautiful day here. So I'm looking forward to getting out there in the sun.

VAMOSI: I’d like to thank James Campbell for coming on The Hacker Mind to talk about his experience in incident response, and how that field is changing as more and more organizations move to the cloud.