The Hacker Mind Podcast: Why Are Blue Team Hackers More L33T?
So you’re in your SOC, your security operations center. You spend your time defending all aspects of the organization, then one day this hacker comes in and sees that blindspot, the one you can’t see, that one corner of the network that is exposed, that is vulnerable. For all the time and money spent, you’re still pwned. That’s not good. In your role, you have to see that everything is secure, 24/7, and think of every crazy attack vector, but for a hacker, they only have to find that one fault--and then they’re inside.
Welcome to The Hacker Mind, an original podcast from ForAllSecure. It's about challenging our expectations about the people who hack for a living. I’m Robert Vamosi, and in this episode I'm defining red teams, blue teams, and even purple teams. How hiring professional hackers has become a business necessity given the ever-changing threat landscape.
Listen to EP 05 on:
Manka: I spent about seven-ish years working at the National Security Agency. There's a wide variety of places that people can go within that agency. One of them is the NSA red team.
Vamosi: This is Scott Manka. Today he’s a federal systems engineer, but, as you’ll soon hear, he spent time working for the U.S. government -- hacking the U.S. Government - as part of a red or blue team experience.
Manka: Really, one of the biggest advantages and use cases I guess for having a red team is just getting that adversary point of view whenever it comes into breaking into networks or testing your network defenses. There's always people who like to say white hat, black hat, and gray hat types of hackers. Really the the red team is in an adversarial mindset. They're in the mindset of attacking and finding any way into the network necessary.
Vamosi: Would you hire a hacker? No, seriously, would you bring a hacker on staff, even for a short amount of time? It’s actually a very good idea. Not only do you get a trained and certified security professional, you also get someone who doesn’t necessarily see the world the way you do, which is how hackers see the world--different from you. So with red teams, you have the attackers on the one side. That means you are also going to need the defenders on the other side. That would be the blue team.
Manka: Yeah, so a blue team is a team of security folks who are really focused on and specialized in just maintaining the security integrity, and the confidentiality of their networks that they're in charge of protecting. So it really is a defensive mindset for the blue team, compared to the offensive mindset of the red team. So, they're pretty much to the exact opposite of each other, really.
Vamosi: The term red team is military in origin. Part of Cold War wargaming exercises, the red teams are always viewed as offensive, or the attackers -- in this case it could be reference to the red Soviet Union. And the blue teams are always viewed as the defenders -- in this case, the true blue U.S.A. In a kinetic war, the two groups don’t necessarily tip their hands, and the same is true in a virtual war--even if both teams are employed by the same organization.
Manka: Oftentimes communication between the red team and blue team is not fully bi-directional, showing their full hand, and how they're able to do this, that, or the other. That's by design. You don't want that whenever it comes to the actual defending actually defending your network. You're not going to know everything that your adversary is using. And so, whenever these red teams have highly specialized folks, they have their certain tactics, techniques, and procedures. They're following certain tools that they keep close to the chest and then they'll give a little bit of detail about how they actually gain access or actually compromise a few systems here or there, and then usually they'll write up a report and then share that with the blue team. Then they'll have a pretty fun back and forth between the red teams and the blue teams, because you know you do have those folks who are a very happy to show that they're able to successfully either break in or give the red team a hard time getting gaining access. So it's a it's a pretty interesting back and forth between the two teams, but yeah that's that's been my experience from a from a higher level.
Vamosi: So we got red teams, and blue teams. Two sides of a coin. Lately there’s been talk about a purple team. What the heck is that?
Manka: Yeah, so a purple team is just a mixture of the two. So, let's say, a lot of times that I've really ran into some purple teams type of things, would be smaller deployments of folks focused on a specific objective, or let's see have some teams either traveling to different bases and they don't have the resources to have a fully dedicated red team and a fully dedicated blue team that is completely knowledgeable and is really in the weeds in the in those environments. I should say in those specific environments not that they're not familiar with securing or attacking, but, in my experience, I've just seen them where it's more of like an ad hoc type, maybe a traveling purple team, if you will. Obviously that's not always the case but purple teams are just a mixture of folks who represent both red and blue it's a single group of people who really do both of the offensive and defensive testing.
Vamosi: So it sounds like you don’t need to hire red, blue and purple teams full time. You can contract with them. And it’s not just the government, the commercial space needs them as well.
Manka: Yeah, yeah absolutely. There's a pretty big market to for either folks, or organizations, or companies that don't have their own team of folks that are dedicated to red and blue. Sometimes they might have a budget for something like a purple team where it's a smaller group of people doing both the attack and defense, but oftentimes there's these third party companies that can come in so a sort of like a how you'd hire like a pen test or pen testing company or a pen testing individual. They have a situation where you can like hire purple teams to really just hop in and try to do posts to get both sides of that visibility into the security of your network.
Vamosi: A penetration or pen test is when you hire a hacker or hackers to purposefully look for vulnerabilities in your network, system, or product. These are usually point in time engagements, specific to an area of concern. With red, blue, and purple teams, whether they are full time or contract, these are more engaged campaigns that can be either point in time or more often they are simply continuous.
Manka: At the very least, from my experience, the red teams and the blue teams were constant; you know, it was a full time thing. There would be a few weeks spent doing some research on any new happenings or whatever. The blue team pretty much always has their job. They have their hands full anyway, and it seemed like the red team would spend several weeks gathering some information together and then once they get their full idea of maybe their next quote unquote attack, there will be a period of a week or so where it would just be a constant barrage of them, attacking. And then the blue team would be there to defend and protect. But, you know, whenever you're doing something like a pen test of course, if you're contracting that out to a third party, that isn't always going to be a constant thing. That could be said for the purple teams as well. It really would depend whether there is a huge benefit in having the continuous testing along those lines because you know the threat landscape is changing -- every single day there's always something new. That is going to allow somebody else access into your network and if you're not constantly keeping your defense folks on their toes, then it's it could lead to problems in the future.
Continuous Testing at the Speed of Development.
Find out how ForAllSecure can bring advanced fuzz testing into your development pipelines.
Request Demo Learn More
Vamosi: Think about it. A job where you either attack your own organization all day long, or you are assigned to defend your organization from the red team. How cool is that? So I would assume at a minimum you’d need to know baseline security skills, but do you grow up wanting to someday be on a red team? Or a blue team?
Manka: So, whenever it came to the military it, they generally picked the folks for the red teams in the blue teams from the same pool of professionals that they had available to them. Whenever you're on the outside of the military, you'll have to have your basic credentials just to get into the cybersecurity field. Really, a degree in a relevant and relevant interest. Maybe a few certifications and then depending on where you want to go once you have those that foundation built of a degree and then, and several certifications that go after that really branch out in those two different directions. From there, you have a specific set of certifications that are focused on the attack or offensive aspect of things and then, as well as the defensive way defensive side of things as well. So, I wouldn't say diving down one of those branches will completely exclude you from the other, but it will really make things easier to find a position whenever you have those relevant credentials associated with those two aspects of things.
Vamosi: Given these terms stem from the military, is there a hierarchy? I mean, from the sound of it, I say maybe Red Team is on top, the elite hackers would go there. But that might not be the case. A blue team needs to be really skilled to fend off these attacks. I mean really skilled.
Manka: I really did think that was the case and, and I initially went into more of a defensive blue team position. And then for several years about four years which was focused on defense and focused on after the fact. For blue teams in your defense to succeed, you're going to need some serious skills because let's say that is the mindset of everybody where they're saying, okay, if you are really apt to learning new things and you're extremely driven and only focus on red team that's going to really, really hurt things on the defense aspect because you're going to need folks with a lot of resources --some really strong backgrounds -- when it comes to defending because if you have folks who are really passionate about things only pursuing the attack and the offensive portion, that's really going to throw things off balance. That's where you're going to see your red teams are going to constantly be handing these reports down to the blue teams and then the blue teams will just never seem to be able to catch up is what I've seen. Now, you'll get like a few folks who are just extremely bright and extremely driven to really focus on defense and that's really where I would say it's like one of those things where like the best offense is a great defense, you know,
Vamosi: So it sounds like blue teams are the more elite--and why not? They are defending the crown jewels, 24/7. They have to think of everything. The red team only has to be right once. On the other hand, red team is what we see in television and in film as the representation of hacking. The black hoodie. It’s not. Really. Hacking.
Manka: Red Team has the easier job in terms of attracting new folks because that seems to be the most movie-like position, but on the blue team that's really where the challenge is, defending against these unknown unknowns. The blue team is really where you're going to have the ability to think outside of the box and have some really cool ideas and if you are someone extremely bright and have these different perspectives and different ways to attack certain problems, blue team is definitely where it's going to be the most beneficial. Red teams, at the end of the day, will refer to themselves as glorified script kiddies at times because they'll be stepping through certain things, and if a certain vulnerability isn't present there's always, there's always another one. There's always something that's easily easily able to be copied and pasted and and transferred over there, but, at the same time, that's not to say like one is better than the other. They're both exactly just important, but I would say that for blue team that is where critical thinkers and you know the thinking outside of the box is really going to be the most beneficial.
Vamosi: So, let’s be cynical. This sounds like it is all just fun and games. I hire two teams of hackers to attack and defend my operations all day. Why on earth would I pay somebody to do this? I mean, if I’m a commercial company, how do I justify this to my investors, my shareholders, or with the government, the taxpayers?
Manka: You're always going to need the network defense, and some form of network defense. You're gonna always need somebody there to defend your networks. Now, if, if you're in a vacuum, and let's say you're in charge of keeping your network secure, you're just going to be performing your regular scans -- whatever is mandated by your organization or the different security controls that you have to follow. You'll just be pretty much following along those lines. Whenever you bring in this sort of adversarial aspect, where not only are you going to be attacked in a very similar way if not identically in a real world scenario, but, after the fact, you'll also get to have this collaboration and discussion between those two teams of what went wrong. How did we get in? Where did this chain break? How is everything blown wide open? Now you get that additional insight so it brings a lot more context into just, 'Oh, hey, we were breached.' Oftentimes you don't have a whole lot of information to go on, whereas whenever you have that red team blue team collaboration, you're really going to be able to get a lot more detail into how that breach occurred. And on top of that, if you're working defense, you know that you're just constantly going to be attacked in some certain way, it's going to force you to think a little, be a little more forward thinking in terms of defense and in terms of what should be done instead of what has to be done. So it really does change the mentality of the folks performing the security and it's it's extremely beneficial and in both of those aspects at the very least.
Vamosi: So beyond the theoretical, what does it look like to be on a red team, or blue team?
Manka: For several weeks, we were getting -- I don't want to say -- owned by the red team, but I don't want to give them too much credit. The red team definitely was not making things easy on us in terms of they found that this vector and they got these credentials on your domain controller. you know, it just seemed like you know one thing after another after another. Most of our team was relatively new. You know, fresh out of boot camp, fresh out of training, and then it has to happen during you know one of the overnight shifts, so two or three in the morning. All this stuff is happening, but one of the more interesting things was after a few weeks of just having our butts handed to us by the red team, folks just started to find that competitive nature. People just started to come out and say we want to just figure something out. At the very least, just hold our ground, and then maybe work from there. And we were just starting to do some research of our own on what's being used, and what sort of things are working for attackers, what's really trending. And this was around the time of a tool called the Low Orbit Ion Cannon.
Vamosi: Low Orbit Ion Cannon is an open source tool perhaps first made famous by Anonymous, a loosely organized group that went after political targets by sometimes causing distributed denial of service attacks. That’s what Low Orbit Ion Cannon does, it floods a site with bogus traffic, preventing legitimate requests from people trying to access the site. Using Low Orbit Ion Cannon for stress testing your own server--that’s perfectly legal. Turning it on another server, not owned by you. That’s not legal. So in the organization where Scott was, the red team might someday use this, so the blue team had to anticipate that.
Manka: It was starting to be talked about a bit more and we were assuming that you know the red team was going to maybe do something like this just to just have one more vector or one more thing to sort of you know rub in our face I guess if you will. And so what we did was download a few different copies that we were able to get, and we spent a lot of time writing these intrusion detection signatures, and trying to write some sort of way to uniquely identify traffic that was coming from this tool or blocking or that would block any downloads either saw somebody downloading it or using it, or using it for some sort of denial service. Then we spent a lot of time writing some signatures to mitigate those two different aspects of things. And it really did help us out because that was one of the avenues that they had ended up trying to take. It's not the most you know like groundbreaking story but it really does reinforce what I was saying earlier about being constantly attacked by by your colleagues is really going to make you think a little more proactively, or be a little more proactive in defense versus just reactive because the reactive approach is just impossible. Okay, so this is how we need to be handling it and what we need to do. We need to be keeping an eye on what's going on, instead of just focusing on the bare minimum, and only looking at ourselves in a vacuum, believing that we're totally safe. That was just a more of an eye opening moment for myself and the and the team that I was on in terms of just going out there and looking for ways that we will be attacked and figuring out how to stop that
Vamosi: So if the red team is using Low Orbit Ion Cannon, they must be using other tools. Like Kali Linux, it has perhaps the most tools already baked in.
Manka: So, for some someone on the red team, yeah, Kali Linux is pretty much the Go To. That's what most folks I know use. Once you start getting into that offensive security type of world, people love just getting into either smaller subsets of a subset, or of an already small subset. It's pretty much Linux. The overlying OS doesn't really matter, or the different branches I guess that they have, but if you download Kali Linux, it's going to have everything that you're going to need to get your hands dirty and get started. And once you actually start knowing your own workflow, knowing the tools you prefer to use, and knowing what works, then really you can choose whatever, whatever type flavor Linux is, is your favorite. I would recommend you know Linux over the other main OSes if possible just because most of the offensive security red team pen testing world is Linux. It seems like the outliers of the smaller community, or smaller portions of that community, would use something like Windows or or Mac. It would just be the easiest to get things going in terms of that because you'll have your packet sniffers, you'll have Metasploit baked into the Kali Linux image already ready to go with so many different exploits that you're able to use right off the bat. Whenever you're getting into the defensive side of things. Some of the tools are going to transfer over as well, like using Wireshark, having some sort of intrusion detection system. You'll maybe use Snort and be able to write signatures to highlight any suspicious traffic and whenever a signature fires off. It's going to collect a minute or two of that traffic that's associated with it, so you can open that up in Wireshark and get an idea of what was actually going on. And, really, it does seem a bit unfair whenever I think about it because the blue team has so has less to work off of whenever they're trying to protect versus the red team where everybody loves writing scripts to destroy things and Metaploit is out there for free and it really takes things to the next level there.
Vamosi: Okay, so being a professional hacker, working on a red or a blue or a purple team, sounds cool. What skills must I have or what CERTS do I need?
Manka: A baseline would be something like security plus and that plus would go over there. I'm the I'm hesitant to endorse something like a Certified Ethical Hacker or there's also like. I think it's like computer hacking forensic investigator because I feel like. Anytime that I was in position, that would be considered or construed as I'm a hacker, if somebody had that if somebody, you know, had those certifications you'll realize very fast that those don't apply at all, and Certified Ethical Hacker and CHFI the hacking forensic investigator those seem to be more of a very high level view of things that don't really translate well into getting your hands dirty. At least, at the very least that was my experience,
Vamosi: For the record, Scott has both CEH and CHFI certifications.
Manka: Those first two, the security plus and net plus, those are very basic those should be just a given to give in to get there's also an whenever you start getting a bit more into, let's say red team there's an offensive security certified professional. So that is a very well respected cert to work toward. It is a very difficult one to get, as well as the, the testing for it is no joke. If you have OSCP and you talk to anybody who's in the pen testing world, that's kind of like, okay, you know what you're doing. You can't just, put a banana peel into your heel and slip into getting that cert. It's not a joke at all. But, I would also recommend a lot of the ones by Offensive Security, actually pretty much all have. All their certifications are pretty specialized. They are not just handed out at all. They are difficult to get, but they definitely hold their weight. So, I would look into those as well.
Vamosi: Given that Scott’s lives in this world, what advice does he have?
Manka: I would say for for anybody who are interested in those two types of or potentially three types of paths red blue or purple. I would, I would really encourage a second look at the defensive aspect of things. Just because that is if defensive network defense is going to be a very lucrative career path as well because not everybody is hiring for a red team not everybody's hiring for a pen tester, you know, 365 days of the year. That is more of a pen testing thing, at the very least, an ad hoc or contracting thing, where they'll spend a month doing one thing and then hop from each different thing. But if you are somebody who's loves thinking outside of the box, and who can really bring their different perspective, and is a very forward thinking individual, being on a blue team or something similar to a blue team is extremely helpful. They definitely need way more folks who can think like that on the blue team side of things. The job is going to be more challenging and a bit more rewarding because it's not just a simple, you know, line and Metasploit and then you hit enter, you set your hosts, and then you automatically get in. Blue team is definitely the more challenging side of things and it's, it's also potentially more lucrative.
Vamosi: So there you go. Rather than be caught unawares, it’s better to hack yourself. It’s better to have smart people on staff who can be creative and think outside the box on your team, whether they be red, blue or purple. So go hack yourself. You’ll be glad you did. For the Hacker Mind, I remain the proudly purple Robert Vamosi
Exploring The Best SBOM/SCA Tools: Docker Scout
Add Mayhem to Your DevSecOps for Free.
Get a full-featured 30 day free trial.
