The Hacker Mind Podcast: The Hacker Revolution Will Be Televised

What if DEF CON CTFs were televised? What if you could see their screens and have interviews with the players in the moment? You can.

Jordan Wiens, from Vector 35, maker of Binary Ninja, is no stranger to CTFs. He’s played in ten final DEF CON CTFs, was a part of DARPA’s Cyber Grand Challenge, and recently he’s moderated the live broadcast of the annual Hack-A-Sat competition. So if anyone can pull off turning CTFs into an eSport, it’s probably Jordan.

The Hacker Mind is available on all podcast platforms.

[Heads Up: This transcription was autogenerated, so there may be errors.]

‍

VAMOSI: I am not a gamer. Last year at Hacker Summer Camp I was invited to Mikko Hypponen’s book launch for If it’s Smart, It’s vulnerable. His publisher held it in an arcade in Las Vegas where competitors sat in a main area, their screens displayed to an audience seated in an arena around the players. It was an interesting idea. That people would pay to see the top gamers competition.

And, of course, given it was hacker summer camp, where the world’s best hackers converge on Las Vegas for a week and half of Dianna Initiative, BSides LAs Vegas, Black Hat and, of course, DEF CON, it seemed only logical that a capture of the flag competition might be held there or someplace like it

Something like that happened, back in 2016, when DARPA commissioned its Cyber Grand Challenge during Hacker summer camp that year. They invited the top cyber reasoning systems, machines that could think like a hacker, to Las Vegas for the finals. Machines, not humans, playing capture the flag. And DARPA made the event interesting. They actually televised it.  

[[CLIP from the show]]

So the question is, What if all DEF CON CTFs were like this? Televised. What if you could see their screens and have interviews with the players in the moment?  In a moment I’ll introduce you to the person who has done this, and given his experience with live broadcasts of CTFs, is working now to make live CTFs a regular e-sport.

[MUSIC]

Welcome to the Hacker Mind, an original podcast from ForAllSecure. It’s about challenging our expectations about people who hack for a living.

I’m your host Robert Vamosi and in this episode I’m talking about gamifying something that is already a game, and packaging it for people who want to view it as an e-port.

[MUSIC]

VAMOSI: First, a quick disclaimer. The Hacker Mind is sponsored by ForAllSecure, which did in fact win the only Cyber Grand Challenge with its computer reasoning system called Mayhem. I mention this because my guest on this episode was also a part of CGC.

WIENS: My name is Jordan Wiens. And I am either founder of Vector 35 or CTF has been or hacker depending on how you want to how do you want to cut slice it

VAMOSI: Jordan has both played and created numerous capture the flag competitions. And Jordan now has his own company.  Vector 35, was born out of a history of capture the flag competitions.

WIENS: Yeah, so So Vector 35 grew out of a number of folks that were playing CTFs that were doing vulnerability research doing reverse engineering for government contracting purposes and then thought like, you know what, it'd be nice to see sunshine, have a window at her office, get outside, do more Hilton commercial. We thought, hey, this reverse engineering market hasn't really changed forever. I think we can. We can do something good here and so we formed a 35 to make binary ninja over eight years ago now. So it was eight years ago, January and still going strong.

VAMOSI: Often there’s an inside joke behind a name. I couldn’t think of one.

WIENS: You wouldn't have a reason. The three co-founders were all 35 years old. When we founded the company eight years ago. So in fact, I'm turning 43 tomorrow, so we should call it you know, vector 43 This year, but we Yeah, we're and then vector was was really just kind of like it has connotations of you know, like the, the minions show that there's the villain called vector direction and magnitude and so that we kind of liked that vibe. But also like vector graphics, like video games, there's, you know that so  we were doing Capture the Flag events and building hackable video games. As for fun like we've to this day never made really money on that. I think we did one training for Google once a long time ago, where we did a paid training using whatever hackable video games but really, we just have done a bunch of CTFs where we, you know, release these hackable things. The opponent venture was a series of games. And so it's kind of part of our culture, even if it wasn't part of the business plan, necessarily, of the company and so vectors have several of those connotations as well as like, attack vector, because we kind of had an offensive work. So we sort of like the backronym some of that stuff didn't do it. Just cut but it was a nice, nice word and was unique enough that we could, you know, get the domain name.

VAMOSI: One of the skills necessary for a successful capture the flag experience is being able to look inside an unknown binary. So, in this episode, we’re going to talk a little bit about reverse engineering.

WIENS: Yeah, so people when they asked me what I do, I say I build software that takes apart other pieces of software. So the idea is that for a variety of purposes, a company might want to understand is this piece of software, a virus or someone else might want to say, is this piece of software that I'm going to run on my computer? Does it have vulnerabilities? Are the things about the software that I need to know? And when there's different types of software, we don't work with all software. If I were to differentiate between compiled and interpreted and web tech versus you know, most like binary native stuff? So I don't usually split that out and just say like the apps running on your phone, for example, are all compiled and built specifically for your phone and your CPU on that phone and to turn it back into something that humans can understand. That's essentially where software comes along.

VAMOSI: So this is important. When we talk about executables, not all binaries are easy to reverse engineer.

WIENS: currently, but for example, Python is not compiled to binary. They have like Python bytecode, which isn't real .Net Java. These other like virtual machines, have separate analysis that you would do for them and there's actually dedicated tools for each of those, right? There's dedicated Python decompilers, dedicated .Net decompilers, dedicated Java decompilers. And we intentionally focused our design on just native Yeah, so just binaries. Again, if I'm explained to a layperson, I'll say like if it runs on your phone, it's probably a native application, but for example, when you go to a website, there's like code in your browser and the JavaScript that makes web apps work. That's not something that we, you know, would be analyzing to kind of differentiate

VAMOSI:  You could do reverse engineering manually. Why would you. There are a few good tools on the market today. I’ve used IDA PRO in Black Hat trainings. It’s the gold standard, but also really expensive. Then there’s the open source free tool from the NSA, Ghidra. Jordan’s product, Binary Ninja, fits nicely in the middle.

WIENS: So it is a mixture and so one of our sort of key things that we did differently from the beginning was we said, look, we're the IDA Pro is like the main commercial tools everybody uses. There's a couple of open source things now we have Deidre which has come out the NSA is an open source tool. We actually used it before, not extensively, but like we had access as government contractors are sort of past life. And so we were familiar with it, at least that it existed. But the thing that we saw that we wanted to focus on was we built an API first. So we built a system for automation. And then we just built a UI on top of that. Whereas like with IDA, for example, they didn't even have a Python API. It was a third party that wrote it, and kind of shipped it in. They have a separate public and private API split. They have sort of like this, what the public sees, and they have what they can use internally. They're not the same thing. We designed our interface to be just a full public API. Same thing that we use to build our UI, any other plugin anybody else can build on top of can do whatever they want. And like I, we care a lot about UI and UX design, and we think we're the only one that does it to the degree that we do certainly because the other tools are clearly like power tool geek tools that were on forever. It just got a ton of features and buttons and knobs and whistles and they're intimidating. Words, we actively fight against the trend. Even though we've been around eight years. We try not to just put everything in a button or menu and try to make it like any anyway. But so, you know, we care about the user experience. We care about it being my manual, but we also have a heavy focus on automation and making sure that we are the best tool for automation, we're still not actually the best tool for some types of manual analysis. There's some features in both Ida and ghidra. It still has been great decompilation, greater still has better project based support for things like collections of binaries or changes over time. There's things that they do better there. But our API is like, universally accepted to be really, really good. And it's kind of a strength, I think and so we're working on wanting to be the best at everything. But that's sort of how we have focused on it sorry, it was a long answer.

VAMOSI:  When I interviewed for my current job, I was asked about Binary Ninja, whether I was comfortable using it. It shows up in the industry. Binary Ninja is lucky in that it serves an interesting niche in the market. It’s neither free like Ghidra, nor is it expensive like IDA Pro.

WIENS: yeah, and obviously, you know, you live with their price is the big one, we knew we had to come in at a cheaper price point. I'm very jealous at the margins that either has and what they can charge as the sort of dominant commercial player. But you know, I think that ghidra has been really interesting because you must be this tall right now. Right? Like no, if you didn't come out when we were like within the first year or two of us starting the company we would have actually had to go out and we would pivoted or something completely different. We could not have competed. We were able to survive. Because we had five, six years of development in the pipeline to be mature enough to have a good enough product but if we had to spend 30 developer Dec years, building a tool to compete to get something it's free, like before we could even be close to comparable like that just wouldn't have scaled, it wouldn't have worked. Even with our focus kind of an API some other things. I think we'd have had a much harder time because one of the ways we're gonna succeed early on is we had really great traction and students and hobbyist markets because we intentionally priced it, had much better discounts and had a non commercial version versus a commercial version. With a very big price difference. And so we had this two prong approach for experts. We had a better API and some program analysis and some modern techniques that like the very high end of people really appreciate it. And so they would, even if they were using it still to do their Manual or re engineering it will start writing a lot of their scripts and automation on binary ninja, and then on the low end it was because we were affordable and accessible and a cleaner interface and those kind of things. Gator would have entirely destroyed that market for us because we wouldn't have been the most affordable in the town. So we're very lucky that we had the run that we did before they released it. The good news though, is that we've never had a competitor because it's just it's gonna be impossible to make money in this space with something that featureful for free, so it's an interesting side effect of of that hidden market.

[MUSIC}

VAMOSI: There are two types of Capture the flag competitions. Perhaps the most common is Jeopardy style, where you have a board with categories and the questions get progressively harder in each. Then there’s the Attack and Defend or King of the Hill, that’s what you see at DEF CON, where you are not only attacking someone else’s server, but defending your own sever from their attacks. Each have their pros and cons. Both require some reverse engineering knowledge. And reverse engineering is particularly helpful when you don't know what's being thrown at you in a King of the Hill competition. You have to deconstruct it and then respond in the moment. Reverse engineering is also important in Jeopardy.

WIENS:  Yeah, there's multiple categories that will come up and right there. So there's even the poem category, which is the opponent bowler exploitation or hangnail, it's sort of pure hacking kind of side, even that really is heavily reverse engineering because you don't just get told exactly what the flaw is. You have to go find the flaw and then you write the exploit. There are often separate categories and separate challenges that are pure reverse engineering pure Ari, where you the whole goal is there is a flag there is something you're going to go steal you're gonna have you're gonna figure out and solve in a binary and just once you've got it you submitted you're done. And that those are kind of distinct, usually from from opponent over your neck attack, you're going to analyze it, figure out it and then attack the server. But yeah, it's a foundational skill for multiple things in Capture the Flag and that's, that is exactly how I got my start. So I started in my career as a network. Network defense guy, I was working at university and had a server get hacked into we're gonna blow report and it was super cool and then ended up getting hired as a security engineer at the university as a student, because I was just like, oh, this is cool. We got hacked to like it was really interesting stuff but had no no excuse or experience. And so started doing network defense, and then sort of playing Capture the Flag was turned into like, oh, this hacking stuff was pretty fun, too. And so that's where I spent, you know, the next seven, eight years of my life doing offensive stuff. And now it's funny because as a building a reverse engineering tool, both offense and defense are going to use it equally, right? Whether it's finding a bug, whether it's for analyzing malware, it's just more of a like a compiler, it dooleys technology. It has a lot of applications. And so I think that kind of balances has been helpful but yeah, it was CTF that took me from network and defense into like this offensive and reverse engineering kind of world for me personally anyway

VAMOSI: Jordan is modest. He’s played in DEF CON’s prestigious final round a number of times. And he’s won. Meaning he has a black badge that will get him into DEF CON for free forever.

WIENS: I joked that I've been. I've played in 10 finals, I think I counted at DEF CON. In the past. I've been around for quite a long time. I'm actually helping now. I'm not officially a member of Nautilus uttering it this year, but I'm this group called Live CTF we've kind of put together and we're sort of like an ancillary to To the Nautilus Institute which is the team that's during the main event. And we're focused really just on broadcast and production and making visible and being able to talk about and show because people have been playing DEF CON for, you know, almost 30 years now. And it's really hard to get a sense of what they're doing and how good they are. And you hear the stories and so I really am passionate about trying to make it digestible, trying  to make it understandable, trying to to show how impressive it is that some of these people that sort of top of their game are. And so that's that's kind of what life CTF is doing but I've been on Team men and black hats team last place team beta Gods we won DEF CON three times I won twice with last place and I won with confetti gods are Team Awesome I forget which name it was that year. And then yeah, played platinum brother years but like I said, it was a long time ago. Back when it was it was easier

VAMOSI: So with DEF CON, anything goes, and that extends to the CTFs. Jordan has a talk where he enumerates some of the best challenges. The way it works is that an organization designs the challenges each year. Organizations like Legit BS or Oder Of The Overflow. Currently it’s the Nautilus Institute, composed of several past winners. Anyway, one year there was this weird hybrid of hardware and software challenge. IT was called Badger.

WEINS: It was that was that's one of the you know, I sort of have a list of like, the most impressive challenges people have ever gotten. And that's that's high up there. Just in terms of terms of what went into it. So I mean, nowadays, badge life is a thing at DEF CON, right like in the last five years. Everybody's building their own badges. There's the you know, these little ESP chips that have like, all in one Wi Fi and a little Linux or a little you know that OS that's just trivial and you download the firmware, you tweak a few things and you've got blinky lights, the magic can talk to other things and like do all sorts of cool stuff. It is really exciting. It's fun, but this was probably 10 years ago when legit BS was the organizers. So if the fair the exact year earlier year, so yeah, it's been at least 10 years I think. And I remember his handle on just adjacent was that was the developer. I don't think he'll get mad at me for for outing his first name. But the guy who built it, designed his own circuit board, but like, designed like an FPGA on it that you know, is a circuit that you can kind of reprogram right like an FPGA is the ability it's sort of like you can you can reprogram it to be a different piece of hardware. So it's kind of in between the sort of software and hardware hardware if you listeners aren't familiar with it. And so it's really like geeky and really interesting and he built these into these badges and then built his own breds spectrum, wireless protocol mesh network where like because DEF CON has always been his hostile like Wi Fi in fact, actually it's only been the recent years that like you could just use Wi Fi at DEF CON to be reasonably good. You get a secure link and actually is generally fine. Like for a long time. You just turn off every wireless interface on your device whatsoever because it was going to be atrocious. Excuse me it's been recently that I feel like it's actually kind of recent, like I don't turn my phone on airplane mode at DEF CON anymore and have been for several years. And partly just because like no one's gonna blow an iOS o day on the DEF CON floor because you're gonna lose it because people are going to capture it and steal it from you and do whatever they want with it. And to it's just it's just too noisy to open too much people can look around but anyway, so this badge was just incredibly advanced on both the wireless and the communication network and had its own little custom like messaging thing. You could do little icons and you could use a little push buttons to just use it like as a wireless chat badge with the other badges. And it was this custom like message passing wireless mesh network that would establish just incredibly complex amount of engineering. I mean even doing the circuit like they actually did the the boards themselves. This was you know, to get it done right they actually were like baking in the oven. The circuit boards to like get the get the chips in place instead of using like a service just kind of did it for you which nowadays prices have come down so much that it's almost always the better option. So it's just yeah, it's super neat. How much work in engineering goes into these like hobby projects. It's essentially going from kind of like a government contractor and commercial and a lot of people when we started our company actually we really thought about do we want to sell a reverse engineering tool or do we want to like do catch the flag as a service or as like a paid like that maybe maybe that's the business there and a lot of a lot of government a lot of companies are like love the idea of capsules like like this would be great for training and for you know, this is this is really cool. And then you tell them how much it costs and they're like, oh, nevermind. It's it's incredibly difficult. It takes so much effort until like for all secure hadn't been for a while that they were trying to like auto generate right and to be able to make new challenges without the labor cost of a manual bespoke custom challenges to try to kind of thread that needle. But there's not been a lot of companies that have really even tried, because like the like the labor of love that is DEFCON CTF, and the organizers the amount of work they put into it. He's just It's millions of dollars like at like any use any standard billable rate. If you were to pay these experts to do this stuff. It's just incredibly expensive. But when they do it for fun, they'll spend six months of their life pouring into something fascinating and then release it for the friends to hack on so it's cool

VAMOSI: So in Episode 4 of the Hacker Mind I spoke with Eyre, a member of the Plaid Parliament of Pawning or PPP, the most celebrated team in DEF CON history. In that episode she told me about one of her experiences. The challenge was called Clemancy. Eyre told me about showing up the day before the competition. So all of the tools, everything that they had worked on in preparation -- they had to start from scratch, and do so the night before the competition.

WIENS: I told the organizers not to so I was friends with the organizers. We knew what they were doing. And they actually even asked us to like do you want to build support and binary ninja for this? I was like heck no, that's way too hard. I like real architecture and real customers. That's gonna take me too much time to do it right. I can't be distracted from trying to launch a commercial company here. I think you're super mean and no one's gonna do well. It's gonna be a huge flop. I'm very glad I was wrong. It was not and I think some teams struggled but a lot of teams prevailed and made it work and it's the essence of hacking is they figured it out. But yet clemency was designed to be enlightening. The author who made that design is me. Like she is notorious for like, making stuff super hard. And it was just weird. Like, you know, it's not Little Endian or Big Endian it's middle Endian and it was three bytes instead of any reasonable power of two, like it was just, just gross. Just really designed to be awkward and awful. And, you know, teams thrived. It's like, you know, poetry, the best poets love weird constraints, right? And because it forces them to be more creative. And so I think to some degree, you see the same thing in hacking, where sometimes it's the weirder, random, the more random something is, the more people will thrive and enjoy it to a point there is there's a breaking point where you know, you can burn some people out but that was the was the success.

VAMOSI: Jordan knows something about the burn out. He’s not played CTFs on that rock star level for many years. And there’s a reason it’s so exhausting, and why if you step away from it, it’s not so easy to get back in and play again at that high level.

I joke about being a has been but it's not entirely wrong. Like I don't keep up with the day to day skills to be topped here at CTF like I'm still okay I can still go play and enjoy and have some fun but like I'm not gonna go qualify for DEF CON these days because I haven't kept like I'm still a decent reverse engineer. And, although I like to solve some of the parts of CTF that have gotten so specialized and advanced and continue to build on the history of other challenges and other events that I like, it's very difficult to be in that mindset. And it's different to like the day to day rehearsals during work. You know, the pacing and the goals are kind of different maybe than like, drop in and go and just super speed rush. It's like people who speed run video games versus just play it. You know, it's a very different way of playing and CTF is a little bit like that people who play CTF have to be fast and so live CTF kind of distills that down. And it's like livectf.com. You can go see the bracket we kind of did last year we took one person from each team and put them up head to head and we just gave them a simple binary inside go and then we captured the screen so we gotta watch what they were doing much like you wouldn't esport I'm trying to get booth this year. We'll see if that works out. I want to get him into nice sound isolation because ironically one of the problems is we couldn't broadcast it in the room and talk about what they're doing because like they would hear it and so we need to sequester them somehow. So we're trying to figure that out. Alright, I hope we can because I want the other players in the room. I know that other players were tuned in; they would watch the stream from their laptops and put in headphones or whatever. So some of them are watching but I want to be able to have it more engaging for the live audience and not just people who were able to watch the stream. Two, but that is we just watched them go. But yeah, those are the easier challenges. You can't have the same advanced things that will take you far too long to dissect into even even those challenges watching some of these torture people do what they do. It's so hard to keep up even with somebody who built the challenge or knows what the answer is just to watch them and try to think what's going through their head can be super challenging, but I think I think it was a success. I think it was a step in the right direction. And yeah, we're still trying to figure out how we do that. Like how we make it accessible.

[MUSIC]

VAMOSI: I opened the show with DARPA’s Cyber Grand Challenge. Jordan was there, in the back, helping out with the live TV broadcast. Yeah, this computer revolution was televised. I talk about CGC with people who participated in the competition in Episode 3 of the Hacker Mind. But Jordan, as I said, was in the back.

WIENS: I was actually the guy running the like you know like in the football things the guys doing the stats behind the scenes that was me and a crew of people backstage at cyber Grand Challenge and effect for the last like 30 minutes. It was me in Viserys earpiece directly telling him because we had some drama with like, we had a whole production pipeline and it was taking too long for the analysis to go to like a video editor to go to the script writer to go to the teleprompter to get on and so we cut all that out and I just got into his ear and I'm like I was yeah, I'm just gonna tell you what happened. Live as you talk to the camera. It was chaos. Sorry to interrupt, but yeah, that was actually when we started our company. It was our first paid contract to work on helping build the visualizations and the kind of behind the scenes analysis for that event. So yes, very familiar.

VAMOSI: So, watching hackers work, that could be kind of boring. Yet somehow, the folks behind the CGC made it interesting. For one thing, they collapsed the 10 hours of hacking-- when, literally, the machines were doing all the work and the teams were just sitting around awaiting the point totals for each round. So they collapsed the first 8 hours into a brisk recap, with graphics, music, and interviews with the different teams as they heard the results. That recap then went live at some point and the final rounds were as it happened. It was amazing.

WIENS: the final round  kind of catch up. There was a point where we went real time which was kind of fun too. So it was time such that it went from just okay we're recap recap recap to to live that's also why it was so stressful being behind the scenes and trying to keep up with everything that was happening. And to get that information to a broadcast. That was yeah, that was really stressful. It was such a cool event and they had done such a good job and it's such a good team of like video and light and all the other stuff. But it was crazy. And you know, like I think we actually went back and did I don't know if you saw there was a recap video that Deb Nuttall and myself did where we had time to actually sit back and like okay, what did we miss, like what really happened? And really recap it in a tighter kind of summary. Because it was so hard to like, really understand what was happening. What were they doing? Are they not you know especially because these were these machines, you couldn't even go talk to them. You're just looking at their output and trying to figure out what was going on. It was a huge volume of stuff too. So I guess that we had a whole team of people backstage whose entire job was just analyze everything that came out and figure out what's working, what's not what are people trying what are the cool tricks and some of them we caught live and some of them we only really were able to dig into, you know, after the fact and post mortem.

VAMOSI: So another competition from the US Government is the annual hack-a-sat. Just like it sounds, the competition is a series of capture the flag challenges that first get you access to a satellite then asks you to control the satellite. In the first three, the satellite in question was digital, a representation. In hack-a-sat 4, in 2023, the satellite will be real and in fact orbiting the earth. This, it would think, would make for some pretty cool television.

WIENS: Yeah, and so I've been lucky enough to be sort of like the tech spokesperson for hackers for the last three years. And you know, the first year it was supposed to be at DEF CON it's when COVID hit. And so they went from there. they actually asked us as a company because of our kind of background in this visualization and gaming and kind of the overlap there. Could you build this like a 3d model that we think would be really cool to show people like, we just want a board scoreboard. That's really neat. And so we were building that? And they said, Oh, do you want to help present a little bit like at DEF CON, because, you know, Jordan, you've done you want to do this kind of CTF stuff and you don't want to do more broadcast stuff. And I said oh sure. So it started as I'll show up, it'll talk a couple of times at DEF CON, I'm probably gonna be anyways, and then it turned into oh, by the way, you're the only face of the show because COVID And so we had to set up a full production studio and figure out the workflows for doing that. So the first year was, I think, kind of rough. Turns out my lighting chops needed a lot of work and I learned more about white balance too late to get good video, but it was a learning experience. But I mean, the actual content was fun. And it was you know, telling people what was happening in this really cool one of a kind thing of teams hacking into satellites. And dealing with technology has new constraints. And so that's kind of, you know, the last two years, thankfully, we knew in advance it was going to be remote. And so I was not the only voice there was actually people that were able to talk about other things and we could interview other people and I was just sort of like the sideline reporter doing tech updates. But it's been Yeah, it's been a super fun experience. The thing I like about it, too, is you know, a lot of people underestimate how much a good marketing team can do just to be able to actually have a company who really can design really cool graphics and can clearly put the work into good video editing beforehand and recaps afterwards. And so that kind of stuff like that's, that's a tremendous amount of time and effort. And obviously, you know, it's, if it's funded, if you've got the budget for it, it makes for a much more entertaining and you know, thing and they do a good job to making sure that like, lay people will understand it because they are the lay people there in the marketing folks are like, Well, if you can explain to me then, you know, I explained to my grandma then then that's good. And so it's, it's a good bar to try to make it more than just for the geeks by the geeks, but to make it something that everyone else can understand.

[MUSIC]

VAMOSI: given the immense creativity and the sheer number of man hours that go into, say the DEF CON finals. We're talking about live CTFs. We're talking about potentially making it an esport in some ways. How does that scale then and still have that, that creativity and brilliance?

WIENS: So yeah, it's a trade off and you do lose that. So for example. So live TTF as a concept. So George Hotz, geo hot started this kind of thing many years ago, and I had done a couple as well like just a long time ago. Just this idea of like, hey, this feels like we could do something with eSports with hacking, kind of like this sort of Twitch and live streaming model, but like, can we make it competitive and head to head and can we add comments? And I mean, there's been experiments of people doing different things, kind of around that even the prosecutor had done some of that as well too, with some of the stuff they were doing. So people were kind of tinkering with this. And trying like, how can we, how can we do this? Even Cisco, actually, recently I saw is doing this and there's a couple of other folks kind of getting into it. But I will say it is very much a trade off. And so the challenge is my friends that were doing the live CTF stuff last year at DEF CON finals, these are like, Baby challenges, right? These have to be for a couple reasons. One, because it's not fun to watch somebody stare at a screen for eight hours. Right? That's just not engaging and to make it something that you can even engage with and talk about to try to make us understandable. You have to make them sort of easier, simpler problems. And so they tend to be what used to be called sort of derogatorily like baby's first challenges or now intro challenges or warm up challenges, which a lot of CTFs do which is a great thing to do is intentionally put some challenges because the CTF as a sort of sport has and as a game has matured to the point that you know, it's it's it's high art,

VAMOSI: It was surprisingly entertaining, for a hacking competition by machines, not humans.  but what Jordan’s talking about is simplifying the tasks, having the hackers  in a soundproof booth, and doing that over, say, Twitch, basically.

WIENS: Yeah, exactly. Yeah, we wanted to make it digestible. We wanted to make it sort of the purest speed of the fastest hackers and you have to be both good and you have to be fast. And that's really the same thing. There are people who are far better, excellent writers and reverse engineers to me, that would not do well in a live CTF because that's just not the pace that they operate on. So it is specialized, and I don't hold it up as a great representation of like, these are the best of the best at like all hacking. It's a very niche skill set, but it is still really good and they're still fantastic and the best people have a combination of just great familiarity with their tools. good intuition. Very, very fast. Little bit of prebuilt analysis. And maybe I'm curious I am really curious if next year we see more people come preloaded with things like you know kind of chat GPT like auto saw some of that stuff can it help you expect I don't know like it'd be interesting to see are people start building automated tools, which ironically, was part of the goal of CGC right probably goals did you see was to do that. And I think most of what was said in CGC didn't really fulfill the vision of true AI machine learning. And it was more about orchestration and automation and kind of more like expert system level stuff. And we're now maybe at the point where actually if you did it again, you would really see actual cyber reasoning systems and that was the name of what they called the system that was cyber reasoning systems. And I think that may have been a little over promised for what, really what all the teams kind of converged on of the solutions that proved to be successful and the way that things worked, and the reasoning is all strong. But what we're seeing today in modern ml really, at least in my opinion, sure. Looks like reasoning to me and so if we ever go back and apply that against Alexei, do you see it be? Yeah, really interesting. But as you said, being a big  budget, I don't even know the total budget for CGC. But it was a lot.

VAMOSI: And we’re already this at DEF CON. Last year, Jordan and his team did a live CTF in parallel with the CTF.

Live CTF:

VAMOSI: And this year …?

WIENS: Well, I mean, live CTF is a part of DEF CON. Right so last year, you got points in the DEFCON CTF based on how well you did at my little show and how far you made it through the tournament. And it was enough that like you had to take it seriously like you didn't want to just phone it in and skip the event. It wasn't you know, it's kind of tuned to do. I think the goal I forget exactly the phrasing of it was we wanted it and again, this was the Nautilus Institute is the group running it. And they sort of made the choices that we just said we're here. We're going to give you a list of people and we'd like to hear some points because it'll incentivize these teams to send the best people it's like you've got the some of the best hackers in the world playing DEFCON CTF, why wouldn't you want to like take advantage of that? And so we wanted to incentivize them. So they made the points such as, you couldn't ignore it, but it wasn't gonna like take somebody from a first right you don't want to like just totally, you know, swing too hard, and so they kind of tried to tune it to do that. And so it was a part of it, and it will be part of it. I guess we haven't actually announced that anywhere yet, but I don't think that they'll get mad at me for saying we're doing it. We should probably start talking about so yeah, I first exclusive announcement, but we are. We're going to be back this this next year. And yeah, so hopefully for for both calls and finals. We'll have some some live CTF stuff.

[MUSIC]

VAMOSI: There are examples of live CTFs today.  George Hotz has done a few markets Hutchinson and other people are doing it on on like YouTube. Is it finding an audience? Is there a commercial value in doing live CTFs? Or is it still just a niche thing that I would probably tune into but no one else would?

WIENS:  That's a great question. I don't know that. I know the answer. It is interesting. You know, when CTF itself started, many years ago, it was very anti commercial, like sponsorship, no, keep your grubby money out of my pure hobby kind of thing. And some of that's kind of like toned down, right? Most conferences that are still around still succeeding commercial backers. You know, DEF CON, even DEF CON is well, sort of its own thing not really sponsored by a company that is still actually owned by a commercial entity that runs Blackhat makes a lot of money and like it's still a commercial venture. And so I feel like there's this kind of interesting thing, people are getting a little more chill about that, a little less of the anarchist hacker vibe. But I don't know I think there is a again, I mean look at like Sousa clearly thought it was valuable to like to talk about and to be able to like, have this kind of diecast eSports casted hacking event. So I think it's there. But I don't know a part of me still wants it to be me, my friends doing the thing for a while. But I also wanted to grow and to get a bigger audience and that sort of necessitates figuring out a way to make it commercially viable. Whether it's, you know, games, you know, LED lighting and gaming chairs, sponsors and keyboards, you know, mechanical keyboard manufacturers or you know, get the keyboard that the best hacker uses like, I don't know what that looks like. I but I would like it to be I think there is still I mean, we had all of the numbers but we had several. I think like 1000 People live watching live and I've used the recap since then for the license. So to me like that's that's a decent start for an audience to so I think there's something there too, and there's getting the first year that we had done it it was still like so hopefully you know if we come back this year, we'll get even more views and and it can be accessible to right because the goal like the goal is you can't have it just be look at this art. This arcane, subtle thing you have to make it accessible which is one of the things I really liked about cyber grand challenge because they had the budget and we at the time, we built ground up visualization technology that didn't exist anywhere. It was actually a game company. And that was our original task in mind and my co-founder rusty like we went to work with a game company that knew 3D visualization. They knew nothing about CTF and the question is, well, how do we show what's happening inside the game? How do we make this comprehensible? And I don't think anybody has really done that since not to that degree. There was a CTF in Japan. It has a really flashy kind of visualization of just and there's been a lot of PTSD. We'll do like, watch the PPU packets and watch like, you know, scores. Everybody tries to do something like that. But the degree to which we were able to really design something custom and think about how to clearly communicate and to show things that were both technically real and also comprehensible. There's just a whole bunch of stuff just even on the visualization that we did that I've not seen since and I think needs to be a part of if it ever gets bigger, but I don't know we'll see. I'm a little busy with the company and you know, I don't know where I don't know where where it goes, but I do want to see you succeed

VAMOSI: So what advice would Jordan give for someone who wants to transfer their skills over to infosec, someone who wants to get started. CTFs?

WIENS:  I mean, he know you know, George, George intimidates me. I remember when he was soloing, beating the best teams in the world and then but you know, he had his reign and then Loki Hart came along and did the same thing to him like there's just always somebody better, which is which is of course, a humbling thing. You know, as you get older, but there and that was earlier it has gotten to some of these top tier CTFs you can't just show up and do well at that's just not possible. Because they're so honed. I mean, not always sometimes just being really good at the topic is sufficient. But they're, you know, there's whole categories of challenges that have like this history of, of, well, this challenge inspired this challenge, which is part of this challenge and no less, you know, the lineage it's just harder to kind of kind of get into it. So that's one of the things that sometimes you have to balance as a CTF organizer. Some of that's fine and perfectly acceptable because it is part of this culture and part of this scene, but also you do want to let people have an on ramp and so you know, there are a lot of CTFs pico CTF is obviously one of the best.

VAMOSI: In EP 29 I interviewed Meghan Kerns who runs PicoCTF out of the CyberLab at Carnegie-Mellon University. It’s open for everyone. It’s designed to get you started with your CTF experience.

WIENS:  Brumley has even said that he's somehow gives me partial credit somehow for the idea that which I think is bogus, because that was that was all him but it's a fantastic Well, him and obviously a lot of other people but it's it's a it's a fantastic CTF. There's no seesaw seats, CTF there's a couple of ETFs that have the quality qualification or an NUS for seesaw. And there's some that focus on that kind of beginner thing. The more important thing is just start doing it and be okay. failing at first and learning and going really write ups afterwards and then just kind of you have to wean yourself off of it. The same way that when you start programming 90% of what you write is from Stack Overflow. And the more you go like it's never zero. I've been doing it for long enough, it's never zero, but it gets smaller and smaller as you're able to do more of it. It's the same way with CTF where at the beginning you're just like overwhelmed and then you just focus on one thing focus on the other even even George you know when George he didn't overnight become the best person in the world of CTF like he methodically said, I don't know crypto. I'm gonna go and read all the research papers. I'm gonna go read all the write ups and practice them. He didn't have a job at this point. He was just between things and kind of had enough money he can literally just devote himself to studying the domain for a while and so he was like, I'm gonna do crypto. I'm gonna go to this like he kind of picked it apart. Most people don't have he's also just very smart to begin with, but have both the time and the the ability to just go and do that across the field, but like picking, okay, let's go look at their lives web app security. Let's go look at the latest bug bounty reports. Let's go read, you know, find the right blogs or tweets or where people are talking about web security and just get really good at that. Because even most teams don't have people that try to do everything. There's the crypto person, there's the reversing person, there's several vulnerable people. There's the web app, there's there's all these different specializations that you have to do and so that's the other part of it,

VAMOSI: So Jordan pretty much embodies the Hacker Mind, having started out with CTFs and now running a company, and hosting televised competitions.

WIENS: We've talked about so many of my passions that I could go on for literally hours on any of them. So it's hard to know where to stop. Yeah, I I'm excited that this like I you know it's funny when I started March 35. I put on my little snarky bio the website which we need to update. It's been years since I took it. Take them down I think but CTF anthropologist slash apologist basically like I really like look, archiving and collecting the stories and what's happened in CTFs. And so a lot of my talks have been about some really cool things and they're, you know, just things I've experienced, other people get I've not even been active in the last decade nearly as much. And so there's tons more stories happening and I want this to be kind of collected and talked about. I think it's a cool world with lots of neat stuff. But also I want to like more people to be involved and active. I want to be accessible. I want there to be more on ramps to get people into it because for a lot of us this was you know, in the 80s and 90s we started getting into hacking you just like you hack in some servers and you just hope you didn't get too much trouble when you got caught eventually and or you crack some software like that was how a lot of people that are now the sort of legitimate got their start. But now you know, the rise of cybercrime like that's actually problematic and it's a bad idea to make that your start into information security. But the great news is we've got CTS, we've got this just unbelievable mechanism and a lot of people have criticisms about CTS that are very fair in terms of like, how realistic they are, to the world what you can be great at CTF and miss a lot of real world skills that makes you mediocre at a real job. But generally speaking, there's actually a lot of overlap. In fact, even you know some of the weirdest challenges that I've either written or known who people who wrote the weirder they are the more likely to actually have some really fascinating basis sometimes even like I know some people who were working like sensitive, you know, stuff that had some really weird bug that they couldn't public ever talk about. But they recreated the kernel of the really interesting bit in this totally benign CTF challenge and people like oh, this is so unrealistic and they're like if you only knew, so it does have a surprising amount of kind of real world overlap. And it's just fun to so you know, I think that as a way to get people into vulnerability research and information security versus hearing. It's pretty fantastic.

VAMOSI: Jordan further cautions that everyone is going to be able to compete in the Live CTFs.

WIENS:  that's actually my favorite things ab out Live CTFs as you see even the best people in the world who are far better than me. I watch them do things like oh wow, they were they just did this or that's where they read this. They could have done this command line tool. They couldn't use this feature of binary ninja either or whatever it like. So it is it is interesting. I remember even the first time George did that I was like Wow, he's so good. And he makes all these mistakes. And it was like again it didn't detract from him in my opinion cuz he was still like gotten to your level. Good. At the time how the he was just so effective. But like it did humanize them a little bit to see him kind of behind the scenes. So I think that's one of the great parts about live CTF is you can both be awed and amazed sometimes just how good people are, but also like, oh, actually, like I need that little thing or I could have done that a little bit better, you know, and so so I think that's, that's a fun, fun aspect to it.

VAMOSI: I’d like to thank Jordan Wiens for coming on the show and talking about his history with CTFs and his company Vector 35 and it’s product Binary Ninja.

‍