The Hacker Mind Podcast: Defending Costa Rica From Conti Ransomware

What is it like to hack an entire country, to take its government services offline, to deny a government an ability to function? Costa Rica knows. Esteban Jimenez of AttiCyber has been helping Costa Rica improve its cybersecurity posture for more than 16 years, and he has been helping them recently recover from a crippling ransomware attack in April 2022 that hit 28 ministries of the government. Central and Latin America appear to be a new playground for bad actors testing new malware. But Central and Latin America are learning how to fight back.

‍

[Heads Up: This transcription was autogenerated, so there may be errors.]

‍

VAMOSI: What is it like to hack an entire country? ‍

I mean what is it like when criminal hackers take out the basic services provided by the government of a country? For instance, they might shut down foreign trade, the immigration process, healthcare? Or limit basic access to payment processing or banking?

‍

Something like that happened in 2007. 

‍

There was this Soviet-Era statue in the center of Talinn, Estonia.  Originally called "Monument to the Liberators of Tallinn '' the statue represented different things to different people. For ethnic Russians, the statue represented the Soviet Union’s victory over Nazism in World War II. However,  for ethnic Estonians, Russian soldiers were not liberators and the statue was a painful symbol of half a century of Soviet oppression that followed. In 2007, the city government voted to remove the statue from the city center and place it within a cemetery. 

‍

Chaos broke out as a result. Here’s the International Centre for Defence and Security 

‍

ICDS: in 2007 a struggle over a divisive soviet statute set the standard for a new form of Russian interference in the affairs of foreign states plans to move the bronze soldier in Tallinn led to riots outrage and the first cyber attack ever attempted on an entire nation-state

‍

By modern standards, the cyber attack on Estonia in 2007 was relatively low-tech. Botnets directed massive waves of spam and huge amounts of automated online requests swamped servers with distributed denial of service attacks. The result for the average Estonian citizen was that cash machines and online banking services were unavailable; purchases of gasoline and food were not possible, government employees were unable to communicate with each other; and newspapers and broadcasters suddenly found they couldn't deliver the news, either online or in print. This lasted a few days.

‍

The next cyber targeted nation appears was Ukraine. As far back as 2014, there began a series of large scale online attacks. The most famous of these was Crash Override, which shut down power plants in the dead of winter. But even in 2022, when Russia physically attacked the nation, Ukraine was never fully shut down. It was resilient and remains so today.

‍

The next such example of an entire nation being shut down by an orchestrated cyber-related attack was in Costa Rica in April of 2022. This was a few weeks after the Russian invasion of Ukraine. And this was a ransomware attack, asking millions of dollars from the Central American nation. 

And in a moment I’ll introduce you to someone who was central to handling the incident response for Costa Rica, a nation that is still today recovering from the ransomware attack some 18 months later. It’s a story about what happened in Costa Rica, and what could happen in the rest of Latin America, in Africa, and in Southeast Asia. I hope you’ll stick around. 

‍

[music]

‍

Welcome to The Hacker Mind, an original podcast from the makers of Mayhem Security. It’s about challenging our expectations about people who hack for a living. I’m Robert Vamosi and in this episode I’m discussing the increasing cybersecurity threats facing Central and Latin America, the perception that these are defenseless targets, and I’m interviewing one of the people who are making changes in those countries to bring them into the twenty-first century in cybersecurity defenses.

‍

[music]

‍

VAMOSI: Costa Rica, like Estonia, is a stable democracy with a highly educated workforce. Bounded on the north by Nicaragua and on the south by Panama in Central America, with access to both the Atlantic and Pacific Oceans, Costa Rica began as a Spanish colony in the 16th century and declared its independence in the mid 1800s. After a civil war in 1948, it abolished its military force, and remains without a standing military in the middle of Central America. Today is home to several large international corporations including Amazon, Intel, IBM, and Microsoft. It is home to many cloud data centers. Among the Central American nations, and even those in Latin America, Costa Rica is perhaps the best prepared to fight off a foreign cybersecurity event, although that strength is relatively recent, largely untested, and the product of a core group of individuals, one of whom is my quest this episode.

‍

JIMENEZ:  All right. Well, my name is Esteban Jimenez. I am the Chief of Technology of the Costa Rica and cyber defense company name at Atticyber and I used to be a cybersecurity engineer for the Intel Corporation. I worked for Intel security for about five years. And I also worked with IBM Security back when he was not called IBM Security, starting with many of the operations that they currently have. And we created the security operation center that's located here in Costa Rica that now holds I guess, is the largest one outside of the United States. Working with teams all over IBM, IBM X Force, research and development, the iris division, so it was about five years to prior to that I also worked in other missions with Bank of America. So I've had some good runs with with some of the big players in the industry.

‍

VAMOSI: So we *are* going to talk about the initial Conti ransomware attack in April 2022, and the subsequent Hive ransomware attack that occurred shortly after, but as I began to talk with Esteben, as I learned more about his role at a private company working with the government, I could see how there was much much more to the story. So I think it is important to understand first how far Costa Rica has progressed with its cybersecurity program in less than two decades. 

‍

JIMENEZ: Well, if you go back 16 years, Costa Rica did not have any, you know, cybersecurity practice in place. So I'm actually one of the founders of the cybersecurity specialization in the country, 16 years ago, I'm 34 right now so I started young and we have created here in the country, many of the cybersecurity trainings for the public education system, for universities, which were built up the National Cybersecurity Strategy. And just recently, I am also the person who wrote a part of the new law  the National Cybersecurity law that now creates the Costa Rican National Cybersecurity Agency that's currently in approvals. So I've done a lot of work with the government for the past 16 years

‍

VAMOSI: So, as I said, Esteban is clearly one of the central figures when it comes to discussing cybersecurity within Costa Rica. 

‍

JIMENEZ: It's interesting, because we were a handful of people, you know, sixteen years ago, I was the youngest, probably the youngest of the team. Some of them work with the law enforcement, some of them work with the University and some of them work with private individuals in the private sector. So we kind of created this public private Alliance at that moment, and our groups join in creating the communities that currently go to regard really develop. We have people in the country who have created pathogens now. We have groups in the universities actually now graduating from doctorate PhDs on cybersecurity in Costa Rica from the technical Technological Institute of the country. You know, there was no there were a lot of good people with theoretical knowledge on what a cyber attack was, but no practical knowledge.

VAMOSI: Okay, this is important. I have CERTs in infosec, but I’d have to admit that apart from some hands-on work, I, too, am more theoretical in my knowledge. I’m not on a SOC battle real world demons. And sixteen years ago, that’s where Esteban and his peers were -- they had a lot of knowledge, but very little hands on experience with fighting live cyber attacks.

‍

JIMENEZ: You know, everything that they was around was just people that came from the United States from working with private companies, who had experience with some light attacks. We received a lot of training and collaboration from allies in Latin America for example, Brazil, Argentina, you can name a lot of those communities who started to work with talent in Costa Rica to develop these new capabilities. And I remembered that one of the first roles that I had growing up in my career was back in 2011, when Anonymous was really big at that time, and we had an operation that's called Operation Pura Vida, you know, like Costa Rica has this phrase, right? Which is really famous in the world, which is Pura Vida, it's like, like our signature phrase.

‍

VAMOSI: Okay, I do recognize this phrase. In my 30s I decided to challenge myself, to get out of my comfort zone, to live dangerously. So I enrolled in an Outward Bound course. Perhaps you’ve heard of it. It’s a program started in the UK designed to get young men to toughened up by learning basic survival skills. Today the program is open to all and is hosted in various countries. I wanted to toughen up so I decided to do a 10-day program … in a foreign country, one where I did not speak the language. I chose Costa Rica. And it was fantastic. I spent 10 days along with five other adults, each armed with machetes, cutting back bamboo as we trekked through the untamed rainforests just west of the  capital San Jose. Over the course of seven days we walked under the canopy of leaves and howler monkeys, over countless fallen trees, and stepped aside poisonous snakes, heading toward the Pacific Ocean, ending up in Quespos, a beach town near Manuel Antonio National Park on the coast. And so, from personal experience, I know Pura Vida. In english it would be “Simple life” or “Pure life” It’s used as a greeting and a farewell. So Operation Pura Vida was perhaps a way to say goodbye to Anonymous. 

‍

JIMENEZ: And it resulted that at that time, there was a law that was trying to get into the Congress to censor the press. So this was a revolution in the country. We never, you know, previously had any of these kinds of really, you know, like really strict laws on things that you could say in public and things like that. So it resulted in the creation of an anonymous group here in Costa Rica, a big cell and affiliate of the big anonymous group started to create attacks against the public apparatus. 

‍

VAMOSI: Anonymous is a hacktivist group that has evolved over time. It used to post these videos demanding changes that it sought. It always ended with the following:

‍

ANONYMOUS: We are Anonymous. We are Legion. We do not forget. Expect us.

‍

VAMOSI: When I first covered Anonymous for ZDNet with Project Chanology. This was Anonymous’ response to the Church of Scientology's attempts to censor or remove material from a highly publicized interview with Scientologist member Tom Cruise from the Internet in January 2008. 

‍

Anonymous: hello leaders of Scientology we are anonymous over the years we have been watching you your campaigns of misinformation your suppression of dissent your litigious nature all of these things have caught a write with the leakage of your latest propaganda video into mainstream circulation the extent of your malign influence over those who have come to trust you as leaders has been made clear to us Anonymous has therefore decided that your organization should be destroyed

‍

VAMOSI: Online, Anonymous proceeded to DDoS various Scientologist websites and enact flash mobs at various Scientology centers around the country. 

‍

Anonymous has since targeted ISIS, supported the George Floyd protests and generally attacked various others seen as harassing journalists and women. Anonymous has resurfaced today and appears to be going after pedohiles and sex traffickers. At the time, in 2011, Anonymous was reacting to attempts to censor the news media in Costa Rica. Politics aside, Esteban’s group stepped up to help the government defend itself against the online attacks.

‍

JIMENEZ: And many of us who were a part of this incipient group of specialists, started work with the government, because the attacks were really aggressive. It was actually the first time that the country was attached directly to critical infrastructure. And back in the day, you remember that the low orbit cannons when we're really big are anonymous, right, so there were a lot of groups building them here in Costa Rica. 

‍

VAMOSI: At a high level, Low Orbit Ion Cannon (LOIC) is an open-source network stress testing and denial-of-service attack application. What that means is it amplified DDoS, so that the volume was much larger than previously seen at that time. While anyone could use it -- again for penetration stress testing of websites -- it was used by Anonymous. Esteban has a heads up that this was coming. 

‍

JIMENEZ: I remember being in some of those chats because we got to infiltrate some of those incipient groups, and we figured out that many of the the manuals that detailed instructions on how there were the anonymous tools because they had a hidden websites where you could not only download different tools, but but also access some of those low orbit cannons who were already built up on some websites and things like that. They will explain to you step by step how to attack certain infrastructures. And I remember one day that an attack was scheduled for midnight against our let's say we have a small FBI here in Costa Rica. Which is the best the organism for judicial investigation, OIJ, and they had a huge attack that was orchestrated against them overnight against them, and also against the systems that the Congress. We figured out that this attack was going to happen that night. And we started to call and try to reach some of the system administrators who were already at their houses, right. So it was really, really a challenge to try to get a hold on them. 

‍

VAMOSI: This is a real problem. When you are not in the habit of dealing with live attacks on your system, you may not be aware how much someone needs to always be on call, always be ready to mitigate any new threat that comes along outside of normal working hours. So Esteban’s groups, aware that this Anonymous attack was likely to happen that night, took matters into their own hands. 

‍

JIMENEZ: So what we decided was to physically go and disconnect the servers at 11pm because we were sure that the attack was going to happen over midnight at some point. So that was the actual, you know, strategy to avoid the system from being hacked. Because there was no administrators ready so we had to go with the group of the investigation organism and physically unplug all the servers and just wait for the attack to pass, you know, and that was one of my first encounters with this kind of environment.

‍

[Music]

‍

VAMOSI: So I'm curious what happened in Costa Rica sixteen years ago that caused this group to come together. Or was it just this hacker collective coming together and saying Costa Rica needs this security.

‍

JIMENEZ: Well, it's really interesting because what you will find 16 years ago it's a it's a change in its strategy of Costa Rica, in terms of how the country wanted to be perceived outside.

‍

NatGEO:  Our next destination is Costa Rica. 

‍

JIMENEZ: So they started out with some new public laws and strategies to develop high technology companies, right, everything started out actually 94 when Intel was brought in, so the Intel revolution is how we call it here. 

‍

INtel: Intel came to Costa Rica was one of the primary reasons was because of the labor force two or three years out of college the engineers we hire out of the schools here in Costa Rica are working on Intel state-of-the-art products the crown jewels of Intel are made in this Factory and we find that you know engineers right out of school here it can be productive in a very short time so this has been a big success story for our site

‍

JIMENEZ: This was the first huge company of you know, high technology that was established here in Costa Rica. When Intel started work here. That changed everything because training from Intel on those specialized lines of of science, started to pour down the whole the whole, the whole academia industry and I started to change the laws inside the country and how technology was perceived as a new engine for development. Right. So along 94 and moving forward right when we kind of started out this whole movement that a lot of new companies started comes Costa Rica, actually today, you'll find that many of the Fortune 100 companies are established in Costa Rica, you will find a lot of them actually Amazon, for example, has the largest operation outside of the United States here in Costa Rica, VMware, Dell, operations are really well established. So a lot of people have started to work with these high tech companies. And the result of that was we started looking at the security as a missing piece on the parcel, right. People were started out working with these big companies realize that all of these attacks were happening in the US right and they were also happening in Costa Rica and some other Latin American countries. 

‍

VAMOSI: This makes sense. The internet knows no national borders. But if a tree falls in the forrest, who is there to here it? In other words, of course these attacks were happening, but how would Esteban and his cohorts even know about them?

‍

JIMENEZ: we had no way of detecting them. You know, there was no sensors. There was no strategy and all this knowledge that we started again, from working with the international companies, at some point, we realized that that knowledge needed to be transferred to the government right needed to be transferred to the government needed to be transferred to the universities. So fortunately, people like me who, for example, thanks to Intel, when I worked at Intel, they gave me my first specialized training on security, even though I started working with computers when I was three years old or something like that. Thanks to my mom. She is actually the first one who taught me how to turn on a computer. But Intel gave me my good training, you know, and I flew to and to the United States. My first role up certifications, naming you know, CISSP G say, I took those with Intel, and I started to be a part of the community. I attended my first DEF CON in 2012 when I was selected by Intel to go there, you know, so this is the story of some of the other foreigners that I grew up with here in the country who went through the same process. And this is the years, you know, starting 2000s 2010, the first decade of the 2000s when Costa Rica decided to be different. Now what you can find here in a hub, that's one of the most important hubs in Central America and the Caribbean is probably one of the most sophisticated technology matrix for Costa Rica has one of most sophisticated technology matrix in Latin America. And in that started in that first decade of the 2000s.

‍

VAMOSI: So I'm imagining this group to be something like the L0pht -- spelled L-zero-ph-T -- here in the United States. In the 1990s, Boston, Massachusetts, was the hub of early hacking with all its colleges and universities, with MIT being central to any hacking story.  For more on that I recommend Steven Levy’s book Hackers, which traces the early days well. The L0pht, then, was literally a group of young hackers who rented an artist’s loft in South Boston and started doing network hacking and even early hardware hacking. Eventually the L0pht was sold to @stake which was eventually sold to Symantec. I’m imaging Esteban’s group was like an early L0pht, a group of like-minded hackers who go together and started asking serious questions about their government’s cybersecurity. At the time Esteben worked for Intel, and others in the group worked for other international companies in Costa Rica so the technical knowledge was there. So how many people were there in the group 16 years ago?

‍

JIMENEZ: I would say that it's less than it was less than that, but it probably isn't 20 votes. So it started out with, you know, you know, brainstorming and talking to each other, and figuring out how to bring all this to Costa Rica, how to how to transfer this knowledge to some of the people here and figure out how to use this the new tools that we learn from to protect the citizens in our country, because it was really clear to us and it's still being very clear to us that Latin America, the same as some parts of Africa, in Southeast Asia, in some regions are used by hacker groups as playground. So some of the things that we have found in you know, throughout my career so far, I've been able to identify that our region because we like response capabilities, I mean, it's not the same as attacking the United States or Israel or the UK or something like that. You what you're going to find is that this is the perfect place. For a hacking group to drain the resources. And most of the times what we find is in those systems, when we you know, go in and attend an incident response process or something like that. Many times we find prototypes of malware that are still being developed, you know, we find even debug codes we find comments on the the exploits, because what they're doing is that they're testing it's much easier to test your weapons against the infrastructures. We're not as mature as the ones in the north. And then once you get your weapon ready, then you can throw that against the United States and other governments, right. But usually what we have found is that our countries are really really vulnerable. People are not aware of how to respond to these kinds of threats. Or threats. And most of the times they don't even share that with anyone. They don't share that with the public. They don't share that with law enforcement. There are no metrics, there is no way to test or measure. What's the index of exposure that the country has because nobody is taking note of that. So those are some of the things that we set set down with this incipient group and we say, hey, we need to change it. It's wage changes. At some point, this is going to cause us not only money, but maybe lives.

‍

VAMOSI: So this at a high level, what sort of opening scenario are we talking about you you go to the government and you say, we know this is going to be an issue, and you find that they're writing passwords down there configurations are all over the place. So what sort of low hanging fruit did the group discover initially 16 years ago?

‍

JIMENEZ:  Okay, well, some of the initial exercises that we developed, were really Academia like exercises for example, one of the first things that we tried here in the country was a war driving across San Jose, which was impressive, you know.

‍

VAMOSI: War Driving. Basically you drive or bike around an area and you identify any open SSIDs, or wi-fi signals. Back in the day these might mean your organization had a rogue access point -- so you’d want to know that, and mitigate it.

‍

JIMENEZ:  you would think that wardriving in its, you know, pure form will go out and we will, I mean, in the United States, we will throw one of your sensors and you can hook it up with a GPS device. Just drive around, find some vulnerable abs, right. And of course, it's really interesting. You can create a heat map and all that. We wanted to tread. We wanted to test that in Costa Rica and figure it out. What was that risk map that was going to result out of that exercise? We were really, really curious about it. So we built a computer with some antennas to it, and a GPS device and we drove around San Jose at the center, you know, San Jose downtown, around for some 30 to 40 minutes. And we went through some of the you know, the central street Main Street, driving around where some of the major banks are located and some of the public institutions are located. At the end, what we found is that a rate is within a radius of around 40 kilometers. We established not only a huge risk to every AP that we found, because many of them were completely out of security. And I'm talking about the huge amount of APS I mean for 40 kilometers radio. We found around 150 aps that you could, you know, survey and and they resulted in some kind of vulnerability, you know, and you're talking 150 on a four kilometer radios and, and in a zone that hosts probably some of the most important public institutions of the country. You know, this first study came out and it was not well received. And this is a really interesting thing because it shows what you know about the low level of maturity in the country 16 years ago, which is something that you will see in Latin America in general, these kinds of studies are not well taken. And we're going to do that because to me, you know, doing a word driving especially with the people that I learned from in the United States, it was really something that you should do, you know, it was like, you know, security 101; you wanted to just understand risk. You had to do a war driving or some sort of survey just to first understand and do some braking, to understand what the problem may be. We came out with this study, and it's a it's a it's a you know, it's a memory that I have when somebody or the government called me because they already knew me, right? We've been in this initial group. Already. They gave me a call and they said hey, listen I got something to tell you. Your just study was not well received by some people in the government. And you might be prepared because these people had your name filed with law enforcement. 

‍

VAMOSI: Yeah, this happens with hacking sometimes. Where a researcher informs an  organization that they have all these security vulnerabilities only to have that organization turn around and call in law enforcement. Esteban, responsibility reporting what he found to a government entity, found himself being turned over to the equivalent of the US FBI.

‍

JIMENEZ:  So we have an intelligence division here in the country. And thankfully, one of those persons was also a friend of mine. And they knew me, you know, they knew that the study was well structured. It was informed. And the good thing is that the reply was, hey, listen, I mean, maybe you should you know, just just hold a little bit of doing any more of this for a while, but we're really interesting. We're really interested in the results. So if you can pass those results to us, that's it. We're not going to do anything against you or anything. We just want to know what the result was. You know, so and so those were typically the things that we used to find many of the people in this initial group, some of them unfortunately, when to the other side, you know, and they start they started release vulnerabilities of the government, publicly, you know, and also charging to charging some of the public employees or something to disclose the vulnerability with details, which was, of course, something unethical, and that is where our group broke down a little bit at that time. 

‍

VAMOSI: So the group that Estaben was a part of … splintered. There were some who felt the work was progressing too slow.

‍

JIMENEZ: Two groups developed one who was completely focused on ethical hacking, some other you know, Rotten Tomatoes, went on creating some local hacking groups. But it was interesting, interesting how you know, things developed at that time. And of course, we knew about the vulnerability level, pretty much on some of those rubies. Every single ministry or Republic institution came out with a lot of vulnerabilities. Some systems were outdated even, you know, since from the same day that they were installed it looks like you know, administrators in contractual controls. were only interested in buying the technology, but they have zero importance on maintenance and updating. 

‍

VAMOSI: This is a universal problem. Organizations and governments alike want the convenience but sometimes lack the technical expertise to realize how their the threat models change and how they should mitigate against them. They just want the shiny new toys to play with.

‍

JIMENEZ:  So when the providers came in, they just plugged in printers, they plugged in new servers, things like that, that when they plugged the infrastructure, the infrastructure what was left to abandon completely, abandon, you know, they forgot about it, so they never patched it. And I'm talking about really, really important databases from the government, really important databases from critical services, hospitals, things like that, you know, simply updating and having a culture of security was not non-existent.

‍

VAMOSI: So this might explain what happened 16 years ago when Esteban and his friends looked around and realized that Costa Rica needed a national cybersecurity strategy. Like plugging an unprotected laptop onto the internet and watching it slowly get infected with all sorts of malware, the government systems in Costa Rica were more or less wide open to attack. 

‍

JIMENEZ:  Well, yeah, you know, to me at that time was was really fun because you know, this is something that I've done all my life and I never charge for any of this with the government back in the day, it was fun, but yeah, sometimes they, you know, because the little bit the line now, you know, retros in retrospect, we did a lot of things for the government that at the end was not really appreciated. And sometimes it was also taken in the wrong way, which also caused, you know, some friction with some of the public institutions, some banks, but at the end, they changed some of their controls, which was positive. I even remember when we disclosed an attack from the ransom we grew maize in the country. It was big news here in Costa Rica because it was the first time that a major public bank was attacked and publicly ashamed by one of these hacking groups. 

‍

VAMOSI: So Distributed Denial of Service attacks gave way to something a bit more profitable for the criminal hackers. That is ransomware, where the files are copied and exfiltrated and then the originals are encrypted. The idea is that you pay for the decryption of your files, and then later pay again if you don’t want the exfiltrated files begin released publicly.  Esteban and his team had experience with a ransomware before Conti. In 2020 the Maze Ransomware group claims to have stolen  11 million credit card details of Banco BCR of Costa Rica customers. While not directly involved with the incident response, Estaben and his group did create some tools to help customers of the bank deal with the incident response. And of course this did not go exactly as planned. 

‍

JIMENEZ:  And we develop a way for the people in the country to double check and see if their credit cards were hijacked. You know, if those credit cards were part of the bridge, and once again, I mean, you guys would think that this is really normal in some cases, right? Where there's a breach so somebody in the community developed a tool where people can just double check. If there was a, their account was compromised, and then you know, take some steps forward to secure themselves because if you think about the ethics of a hacker, or an ethical hacker, in this case, you know that you have to put people with vulnerability first, that's your first mission is to protect people. You have to protect the user that's in danger. And then you have to put the critical infrastructure next, and that the other things right. So this is something that, you know, this is the chip that we have, but when we released the tool for people to double check, if their accounts were compromised, that blew up, you know, completely on some of the building institutions, they were outraged by this tool. They have never seen something like that. They thought that we were actually the ones who had the bank. You know, they didn't show any care for the customers. People were using the tool they started, you know, until the first day, the tool blew up because so many people logged in to check their accounts. And people that started to figure out that their accounts were compromised. They were heading to the bank to change their plastic and the bank started to return them to their houses, because they didn't want it to get the fee. You know, people didn't want to pay the fee to get a new card. And they started to withhold the cards in the bank, the bank accounts, which was of course, you know, completely unethical from any point of view. People were trying to protect themselves by going to the bank, changing the plastic, but at some point this was something that we released with a bleeping computer, living computer in ourselves, we created the first you know, row of information about what's happened with this bank in the base group. And it was really not well received and those are some scenarios that we have had to deal with for the past 16 years trying to change the structures because they were not ready for this.

‍

VAMOSI: So then in April of 2022, having taken on Anonymous, having helped educate the government on various vulnerabilities, and having helped with the Maze Ransomware attacks, would Estaben agree that his group was battle-tested enough to go against the Conti Ransomware group?

‍

JIMENEZ:  Yeah. April, April 18 of 2022. I received a call in the early morning from the minister of the finance ministry, right. The Minister gave us a cold and then he was the person in charge of it. And they said to us, we have a big problem. At this time, all our infrastructure has been compromised. We're talking 1000s of servers that are at this time, completely disconnected. And this is affecting not just us, not the ministry of hacienda, but also a number of other institutions and private entities they were talking about at the beginning we were thinking about some 20 institutions, large institutions, including the Ministry of Health, you know, but this started to create waves over because when the minister the ministry of Hacienda lost their systems, pretty much the whole country was attached to that right you're talking about customs. People were not able to ship anything outside because they were not able to charge taxes or calculate taxes right? collection was completely stuck in the whole country. Nobody could go and pay their taxes. Nobody could consult what they own to the ministry and every single digital service from every public institution that somehow was attached to the Ministry of Hacienda and we have to, you know, extrapolate this and think about the IRS of the United States completely blocked even for one second or 15 minutes. This is starting to create a hole in the public finance right, everyone is completely stopped, nobody can pass any invoice, nobody can pass anything.

‍

VAMOSI: So maybe we talk about the inconvenience of this type of attack and not enough about real costs. The government of Costa Rica was losing money every minute their tax systems were offline. And those costs were starting to get astronomical. And that’s the point of ransomware. At some point mitigating this will be more expensive than to pay the attackers. So the clock is already ticking. And Esteban was not called when it happened, but shortly after, so there’s time that’s already been wasted. This was when Esteban and team really had to step up and deliver for the government. 

‍

JIMENEZ:  So this is the moment when we knew that this was the most devastating scenario we've seen since the beginning of, you know, the studies that we started in Costa Rica over 16 years ago. This was critical. We knew that the vulnerability of the ministry was real because we practiced on this ministry for a number of times. And along with other institutions. We informed them three years prior to this event that the status of their infrastructure was really risky, that they had a lot of vulnerabilities that they needed to update their systems. And there are a number of reports about this, both from private companies and also from the regulators in the country where it is proven than for probably five to you know, some five years prior to the attack three years prior to the attack. Many, many people try to tell them to, you know, to update their infrastructure, but it didn't happen, you know, didn't happen. 

‍

VAMOSI: So this is a government that didn’t take the prior warnings seriously. Now, Esteban needed to step in and gain control of the situation, do some incident response, and do some forensics on the systems.

‍

JIMENEZ:  So what we found is that some of the first things that I did was I immediately when they gave us a call, I grabbed my kit, you know my incident response kit that some of us have ready for this kind of event scenario. And we kicked off the first meeting. What we found in this first meeting was that there was complete chaos inside the ministry. Nobody knew what to do and the attack didn't start on the 18th. This is a Monday. Right. But what we have found is that the attack actually started a week before on April 11. It was the first alert that was found on the on the on the SAM were in the monitoring system on the usage of Cobalt Strike.

‍

 VAMOSI: Oh, man, that’s a definite red flag. Cobalt Strike is a paid penetration testing tool that allows an attacker to deploy an agent named 'Beacon' on the victim machine. Beacon provides the attacker with information including command execution, key logging, file transfer, SOCKS proxying, privilege escalation, mimikatz, port scanning and lateral movement.  Beacon is file-less and supports C2 and staging over HTTP, HTTPS, DNS, SMB. It can also do forward and reverse TCP. So it’s a kind of swiss-amry knife of tools which penetration testers use and so do criminal hackers. Often Cobalt Strike is used for reconnaissance. 

‍

JIMENEZ: You know, it was not literal in the alert that said cobalt strike detected there was an alert about the usage of cobalt strike within the network. But what happens is that on April 11, every public employee was out on vacation. You know, so timing was perfect, and that was one of the indicators that started to give the give us some suspicious about the nature of this attack, you know, because they they waited, they knew that people was going to go out on vacation that the complete ministry was going to be left out of monitoring for around a week because this is the Holy Week that celebrated in many of the Christian countries, right.

‍

VAMOSI: Okay, this sounds familiar. If you’ve heard about the Bank of Bangladesh heist, by the Lazarus group out of North Korea. They used the weekend and the Chinese New Year to arrange for a large illegal transfer of funds hoping that most of the funds would  be transferred because there would be so few people watching the system. And that was true, to a degree; in this case, a majority of the funds were held back only because someone in Europe noticed a spelling error and stopped the international transfer of the remaining funds. Something similar appears to have happened here. The Conti Ransomware struck during Holy Week in Costa Rica, which is largely a Christian nation. So the alerts in Costa Rica weren't being tracked as closely. Perhaps in the IT world, it would be a good idea to make all holidays flagged as opportunities for bad actors and therefore considered high alert for those guarding our most critical institutions.

‍

JIMENEZ:  So on the on the eleventh, we found about the cold strike alerts and subsequent alerts referred to lateral movement, but there was nobody to attend those alerts. And it was until the 15th, 16th, and 17 that the real ransomware started to unfold, right. So typically, once you get a certain experience in this field, you know that a ransomware attack will start around a week before you know five to seven days if you know, like regular, what you will find about triggering the ransomware and starting the deciphering. But prior you'll find some indications of infiltration, you know, information being infiltrated out and then the ransomware it starts some three days you know prior to the the whole thing to be completely wiped out. Right. So that's exactly what happened. So 15,16, and 17. We started the notice in the logs, the lateral movements, the ciphering. And one of the most interesting things that we saw is that within those first or early stages of deciphering, the first, let's say infrastructure that this got a hold on was precisely that backup cert. And they knew that this was the only backup server that existed in the ministry.

‍

VAMOSI:  Okay, so this is bad. The backups for the government ministries were singular and now wiped out. 

‍

JIMENEZ: to me, it was incredible that a public institution of this importance, had only one backup system, which was not there was no redundancy whatsoever , and they knew about it, because he was the first system that they took down. So that was the other suspicious activity that we found out. That, you know, gave us some idea about maybe there was some assistance to continue in this context, you know, helping them to understand when to attack, what to attack, because after that some of the other servers that were hit was not only the backup server, but they also stopped the there was a Team Foundation Server. That was the principal server that hold the code for every application in the ministry. So this is also really, really interesting because they knew that the code was not being synced with any good or any repository on the cloud. They knew that there was a local web team foundation server, where the code for every single web application's internal systems where everything was stored, and that was the second point that they wiped out.

‍

VAMOSI:  And it wasn’t just one government ministry -- it was that and 27 others. Clearly this attack had to have some inside knowledge. And clearly, in order to have happened all at once, there was external coordination. So was it possible from the 11th to the 15th, that the criminal hackers were doing reconnaissance and they figured all this out?

‍

JIMENEZ:  Well, we believe that they actually started much earlier because we found that the logo of the Ministry of Hacienda was changed in February of this of that same year.

‍

VAMOSI: So in February of 2022, someone did a website defacement of the Ministry of Hacienda. This is a low sophistication attack, changing just the site’s logo, nothing else.

‍

JIMENEZ:  It was changed with the logo of the Finance Ministry of Russia. You know, and this was there was a there was this was really interesting too, because this happened in February and nobody noticed that the logo in Google search was changed with one from the Russian Ministry of Finance, you know, so we had some indicators that probably the reconnaissance and intelligence activities of this group started at least at the beginning of the year, maybe some surveys around, you know, December or something like that. But that was definitely a lot of Intel, moving forward, because as I said before, this attack, not only striked the minister, the Ministry of hacienda, but it also strike 27 more public institutions, you know, so it was a wave of complete, you know, destruction in in some of the most important parts of the of the government.

‍

VAMOSI: What Esteban and others did next was first to secure the government’s remaining system and get them back up and running. He also started to dive into what might have happened. And possibly who was behind it. 

‍

What we have not covered yet is that this Conti Ransomware attack happened during an important presidential election in Costa Rica and how in the midst of that, the country, weakened, on guard, and defending against online threats, had to deal with a second ransomware attack from a different organization. Through it all Estaben was leading the incident response, both learning in the moment and educating others, laying a foundation for the future.. 

‍

There’s so much more to this story, and I’ve only scratched the surface, We haven’t, for example, talked about what all this might mean for other Central American and Latin America countries as well. So I’m going to tell that part of the story in the next episode of The Hacker Mind, Episode 79. Join me again in two weeks when I conclude the story of Costa Rica’s ransomware attack. 

‍