Even before COVID-19 drove homebound digital forensics examiners to seek out new modes of training and skill-building, Capture the Flag (CTF) challenges were a popular feature at numerous conferences. Now offered in virtual formats both during and outside of conferences, CTFs continue to attract participants. What can you expect from the experience, and what do you need to participate? Bring your own box (BYOB) and find out!
What is “Capture the Flag”?
CTFs take their name from a live-action game, Capture the Flag, defined on Wikipedia as “a traditional outdoor game where two teams each have a flag (or other marker) and the objective is to capture the other team’s flag, located at the team’s “base,” and bring it safely back to their own base.”
In a digital forensics or cybersecurity context, a CTF competition comprises a set of puzzles to solve. In a session delivered at the virtual National Cyber Crime Conference in July, Chris Atha, a High Tech Crime Specialist at the National White Collar Crime Center (NW3C), said the “flag” in each puzzle is “one distinct correct answer with several different ways to get there, [and] several steps to derive the answer.” For example:
- Competitors might have to identify a string of text within an ambiguous file type, recover a password using a cryptographic scheme, or use encryption keys to find out what type of encryption a file is using.
- The puzzle might involve a single file, or a full computer image.
- A CTF like FireEye’s FlareOn challenge, which focuses more on malware reverse engineering, might ask competitors to deobfuscate pieces of code to find the flag.
- Some CTFs can be fun, said Atha, including “Easter eggs” such as finding a four leaf clover for St. Patrick’s Day.
The Diana Initiative offers another alternative: a Jeopardy-style CTF featuring “challenges ranging from forensics to open source intelligence (OSINT) to hack-the-box to malware analysis to hacker trivia,” according to its website.
“When I build a CTF I like to include a wide variety of challenges, from forensics to trivia to hacker methodology,” said Marcelle Lee, a security researcher and educator who also serves as Chief of Competition Programming at the Women’s Society of Cyberjutsu. “The thought process behind this is to expose participants to a variety of disciplines within the field of cybersecurity. I also want my competitions to be attainable, so that everyone can get at least a few answers correct. It is very discouraging to not score any points, especially if it is your first time competing!”
Lee pointed to her own experience: “I analyzed my first PCAP file in a cyber competition,” she said. “At the time I had no idea what a packet capture was but I dug into it and fell in love.” Now she’s considered an expert in network traffic analysis.
Santiago Ayala, director and founding partner of Florida, U.S.-based ATX Forensics, said his favorite CTFs rely on a “case-like” structure. “The reason is, as I start the game (investigation) I can foresee issues based on the knowledge of artifacts,” he explained.
Ayala first competed in a SANS NetWars held at the Computer Enterprise Investigation Conference (CEIC) in 2013. Initially hesitant — “I was thinking that there was too much of a risk to expose your knowledge like that in an environment where you could see opposing experts/analysts everywhere” — Ayala and colleague Stacey Randolph decided to give it a try. Pitted against well-known experts in the field, Ayala “just focussed on answering the questions without looking at the scoreboard.” He ended up tied in second place.
Since then, he’s played in four NetWars tournaments, including two online; the SANS Holiday Hack Challenge; and Magnet Forensics’ 2018 2019, and 2020 competitions. He won the CEIC NetWars match in 2016.
The majority of CTFs are time-limited to a few hours, a day, week, or the duration of a conference. That can make it challenging to compete at the same time as working and/or attending talks. “[G]iven the choice I will always go for the CTF,” said Lee. “Talks are typically recorded, so I know I can go back and watch them later.”
Trace Labs’ OSINT Search Party can take place at events, but is also run monthly on Saturdays. For its part, the NW3C began to offer daily virtual CTF challenges in April, offering prizes from commercial sponsors.
Why are CTFs important?
In his presentation, Atha said that whereas a tool certification reflects that the holder learned what a vendor believes is needed to use their tool, CTF participation — which is not generally reliant on commercial tools — demonstrates an effort to learn a certain skill: to reverse engineer to a point, for example.
From there, a prospective employer or other interested party can look at the CTF’s statistics and what others have written to validate that training. “It’s a directed path to learn a new skillset commensurate with technology that is on the leading edge,” Atha explained.
CTFs are a good way to “force people out of their comfort zone,” said Atha, broadening their skillset in a low-stakes environment like a game. “Most CTFs are looking for out of the box thinking,” he added — one reason why some organizations use CTFs as part of the hiring process.
That can be important for participants from underrepresented backgrounds, who Lee said often have to work harder for recognition. “Whether it is through certification, degrees, or extracurricular activities like CTFs we have many more boxes to check before we can enter the cybersecurity playing field on a somewhat even level,” she explained.
Hiring isn’t the only practical application for a CTF. Trace Labs’ OSINT Global Search Party CTF is a gamified effort to find real-life missing persons. Leads that contestants identify are passed along to law enforcement working the cases — an outcome that can win the participants additional points.
Kyle Naish, a detective with the Springdale (Arkansas, USA) Police Department, has both participated in and judged Search Party CTFs. He added that a CTF like this can help digital forensics examiners learn a complementary skillset — how OSINT data can contextualize the information found on the device.
“[OSINT and digital forensics are on the] same spectrum,” Naish explained. “OSINT is our own unique thing, but partnership is critical because [the disciplines] feed off each other.” That’s important because digital forensics alone may provide only a fraction of the answers needed to solve a case.
For example, OSINT can help to identify whether a device owner has more than one social media or other account that doesn’t appear on the seized device — or even identify a totally different device. Naish said one CTF awarded points to a team that identified an iPhone from a “mirror selfie” posted to social media.
Ayala has used CTFs to improve his network forensic skills. “I learn on every CTF I play,” he said. “The other cool thing about playing them is that you most of the time get presented with a new or recently discovered artefact that you have not seen before on your day to day work.”
Learning new skillsets is challenging, of course, but Atha said that’s where CTF proctors can help. Questions start without a hint, and the proctors keep an eye on how well participants do. If many people seem to be struggling with a particular question, proctors give them a hint.
Ayala said sometimes the CTF structure itself can be challenging. “[S]o far, the most difficult CTF I have played was the Magnet 2020. I had a hard time connecting the dots between challenges. I could not find a structure or a case-like theme to it. That said, it was very fun to play it and I think I did okay.”
CTFs help to build soft skills, too, especially when played as teams. The Diana Initiative is managing this virtually, said Lee, by hosting its CTF within its own Hopin session. There, attendees can break out into additional private sessions for teaming.
What skills and tools do you need?
Atha said any good CTF starts with “foundational statements” such as security best practices and sound forensic science. The NW3C’s CTF asks participants to confirm their results with a degree of certainty, for instance, whether an Android image has malware.
Because the CTF isn’t meant to be an in-depth investigation, though, Atha cautioned against overthinking. “It isn’t necessarily analysis of the malware,” he said, “just [showing] whether it’s present. You don’t need to know, say, the purpose of an APK file or localhost.”
This kind of extra step can help participants think in terms of preparing for trial testimony and the real-world outcome in which their work may assist in taking someone’s liberty. Likewise the Global Search Party CTF, which Naish says can be an “emotional rollercoaster” for participants.
“The stakes are higher, so an investigator can feel drained or burned out,” he explained. “Volunteers need to know how to breathe and acknowledge that they did everything they know how to do to find someone” — without comparing their efforts to others’ because ultimately, anyone’s find could be valuable. “We just want to give families the knowledge of whether [their loved one is] okay,” Naish said.
After laying these foundations, a good CTF focuses on methodology rather than tools. This way, competitors can solve the puzzles even if they don’t have access to premium commercial — or law enforcement only — tools. (Besides, many tools’ end user license agreements limit their use.)
Naish said not having tools to fall back on forces CTF participants to adapt their methods; Atha said it can also be a matter of fairness. For instance, automated analysis is inhibited, and methods like brute forcing are generally not necessary when it comes to encryption challenges.
For environments, Atha recommended one of two operating system (OS) configurations:
- A virtual machine that you can operate “with less fear of messing up,” Atha said. “[With a CTF like FireEye’s] FlareOn, where you’re reversing malware, you don’t want to do this in an operational environment.” In general, you shouldn’t compete in a CTF on a full stock forensic machine actively working cases (nor internet unless proper steps – legal authority to analyze doesn’t mean recklessness with someone else’s data/rprivacy)
- A Linux box running Kali Linux or one of the other flavors; Ubuntu, Debian, Arch, or Fedora, though Lee recommends Kali Linux. “[I]t comes equipped with many tools you might need in a CTF, i.e. Wireshark, Autopsy, nmap,” she explained. The open-source OS has a large user base and is already configured as a VM.
If the OS is a chassis, Atha said, then it becomes possible to “bolt parts on” in building out a toolkit. For a CTF like NW3C’s or the Diana Initiative’s, common tools include an image hashing algorithm, an archive utility to work with compressed files, a text editor (Atha recommends NotePad++), a multimedia viewing tool like VLC, and a SQLite database viewer. A tool like CyberChef can help CTF participants to understand code, while the NSA’s Ghidra helps with reverse engineering. A sandbox may be necessary to protect a system or virtual machine from malware. OSINT CTF tools — publicly facing social media sites, people-finder websites, and public records among other resources — are already available.
Part of the challenge with toolboxes is that each puzzle may require a different set of tools. “You can never have all the tools,” Atha said. Naish concurred: “Tools aren’t a one stop shop. [You need to] keep all of them sharpened because each serves a specific purpose, and you never know which one will be useful.”
So, said Atha, while you might be able to obtain a fairly well built out toolkit up front, another option is just to jump in and figure out what you need next as you go. “Don’t get overwhelmed by tool overload,” he advised. “Pick something and learn how to use what you pick. If it’s efficient and it works for you, become a master of it. Work through its capabilities and flaws.” Just like in sports, he added, athletes are limited only by their skill and fitness — not their equipment.
Most CTFs, said Atha, allow for unlimited tries, though some limit the number of wrong answers you can get. Googling for information is accepted, as is collaboration, though Atha cautions that with NW3C’s CTF, only one competitor at a time will be able to get credit. The Diana Initiative sets up its CTF to allow both individuals and teams to play.
Part of the learning process is some type of after-action review. At the National Cybercrime Conference, this consisted of a 50-minute session to walk through all 9 challenges, encouraging participants to share the method(s) they used to solve the puzzles. “For me, this is where CTFs have some of the best and most impactful learning,” Atha said. “There could be 8 or 9 or 10 different ways to solve one problem.”
For first-time competitors, Ayala has the following advice:
- “Make sure you rest before the CTF.
- “Make sure you have some type of protein bar or healthy food before you start.
- “Play for yourself and against nobody. it does not matter if you win or not, you will have fun and you will be learning.
- “Play until the end, don’t give up.
- “If you are doing Netwars, enjoy the beer and music!
- “Perhaps on the Magnet ones, enjoy the awesome commentary.”
Naish said a CTF like Trace Labs’ is all about teamwork, bringing together people with different skills and talents to get more done together. That way, when other investigators need help, they can pursue the answer “not for, but with,” he said.
“I encourage people to compete!” said Lee. “It is not only fun (and a bit addictive) but you will learn new things, build experience, and meet like-minded individuals. Don’t think that you don’t have something to contribute, because everyone does!”
Looking for your next challenge, or want to share your experiences? Talk about it in the Forensic Focus forums!
Christa Miller is a Content Manager at Forensic Focus. She specializes in writing about technology and criminal justice, with particular interest in issues related to digital evidence and cyber law.
