Hello everyone,
My name is Tobias Scharnowski (@ScepticCtf). I’m a student employee at VMRay and a member of FluxFingers, the official Capture the Flag (CTF) team at Ruhr University Bochum (RUB), supported by VMRay and also part of the German team, Sauercloud. This August, my FluxFingers teammates and I traveled to Las Vegas, NV to participate in this year’s DEF CON CTF. Our team ranked fifth out of 24 teams at the end of the competition, which is a pretty awesome outcome.
As you may know already, DEF CON CTF is one of the original and most competitive cybersecurity competitions. In this blog post, I will shed some light on how to qualify for this event, what is actually going on in this competition, what kinds of thoughts go into it and what preparation is necessary in order to be successful.
What is a CTF competition?
The first question that naturally arises when talking about the topic is: what is a CTF? As the name implies, the goal is for participants to “capture” flags of some sort. Most of the time a flag is a textual string of a pre-defined format, e.g., “flag{this_is_a_flag}”. The way a flag can be “captured” varies from challenge to challenge, which I will cover below.
In Jeopardy-style CTFs, the organizers present the teams with a set of challenges in several main categories:
- Web security
- Cryptography
- Binary software exploitation
- Reverse engineering
Participants have to penetrate websites, break cryptosystems, find vulnerabilities in software components or recover the behavior of an unknown piece of software to obtain flags. The goal is simple: solve as many challenges as possible in the given time. The number of points awarded for one flag is typically adjusted as more and more teams solve a given problem. This means that solving a difficult task can boost your team’s score quite considerably.
The qualification system for the DEF CON CTF is based primarily on Jeopardy-style CTFs. Teams can also qualify by winning one of the bigger community events during the year or winning the previous year’s finals. We qualified for the finals by placing 4th out of 586 teams in the CTF Qualifier.
This brings us to the second CTF format, what the DEF CON Finals categorize as an Attack & Defense CTF. This format adds another dimension to the competition, because here teams additionally have to penetrate the infrastructure of their opponents. Starting off, each team is provided with a set of vulnerable services. During the game, these services are holding flags and by exploiting them, the teams can steal them from their opponents. Every few minutes (every tick), the organizer infrastructure checks whether teams were able to submit such stolen flags to a dedicated submission interface. After that, fresh flags are placed into each team’s service again. Points are awarded for three different reasons: capturing an opposing flag, defending one’s own services, and keeping services fully functional.
Depending on the implementation, placing fresh flags and checking service functionality can go hand in hand. In this case a checker interacts with a service in a predefined way, leaving behind the new flag. If the bot encounters an error while using the service, it is considered to be non-functional (sometimes this state is dubbed ‘mumbling’).
In order to defend a service, teams need to identify the vulnerability and patch it, while not breaking its functionality at the same time. Keep in mind that in most cases (and this has historically been true for the DEF CON services), no source code is available for the participants. This means that all analysis and patching needs to be performed on binary level.
Preparing for the DEF CON CTF
This year’s DEF CON CTF organizers, Order of the Overflow, decided to mix things up. Instead of giving the teams full access to the virtual machines that host the services, the organizers provided them with an interface to submit patched versions of the their binaries. Upon submission of a patch, functionality testing was automatically performed on the modified version. Only if all functional tests were passed, the new file was deployed. This way, the organizers ensured that only fully functional services are running for every team and that functionality breaking patches never made it into production.
Additional context heavily influences the way an Attack & Defense CTF plays out as I’ll show with two examples.
Firstly, an important aspect of the game is deciding whether the versions of service binaries currently running on each team’s infrastructure are accessible by the other teams. Giving access to those patches potentially answers the following questions:
- Where is a vulnerability located?
- How to patch it?
- Does the patch the other team applied fully fix a vulnerability or merely make it harder to exploit?
Can we just copy another team’s service version?
Taking this into account significantly increases the amount of preparation necessary for the CTF. A well-prepared solution ensures that other teams cannot simply use your patched service version and “steal” your work. In practice, there are different ways to prevent this. For example a binary patching framework can be used to obfuscate binaries, or hidden and hard-to-remove backdoors can be integrated in the patched versions. Obfuscation will make it much harder for other teams to figure out what actually was patched and whether the patches leave additional holes. Inserting backdoors keeps opposing teams from just copying the patches to their own binaries. Keep in mind that services from one CTF may use all different kinds of architectures. Hence, obfuscation frameworks and backdoors need to be implemented wholesale
Secondly, traffic capture is probably the next-most important aspect of the game. If and when the organizers hand out traffic capture has a significant impact. Similar to the information revealed by making patched services public, handing out traffic captures can answer another set of questions:
- What part of the functionality contains security issues?
- In what format do flags hide in a service?
- How does a particular exploit work?
- Can we just replay the exploit?
Ideally, the exploits that one throws against other teams should be as hard to analyze and replay as possible. In this case, preparing generic obfuscations may be hard as the types of vulnerabilities that teams will have to exploit are hard to predict. An obvious approach is to drown the opponents in a sea of requests that makes it hard to find the real exploits in a traffic dump.
Another approach is to mess with the tooling of other teams. This notoriously led to bugs in Wireshark dissectors being exploited by crafted packets spread over the DEF CON CTF Finals network over the years. Building traffic analysis tooling that is independent of Wireshark and may answer the questions above is another area of preparation that comes in handy during a CTF.
Aside from building technology that automates tasks in a CTF, theoretical questions arise depending on the game conditions:
- Do we actually want to upload a patch for a service that no other team successfully exploited so far?
- Are there certain teams we do not want to expose our newly created exploit to?
- Can we automate the process of adapting the decisions as the CTF progresses?
For continued reading, check out the paper How Shall We Play a Game? that looks at the topic from a more systematic perspective.
DEF CON CTF Finals 2018 Format
Now let’s look at the format for this year’s DEF CON CTF Finals. Firstly, no specifics were publicized to the teams prior to the start of the CTF. As it turned out, the organizers decided to not share any patches or upload traffic dumps before a service was already exploited heavily. In addition, patches could only contain a small number of modified bytes. This meant that service reimplementation, patch obfuscation, and backdooring was out of the question. Teams could also not use automated systems to replay exploits.
Instead, teams had to find and patch vulnerabilities on their own and were able to analyze other teams’ exploits only way after the first team started using them. This way, the Order of the Overflow returned back to the basics of what I think an Attack & Defense CTF is about: Finding, exploiting, and patching vulnerabilities. In addition, teams with an established tool base lost parts of their advantage, giving way for teams new to the format to perform. Overall, I really liked the general path the CTF headed this year.
Having touched on some of the main points that go into preparing and making decisions during the CTF, other aspects can easily fly under a team’s radar. With teams growing in size and professionalism, team management related issues like parallelization of work, combining the correct skillsets between team members, and coordinating offense and defense come into play. For teams playing from different time zones, communication between the different groups can be an issue as well. I’ve heard even the issue of food orders to a hotel for 15 in-the-zone people can be quite a challenging endeavor if it was not organized up front.
As one can see, there are quite some considerations to take into account when preparing for and playing a CTF, especially of the scale of the DEF CON CTF Finals. We certainly learned a lot, enjoyed the ride and look forward to next year. A big shout out goes to all the incredibly talented participants of the finals and the organizers as well. See you all next year!
If you would like to get started playing CTFs, come to one of our weekly meetings at RUB. Find us on IRC and check out our intro challenges at https://rookies.fluxfingers.net/.
You may also follow FluxFingers on Twitter or visit https://fluxfingers.net/
