chelmzy.tech

Thoughts and Experiences

DOE CDC 2018 Writeup

Intro:

Back in April I participated in the US Department of Energy's cyber defense competition. The competition is strictly collegiate and is held once a year. This is the third year for the competition and it was bigger than ever. Twenty-five colleges from around the nation sent undergrad and grad students to one of three DOE labs: Argonne, Oak Ridge, and Pacific Northwest. Each team was tasked with defending various systems from a group of attackers that was composed of lab employees, industry, and government workers. I'm going to write a bit about my experience and provide some tips for future contenders.

Team:

I represented the University of South Alabama as the team captain. Our team composition was one senior(IS), a junior(IT), sophmore(CS), and two freshmen(CS). The staff informed us that we had the only two freshmen in the entire competition, making us the youngest team in attendance. None of us had attended a competition of this nature and some of the younger team members had never touched the command line. I have never really been in a position of leadership/teaching so this was a wonderful learning experience for me as well.

Rules:

The scenario was typical for these types of competitions. Our team was hired by Natural Gas Demand Corporation to help secure their infrastructure. The scoring was roughly broken down into uptime, defense, documentation, and customer support. Differing from most other competitions, we were given access to our environment a month in advance to secure our systems and setup the necessary services.

The scored services were as follows:

  • LDAP
  • DNS
  • NTP
  • Modbus
  • HTTP
  • FTP
  • SSH
  • POP3/SMTP

Of all the services scored, we were initially only given HTTP, FTP, and Modbus. We were responsible for implementing and maintaining the rest of the scored services. This is why we were given access to our environment a month in advance.

Environment:

The environment was almost completely in Azure except the industrial control system which was hosted on a Raspberry Pi. We were given access to the Azure dashboard and access to an Openvpn server that allowed us to connect to both the Azure VMs and the Raspi.

We were initially given the following VMs:

  • CentOS 7 - Wordpress
  • Ubuntu 16.04 - FTP/MySQL (MySQL database for the Wordpress site)
  • Ubuntu 16.04 - TightVNC/Cybatiworks (ICS education software)

Of course these systems were riddled with backdoors, vulnerabilties, and misconfigurations. The most notable backdoor was the robots.txt file on the Wordpress site. Instead of fending off law abiding robots like it normally does, this file was actually a backdoor called “c100 shell”. It allowed anyone to run commands as the apache user from a convenient web interface.

We setup the following VMs:

  • CentOS 7 - FreeIPA
  • Ubuntu 16.04 - osTicket
  • Ubuntu 16.04 - Postfix/Dovecot
  • Ubuntu 14.04 - Security Onion

All machines were joined to the FreeIPA domain. We setup the Wordpress site and osTicket to do LDAP authentication for users added in FreeIPA. The Cybatiworks VM acted as a human machine interface to the Raspi PLC. The users in the scenario would connect to the VNC server and interact with the program to control the PLC. The Wordpress site also had a page that would pull info directly from the Raspi PLC and display it to the users. Any changes we made to user documentation would be accessible on the FTP server. The Postfix/Dovecot server was running but we never actually added the users in since there was no use for it. All support was handled through osTicket and a live chat plugin on the Wordpress site. As for the Security Onion instance, we never got around to feeding all of the log sources to it. Given more time and manpower we would have forwarded events from syslog, OSSEC, etc.

Results:

We placed 2nd at the Oak Ridge location and 9th in the nation. I'm very proud to say that we finished the competition with ZERO intrusions. Our defensive score was the second highest at the competition. The largest chunk of points we missed were from the documentation scores. Quoting some of the judges “Docs will BURN you in this competition!", and they certainly did. We scored middle of the road in the documentation category and had we scored higher we could easily be in the top 5. The scoring was very fair overall but I will say that the judges should be more clear on how detailed they want the network diagram. The network diagram example they gave us was not very detailed so we thought we could get away by doing the bare minimal. This was not the case as the judges were looking for enterprise tier network diagrams. The usability and user support scoring was extremely low for all teams in attendance but we scored pretty well in this category.

Advice:

Start working on you documentation day one. Docs WILL make or break you in this competition. Be as verbose as possible on all aspects, you can never have too much information. I would also recommend getting a few people to look over your user docs and run through user support scenarios. Users will always find a way to break the system! I personally believe that sticking with an all Linux environment was beneficial. A few teams used Active Directory as their domain controller and I think it caused more problems than it solved.