Hunting for C3

The ability to detect and identify adversary Command and Control (C2) channels has always been one of the most important elements to a strong defensive capability. The detection of an attacker’s C2 mechanism not only allows threat hunters to pivot on C2 characteristics and behaviour to find other compromised endpoints, but it also reveals a lot about the capability and sophistication of an attacker. Indicators of Compromise (IOCs) extracted from the C2 phase of the killchain can also be shared through threat feeds and threat intelligence partnerships to gain insight into an attacker’s targets, their motivation, and to gain a better understanding of the size of an adversary’s campaign.

Given the importance of this element in the killchain, blue teams have spent considerable effort focusing on developing defensive techniques to detect C2 communication. This increase in effort has meant that adversaries using traditional C2 channels are likely to be detected in modern and mature enterprise environments, where particular effort has been spent on defensive security.

A typical blue team will rely on data from either network proxies or some form of network sensor device to enable them to query for the “low-hanging fruit” of C2 characteristics such as: newly seen or recently registered domains, unusual user-agents, regular beaconing behaviour, and domains with low or unknown reputation scores. More advanced organisations will also have the ability to link abnormal network activity back to individual endpoint processes instead of endpoints as a whole. This has bought in a whole new level of capability for C2 detection as defenders are now able to combine and augment the C2 characteristics listed above, with other suspicious indicators such as: unsigned applications making regular network connections, process injection events followed by network connections, and abnormal per-process network activity.

These developments in defensive techniques have forced more capable adversaries to use esoteric C2 channels to avoid detection. In recent years we have seen well-known Advanced Persistent Threat (APT) groups using popular web services such as Instagram [1], Photobucket [2] and Google Drive [3] as mechanisms for their C2 communication due to their good domain reputation and common presence on target networks.

The implementation of these more advanced C2 channels has meant that red teams running adversary simulations have also had to adopt sophisticated C2 mechanisms to avoid blue team detection, and to also deliver a realistic simulation of APT or near-peer APT capability. This has mostly been achieved with custom internal tooling on a per red team basis, with little to no open source framework to fill the requirement of esoteric C2 channels for red team threat emulation.

From a threat hunting perspective the release of C3 poses an interesting challenge for effective threat detection. The release of the framework lowers the bar for mainstream attackers to leverage the legitimate functionality of popular applications to disguise the communication between compromised endpoints and C2 servers. Due to this, it is likely that organisations will begin to see an uptick in the number of attackers who are utilising stealthier C2 channels such as those offered in the C3 framework.

To defend against attackers employing new and advanced techniques, the threat hunting team at Countercept analyses a huge amount of endpoint machine data in order to rapidly detect threats across the whole killchain. The coverage this data provides means that when an attacker employs new techniques at one element of the killchain (e.g. the command and control phase) we are often able to detect them at other elements and then pivot from this information to deconstruct and analyse the rest of the attack path. For advanced C2 channels - such as those offered by the C3 framework – we are more likely to detect the adversary at the delivery, exploitation, installation and actions phases of the killchain.

However, it is necessary to prepare for a situation where we either do not have access to this data (such as when a client contacts us to start an Incident Response (IR) investigation after an attacker has implanted themselves in the network) or where an attacker is utilising sophisticated Tools, Techniques and Procedures (TTPs) allowing them to undertake the initial steps of the killchain without being detected. For this reason we decided to investigate how it would be possible to detect the use of C3 without relying on historical data from the rest of the killchain.

–    “A system has been compromised and the attacker is utilising C3 to disguise C2 communication. The attacker may have also decided to inject malicious code into a legitimate process on the endpoint, and then use an esoteric C2 channel which matches the injected processes typical network behaviour”

The detection portion of this document is broken down into two sections; Injected and non-Injected. The non-injected section relates to a scenario where an attacker has executed a C3 payload on an endpoint and is beaconing out using a C3 module to disguise the C2 communication. In this example no memory injection has taken place. The second section is slightly more complicated and works from a scenario where an attacker has exploited an endpoint and then injected malicious code into a legitimate process. The attacker has then begun using an esoteric C2 channel – such as those offered by C3 – which matches the process they have injected into.

For both the injected and non-injected sections of this document, we will be working off of the scenario that the attacker is using a Slack C2 channel, meaning that all C2 communication should be disguised as slack application traffic (as provided by the “Slack” C3 module). Although we are only focusing on one module in this document, most of the underlying principles of the discussed detection techniques should be expandable to cover other C2 channels provided by C3.

Sysmon and Event Tracing for Windows (ETW) providers can be used by blue teams to log and examine the DNS requests created by processes. This is a very rich data source for threat hunting as it allows defenders to link network activity back to a specific process running on an endpoint, thus enabling the ability to pivot between different data sources.

Using this dataset we can look for quick wins by querying for any unsigned executable which is making DNS requests to the “slack.com” domain or related subdomains. In a typical environment we would only expect to see signed executables (such as chrome, slack, svchost and AV software) creating DNS connections to popular domains such as slack.com.

We can also compare the DNS requests made by a legitimate slack executable to the requests made by an implant utilising the C3 Slack module. Comparing the results reveals some interesting differences. As you can see from the image below, the C3 slack module heavily utilises the “slack.com” DNS query with little variation other than the occasional query for "files.slack.com" which is used when an attacker is trying to upload a large amount of data from a compromised endpoint.

When we compare this data to the DNS requests which are made by legitimate slack executables we can see a clear difference in the variation of queries. We observed that legitimate Slack executables typically make DNS requests for 19 different domains:

It is worth noting that this list was stripped down to only include domains owned by Slack. The full list of DNS queries includes domains such as bugsnag.com, fonts.gstatic.com and various giphy.com sub-domains, the presence and frequency of these DNS queries is obviously dependant on the user activity on Slack and the types of plugins being used.

From a detection perspective the differences in the DNS queries used by the legitimate slack executable and the C3 slack module presents a potential method of detection. It is a relatively straight forward process to monitor the DNS queries made by an executable querying “*.slack.com” and then see if the queried DNS names fit the DNS profile of the legitimate slack executable.

Detecting esoteric C2 channels becomes more interesting when we consider that an attacker may attempt to inject malicious code into a process and then utilise a C2 channel which matches the source process’s typical network behaviour. For the purpose of this research, we are using the scenario where an attacker first injects their malicious code into a legitimate Slack process, and then beacons out from that process using a C2 channel similar to the C3 Slack module. In this case - from a high level view - we would initially only see a Slack process talking to legitimate slack domains, which is obviously expected behaviour and not an event which would be suspicious to a blue team.

However, by injecting into the Slack process the attacker will undoubtedly change the behaviour of that process. We can try to identify these changes in order to detect C3 functionality being used in this stealthier manner.

Using ETW providers we can also trace network connections back to the module within a process which created the connection. By comparing the network connections created by a legitimate Slack process to the connections created by a Slack process which has been injected into by an attacker, we can start to see some interesting differences.

If we look at the network connections for a normal slack deployment over a 30 day period, we can see that the connections originate from the main executable (slack.exe):

In comparison, if we look at an example where an attacker has injected into one of the Slack executables and has been using it to beacon out to a C2 server, we can see that these connections originate from the ntdll.dll module instead of the core Slack executable:

To confirm our suspicions that this Slack process is being used by an attacker, we can look for thread injection events prior to the first instance of the ntdll.dll module creating a network connection from the Slack process. As we can see in the image below, our suspicions are confirmed as there was a thread injection event immediately followed by a series of network connections originating from the ntdll.dll module:

A key point to remember is that regardless of what process an attacker injects into, they will almost always cause that process to behave in a different manner. Working from this knowledge, we can look at other network communication based changes that may occur when an attacker is beaconing from an injected process.

The scatter graph above shows a 12 hour period encompassing a normal working day of a medium sized company where Slack is used for day-to-day business communication. The Y axis shows the number of packets that the Slack executable transmits every five minutes. For the vast majority of the hosts on the network, we can see that the Slack executable transmits between 0 to 10 packets every five minutes, with the occasional peak between 10 and 20 packets which is most likely due to file uploads and downloads.

However, the packet count for the host marked “Compromised Endpoint” on the graph can be clearly seen as having an elevated baseline count of between 15 to 20 packets transmitted every five minutes with the occasional spike of up to 25 packets. During our testing, this endpoint was where an attacker has injected into the slack.exe process and had then opened a C2 channel using the legitimate Slack domains. We can clearly see that this set of actions has caused the Slack executable to behave in an anomalous manner, with the overall network profile of the process being raised above the profile of the other Slack processes running on the network.

In addition to the anomalies and techniques discussed already, it is important to remember that the purpose of a C2 mechanism is to enable adversary connectivity and to allow an attacker to perform future actions on an endpoint. The result of these actions can also be used to facilitate detection, for example data exfiltration will almost always result in increased and extended levels of traffic flowing from an endpoint process across the perimeter of the target network.

Furthermore, poor attacker TTPs once on an endpoint can often lead to the identification of a C2 mechanism. In the image below we can see an attacker deploying a simple persistence mechanism from a C3 relay. Actions of such an obvious nature will usually trigger a detailed examination of the parent process which in this case would likely lead to the identification of the C2 mechanism.  

This example of obviously malicious attacker behaviour is a good example of when low-skilled attackers use advanced frameworks - like C3 - in an attempt to reduce their chances of being detected, but then lack the necessary skills and experience to remain undetected outside the confines of the framework.

In order for blue teams to effectively undertake threat hunting and to have the best ability to detect attackers, they require continuous access to high quality data, enabling detection across as many phases of the killchain as possible. Although we were able to find several methods to detect the presence of C3 without relying on historical data from the initial phases of the killchain, it is clear that when attackers are utilising advanced methods (such as those provided by C3 modules) to disguise their C2 traffic it is especially difficult to detect their presence. It is therefore important that blue teams ensure that they have detailed coverage at the other phases of the killchain, in order to detect and prevent attackers before they are able to secure their access in a compromised network.

Detecting more advanced attacker TTPs is made significantly easier if you can establish what ‘normal’ looks like on a network. By ensuring that defenders are able to pull data from all endpoints across a network, not only ensures suitable coverage, but also equips them to hunt for deviations from normal activity by comparing the data that they have access to.

Given the rapidly shifting nature of attacker TTPs, larger and more mature organisations that have both internal offensive and defensive teams should feed knowledge to each other to further develop their capability. MWR Labs and Countercept have a long history of working together to further their overall objectives, with the most recent example being the C3 project. The results of our threat hunting research detailed in this blog post will be injected back into the development process for C3, ensuring that the framework continues to be a valuable tool for red teams to use during adversary simulation engagements.

[1] https://www.welivesecurity.com/2017/06/06/turlas-watering-hole-campaign-updated-firefox-extension-abusing-instagram

[2] https://securingtomorrow.mcafee.com/mcafee-labs/vpnfilter-botnet-targets-networking-devices

[3] https://www.fireeye.com/blog/threat-research/2018/08/fin7-pursuing-an-enigmatic-and-evasive-global-criminal-operation.html

[4] https://labs.mwrinfosecurity.com/tools/c3