Catching Lazarus: Threat Intelligence to Real Detection Logic - Part Two

To decrease the chances of being detected, the Lazarus Group removed evidence of their activity as they moved within the victim network. For instance, they were observed using the Wevtutil tool to export Windows security event logs:

In the command snippet shown above, we can see that the Lazarus Group exported copies of two Event IDs with the Wevtutil tool. These Event IDs, 4778 and 4624, respectively log events showing when a user reconnects to a disconnected terminal (meaning that user previously had logged in to that terminal), and when a user successfully logs into a machine.

Subsequently, all 4778 and 4624 events pertaining to the compromised user accounts were removed and could not be found during the investigation of the Windows Event Logs. This made it challenging to piece together the movement of the attacker during the incident response investigation.

Detection

Although the specific technique used to clear the event logs was not identified in the TI report, blue teams can still build detection logic for known techniques. As the Lazarus group used the Wevtutil tool to export Event Logs we can first focus on how this tool can be utilized to clear event logs:

wevtutil cl <Logname>

There are two existing Sigma rules[8][9] to detect Windows Event Logs being cleared. Both rules provide suitable detection coverage, but our testing showed that they generate a high number of false positives related to legitimate administrative activities. The number of false positives can be reduced if we focus specifically on the Security logs, also referred to as Audit logs. Another Sigma rule[10] uses this approach, and looks for Event IDs 517 and 1102 (both mean that the Audit log was cleared but on different Windows versions). This rule is powerful as it detects Security logs being cleared regardless of the command used by the attacker.

However, none of the existing rules look for the Wevtutil tool being used to export Security logs, as used by the Lazarus Group. We created a Sigma rule to detect both exporting and clearing of Security logs using the Wevtutil tool, so that defenders can get insight into this activity happening on their estate. The command for clearing logs, “cl” (or “clear-log”), when it is focused on the Security logs, yields significantly fewer false positives based on historical endpoint data from F-Secure Countercept. The Sigma rule created to detect this behavior can be found on our GitHub and is named win_wevtutil_clear_export_logs. The executable “MsSense.exe” is excluded as it is used for legitimate purposes to export the Security logs by Windows Defender Advanced Threat Protection.

Our advice would be to implement all the above rules with different values for the “level” field based on the false-positive rate. As the “level” corresponds to the severity for the rule, we would recommend that rules that have a low false positive rate have a “critical” or “high” level, whereas rules that are more prone to false-positives should be “low” or “medium”.

However, as pointed out at the beginning of this section, there are many other techniques that a threat actor can use to clear Event Logs without using the Wevtutil tool. Furthermore, while we are only focusing on the clearing of Event Logs, an attacker could also disable log generation or hide from Event Tracing for Windows (ETW)[10]. ETW is the kernel-level event tracing mechanism built into the Windows operating system that handles the logging of all system events. In fact, there are so many techniques that could be used to disable or hide from ETW that it could fit in a separate blog post series. As we are focusing on generating detection from the Lazarus TI report, they fall outside the scope of this blog post.

Regardless of whether the attacker clears the logs or disables ETW, this section underlines the importance of streaming logs to a central source in real-time. With this implemented, if an attacker clears the logs on an endpoint – as the Lazarus Group did - it would only affect the logs stored locally and therefore blue teams would not lose valuable forensic information. Furthermore, streaming logs to a central source provides the detection opportunity to alert when an endpoint stops sending logs while it is still online, this provides another “tripwire” for detecting suspicious endpoint activity.

Disabling Credential Guard

The Lazarus group were observed using a custom version of Mimikatz to harvest credentials from LSASS memory. To accomplish this, they first disabled Windows Defender Credential Guard.

Older versions of Windows used to store credentials in the memory of the Local Security Authority (LSA) process. From Windows 10 onwards, credentials used by the operating system are stored in a new component called the isolated LSA, also known as the “Credential Guard”.[11] This isolated LSA process (LSAIso.exe) is protected using virtualization-based security and is not accessible to the rest of the operating system except by the LSASS process through Remote Procedure Calls (RPC). Therefore, threat actors often disable this security mechanism before attempting to extract system credentials.

By modifying the registry entry shown below, the Lazarus Group disabled Credential Guard which caused LSA to store credentials in its process memory instead:

Detection

As the command above modifies the registry entry 'HKLM\System\CurrentControlSet\Control\LSA\LsaCfgFlags', the detection will be based on the registry_event and process_creation log sources to be able to monitor any registry modification.

Using the registry_event log source, we must filter for any event of "SetValue" or "AddValue" with the value of 'DWORD (0x00000000)' in the above-mentioned registry entry. This log source is important for detection as we can monitor any changes to this registry entry, regardless of the tool used by the attacker.

That said, if there is an issue with the registry event log (let’s say an attacker clears or tampers with logs, as previously mentioned), it is still good to have something for the process creation logs. Here we could look for the Reg.exe command with the arguments ‘add HKLM\System\CurrentControlSet\Control\LSA /v LsaCfgFlags /t REG_DWORD /d 0’.

This works well, but our telemetry showed that we can expand that detection rule by restricting the command line arguments only to “LsaCfgFlags” and still have a very low false positive rate. This way it will also show an attacker deleting that registry entry (which also disables CredentialGuard) or querying it (to check if CredentialGuard is enabled).

Based on our telemetry, querying or modifying the Credential Guard registry value has a very low false positive rate. The Sigma rule created to detect this behavior can be found on our GitHub and is named win_disable_credential_guard.

Once Credential Guard has been disabled, it is significantly easier to extract credential material from LSASS’ memory. The Lazarus group took advantage of that by performing the following procedures:

Enabling storage of credentials in plaintext (WDigest)

WDigest credential caching is a legacy authentication protocol.[12] This protocol causes LSASS to store credentials in its memory in plain text and is enabled by default on Windows Server versions prior to Server 2012 R2.[13] While it is disabled by default on more recent versions of Windows, threat actors can modify the registry to enable WDigest credential caching. The Lazarus Group were observed using the following command to achieve this:

The detection for this technique is similar to the detection of Credential Guard being disabled as discussed earlier in this blog post. Any registry event or process creation log where the registry entry 'HKLM\System\CurrentControlSet\Control\SecurityProviders\WDigest\UseLogonCredential' has its "DWORD" value modified to 1 is an indicator of this technique being used. While there is an existing Sigma rule[14] which includes the detection for this behavior, it also includes detections for other techniques specific to another APT group. As such, we have written a separate rule specifically detecting the modification of the WDigest credential caching registry key. During our testing of this detection rule, we had almost no false positives except for a few instances caused by vulnerability management scripts. If this detection rule causes too many false positives in your environment, you could modify the rule to only focus on the modification of the registry entry to the value ”1”. The Sigma rule created to detect this behavior can be found on our GitHub and is named win_reg_enable_wdigest.

Using standard Mimikatz functions to dump credentials

With the previous preparation steps completed; threat actors can then use tools like Mimikatz to harvest credentials from the LSASS process memory. 

While the Lazarus Group were observed using a custom or simply a renamed version of Mimikatz, traces of the commonly used functions of Mimikatz could still be identified through the command line arguments, such as “privilege” and “sekurlsa”:

Detecting Usage of Mimikatz-like tools

Since the Lazarus Group did not obfuscate commonly used functions from the Mimikatz tool, blue teams can detect commonly used Mimikatz functions in command line arguments with an existing Sigma rule[15].

Apart from the quick win above, blue teams can also hunt for processes accessing the LSASS process. For example, running Mimikatz with the arguments SEKURLSA::LogonPasswords will cause Mimikatz to dump password data from the LSASS process memory for recently logged on accounts as well as services running under the context of user credentials[16]. This generates a process access event, which is created when a process accesses another process. This behavior usually occurs as a result of reading and writing to the memory address space of the target process[17]. In this case, the target process is Lsass.exe and credentials are read from its address space. An existing Sigma rule is available to detect this behavior[18].

Leveraging scheduled tasks for lateral movement is a very common technique. The Lazarus Group used scheduled tasks to modify the “HKLM\SYSTEM\CurrentControlSet\Control\Lsa\Security Packages” registry key on a remote machine:

Detection

Simply detecting the use of the Schtasks utility can be challenging as it is widely used by legitimate Windows applications and for administrative purposes. However, we can reduce the number of false positives by detecting the creation of a scheduled task for the purpose of launching anomalous programs.

Suspicious scheduled task creation:

The Lazarus Group's scheduled task creation can be considered anomalous behavior as Reg.exe is rarely executed by the scheduled task utility. Therefore, monitoring for the use of the Reg.exe utility in a scheduled task creation command can prove to be an effective detection strategy. Another example for anomalous programs originating from a scheduled task is encoded PowerShell. While encoded PowerShell alone is not a high-fidelity detection, we have found that hunting for scheduled tasks being created with encoded PowerShell as the “scheduled action” has a low false positive rate, often indicating persistence or lateral movement activity. The Sigma rule created to detect this behavior can be found on our GitHub and is named win_anom_schtasks_creation.

Remote scheduled task creation:

Another opportunity for detection would be the use of command line options "/S", “/U” and "/P", which are used to specify the IP address for a remote computer, the user context under which Schtasks.exe should run, and the password for a given user context respectively. While these command line options can be used legitimately to create scheduled tasks on remote machines, they are not commonly observed based on historical data from F-Secure Countercept. The Sigma rule created to detect this behavior can be found on our GitHub and is named win_remote_schtasks_creation.

Suspicious scheduled task execution:

While the creation of remote scheduled tasks is often observed on the first compromised host to move laterally to other hosts in the network, the tasks themselves will eventually execute on these hosts. Thus, it would also be valuable to build detections for suspicious processes that are executed by the Windows Task Scheduler.

This can be achieved by detecting a combination of suspicious processes or processes that are commonly used to launch suspicious scripts; examples include Cmd.exe, Powershell.exe, Cscript.exe and Reg.exe. For these processes we can filter for where the parent process is either Windows Task Scheduler Service (“svchost.exe -k netsvcs –p  -s Schedule”) or Windows Task Scheduler Engine (Taskeng.exe). We observed some difference in scheduled tasks execution in different versions of Windows. In newer versions such as Windows 10 and Windows Server 2016, scheduled tasks are executed directly from Windows Task Scheduler Service where older versions like Windows 7 and Windows Server 2012 are executed from Windows Task Scheduler Engine.

The Sigma rule created to detect this behavior can be found on our GitHub and is named win_susp_schtasks_execution. However, it is worth noting that during testing we found that without any tuning this rule will often trigger false positives, especially in environments where scheduled tasks are used to regularly run custom administrative scripts. Therefore, blue teams should identify and exclude any known administrative scripts in the detection rule before implementation in production environments.

With the credentials acquired using the techniques explained in the Credential Access section, the Lazarus Group were able to move laterally across the network. To achieve this, they made use of common remote services such as RDP and VNC.

Detection

It is challenging to write detection rules for the use of RDP and VNC as they are generally widely used within networks of all sizes for legitimate purposes. Context is crucial for such activities, and malicious activity is often only apparent by looking at a combination of user and endpoint activity across many systems on the network over hours or days, rather than focusing on a single event on a single endpoint.

To combat this detection challenge, at F-Secure Countercept we use User Behavior Analytics (UBA), a technique focused on detecting when users perform new activity that they have not performed historically.

Traditionally, blue teams’ focus has been on detecting generic "bad" behavior. As with most of the content in this detection report, it involves searching for a wide variety of offensive tactics, techniques, and procedures (TTPs) and general anomalies that relate to them. UBA differs from this in that it focuses on detecting anomalies based on user behavior, regardless of whether any offensive techniques have been used. This focus makes it a perfect complement to our traditional focus and has the potential to fill in some key detection gaps.

For example, by analyzing 4624 and 4648 logon events from Windows security Event Logs, it is possible to identify when a standard user account authenticates to multiple new (for that user) endpoints, which can be an indication of lateral movement.