Memory Tracing: Forensic Reverse Engineering
*Memory tracing* is an entirely novel reverse engineering technique, which we have developed and used over the last couple of years. In a nutshell, the technique consists of a *recorder* which records RAM snapshots of an instrumented system - running the target software to be reverse engineered - with a high frequency (up to 100 snapshots per second), and an *analysis toolchain* which processes and visualizes the recorded snapshots to reverse engineer the behavior of the target. Memory tracing allows for various novel reverse engineering techniques, and in some cases simplifies and speeds up or automates existing ones.
In the following we describe our recorder and analysis toolchain and their capabilities in more detail. We note that some concrete reversing examples are illustrated in the supporting material we have submitted (see below).
## The recorder
Our recorder is an extension of the KVM (http://www.linux-kvm.org) hypervisor. We can thus record memory traces (a series of memory snapshots) for any x86 / x64 OS that is supported by KVM, which for instance currently includes various versions of Windows as well as Linux systems. The snapshots being recorded contain the entire physical memory, including memory mapped I/O. A memory trace thus contains a wealth of information on the system being monitored, i.e., we get information on all memory resident processes, the OS kernel state, virtual memory, code, data etc. and the evolution of these entities over time (the recording period).
A crucial aspect of the recorder is what we call *triggering*, i.e., the events in the guest OS being monitored that trigger a memory snapshot. We currently support two triggering mechanisms. The *time based trigger* fires at periodic time intervals, whose duration is configurable. While this is kind of an obvious approach to triggering, it is not the most effective one. We rather would like to trigger snapshots when, loosely speaking, interesting things happen in the system. To this end we have implemented a *system call trigger*, which triggers snapshots on entry and / or exit of system calls. The selection of system calls as well as the entry or exit triggering are fully configurable in our system.
Let's briefly illustrate the benefits of system call triggers. For instance, in the analysis of self modifying code, code injections etc., where typically memory regions are allocated and written, we would set triggers on NtAllocateVirtualMemory, NtWriteVirtualMemory and NtProtectVirtualMemory system calls to have the relevant modifications recorded. More generally, we typically trigger system calls related to memory modifications, processes, disk and network I/O.
While we have not yet systematically measured the performance overhead incurred by the recorder, we can say from our working experience that it is quite moderate. In fact, one can work interactively with the system being recorded with no to very little lag.
Another advantage for placing the recording component within the KVM hypervisor is "stealthiness", i.e., it is rather difficult to detect and thus evade. While we are of course aware that timing attacks and possibly other advanced techniques can detect its presence, it seems that such detections are rare in the wild today.
Last but not least we note that a stable and performant implementation of the recorder was at times technically challenging, so we can not only talk about the recorders functionality, but also about some interesting implementation details.
## Analysis toolchain
In a typical analysis we run the sample for a couple of minutes to produce a memory trace (e.g,. we get 2057 snapshots in a 1 min 3 sec run). As mentioned above, this results in a wealth of information on the system behavior. While intuition suggests that this is pretty cool and certainly useful somehow, at second sight it is not clear how to foster this wealth of information for reverse engineering in a systematic way.
We have thus in fact invested considerable effort in the conception and implementation of an analysis toolbox that offers various code analysis / reversing functionalities, and we are still constantly improving and extending our toolbox.
Technically, our toolbox consists of a Python API to access memory traces and various analysis scripts that support the reversing process, and an interactive visualization component that displays memory traces and offers some visual analysis capabilities such as the detection of memory modifications (e.g., API hooks), code injections etc., color coding of memory, etc.
The main goal and strength of our tool is that it essentially automatically generates an overview of the behavior, actions, relevant code parts etc. e.g. of a malware sample. This information alone in some cases can be sufficient to get a *first grasp of malware behavior*. But probably more interestingly this information will *guide the experienced reverse engineer* to the snapshots, processes, code regions, etc. which contain interesting behavior and require further investigation using traditional “manual” reversing techniques like disassembly etc. The fact that we are working with memory snapshots allows the use of the standard toolchain of a reverse engineer such as IDA Pro (by exporting PE files, code etc. from memory snapshots), Volatility, WinDBG etc. We believe that this is another strength of our approach, since many reverse engineers are reluctant to give up their well proven tools.
Our current analysis tools cover the following functionalities:
- Detection of memory modifications: Detection of hooking (EAT/IAT/inline), memory injections and in fact essentially any code modification with a very simple but reliable (very little false positives) detection algorithm, which does not need to resort detection heuristics.
- Self modifying code: By simply analyzing how memory regions containing code evolve over time, we reliably find self modifying code, packing, etc. Since we have the memory snapshots, we can also easily extract the unpacked code.
- Detection of crypto usage: By using a similar approach as for self modifying code, we look for the temporal behavior and modification of regions of high entropy. Using this technique we can automatically detect crypto activity and crypto buffers (with some false positive rate). This may guide a reverser to crypto functionality.
- Pattern matching: Fuzzy search of binary data across all snapshots. This is e.g., quite useful for searching malicious code in different virtual address spaces, and figuring out its origin, or to identify infected processes etc.
- Visual analysis: By visualizing memory traces some analysis insights can be essentially read off from our graphs. These are for instance:
- Library loading “long” after process start
- Detection of unknown modules
- Unpacking
- Memory injections
- Etc.
- Traditional sandbox features: From memory traces we can extract infer interactions with the system much like in “traditional” sandboxes. Information we can extract includes:
- Process creation / exiting
- Thread injections
- Registry modifications
- Services modifications (start, stop, autostart)
- File usage
- Network communication
Finally, we believe that memory tracing adds a forensic quality to dynamic analysis. In fact, unlike classical sandboxes and similar dynamic techniques where the analyst only gets relatively high level information, memory traces yield a substantially more *complete and deep recording of system behavior at the time of capture*. This allows a reverser any time to go back to a memory trace and perform essentially any follow up analysis at will, without the need to relaunch the sample in a debugger or similar tools. This is particularly interesting for samples that use anti-* techniques, and only feature interesting behavior under some circumstances, e.g., during some time windows, upon first execution etc.
## Demos
Since we have used our memory tracing tools ourselves quite a lot in the last couple of months, we will be able to illustrate our talk with various real world examples. So far we have mainly used our tools for reversing malware, but we were also able to break packers.
Finally, we'd like to stress that despite our university background the system is completely practical and not just yet another academic proof of concept prototype. ;-) The only non-practical aspect is that some analysis tools, e.g., finding self modifying code, require quite some preprocessing time and are thus not interactive.
Speakers
 |
Endre Bangerter |
|
Dominic Fischer |