Understanding Rootkits: Using Memory Dump Analysis for Rootkit Detection

Dmitry Korolev, Yuri Gubanov, Oleg Afonin research@belkasoft.com © Belkasoft Research, 2013

Every rootkit employs a wide range of masquerading techniques to prevent its detection. Anti-virus and anti-malware tools must perform what is called, in forensic terms, “live box analysis”, performing a real-time scan of a live system. No wonder rootkits can actively resist detection by either hiding themselves or messing with anti-virus software or the system kernel. This constant battle makes rootkit detection not only difficult and unreliable, but disruptive and potentially dangerous to system stability and the integrity of user data.

This paper proposes going away from live box analysis approach, and analyzing raw memory dumps offline. Authors propose a new rootkit analysis methodology based on using Windows built-in debugger, WinDbg, to analyze snapshots of the computer’s volatile memory. This paper proposes comprehensive classification of rootkits and their masquerading techniques, and demonstrates which types of rootkits can be detected with proposed analysis methodology. Some of the described algorithms can be quickly implemented by using WinDbg’s built-in scripting language. Finally, the proposed methodology is tested in real-world to detect existing rootkits.

Keywords: memory dump analysis, malware detection, malware debugging, offline analysis, system call interception, rootkit detection, system hook detection

1. Introduction

The term “rootkit” is known for more than 15 years [1]. Originally, the term “rootkit” referred to a group of small, handy tools allowing developers keep “root” (“administrator”, in Windows terms) access to the functions of the operating system. “Rootkits: Subverting the Windows Kernel” [2]defines rootkits as programs ensuring persistent, robust and undetectable presence in the computer.

Essentially, rootkits are merely tools using certain techniques to bypass system protection mechanisms and algorithms to masquerade their very presence in the system. As pretty much any technology, rootkits can be used for malicious and legitimate purposes. As much as having sustained associations with malware, rootkits were still used (although in a quite controversial manner) by legitimate companies. For example, SONY used to embed rootkit-based technology into several audio CD titles in an attempt to prevent unauthorized duplication. As it turned out, using rootkits was not the best idea after all, and following several lawsuits the company was forced to recall affected CD titles [3].

Get The Latest DFIR News

Join the Forensic Focus newsletter for the best DFIR articles in your inbox every month.

Unsubscribe any time. We respect your privacy - read our privacy policy.

Interestingly, rootkits can still be used for legitimate purposes. For example, many anti-debugging protection schemes are using rootkit-like technologies. Most anti-rootkit programs are also using rootkit-like technologies.

At the same time, most rootkits are used in combination with viruses, Trojans and computer worms for explicitly malicious purposes. Therefore, antivirus developers created a variety of tools for discovering rootkits. There are multiple types of anti-rootkit tools based on several principles.

  • Firewalls [4] filtering network traffic and detecting suspicious network activities;
  • Intrusion Detection Systems (IDS) [5] locating and removing rootkits already installed;
  • Intrusion Prevention Systems (IPS) [6] identifying and neutralizing rootkits before they can be installed into the system.

Let’s have a look at certain rootkit detection techniques based on memory dump analysis

2. Material and Methods

This paper proposes a method of discovering the presence of a hidden rootkit by performing an offline analysis of raw memory dumps captured on the suspected computer. Authors propose methodology based on using Windows built-in debugger, WinDbg, to perform the analysis. Repeated actions are implemented in the form of WinDbg scripts by using the debugger’s built-in scripting language.

2.1. Memory Dump Analysis

Memory dumps contain static snapshots of the computer’s volatile memory (RAM). It is possible to create a memory dump for a single process, system kernel or the entire system. By analyzing memory dumps, examiners can ensure clean working environment and no active resistance from the rootkit. Techniques used in memory dump analysis can be also deployed on a live system, with restrictions.

2.2. Rootkit Detection Techniques

There are several techniques available to detect rootkits.

2.2.1. Signature-based Analysis

This method uses characteristic signatures (fixed byte sequences) from known rootkits. Most anti-virus tools use signatures extracted from rootkit bodies (e.g. file or network packet). In addition, there are heuristic and behavioral patterns based on certain activities typical for rootkits (e.g. an aggressive use of a certain network port on the computer).

2.2.2. Detecting Interceptions

Windows uses pointer tables to redirect system requests e.g. IDT (Interrupt Descriptor Table), SSDT (System Service Descriptor Table), IAT (Import Address Table).Rootkits can either replace or modify these tables to specify its own handlers for certain interrupts. As a result, requests for certain OS functions will always be handed to the rootkit, allowing the rootkit to detect and counter detection efforts.

2.2.3. Comparing Data from Different Sources

In order to masquerade their presence, rootkits may alter certain data returned to the examiner. For example, by substituting high-level API’s with their own handlers. Therefore, it is possible to detect their presence by comparing results returned with high-level and low-level system calls.

Another variation of this method is based on comparing process memory loaded into the computer’s volatile memory with the content of the file stored on the hard disk (tools such as RootkitRevealer [7] are using this approach).

This method helps identifying hidden resources, but does not identify what exactly was intercepted.

2.2.4. Integrity Check

A digital signature (e.g. with a cryptographic hash function) is calculated for every system library at the very beginning (when the system is clean). After that, libraries can be checked for code alteration. Tools such as Tripwire [8] are using this method.

2.3. Classification of rootkits

In Windows, rootkits work by modifying data and executable code in the computer’s volatile memory (RAM) [2]. From this point of view there are two groups of rootkits [2]:

  • Rootkits that modify execution path;
  • Rootkits performing direct kernel object manipulation.

Another classification deals with memory areas affected by rootkits.

2.3.1. User-mode rootkits

These rootkits execute their code in the lease privileged user mode (“Ring 3” [9]). These rootkits use program extensions and plugins (e.g. Windows Explorer or Internet Explorer extensions). They can intercept interrupts or use a debugger , exploit security weaknesses and use function hooking of widely used API in data sets of every process’.

2.3.2. Kernel-mode rootkits

These rootkits are running in the system’s most privileged kernel mode (“Ring 0” [9]). They can embed into device drivers, directly modify kernel objects (DKOM), and affect the interaction between user and kernel modes. The following figure displays a kernel-mode rootkit.

2.3.3. Boot Loaders

This type of rootkits modifies the master boot record (MBR).

2.3.4. Hypervisor Mode

Some recent CPUs offer an additional operation mode known as the Hypervisor mode. Hypervisor is a small kernel that controls resource distribution between multiple operating systems. It works one level lower than the operating systems (see image below). In x86 terms, this mode is referred to as “Ring -1”. At this level, rootkits can intercept hardware requests of the host operating system. At this time, there are no known rootkits implementing this approach in the wild, only some demonstration versions exist [10].

2.3.5. Firmware

These rootkits modify firmware of certain devices such as network cards, hard drives or computer’s BIOS (see image above).

2.4. Memory Dumps

There are four types of memory dumps:

  • Crash dumps;
  • Raw dumps;
  • hiberfil.sys;
  • Vmem.

Crash dumps are files created when the computer crashes. The entire content of system memory is saved into a file.

Raw dumps are complete snapshots of the operating system memory.

Hiberfil.sys is a file used by the operating system for enabling hibernation. When the operating system is hibernated, the content of the computer’s volatile memory is stored into this file. The size of this file will equal the size of the computer’s RAM. Note that memory-mapped areas that are stored on the hard drive are not saved into hiberfil.sys.

Vmem is a file created by VMware [11]. This file contains the entire volatile memory of a virtual machine (including page file content).

2.5. WinDbg

WinDbg is a multi-functional debugging tool provided by Microsoft with Microsoft Windows. It can be used to debug applications, drivers, or the operating system itself in kernel mode. WinDbg offers a convenient graphical user interface.

WinDbg works exclusively with crash dumps. Every other type of memory dump can be converted into that format with Moonsols Windows Memory Toolkit Community Edition [12].

3. Using WinDbg to Detect Rootkits in Memory Dumps

Windows contains several pointer tables to handle request redirects. These tables include IDT, SSDT, and IAT.

A rootkit can modify any pointer in these tables, or swap the table itself.

3.1. IDT

User-mode applications requesting execution of a system-level function will need to raise their privilege level. This is implemented via program interrupts.

In Windows NT, system request are initialized by executing the “int 2e” command. The “int” command makes the CPU generate a program interrupt, referring to IDT index “2e” and reading data from that address. The processor then sets pointer value to the offset of the interrupt service routine stored in the table. System calls are using dispatcher service (KiSystemService).

In WinDbg, the “!idt 2e” command displays the routine pointer value. Normally, the routine pointer is set to the address of “KiSystemService”. If this is not the case, the routine has been intercepted.

3.2. Modifying code of system functions

In order to alter the program’s execution, one can either modify system call addresses or alter the code of those system calls. The calls can be modified in-place, or can be altered to re-address execution to a certain code segment (in simple cases, via a single “jump” command).

WinDbg includes facilities for discovering such alterations. A command is available to check code integrity of system calls by comparing volatile memory content with code stored on the symbol server. In order to validate the integrity of system calls, one can use the following command:

!chkimg -d nt

As a result, the debugger will list the differences, if any, between resident code and symbol copies.

kd> !chkimg -d nt

804d90c9-804d90cd 5 bytes – nt!KiXMMIZeroPage+30

[ fa f7 80 0c 02:e9 08 82 c7 00 ]

804d910c – nt!KiXMMIZeroPage+73 (+0x43)

[ fb:90 ]

804d9112-804d9115 4 bytes – nt!KiXMMIZeroPage+79 (+0x06)

[ 57 ff ff ff:6d c6 c1 00 ]

804d9545-804d954a 6 bytes – nt!ExAcquireResourceSharedLite+10 (+0x433)

[ fa 8b 75 08 33 db:e9 a3 c2 c1 00 cc ]

804d9564 – nt!ExAcquireResourceSharedLite+98 (+0x1f)

[ fb:90 ]

……………………..some entries were left out……………………..

804ed809-804ed80f 7 bytes – nt!CcGetActiveVacb+5 (+0x4670)

[ fa 8b 45 08 8b 48 48:e9 ee 7f c0 00 cc cc ]

804ef1dc-804ef1e3 8 bytes – nt!CcSetActiveVacb+7 (+0x19d3)

[ fa 8b 45 08 83 78 48 00:e9 70 66 c0 00 cc cc cc ]

804ef1ff-804ef20c 14 bytes – nt!CcSetActiveVacb+a3 (+0x23)

[ 8b 0a 89 48 48 89 58 50:e9 3d 66 c0 00 e9 2c 66 ]

804f0c70-804f0c73 4 bytes – nt!ExAcquireSharedStarveExclusive+7 (+0x1a71)

[ 64 a1 24 01:e9 cf 05 c6 ]

805745a3-805745a7 5 bytes – nt!NtOpenProcess+5

[ 68 60 a5 4e 80:e9 d8 b4 7b 78 ]

161 errors : nt (804d90c9-805745a7)

The listing above demonstrates that certain parts of system code were altered.

3.3. Searching for Hidden Resident Modules

Some rootkits load their executable images directly into the system memory.

Windows executable files in PE (Portable Executable) format can be identified with the “MZ” signature appearing in the very beginning of the image. WinDbg can help searching for this signature through the entire OS memory.

The search command:

s -d 0x0 L?0xffffffff 0x00905a4d

As a result, the debugger will list all entries containing the “MZ” signature.

0:001> s -d 0x0 L?0xffffffff 0x00905a4d

01000000 00905a4d 00000003 00000004 0000ffff MZ…………..

5ad70000 00905a4d 00000003 00000004 0000ffff MZ…………..

74720000 00905a4d 00000003 00000004 0000ffff MZ…………..


The first column contains the address of the executable module. The other four columns display the first four words in the module.

If the module was properly loaded into the system memory, the “!lmi” command will list information about the module.

0:001> !lmi 01000000

Loaded Module Info: [01000000]

Module: calc

Base Address: 01000000

Image Name: C:\windows\system32\calc.exe

Machine Type: 332 (I386)

Time Stamp: 3b7d8410 Fri Aug 17 15:52:32 2001

Size: 1f000

CheckSum: 1d7fc

Characteristics: 10f

Debug Data Dirs: Type Size VA Pointer

CODEVIEW 19, 160c, a0c NB10 – Sig: 3b7d8410, Age: 1, Pdb: calc.pdb

Symbol Type: DEFERRED – No error – symbol load deferred

Load Report: no symbols loaded

If, however, the module was infused directly into the memory, the system will know nothing about the process. If this is the case, the “!lmi” command will produce the following output:

Loaded Module Info: [0xf883ecc0]

fffffffff883ecc0 is not a valid address

It is possible to automate this search, as manually checking each and every entry can be extremely time-consuming. Automating procedures in WinDbg can be performed via the use of a script written in WinDbg internal language.

WinDbg scripts are plain-text files containing any set of debugger commands along with control directives such as .if, .else, .elsif, .foreach, .for, .while, .do, .break, .continue, .catch, .leave, .printf, .block. Variables are substituted with aliases or pseudo-registers ($tn, with n being an integer) assigned with the “r” command. Logical constructions can be programmed with MASM or C type syntaxes. C-like expressions are used like this: @@c++(expression).

.foreach ( start { s -[1]d 0x0 L?0xffffffff 0x00905a4d })


.echo start;

!lmi start;


WinDbg can be used for further in-depth analysis of suspicious memory areas. Extracting the executable module is possible by reading its size (data located at offset 0x140) and saving the data to a file with “.writemem” command.

3.4. Intercepting routines via SSDT

SSDT (System Service Descriptor Table) is a system service dispatch table that contains entry point addresses of NT kernel services [13]. The following figures demonstrate normal execution of a system routine “WriteFile” vs. the execution of the same service after the rootkit replaced an SSDT pointer to its own routine. This way, a rootkit can control system routines, filtering out “unwanted” data.

Normal execution of WriteFile

Code flow using the WriteFile routine from modified SSDT. This leads to rootkit activation.

All pointers referenced in SSDT must refer to routines implemented in either “nt” or “win32k” library. Therefore, when checking these pointers for interceptions, one must verify whether SSDT pointers actually refer to one of those memory areas.

We developed a script to automate the process. The script returns a list of all SSDT pointers as well as memory modules they refer to. If the script detects a third-party library instead of a system one, it will mark such occurrences with the word “HOOK”.

3.5. Modifying the KTHREAD.pServiceDescriptorTable pointer

Threads are the system’s base execution units. Threads are represented by kernel structure KTHREAD (as well as by its executive extension ETHREAD). The (K/E)THREAD structure varies greatly from one kernel version to another. However, certain fields keep their format and purpose, e.g. “ServiceTable”. Normally, the field stores a pointer to either KeServiceDescriptorTable or KeServiceDescriptorTableShadow; however, a thread can redefine its table for its own use. If this value is redefined, this should be a red flag.

The following figure displays two threads. The first thread did not invoke a USER or GDI routine, and its “pServiceDescriptorTable” points to routines serviced by “ntoskrnl”. The “PServiceDescriptorTable” of the second thread points to the descriptor table, including the descriptor serving USER and GDI routines, the “win32k”. At the following image, one can observe that the rootkit copied an old SSDT, modified its pointers with its own handlers, and set a new pointer to the modified descriptor table into KTHREAD.

Ppointer in ETHREAD structures points to SSDT of the core

SSDT pointer in ETHREAD structures points to rootkit SSDT

In order to verify this, one can enumerate all processes and their threads, and check this field’s value. This algorithm is implemented in WinDbg scripting language. The script returns all executable threads with their SSDT pointers. The validity of the pointers is verified by comparing their values with the result returned by system funtsions “nt!KeServiceDescriptorTable” (SSDT table pointer) and “nt!KeServiceDescriptorTableShadow” (shadow SSDT table pointer) [14]. If the pointer set in the thread has a different value, a warning such as “Changed value of KTHREAD::ServiceTable” will be displayed.

3.6. Searching hidden processes

Rootkits that modify system’s kernel object queue can hide processes as follows. In Windows NT 5.x all processes active in the system are linked in a single bidirectional list. The rootkit swaps pointers of the two objects neighboring its own process. This way, when the kernel enumerates processes by following the links, it won’t see that process. The following figure demonstrates system queue, while the other image demonstrates that the rootkit hid the second process.

Process queue in OS

Process queue modified by a rootkit

Windows NT distributes CPU cycles between threads. Processes do not participate in this distribution; the system gives equal treatment to threads belonging to the same or different process. Context switching is implemented by the system scheduler (thread dispatcher). There are three queues: KiDispatcherReadyListHead (queues threads that are ready to execute), KiWaitInListHead and KiWaitOutListHead (two queues listing threads that are not yet scheduled because they are waiting for a certain event, e.g. the completion of an i/o operation).

A script automating this approach enumerates the four thread lists (KiDispatcherReadyListHead, KiWaitInListHead, KiWaitOutListHead, KiStackInSwapListHead), refers to a process owning a particular thread, and verifies if the process is listed in the process list.

4. Testing the scripts

In order to verify the ability of proposed detection methods to discover actual rootkits, the following tools were used:

  • VMware Workstation 7.1.4 (to make a snapshot of the system’s memory);
  • Windows XP Professional;
  • Moonsols Windows Memory Toolkit Community Edition (for converting a full memory dump into format accepted by WinDbg);
  • WinDbg 6.1 (debugger).

4.1. Testing methodology

  1. Creating a virtual machine in VMware with Windows XP Professional SP3.
  2. Taking a clean snapshot of the OS.
  3. Installing a rootkit.
  4. Capturing a memory dump (.vmem).
  5. The memory dump is converted into format accepted by WinDbg.
  6. The memory dump is loaded into WinDbg.
  7. Launching the scripts described earlier in this paper.
  8. Restoring the OS from the clean snapshot.

Steps 3 through 8 are repeated for every rootkit.

4.1.1. Using the scripts

The scripts are available as plain text files. Before launching the scripts, one needs to insert the complete path to the script file, e.g.

aS ${/v:rep_root} “C:\\rootkit”;

To launch scripts in WinDbg command line, the following command must be executed:


4.2. Test results

The results of using the detection scripts on a range of malware applications are summarized in the following table.


IDT 2e interception

Modifying code of system functions

Searching for Hidden Resident Modules

Intercepting routines via SSDT

Modifying the KTHREAD.pServiceDescriptorTable pointer

Searching hidden processes













































































4.3. Discussion

As shown in the table, not all algorithms demonstrated equal performance. For example, out of 27 viruses tested, only one uses IDT 2e interception, and only 3 hide their processes by removing them from the process queue. However, we still believe that the test results prove our proposed methodology for detecting rootkits based on the analysis of a memory dump. The results, however, clearly show that just the few detection algorithms reviewed in this paper are not enough to reliably detect the presence of a rootkit; additional research and additional algorithms are required to make the detection really effective.

5. Conclusion and Commercial Implementation

The work allows detecting many types of rootkits by analyzing offline memory dumps. Belkasoft already had a tool for acquiring memory dumps. Would you like to chat with Belkasoft about Malware Detection Plugin? Write us at contact@belkasoft.com for an insight! Belkasoft is looking forward to enhancing its forensic software with highly innovative heuristic rootkit detection algorithms. Detecting rootkits residing in the computer’s volatile memory opens additional opportunities in digital investigations, allowing forensic specialists differenciate between suspicious activities and ones generated by the rootkit.

6. Authors

  • Dmitry Korolev graduated St-Petersburg State University and is a member of Belkasoft Research team.
  • Yuri Gubanov is a renowned computer forensics expert. He is a frequent speaker at industry-known conferences such as CEIC, HTCIA, China Forensic Conference, FT-Day, ICDDF, TechnoForensics and others. Yuri is the Founder and CEO of Belkasoft. Besides, Yuri is an author of f-interviews.com, a blog where he takes interviews with key persons in digital forensics and security domain.You can add Yuri Gubanov to your LinkedIn network at http://linkedin.com/in/yurigubanov
  • Oleg Afonin is Belkasoft sales and marketing director. He is an expert and consultant in computer forensics.

Authors can be contacted via email: research@belkasoft.com

Belkasoft Research is based in St. Petersburg State University. The company performs non-commercial researches and scientific activities.

7. References

[1] A. Shevchenko, “Rootkit evolution,” 28 August 2008. [Online]. Available: http://www.securelist.com/en/analysis/204792016/Rootkit_evolution.
[2] J. B. Greg Hoglund, Rootkits: Subverting the Windows Kernel, Boston: Addison-Wesley Professional, 2005.
[3] Wikipedia, “Sony BMG copy protection rootkit scandal,” 24 апрель 2012. [Online]. Available: http://en.wikipedia.org/wiki/Sony_BMG_copy_protection_rootkit_scandal.
[4] wikipedia, “Windows Firewall,” 7 March 2013. [Online]. Available: http://en.wikipedia.org/wiki/Windows_Firewall.
[5] wikipedia, “Intrusion detection system,” 21 March 2013. [Online]. Available: http://en.wikipedia.org/wiki/Intrusion_detection_system.
[6] wikipedia, “Intrusion prevention system,” 3 April 2013. [Online]. Available: http://en.wikipedia.org/wiki/Intrusion_prevention_system.
[7] B. C. Mark Russinovich, “RootkitRevealer v1.71,” 1 November 2006. [Online]. Available: http://technet.microsoft.com/en-us/sysinternals/bb897445.aspx.
[8] Wikipedia, “Open Source Tripwire,” 2 апрель 2012. [Online]. Available: http://en.wikipedia.org/wiki/Open_Source_Tripwire.
[9] G. F. Alexander Frolov, Защищенный режим процессоров Intel 80286/80386/80486, vol. 4, Moscow: Диалог-МИФИ, 1993, p. 234.
[10] Gangsta, “Гипервизор в роли антируткита,” 10 Febrary 2011. [Online]. Available: http://habrahabr.ru/post/113519/.
[11] VMware, “VMware,” [Online]. Available: http://www.vmware.com.
[12] Moonsols, “Moonsols Windows Memory Toolkit,” [Online]. Available: http://www.moonsols.com.
[13] S. Schreiber, Undocumented Windows 2000 Secrets: A Programmer’s Cookbook, Boston: Addison-Wesley Professional, 2001.
[14] M. Russinovich and D. Solomon, Microsoft Windows Internals (Fourth ed.), Microsoft Press, 2004.
[15] S. Advisory, “Microsoft Security Advisory (2286198),” 2 Август 2010. [Online]. Available: http://technet.microsoft.com/en-us/security/advisory/2286198.
[16] M. Rem, “Process Hunter,” [Online]. Available: http://ms-rem.dot-link.net/.
[17] F-Secure, “F-Secure,” [Online]. Available: http://www.f-secure.com.
[18] Greatis, “Unhackme,” [Online]. Available: http://www.greatis.com/unhackme/.
[19] IceSword, “IceSword,” [Online]. Available: http://www.antirootkit.com.
[20] J. Rutkovska, “System Virginity Verifier,” [Online]. Available: http://www.softpedia.com/get/System/System-Info/System-Virginity-Verifier.shtml.
[21] “GMER,” [Online]. Available: http://www.gmer.net/.
[22] GPL, “The Volatility Framework,” [Online]. Available: https://www.volatilesystems.com/default/volatility.
[23] Kaspersky lab, “Rootkit.Win32.Fu.b,” 9 June 2006. [Online]. Available: http://www.securelist.com/en/descriptions/158955/Rootkit.Win32.Fu.b.
[24] Kaspersky lab, “Rootkit.Win32.Stuxnet.a,” 20 September 2010. [Online]. Available: http://www.securelist.com/en/descriptions/15071647/Rootkit.Win32.Stuxnet.a.
[25] Kaspersky lab, “Trojan-Spy.Win32.Zbot.a,” 24 December 2008. [Online]. Available: http://www.securelist.com/en/descriptions/253057/Trojan-Spy.Win32.Zbot.a.
[26] wikipedia, “Direct memory access,” 7 March 2013. [Online]. Available: http://en.wikipedia.org/wiki/Direct_memory_access.

Leave a Comment

Latest Articles