As is widely known, the new release of Windows 10 is imminent. Microsoft is advertising this release and urging customers to upgrade. This means that we should expect so see more and more Windows 10 machines that require Rekall’s advanced memory analysis!
Each time a new version of Windows is released, it’s a little bit like Christmas for me - full of surprises, you never really know what you are going to get! Microsoft did not disappoint this time either. There are some differences in Windows 10 that require Rekall to vary its approach to certain analysis techniques. It has been fun to make Rekall work with windows 10 and this post will cover some of the interesting differences between Windows 10 and earlier versions. I will also explain some of the changes we implemented in Rekall.
If you want to play with Windows 10 yourself, you could simply fetch the sample image from http://images.rekall-forensic.com/images/ and try it with the latest git head checkout. If you discover a bug, please let us know on the issues page so we can fix it before the next release.
To start off I installed Windows 10 preview (en_windows_10_pro_insider_preview_10074_x64_dvd_6651360.iso) on Virtual Box. Once the install was done, I downloaded Rekall from the release page and used winpmem_2.0.1.exe to acquire an AFF4 memory image together with the pagefile. So far, there were no surprises! Winpmem just worked out the box. I then continued to work on the image.
When I ran Rekall on this image, the profile was not found in the profile repository but this is not really a problem. Recent versions of Rekall can just fetch their own profiles from the Microsoft Symbol server by themselves (you can see this if you specify the -v flag):
$ rekal -v -f ~/images/win10.aff4
DEBUG:root:Will detect profile using these Detectors: nt_index,osx,pe,rsds,ntfs,linux
DEBUG:root:Opened url https://raw.githubusercontent.com/google/rekall-profiles/master/v1.0/inventory.gz
DEBUG:root:Opened url http://profiles.rekall-forensic.com/v1.0/inventory.gz
DEBUG:root:Skipped profile nt/GUID/2F35AA77F3484FE59775E55B7FF1EDE61 from https://raw.githubusercontent.com/google/rekall-profiles/master (Not in inventory)
DEBUG:root:Skipped profile nt/GUID/2F35AA77F3484FE59775E55B7FF1EDE61 from http://profiles.rekall-forensic.com (Not in inventory)
DEBUG:root:Will build local profile nt/GUID/2F35AA77F3484FE59775E55B7FF1EDE61
Trying to fetch http://msdl.microsoft.com/download/symbols/ntoskrnl.pdb/2F35AA77F3484FE59775E55B7FF1EDE61/ntoskrnl.pd_
ERROR:root:Error: HTTP Error 404: Not Found
Trying to fetch http://msdl.microsoft.com/download/symbols/ntkrnlpa.pdb/2F35AA77F3484FE59775E55B7FF1EDE61/ntkrnlpa.pd_
ERROR:root:Error: HTTP Error 404: Not Found
Trying to fetch http://msdl.microsoft.com/download/symbols/ntkrpamp.pdb/2F35AA77F3484FE59775E55B7FF1EDE61/ntkrpamp.pd_
ERROR:root:Error: HTTP Error 404: Not Found
Trying to fetch http://msdl.microsoft.com/download/symbols/ntkrnlmp.pdb/2F35AA77F3484FE59775E55B7FF1EDE61/ntkrnlmp.pd_
Extracting cabinet: /tmp/tmpo9ZU84/ntkrnlmp.pd_
All done, no errors.
DEBUG:root:Opened url https://raw.githubusercontent.com/google/rekall-profiles/master/v1.0/mspdb.gz
DEBUG:root:Adding mspdb to local cache.
INFO:root:Loaded profile mspdb from Local Cache Directory:/tmp/.rekall_cache
WARNING:root:Profile nt/GUID/2F35AA77F3484FE59775E55B7FF1EDE61 fetched and built. Please consider reporting this profile to the Rekall team so we may add it to the public profile repository.
DEBUG:root:Opened local file /tmp/.rekall_cache/v1.0/nt/GUID/2F35AA77F3484FE59775E55B7FF1EDE61.gz
INFO:root:Loaded profile nt/GUID/2F35AA77F3484FE59775E55B7FF1EDE61 from Local Cache Directory:/tmp/.rekall_cache
DEBUG:root:Found _EPROCESS @ 0x225AA40 (DTB: 0x1AA000)
INFO:root:Detected ntkrnlmp.pdb with GUID 2F35AA77F3484FE59775E55B7FF1EDE61
DEBUG:root:Detection method rsds worked at offset 0x21128d0
The Rekall Memory Forensic framework 1.3.2 (Dammastock).
"We can remember it for you wholesale!"
This program is free software; you can redistribute it and/or modify it under
the terms of the GNU General Public License.
See http://www.rekall-forensic.com/docs/Manual/tutorial.html to get started.
 win10.aff4 00:45:18>
The Object Manager.
The most important difference from a Memory analysis perspective is that the Object Header’s TypeIndex field is obfuscated. In order to understand how this impacts Rekall, I will summarize quickly how the windows Object Manager works, and how we use this in Rekall.
The windows kernel manages Objects (Please do not confuse this with object oriented languages like C++ objects - kernel objects are not really Object Oriented in the same sense of the word). An object is just an entity that the kernel is responsible for managing (i.e. create, destroy etc). There are many kernel objects, such as Process, Thread etc. You can actually see the kernel objects by running the object_types plugin:
 win10.aff4 00:08:34> object_types
Type Index Number Objects PoolType Name
-------------- ----- --------------- -------------------- ----
0xe0003484c5b0 2 53 NonPagedPoolNx Type
0xe0003485ea80 3 45 PagedPool Directory
0xe0003485d630 4 160 PagedPool SymbolicLink
0xe00034860c20 5 1182 PagedPool Token
0xe00034872c40 6 9 NonPagedPoolNx Job
0xe00034865d70 7 46 NonPagedPoolNx Process
0xe00034856440 8 682 NonPagedPoolNx Thread
Overall there are 48 different types of objects the Windows 10 kernel manages (you can see above how many objects of each type are outstanding as well as some statistics - e.g. we can see that there are 46 processes currently running).
The kernel has an array of _OBJECT_TYPE pointers located at nt!ObTypeIndexTable, which contains one pointer for each type (this is the meaning of the Index column above - it is just the index into this array.).
When the kernel wants to allocate one of these object (e.g. create a new process) it calls the object manager to allocate a new object of the required type (e.g. an _EPROCESS) object. It’s called the Object manager because it is responsible for tracking and managing different aspects of these objects. The object manager does this by adding additional headers before the allocated header.
Figure 1. Object layout in the kernel pool
See more information in our workshop slides: here.
So when we have a pointer to an _EPROCESS struct we know that before that address there will be a an _OBJECT_HEADER struct, and before that there is a variable number of optional headers. Before the optional headers we find the _POOL_HEADER.
The first sign of trouble was a failure of the handles and object_tree plugins which inspect the object manager’s objects. Here is the output ofobject_tree:
This is very weird. The Type of many objects is completely wrong! e.g. "Sessions" and "ArcName" should be of type Directory, while "PowerPort" should be an ALPC Port.
Lets take a closer look at valid objects. First we list all processes - which works because Rekall can just follow the linked list of active processes fromnt!PsActiveProcessHead. We then try to manually construct an _OBJECT_HEADER before one of the reported _EPROCESS structs:
We can see that at offset 0x18, the TypeIndex field has a value of 0x92 which is completely wrong! Since Windows 7, the_OBJECT_HEADER.TypeIndex is supposed to refer to the index of the relevant _OBJECT_TYPE struct in the global array atnt!ObTypeIndexTable. But we know there are only 48 different types so clearly 0x92 is nonsense.
To work out what is happening, I reasoned that the functions in the object manager which allocate and free these objects will need to be able to go back to the object’s _OBJECT_TYPE record, if only to increment/decrement the use count statistic we saw before. One of the functions seem promising:
This function accepts the address of the _OBJECT_HEADER in RCX it immediately shifts it right by 8 bits, and XORs it with TypeIndex field (at 0x18 from the start of the header). It then fetches a cookie from the symbol nt!ObHeaderCookie and XORs that as well.
Let us use this example and work this by hand. First dump the cookie then calculate:
This is exactly the correct type for a Process (see the output of object_types). So the _OBJECT_HEADER.TypeIndex field is obfuscated. We can de-obfuscate the field automatically by adding an object @property:
cookie = self.obj_profile.get_constant_object("ObHeaderCookie", target="byte").v()# Windows 7 has no cookie.if cookie == None:return self.m("TypeIndex")# Windows 10 xors the virtual address into this field so we need to use# the virtual address to decode it.
vaddr = self.obj_offset
return((vaddr >>8)^ cookie ^int(self.m("TypeIndex")))&0xFF
We simply retrieve the cookie using the profile (Rekall profiles have full symbol information) and then retrieve the original TypeIndex fields with them() method. This automatically de-obfuscates the field whenever it is accessed. Now both object_tree and handles work correctly.
We are not out of the woods yet! Alas, none of the pool scanning plugins like psscan work at all!
One of the more popular techniques in memory analysis is pool scanning. This is essentially carving out common signatures of typical kernel pool allocations in the hope of finding hidden kernel objects (such as hidden processes or file objects).
Each kernel pool allocation starts with the _POOL_HEADER struct, and these typically have a unique pool tag - a four byte sequence which identify the purpose of the allocation. The idea is to locate pool tags for the objects of interest and then recover the actual objects from the allocation. So for example to scan for processes:
Scan the physical address space for the pool tag (since Windows 8 this is Proc).
Verify other fields from the _POOL_HEADER (e.g. pool type and required allocation size).
Now we need to search forward from the _POOL_HEADER to find the _OBJECT_HEADER. Due to the variable number of optional headers, Rekall simply attempts to instantiate an _OBJECT_HEADER at ever increasing offsets and see if the struct "fits" (i.e. it is of the correct type and has the right number of optional headers).
There are some common limitations of pool scanning techniques. Pool tags are only used for debugging so attackers can easily manipulate them. Further, scanning the physical address space is much faster but provides no context - it is possible for attackers to "fake" kernel objects by simply having a pattern in data under their control which happens to look a bit like a process. Despite these shortcomings, scanning is still used frequently.
The astute reader might have noticed the problem already!
Since the pool scanner inspects the physical address space it is impossible to de-obfuscate the TypeIndex field since one must XOR the field with bits 8-16 of the virtual address of the _OBJECT_HEADER, but when finding it in the physical memory we have no idea where it is actually mapped in the kernel’s address space.
Here is an example to stress the point. Consider the same _OBJECT_HEADER we looked at before, only this time we check its physical address. First convert the virtual address to physical using the vtop plugin and then try to repeat the calculation with this physical address:
We can not use the physical address of the _OBJECT_HEADER to de-obfuscate since bits 12-16 are random! The resulting object type is completely wrong.
The solution involves ensuring that we find the virtual address for every physical address we scan through. Rekall implements two methods for finding where each physical page is mapped:
The ptov plugin uses the PFN database to find the virtual address of every physical frame. This is pretty fast.
The pas2vas plugin takes the brute force approach of building a large lookup table by iterating over all virtual addresses, and noting their physical address. While building the initial lookup table takes some time, looking it up is very fast.
The solution we chose was to re-implement the pas2vas as a Rekall service. This allows the lookup map (which takes a few seconds to construct) to be globally cached and reused in general. Furthermore we can specify that the virtual address we seek is in the kernel address space:
The lookup maps are cached in the physical_address_resolver object and then can be queried for a specific address space. In the example above we restrict the search to the kernel address space (which is also shared with the System process PID=4).
If you just want to know which process owns any specific physical address, use the pas2vas plugin.
Once the lookup maps are built, they can be very quickly used to resolve the virtual address and the TypeIndex field can be easily de-obfuscated. Lets see the process scan:
 win10.aff4 12:39:05> psscan
- _EPROCESS (P) Name PID Offset(V) PPID PDB Stat Time created Time exited
-------------- -------------------- ----- -------------- ------ -------------- ---- ------------------------ ------------------------
0x000006fa4080 svchost.exe 804 0xe00036471080 496 0x00000713a000 EP 2015-06-03 06:56:05+0000
0x000007922680 SearchProtocol 3720 0xe000361b3680 2984 0x00001832a000 EP 2015-06-03 06:56:28+0000
0x00000874b680 ShellExperienc 2656 0xe00035fad680 572 0x000009aaa000 EP 2015-06-03 06:56:16+0000
0x000008a86680 WSHost.exe 2744 0xe0003604f680 572 0x000034ff1000 EP 2015-06-03 06:56:17+0000
0x000008bee080 svchost.exe 856 0xe000364ec080 496 0x000009022000 EP 2015-06-03 06:56:05+0000
0x000009027680 svchost.exe 880 0xe000364f5680 496 0x000008f38000 EP 2015-06-03 06:56:05+0000
As we work more with Windows 10 we might find some more minor differences. We hope to have complete tested support for Windows 10 by the time it is officially released!