Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Analyst’s Perspective: Analyzing User Mode State from a Kernel Connection

Analyzing user mode state from the kernel debugger appears to have become something of a black art. Some people swear it can’t be done, others swear that it can’t be done reliably, and a small few claim that they do it all the time without any problems. I’m here to say that, yes, it can be done and to grow that small few to the vast majority…

When it comes down to it, there are only three things that you need to understand in order to properly work with user mode state from a kernel debug connection. So, let’s explore each of these.

The Virtual Address Space in Windows

Windows maintains two different virtual address spaces, the user virtual address space and the kernel virtual address space. In a standard x86 installation, this division results in the low 2GB of virtual memory being given to the current user process and the high 2GB being the kernel virtual address space.

The lower portion of the address space changes depending on the thread currently executing on the processor. The higher portion of the address space however is the same across all process contexts. Thus, the lower portion of the address space is process context specific whereas the higher portion of the address space is process context independent.

WinDBG and Process Context

Understanding the virtual address space in Windows is a critical point to any analyst who wants to inspect user mode state. What one has to realize is that the debugger can only use one process context at a time to translate virtual addresses. This means that if you want to inspect user state you must make sure that you have instructed WinDBG to use the correct process context for that state. Failure to do so will lead to access errors or, even worse, incorrect or misleading information being returned.

Also worth noting at this point is that the .thread command does not change process context by default, thus simply switching to a different thread context is not sufficient to change your process context.

WinDBG and the User Mode Loaded Module List

The user mode loaded module list is our final piece to understanding working with WinDBG and user mode state. Unlike in kernel mode where we have a single loaded module list that WinDBG keeps track of, WinDBG does not keep track of the user module list for each process. Instead, WinDBG keeps a single list that represents the user module list at the time of the last .reload. What this means for you is that any time you begin working with a new user mode state, you want to make sure you refresh the user module list so that it matches the process you are analyzing.

Inspecting User State in Practice

Given that we now have the foundation, let’s put the pieces together and see some practical examples. I’ll start off by breaking in to an idle system from a live kernel debug session and inspecting the current process context:

It’s the Idle process, which isn’t much of a shock. The Idle process is interesting in that it’s one of the two system processes, which are processes with no user mode state. After a .reload we can inspect the user module list with lmu:

Note that there are no modules on the user mode loaded module list, which makes sense considering the fact that this is a system process. However, in the !process 0 0 output I see an instance of Notepad and I really want to set a breakpoint in that process:

In order to do that, I need to use WinDBG’s .process command to switch to the Notepad process context. In a live debug session we also want to specify the /i to inspect the process state invasively. This will require that we resume the target machine, after which the target will break in to the debugger in the correct process context:

From here, we should be able to inspect the current process and see that we’re in the Notepad process:

However, we still do not have any user modules on our loaded module list!

Remember, WinDBG caches the user module list from the last .reload, thus we’re still using the original loaded module list from the Idle process. In order to get WinDBG to refresh the user loaded module list, we need to perform a .reload again. Though we can save a bit of time here by just instructing WinDBG to reload the user module list with .reload /user:

Now we can actually see some results when inspecting the user module list:

At this point in the analysis, we are free to inspect user mode state or set breakpoints in user mode routines. However, be aware that setting a breakpoint in a DLL mapped into multiple processes will result in the breakpoint being set in all of those processes. Writes from the kernel mode debugger are not subject to copy-on-write, thus setting a breakpoint with bp will put an int 3 instruction in the shared physical page. You can see the results of this here:

A process specific breakpoint can be your savior here though:

Though the breakpoint will still be set in all processes sharing the page, the process specific breakpoint will cause WinDBG to only break if the breakpoint is hit by the specified process. Here we use the $proc pseudo register, which always maps to the current process.

Note what happens now if we become interested in a different process, say VMWareUser.exe:

We do all of the same processing as above and then check the user module list:

Note how it looks like Notepad is mapped into the VMWareUser.exe process. Clearly this is bogus, it’s just WinDBG using the cached user module list from the last .reload performed. Because our analysis has brought us to a new user process, we will again need to perform a .reload /user to have our module list updated:

What About Crash Dumps?

If you try to perform a .process /i command from within a crash dump, you’ll be greeted with an error:

This operation only works on live kernel debug sessions due to the fact that the invasive switch requires that code actually execute on the target machine. Luckily, there is a way to force WinDBG to internally switch to a different process context without changing the state of the target. For that, we’ll use .process with the /r and /p switches. In addition to getting us into the correct process context, this will force a reload of the user symbol list:

Additionally, .thread also takes /r and /p switches to automatically switch the debugger to the correct process context for a particular thread. This is extremely helpful if you’re moving around a full memory dump and would like to automatically have your process context set for each thread you inspect:

Seeing User State with !process and !thread

Last but not least, both !process and !thread take a flag value of 0x10, which causes the extension command to perform the equivalent of a .process /r /p for the appropriate process before displaying the call stacks of the threads. Thus, instead of this:

Which aborts once entering user mode, you will see this:

Black Art No More!

While there’s always more to explore, hopefully this article serves to pique your interest and allow you to incorporate more user mode analysis into your kernel debugging sessions!

 

Analyst’s Perspective is a column by OSR consulting associate, Scott Noone. When he’s not root-causing complex kernel issues, he’s leading the development and instruction of OSR’s Kernel Debugging seminar. Comments or suggestions for this or future Analyst’s Perspective columns can be addressed to ap@osr.com.

Summary
Article Name
Analyst's Perspective: Analyzing User Mode State from a Kernel Connection
Description
This article uses a practical example to explain how to perform user mode analysis from a kernel debug session
Author