Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

The Basics of Debugger Extensions

In order to talk about debugger extensions, we need to first talk about the Debugger Engine library, (DbgEng). DbgEng is a generic interface that can be used to manipulate various debuggingtargets, which can be things such as crash dumps, a live kernel, an application, etc. The library provides access to all of the types of actions that you might want to perform on a debugging target, such as setting or clearing breakpoints, reading or writing memory, translating addresses into symbolic debugging information, receiving events from the target, and so on.

The DbgEng library is quite flexible and can actually be used in standalone applications. If you wrote an application that exported all of the features of the DbgEng library, you’d end up with a debugger exactly like WinDBG or KD because, well, that’s what they are. In addition, the DbgEng library is what we’re going to talk to when we write our debugger extensions. Thus, in effect we have the same API set but two different uses. This can lead to a fair amount of confusion if you start looking through the WinDBG SDK samples because there’s a mix of applications and extensions supplied. If you’re not aware of this fact it can be difficult to find a sample to get started with.

Sessions
In either case, DbgEng relies on the concept of sessions. In order to examine or manipulate the target, a debugging session must be active. When a debug event is raised from the target, the target may become acquired at which point the debug session is active. The application or extension is now allowed to perform whatever options it wants against the target. Once finished with the target, it must be released, at which point the session is no longer active and operations are no longer allowed against the target.

Aside from sessions, the other major concept to learn about DbgEng is the set of objects that are available to the application or extension. The objects supplied are broken down into two basic categories: client objects and callback objects.

Client Objects
Client objects are used to interact with the debugger engine and provide access to all of the available DbgEng APIs used to manipulate the target via COM interfaces. The way this works is that each client object provides a QueryInterface method that is used to create instances of the various COM interfaces. So, for example, if you wanted to set a breakpoint on the target, you’d call the QueryInterface method on a client object to retrieve an instance of the COM interface that exports the set breakpoint API. Sounds a little scary, but it’s going to turn out to be straightforward so don’t worry. The list of available client COM interfaces and their uses are as follows:

  • IDebugAdvanced – Thread context, source server, and some symbol information
  • IDebugClient – Connect to targets, register input/output callbacks, register debugger event callbacks, etc.
  • IDebugControl – Evaluate expressions, disassemble, execute commands, add/remove breakpoints, get input, send output, etc.
  • IDebugDataSpaces – Read/write virtual memory, physical memory, I/O ports, MSRs, etc.
  • IDebugRegisters – Read/write pseudo and hardware registers
  • IDebugSymbols – Type information, module information, source file information, etc.
  • IDebugSystemObjects – Thread, process, and processor information

Callback Objects
As their name implies, callback objects export the COM interfaces necessary to receive notifications of events happening on the target. There are three types of callback objects: input objects, output object, and debugger event objects. These allow you to scan the input supplied to the debugger engine from the user as well as the output of any debugger commands or the debugging target. In addition to this, callback objects provide a way to receive notification of process creation, thread creation, breakpoints, etc. The list of available callback COM interfaces and their uses are as follows:

  • IDebugInputCallbacks – Receive notification of the engine waiting for user input
  • IDebugOutputCallbacks – Receive notification of output being sent to debugger engine
  • IDebugEventCallbacks – Receive notification of debug events

Creating Your Own Extension
Now that we have some of the basics down, we can see how to actually create a debugger extension. Debugger extensions are nothing more than DLLs that provide commands via exports and rely on the DbgEng library and any other Win32 API of their choosing. They can be built just like any other DLL, though we’ll use the WDK seeing as how that’s what the available samples use. The only other thing that you’ll need is the SDK subdirectory of the WinDBG installation, as that’s where the necessary header and library files are located.

Extension DLL Entry Points
There is a single required entry point for extension DLLs, DebugExtensionInitialize. This is where you will do all of your one time initialization for the extension DLL. A description of the entry point and the function prototype can be found in dbgeng.h:

A simple example of a complete DebugExtensionInitialize is as follows:

There are two other entirely optional exports that you driver can provide, DebugExtensionUninitialize  and DebugExtensionNotify. DebugExtensionUninitialize can be used to undo anything that you might have done in DebugExtensionInitialize:

If present, DbgEng calls the DebugExtensionNotify callback for session state changes:

The entire point of writing this DLL is to create your own debugger commands, and those will also be exports of your DLL. These commands or, extensions must appear in the .DEF file associated with your DLL and the exports must contain only lower case letters. The function prototype of an extension command is as follows:

You’ll note that the function prototype indicates that a pointer to a DEBUG_CLIENT structure is passed as the first parameter to the extension command.  This is actually the client object whose QueryInterface method you will use to gain access to the various client COM interfaces for target manipulation.

Let’s see a simple example command that uses the built in expression evaluator to add two and two together and then displays the result. This would be the equivalent of typing ? 2 + 2 in the WinDBG command prompt.

Hopefully the steps followed are fairly straightforward at this point. We’ve taken the passed-in client object, created an instance of IDebugControl, and then executed some methods on it to perform actions.

To further drive home the pattern, we can see how we’d get the offset of a field of a data structure. For that, we need an instance of IDebugSymbols:

Dealing with 32-bit vs. 64-bit
Note that when you’re writing a debugger extension command, your code is always running on the host machine. Also note that the pointer size of the host machine does not necessarily match the pointer size of the target machine, due to the fact that 32-bit hosts can debug 64-bit targets and vice versa. In order to deal with this, debugger extensions treat all addresses from the target as 64-bit values. Thus you’ll note that the DbgEng APIs express pointer addresses as ULONG64 values. Any 32-bit value used in your extension command must be sign extended out to a full 64-bit value.

Alternative Debugger Extension Interfaces
To add to the confusion when it comes to writing a debugger extension, there are two alternative interfaces that you can use to write your debugger extensions. The first is the WdbgExts interface, which is the legacy debugger extension interface that existed before DbgEng came around. This interface is still available in the newest versions of the debugger, however it has two drawbacks. First, it is not the forward moving API thus it is frozen in time and will provide no new features. Second, this interface does not have any support for writing standalone applications, thus it won’t port to any kind of automated analysis tool that you might write.

The other interface available to you is the EngExtCpp interface. This is actually just a C++ wrapper library around DbgEng to simplify common tasks such as manipulating typed data. Unlike WdbgExts, this is a fully supported interface and can be used alongside the direct DbgEng calls that we’ve discussed in this article.

Go forth!
Hopefully we’ve been able to clear up the cloud of mystery that hangs over writing debugger extensions and have set you on a journey of creating your own.

 

Download source to a debug extension (!uniqstack)

Summary
Article Name
The Basics of Debugger Extensions
Description
After a quick tour of the Debugger Engine library, we explain how to create your own debugger extensions to add to your trough of debugging and analysis tools
Author