3/16/09

Debugger Tips

Visual Studio's (native) C++ debugger has many useful features that can make your debugging much more pleasant, if you know what they are. These tend to accumulate over releases, and get forgotten and unused, unless you happen upon an archaic piece of documentation. On this topic, then, there are special expression and format specifiers that you can use to better examine the content in the debugger's watch windows.

For example, say we break after the following bit of code:

int i = 0x12345678;

You can use the by, wo, and dw operators to view contents of a variable as an unsigned byte, word, or dword:

i 0x12345678 int

by i 0x78 'x' unsigned char

wo i 0x5678 unsigned short

dw i 0x12345678 unsigned long

You can also use the operators on a register to do the same to the destination of the register:

eax 0x0012ff2c unsigned long

by eax 0x78 'x' unsigned char

wo eax 0x5678 unsigned short

dw eax 0x12345678 unsigned long

These come in handy when debugging through assembly.

Another way to change debugger output is through format specifiers. These are directives passed after the expression, separated by a comma. For example, to change the radix out the output, you can append ',o' for octal, ',d' for decimal, or ',x' for hex:

i 42 int

i,o 052 int

i,d 42 int

i,x 0x0000002a int

To interpret a pointer expression as a string, you can use ',s' for an simple null-terminated string, ',s8' for a UTF-8 string, or ',su' for a Unicode string. (Note that the expression has to be a pointer type for this to work).

char str[] = "hello";

wchar_t str2[] = L"world";

str 0x0012ff00 "hello" char [6]

str,s "hello" char [6]

str2 0x0012fee8 "world" wchar_t [6]

(void*)str2,su "world" void *

The memory operators can be used to display up to 64 bytes of memory in the preview line, as bytes, words, dwords, quads, or ascii characters.

str,m 0x0012ff00 68 65 6c 6c 6f 00 cc cc cc cc cc cc cc cc cc cc hello. char [6]

str,mb 0x0012ff00 68 65 6c 6c 6f 00 cc cc cc cc cc cc cc cc cc cc hello. char [6]

str,mw 0x0012ff00 6568 6c6c 006f cccc cccc cccc cccc cccc char [6]

str,md 0x0012ff00 6c6c6568 cccc006f cccccccc cccccccc char [6]

str2,mu 0x0012feec 0077 006f 0072 006c 0064 0000 cccc cccc world.?? wchar_t [6]

str,mq 0x0012ff00 cccc006f6c6c6568 cccccccccccccccc char [6]

str,ma 0x0012ff00 hello.(..(......T.. char [6]

You can use ,wc ,wm and ,hr to view data as a window class, window message, or HRESULT.

0x00400000,wc WS_OVERLAPPEDWINDOW int

0x10,wm WM_CLOSE int

0x10,hr 0x00000010 The directory cannot be removed. int

Finally, you can use ,! to turn off STL visualizations on the expression:

str "hello world" std::basic_string< ... >

str,! {_Bx={...} _Mysize=0x0000000b _Myres=0x0000000f} std::basic_string<...>

All of these operators can be used to ease the way you get to data while debugging, and become necessary whern creating custom visualizations. You can check-out the autoexp.dat file in your Visual Studio directory for examples of how to combine these operators and the visualization language to create custom visualizers for your own data.

No comments: