Case Study 241
CASE STUDY
Consider the following legacy C++/Win32 code fragment highlighted in WinDbg
after opening a crash dump:
1: HANDLE hFile = CreateFile(str.GetBuffer(), GENERIC_READ,
FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
2: if (hFile != INVALID_HANDLE_VALUE)
3: {
4: DWORD dwSize = GetFileSize(hFile, NULL);
5: DWORD dwRead = 0;
6: CHAR *bufferA = new CHAR[dwSize+2];
7: memset(bufferA, 0, dwSize+2);
8: if (ReadFile(hFile, bufferA, dwSize, &dwRead, NULL))
9: {
10: DWORD i = 0, j = 0;
11: for (; i < dwSize+2-7; ++i)
12: {
13: if (bufferA[i] == 0xD && bufferA[i+1] != 0xA)
At the first glance the code seems to be right: we open a file, get its size and
allocate a buffer to read. All loop indexes are within array bounds too. Let’s look at
disassembly and crash point:
0:000> uf component!CMyDlg::OnTimer
…
…
…
004021bc push 0
004021be push esi
004021bf call dword ptr [component!_imp__GetFileSize (0042e26c)]
004021c5 mov edi,eax ; dwSize
004021c7 lea ebx,[edi+2] ; dwSize+2
004021ca push ebx

004021cb mov dword ptr [esp+34h],0
004021d3 call component!operator new[] (00408e35)
004021d8 push ebx
004021d9 mov ebp,eax ; bufferA
004021db push 0
004021dd push ebp
004021de call component!memset (00418500)
004021e3 add esp,10h
004021e6 push 0
004021e8 lea edx,[esp+34h]
004021ec push edx
004021ed push edi
004021ee push ebp
004021ef push esi
004021f0 call dword ptr [component!_imp__ReadFile (0042e264)]
004021f6 test eax,eax
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
242 PART 2: Professional Crash Dump Analysis
004021f8 jne component!CMyDlg::OnTimer+0×3b1 (00402331)
…
…
…
00402331 xor esi,esi ; i
00402333 add edi,0FFFFFFFBh ; +2-7 (edi contains dwSize)
00402336 cmp edi,esi ; loop condition
00402338 mov dword ptr [esp+24h],esi
0040233c jbe component!CMyDlg::OnTimer+0×43e (004023be)
00402342 mov al,byte ptr [esi+ebp] ; bufferA[i]
0:000> r
eax=00002b00 ebx=00000002 ecx=00431000 edx=00000000 esi=00002b28

edi=fffffffb
eip=00402342 esp=0012efd4 ebp=0095b4d8 iopl=0 nv up ei pl nz ac pe cy
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00000217
component!CMyDlg::OnTimer+0×3c2:
00402342 8a042e mov al,byte ptr [esi+ebp] ds:0023:0095e000=??
If we look at ebx (dwSize+2) and edi registers (array upper bound, dwSize+2-7)
we can easily see that dwSize was zero. Clearly we had buffer overrun because upper
array bound was calculated as 0+2-7 = FFFFFFFB (the loop index was unsigned integer,
DWORD). Were the index signed integer variable (int) we wouldn’t have had any prob-
lem because the condition 0 < 0+2-7 is always false and the loop body would have never
been executed.
Based on that the following fix was proposed:
1: HANDLE hFile = CreateFile(str.GetBuffer(), GENERIC_READ,
FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
2: if (hFile != INVALID_HANDLE_VALUE)
3: {
4: DWORD dwSize = GetFileSize(hFile, NULL);
5: DWORD dwRead = 0;
6: CHAR *bufferA = new CHAR[dwSize+2];
7: memset(bufferA, 0, dwSize+2);
8: if (ReadFile(hFile, bufferA, dwSize, &dwRead, NULL))
9: {
10: DWORD i = 0, j = 0;
10: int i = 0, j = 0;
11: for (; i < dwSize+2-7; ++i)
11: for (; i < (int)dwSize+2-7; ++i)
12: {

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Case Study 243

GetFileSize can return INVALID_FILE_SIZE (0xFFFFFFFF) and operator new can fail
theoretically (if the size is too big) so we can correct the code even further:
1: HANDLE hFile = CreateFile(str.GetBuffer(), GENERIC_READ,
FILE_SHARE_READ, NULL, OPEN_EXISTING, FILE_ATTRIBUTE_NORMAL, NULL);
2: if (hFile != INVALID_HANDLE_VALUE)
3: {
4: DWORD dwSize = GetFileSize(hFile, NULL);
4a: if (dwSize != INVALID_FILE_SIZE)
4b: {
5: DWORD dwRead = 0;
6: CHAR *bufferA = new CHAR[dwSize+2];
6a: if (bufferA)
6b: {
7: memset(bufferA, 0, dwSize+2);
8: if (ReadFile(hFile, bufferA, dwSize, &dwRead, NULL))
9: {
10: int i = 0, j = 0;
11: for (; i < (int)dwSize+2-7; ++i)
12: {


Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
244 PART 2: Professional Crash Dump Analysis
DETECTING LOOPS IN CODE
Sometimes when we look at a stack trace and disassembled code we see that a
crash couldn’t have happened if the code path was linear. In such cases we need to see
if there is any loop that changes some variables. This is greatly simplified if we have
source code but in cases where we don’t have access to source code it is still possible to
detect loops. We just need to find a direct (JMP) or conditional jump instruction (Jxxx,
for example, JE) after the crash point branching to the beginning of the loop before the

crash point as shown in the following pseudo code:
set the pointer value
…
label:
…
>>> crash when dereferencing the pointer
…
change the pointer value
…
jmp label
Let’s look at one example I found very interesting because it also shows
__thiscall calling convention for C++ code generated by Visual С++ compiler. Before we
look at the dump I quickly remind you about how C++ non-static class methods are
called. Let’s first look at non-virtual method call.
class A
{
public:
int foo() { return i; }
virtual int bar() { return i; }
private:
int i;
};
Internally class members are accessed via implicit this pointer (passed via ECX):
int A::foo() { return this->i; }
Suppose we have an object instance of class A and we call its foo method:
A obj;
obj.foo();
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Detecting Loops in Code 245
The compiler has to generate code which calls foo function and the code inside

the function has to know which object it is associated with. So internally the compiler
passes implicit parameter - a pointer to that object. In pseudo code:
int foo_impl(A *this)
{
return this->i;
}
A obj;
foo_impl(&obj);
In x86 assembly language it should be similar to this code:
lea ecx, obj
call foo_impl
If we have obj declared as a local variable the code is similar:
lea ecx, [ebp-N]
call foo_impl
If we have a pointer to an obj then the compiler usually generates MOV instruc-
tion instead of LEA instruction:
A *pobj;
pobj->foo();
mov ecx, [ebp-N]
call foo_impl
If we have other function parameters they are pushed on the stack from right to
left. This is __thiscall calling convention. For virtual function call we have an indirect call
through a virtual function table. The pointer to it is the first object layout member and
in the latter case where the pointer to obj is declared as the local variable we have the
following x86 code:
A *pobj;
pobj->bar();
mov ecx, [ebp-N]
mov eax, [ecx]
call [eax]


Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
246 PART 2: Professional Crash Dump Analysis
Now let’s look at the crash point and stack trace:
0:021> r
eax=020864ee ebx=00000000 ecx=0000005c edx=7518005c esi=020864dc
edi=00000000
eip=67dc5dda esp=075de820 ebp=075dea78 iopl=0 nv up ei pl nz na po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000 efl=00010202
component!CDirectory::GetDirectory+0×8a:
67dc5dda 8b03 mov eax,dword ptr [ebx] ds:0023:00000000=????????
0:021> k
ChildEBP RetAddr
075dea78 004074f0 component!CDirectory::GetDirectory+0x8a
075deaac 0040e4fc component!CDirectory::FindFirstFileW+0xd0
075dffb8 77e64829 component!MonitorThread+0x13
075dffec 00000000 kernel32!BaseThreadStart+0x34
If we look at GetDirectory code we would see:
0:021> .asm no_code_bytes
Assembly options: no_code_bytes
0:021> uf component!CDirectory::GetDirectory
component!CDirectory::GetDirectory:
67dc5d50 push ebp
67dc5d51 mov ebp,esp
67dc5d53 push 0FFFFFFFFh
67dc5d55 push offset component!CreateErrorInfo+0x553 (67ded93b)
67dc5d5a mov eax,dword ptr fs:[00000000h]
67dc5d60 push eax
67dc5d61 mov dword ptr fs:[0],esp
67dc5d68 sub esp,240h

67dc5d6e mov eax,dword ptr [component!__security_cookie (67e0113c)]
67dc5d73 mov dword ptr [ebp-10h],eax
67dc5d76 mov eax,dword ptr [ebp+8]
67dc5d79 test eax,eax
67dc5d7b push ebx
67dc5d7c mov ebx,ecx
67dc5d7e mov dword ptr [ebp-238h],ebx
67dc5d84 je component!CDirectory::GetDirectory+0×2a1 (67dc5ff1)
component!CDirectory::GetDirectory+0x3a:
67dc5d8a cmp word ptr [eax],0
67dc5d8e je component!CDirectory::GetDirectory+0x2a1 (67dc5ff1)
component!CDirectory::GetDirectory+0x44:
67dc5d94 push esi
67dc5d95 push eax
67dc5d96 call dword ptr [component!_imp__wcsdup (67df050c)]
67dc5d9c add esp,4
67dc5d9f mov dword ptr [ebp-244h],eax
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Detecting Loops in Code 247
67dc5da5 mov dword ptr [ebp-240h],eax
67dc5dab push 5Ch
67dc5dad lea ecx,[ebp-244h]
67dc5db3 mov dword ptr [ebp-4],0
67dc5dba call component!CStrToken::Next (67dc4f80)
67dc5dbf mov esi,eax
67dc5dc1 test esi,esi
67dc5dc3 je component!CDirectory::GetDirectory+0x28c (67dc5fdc)
component!CDirectory::GetDirectory+0x79:
67dc5dc9 push edi
67dc5dca lea ebx,[ebx]

component!CDirectory::GetDirectory+0x80:
67dc5dd0 cmp word ptr [esi],0
67dc5dd4 je component!CDirectory::GetDirectory+0x28b (67dc5fdb)
component!CDirectory::GetDirectory+0x8a:
>>> 67dc5dda mov eax,dword ptr [ebx]
67dc5ddc mov ecx,ebx
…
If we trace EBX backwards we would see that it comes from ECX so ECX could be
considered as an implicit this pointer according to __thiscall calling convention. There-
fore it looks like the caller passed NULL this pointer via ECX.
Let’s look at the caller. To see the code we can either disassemble FindFirstFileW
or disassemble backwards at the GetDirectory return address. We’ll do the latter:
0:021> k
ChildEBP RetAddr
075dea78 004074f0 component!CDirectory::GetDirectory+0×8a
075deaac 0040e4fc component!CDirectory::FindFirstFileW+0xd0
075dffb8 77e64829 component!MonitorThread+0×13
075dffec 00000000 kernel32!BaseThreadStart+0×34
0:021> ub 004074f0
component!CDirectory::FindFirstFileW+0xbe:
004074de pop ebp
004074df clc
004074e0 mov ecx,dword ptr [esi+8E4h]
004074e6 mov eax,dword ptr [ecx]
004074e8 push 0
004074ea push 0
004074ec push edx
004074ed call dword ptr [eax+10h]

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.

248 PART 2: Professional Crash Dump Analysis
We see that ECX is our this pointer. However the virtual table pointer is taken
from the memory it references:
004074e6 mov eax,dword ptr [ecx]
…
…
004074ed call dword ptr [eax+10h]
Were ECX a NULL we would have had our crash at this point. However we
have our crash in the called function. So it couldn’t be NULL. There is a contradiction
here. The only plausible explanation is that in GetDirectory function there is a loop that
changes EBX (shown in bold in GetDirectory function code above). If we have a second
look at the code we would see that EBX is saved in [ebp-238h] local variable before it is
used:
0:021> uf component!CDirectory::GetDirectory
component!CDirectory::GetDirectory:
67dc5d50 push ebp
67dc5d51 mov ebp,esp
67dc5d53 push 0FFFFFFFFh
67dc5d55 push offset component!CreateErrorInfo+0x553 (67ded93b)
67dc5d5a mov eax,dword ptr fs:[00000000h]
67dc5d60 push eax
67dc5d61 mov dword ptr fs:[0],esp
67dc5d68 sub esp,240h
67dc5d6e mov eax,dword ptr [component!__security_cookie (67e0113c)]
67dc5d73 mov dword ptr [ebp-10h],eax
67dc5d76 mov eax,dword ptr [ebp+8]
67dc5d79 test eax,eax
67dc5d7b push ebx
67dc5d7c mov ebx,ecx
67dc5d7e mov dword ptr [ebp-238h],ebx

67dc5d84 je component!CDirectory::GetDirectory+0×2a1 (67dc5ff1)
component!CDirectory::GetDirectory+0x3a:
67dc5d8a cmp word ptr [eax],0
67dc5d8e je component!CDirectory::GetDirectory+0x2a1 (67dc5ff1)
component!CDirectory::GetDirectory+0x44:
67dc5d94 push esi
67dc5d95 push eax
67dc5d96 call dword ptr [component!_imp__wcsdup (67df050c)]
67dc5d9c add esp,4
67dc5d9f mov dword ptr [ebp-244h],eax
67dc5da5 mov dword ptr [ebp-240h],eax
67dc5dab push 5Ch
67dc5dad lea ecx,[ebp-244h]
67dc5db3 mov dword ptr [ebp-4],0
67dc5dba call component!CStrToken::Next (67dc4f80)
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Detecting Loops in Code 249
67dc5dbf mov esi,eax
67dc5dc1 test esi,esi
67dc5dc3 je component!CDirectory::GetDirectory+0x28c (67dc5fdc)
component!CDirectory::GetDirectory+0x79:
67dc5dc9 push edi
67dc5dca lea ebx,[ebx]
component!CDirectory::GetDirectory+0x80:
67dc5dd0 cmp word ptr [esi],0
67dc5dd4 je component!CDirectory::GetDirectory+0x28b (67dc5fdb)
component!CDirectory::GetDirectory+0x8a:
>>> 67dc5dda mov eax,dword ptr [ebx]
67dc5ddc mov ecx,ebx
…

If we look further past the crash point we would see that [ebp-238h] value is
changed and then used again to change EBX:
component!CDirectory::GetDirectory+0x80:
67dc5dd0 cmp word ptr [esi],0
67dc5dd4 je component!CDirectory::GetDirectory+0×28b (67dc5fdb)
component!CDirectory::GetDirectory+0x8a:
>>> 67dc5dda mov eax,dword ptr [ebx]
67dc5ddc mov ecx,ebx
…
…
…
component!CDirectory::GetDirectory+0×11e:
67dc5e6e mov eax,dword ptr [ebp-23Ch]
67dc5e74 mov ecx,dword ptr [eax]
67dc5e76 mov dword ptr [ebp-238h],ecx
67dc5e7c jmp component!CDirectory::GetDirectory+0×20e (67dc5f5e)
…
…
…
component!CDirectory::GetDirectory+0×23e:
67dc5f8e cmp esi,edi
67dc5f90 mov ebx,dword ptr [ebp-238h]
67dc5f96 jne component!CDirectory::GetDirectory+0×80 (67dc5dd0)
We see that after changing EBX the code jumps to 67dc5dd0 address and this ad-
dress is just before our crash point. It looks like a loop. Therefore there is no contradic-
tion. ECX as this pointer was passed as non-NULL and valid pointer. Before the loop
started its value was passed to EBX. In the loop body EBX was changed and after some
loop iterations the new value became NULL. It could be the case that there were no
checks for NULL pointers in the loop code.
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.

250 PART 2: Professional Crash Dump Analysis

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
Crash Dump Analysis Checklist 251
CRASH DUMP ANALYSIS CHECKLIST
Often the root cause of a problem is not obvious from a memory dump. Here is
the first version of crash dump analysis checklist to help experienced engineers not to
miss any important information. The check list doesn’t prescribe any specific steps, just
lists all possible points to double check when looking at a memory dump.
General:
• Internal database(s) search
• Google or Microsoft search for suspected components as this could be a known issue.
Sometimes a simple search immediately points to the fix on a vendor’s site
• The tool used to save a dump (to flag false positive, incomplete or inconsistent dumps)
• OS/SP version
• Language
• Debug time
• System uptime
• Computer name
Application crash or hang:
• Default analysis (!analyze -v or !analyze -v -hang for hangs)
• Critical sections (!locks) for both crashes and hangs
• Component timestamps. DLL Hell?
• Do any newer components exist?
• Process threads (~*kv or !uniqstack)
• Process uptime
• Your components on the full raw stack of the problem thread
• Your components on the full raw stack of the main application thread
• Process size
• Number of threads

• Gflags value (!gflag)
• Time consumed by thread (!runaway)
• Environment (!peb)
• Import table (!dh)
• Hooked functions (!chkimg)
• Exception handlers (!exchain)
Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.
252 PART 2: Professional Crash Dump Analysis
System hang:
• Default analysis (!analyze -v -hang)
• ERESOURCE contention (!locks)
• Processes and virtual memory including session space (!vm 4)
• Pools (!poolused)
• Waiting threads (!stacks)
• Critical system queues (!exqueue f)
• I/O (!irpfind)
• The list of all thread stack traces (!process 0 ff for W2K3/XP/Vista, ListProcessStacks
script for Windows 2000, see page 222)
• LPC chain for suspected threads (!lpc message)
• Critical sections for suspected processes (!ntsdexts.locks)
• Sessions, session processes (!session, !sprocess)
• Processes (size, handle table size) (!process 0 0)
• Running threads (!running)
• Ready threads (!ready)
• DPC queues (!dpcs)
• The list of APCs (!apc)
BSOD:
• Default analysis (!analyze -v)
• Pool address (!pool)
• Component timestamps.

• Processes and virtual memory (!vm 4)
• Current threads on other processors
• Raw stack
• Bugcheck description (including ln exception address for corrupt or truncated dumps)

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.