www.it-ebooks.info
Inside Windows®
Debugging
Tarik Soulami
Published with the authorization of Microsoft Corporation by:
O’Reilly Media, Inc.
1005 Gravenstein Highway North
Sebastopol, California 95472
Copyright © 2012 by Tarik Soulami
All rights reserved. No part of the contents of this book may be reproduced or transmitted in any form or by any
means without the written permission of the publisher.
ISBN: 978-0-7356-6278-0
1 2 3 4 5 6 7 8 9 LSI 6 5 4 3 2 1
Printed and bound in the United States of America.
Microsoft Press books are available through booksellers and distributors worldwide. If you need support related
to this book, email Microsoft Press Book Support at mspinput@microsoft.com. Please tell us what you think of
this book at http://www.microsoft.com/learning/booksurvey.
Microsoft and the trademarks listed at http://www.microsoft.com/about/legal/en/us/IntellectualProperty/
Trademarks/EN-US.aspx are trademarks of the Microsoft group of companies. All other marks are property of
their respective owners.
The example companies, organizations, products, domain names, email addresses, logos, people, places, and
events depicted herein are fictitious. No association with any real company, organization, product, domain name,
email address, logo, person, place, or event is intended or should be inferred.
This book expresses the author’s views and opinions. The information contained in this book is provided without
any express, statutory, or implied warranties. Neither the authors, O’Reilly Media, Inc., Microsoft Corporation,
nor its resellers, or distributors will be held liable for any damages caused or alleged to be caused either directly
or indirectly by this book.
Acquisitions Editor: Russell Jones
Developmental Editor: Russell Jones
Production Editor: Melanie Yarbrough
Editorial Production: Waypoint Press
Technical Reviewer: John Mueller
Copyeditor: Roger LeBlanc
Indexer: Christina Yeager
Cover Design: Twist Creative • Seattle
Cover Composition: Karen Montgomery
Illustrator: Steve Sagman
www.it-ebooks.info
Contents at a Glance
Foreword
Introduction
PART I
A BIT of BACkgRounD
ChapTer 1
Software Development in Windows
PART II
DEBuggIng foR fun AnD PRofIT
ChapTer 2
ChapTer 3
ChapTer 4
ChapTer 5
ChapTer 6
ChapTer 7
ChapTer 8
ChapTer 9
ChapTer 10
Getting Started
how Windows Debuggers Work
postmortem Debugging
Beyond the Basics
Code analysis Tools
expert Debugging Tricks
Common Debugging Scenarios, part 1
Common Debugging Scenarios, part 2
Debugging System Internals
PART III
oBsERvIng AnD AnAlyzIng sofTwARE BEhAvIoR
ChapTer 11
ChapTer 12
ChapTer 13
Introducing Xperf
Inside eTW
Common Tracing Scenarios
appenDIX a WinDbg User-Mode Debugging Quick Start
appenDIX B WinDbg Kernel-Mode Debugging Quick Start
Index
xv
xvii
3
33
85
125
159
195
219
267
323
365
391
415
457
505
519
527
www.it-ebooks.info
Contents
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
Who Should Read This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii
Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii
Organization of This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xviii
Conventions in This Book . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xix
System Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx
Code Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi
Installing the Code Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxi
Running the Code Samples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxii
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvi
Errata & Book Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii
We Want to Hear from You . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxvii
Stay in Touch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xxviii
PART I
A BIT of BACkgRounD
Chapter 1 software Development in windows
3
Windows Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3
Windows Release History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Supported CPU Architectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
Windows Build Flavors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
Windows Servicing Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Windows Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7
Kernel Mode vs. User Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
User-Mode System Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
User-Mode Application Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . .10
Low-Level Windows Communication Mechanisms . . . . . . . . . . . . . .13
Windows Developer Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16
v
www.it-ebooks.info
Developer Documentation Resources . . . . . . . . . . . . . . . . . . . . . . . . .16
WDM, KMDF, and UMDF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17
The NTDLL and USER32 Layers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
The Win32 API Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18
The COM Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19
The CLR (.NET) Layer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25
Microsoft Developer Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28
The Windows DDK (WDK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
The Windows SDK . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30
PART II
DEBuggIng foR fun AnD PRofIT
Chapter 2 getting started
33
Introducing the Debugging Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34
Acquiring the Windows Debuggers Package . . . . . . . . . . . . . . . . . . .34
Acquiring the Visual Studio Debugger . . . . . . . . . . . . . . . . . . . . . . . . .38
Comparing the WinDbg and Visual Studio Debuggers . . . . . . . . . .38
User-Mode Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39
Debugging Your First Program with WinDbg . . . . . . . . . . . . . . . . . . .39
Listing the Values of Local Variables and Function Parameters . . . .47
Source-Level Debugging in WinDbg . . . . . . . . . . . . . . . . . . . . . . . . . .52
Symbol Files, Servers, and Local Caches . . . . . . . . . . . . . . . . . . . . . . .53
Caching Symbols Offline for WinDbg . . . . . . . . . . . . . . . . . . . . . . . . .55
Troubleshooting Symbol Resolution Issues in WinDbg . . . . . . . . . . .56
Name Decoration Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57
Getting Help for WinDbg Commands . . . . . . . . . . . . . . . . . . . . . . . . .58
Kernel-Mode Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60
Your First (Live) Kernel Debugging Session . . . . . . . . . . . . . . . . . . . . .61
Setting Up a Kernel-Mode Debugging Environment Using
Physical Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67
Setting Up a Kernel-Mode Debugging Environment Using
Virtual Machines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .73
vi
Contents
www.it-ebooks.info
Diagnosing Host/Target Communication Issues . . . . . . . . . . . . . . . .76
Understanding the KD Break-in Sequence . . . . . . . . . . . . . . . . . . . . .77
Controlling the Target in the Kernel Debugger . . . . . . . . . . . . . . . . .78
Setting Code Breakpoints in the Kernel Debugger . . . . . . . . . . . . . .81
Getting Help for WinDbg Kernel Debugging Commands . . . . . . . .83
Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .83
Chapter 3 how windows Debuggers work
85
User-Mode Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .85
Architecture Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .86
Win32 Debugging APIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .87
Debug Events and Exceptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .88
The Break-in Sequence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .91
Setting Code Breakpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .93
Observing Code Breakpoint Insertion in WinDbg . . . . . . . . . . . . . . .93
Kernel-Mode Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .98
Architecture Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .98
Setting Code Breakpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100
Single-Stepping the Target . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .100
Switching the Current Process Context . . . . . . . . . . . . . . . . . . . . . . .101
Managed-Code Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103
Architecture Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .103
The SOS Windows Debuggers Extension . . . . . . . . . . . . . . . . . . . . . .106
Script Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .112
Architecture Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .112
Debugging Scripts in Visual Studio . . . . . . . . . . . . . . . . . . . . . . . . . . .114
Remote Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116
Architecture Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .116
Remote Debugging in WinDbg . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .117
Remote Debugging in Visual Studio . . . . . . . . . . . . . . . . . . . . . . . . . .121
Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .123
vii
www.it-ebooks.info
Chapter 4 Postmortem Debugging
125
Just-in-Time Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .125
Your First JIT Debugging Experiment . . . . . . . . . . . . . . . . . . . . . . . . .126
How Just-in-Time Debugging Works . . . . . . . . . . . . . . . . . . . . . . . . .128
Using Visual Studio as Your JIT Debugger . . . . . . . . . . . . . . . . . . . . .132
Run-Time Assertions and JIT Debugging . . . . . . . . . . . . . . . . . . . . . .138
JIT Debugging in Session 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .139
Dump Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .139
Automatic User-Mode, Crash-Dump Generation . . . . . . . . . . . . . . .139
Analyzing Crash Dumps Using the WinDbg Debugger . . . . . . . . .143
Analyzing Crash Dumps in Visual Studio . . . . . . . . . . . . . . . . . . . . . .150
Manual Dump-File Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .151
“Time Travel” Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .153
Kernel-Mode Postmortem Debugging . . . . . . . . . . . . . . . . . . . . . . .153
Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .157
Chapter 5 Beyond the Basics
159
Noninvasive Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .159
Data Breakpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .162
Deep Inside User-Mode and Kernel-Mode Data Breakpoints . . . .163
Clearing Kernel-Mode Data Breakpoints . . . . . . . . . . . . . . . . . . . . . .165
Execution Data Breakpoints vs. Code Breakpoints . . . . . . . . . . . . . .166
User-Mode Debugger Data Breakpoints in Action: C++
Global Objects and the C Runtime Library . . . . . . . . . . . . . . . . . . . .168
Kernel-Mode Debugger Data Breakpoints in Action:
Waiting for a Process to Exit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .170
Advanced Example: Who Is Changing a Registry Value? . . . . . . . .172
Scripting the Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .176
Replaying Commands Using Debugger Scripts . . . . . . . . . . . . . . . .176
Debugger Pseudo-Registers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .178
Resolving C++ Template Names in Debugger Scripts . . . . . . . . . . .180
Scripts in Action: Listing Windows Service Processes in the
Kernel Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .181
viii Contents
www.it-ebooks.info