• Navigate, comment, and modify disassembly
• Identify known library routines, so you can focus your
analysis on other areas of the code

Whether you’re analyzing malware, conducting vulnerability research, or reverse engineering software, a mastery
of IDA Pro is crucial to your success. Take your skills to the
next level with this 2nd edition of The IDA Pro Book.
ABOUT THE AUTHOR

Chris Eagle is a Senior Lecturer of Computer Science
at the Naval Postgraduate School in Monterey, CA.
He is the author of many IDA plug-ins and co-author of
Gray Hat Hacking (McGraw-Hill), and he has spoken
at numerous security conferences, including Blackhat,
Defcon, Toorcon, and Shmoocon.

$69.95 ($79.95 CDN)
SHELVE IN:
PROGRAMMING/
SOFTWARE DEVELOPMENT

“ I L I E F L AT .”
This book uses a lay-flat binding that won’t snap shut.

UNOFFICIAL GUIDE TO THE

W O R L D’ S

M O S T

P O P U L A R

D I S A S S E M B L E R

CHRIS E AGLE
“I wholeheartedly recommend The
IDA Pro Book to all IDA Pro users.”
—Ilfak Guilfanov,
creator of IDA Pro

E BP

P

B
SU

E AGL E

w w w.nostarch.com

T H E

JM

T H E F I N E ST I N G E E K E N T E RTA I N M E N T ™

BOOK

B


• Use IDA’s built-in debugger to tackle hostile and
obfuscated code

• Explore popular plug-ins that make writing IDA scripts
easier, allow collaborative reverse engineering, and
much more

IDA PRO

SU

B
SU

Hailed by the creator of IDA Pro as “profound, comprehensive, and accurate,” the second edition of The IDA
Pro Book covers everything from the very first steps to
advanced automation techniques. You’ll find complete
coverage of IDA’s new Qt-based user interface, as
well as increased coverage of the IDA debugger, the
Bochs debugger, and IDA scripting (especially using
IDAPython). But because humans are still smarter than
computers, you’ll even learn how to use IDA’s latest
interactive and scriptable interfaces to your advantage.
Save time and effort as you learn to:

• Extend IDA to support new processors and filetypes
using the SDK

D N
N O

2 TI
I
D

E

• Use code graphing to quickly make sense of crossreferences and function calls

2ND EDITION

THE IDA PRO BOOK

P
No source code? No problem. With IDA Pro, the interactive disassembler, you live in a source code–optional
world. IDA can automatically analyze the millions of
opcodes that make up an executable and present you
with a disassembly. But at that point, your work is just
beginning. With The IDA Pro Book, you’ll learn how
to turn that mountain of mnemonics into something you
can actually use.

E BP

P

JM

E BP

THE


JM

IDA PRO
DE-OBF USC AT E D



PRAISE FOR THE FIRST EDITION OF THE IDA PRO BOOK
“I wholeheartedly recommend The IDA Pro Book to all IDA Pro users.”
—ILFAK GUILFANOV, CREATOR OF IDA PRO
“A very concise, well laid out book. . . . The step by step examples, and much
needed detail of all aspects of IDA alone make this book a good choice.”
—CODY PIERCE, TIPPINGPOINT DVLABS
“Chris Eagle is clearly an excellent educator, as he makes the sometimes very
dense and technically involved material easy to read and understand and also
chooses his examples well.”
—DINO DAI ZOVI, TRAIL OF BITS BLOG
“Provides a significantly better understanding not of just IDA Pro itself, but
of the entire RE process.”
—RYAN LINN, THE ETHICAL HACKER NETWORK
“This book has no fluff or filler, it’s solid information!”
—ERIC HULSE, CARNAL0WNAGE BLOG
“The densest, most accurate, and, by far, the best IDA Pro book ever
released.”
—PIERRE VANDEVENNE, OWNER AND CEO OF DATARESCUE SA
“I highly recommend this book to anyone, from the person looking to begin
using IDA Pro to the seasoned veteran.”
—DUSTIN D. TRAMMELL, SECURITY RESEARCHER
“This book does definitely get a strong buy recommendation from me. It’s

well written and it covers IDA Pro more comprehensively than any other
written document I am aware of (including the actual IDA Pro Manual).”
—SEBASTIAN PORST, SENIOR SOFTWARE SECURITY ENGINEER, MICROSOFT
“Whether you need to solve a tough runtime defect or examine your
application security from the inside out, IDA Pro is a great tool and this book
is THE guide for coming up to speed.”
—JOE STAGNER, PROGRAM MANAGER, MICROSOFT



THE IDA PRO BOOK
2ND EDITION
The Unofficial Guide to the
World’s Most Popular
Disassembler

by Chris Eagle

San Francisco


THE IDA PRO BOOK, 2ND EDITION. Copyright © 2011 by Chris Eagle.
All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or
mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior
written permission of the copyright owner and the publisher.
Printed in Canada
15 14 13 12 11

123456789


ISBN-10: 1-59327-289-8
ISBN-13: 978-1-59327-289-0
Publisher: William Pollock
Production Editor: Alison Law
Cover and Interior Design: Octopod Studios
Developmental Editor: Tyler Ortman
Technical Reviewer: Tim Vidas
Copyeditor: Linda Recktenwald
Compositor: Alison Law
Proofreader: Paula L. Fleming
Indexer: BIM Indexing & Proofreading Services
For information on book distributors or translations, please contact No Starch Press, Inc. directly:
No Starch Press, Inc.
38 Ringold Street, San Francisco, CA 94103
phone: 415.863.9900; fax: 415.863.9950; ; www.nostarch.com
The Librar y of Congress has cataloged the first edition as follows:
Eagle, Chris.
The IDA Pro book : the unofficial guide to the world's most popular disassembler / Chris Eagle.
p. cm.
Includes bibliographical references and index.
ISBN-13: 978-1-59327-178-7
ISBN-10: 1-59327-178-6
1. IDA Pro (Electronic resource) 2. Disassemblers (Computer programs) 3. Debugging in computer science.
Title.
QA76.76.D57E245 2008
005.1'4--dc22
2008030632

I.


No Starch Press and the No Starch Press logo are registered trademarks of No Starch Press, Inc. Other product and
company names mentioned herein may be the trademarks of their respective owners. Rather than use a trademark
symbol with every occurrence of a trademarked name, we are using the names only in an editorial fashion and to the
benefit of the trademark owner, with no intention of infringement of the trademark.
The information in this book is distributed on an “As Is” basis, without warranty. While every precaution has been
taken in the preparation of this work, neither the author nor No Starch Press, Inc. shall have any liability to any
person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the
information contained in it.


This book is dedicated to my mother.



BRIEF CONTENTS
Acknowledgments .........................................................................................................xix
Introduction ..................................................................................................................xxi

PART I: INTRODUCTION TO IDA
Chapter 1: Introduction to Disassembly ..............................................................................3
Chapter 2: Reversing and Disassembly Tools ....................................................................15
Chapter 3: IDA Pro Background......................................................................................31

PART II: BASIC IDA USAGE
Chapter 4: Getting Started with IDA ................................................................................43
Chapter 5: IDA Data Displays.........................................................................................59
Chapter 6: Disassembly Navigation ................................................................................79
Chapter 7: Disassembly Manipulation ...........................................................................101
Chapter 8: Datatypes and Data Structures......................................................................127
Chapter 9: Cross-References and Graphing....................................................................167

Chapter 10: The Many Faces of IDA .............................................................................189

PART III: ADVANCED IDA USAGE
Chapter 11: Customizing IDA.......................................................................................201
Chapter 12: Library Recognition Using FLIRT Signatures...................................................211
Chapter 13: Extending IDA’s Knowledge .......................................................................227
Chapter 14: Patching Binaries and Other IDA Limitations.................................................237


PART IV: EXTENDING IDA’S CAPABILITIES
Chapter 15: IDA Scripting............................................................................................249
Chapter 16: The IDA Software Development Kit ..............................................................285
Chapter 17: The IDA Plug-in Architecture .......................................................................315
Chapter 18: Binary Files and IDA Loader Modules ..........................................................347
Chapter 19: IDA Processor Modules..............................................................................377

PART V: REAL-WORLD APPLICATIONS
Chapter 20: Compiler Personalities ...............................................................................415
Chapter 21: Obfuscated Code Analysis.........................................................................433
Chapter 22: Vulnerability Analysis ................................................................................475
Chapter 23: Real-World IDA Plug-ins.............................................................................499

PART VI: THE IDA DEBUGGER
Chapter 24: The IDA Debugger ....................................................................................513
Chapter 25: Disassembler/Debugger Integration ............................................................539
Chapter 26: Additional Debugger Features ....................................................................569

Appendix A: Using IDA Freeware 5.0 ...........................................................................581
Appendix B: IDC/SDK Cross-Reference..........................................................................585


Index .........................................................................................................................609

viii

Brief Contents


CONTENTS IN DETAIL
A C KN O W L E D G M E N T S

xix

INTRODUCTION

xxi

PART I
INTRODUCTION TO IDA
1
INTRODUCTION TO DISASSEMBLY

3

Disassembly Theory ................................................................................................... 4
The What of Disassembly ........................................................................................... 5
The Why of Disassembly ............................................................................................ 6
Malware Analysis ........................................................................................ 6
Vulnerability Analysis ................................................................................... 6
Software Interoperability ............................................................................... 7
Compiler Validation ..................................................................................... 7

Debugging Displays ..................................................................................... 7
The How of Disassembly ............................................................................................ 7
A Basic Disassembly Algorithm ...................................................................... 8
Linear Sweep Disassembly ............................................................................ 9
Recursive Descent Disassembly .................................................................... 11
Summary................................................................................................................ 14

2
REVERSING AND DISASSEMBLY TOOLS

15

Classification Tools.................................................................................................. 16
file ........................................................................................................... 16
PE Tools .................................................................................................... 18
PEiD ......................................................................................................... 19
Summary Tools ....................................................................................................... 20
nm ........................................................................................................... 20
ldd ........................................................................................................... 22
objdump ................................................................................................... 23
otool......................................................................................................... 24
dumpbin ................................................................................................... 25
c++filt ....................................................................................................... 25
Deep Inspection Tools .............................................................................................. 27
strings....................................................................................................... 27
Disassemblers ............................................................................................ 28
Summary................................................................................................................ 29


3

I D A P R O B AC K G R O U N D

31

Hex-Rays’ Stance on Piracy ...................................................................................... 32
Obtaining IDA Pro................................................................................................... 33
IDA Versions.............................................................................................. 33
IDA Licenses .............................................................................................. 33
Purchasing IDA .......................................................................................... 34
Upgrading IDA .......................................................................................... 34
IDA Support Resources............................................................................................. 35
Your IDA Installation ................................................................................................ 36
Windows Installation .................................................................................. 36
OS X and Linux Installation.......................................................................... 37
IDA and SELinux ........................................................................................ 38
32-bit vs. 64-bit IDA .................................................................................. 38
The IDA Directory Layout............................................................................. 38
Thoughts on IDA’s User Interface ............................................................................... 40
Summary................................................................................................................ 40

PART II
BASIC IDA USAGE
4
G E T T I N G S T A R T E D W IT H I D A

43

Launching IDA ........................................................................................................ 44
IDA File Loading ........................................................................................ 45
Using the Binary File Loader ........................................................................ 47

IDA Database Files.................................................................................................. 48
IDA Database Creation............................................................................... 50
Closing IDA Databases ............................................................................... 51
Reopening a Database ............................................................................... 52
Introduction to the IDA Desktop ................................................................................. 53
Desktop Behavior During Initial Analysis .................................................................... 56
IDA Desktop Tips and Tricks ..................................................................................... 57
Reporting Bugs ....................................................................................................... 58
Summary................................................................................................................ 58

5
IDA DATA DISPLAYS

59

The Principal IDA Displays........................................................................................ 60
The Disassembly Window ........................................................................... 60
The Functions Window ............................................................................... 66
The Output Window ................................................................................... 66
Secondary IDA Displays........................................................................................... 66
The Hex View Window............................................................................... 67
The Exports Window .................................................................................. 68
The Imports Window .................................................................................. 68

x

Contents in D etai l


The Structures Window ............................................................................... 69

The Enums Window.................................................................................... 70
Tertiary IDA Displays ............................................................................................... 70
The Strings Window ................................................................................... 70
The Names Window .................................................................................. 72
The Segments Window ............................................................................... 74
The Signatures Window.............................................................................. 74
The Type Libraries Window ......................................................................... 75
The Function Calls Window......................................................................... 76
The Problems Window................................................................................ 76
Summary................................................................................................................ 77

6
D I S A S S E M B L Y N A V IG A T I O N

79

Basic IDA Navigation .............................................................................................. 80
Double-Click Navigation ............................................................................. 80
Jump to Address......................................................................................... 82
Navigation History ..................................................................................... 82
Stack Frames .......................................................................................................... 83
Calling Conventions ................................................................................... 85
Local Variable Layout ................................................................................. 89
Stack Frame Examples ................................................................................ 89
IDA Stack Views......................................................................................... 93
Searching the Database ........................................................................................... 98
Text Searches ............................................................................................ 99
Binary Searches ......................................................................................... 99
Summary.............................................................................................................. 100


7
D I S A S S E M B L Y M A N IP U L A T I O N

101

Names and Naming.............................................................................................. 102
Parameters and Local Variables ................................................................. 102
Named Locations ..................................................................................... 103
Register Names........................................................................................ 105
Commenting in IDA ............................................................................................... 106
Regular Comments ................................................................................... 107
Repeatable Comments .............................................................................. 107
Anterior and Posterior Lines ....................................................................... 108
Function Comments .................................................................................. 108
Basic Code Transformations ................................................................................... 108
Code Display Options .............................................................................. 109
Formatting Instruction Operands................................................................. 112
Manipulating Functions ............................................................................. 113
Converting Data to Code (and Vice Versa).................................................. 119
Basic Data Transformations .................................................................................... 120
Specifying Data Sizes ............................................................................... 121
Working with Strings ................................................................................ 122
Specifying Arrays..................................................................................... 124
Summary.............................................................................................................. 126

Contents in D etai l

xi



8
DATATYPES AND DATA STRUCTURES

127

Recognizing Data Structure Use .............................................................................. 130
Array Member Access .............................................................................. 130
Structure Member Access .......................................................................... 135
Creating IDA Structures.......................................................................................... 142
Creating a New Structure (or Union) .......................................................... 142
Editing Structure Members......................................................................... 144
Stack Frames as Specialized Structures ....................................................... 146
Using Structure Templates....................................................................................... 146
Importing New Structures ....................................................................................... 149
Parsing C Structure Declarations ................................................................ 149
Parsing C Header Files ............................................................................. 150
Using Standard Structures ...................................................................................... 151
IDA TIL Files.......................................................................................................... 154
Loading New TIL Files ............................................................................... 155
Sharing TIL Files ....................................................................................... 155
C++ Reversing Primer ............................................................................................ 156
The this Pointer ........................................................................................ 156
Virtual Functions and Vtables ..................................................................... 157
The Object Life Cycle................................................................................ 160
Name Mangling ...................................................................................... 162
Runtime Type Identification ........................................................................ 163
Inheritance Relationships ........................................................................... 164
C++ Reverse Engineering References.......................................................... 165
Summary.............................................................................................................. 166


9
C R O S S - R E F E R E N C E S A N D G R AP H IN G

167

Cross-References ................................................................................................... 168
Code Cross-References ............................................................................. 169
Data Cross-References .............................................................................. 171
Cross-Reference Lists ................................................................................. 173
Function Calls .......................................................................................... 175
IDA Graphing....................................................................................................... 176
IDA External (Third-Party) Graphing ............................................................ 176
IDA’s Integrated Graph View..................................................................... 185
Summary.............................................................................................................. 187

10
T H E M AN Y F A C E S O F I D A

189

Console Mode IDA................................................................................................ 190
Common Features of Console Mode ........................................................... 190
Windows Console Specifics ...................................................................... 191
Linux Console Specifics............................................................................. 192
OS X Console Specifics ............................................................................ 194
Using IDA’s Batch Mode ........................................................................................ 196
Summary.............................................................................................................. 198

xii


C on t e n t s i n D e t a i l


PART III
ADVANCED IDA USAGE
11
C U ST OM IZ IN G ID A

201

Configuration Files ................................................................................................ 201
The Main Configuration File: ida.cfg .......................................................... 202
The GUI Configuration File: idagui.cfg........................................................ 203
The Console Configuration File: idatui.cfg ................................................... 206
Additional IDA Configuration Options ..................................................................... 207
IDA Colors .............................................................................................. 207
Customizing IDA Toolbars ......................................................................... 208
Summary.............................................................................................................. 210

12
L I B R A R Y R E C O G N I T I O N U S I NG F L IR T S I G N A T U R E S

211

Fast Library Identification and Recognition Technology............................................... 212
Applying FLIRT Signatures ...................................................................................... 212
Creating FLIRT Signature Files ................................................................................. 216
Signature-Creation Overview ..................................................................... 217
Identifying and Acquiring Static Libraries .................................................... 217
Creating Pattern Files................................................................................ 219

Creating Signature Files ............................................................................ 221
Startup Signatures .................................................................................... 224
Summary.............................................................................................................. 225

13
EXTENDING IDA’S KNOWLEDGE

227

Augmenting Function Information ............................................................................ 228
IDS Files.................................................................................................. 230
Creating IDS Files..................................................................................... 231
Augmenting Predefined Comments with loadint......................................................... 233
Summary.............................................................................................................. 235

14
P A T C H I N G B I N AR I E S A N D O TH E R I D A L I M IT A T I O N S

237

The Infamous Patch Program Menu.......................................................................... 238
Changing Individual Database Bytes .......................................................... 238
Changing a Word in the Database ............................................................ 239
Using the Assemble Dialog........................................................................ 239
IDA Output Files and Patch Generation.................................................................... 241
IDA-Generated MAP Files.......................................................................... 242
IDA-Generated ASM Files.......................................................................... 242
IDA-Generated INC Files........................................................................... 243
IDA-Generated LST Files ............................................................................ 243
IDA-Generated EXE Files ........................................................................... 243


Contents i n Detail

xiii


IDA-Generated DIF Files ............................................................................ 244
IDA-Generated HTML Files......................................................................... 245
Summary.............................................................................................................. 245

PART IV
EXTENDING IDA’S CAPABILITIES
15
IDA SCRIPTING

249

Basic Script Execution ............................................................................................ 250
The IDC Language................................................................................................. 252
IDC Variables .......................................................................................... 252
IDC Expressions ....................................................................................... 253
IDC Statements ........................................................................................ 254
IDC Functions .......................................................................................... 254
IDC Objects ............................................................................................ 256
IDC Programs .......................................................................................... 257
Error Handling in IDC ............................................................................... 258
Persistent Data Storage in IDC ................................................................... 259
Associating IDC Scripts with Hotkeys ....................................................................... 261
Useful IDC Functions .............................................................................................. 261
Functions for Reading and Modifying Data.................................................. 262

User Interaction Functions.......................................................................... 263
String-Manipulation Functions .................................................................... 264
File Input/Output Functions........................................................................ 264
Manipulating Database Names ................................................................. 266
Functions Dealing with Functions ................................................................ 266
Code Cross-Reference Functions................................................................. 267
Data Cross-Reference Functions.................................................................. 268
Database Manipulation Functions............................................................... 268
Database Search Functions........................................................................ 269
Disassembly Line Components ................................................................... 270
IDC Scripting Examples.......................................................................................... 270
Enumerating Functions .............................................................................. 270
Enumerating Instructions ............................................................................ 271
Enumerating Cross-References.................................................................... 272
Enumerating Exported Functions................................................................. 275
Finding and Labeling Function Arguments ................................................... 275
Emulating Assembly Language Behavior ..................................................... 278
IDAPython ............................................................................................................ 280
Using IDAPython ...................................................................................... 281
IDAPython Scripting Examples ................................................................................ 282
Enumerating Functions .............................................................................. 282
Enumerating Instructions ............................................................................ 282
Enumerating Cross-References.................................................................... 283
Enumerating Exported Functions................................................................. 283
Summary.............................................................................................................. 284

xiv

Contents in D etai l



16
THE IDA SOFTWARE DEVELOPMENT KIT

285

SDK Introduction ................................................................................................... 286
SDK Installation........................................................................................ 287
SDK Layout.............................................................................................. 287
Configuring a Build Environment ................................................................ 289
The IDA Application Programming Interface ............................................................. 289
Header Files Overview ............................................................................. 290
Netnodes ................................................................................................ 294
Useful SDK Datatypes ............................................................................... 302
Commonly Used SDK Functions.................................................................. 304
Iteration Techniques Using the IDA API........................................................ 310
Summary.............................................................................................................. 314

17
THE IDA PLUG-IN ARCHITECTURE

315

Writing a Plug-in ................................................................................................... 316
The Plug-in Life Cycle ................................................................................ 318
Plug-in Initialization .................................................................................. 320
Event Notification..................................................................................... 321
Plug-in Execution ...................................................................................... 322
Building Your Plug-ins ............................................................................................ 324
Installing Plug-ins ................................................................................................... 329

Configuring Plug-ins .............................................................................................. 330
Extending IDC ...................................................................................................... 331
Plug-in User Interface Options ................................................................................. 333
Using the SDK’s Chooser Dialogs ............................................................... 334
Creating Customized Forms with the SDK.................................................... 337
Windows-Only User Interface–Generation Techniques .................................. 341
User Interface Generation with Qt .............................................................. 342
Scripted Plug-ins.................................................................................................... 344
Summary.............................................................................................................. 346

18
B IN AR Y F IL E S A N D ID A L O AD E R M O D U L E S

347

Unknown File Analysis ........................................................................................... 348
Manually Loading a Windows PE File...................................................................... 349
IDA Loader Modules .............................................................................................. 358
Writing an IDA Loader Using the SDK ..................................................................... 358
The Simpleton Loader ............................................................................... 361
Building an IDA Loader Module ................................................................. 366
A pcap Loader for IDA.............................................................................. 366
Alternative Loader Strategies .................................................................................. 372
Writing a Scripted Loader ...................................................................................... 373
Summary.............................................................................................................. 375

C on t e n t s i n D e t a i l

xv



19
IDA PROCESSOR MODULES

377

Python Byte Code.................................................................................................. 378
The Python Interpreter ............................................................................................ 379
Writing a Processor Module Using the SDK .............................................................. 380
The processor_t Struct ............................................................................... 380
Basic Initialization of the LPH Structure ........................................................ 381
The Analyzer ........................................................................................... 385
The Emulator............................................................................................ 390
The Outputter........................................................................................... 394
Processor Notifications.............................................................................. 399
Other processor_t Members....................................................................... 401
Building Processor Modules .................................................................................... 403
Customizing Existing Processors .............................................................................. 407
Processor Module Architecture ................................................................................ 409
Scripting a Processor Module ................................................................................. 411
Summary.............................................................................................................. 412

PART V
REAL-WORLD APPLICATIONS
20
C O M PI L E R P E R S O N A L I T I E S

415

Jump Tables and Switch Statements ......................................................................... 416

RTTI Implementations ............................................................................................. 420
Locating main ....................................................................................................... 421
Debug vs. Release Binaries..................................................................................... 428
Alternative Calling Conventions .............................................................................. 430
Summary.............................................................................................................. 432

21
O B F U S CA T E D C O D E A N AL YS I S

433

Anti–Static Analysis Techniques............................................................................... 434
Disassembly Desynchronization ................................................................. 434
Dynamically Computed Target Addresses.................................................... 437
Imported Function Obfuscation .................................................................. 444
Targeted Attacks on Analysis Tools............................................................. 448
Anti–Dynamic Analysis Techniques .......................................................................... 449
Detecting Virtualization ............................................................................. 449
Detecting Instrumentation .......................................................................... 451
Detecting Debuggers ................................................................................ 452
Preventing Debugging .............................................................................. 453
Static De-obfuscation of Binaries Using IDA .............................................................. 454
Script-Oriented De-obfuscation................................................................... 455
Emulation-Oriented De-obfuscation ............................................................. 460
Virtual Machine-Based Obfuscation ......................................................................... 472
Summary.............................................................................................................. 474
xvi

Contents in D etai l



22
V U L N E R AB I L IT Y A N A L Y S I S

475

Discovering New Vulnerabilities with IDA................................................................. 476
After-the-Fact Vulnerability Discovery with IDA .......................................................... 483
IDA and the Exploit-Development Process ................................................................. 488
Stack Frame Breakdown ........................................................................... 488
Locating Instruction Sequences ................................................................... 492
Finding Useful Virtual Addresses ................................................................ 494
Analyzing Shellcode.............................................................................................. 495
Summary.............................................................................................................. 498

23
R E A L - W O R L D I D A P L U G- I N S

499

Hex-Rays.............................................................................................................. 500
IDAPython ............................................................................................................ 503
collabREate .......................................................................................................... 503
ida-x86emu .......................................................................................................... 506
Class Informer....................................................................................................... 506
MyNav ................................................................................................................ 508
IdaPdf.................................................................................................................. 509
Summary.............................................................................................................. 510

PART VI

THE IDA DEBUGGER
24
THE IDA DEBUGGER

513

Launching the Debugger ........................................................................................ 514
Basic Debugger Displays........................................................................................ 518
Process Control ..................................................................................................... 521
Breakpoints ............................................................................................. 522
Tracing ................................................................................................... 526
Stack Traces ............................................................................................ 528
Watches ................................................................................................. 529
Automating Debugger Tasks ................................................................................... 530
Scripting Debugger Actions ....................................................................... 530
Automating Debugger Actions with IDA Plug-ins........................................... 536
Summary.............................................................................................................. 538

25
DISASSEMBLER/DEBUGGER INTEGRATION

539

Background.......................................................................................................... 540
IDA Databases and the IDA Debugger..................................................................... 541
Debugging Obfuscated Code ................................................................................. 543
Launching the Process ............................................................................... 545
Simple Decryption and Decompression Loops .............................................. 546
Contents in D etai l


xvii


Import Table Reconstruction ....................................................................... 550
Hiding the Debugger ................................................................................ 555
IdaStealth............................................................................................................. 560
Dealing with Exceptions ......................................................................................... 561
Summary.............................................................................................................. 568

26
A D D IT I O N A L D E B U G G E R F E A T U R E S

569

Remote Debugging with IDA................................................................................... 569
Using a Hex-Rays Debugging Server .......................................................... 570
Attaching to a Remote Process ................................................................... 573
Exception Handling During Remote Debugging............................................ 574
Using Scripts and Plug-ins During Remote Debugging ................................... 574
Debugging with Bochs ........................................................................................... 574
Bochs IDB Mode ...................................................................................... 575
Bochs PE Mode........................................................................................ 576
Bochs Disk Image Mode............................................................................ 577
Appcall................................................................................................................ 578
Summary.............................................................................................................. 579

A
USING IDA FREEWARE 5.0

581


Restrictions on IDA Freeware .................................................................................. 582
Using IDA Freeware .............................................................................................. 583

B
IDC/SDK CROSS-REFERENCE

585

INDEX

609

xviii

C on t e n t s i n D e t a i l


ACKNOWLEDGMENTS

As with the first edition, I would like to thank my family
for putting up with me while I worked on this project.
I am ever grateful for their patience and tolerance.
I would also like to thank everyone who helped make the first edition
a success, in particular the readers who I hope have found it to be a useful
addition to their reverse engineering libraries. Without your support and
many kind words, this edition would never have been possible.
Once again I wish to thank my technical editor Tim Vidas for all of his
input over the course of this project, as well as his wife Sheila for allowing me
to borrow him a second time.

Thanks also to the developers at Hex-Rays, not only for the product you
have built but also for putting up with my “bug” reports, too many of which
turned out to be false alarms. Ilfak, you have as usual been more than generous with your time; Elias, Igor, and Daniel, you have all provided insights
that I could have obtained nowhere else. Together you all make IDA my
favorite piece of software.
Finally, I would like to thank Alison Law and everyone else at No Starch
Press for their hard work in keeping this version of the book moving along as
smoothly as I could ever have hoped.



INTRODUCTION

Writing a book about IDA Pro is a challenging task. The fact that it is a complex piece
of software with more features than can even
be mentioned, let alone detailed in a book of
reasonable size, is the least of the difficulties. New
releases of IDA also tend to occur frequently enough
that any book will almost certainly be one, if not two,

P
JM
SU
B

EB
P

versions behind by the time it hits the streets. Including version 5.3, which
was released just as the first edition was going to press, seven new versions of

IDA have been released since the first edition was published. The release of
version 6.0 with a new, Qt-based graphical user interface motivated me to
update the book and address many of the features that have been introduced
in the interim. Of course, true to form, another version of IDA (6.1) was
released late in the process just to make things more exciting.
My goal with this edition remains to help others get started with IDA and
perhaps develop an interest in reverse engineering in general. For anyone
looking to get into the reverse engineering field, I can’t stress how important


it is that you develop competent programming skills. Ideally, you should love
code, perhaps going so far as to eat, sleep, and breathe code. If programming
intimidates you, then reverse engineering is probably not for you. It is possible
to argue that reverse engineering requires no programming at all because all
you are doing is taking apart someone else’s program; however, without committing to developing scripts and plug-ins to help automate your work, you
will never become a truly effective reverse engineer. In my case, programming
and reverse engineering substitute for the challenge of The New York Times
Sunday crossword puzzle, so it is rarely tedious.
For continuity purposes, this edition preserves the overall structure of
the first edition while elaborating and adding material where appropriate.
There are a number of ways to read this book. Users with little reverse engineering background may wish to begin with Chapters 1 and 2 for some
background information on reverse engineering and disassemblers. Users
without much IDA experience who are looking to dive right in can begin
with Chapter 3, which discusses the basic layout of an IDA installation, while
Chapter 4 covers what goes on when you launch IDA and load a file for analysis. Chapters 5 through 7 discuss IDA’s user interface features and basic
capabilities.
Readers possessing some familiarity with IDA may wish to begin with
Chapter 8, which discusses how to use IDA to deal with complex data structures, including C++ classes. Chapter 9, in turn, covers IDA cross-references,
which are the foundation for IDA’s graph-based displays (also covered in
Chapter 9). Chapter 10 provides a bit of a diversion useful for readers interested in running IDA on non-Windows platforms (Linux or OS X).

More advanced IDA users may find Chapters 11 through 14 a good place
to start, because they cover some of the fringe uses of IDA and its companion
tools. A brief run-through of some of IDA’s configuration options is presented
in Chapter 11. Chapter 12 covers IDA’s FLIRT/FLAIR technology and related
tools that are used to develop and utilize signatures to distinguish library code
from application code. Chapter 13 offers some insight into IDA type libraries
and ways to extend them, while Chapter 14 addresses the much-asked question of whether IDA can be used to patch binary files.
IDA is a quite capable tool right out of the box; however, one of its
greatest strengths is its extensibility, which users have taken advantage of to
make IDA do some very interesting things over the years. IDA’s extensibility
features are covered in Chapters 15 through 19, which begin with coverage
of IDA’s scripting features, including increased coverage of IDAPython, and
follow with a systematic walk through IDA’s programming API, as provided
by its software development kit (SDK). Chapter 16 provides an overview of
the SDK, while Chapters 17 through 19 walk you through plug-ins, file
loaders, and processor modules.
With the bulk of IDA’s capabilities covered, Chapters 20 through 23
turn to more practical uses of IDA for reverse engineering by examining how
compilers differ (Chapter 20); how IDA may be used to analyze obfuscated
code, as is often encountered when analyzing malware (Chapter 21); and

xxii

I n t ro d u c t i o n


how IDA may be used in the vulnerability discovery and analysis process
(Chapter 22). Chapter 23 concludes the section by presenting some useful
IDA extensions (plug-ins) that have been published over the years.
The book concludes with expanded coverage of IDA’s built-in debugger

in Chapters 24 through 26. Chapter 24 begins by introducing the basic features of the debugger. Chapter 25 discusses some of the challenges of using
the debugger to examine obfuscated code, including the challenge of dealing with any anti-debugging feature that may be present. Chapter 26 concludes
the book with a discussion of IDA’s remote debugging capabilities and the
use of the Bochs emulator as an integrated debugging platform.
At the time of this writing, IDA version 6.1 was the most current version
available, and the book is written largely from a 6.1 perspective. Hex-Rays is
generous enough to make an older version of IDA available for free; the
freeware version of IDA is a reduced-functionality version of IDA 5.0. While
many of the IDA features discussed in the book apply to the freeware version
as well, Appendix A provides a brief rundown of some of the differences a
user of the freeware version can expect to encounter.
Finally, since it is a somewhat natural progression to begin with IDA
scripting and move on to creating compiled plug-ins, Appendix B provides a
complete mapping of every IDC function to its corresponding SDK counterparts. In some cases you will find a one-to-one correspondence between
an IDC function and an SDK function (though in all cases the names of
those functions are different); in other cases, you will find that several SDK
function calls are required to implement a single IDC function. The intent
of Appendix B is to answer questions along the lines of “I know how to do X
in IDC, how can I do X with a plug-in?” The information in Appendix B was
obtained by reverse engineering the IDA kernel, which is perfectly legal
under IDA’s atypical licensing agreement.
Throughout the book, I have tried to avoid long sequences of code in
favor of short sequences that demonstrate specific points. The vast majority
of sample code, along with many of the binary files used to generate examples,
is available on the book’s official website, where you
will also find additional examples not included in the book as well as a comprehensive list of references used throughout the book (such as live links to
all URLs referred in footnotes).

I n t ro d u c t i o n


xxiii