Python Digital Forensics Cookbook

Effective Python recipes for digital investigations


Preston Miller
Chapin Bryce

BIRMINGHAM - MUMBAI


Python Digital Forensics Cookbook
Copyright © 2017 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embedded in
critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of
the information presented. However, the information contained in this book is sold
without warranty, either express or implied. Neither the authors, nor Packt
Publishing, and its dealers and distributors will be held liable for any damages
caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.

First published: September 2017

Production reference: 1220917

Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-78398-746-7


www.packtpub.com


Credits

Authors
Copy Editor
Preston Miller
Stuti Srivastava
Chapin Bryce

Reviewer

Project Coordinator

Dr. Michael Spreitzenbarth

Virginia Dias

Commissioning Editor

Proofreader


Kartikey Pandey

Safis Editing


Acquisition Editor

Indexer

Rahul Nair

Aishwarya Gangawane

Content Development Editor

Graphics

Sharon Raj

Kirk D'Penha

Technical Editor

Production Coordinator

Prashant Chaudhari

Aparna Bhagat



About the Authors
Preston Miller is a consultant at an internationally recognized risk management
firm. He holds an undergraduate degree from Vassar College and a master’s degree
in Digital Forensics from Marshall University. While at Marshall, Preston
unanimously received the prestigious J. Edgar Hoover Foundation’s Scientific
Scholarship. He is a published author, recently of Learning Python for Forensics, an
introductory Python Forensics textbook. Preston is also a member of the GIAC
advisory board and holds multiple industry-recognized certifications in his field.

Chapin Bryce works as a consultant in digital forensics, focusing on litigation
support, incident response, and intellectual property investigations. After studying
computer and digital forensics at Champlain College, he joined a firm leading the
field of digital forensics and investigations. In his downtime, Chapin enjoys working
on side projects, hiking, and skiing (if the weather permits). As a member of multiple
ongoing research and development projects, he has authored several articles in
professional and academic publications.


About the Reviewer
Dr. Michael Spreitzenbarth, after finishing his diploma thesis with the major topic
of mobile phone forensics, worked as a freelancer in the IT security sector for
several years. In 2013, he finished his PhD at the University of Erlangen-Nuremberg
in the field of Android forensics and mobile malware analysis. Since then, he has
been working as a team lead in an internationally operating CERT.
Dr. Michael Spreitzenbarth's daily work deals with the security of mobile systems,
forensic analysis of smartphones and suspicious mobile applications, as well as the
investigation of security-related incidents within ICS environments. At the same
time he is working on the improvement of mobile malware analysis techniques and
research in the field of Android and iOS forensics as well as mobile application

testing.


www.PacktPub.com
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF
and ePub files available? You can upgrade to the eBook version at www.PacktPub.com
and as a print book customer, you are entitled to a discount on the eBook copy. Get
in touch with us at for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up
for a range of free newsletters and receive exclusive discounts and offers on Packt
books and eBooks.

/>
Get the most in-demand software skills with Mapt. Mapt gives you full access to all
Packt books and video courses, as well as industry-leading tools to help you plan
your personal development and advance your career.


Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser


Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our
editorial process. To help us improve, please leave us an honest review on this
book's Amazon page at />If you'd like to join our team of regular reviewers, you can email us at
We award our regular reviewers with free eBooks and

videos in exchange for their valuable feedback. Help us be relentless in improving
our products!


To my mother, Mary, whose love, courage, and guidance have had an indelible
impact on me.
I love you very much.
Preston Miller

This book is dedicated to the love of my life and my best friend, Alexa.
Thank you for all of the love, support, and laughter.
Chapin Bryce


Table of Contents
Preface
What this book covers
What you need for this book
Who this book is for
Sections
Getting ready
How to do it…
How it works…
There's more…
See also
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book

Errata
Piracy
Questions
1. Essential Scripting and File Information Recipes
Introduction
Handling arguments like an adult
Getting started
How to do it…
How it works…
There's more…
Iterating over loose files
Getting started
How to do it…
How it works…
There's more…
Recording file attributes
Getting started
How to do it…
How it works…
There's more…
Copying files, attributes, and timestamps
Getting started
How to do it…


How it works…
There's more…
Hashing files and data streams
Getting started
How to do it…

How it works…
There's more…
Keeping track with a progress bar
Getting started
How to do it…
How it works…
There's more…
Logging results
Getting started
How to do it…
How it works…
There’s more…
Multiple hands make light work
Getting started
How to do it…
How it works…
There's more…
2. Creating Artifact Report Recipes
Introduction
Using HTML templates
Getting started
How to do it...
How it works...
There's more...
Creating a paper trail
Getting started
How to do it...
How it works...
There's more...
Working with CSVs

Getting started
How to do it...
How it works...
There's more...
Visualizing events with Excel
Getting started


How to do it...
How it works...
Auditing your work
Getting started
How to do it...
How it works...
There's more...
3. A Deep Dive into Mobile Forensic Recipes
Introduction
Parsing PLIST files
Getting started
How to do it...
How it works...
There's more…
Handling SQLite databases
Getting started
How to do it...
How it works...
Identifying gaps in SQLite databases
Getting started
How to do it...
How it works...

See also
Processing iTunes backups
Getting started
How to do it...
How it works...
There's more...
Putting Wi-Fi on the map
Getting started
How to do it...
How it works...
Digging deep to recover messages
Getting started
How to do it...
How it works...
There's more…
4. Extracting Embedded Metadata Recipes
Introduction
Extracting audio and video metadata
Getting started


How to do it...
How it works...
There's more...
The big picture
Getting started
How to do it...
How it works...
There's more...
Mining for PDF metadata

Getting started
How to do it...
How it works...
There's more...
Reviewing executable metadata
Getting started
How to do it...
How it works...
There's more...
Reading office document metadata
Getting started
How to do it...
How it works...
Integrating our metadata extractor with EnCase
Getting started
How to do it...
How it works...
There's more...
5. Networking and Indicators of Compromise Recipes
Introduction
Getting a jump start with IEF
Getting started
How to do it...
How it works...
Coming into contact with IEF
Getting started
How to do it...
How it works...
Beautiful Soup
Getting started

How to do it...
How it works...


There's more...
Going hunting for viruses
Getting started
How to do it...
How it works...
Gathering intel
Getting started
How to do it...
How it works...
Totally passive
Getting started
How to do it...
How it works...
6. Reading Emails and Taking Names Recipes
Introduction
Parsing EML files
Getting started
How to do it...
How it works...
Viewing MSG files
Getting started
How to do it...
How it works...
There’s more...
See also
Ordering Takeout

Getting started
How to do it...
How it works...
There’s more...
What’s in the box?!
Getting started
How to do it...
How it works...
Parsing PST and OST mailboxes
Getting started
How to do it...
How it works...
There’s more...
See also
7. Log-Based Artifact Recipes


Introduction
About time
Getting started
How to do it...
How it works...
There's more...
Parsing IIS web logs with RegEx
Getting started
How to do it...
How it works...
There's more...
Going spelunking
Getting started

How to do it...
How it works...
There's more...
Interpreting the daily.out log
Getting started
How to do it...
How it works...
Adding daily.out parsing to Axiom
Getting started
How to do it...
How it works...
Scanning for indicators with YARA
Getting started
How to do it...
How it works...
8. Working with Forensic Evidence Container Recipes
Introduction
Opening acquisitions
Getting started
How to do it...
How it works...
Gathering acquisition and media information
Getting started
How to do it...
How it works...
Iterating through files
Getting started
How to do it...



How it works...
There's more...
Processing files within the container
Getting started
How to do it...
How it works...
Searching for hashes
Getting started
How to do it...
How it works...
There's more...
9. Exploring Windows Forensic Artifacts Recipes - Part I
Introduction
One man's trash is a forensic examiner's treasure
Getting started
How to do it...
How it works...
A sticky situation
Getting started
How to do it...
How it works...
Reading the registry
Getting started
How to do it...
How it works...
There's more...
Gathering user activity
Getting started
How to do it...
How it works...

There's more...
The missing link
Getting started
How to do it...
How it works...
There's more...
Searching high and low
Getting started
How to do it...
How it works...
There's more...


10. Exploring Windows Forensic Artifacts Recipes - Part II
Introduction
Parsing prefetch files
Getting started
How to do it...
How it works...
There's more...
A series of fortunate events
Getting started
How to do it...
How it works...
There's more...
Indexing internet history
Getting started
How to do it...
How it works...
There's more...

Shadow of a former self
Getting started
How to do it...
How it works...
There's more...
Dissecting the SRUM database
Getting started
How to do it...
How it works...
There's more...
Conclusion


Preface
At the outset of this book, we strove to demonstrate a nearly endless corpus of use
cases for Python in today’s digital investigations. Technology plays an increasingly
large role in our daily life and shows no signs of stopping. Now, more than ever, it is
paramount that an investigator develop programming expertise to work with
increasingly large datasets. By leveraging the Python recipes explored throughout
this book, we make the complex simple, efficiently extracting relevant information
from large data sets. You will explore, develop, and deploy Python code and libraries
to provide meaningful results that can be immediately applied to your investigations.
Throughout the book, recipes include topics such as working with forensic evidence
containers, parsing mobile and desktop operating system artifacts, extracting
embedded metadata from documents and executables, and identifying indicators of
compromise. You will also learn how to integrate scripts with Application Program
Interfaces (APIs) such as VirusTotal and PassiveTotal, and tools, such as Axiom,
Cellebrite, and EnCase. By the end of the book, you will have a sound understanding
of Python and will know how you can use it to process artifacts in your
investigations.



What this book covers
Chapter 1, Essential Scripting and File Information Recipes, introduces you to the

conventions and basic features of Python used throughout the book. By the end of
the chapter, you will create a robust and useful data and metadata preservation script.
Chapter 2, Creating Artifact Report Recipes, demonstrates practical methods of

creating reports with forensic artifacts. From spreadsheets to web-based dashboards,
we show the flexibility and utility of various reporting formats.
Chapter 3, A Deep Dive into Mobile Forensic Recipes, features iTunes' backup

processing, deleted SQLite database record recovery, and mapping Wi-Fi access
point MAC addresses from Cellebrite XML reports.
Chapter 4, Extracting Embedded Metadata Recipes, exposes common file types

containing embedded metadata and how to extract it. We also provide you with
knowledge of how to integrate Python scripts with the popular forensic software,
EnCase.
Chapter 5, Networking and Indicators of Compromise Recipes, focuses on network

and web-based artifacts and how to extract more information from them. You will
learn how to preserve data from websites, interact with processed IEF results, create
hash sets for X-Ways, and identify bad domains or IP addresses.
Chapter 6, Reading Emails and Taking Names Recipes, explores the many file types

for both individual e-mail messages and entire mailboxes, including Google Takeout
MBox, and how to use Python for extraction and analysis.
Chapter 7, Log-Based Artifact Recipes, illustrates how to process artifacts from


several log formats, such as IIS, and ingest them with Python info reports or other
industry tools, such as Splunk. You will also learn how to develop and use Python
recipes to parse files and create artifacts within Axiom.
Chapter 8, Working with Forensic Evidence Container Recipes, shows off the basic

forensic libraries required to interact and process forensic evidence containers,
including EWF and raw formats. You will learn how to access data from forensic
containers, identify disk partition information, and iterate through filesystems.


Chapter 9, Exploring Windows Forensic Artifacts Recipes Part I, leverages the
framework developed in Chapter 8, Working with Forensic Evidence Container

Recipes, to process various Windows artifacts within forensic evidence containers.
These artifacts include $I Recycle Bin files, various Registry artifacts, LNK files,
and the Windows.edb index.
Chapter 10, Exploring Windows Forensic Artifacts Recipes Part II, continues to
leverage the framework developed in Chapter 8, Working with Forensic Evidence

Container Recipes, to process more Windows artifacts within forensic evidence
containers. These artifacts include Prefetch files, Event logs, Index.dat, Volume
Shadow Copies, and the Windows 10 SRUM database.


What you need for this book
In order to follow along with and execute the recipes within this cookbook, use a
computer with an Internet connection and the latest Python 2.7 and Python 3.5
installations. Recipes may require additional third-party libraries to be installed;
instructions for doing that are provided in the recipe.

For ease of development and implementation of these recipes, it is recommended
that you set up and configure an Ubuntu virtual machine for development. These
recipes, unless otherwise noted, were built and tested within an Ubuntu 16.04
environment with both Python 2.7 and 3.5. Several recipes will require the use of a
Windows operating system, as many forensic tools operate only on this platform.