Mastering Go

Create Golang production applications using network libraries,
concurrency, and advanced Go data structures

Mihalis Tsoukalos


BIRMINGHAM - MUMBAI



Mastering Go
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or
transmitted in any form or by any means, without the prior written permission of the
publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the
information presented. However, the information contained in this book is sold without
warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers
and distributors, will be held liable for any damages caused or alleged to have been caused
directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.
Acquisition Editors: Frank Pohlmann, Suresh Jain
Project Editor: Kishor Rit
Content Development Editor: Gary Schwarts
Technical Editors: Gaurav Gavas, Nidhisha Shetty
Proofreader: Tom Jacob

Indexer: Mariammal Chettiyar
Graphics: Tom Scaria
Production Coordinator: Shantanu Zagade
First published: April 2018
Production reference: 1270418
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-78862-654-5
www.packtpub.com


Packt Upsell

mapt.io

Mapt is an online digital library that gives you full access to over
5,000 books and videos, as well as industry leading tools to help
you plan your personal development and advance your career. For
more information, please visit our website.


Why subscribe?
Spend less time learning and more time coding with practical
eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable

Copy and paste, print, and bookmark content


PacktPub.com
Did you know that Packt offers eBook versions of every book
published, with PDF and ePub files available? You can upgrade to
the eBook version at www.PacktPub.com and as a print book customer,
you are entitled to a discount on the eBook copy. Get in touch with
us at for more details.
At www.PacktPub.com, you can also read a collection of free technical
articles, sign up for a range of free newsletters, and receive
exclusive discounts and offers on Packt books and eBooks.


Contributors


About the author
Mihalis Tsoukalos is a technical author, a Unix administrator, a
developer, and a mathematician, who enjoys learning new things.
He has written more than 250 technical articles for many
publications, including Sys Admin, MacTech, Linux User and
Developer, Usenix ;login:, Linux Format, and Linux Journal.
Mihalis is also the author of Go Systems Programming, by Packt
Publishing, 2017 and the technical editor for MongoDB in Action,
Second Edition, by Manning. Mihalis' research interests include
databases, operating systems, and statistics. You can reach him at ht
tp://www.mtsoukalos.eu/ and @mactsouk. He is also a photographer ( http://www.h
ighiso.net/).
I would like to thank the people at Packt Publishing for helping me write this book,

including Frank Pohlmann and Gary Schwartz, my technical reviewer, Mat Ryer,
Radhika Atitkar, for her encouragement and trust, and Kishor Rit, for answering all my
questions and encouraging me during the whole process.
For all people everywhere: You will never change your life until you change something
you do daily!


About the reviewer
Mat Ryer has been programming computers since he was 6 years
old. He would build games and programs, first in BASIC on a ZX
Spectrum and then in AmigaBASIC and AMOS on Commodore
Amiga with his father. Many hours were spent on manually
copying the code from the Amiga Format magazine and tweaking
variables or moving GOTO statements around to see what might
happen. The same spirit of exploration and obsession with
programming led Mat to starting work with a local agency in
Mansfield, England, when he was 18, where he started to build
websites and other online services.
After several years of working with various technologies and
industries in London and around the world, Mat noticed a new
systems language called Go that Google was pioneering. Since it
addressed very pertinent and relevant modern technical challenges,
Mat started using it to solve problems while the language was still
in the beta stage. He has used it ever since. Mat contributes to
open-source projects and founded Go packages, including Testify,
Moq, Silk, and Is, as well as a macOS developer tool called BitBar.
In 2018, Mat co-founded Machine Box and still spends a lot of time
speaking at conferences, writing about Go on his blog, and is an
active member of the Go community.



What this book covers
, Go and the Operating System, begins by talking about the
history of Go and the advantages of Go before describing the godoc
utility and explaining how you can compile and execute Go
programs. After that, it talks about printing the output and getting
user input, working with the command-line arguments of a
program, and using log files. The last topic of the first chapter is
error handling, which plays a key role in Go.
Chapter 1

, Understanding Go Internals, discusses the Go garbage
collector and the way it operates. Then it talks about unsafe code
and the unsafe package, how to call C code from a Go program, and
how to call Go code from a C program. After that, it showcases the
use of the defer keyword and presents the strace(1) and dtrace(1)
utilities. In the remaining sections of the chapter, you will learn
how to find information about your Go environment and the use of
the Go assembler.
Chapter 2

, Working with Basic Go Data Types, talks about the data
types offered by Go, which includes arrays, slices, and maps as well
as Go pointers, constants, loops, and working with dates and times.
You would not want to miss this chapter!
Chapter 3

, The Uses of Composite Types, begins by teaching you about
Go structures and the struct keyword before discussing tuples,
strings, runes, byte slices, and string literals. The rest of the chapter

talks about regular expressions and pattern matching, the switch
statement, the strings package, the math/big package, and about
developing a key-value store in Go.
Chapter 4

, Enhancing Go Code with Data Structures, is about
developing your own data structures when the structures offered
by Go do not fit a particular problem. This includes developing
binary trees, linked lists, hash tables, stacks, and queues and
learning about their advantages. This chapter also showcases the
use of the structures found in the container standard Go package.
The last topic of this chapter is random number generation.
Chapter 5

, What You Might Not Know About Go Packages, is all
about packages and functions, which also includes the use of the
init() function, the syscall standard Go package, and the text/template
and html/template packages. This chapter will definitely make you a
Chapter 6


better Go developer!
, Reflection and Interfaces for All Seasons, discusses three
advanced Go concepts: reflection, interfaces, and type methods.
The last part of the chapter is about object oriented programming
in Go!
Chapter 7

, Telling a Unix System What to Do, is about systems
programming in Go, which includes subjects such as the flag

package for working with command-line arguments, handling Unix
signals, file input and output, the bytes package, and the io.Reader
and io.Writer interfaces. As I told you before, if you are really into
systems programming in Go, then getting Go Systems
Programming after reading Mastering Go is highly recommended!
Chapter 8

, Concurrency in Go – Goroutines, Channels, and Pipelines,
discusses goroutines, channels and pipelines, which is the Go way
of achieving concurrency. You will also learn about the differences
between processes, threads, and goroutines, and the sync package
and the way the Go scheduler operates.
Chapter 9

, Concurrency in Go – Advanced Topics, will continue from
the point where the previous chapter left off and make you a
master of goroutines and channels! You will learn more about the
Go scheduler, the use of the powerful select keyword and the
various types of Go channels as well as shared memory, mutexes,
the sync.Mutex type, and the sync.RWMutex type. The last part of the
chapter will talk about the context package, worker pools, and how to
detect race conditions.
Chapter 10

, Code Testing, Optimization, and Profiling, discusses code
testing, code optimization, and code profiling as well as about cross
compilation, creating documentation, benchmarking Go code,
creating example function, and finding unreachable Go code.
Chapter 11


, The Foundations of Network Programming in Go, is all
about the net/http package and how you can develop web clients and
web servers in Go. This also includes the use of the http.Response,
http.Request and http.Transport structures and the http.NewServeMux type. You
will even learn how to develop an entire website in Go!
Furthermore, in this chapter, you will learn how to read the
configuration of your network interfaces and how to perform DNS
lookups in Go.
Chapter 12


, Network Programming – Building Your Own Servers and
Clients, talks about creating UDP and TCP servers and clients in
Go, using the functionality offered by the net package. Other topics
included in this chapter are how to create RPC clients and servers
as well as develop a concurrent TCP server in Go and read raw
network packages!
Chapter 13


Packt is searching for
authors like you
If you're interested in becoming an author for Packt, please visit aut
hors.packtpub.com and apply today. We have worked with thousands of
developers and tech professionals, just like you, to help them share
their insight with the global tech community. You can make a
general application, apply for a specific hot topic that we are
recruiting an author for, or submit your own idea.



Table of Contents
Title Page
Copyright and Credits
Mastering Go
Packt Upsell
Why subscribe?
PacktPub.com
Contributors
About the author
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews

1.

Go and the Operating System
The structure of the book
The history of Go
Why learn Go?
Go advantages
Is Go perfect?
What is a preprocessor?

The godoc utility
Compiling Go code
Executing Go code
Two Go rules
You either use a Go package or do not include it
There is only one way to format curly braces
Downloading Go packages
Unix stdin, stdout, and stderr
About printing output
Using standard output
Getting user input
About := and =
Reading from standard input
Working with command-line arguments
About error output
Writing to log files
Logging levels
Logging facilities
Log servers


A Go program that sends information to log files
About log.Fatal()
About log.Panic()
Error handling in Go
The error data type
Error handling
Additional resources
Exercises
Summary


2.

Understanding Go Internals
The Go compiler
Garbage Collection
The Tricolor algorithm
More about the operation of the Go Garbage Collector
Unsafe code
About the unsafe package
Another example of the unsafe package
Calling C code from Go
Calling C code from Go using the same file
Calling C code from Go using separate files
The C code
The Go code
Mixing Go and C code
Calling Go functions from C code
The Go package
The C code
The defer keyword
Panic and Recover
Using the panic function on its own
Two handy Unix utilities
The strace tool
The dtrace tool
Your Go environment
The Go Assembler
Node Trees
Learning more about go build

General Go coding advices
Additional Resources
Exercises
Summary

3.

Working with Basic Go Data Types
Go loops
The for loop
The while loop
The range keyword
Examples of Go for loops


Go arrays
Multi-dimensional arrays
The shortcomings of Go arrays
Go slices
Performing basic operations on slices
Slices are being expanded automatically
Byte slices
The copy() function
Multidimensional slices
Another example of slices
Sorting slices using sort.slice()
Go maps
Storing to a nil map
When you should use a map?
Go constants

The constant generator iota
Go pointers
Dealing with times and dates
Working with times
Parsing times
Working with dates
Parsing dates
Changing date and time formats
Additional resources
Exercises
Summary

4.

The Uses of Composite Types
About composite types
Structures
Pointers to structures
Using the new keyword
Tuples
Regular expressions and pattern matching
Now for some theory
A simple example
A more advanced example
Matching IPv4 addresses
Strings
What is a rune?
The Unicode package
The strings package
The switch statement

Calculating Pi with great accuracy
Developing a key/value store in Go
Additional resources
Exercises


Summary

5.

Enhancing Go Code with Data Structures
About graphs and nodes
Algorithm complexity
Binary trees in Go
Implementing a binary tree in Go
Advantages of binary trees
Hash tables in Go
Implementing a hash table in Go
Implementing the lookup functionality
Advantages of hash tables
Linked lists in Go
Implementing a linked list in Go
Advantages of linked lists
Doubly linked lists in Go
Implementing a doubly linked list in Go
Advantages of doubly linked lists
Queues in Go
Implementing a queue in Go
Stacks in Go
Implementing a stack in Go

The container package
Using container/heap
Using container/list
Using container/ring
Generating random numbers
Generating random strings
Additional Resources
Exercises
Summary

6.

What You Might Not Know About Go Packages
About Go packages
About Go functions
Anonymous functions
Functions that return multiple values
The return values of a function can be named!
Functions with pointer parameters
Functions that return pointers
Functions that return other functions
Functions that accept other functions as parameters
Developing your own Go packages
Compiling a Go package
Private variables and functions
The init() function
Reading the Go code of a standard Go package
Exploring the code of the net/url package
Looking at the Go code of the log/syslog package
Creating good Go packages



The syscall package
Finding out how fmt.Println() really works
Text and HTML templates
Generating text output
Constructing HTML output
Basic SQLite3 commands
Additional resources
Exercises
Summary

7.

Reflection and Interfaces for All Seasons
Type methods
Go interfaces
About type assertion
Developing your own interfaces
Using a Go interface
Using switch with interface and data types
Reflection
A simple Reflection example
A more advanced reflection example
The three disadvantages of reflection
Object-oriented programming in Go!
Additional resources
Exercises
Summary


8.

Telling a Unix System What to Do
About Unix processes
The flag package
The io.Reader and io.Writer interfaces
Buffered and unbuffered file input and output
The bufio package
Reading text files
Reading a text file line by line
Reading a text file word by word
Reading a text file character by character
Reading from /dev/random
Reading the amount of data you want from a file
Why are we using binary format?
Reading CSV files
Writing to a file
Loading and saving data on disk
The strings package revisited
About the bytes package
File permissions
Handling Unix signals
Handling two signals
Handling all signals


Programming Unix pipes in Go
Implementing the cat(1) utility in Go
Traversing directory trees
Using eBPF from Go

About syscall.PtraceRegs
Tracing system calls
User ID and group ID
Additional resources
Exercises
Summary

9.

Go Concurrency – Goroutines, Channels, and Pipelines
About processes, threads, and goroutines
The Go scheduler
Concurrency and parallelism
Goroutines
Creating a goroutine
Creating multiple goroutines
Waiting for your goroutines to finish
What if the number of Add() and Done() calls do not agree?
Channels
Writing to a channel
Reading from a channel
Channels as function parameters
Pipelines
Additional resources
Exercises
Summary

10.

Go Concurrency – Advanced Topics

The Go scheduler revisited
The GOMAXPROCS environment variable
The select keyword
Timing out a goroutine
Timing out a goroutine – take 1
Timing out a goroutine – take 2
Go channels revisited
Signal channels
Buffered channels
Nil channels
Channel of channels
Specifying the order of execution for your goroutines
Shared memory and shared variables
The sync.Mutex type
What happens if you forget to unlock a mutex?
The sync.RWMutex type
Sharing memory using goroutines
Catching race conditions
The context package
An advanced example of the context package


Worker pools
Additional resources
Exercises
Summary

11.

Code Testing, Optimization, and Profiling

The Go version used in this chapter
Comparing Go version 1.10 with Go version 1.9
Installing a beta or RC version of Go
About optimization
Optimizing Go code
Profiling Go code
The net/http/pprof standard Go package
A simple profiling example
A convenient external package for profiling
The web interface of the Go profiler
A profiling example that uses the web interface
A quick introduction to Graphviz
The go tool trace utility
Testing Go code
Writing tests for existing Go code
Benchmarking Go code
A simple benchmarking example
A wrong benchmark function
Benchmarking buffered writing
Finding unreachable Go code
Cross-compilation
Creating example functions
Generating documentation
Additional resources
Exercises
Summary

12.

The Foundations of Network Programming in Go

About net/http, net, and http.RoundTripper
The http.Response type
The http.Request type
The http.Transport type
About TCP/IP
About IPv4 and IPv6
The nc(1) command-line utility
Reading the configuration of network interfaces
Performing DNS lookups
Getting the NS records of a domain
Getting the MX records of a domain
Creating a web server in Go
Profiling an HTTP server
Creating a website in Go
HTTP tracing


Testing HTTP handlers
Creating a web client in Go
Making your Go web client more advanced
Timing out HTTP connections
More information about SetDeadline
Setting the timeout period on the server side
Yet another way to time out!
Wireshark and tshark tools
Additional resources
Exercises
Summary

13.


Network Programming – Building Servers and Clients
The net standard Go package
A TCP client
A slightly different version of the TCP client
A TCP server
A slightly different version of the TCP server
A UDP client
Developing a UDP server
A concurrent TCP server
A handy concurrent TCP server
Remote Procedure Call (RPC)
The RPC client
The RPC server
Doing low-level network programming
Grabbing raw ICMP network data
Where to go next?
Additional resources
Exercises
Summary
Other Books You May Enjoy
Leave a review - let other readers know what you think


Preface
The book you are reading right now is called Mastering Go and is
all about helping you become a better Go developer!
I tried to include the right amount of theory and hands on practice,
but only you, the reader, can tell if I succeeded or not! Additionally,
all presented examples are self-contained, which means that they

can be used on their own or as templates for creating more
complex applications.
Please try to do the exercises located at the end of each chapter and
do not hesitate to contact me with ways to make any future
editions of this book even better!


Who this book is for
This book is for amateur and intermediate Go programmers who
want to take their Go knowledge to the next level as well as for
experienced developers in other programming languages who want
to learn Go without learning again how a for loop works.
Some of the information found in this book can be also found in
my other book, Go Systems Programming by Packt Publishing.
The main difference between these two books is that Go Systems
Programming is about developing system tools using the
capabilities of Go, whereas Mastering Go is about explaining the
capabilities and the internals of Go in order to become a better Go
developer. Both books can be used as a reference after reading
them for the first or the second time.