1
RPC Decodes – SANS 2001
RPC
Your Friend or Foe?
David Hoelzer
SANS 2001
Version 1.5
This page intentionally left blank.
2
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 2
Contents
RPC Basics 4
Portmapper 13
Decoding RPC 18
Anomalous Traffic 30
Appendices 58
This page intentionally left blank.
3
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 3
What you should learn
!
Understanding of what RPC’s are
!
Purpose of Portmap
!
Transport Methods
!
What RPC’s should look like
!

How to identify anomalous RPC traffic
Welcome to the wonderful world of RPCs!
The purpose of this course is to provide you with a basic understanding of how RPCs function, what
their purpose is and, most importantly, what they look like.
The idea is that if we can figure out how RPCs should look, we can identify “anomalous” RPCs, or
watch for particular types of RPCs with a minimum of effort.
4
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 4
RPC Basics
!
What are RPC’s anyway?
!
What are the advantages to using them?
!
How do they work?
!
What sorts of RPC’s are available?
So now it’s time to get to work. We’ll see if we can lay a groundwork of understanding for the RPC
paradigm, explain why it can be a powerful and valuable service, how they work and what sorts of
things are available through RPC calls.
5
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 5
What are RPCs for?
Imagine for a moment that you have a number of workstations, each of which needs to use a
particular application to operate on a dataset. I know that this sounds antiquated, but we’re trying to
get the mindset of the persons who came up with RPCs.
Imagine the issues involved with trying to synchronize the dataset across all of the hosts. You would
very shortly be motivated to come up with some sort of distributed file system to simplify your life.

However, since we’re talking about programmer types here, of course we’re going to go for the all-
encompassing universal solution!!
6
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 6
What are RPCs for?
In this slide, we have a picture of the service that we very very quickly realize that we need to have
to facilitate easy maintenance of our data set. But there’s actually much more to this! The “Data”
that is depicted beneath the central workstation, using RPCs, could be virtually ANY resource!
These resources thus become platform independent. In other words, we could create a solution to a
programming problem on one platform, design it as an RPC, and then access that RPC from any
other platform that supports RPCs!
7
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 7
RPC Basics
!
RPC : Remote Procedure Call
!
Provides transparent service to programmer
This page intentionally left blank.
8
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 8
RPC Basics
Advantages
!
Standard high level interface to services.
!
Easy way of providing a single point of service in a

networked environment
!
Good framework for Distributed Computing
Environments
Some persons coming into this class may have the notion, “RPCs are all bad. Why don’t you just not
use them?” As we can see from this brief discussion, though, RPCs fill a very important need in the
distributed computing environment.
9
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 9
RPC Basics
How do they work?
!
A local program makes a call to a “stub” function.
!
Stub function packs the data up and ships it to an
appropriate server
!
Server processes the request and ships pack the
response to the client
!
The stub function returns the unpacked data to the
calling program
We’ll take a very brief look at how RPCs work internally. A basic understanding of this is necessary
in order to understand the general flow of RPC traffic. At least, what that flow should look like!
By “Stub” function, what we mean is that the runtime library has a function defined in it that, in this
case, exists only to forward the request on to somewhere else. The actual functionality exists
somewhere else.
The four steps above are somewhat simplified. There is actually some more behind the scenes work
going on when we use an RPC resource.

10
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 10
RPC Basics
This slide is a simple representation of what we discussed on the previous slide. The flow is from
top to bottom, from left to right.
1. Our application has a need for some resource
2. This need is formulated into a function call for the resource (Resource Request)
3. The request is passed to the OS
4. Somewhere in the runtime library there is a realization that this is a “remote” resource. An RPC
request is formulated and dropped on the wire
5. The RPC Server receives the request and forwards it on into the OS/runtime library
6. The resource is accessed
The result from the RPC is passed back in roughly the reverse order.
11
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 11
RPC Basics
What sorts of RPC’s are there?
!
NFS
!
Yellow Pages Services (yppasswd)
!
rusers
!
rquota
!
NIS
!

sadmind
In truth, you can implement an RPC for absolutely anything. There is provision within the
specification for “User” space RPCs.
12
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 12
RPC Basics
How do we find RPC services on a system?
But now the real question. What happens behind the scenes that allows us to find the right service
on the remote machine?
13
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 13
The Portmapper
!
The portmapper maintains a directory of available
RPC services on a system
!
Allows a service to map to a particular version of
an RPC program
The portmapper is an essential part of the RPC paradigm. In fact, RPCs will not function without it.
We’ll go into more detail on it in the next few slides.
14
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 14
The Portmapper
How it works
Familiar error?
RPC Program not registered.
If you’ve ever seen this error, you were probably trying to do a quick NFS share/mount to facilitate

the transfer of a file system or something similar. Essentially, what this error is telling you is that the
portmapper has no idea where the program you are requesting is because it has not “registered” itself
as open for business.
When a program registers with the portmapper, it contacts the portmapper with the following
information:
•Which program number it is (more on that later)
•Which versions/protocols are available
•Which port/ports it is listening on.
15
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 15
The Portmapper
!
All services must register. ‘rpcinfo’ can be used to
see what’s what.
snoopy# rpcinfo -p
program vers proto port service
100000 4 tcp 111 rpcbind
100000 3 tcp 111 rpcbind
100000 2 tcp 111 rpcbind
100000 4 udp 111 rpcbind
100000 3 udp 111 rpcbind
100000 2 udp 111 rpcbind
100021 1 udp 4045 nlockmgr
100024 1 udp 32772 status
100024 1 tcp 32771 status
(Notes)
RPCInfo basically calls the “DUMP’ pragma on the portmapper. This asks the portmapper to dump
a list of all currently registered programs with their available versions and protocols.
Notice that there are six versions of “rpcbind” available. This is an internal name for “portmap”. In

reality, there are (most likely) not six separate daemons running on the system. More likely, the
daemon that is running and has registered itself with the portmapper understands (and is listening)
both UDP and TCP (and has bound both the UDP and the TCP ports required) as well as versions 2-
4 of portmap.
What if I could insert my own RPC program into the portmap list?
The portmapper has 4 different possible functions that can be called:
PMAPPROC_SET: Called to register a program number to a port number
PMAPPROC_UNSET: Called to remove a program from the portmapper directory
PMAPPROC_GETPORT: Called to retrieve the port number that a particular service and version
lives on
PMAPPROC_DUMP: Dumps a list of all programs, versions and port numbers
16
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 16
Traffic Decodes
(Anomalous)
00:06:55.753351 127.0.0.1.43862 > 10.0.0.20.111: udp 56 (DF) (ttl
248, id 55069)
00:18:55.780919 127.0.0.1.43877 > 10.0.0.20.111: udp 56 (DF) (ttl
248, id 54248)
00:30:55.644708 127.0.0.1.43894 > 10.0.0.20.111: udp 56 (DF) (ttl
248, id 53277)
00:42:55.590878 127.0.0.1.43909 > 10.0.0.20.111: udp 56 (DF) (ttl
248, id 52381)
00:54:55.535681 127.0.0.1.43927 > 10.0.0.20.111: udp 56 (DF) (ttl
248, id 51485)
01:04:55.443822 127.0.0.1.43939 > 10.0.0.20.111: udp 56 (DF) (ttl
248, id 61621)
01:14:55.400732 127.0.0.1.43953 > 10.0.0.20.111: udp 56 (DF) (ttl
248, id 6261)

01:24:55.356257 127.0.0.1.43966 > 10.0.0.20.111: udp 56 (DF) (ttl
248, id 16437)
01:34:55.309417 127.0.0.1.43981 > 10.0.0.20.111: udp 56 (DF) (ttl
248, id 26613)
This isn’t the anomalous decodes section, but since we’re talking about the portmapper and
registered services, we can’t put this one off til later.
10.0.0.20 is an internet visible machine providing some RPC services. While we don’t have any
more data than the packet headers above, what sort of activity could we be looking at here?
RPC Services are inserted into the portmapper only from the local machine and require root
privileges. Is this an attempt to either insert or delete a new service?
In this instance, an excellent correlation would be a subsequent request from a host for a dump to
check their work. In this case, the site in question wasn’t watching for port 111, they were watching
for illegal IP addresses, so we can’t be sure.
17
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 17
The Portmapper
!
Programs are identified by number. Clients
request the information for a service based on this
ID number.
0 - 1fffffff Defined by Sun
20000000 - 3fffffff User defined
40000000 - 5fffffff Transient
60000000 - ffffffff Reserved
This list comes from RFC 1050. While Sun is designated as the authority for RPC program numbers,
they unfortunately have not produced an exhaustive list of the RPC programs that have been
registered thus far. You can go to their website and request the information, but so far the only
response I’ve ever received is “We’re in the process of compiling the data and will forward it on
when it is complete.” I have found that various network information tools such as “Nmap” have a

fairly comprehensive list. If you obtain the Nmap source distribution, you will find various RPC
resource files enumerating most of the common services, what program # they are, and where they
commonly live port wise.
The next slide has a list of some of the more common ones.
18
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 18
The Portmapper
RPC Service Numbers
100000 Portmap (rpcbind)
100001 rstatd
100002 rusersd
100003 nfsd
100005 mountd
100008 walld
100011 rquota
100012 sprayd
100021 nlockmgr (nfs locking manager)
100024 status
100026 bootparam
100227 nfs_acl
100232 sadmind (Sun admin daemon)
Here are a few RPC Program numbers. While there are many many more, this is a list of a few of
the most common ones that I have seen probes for.
19
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 19
RPC Decode
What are RPCs supposed to look like?
Now that we understand some of the theory behind RPCs and how they can be helpful, it’s time to

start taking a look at what actual RPC decodes look like at the byte level. The information that was
used to put together the ‘pictures’ of what RPCs ought to look like comes from applying the RFC on
the XDR (External Data Representation) language and the RPC RFC (which is specified using
XDR). It would be a good idea to take some time to go through the XDR RFC and make sure you
understand the basics of how an XDR specification works since nearly all RPCs for which there are
RFCs are specified in this way.
This becomes important if you decide that you have a need to decode something further up the
application layer. For instance, if you have an interest in monitoring for particular NFS result codes,
you will need to work out the XDR definition of the NFS application.
20
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 20
RPC Decode
!
The RPC specification is written in XDR
!
RFC 1057 - RPC Version 2
!
RFC 1832 - XDR
Notes
RFC 1057 defines the actual internal structure of an RPC call. The format provides a prototype style
definition for where each piece of data falls within the header.
RFC 1832 defines the XDR for us. XDR is the “External Data Representation Standard”. Here are
some important pieces to know:
Basic Data Size: 4 bytes (double word)
Integer 4 bytes
Enum 4 bytes
Boolean 4 bytes (enum { FALSE = 0, TRUE = 1}
Hyper Int 8 bytes
Float 4 bytes (IEEE standard for normalized signle precision)

Double 8 bytes
Quad 16 bytes
Opaque 4 byte length followed by bytes padded to 4 byte boundary
String 4 byte length followed by bytes padded to 4 byte boundary
21
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 21
RPC Headers
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Transaction ID (XID) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Message Type (Call, 0) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| RPC Version |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| RPC Program |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Program Version |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Procedure Number |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|< Four Bytes Wide >|
This format is consistent for ALL RPC calls. We’re in for a wild ride for the rest though!
The RPC header will immediately follow the UDP header if a UDP transport is being used.
However, if TCP is the transport layer, then the RPC header will be preceded by a 4 byte length field
used to define the total length of the entire RPC that will follow.
22
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 22
RPC Headers

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Transaction ID |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Message Type (Reply, 1) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Status (Accepted = 0) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Verifier (Up to 408) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Accept Status |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|< Four Bytes Wide >|
This format is consistent for ALL RPC replies. We’re in for a wild ride for the rest though!
The RPC header will immediately follow the UDP header if a UDP transport is being used.
However, if TCP is the transport layer, then the RPC header will be preceded by a 4 byte length field
used to define the total length of the entire RPC that will follow.
We will not go into each of the various cases possible (the various Accept status reports), however
an value other than zero for the status indicates an error condition of some kind. Additionally, if the
Status is not equal to zero, the request has been refused or rejected. Either of these might be notable
events.
23
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 23
RPC Decode
10:24:46.396484 XXX.XXX.160.64.961453077 >
XXX.XXX.85.20.2049: 40 null (DF)
4500 0044 feca 4000 fd11 3486 xxxx a040
xxxx 5514 b5e7 0801 0030 9b40 394e 9c15
0000 0000 0000 0002 0001 86a3 0000 0003
0000 0000 0000 0000 0000 0000 0000 0000

0000 0000
5 double words for IP = 20 bytes
8 byte UDP header with a length field of 48
394e 9c15 XID 0000 0000 Message Type (CALL)
0000 0002 RPC Version 0001 86a3 RPC Program
0000 0003 Program Vers 0000 0000 Procedure #
0000 0000 Authentication
Here we have a full decode of the RPC header in a UDP packet. When first seeing this packet, it
might strike us as anomalous. Notice the source port number that TCPDump comes up with.
Decoding the packet, however, we find that there is apparently nothing wrong with it. If TCPDump
is able to determine based on the packet contents that it is looking at an RPC that it can understand,
will attempt to decode the RPC. If it is able to do so, it will replace the source port number with the
RPC Transaction ID.
With an understanding of how to decode an RPC request header, we are able to determine which
particular service an attacker is seeking. While the port numbers for common services do tend to be
fairly constant, the port assignments are not part of the RPC specification. This means that the port
number is subject to change at any time. For instance, if some other service had already bound port
2049 in the server, some other port would have been picked. This is where the port mapper comes in
for regular network traffic.
Many sites watch for probes to their portmapper, but how many watch for someone to go right to the
common port for the service?
24
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 24
RPC Decode (UDP Filter)
Please take a moment and design a TCPDump filter
to capture all UDP packets that might be RPC
version 2 Calls. Please feel free to collaborate.
Using the definition that we already have of an RPC call header, it should be possible for you to
create a TCPDump filter to capture all traffic that fits into our definition.

25
RPC Decodes – SANS 2001
RPCs - Friends or Foe? SANS 2001 - 25
RPC Decode
Assuming we have a failed/denied RPC:
Here’s the hex dump from a failed RPC request:
4500 0034 9bb4 4000 fe11 96bb xxxx 7732
xxxx 8821 0801 85c9 001c 63f9 3944 8a5c
0000 0001 0000 0001 0000 0001 0000 0004
udp[12:4] = 1 and udp[16:4] = 1 and udp[20:4] = 1
This hex dump shows an UDP packet (type 0x11) with no IP options (length 5) of length 0x1c (28
total bytes including the header). The RPC header is as follows:
3944 8a5c XID (Transaction ID)
0000 0001 Message Type (1 = reply)
0000 0001 Reply Type (1 = rejected)
0000 0001 Rejection reason (1 = authentication error)
0000 0004 Authentication error (4 = verifier expired or replayed)
What makes RPC headers trickier than normal network headers is that the header format changes
depending upon the discriminants in the unions. However, could we build an IDS filter to capture
RPC replies that failed authentication? We have everything we need for a signature:
Udp[12:4] = 1 and udp[16:4] = 1 and udp[20:4] = 1