Annals of the International Society of Dynamic Games
Volume 7
Series Editor
Tamer Bas¸ar
Editorial Board
Tamer Bas¸ar, University of Illinois, Urbana
Pierre Bernhard, I3S-CNRS and University of Nice-Sophia Antipolis
Maurizio Falcone, University of Roma “La Sapieza”
Jerzy Filar, University of South Australia, Adelaide
Alain Haurie, HEC-University of Geneva
Arik A. Melikyan, Russian Academy of Sciences, Moscow
Andrzej S. Nowak, Wroclaw Univeristy of Technology
and University of Zielona G
´
ora
Leo Petrosjan, St. Petersburg State University
Alain Rapaport, INRIA, Montpelier
Josef Shina, Technion, Haifa
Annals of the International Society of Dynamical Games
Advances in Dynamic Games
Applications to Economics, Finance,
Optimization, and Stochastic Control
Andrzej S. Nowak
Krzysztof Szajowski
Editors
Birkh
¨
auser
Boston
•

Basel
•
Berlin
Andrzej S. Nowak
Wroc
law University of Technology
Institute of Mathematics
Wybrze˙ze Wypia
´
nskiego 27
50-370 Wroc
law
Poland
Krzysztof Szajowski
Wroc
law University of Technology
Institute of Mathematics
Wybrze˙ze Wypia
´
nskiego 27
50-370 Wroc
law
Poland
and
Faculty of Mathematics, Computer Science,
and Econometrics
University of Zielona G
´
ora
Podg

´
orna 50
65-246 Zielona G
´
ora
Poland
AMS Subject Classifications: 91A-xx, 91A05, 91A06, 91A10, 91A12, 91A13, 91A15, 91A18, 91A20,
91A22, 91A23, 91A25, 91A28, 91A30, 91A35, 91A40, 91A43, 91A46, 91A50, 91A60, 91A65, 91A70,
91A80, 91A99
Library of Congress Cataloging-in-Publication Data
International Symposium of Dynamic Games and Applications (9th : 2000 : Adelaide, S. Aust.)
Advances in dynamic games : applications to economics, finance, optimization, and
stochastic control / Andrzej S. Nowak, Krzysztof Szajowski, editors.
p. cm. – (Annals of the International Society of Dynamic Games ; [v. 7])
Papers based on presentations at the 9th International Symposium on Dynamic Games
and Applications held in Adelaide, South Australia in Dec. 2000.
ISBN 0-8176-4362-1 (alk. paper)
1. Game theory–Congresses. I. Nowak, Andrzej S. II. Szajowski, Krzysztof. III. Title.
IV. Series.
HB144.I583 2000
330’.01’5193–dc22 2004048826
ISBN 0-8176-4362-1 Printed on acid-free paper.
c

2005 Birkh
¨
auser Boston
All rights reserved. This work may not be translated or copied in whole or in part without the written
permission of the publisher (Birkh
¨

auser Boston, c/o Springer Science+Business Media, Inc., Rights
and Permissions, 233 Spring Street, New York, NY, 10013, USA), except for brief excerpts in con-
nection with reviews or scholarly analysis. Use in connection with any form of information storage
and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now
known or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks and similar terms, even if they
are not identified as such, is not to be taken as an expression of opinion as to whether or not they are
subject to proprietary rights.
Printed in the United States of America. (KeS/SB)
987654321 SPIN 10988183
www.birkhauser.com
Contents
Preface ix
Contributors xi
Part I Repeated and Stochastic Games
Information and the Existence of Stationary Markovian Equilibrium 3
Ioannis Karatzas, MartinShubik and William D. Sudderth
Markov Games under a GeometricDrift Condition 21
Heinz-UweK
¨
uenle
ASimple Two-Person Stochastic Game with Money 39
Piercesare Secchi and William D. Sudderth
New Approaches and Recent Advances in Two-Person Zero-Sum
Repeated Games 67
Sylvain Sorin
Notes on Risk-Sensitive Nash Equilibria
95
Andrzej S. Nowak
Continuous Convex Stochastic Games of Capital Accumulation 111

Piotr Wi

ecek
Part II Differential Dynamic Games
Dynamic Core of Fuzzy Dynamical Cooperative Games 129
Jean-Pierre Aubin
Normalized Overtaking Nash Equilibrium for a Class of Distributed
Parameter Dynamic Games 163
Dean A. Carlson
Cooperative Differential Games 183
Leon A. Petrosjan
vi Contents
Part III Stopping Games
Selection by Committee 203
Thomas S. Ferguson
Stopping Game Problem for Dynamic Fuzzy Systems 211
Yuji Yoshida, Masami Yasuda, Masami Kurano and Jun-ichi Nakagami
On Randomized Stopping Games 223
El˙zbieta Z. Ferenstein
Stopping Games – Recent Results 235
Eilon Solan and Nicolas Vieille
Dynkin’s Games with Randomized Optimal Stopping Rules 247
Victor Domansky
Modified Strategies in a Competitive Best Choice Problem with
Random Pr
iority 263
Zdzisław Porosi
´
nski
Bilateral Approach to the Secretary Problem 271

David Ramsey and Krzysztof Szajowski
Optimal Stopping Games where Players have Weighted Privilege . 285
Minoru Sakaguchi
Equilibrium inanArbitration Procedure 295
VladimirV. Mazalov and AnatoliyA. Zabelin
Part IV Applications of Dynamic Games to Economics,
Finance and Queuing Theory
Applications of Dynamic Games in Queues 309
Eitan Altman
Equilibria for Multiclass Routing Problems in Multi-Agent Networks 343
Eitan Altman and Hisao Kameda
Endogenous Shocks and Evolutionary Strategy: Application to a
Three-Players Game 369
Ekkehard C. Ernst, Bruno Amable and Stefano Palombarini
Contents vii
Robust Control Approach to Option Pricing, Including Transaction
Costs 391
Pierre Bernhard
S-Adapted Equilibria in Games Played over Event Trees: An Overview 417
Alain Haurie and Georges Zaccour
Existence of Nash Equilibria in Endogenous Rent-Seeking Games 445
Koji Okuguchi
A Dynamic Game with Continuum of Players and its Counterpart with
Finitely Many Players 455
Agnieszka Wiszniewska-Matyszkiel
Part V Numerical Methods and Algorithms for Solving
Dynamic Games
Distributed Algorithms for Nash EquilibriaofFlow Control Games 473
Tansu Alpcan and Tamer Bas¸ar
A Taylor Series Expansion for H

∞
Control of Perturbed Markov Jump
Linear Systems 499
Rachid El Azouzi,Eitan Altman and Mohammed Abbad
Advances in Parallel Algorithms for the Isaacs Equation 515
Maurizio Falcone and Paolo Stefani
Numerical Algorithm for Solving Cross-Coupled AlgebraicRiccati
Equations of Singularly Perturbed Systems 545
Hiroaki Mukaidani, Hua Xu and Koichi Mizukami
Equilibrium Selection via Adaptation: Using Genetic Programming to
Model Learning in a Coordination Game 571
Shu-Heng Chen, John Duffy and Chia-Hsuan Yeh
Two Issues Surrounding Parrondo’s Paradox 599
Andre Costa, Mark Fackrell and Peter G. Taylor
Part VI Parrondo’s Games and Related Topics
State-Space Visualization and Fractal Properties of Parrondo’s Games 613
Andrew Allison, Derek Abbott and Charles Pearce
viii Contents
Parrondo’s Capital and History-Dependent Games 635
Gregory P. Harmer, Derek Abbott and Juan M. R. Parrondo
Introduction to Quantum Games and a Quantum Parrondo Game . 649
Joseph Ng and Derek Abbott
A Semi-quantum Version of the Game of Life 667
Adrian P. Flitney and Derek Abbott
Preface
Modern game theory has evolved enormously since its inception in the 1920s in the
works of Borel and von Neumann. The branch of game theory known as dynamic
games descended from the pioneering work on differential games by R. Isaacs,
L. S. Pontryagin and his school, and from seminal papers on extensive form games
by Kuhn and on stochastic games by Shapley. Since those early developmental

decades, dynamic game theory has had a significant impact on such diverse dis-
ciplines as applied mathematics, economics, systems theory, engineering, oper-
ations research, biology, ecology, and the environmental sciences. On the other
hand, a large variety of mathematical methods from differential equations to
stochasti
c processes has been applied to formulate and solve many different prob-
lems.
Thisnewedited book focuses on various aspects of dynamic game theory, pro-
viding authoritative, state-of-the-art information and serving as a guide to the
vitality of the field and its applications. Most of the selected, peer-reviewed papers
are based on presentations at the 9th International Symposium on Dynamic Games
and Applications held in Adelaide, South Australia in December 2000. This con-
ference took place under the auspices of the International Society of Dynamic
Games (ISDG), established in 1990. The conference has been cosponsored by
Centre for Industrial and Applicable Mathematics (CIAM), University of South
Australia, IEEE Control Systems Society, Institute of Mathematics, Wrocław Uni-
versity of Technology (Poland), Faculty of Mathematics, Computer Science and
Econometrics, University of Zielona G
´
ora (Poland), ISDG Organizing Society,
and the University of South Australia.Every paper that appears inthis volume has
passed through a stringent reviewing process, as is the case with publications for
archival journals.
Avariety of topics of current interest are presented. They are divided intosix
parts: the first (five papers) treat repeated games and stochastic games, and the sec-
ond (three papers) covers differential dynamic games
. The third part of the volume
(nine papers) is devoted to the various extensions of stopping games, which are
also known as Dynkin’s games. In the fourth part there are seven papers on applica-
tions of dynamic games to economics, finance, and queuing theory. The final two

parts contain five papers which are devoted to algorithms and numerical solution
approaches for dynamic games, and the section on Parrondo’s games (five papers).
We wish to thank all the associate editors and the referees for their valuable
contributions that made this volume possible.
Wrocław and Zielona G
´
ora Andrzej S. Nowak
Wrocław Krzysztof Szajowski
Contributors
Mohammed Abbad,D
´
epartement de Mathematiques et Informatique, Facult
´
e des
Sciences B.P. 1014, Universit
´
e Mohammed V, 10000 Rabat, Morocco
Derek Abbott, Centre for Biomedical Engineering (CBME) and Department of
Electrical and Electronic Engineering, The University of Adelaide, Adelaide,
SA 5005, Australia
Andrew Allison, Centre for Biomedical Engineering (CBME) and Department of
Electrical and Electronic Engineering, University of Adelaide, Adelaide,
SA 5005, Australia
Tansu Alpcan, Coordinated Science Laboratory, University of Illinois, 1308 West
Main Street, Urbana, IL 61801, USA
Eitan Altman, INRIA, B.P. 93, 2004 route des Lucioles, 06902 Sophia-Antipolis
Cedex, France
Bruno Amable, Facult
´
e des Sciences Economies, Universit

´
eParis X-Nanterre,
200 av. de la R
´
epublique, 92000 Nanterre, France
Jean-Pierre Aubin, Centre de Recherche Viabilit
´
e, Jeux, Contr
ˆ
ole, Universit
´
e
Paris-Dauphine, 75775 Paris cx (16), France
Rachid El Azouzi,University of Avignon, LIA, 339, chemin des Meinajaries,
Agroparc B.P. 1228, 84911 Avignon Cedex 9, France
Tamer Bas¸ar, Coordinated Science Laboratory, University of Illinois, 1308 West
Main Street, Urbana, IL 61801, USA
Pierre Bernhard, Laboratoire I3S, UNSA and CNRS, 2000 route des Lucioles, Les
Algorithmes – b
ˆ
at.Euclide 8, BP.121, 106903 Sophia Antipolis-Cedex, France
Dean A. Carlson, Mathematical Reviews 416 Fourth Street, P.O.Box 8604, Ann
Arbor, MI 48107-8604, USA
Shu-Heng Chen, AI-ECON Research Center, Department of Economics,
National Chengchi University,64Chi-Nan Rd., Sec.2, Taipei 11623, Taiwan
Andre Costa, School of Applied Mathematics, University of Adelaide, Adelaide,
SA 5005, Australia
xii Contributors
Victor Domansky,St. Petersburg Institute for Economics and Mathematics,
Russian Academy of Science, Tchaikovskogo 1, 191187, St. Petersburg, Russia

John Duffy, Department of Economics, University of Pittsburgh, 4S01 Posvar
Hall, 230 S. Bouquet Street, Pittsburgh, PA 15260, USA
Ekkehard C. Ernst,Directorate General Economics, European Central Bank,
Kaiserstrasse 29, 60311 Frankfurt, Germany
Mark Fackrell, Department of Mathematics and Statistics, University of
Melbourne, Victoria, 3010, Australia
Maurizio Falcone,Dipartimento di Matematica, Universit
`
adi Roma "La
Sapienza", P. Aldo Moro 2, 00185 Roma, Italy
El
˙
zbieta Z. Ferenstein, Faculty of Mathematics and Information Science,
Warsaw University of Technology, Plac Politechniki 1, 00-661 Warsaw, Poland;
and Polish-Japanese Institute of Information Technology, Koszykowa 86,
02-008 Warsaw, Poland
Thomas S. Ferguson, Department of Mathematics, University of Californiaat
Los Angeles, 405 Hilgard Ave., Los Angeles, CA 90095-1361, USA
Adrian P. Flitney, Centre for Biomedical Engineering (CBME) and Department
of Electrical and Electronic
Engineering, The University of Adelaide, Adelaide,
SA 5005, Australia
Gregory P. Harmer, Centre for Biomedical Engineering (CBME) and
Department of Electrical and Electronic Engineering, University of Adelaide,
Adelaide, SA 5005, Australia
Alain Haurie,HEC-Management Studies, Faculty of Economics and Social
Science, 40 Blvd. du Pont-d’Arve, CH-1211 Geneva 4, Switzerland
Hisao Kameda, Institute of Information Science and Electroni
cs, University of
Tsukuba, Tsukuba Science City, Ibaraki 305-8573, Japan

Ioannis Karatzas, Department of Mathematics and Statistics, Columbia
University, New York, NY 10027, USA
Masami Kurano, Department of Mathematics, Chiba University, Inage-ku, Chiba
263-8522, Japan
Contributors xiii
Heinz-UweK
¨
uenle, Brandenburgische Technische Universit
¨
at Cottbus, PF 10 13
44, D-03013 Cottbus, Germany
VladimirV. Mazalov, Institute of Applied Mathematical Research, Karelian
Research Center of Russian Acad. Sci., Pushkinakaya st. 11, Petrozavodsk,
185610, Russia
Koichi Mizukami, Graduate School of Engineering, Department of Computer
Science, Hiroshima Kokusai GakuinUniversity, 20-1 Nakano 6, Aki-ku,
Hiroshima, 739-0321, Japan
Hiroaki Mukaidani, Graduate School of Educat
ion, Hiroshima University,
1–1–1, Kagamiyama, Higashi–Hiroshima, 739–8524, Japan
Jun-ichi Nakagami, Department of Mathematics and Informatics, Chiba
University, Inage-ku, Chiba 263-8522, Japan
Joseph Ng, Centre for Biomedical Engineering (CBME) and Department of
Electrical and Electronic Engineering, University of Adelaide, Adelaide,
SA 5005, Australia
Andrzej S. Nowak, Wrocław University of Technology, Institute of Mathematics
Wybrze
˙
z
eWypianskiego 27, PL-50-370 Wrocław Poland; and Faculty of

Mathematics, Computer Science and Econometrics, University of Zielona G
´
ora,
Podgorna 50, 65-246 Zielona G
´
ora, Poland
Koji Okuguchi, Department of Economics and Information, Gifu Shotoku
Gakuen University, Gifu-shi,Gifu-ken 500-8288, Japan
Stefano Palombarini, Facult
´
e des Sciences Economies, Universit
´
eParis VIII, 2
rue de la Libert
´
e, 93526 Saint-Denis Cedex 02, France
Juan M.R. Parrondo, Departamento de F
´
isica At
´
omica, Molecular y Nuclear,
Universidad Complutense de Madrid, 28040 Madrid, Spain
Charles Pearce, Department of Applied Mathematics, The University of
Adelaide, Adelaide, SA 5005, Australia
Leon A. Petrosjan, Faculty of Applied Mathematics, St. Petersburg State
University, Bibliotechnaya pl. 2, Petrodvorets 199504, St. Petersburg, Russia
Zdzisław Porosi
´
nski, Institute of Mathematics, Wrocław University of
Technology, Wybrze

˙
ze Wyspia
´
nskiego 27, 50-370 Wrocław, Poland
xiv Contributors
David Ramsey, Institute of Mathematics, Wrocław University of Technology,
Wybrze
˙
ze Wyspia
´
nskiego 27, 50-370 Wrocław, Poland
Minoru Sakaguchi, 3-26-4 Midorigaoka, Toyonaka, Osaka 560-0002, Japan
Piercesare Secchi,Dipartimento di Matematica, Politecnico di Milano, Piazza
Leonardo da Vinci 32, I-20133 Milano, Italia
MartinShubik, Cowles Foundation for Research in Economics, Yale University,
New Haven, CT 06520, USA
Eilon Solan, Department of Managerial Economics and Deci
sion Sciences,
Kellogg School of Management, Northwestern University; and School of
Mathematical Sciences, Tel AvivUniversity, Tel Aviv 69978, Israel
Sylvain Sorin, Equipe Combinatoire et Optimisation, UFR 921, Université Pierre
et Marie Curie-Paris 6, 4 place fussieu, 75230 Paris, France; and Laboratoire
d’Econometrie, Ecole Polytechnique, 1 rue Descartes, 75005 Paris, France
Paolo Stefani, CASPUR, P. Aldo Moro 2, 00185 Roma, Italy
William D. Sudderth, School of Statistics, Un
iversity of Minnesota, Church
Street SE 224, Minneapolis, MN 55455, USA
Krzysztof Szajowski, Institute of Mathematics, Wrocław University of
Technology, Wybrze
˙

ze Wyspia
´
nskiego 27, 50-370 Wrocław, Poland
Peter G. Taylor, Department of Mathematics and Statistics, University of
Melbourne, Victoria, 3010, Australia
Ra
´
ul Toral, Departamento de F
´
isica, Universitat de les Illes Balears; and Instituto
Mediterr
´
aneo de Estudios Avanzados, IMEDEA (CSIC-UIB), 07071 Palma de
Mallorca, Spain
Nicolas Viei
lle,D
´
epartement Finance et Economie, HEC School of Management
(HEC), 78 Jouy-en-Josas, France
Piotr Wi

ecek, Institute of Mathematics, Wrocław University of Technology,
Wybrze
˙
ze Wyspia
´
nskiego 27, 50-370 Wrocław, Poland
Agnieszka Wiszniewska-Matyszkiel, Institute of Applied Mathematics and
Mechanics, Warsaw University, Banacha 2, 02-097 Warsaw, Poland
Contributors xv

Hua Xu, Graduate School of Business Sciences, The University of Tsukuba,
3–29–1, Otsuka, Bunkyou–ku, Tokyo, 112–0012, Japan
Masami Yasuda, Department of Mathematics and Informatics, Chiba University,
Inage-ku, Chiba 263-8522, Japan
Chia-Hsuan Yeh, Department of Information Management, Yuan Ze University,
Chungli, Taoyuan 320, Taiwan
Yuji Yos hida, Faculty of Economics and Business Administration, The University
of Kitakyushu, Kitakyushu 802-8577, Japan
AnatoliyA. Zabelin,Chita State Pedagogical University, Babushkinst. 121, Chita
672090, Russia
Georges Zaccour,GE
RAD and Ecole des H.E.C. Montr
´
eal, 300 Cote
S. Catherine, H3T 2A7, Montreal, Canada
PART I
Repeated and Stochastic Games

Information and the Existence of Stationary
Markovian Equilibrium
Ioannis Karatzas
Department of Mathematics and Statistics
ColumbiaUniversity
New York, NY 10027

Martin Shubik
Cowles Foundation for Research in Economics
Yale University
New Haven, CT 06520


William D. Sudderth
School of Statistics
University of Minnesota
Minneapolis, MN 55455

Abstract
We describe conditions for the existence of a stationary Markovian equi-
librium when total production or total endowment is a random variable. Apart
from regularity assumptions, there are two crucial conditions: (i) lowinforma-
tion—agents are ignorant of both total endowment and their own endowments
when they make decisions inagiven period, and (ii) proportional endow-
ments—the endowment of each agent is in proportion, possibly random, to the
total endowment. When these conditions hold, there is a stationary equilibrium.
When they do not hold, such an equilibrium need not exist.
1 Introduction
This paper is part of an effort to investigate a mass-market economy with stochastic
elements, inwhich the optimization problems faced by each of a continuum of
agents are modeled as parallel dynamic programming problems. The model used is
a strategic market game at the highest level of aggregation, in order to concentrate
on the monetary aspects of a stochasticenvironment. Although there are several
previous papers which provide economic motivation and modeling details [2]–[4],
4 I. Karatzas, M. Shubik, and W. D. Sudderth
we have attempted to make this paper as self-contained as possible. However, we
shall make use of several results established in these earlier works.
We consider an economy with a stochastic supply of goods, where: (i) the
endowment of each agent is in proportion (possibly random) to the total amount
of goods available; and (ii) the agents must bid for goods in each period without
knowing either the total supply of goods available, or the realization of theirown
random endowments.
For such an economy, we shall show the existence of a stationary equilibrium,

where the optimal amount bid by an agent in each period depends only on the
agent’s current wealth. In equilibrium, there will be a stat
ionary distribution of
wealth among agents, although prices and wealth-levels of individual agents will
fluctuate randomly with time. Thiswill be true whether or not the opportunity
isavailable for agents to borrow from, or deposit in, an outside (government)
bank.
When either the individual endowments are not proportional to the total available
supply of goods, or the agents have additional information (in the form of advance
knowledge of the total supply of goods), there need not exist such an equilibrium.
Thiswill be illustrated by two examples. One interpretation of these results is that
better short-term forecasting can be destabili
zing. We plan further investigation
of these “high information” phenomena in a subsequent paper.
The next section has some preliminary discussion of our model. Sections 3 and
4 treat the model without lending, sections 5 and 6 are on the model with lending
and possible bankruptcy, whereas the final section 7 treats five simple examples
that illustrate the existence and non-existence of stationary equilibrium.
2Preliminaries
For simplicity we omit production from consideration. Instead, we consider an
economy where all consumption goods are bought for cash (fiat money) in a com-
petitive market.Each individual agent begins with an initial endowment of money
and a claim to the proceeds from consumption goods that are sold in the market.
The goods enter the economy in each period as if they were “manna” from an
undescribed production process, and are owned by the individual agents. How-
ever, the agents are required to offer the goods in the market, and do not receive
the proceeds until the start of the subsequent period. The assumption that all goods
go through the market is probably a better approx
imation of the realities in a mod-
ern economy than the reverse, where each agent can consume everything directly,

without the interface of markets or prices.
Our model has a continuum of agents indexed by the unit interval I = [0, 1],
and distributed according to a non-atomic probability measure ϕ on the σ -algebra
B(I) of Borel subsets of I. Time runs indiscrete time-periods n = 0, 1, ···. At the
beginning of each time-period n, every agent α ∈ I receives an endowment Y
α
n
(ω)
inunits of a nondurable commodity. The random variables {Y
α
n
;α ∈ I,n ∈ N},
Information and theExistence of Stationary Markovian Equilibrium 5
and all other random variables encountered inthis paper, are defined on a given
probability space (, F , P).
We shall consider the no-lending model of [3], and also the lending with pos-
sible bankruptcy model of [2]. Unlike these earlier papers, itwill no longer be
assumed that total production Q is constant from period to period, but instead that
production
Q
n
(ω) =

Y
α
n
(ω)ϕ(dα)
in period n is a random variable, for all n = 1, 2, ···.
The following assumption will be in force throughout sections 2–6.
Assumption 2.1. (a) The total-production variables Q

1
,Q
2
, ··· are I.I.D.
(independent and identically distributed) with common distribution ζ . It will
also be assumed that the Q
n
’s are strictly positive with finite mean.
(b) The individual endowment variables Y
α
n
(ω) are proportional to the Q
n
(ω), in
the sense that
Y
α
n
(ω) = Z
α
n
(ω)Q
n
(ω) for all α ∈ I, n ∈ N,ω∈ . (1)
Here the sequences {Z
α
1
,Z
α
2

, } and {Q
1
,Q
2
, } are independent; Z
α
n
≥
0, E(Z
α
n
) = 1; and Z
α
1
,Z
α
2
, ··· are I.I.D. with common distribution λ
α
, for
each α ∈ I.
This is the simplest set of assumptions that permit both the total-production
random variables to fluctuate with time, and a stationary equilibrium to exist; their
negation precludes the existence of such an equilibrium, as Example 7.4 below
demonstrates. A consequence of these assumptions is that
E(Y
α
n
) = E(Z
α

n
) · E(Q
n
) = E(Q
n
). (2)
3 The Model without Lending
For α ∈ I and n ∈ N, let S
α
n−1
(ω) and F
α
n−1
denote respectively the wealth and
information σ-algebra available to agent α at the beginning of period n. As in [3],
agent α bids an F
α
n−1
-measurable amount b
α
n
(ω) ∈ [0,S
α
n−1
(ω)] of money for
the consumption good before knowing the value of Q
n
(ω) or Y
α
n

(ω). We call this
the low-information condition. (In other words, the information σ -algebra F
α
n−1
available to the agent at the beginning of period n, measures the values of past
quantities including S
α
0
,S
α
k
,Q
k
,Z
α
k
,b
α
k
for k = 1, ···,n−1, but not of Q
n
,Y
α
n
.)
Once all agents have placed theirbids, the total amount of fiat money bid for
the consumption good isgiven by
B
n
(ω) =


b
α
n
(ω)ϕ(dα),
6 I. Karatzas, M. Shubik, and W. D. Sudderth
and a new price is formed as
p
n
(ω) =
B
n
(ω)
Q
n
(ω)
for period t = n.Each agent α receives an amount
x
α
n
(ω) =
b
α
n
(ω)
p
n
(ω)
=
b

α
n
(ω)
B
n
(ω)
· Q
n
(ω)
of goods, equal to hisbid’s worth in the price of the goods for period t = n,as
well as
p
n
(ω)Y
α
n
(ω) =
B
n
(ω)
Q
n
(ω)
· Z
α
n
(ω)Q
n
(ω) = B
n

(ω)Z
α
n
(ω) (3)
in cash income, and then enters the next period with wealth in fiat money
S
α
n
(ω) = S
α
n−1
(ω) − b
α
n
(ω) + B
n
(ω)Z
α
n
(ω). (4)
Each agent α ∈ I is assumed to have a utility function u
α
: [0, ∞) → [0, ∞) for
consumption of goods; this function is continuous and continuously differentiable,
strictly concave, strictly increasing, and satisfies u
α
(0) = 0,(u
α
)
′

+
(0) ∈ (0, ∞).
The utility earned by agent α in period n is u
α
(x
α
n
(ω)), and the agent seeks to
maximize the expected value of his total discounted utility
∞

n=0
β
n
u
α

x
α
n+1
(ω)

.
A strategy π
α
for agent α specifies the sequence of bids {b
α
n
}
∞

n=1
. The strategy
π
α
is called stationary, if it specifies the bids in terms of a single function c
α
:
[0, ∞) → [0, ∞) of wealth, in the form
b
α
n
(ω) = c
α
(S
α
n−1
(ω)), n ≥ 1, (5)
where c
α
(s) ∈ [0,s], ∀s ≥ 0. We call c
α
(·) the consumption function for the
strategy π
α
.
The wealth distribution in period n is the random measure ν
n
(·,ω)given by
ν
n

(A, ω) = ϕ

{α ∈ I : S
α
n
(ω) ∈ A}

,A∈ B([0, ∞)). (6)
We are now ready to define the type of equilibrium that we want to study inthis
note.
Definition 3.1. A collection of stationary strategies {π
α
,α ∈ I }and a probability
distribution μ on B((0, ∞)) form a stationary equilibrium, if
(a) given that ν
0
= μ and that every agent α plays strategy π
α
,wehaveν
n
= μ
for all n ≥ 1, and
(b) given that ν
0
= μ, the strategy π
α
is optimal for agent α, when every other
agent β plays π
β
(β ∈ I,β = α), for each α ∈ I.

Information and theExistence of Stationary Markovian Equilibrium 7
Unlike [3], there is no mention of price in Definition 3.1. This is, in part, because
the sequence of prices {p
n
} will not be constant – even in stationary equilibrium –
for the model studied here. Indeed, if the consumption function for π
α
is the same
across all agents α ∈ I, and equal to c
α
(·) ≡ c(·), then
p
n
(ω) =

b
α
n
(ω)ϕ(dα)
Q
n
(ω)
=

c(s)μ(ds)
Q
n
(ω)
,
where the sequence of total bids

B
n
(ω) ≡ B :=

c(s)μ(ds)
is constant in equilibrium; see Theorem 4.1 below. Thus, the prices {p
n
}form then
a sequence of I.I.D. random variables, because the {Q
n
}do so by assumption. The
constant B will play the same mathematical role that was played by the price p in
the earlier works [3] and [2], but of course the interpretation here will be different.
4Existence of Stationary Equilibrium for the Model
without Lending
The methods of the paper [3] can be adapted, to construct a stationary equilibrium
for the present model. As in [3], we consider first the one-person game faced
by an agent α, assuming that the economy is in stationary equilibrium.For ease
of notation we suppress the superscript α while discussing the one-person game.
Furthermore, we also assume that the agents are homogeneous, in the sense that
they all have the same utility function u(·) and the same distribution λ for their
income variables. This assumption makes the existence proof more transparent,
but is not necessary; the proof in [3] works for many types of agents, and can be
adapted to the present context as well.
We introduce a new ut
ility function defined by
˜u(b) := E
[
u(bQ(ω))
]

=

u(bq)ζ (dq), b ≥ 0. (7)
Observe that the expected utility earned by an agent who bids b when faced by a
random price p(ω) = B/Q(ω), can be written
E

u

b
p(ω)

= E

u

b
B
Q(ω)

=˜u

b
B

. (8)
It is straightforward to verify that ˜u(·) has all the properties, such as strict concavity,
that were assumed for u(·).
8 I. Karatzas, M. Shubik, and W. D. Sudderth
Let V(·) be the value function for an agent playing in equilibrium. In essence,

the agent faces a discounted dynamic programming problem and, just as in [3],
the value function V(·) satisfies the Bellman equation
V(s)= sup
0≤b≤s

˜u

b
B

+ β · E[V(s − b + BZ)]

. (9)
This dynamic programming problem is of the type studied in [3], and Theorem
4.1 of that paper has information about it. In particular, there isaunique optimal
stationary plan π = π(B) corresponding to a consumption function c : [0, ∞) →
[0, ∞). We sometimes write this function as c(s) = c(s;B), to make explicit its
dependence on the quantity B.
Consider now the Markov chain {S
n
} of successive fortunes for an agent who
plays the optimal strategy π given by c(·). Then we have
S
n+1
= S
n
− c(S
n
;B) + BZ
n+1

,n∈ N
0
(10)
where Z
1
,Z
2
, are I.I.D. with common distribution λ. By Theorem 5.1of
[3], this chainhasaunique stationary distribution μ(·) = μ(·;B) defined on
B([0, ∞)). Now assume that Z hasafinite second moment: E(Z
2
)<∞. Then,
by Theorem 5.7 of [3], the stationary distribution μ hasafinite mean, namely

(0,∞)
s μ(ds) < ∞. The following lemma expresses the fact that the total amount
bid by all agents is equal to B, when the wealth distribution is μ(·;B).
Lemma 4.1.

c(s;B)μ(ds;B) = B.
Proof. Assume that S
0
has the stationary distribution μ. Then take expectations
in (10) to obtain
E(S
n+1
) = E(S
n
) −


c(s;B)μ(ds;B) + B ·E(Z), n ∈ N
0
and the desired formula follows, since E(S
n+1
) = E(S
n
) by stationarity and
E(Z) = 1 by assumption of our model. ✷
Theorem 4.1. For each B>0,there is a stationary equilibrium for the no-
lending model, withwealth distribution μ(·) = μ(·;B), and with stationary strate-
gies π
α
≡ π(B) for all agents α ∈ I .
Proof. Construct the variables Z
α
n
(ω) = Z
n
(α, ω) using the technique of Feldman
and Gilles [1], so that
Z
1
(α, ·), Z
2
(α, ·), are I.I.D. with distribution λ, for every α ∈ I , and
Z
1
(·, ω), Z
2
(·,ω), are I.I.D. with distribution λ, for every ω ∈ .

Then the chain {S
n
(α, ω)} has the same dynamics for each fixed ω ∈  as it does
for each fixed α ∈ I . The distribution μ is stationary for the chain when α is
Information and theExistence of Stationary Markovian Equilibrium 9
fixed, and will therefore be a stationary wealth distribution for the many-person
game if the total bids B
1
(ω), B
2
(ω), remain equal to B. Now, if S
0
(·,ω)has
distribution μ, then
B
1
(ω) =

c(S
0
(α, ω))ϕ(dα) =

c(s)μ(ds) = B,
by Lemma 4.1. By induction, B
n
(ω) = B for all n ≥ 1 and ω ∈ . Hence, the
wealth-distributions ν
n
are all equal to μ.
The optimality of π

α
= π(B) follows from its optimality in the one-person
game together with the fact that a single player cannot affect the value of the total
bid. ✷
5 The Model with Lending and Possible Bankruptcy
We now assume that there is a Central Bank which gives loans and accepts deposits.
The bank sets two interest rates in each time-period n, namely r
1n
(ω) = 1+ρ
1n
(ω)
to be paid by borrowers and r
2n
(ω) = 1 +ρ
2n
(ω) to be paid to depositors. These
rates are assumed to satisfy
1 ≤ r
2n
(ω) ≤ r
1n
(ω), r
2n
(ω) ≤ 1/β, (11)
for all n ∈ N,ω ∈ .
Agents are required to pay their debts back at the beginning of the next period,
when they have sufficient funds to do so. However, it can happen that they are
unable to pay back their debts in full, and are thus forced to pay a bankruptcy
penalty inunits of utility, before they are allowed to continue in the game.For
this reason, we assume now that each agent α hasautility function u

α
: R → R
defined on the entire real line, and satisfies all the other assumptions made above.
For x<0, the quantity u
α
(x) is negative and measures the “disutility” for agent
α of going bankrupt by an amount x; for x>0, the quantity u
α
(x) is positive and
measures the utility derived by α from consuming x units of the commodity, just
as before.
Suppose that an agent α ∈ I begins in period n with wealth S
α
n−1
(ω). If
S
α
n−1
(ω) < 0, then agent α has an unpaid debt from the previous period and is
assessed a penalty of u(S
α
n−1
(ω)/p
n−1
(ω)). The debt is then forgiven, and the
agent continues play from wealth-position 0. If S
α
n−1
(ω) ≥ 0, then agent α is not
in debt and plays from position S

α
n−1
(ω). In both cases, an agent α, possibly after
being punished, plays from the wealth-position (S
α
n−1
(ω))
+
= max{S
α
n−1
(ω), 0}.
Based on knowledge of past quantities S
α
0
,S
α
k
,Z
α
k
,Q
k
,r
1k
,r
2k
for k =
1, ···,n− 1, agent α chooses a bid
b

α
n
(ω) ∈ [0,(S
α
n−1
(ω))
+
+ k
α
],
10 I. Karatzas, M. Shubik, and W. D. Sudderth
where k
α
≥ 0 is an upper bound on loans to agent α. As before, agent α must
bid in ignorance of both the total endowment Q
n
(ω) and his personal endowment
Y
α
n
(ω) for period n.
The total bid B
n
, the price p
n
, and agent α’s quantities of goods x
α
n
and fiat
money p

n
Y
α
n
= B
n
Z
α
n
are formed exactly as in the no-lending model of Section
3.Formula (4) for the dynamics now takes the form
S
α
n
=

−r
1n
(b
α
n
− (S
α
n−1
)
+
) + B
n
Z
α

n
, if (S
α
n−1
)
+
≤ b
α
n
,
r
2n
((S
α
n−1
)
+
− b
α
n
) + B
n
Z
α
n
, if (S
α
n−1
)
+

>b
α
n
.
(12)
The wealth-distribution ν
n
on day n is defined by formula (6) as before, but with
the understanding that the set A now ranges over Borel subsets of the whole real
line, since some agents may have negative wealth. An agent α’sutility in period n
isnowgiven by
ξ
α
n
(ω) =

u
α
(x
α
n
(ω)), if S
α
n−1
(ω) ≥ 0,
u
α
(x
α
n

(ω)) + u
α
(S
α
n−1
(ω)/p
n−1
(ω)), if S
α
n−1
(ω) < 0.
As before, agent α seeks to maximize the expected value of his total discounted
utility
∞

n=0
β
n
ξ
α
n
(ω).
We extend now the definition of stationary equilibrium to the model with lending.
Definition 5.1. A stationary equilibrium for the model with lending, consists of
a wealth distribution μ (i.e. a probability distribution) on the Borel subsets of the
real line, of interest rates r
1
,r
2
with 1 ≤ r

2
≤ r
1
,r
2
≤ 1/β, and of a collection of
stationary strategies {π
α
,α ∈ I } such that, if the bank sets interest rates r
1
and r
2
in every period, and if the initial wealth distribution is ν
0
= μ, then
(a) ν
n
= μ for all n ≥ 1 when every agent α plays strategy π
α
, and
(b) the strategy π
α
is optimal for agent α when every other agent β plays
π
β
(β ∈ I,β = α), for each α ∈ I.
Suppose that the model is in stationary equilibrium, and that each stationary
strategy π
α
specifies its bids b

α
n
by the same consumption function c
α
≡ c(·) with
0 ≤ c
α
(s) ≤ s + k for all s ≥ 0, and the same upper-bound k
α
≡ k on loans, for
all α ∈ I. Then the total bid,
B = B
n
(ω) =

c(s
+
)μ(ds),
remains constant from period to period, while the prices {p
n
} form an I.I.D.
sequence, just as in the no-lending model of Section 3.
Information and theExistence of Stationary Markovian Equilibrium 11
6Existence of Stationary Equilibrium for the Model
with Lending and Possible Bankruptcy
The methods and results of [2] can be used here, as those of [3] were used in
Section 4. We consider the one-person game faced by an agent when the economy
is in stationary equilibrium. We suppress the superscript α and assume that agents
are homogeneous, with common utility function u(·), income distribution λ, and
loan limit k. We define the utility function ˜u(·) as in (7) and observe that (8) remains

valid.Formula (12) for the dynamics can be written in the simpler form
S
n
= g

(S
n−1
)
+
− b
n

+ BZ
n
,n∈ N (13)
where
g(x) =

r
1
x, x < 0,
r
2
x, x ≥ 0.
The Bellman equation becomes
V(s)=

sup
0≤b≤s+k
[ ˜u(b/B) + β · E[V(g(s− b) + BZ]],s≥ 0

˜u(s/B) + V(0), s < 0.
(14)
This equation is of the type studied in [2], and all the major results of that paper
have counterparts here.For example, Theorem 4.2 of [2] applies, to tell us that there
isaunique stationary optimal strategy π = π(B) corresponding to a consumption
function c(·) = c(·;B). The Markov chain {S
n
} for the fortunes of an agent who
plays π(B) evolves according to the dynamics
S
n+1
= g((S
n
)
+
− c((S
n
)
+
;B)) + BZ
n+1
,n∈ N
0
. (15)
Conditions for this chain to have a stationary distribution μ with finite mean are
available in Theorem 4.3of[2].For μ to be the wealth-distribution of a stationary
equilibrium, we must also assume that the bank balances its books under μ.
Assumption 6.1. (i) The Markov chain {S
n
} of (15) has an invariant distribu-

tion μ with finite mean.
(ii) Under the wealth-distribution μ, the total amount of money paid back to
the bank by borrowers inagiven period, is equal to the sum of the total
amount borrowed, plus the amount of interest paid by the bank to lenders.
This condition can be written as

[Bz ∧ r
1
d(s
+
)] μ(ds)λ(dz) =

d(s
+
)μ(ds)+ ρ
2

ℓ(s
+
) μ(ds),
where d(s) = (c(s) −s)
+
and ℓ(s) = (s −c(s))
+
are the amounts borrowed
and deposited, respectively, under the stationary strategy c(·), by an agent
with wealth s ≥ 0.