Stochastic Control
edited by
Chris Myers
SCIYO
Stochastic Control
Edited by Chris Myers
Published by Sciyo
Janeza Trdine 9, 51000 Rijeka, Croatia
Copyright © 2010 Sciyo
All chapters are Open Access articles distributed under the Creative Commons Non Commercial Share
Alike Attribution 3.0 license, which permits to copy, distribute, transmit, and adapt the work in any
medium, so long as the original work is properly cited. After this work has been published by Sciyo,
authors have the right to republish it, in whole or part, in any publication of which they are the author,
and to make other personal use of the work. Any republication, referencing or personal use of the work
must explicitly identify the original source.
Statements and opinions expressed in the chapters are these of the individual contributors and
not necessarily those of the editors or publisher. No responsibility is accepted for the accuracy of
information contained in the published articles. The publisher assumes no responsibility for any
damage or injury to persons or property arising out of the use of any materials, instructions, methods
or ideas contained in the book.

Publishing Process Manager Jelena Marusic
Technical Editor Goran Bajac
Cover Designer Martina Sirotic
Image Copyright Evv, 2010. Used under license from Shutterstock.com
First published September 2010
Printed in India
A free online edition of this book is available at www.sciyo.com
Additional hard copies can be obtained from
Stochastic Control, Edited by Chris Myers
p. cm.

ISBN 978-953-307-121-3
SCIYO.COM
WHERE KNOWLEDGE IS FREE
free online editions of Sciyo
Books, Journals and Videos can
be found at www.sciyo.com

Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 9
Chapter 10
Chapter 11
Preface IX
The Fokker-Planck equation 1
Shambhu N. Sharma and Hiren G. Patel
The Itô calculus for a noisy dynamical system 21
Shambhu N. Sharma
Application of coloured noise as a driving
force in the stochastic differential equations 43
W.M.Charles
Complexity and stochastic synchronization
in coupled map lattices and cellular automata 59
Ricardo López-Ruiz and Juan R. Sánchez
Zero-sum stopping game associated with threshold probability 81

Yoshio Ohtsubo
Stochastic independence with respect to upper and lower conditional
probabilities defined by Hausdorff outer and inner measures 87
Serena Doria
Design and experimentation of a large scale distributed stochastic
control algorithm applied to energy management problems 103
Xavier Warin and Stephane Vialle
Exploring Statistical Processes with Mathematica7 125
Fred Spiring
A learning algorithm based on PSO and L-M for parity problem 151
Guangyou Yang, Daode Zhang, and Xinyu Hu
Improved State Estimation of Stochastic Systems
via a New Technique of Invariant Embedding 167
Nicholas A. Nechval and Maris Purgailis
Fuzzy identification of discrete time nonlinear stochastic systems 195
Ginalber L. O. Serra
Contents
VI
Chapter 12
Chapter 13
Chapter 14
Chapter 15
Chapter 16
Chapter 17
Chapter 18
Chapter 19
Chapter 20
Chapter 21
Chapter 22
Chapter 23

Fuzzy frequency response for stochastic
linear parameter varying dynamic systems 217
Carlos C. T. Ferreira and Ginalber L. O. Serra
Delay-dependent exponential stability and filtering
for time-delay stochastic systems with nonlinearities 235
Huaicheng Yan, Hao Zhang, Hongbo Shi and Max Q H. Meng
Optimal filtering for linear states over polynomial observations 261
Joel Perez, Jose P. Perez and Rogelio Soto
The stochastic matched filter
and its applications to detection and de-noising 271
Philippe Courmontagne
Wireless fading channel models:
from classical to stochastic differential equations 299
Mohammed Olama, Seddik Djouadi and Charalambos Charalambous
Information flow and causality quantification
in discrete and continuous stochastic systems 329
X. San Liang
Reduced-Order LQG Controller
Design by Minimizing Information Loss 353
Suo Zhang and Hui Zhang
The synthesis problem of the optimum control
for nonlinear stochastic structures in the multistructural
systems and methods of its solution 371
Sergey V. Sokolov
Optimal design criteria for isolation devices in vibration control 393
Giuseppe Carlo Marano and Sara Sgobba
Sensitivity analysis and stochastic modelling
of the effective properties for reinforced elastomers 411
Marcin Kamiński and Bernd Lauke
Stochastic improvement of structural design 437

Soprano Alessandro and Caputo Francesco
Modelling earthquake ground motions by stochastic method 475
Nelson Lam, John Wilson and Hing Ho Tsang
VII
Chapter 24
Chapter 25
Chapter 26
Chapter 27
Chapter 28
Chapter 29
Chapter 30
Quasi-self-similarity for laser-plasma interactions
modelled with fuzzy scaling and genetic algorithms 493
Danilo Rastovic
Efficient Stochastic Simulation to Analyze
Targeted Properties of Biological Systems 505
Hiroyuki Kuwahara, Curtis Madsen, Ivan Mura,
Chris Myers, Abiezer Tejeda and Chris Winstead
Stochastic Decision Support Models and Optimal
Stopping Rules in a New Product Lifetime Testing 533
Nicholas A. Nechval and Maris Purgailis
A non-linear double stochastic model
of return in financial markets 559
Vygintas Gontis, Julius Ruseckas and Aleksejus Kononovičius
Mean-variance hedging under partial information 581
M. Mania, R. Tevzadze and T. Toronjadze
Pertinence and information needs of different subjects
on markets and appropriate operative (tactical or strategic)
stochastic control approaches 609
Vladimir Šimović and Vladimir Šimović, j.r.

Fractional bioeconomic systems:
optimal control problems, theory and applications 629
Darya V. Filatova, Marek Grzywaczewski and Nikolai P. Osmolovskii

Uncertainty presents signicant challenges in the reasoning about and controlling of
complex dynamical systems. To address this challenge, numerous researchers are developing
improved methods for stochastic analysis. This book presents a diverse collection of some of
the latest research in this important area. In particular, this book gives an overview of some
of the theoretical methods and tools for stochastic analysis, and it presents the applications of
these methods to problems in systems theory, science, and economics.
The rst section of the book presents theoretical methods and tools for the analysis of stochastic
systems. The rst two chapters by Sharma et al. present the Fokker-Planck equation and the
Ito calculus. In Chapter 3, Charles presents the use of colored noise with stochastic differential
equations. In Chapter 4, Lopez-Ruiz and Sanchez discuss coupled map lattices and cellular
automata. In Chapter 5, Ohtsubo presents a game theoretic approach. In Chapter 6, Doria
presents an approach that uses Hausdorff outer and inner measures. In Chapter 7, Warin and
Vialle for analysis using distributed algorithms. Finally, in Chapter 8, Spiring explores the use
of Mathematica7.
The second section of the book presents the application of stochastic methods in systems
theory. In Chapter 9, Yang et al. present a learning algorithm for the parity problem. In
Chapter 10, Nechval and Pugailis present an improved technique for state estimation. In
Chapter 11, Serra presents a fuzzy identication method. In Chapter 12, Ferreira and Serra
present an application of fuzzy methods to dynamic systems. The next three chapters by Yan
et al., Perez et al., and Courmontagne explore the problem of ltering for stochastic systems.
In Chapter 16, Olama et al. look at wireless fading channel models. In Chapter 17, Liang
considers information ow and causality quantication. The last two chapters of this section
by Zhang and Zhang and Sokolov consider control systems.
The third section of the book presents the application of stochastic methods to problems in
science. In Chapter 20, Marano and Sgobba present design criteria for vibration control. In
Chapter 21, Kaminski and Lauke consider reinforced elastomers. In Chapter 22, Alessandro

and Francesco discuss structural design. In Chapter 23, Lam et al. apply stochastic methods to
the modeling of earthquake ground motion. In Chapter 24, Rastovic addresses laser-plasma
interactions. Finally, in Chapter 25, Kuwahara et al. apply new, efcient stochastic simulation
methods to biological systems.
Preface
X
The nal section of the book presents the application of stochastic methods to problems in
economics. In Chapter 26, Nechval and Purgailis consider the problem of determining a
products lifetime. In Chapter 27, Gontis et al. applies a stochastic model to nancial markets.
In Chapter 28, Mania et al. take on the problem of hedging in the market. In Chapter 29,
Simovic and Simovic apply stochastic control approaches to tactical and strategic operations
in the market. Finally, in Chapter 30, Darya et al. consider optimal control problems in
fractional bio-economic systems.
Editor
Chris Myers
University of Utah
U.S.A.
The Fokker-Planck equation 1
The Fokker-Planck equation
Shambhu N. Sharma and Hiren G. Patel
X

The Fokker-Planck equation

Shambhu N. Sharma † and Hiren G. Patel‡
Department of Electrical Engineering†
National Institute of Technology, Surat, India

Department of Electrical Engineering
‡

National Institute of Technology, Surat, India


In 1984, H. Risken authored a book (H. Risken, The Fokker-Planck Equation: Methods of
Solution, Application
s, Springer-Verlag, Berlin, New York) discussing the Fokker-Planck
equation for one variable, several variables, methods of solution and its applications,
especially dealing with laser statistics. There has been a considerable progress on the topic
as well as the topic has received greater clarity. For these reasons, it seems worthwhile again
to summarize previous as well as recent developments, spread in literature, on the topic.
The Fokker-Planck equation describes the evolution of conditional probability density for
given initial states for a Markov process, which satisfies the Itô stochastic differential
equation. The structure of the Fokker-Planck equation for the vector case is

,
),,(),(
(
2
1
)
),,(),(
(
),,(
0
2
00
00
T
t
T

tt
x
x
txtxptxGG
tr
x
txtxptxf
tr
t
txtxp
o











where ),( txf
t
is the system non-linearity, ),( txG
t
is termed as the process noise
coefficient, and ),,(
0
ot

txtxp is the conditional probability density. The Fokker-Planck
equation, a prediction density evolution equation, has found its applications in developing
prediction algorithms for stochastic problems arising from physics, mathematical control
theory, mathematical finance, satellite mechanics, as well as wireless communications. In
this chapter, the Authors try to summarize elementary proofs as well as proofs constructed
from the standard theories of stochastic processes to arrive at the Fokker-Planck equation.
This chapter encompasses an approximate solution method to the Fokker-Planck equation
as well as a Fokker-Planck analysis of a Stochastic Duffing-van der Pol (SDvdP) system,
which was recently analysed by one of the Authors.

Key words: The Duffing-van der Pol system, the Galerkin approximation, the Ornstein-
Uhlenbeck process, prediction density, second-order fluctuation equations.

1
Stochastic Control2
1. Introduction
The stochastic differential equation formalism arises from stochastic problems in diverse
field, especially the cases, where stochastic problems are analysed from the dynamical
systems’ point of view. Stochastic differential equations have found applications in
population dynamics, stochastic control, radio-astronomy, stochastic networks, helicopter
rotor dynamics, satellite trajectory estimation problems, protein kinematics, neuronal
activity, turbulence diffusion, stock pricing, seismology, statistical communication theory,
and structural mechanics. A greater detail about stochastic differential equations’
applications can be found in Kloeden and Platen (1991). Some of the standard structures of
stochastic differential equations are the Itô stochastic differential equation, the Stratonovich
stochastic differential equation, the stochastic differential equation involving
p
-differential,
stochastic differential equation in Hida sense, non-Markovian stochastic differential
equations as well as the Ornstein-Uhlenbeck (OU) process-driven stochastic differential

equation. The Itô stochastic differential equation is the standard formalism to analyse
stochastic differential systems, since non-Markovian stochastic differential equations can be
re-formulated as the Itô stochastic differential equation using the extended phase space
formulation, unified coloured noise approximation (Hwalisz et al. 1989). Stochastic
differential systems can be analysed using the Fokker-Planck equation (Jazwinski 1970). The
Fokker-Planck equation is a parabolic linear homogeneous differential equation of order two
in partial differentiation for the transition probability density. The Fokker-Planck operator is
an adjoint operator. In literature, the Fokker-Planck equation is also known as the
Kolmogorov forward equation. The Kolmogorov forward equation can be proved using
mild regularity conditions involving the notion of drift and diffusion coefficients (Feller
2000). The Fokker-Planck equation, definition of the conditional expectation, and integration
by part formula allow to derive the evolution of the conditional moment. In the Risken’s
book, the stochastic differential equation involving the Langevin force was considered and
subsequently, the Fokker-Planck equation was derived. The stochastic differential equation
with the Langevin force can be regarded as the white noise-driven stochastic differential
equation, where the input process satisfies
).(,0 stwww
stt


He considered
the approximate solution methods to the scalar and vector Fokker-Planck equations
involving change of variables, matrix continued-fraction method, numerical integration
method, etc. (Risken 1984, p. 158). Further more, the laser Fokker-Planck equation was
derived.
This book chapter is devoted to summarize alternative approaches to derive the Fokker-
Planck equation involving elementary proofs as well as proofs derived from the Itô
differential rule. In this chapter, the Fokker-Planck analysis hinges on the stochastic
differential equation in the Itô sense in contrast to the Langevin sense. From the
mathemacians’ point of view, the Itô stochastic differential equation involves rigorous

interpretation in contrast to the Langevin stochastic differential equation. On the one hand,
the stochastic differential equation in Itô sense is described as
,),(),(
tttt
dBtxGdttxfdx  on the other, the Langevin stochastic differential
equation assumes the structure ,),(),(
tttt
wtxGtxfx 

where
t
B and
t
w are the
Brownian and white noises respectively. The white noise can be regarded as an informal
non-existent time derivative
t
B

of the Brownian motion .
t
B Kiyoshi Itô, a famous Japanese
mathematician, considered the term
dtBdB
tt

''
and developed Itô differential rule. The
results of Itô calculus were published in two seminal papers of Kiyoshi Itô in 1945. The
approach of this chapter is different and more exact in contrast to the Risken’s book in the

sense that involving the Itô stochastic differential equation, introducing relatively greater
discussion on the Kolmogorov forward and Backward equations. This chapter discusses a
Fokker-Planck analysis of a stochastic Duffing-van der Pol system, an appealing case, from
the dynamical systems’ point of view as well.
This chapter is organised as follows: (i) section 2 discusses the evolution equation of the
prediction density for the Itô stochastic differential equation. A brief discussion about
approximate methods to the Fokker-Planck equation, stochastic differential equation is also
given in section 2 (ii) in section 3, the stochastic Duffing-van der Pol system was analysed to
demonstrate a usefulness of the Fokker-Planck equation. (iii) Section 4 is about the
numerical simulation of the mean and variance evolutions of the SDvdP system. Concluding
remarks are given in section (5).

2. Evolution of conditional probability density
The Fokker-Planck equation describes the evolution of conditional probability density for
given initial states for the Itô stochastic differential system. The equation is also known as
the prediction density evolution equation, since it can be utilized to develop prediction
algorithms, especially where observations are not available at every time instant. One of the
potential applications of the Fokker-Planck equation is to develop estimation algorithms for
the satellite trajectory estimation. This chapter summarizes four different proofs to arrive at
the Fokker-Planck equation. The first two proofs can be regarded as elementary proofs and
the last two utilize the Itô differential rule. Moreover, the Fokker-Planck equation for the OU
process-driven stochastic differential equation is discussed here, where the input process
has non-zero, finite, relatively smaller correlation time.
The first proof of this chapter begins with the Chapman-Kolmogorov equation. The
Chapman-Kolmogorov equation is a consequence of the theory of the Markov process. This
plays a key role in proving the Kolmogorov backward equation (Feller 2000). Here, we
describe briefly the Chapman-Kolmogorov equation and subsequently, the concept of the
conditional probability density as well as transition probability density are introduced to
derive the evolution of conditional probability density for the non-Markov process. The
Fokker-Planck equation becomes a special case of the resulting equation. The conditional

probability density
).() ,(),(
32321321
xxpxxxpxxxp 

Consider the random variables
321
,,
ttt
xxx
at the time instants
321
,, ttt , where
321
ttt  and take values .,,
321
xxx In the theory of the Markov process, the above can
be re-stated as
)()(),(
3221321
xxpxxpxxxp  ,
The Fokker-Planck equation 3
1. Introduction
The stochastic differential equation formalism arises from stochastic problems in diverse
field, especially the cases, where stochastic problems are analysed from the dynamical
systems’ point of view. Stochastic differential equations have found applications in
population dynamics, stochastic control, radio-astronomy, stochastic networks, helicopter
rotor dynamics, satellite trajectory estimation problems, protein kinematics, neuronal
activity, turbulence diffusion, stock pricing, seismology, statistical communication theory,
and structural mechanics. A greater detail about stochastic differential equations’

applications can be found in Kloeden and Platen (1991). Some of the standard structures of
stochastic differential equations are the Itô stochastic differential equation, the Stratonovich
stochastic differential equation, the stochastic differential equation involving
p
-differential,
stochastic differential equation in Hida sense, non-Markovian stochastic differential
equations as well as the Ornstein-Uhlenbeck (OU) process-driven stochastic differential
equation. The Itô stochastic differential equation is the standard formalism to analyse
stochastic differential systems, since non-Markovian stochastic differential equations can be
re-formulated as the Itô stochastic differential equation using the extended phase space
formulation, unified coloured noise approximation (Hwalisz et al. 1989). Stochastic
differential systems can be analysed using the Fokker-Planck equation (Jazwinski 1970). The
Fokker-Planck equation is a parabolic linear homogeneous differential equation of order two
in partial differentiation for the transition probability density. The Fokker-Planck operator is
an adjoint operator. In literature, the Fokker-Planck equation is also known as the
Kolmogorov forward equation. The Kolmogorov forward equation can be proved using
mild regularity conditions involving the notion of drift and diffusion coefficients (Feller
2000). The Fokker-Planck equation, definition of the conditional expectation, and integration
by part formula allow to derive the evolution of the conditional moment. In the Risken’s
book, the stochastic differential equation involving the Langevin force was considered and
subsequently, the Fokker-Planck equation was derived. The stochastic differential equation
with the Langevin force can be regarded as the white noise-driven stochastic differential
equation, where the input process satisfies
).(,0 stwww
stt


He considered
the approximate solution methods to the scalar and vector Fokker-Planck equations
involving change of variables, matrix continued-fraction method, numerical integration

method, etc. (Risken 1984, p. 158). Further more, the laser Fokker-Planck equation was
derived.
This book chapter is devoted to summarize alternative approaches to derive the Fokker-
Planck equation involving elementary proofs as well as proofs derived from the Itô
differential rule. In this chapter, the Fokker-Planck analysis hinges on the stochastic
differential equation in the Itô sense in contrast to the Langevin sense. From the
mathemacians’ point of view, the Itô stochastic differential equation involves rigorous
interpretation in contrast to the Langevin stochastic differential equation. On the one hand,
the stochastic differential equation in Itô sense is described as
,),(),(
tttt
dBtxGdttxfdx

 on the other, the Langevin stochastic differential
equation assumes the structure ,),(),(
tttt
wtxGtxfx



where
t
B and
t
w are the
Brownian and white noises respectively. The white noise can be regarded as an informal
non-existent time derivative
t
B


of the Brownian motion .
t
B Kiyoshi Itô, a famous Japanese
mathematician, considered the term
dtBdB
tt

''
and developed Itô differential rule. The
results of Itô calculus were published in two seminal papers of Kiyoshi Itô in 1945. The
approach of this chapter is different and more exact in contrast to the Risken’s book in the
sense that involving the Itô stochastic differential equation, introducing relatively greater
discussion on the Kolmogorov forward and Backward equations. This chapter discusses a
Fokker-Planck analysis of a stochastic Duffing-van der Pol system, an appealing case, from
the dynamical systems’ point of view as well.
This chapter is organised as follows: (i) section 2 discusses the evolution equation of the
prediction density for the Itô stochastic differential equation. A brief discussion about
approximate methods to the Fokker-Planck equation, stochastic differential equation is also
given in section 2 (ii) in section 3, the stochastic Duffing-van der Pol system was analysed to
demonstrate a usefulness of the Fokker-Planck equation. (iii) Section 4 is about the
numerical simulation of the mean and variance evolutions of the SDvdP system. Concluding
remarks are given in section (5).

2. Evolution of conditional probability density
The Fokker-Planck equation describes the evolution of conditional probability density for
given initial states for the Itô stochastic differential system. The equation is also known as
the prediction density evolution equation, since it can be utilized to develop prediction
algorithms, especially where observations are not available at every time instant. One of the
potential applications of the Fokker-Planck equation is to develop estimation algorithms for
the satellite trajectory estimation. This chapter summarizes four different proofs to arrive at

the Fokker-Planck equation. The first two proofs can be regarded as elementary proofs and
the last two utilize the Itô differential rule. Moreover, the Fokker-Planck equation for the OU
process-driven stochastic differential equation is discussed here, where the input process
has non-zero, finite, relatively smaller correlation time.
The first proof of this chapter begins with the Chapman-Kolmogorov equation. The
Chapman-Kolmogorov equation is a consequence of the theory of the Markov process. This
plays a key role in proving the Kolmogorov backward equation (Feller 2000). Here, we
describe briefly the Chapman-Kolmogorov equation and subsequently, the concept of the
conditional probability density as well as transition probability density are introduced to
derive the evolution of conditional probability density for the non-Markov process. The
Fokker-Planck equation becomes a special case of the resulting equation. The conditional
probability density
).() ,(),(
32321321
xxpxxxpxxxp 

Consider the random variables
321
,,
ttt
xxx
at the time instants
321
,, ttt , where
321
ttt  and take values .,,
321
xxx In the theory of the Markov process, the above can
be re-stated as
)()(),(

3221321
xxpxxpxxxp  ,
Stochastic Control4
integrating over the variable
2
x , we have

,)()()(
2322131
dxxxpxxpxxp



introducing the notion of the transition probability density and time instants

.),(),(),(
2322131
dxxxqxxqxxq
stst





Consider the multi-dimensional probability density
)()(),(
22121
xpxxpxxp  and
integrating over the variable
2

x
, we have

,)()()(
22211
dxxpxxpxp


or


,)(),()(
2221,1
21
dxxpxxqxp
tt

 (1)
where
),(
21,
21
xxq
tt
is the transition probability density and .
.21
tt  The transition
probability density
),(
21,

21
xxq
tt
is the inverse Fourier transform of the characteristic
function
,
)(
21
tt
xxiu
Ee

i.e.

.
2
1
),(
)(
)(
21,
2121
21
duEeexxq
tt
xxiu
xxiu
tt






(2)

Equation (1) in combination with equation (2) leads to


.)()(
2
1
)(
22
)(
)(
1
2121
dudxxpEeexp
tt
xxiu
xxiu





(3)

The characteristic function is the moment generating function, the characteristic
function

n
tt
n
n
xxiu
xx
n
iu
Ee
tt
)(
!
)(
21
11
0
)(




. After introducing the definition of the
characteristic function, equation (3) can be recast as

dudxxpxx
n
iu
exp
n
n

tt
n
xxiu
22
0
)(
1
)())(
!
)(
(
2
1
)(
21
21







.)()())(
2
1
(
!
1
2

0
2
)(
21
21
dxxpxxdueiu
n
n
n
tt
xxiu
n

 





The term



duiue
n
xxiu
)(
2
1
)(

21

)()(
21
1
xx
x
n





and leads to the probability
density

)(
1
xp .)()()()(
!
1
22
0
21
1
21
dxxpxxxx
xn
n
tt

n
n








(4)

For the short hand notation, introducing the notion of the stochastic process,
taking
xxxx


21
,

, where the time instants
,,
21
tttt




equation (4) can be
recast as

dxxpxxxx
xn
xp
n
n
n
)()()()(
!
1
)(
0












dxxpxkxx
xn
n
n
n
)()()()(
!

1
0










,
where
)(
)(
xk
xx
n
n




and the time interval condition 0

leads to
),()()(
!
1

)()(
1
0
xpxk
xn
xpxp
Lt
n
n
n













or
).()()(
!
1
)(
1
xpxk

xn
xp
n
n
n








Note that the above density evolution equation is derived for the arbitrary stochastic
process
)0,(



 txX
t
. Here, the arbitrary process means that there is no restriction
imposed on the process while deriving the density evolution equation and can be regarded
as the non-Markov process. Consider a Markov process, which satisfies the Itô stochastic
differential equation, the evolution of conditional probability density retains only the first
two terms
)(
1
xk and ),(
2

xk which is a direct consequence of the stochastic differential rule
for the Itô stochastic differential equation in combination with the definition
).(
)(
xk
xx
n
n




As a result of these, the evolution of conditional probability
density for the scalar stochastic differential equation of the form

,),(),(
tttt
dBtxgdttxfdx




The Fokker-Planck equation 5
integrating over the variable
2
x , we have

,)()()(
2322131
dxxxpxxpxxp




introducing the notion of the transition probability density and time instants

.),(),(),(
2322131
dxxxqxxqxxq
stst





Consider the multi-dimensional probability density
)()(),(
22121
xpxxpxxp  and
integrating over the variable
2
x
, we have

,)()()(
22211
dxxpxxpxp


or



,)(),()(
2221,1
21
dxxpxxqxp
tt

 (1)
where
),(
21,
21
xxq
tt
is the transition probability density and .
.21
tt  The transition
probability density
),(
21,
21
xxq
tt
is the inverse Fourier transform of the characteristic
function
,
)(
21
tt
xxiu

Ee

i.e.

.
2
1
),(
)(
)(
21,
2121
21
duEeexxq
tt
xxiu
xxiu
tt





(2)

Equation (1) in combination with equation (2) leads to


.)()(
2

1
)(
22
)(
)(
1
2121
dudxxpEeexp
tt
xxiu
xxiu





(3)

The characteristic function is the moment generating function, the characteristic
function
n
tt
n
n
xxiu
xx
n
iu
Ee
tt

)(
!
)(
21
11
0
)(




. After introducing the definition of the
characteristic function, equation (3) can be recast as

dudxxpxx
n
iu
exp
n
n
tt
n
xxiu
22
0
)(
1
)())(
!
)(

(
2
1
)(
21
21







.)()())(
2
1
(
!
1
2
0
2
)(
21
21
dxxpxxdueiu
n
n
n
tt

xxiu
n

 





The term



duiue
n
xxiu
)(
2
1
)(
21

)()(
21
1
xx
x
n






and leads to the probability
density

)(
1
xp .)()()()(
!
1
22
0
21
1
21
dxxpxxxx
xn
n
tt
n
n









(4)

For the short hand notation, introducing the notion of the stochastic process,
taking
xxxx 
21
,

, where the time instants
,,
21
tttt 

equation (4) can be
recast as
dxxpxxxx
xn
xp
n
n
n
)()()()(
!
1
)(
0













dxxpxkxx
xn
n
n
n
)()()()(
!
1
0










,
where
)(

)(
xk
xx
n
n




and the time interval condition 0

leads to
),()()(
!
1
)()(
1
0
xpxk
xn
xpxp
Lt
n
n
n














or
).()()(
!
1
)(
1
xpxk
xn
xp
n
n
n








Note that the above density evolution equation is derived for the arbitrary stochastic
process

)0,(  txX
t
. Here, the arbitrary process means that there is no restriction
imposed on the process while deriving the density evolution equation and can be regarded
as the non-Markov process. Consider a Markov process, which satisfies the Itô stochastic
differential equation, the evolution of conditional probability density retains only the first
two terms
)(
1
xk and ),(
2
xk which is a direct consequence of the stochastic differential rule
for the Itô stochastic differential equation in combination with the definition
).(
)(
xk
xx
n
n




As a result of these, the evolution of conditional probability
density for the scalar stochastic differential equation of the form

,),(),(
tttt
dBtxgdttxfdx 


Stochastic Control6
leads to the Fokker-Planck equation,

),(
),(
2
1
)(),()(
2
22
xp
x
txg
xptxf
x
xp









where
),,()(
1
txfxk  ).,()(
2

2
txgxk  The Fokker-Planck equation can be recast as
,)()( dtxLpxdp  where the vector version of the Fokker-Planck operator

(.)L







ji
ji
ij
T
i
i
i
xx
txGG
txf
x
,
2
)(.),()(
2
1
)(.),(
.


The Fokker-Planck operator is an adjoint operator, since
,,)(, pLpL



where
(.)L

is the Kolmogorov backward operator. This property is utilized in deriving the
evolution
)(
t
xd


of the conditional moment (Jazwinski 1970). The Fokker-Planck equation
is also known as the Kolmogorov Forward equation.
The second proof of this chapter begins with the Green function, the Kolmogorov forward
and backward equations involve the notion of the drift and diffusion coefficients as well as
mild regularity conditions (Feller 2000). The drift and diffusion coefficients are regarded as
the system non-linearity and the ‘stochastic perturbation in the variance evolution’
respectively in noisy dynamical system theory. Here, we explain briefly about the formalism
associated with the proof of the Kolmogorov forward and backward equations. Consider the
Green’s function
,)(),()(
0
dyyuyxqxu
tt


 (5)
where ),( yxq
t
is the transition probability density, )(xu
t
is a scalar function,
x
is the
initial point and
y
is the final point. Equation (5) is modified at the time duration
ht 
as

.)(),()(
0
dyyuyxqxu
htht


 (6)

The Chapman-Kolmogorov equation can be stated as


.),(),(),(

dyqxqyxq
htht




(7)

Making the use of equations (6)-(7) and the Taylor series expansion with mild regularity
conditions leads to
,
)(
)(
2
1
)(
)(
)(
2
2
x
xu
xa
x
xu
xb
t
xu
ttt









(8)

where
h
uu
Lt
t
xu
tht
h
t





0
)(
,
)(xb
and
)(xa
are the drift and diffusion coefficients
respectively (Feller 2000), and the detailed proof of equation (8) can be found in a celebrated
book authored by Feller (2000). For the vector case, the Kolmogorov backward equation can
be recast as
 









,
)(
)(
2
1
)(
)(
)(
2
ji
t
ij
i
t
i
t
xx
xu
xa
x
xu
xb

t
xu


where the summation is extended for
.1,1 njni




From the dynamical systems’
point of view, the vector case of the Kolmogorov backward equation can be reformulated as

 








,
)(
),()(
2
1
)(
),(
)(

2
ji
t
ij
i
t
i
t
xx
xu
txGG
x
xu
txf
t
xu


where the mappings
f and G are the system non-linearity and process noise coefficient
matrix respectively and the Kolmogorov backward operator



(.)L .
(.)
),()(
2
1(.)
),(

2
 





i
ji
ij
i
i
xx
txGG
x
txf


Note that the Kolmogorov backward equation is a parabolic linear homogeneous differential
equation of order two in partial differentiation, since the backward operator is a linear
operator and the homogeneity condition holds. The Kolmogorov forward equation can be
derived using the relation
,)(),()(
0
dxxvyxqyv
ss


in combination with integration by part formula as well as mild regularity conditions (Feller
2000) lead to the expression



.
)()(
2
1
)()()(
2
2
y
yvya
y
yvyb
s
yv
sss








(9)

The terms
)(yb and )(ya of equation (9) have similar interpretations as the terms of
equation (8). The vector version of equation (9) is
The Fokker-Planck equation 7

leads to the Fokker-Planck equation,

),(
),(
2
1
)(),()(
2
22
xp
x
txg
xptxf
x
xp









where
),,()(
1
txfxk  ).,()(
2
2

txgxk  The Fokker-Planck equation can be recast as
,)()( dtxLpxdp  where the vector version of the Fokker-Planck operator

(.)L







ji
ji
ij
T
i
i
i
xx
txGG
txf
x
,
2
)(.),()(
2
1
)(.),(
.


The Fokker-Planck operator is an adjoint operator, since
,,)(, pLpL



where
(.)L

is the Kolmogorov backward operator. This property is utilized in deriving the
evolution
)(
t
xd


of the conditional moment (Jazwinski 1970). The Fokker-Planck equation
is also known as the Kolmogorov Forward equation.
The second proof of this chapter begins with the Green function, the Kolmogorov forward
and backward equations involve the notion of the drift and diffusion coefficients as well as
mild regularity conditions (Feller 2000). The drift and diffusion coefficients are regarded as
the system non-linearity and the ‘stochastic perturbation in the variance evolution’
respectively in noisy dynamical system theory. Here, we explain briefly about the formalism
associated with the proof of the Kolmogorov forward and backward equations. Consider the
Green’s function
,)(),()(
0
dyyuyxqxu
tt

 (5)

where ),( yxq
t
is the transition probability density, )(xu
t
is a scalar function,
x
is the
initial point and
y
is the final point. Equation (5) is modified at the time duration
ht 
as

.)(),()(
0
dyyuyxqxu
htht


 (6)

The Chapman-Kolmogorov equation can be stated as


.),(),(),(

dyqxqyxq
htht




(7)

Making the use of equations (6)-(7) and the Taylor series expansion with mild regularity
conditions leads to
,
)(
)(
2
1
)(
)(
)(
2
2
x
xu
xa
x
xu
xb
t
xu
ttt









(8)

where
h
uu
Lt
t
xu
tht
h
t





0
)(
,
)(xb
and
)(xa
are the drift and diffusion coefficients
respectively (Feller 2000), and the detailed proof of equation (8) can be found in a celebrated
book authored by Feller (2000). For the vector case, the Kolmogorov backward equation can
be recast as
 









,
)(
)(
2
1
)(
)(
)(
2
ji
t
ij
i
t
i
t
xx
xu
xa
x
xu
xb
t

xu


where the summation is extended for
.1,1 njni




From the dynamical systems’
point of view, the vector case of the Kolmogorov backward equation can be reformulated as

 








,
)(
),()(
2
1
)(
),(
)(
2

ji
t
ij
i
t
i
t
xx
xu
txGG
x
xu
txf
t
xu


where the mappings
f and G are the system non-linearity and process noise coefficient
matrix respectively and the Kolmogorov backward operator



(.)L .
(.)
),()(
2
1(.)
),(
2

 





i
ji
ij
i
i
xx
txGG
x
txf


Note that the Kolmogorov backward equation is a parabolic linear homogeneous differential
equation of order two in partial differentiation, since the backward operator is a linear
operator and the homogeneity condition holds. The Kolmogorov forward equation can be
derived using the relation
,)(),()(
0
dxxvyxqyv
ss


in combination with integration by part formula as well as mild regularity conditions (Feller
2000) lead to the expression



.
)()(
2
1
)()()(
2
2
y
yvya
y
yvyb
s
yv
sss








(9)

The terms
)(yb and )(ya of equation (9) have similar interpretations as the terms of
equation (8). The vector version of equation (9) is
Stochastic Control8
,

)()(
2
1
)()()(
,
2









ji
ji
sij
i
i
sis
yy
yvya
y
yvyb
s
yv

and the Kolmogorov forward operator
(.)L .

)(.)(
2
1
)(.)(
,
2







ji
ji
ij
i
i
i
yy
ya
y
yb
For
ii
fb  and
j
i
T
j

i
GGa )( , the Kolmogorov forward operator assumes the structure
of the Fokker-Planck operator and is termed as the Kolmogorov-Fokker-Planck operator.
The third proof of the chapter explains how the Fokker-Planck equation can be derived using
the definition of conditional expectation and Itô differential rule.


)).)((()( xxxEExE
tdttdtt



(10)
The Taylor series expansion of the scalar function )(
!
)(
)(
t
m
m
m
t
dtt
x
m
dx
x





and

),(
!
),(
))((
20
x
m
tx
xxxE
m
m
m
tdtt






 (11)

where
))((),( xxdxEtx
t
m
tm



and the summing variable m takes values upto
two for the Brownian motion process-driven stochastic differential equation that can be
explained via the Itô differential rule. Equation (10) in conjunction with equation (11) leads
to


)(
dtt
xE

)),(
!
),(
(
20
x
m
tx
E
m
m
m






the definition of ‘expectation’ leads to the following expression:




dxdttxpx ),()(

,),()(
!
),(
20
dxtxpx
m
tx
m
m
m





(12)

the integration by part, applying to equation (12), leads to the Fokker-Planck equation,

,
),(),(
!
)1(
),(
21









m
m
m
m
m
x
txptx
mt
txp



where

),,(
1
txf

).,(
2
2
txg



Finally, we derive the Fokker-Planck equation using the concept of the evolution of the
conditional moment and the conditional characteristic function. Consider the state
vector ,Ux
t
 RU :

, i.e. ,)( Rx
t


and the phase space .
n
RU  The state
vector
t
x satisfies the Itô SDE as well. Suppose the function )(
t
x

is twice differentiable.
The evolution
)(
t
xd


of the conditional moment is the standard formalism to analyse
stochastic differential systems. Further more, )(

t
xd


),)((
0
0
txxdE
tt

 holds. A
greater detail can be found in Sharma (2008). The stochastic evolution )(
t
xd

of the scalar
function )(
t
x

(Sage and Melsa 1971) can be stated as

2
2
)(
),()(
2
1
),(
)(

()(
i
t
tii
T
i
ti
i
i
t
t
x
x
txGGtxf
x
x
xd











dt
xx

x
txGG
ji
t
ji
tij
T
)
)(
),()(
2






,),(
)(
1,1



dBtxG
x
x
ti
rni
i
t






(13)
thus
.)
)(
),()(
)(
),()(
2
1
)(
),(()(
2
2
2
dt
xx
x
txGG
x
x
txGG
x
x
txfxd
qp

qp
t
tpq
T
p
p
t
tpp
T
p
p
t
tpt













  
























Note that the expected value of the last term of the right-hand side of equation (13) vanishes,
i.e.
.0),( 

dBtxG
ti
Consider ,)(
t
T
xS

t
ex 

the evolution of the characteristic
function becomes

.)),()(
)(
),()(
2
1
),(()(
2
2
2
dtetxGGSS
x
x
txGGSetxfSedE
qp
xS
tpq
T
qp
p
p
t
tpp
T
p

p
xS
tpp
xS
t
T
t
T
t
T









  














  



Making the use of the definition of the characteristic function as well as the integration by
part formula, we arrive at the Fokker-Planck equation.
The Kushner equation, the filtering density evolution equation for the Itô stochastic
differential equation, is a ‘generalization’ of the Fokker-Planck equation. The Kushner
equation is a partial-integro stochastic differential equation, i.e.


,)()()(
1
pdthdzhhdtpLdp
tn
T





(14)
The Fokker-Planck equation 9
,
)()(
2
1

)()()(
,
2









ji
ji
sij
i
i
sis
yy
yvya
y
yvyb
s
yv

and the Kolmogorov forward operator
(.)L .
)(.)(
2
1

)(.)(
,
2







ji
ji
ij
i
i
i
yy
ya
y
yb
For
ii
fb  and
j
i
T
j
i
GGa )( , the Kolmogorov forward operator assumes the structure
of the Fokker-Planck operator and is termed as the Kolmogorov-Fokker-Planck operator.

The third proof of the chapter explains how the Fokker-Planck equation can be derived using
the definition of conditional expectation and Itô differential rule.


)).)((()( xxxEExE
tdttdtt



(10)
The Taylor series expansion of the scalar function )(
!
)(
)(
t
m
m
m
t
dtt
x
m
dx
x




and


),(
!
),(
))((
20
x
m
tx
xxxE
m
m
m
tdtt






 (11)

where
))((),( xxdxEtx
t
m
tm


and the summing variable m takes values upto
two for the Brownian motion process-driven stochastic differential equation that can be

explained via the Itô differential rule. Equation (10) in conjunction with equation (11) leads
to


)(
dtt
xE

)),(
!
),(
(
20
x
m
tx
E
m
m
m






the definition of ‘expectation’ leads to the following expression:




dxdttxpx ),()(

,),()(
!
),(
20
dxtxpx
m
tx
m
m
m





(12)

the integration by part, applying to equation (12), leads to the Fokker-Planck equation,

,
),(),(
!
)1(
),(
21









m
m
m
m
m
x
txptx
mt
txp



where

),,(
1
txf


).,(
2
2
txg



Finally, we derive the Fokker-Planck equation using the concept of the evolution of the
conditional moment and the conditional characteristic function. Consider the state
vector ,Ux
t
 RU :

, i.e. ,)( Rx
t


and the phase space .
n
RU  The state
vector
t
x satisfies the Itô SDE as well. Suppose the function )(
t
x

is twice differentiable.
The evolution
)(
t
xd


of the conditional moment is the standard formalism to analyse
stochastic differential systems. Further more, )(
t
xd



),)((
0
0
txxdE
tt

 holds. A
greater detail can be found in Sharma (2008). The stochastic evolution )(
t
xd

of the scalar
function )(
t
x

(Sage and Melsa 1971) can be stated as

2
2
)(
),()(
2
1
),(
)(
()(
i

t
tii
T
i
ti
i
i
t
t
x
x
txGGtxf
x
x
xd











dt
xx
x
txGG

ji
t
ji
tij
T
)
)(
),()(
2






,),(
)(
1,1



dBtxG
x
x
ti
rni
i
t






(13)
thus
.)
)(
),()(
)(
),()(
2
1
)(
),(()(
2
2
2
dt
xx
x
txGG
x
x
txGG
x
x
txfxd
qp
qp
t

tpq
T
p
p
t
tpp
T
p
p
t
tpt













  
























Note that the expected value of the last term of the right-hand side of equation (13) vanishes,
i.e.
.0),( 

dBtxG
ti
Consider ,)(
t
T
xS
t
ex 


the evolution of the characteristic
function becomes

.)),()(
)(
),()(
2
1
),(()(
2
2
2
dtetxGGSS
x
x
txGGSetxfSedE
qp
xS
tpq
T
qp
p
p
t
tpp
T
p
p
xS

tpp
xS
t
T
t
T
t
T









  
  
  



Making the use of the definition of the characteristic function as well as the integration by
part formula, we arrive at the Fokker-Planck equation.
The Kushner equation, the filtering density evolution equation for the Itô stochastic
differential equation, is a ‘generalization’ of the Fokker-Planck equation. The Kushner
equation is a partial-integro stochastic differential equation, i.e.



,)()()(
1
pdthdzhhdtpLdp
tn
T





(14)
Stochastic Control10
where (.)L is the Fokker-Planck operator, ),,,(
0
ttztxpp 


the observation
,),(
0


t
t
tt
Bdxhz


and ),( txh
t

is the measurement non-linearity. Harald J
Kushner first derived the expression of the filtering density and subsequently, the filtering
density evolution equation using the stochastic differential rule (Jazwinski 1970). Liptser-
Shiryayev discovered an alternative proof of the filtering density evolution, equation (14),
involving the following steps: (i) derive the stochastic evolution


)(
t
xd

of the conditional
moment, where
),)(()(
0
ttzxEx
tt





(ii) subsequently, the stochastic
evolution of the conditional characteristic function can be regarded as a special case of the
conditional moment evolution, where
t
T
xS
t
ex )(


(iii) the definition of the conditional
expectation as well as integration by part formula lead to the filtering density evolution
equation, see Liptser and Shiryayev (1977). RL Stratonovich developed the filtering density
evolution for stochastic differential equation involving the
2
1
-differential as well. For this
reason, the filtering density evolution equation is also termed as the Kushner-Stratonovich
equation.
Consider the stochastic differential equation of the form


,)()(
tttt
xgxfx



(15)

where
t

is the Ornstein-Uhlenbeck process and generates the process
t
x , a non-Markov
process. The evolution of conditional probability density for the non-Markov process with
the input process with a non-zero, finite, smaller correlation time
cor


, i.e. 10 
cor

,
reduces to the Fokker-Planck equation. One of the approaches to arrive at the Fokker-Planck
equation for the OU process-driven stochastic differential equation with smaller correlation
time is function calculus. The function calculus approach involves the notion of the
functional derivative. The evolution of conditional probability density for the output
process
t
x , where the input process
t

is a zero mean, stationary and Gaussian process,
can be written (Hänggi 1995, p.85) as

),)()()((
)(
)(
2
ds
x
xxstC
x
xg
xx
pxf
xp
s

t
t
t
t
t
o














(16)
where the second-order cumulant of the zero mean, stationary and Gaussian process is
)(),cov(),(
2
stRstC
st







and
s
t
x


is the functional derivative of the process
t
x with respect to the input process .
s

The integral counterpart of equation (15) is

.)()(
0
0


dxgxfxx
t
t
tt


The functional derivative
s
t
x



depends on the time interval ts



and can be stated as

,))()()((














dxg
x
xg
x
xf
x
s

t
s
s
t
ss
t











)(
s
xg
,))()((











d
x
xg
x
xf
t
s
ss




(17)

Making the repetitive use of the expression
s
t
x


within the integral sign of equation (17),
we have
s
t
x


exp()(
s

xg

,)



t
s
d
x
x






)).)()((exp()(





t
s
s
dxgxfxg


(18)


More over, the time derivative of the process noise coefficient )(
t
xg of equation (15) can be
written as
ttt
xxgxg

)()(



),)()()((
tttt
xgxfxg






after some calculations, the integral counterpart of the above equation can be stated as


)).))(
)(
)()(
((exp()()(





dxg
xg
xfxg
xgxg
t
s
ts





(19)
The Fokker-Planck equation 11
where (.)L is the Fokker-Planck operator, ),,,(
0
ttztxpp 


the observation
,),(
0


t
t
tt
Bdxhz



and ),( txh
t
is the measurement non-linearity. Harald J
Kushner first derived the expression of the filtering density and subsequently, the filtering
density evolution equation using the stochastic differential rule (Jazwinski 1970). Liptser-
Shiryayev discovered an alternative proof of the filtering density evolution, equation (14),
involving the following steps: (i) derive the stochastic evolution




)(
t
xd

of the conditional
moment, where
),)(()(
0
ttzxEx
tt








(ii) subsequently, the stochastic
evolution of the conditional characteristic function can be regarded as a special case of the
conditional moment evolution, where
t
T
xS
t
ex )(

(iii) the definition of the conditional
expectation as well as integration by part formula lead to the filtering density evolution
equation, see Liptser and Shiryayev (1977). RL Stratonovich developed the filtering density
evolution for stochastic differential equation involving the
2
1
-differential as well. For this
reason, the filtering density evolution equation is also termed as the Kushner-Stratonovich
equation.
Consider the stochastic differential equation of the form


,)()(
tttt
xgxfx




(15)


where
t

is the Ornstein-Uhlenbeck process and generates the process
t
x , a non-Markov
process. The evolution of conditional probability density for the non-Markov process with
the input process with a non-zero, finite, smaller correlation time
cor

, i.e. 10 
cor

,
reduces to the Fokker-Planck equation. One of the approaches to arrive at the Fokker-Planck
equation for the OU process-driven stochastic differential equation with smaller correlation
time is function calculus. The function calculus approach involves the notion of the
functional derivative. The evolution of conditional probability density for the output
process
t
x , where the input process
t

is a zero mean, stationary and Gaussian process,
can be written (Hänggi 1995, p.85) as

),)()()((
)(
)(
2

ds
x
xxstC
x
xg
xx
pxf
xp
s
t
t
t
t
t
o














(16)

where the second-order cumulant of the zero mean, stationary and Gaussian process is
)(),cov(),(
2
stRstC
st




and
s
t
x


is the functional derivative of the process
t
x with respect to the input process .
s

The integral counterpart of equation (15) is

.)()(
0
0


dxgxfxx
t
t

tt


The functional derivative
s
t
x


depends on the time interval ts



and can be stated as

,))()()((















dxg
x
xg
x
xf
x
s
t
s
s
t
ss
t








 )(
s
xg
,))()((











d
x
xg
x
xf
t
s
ss




(17)

Making the repetitive use of the expression
s
t
x


within the integral sign of equation (17),
we have
s
t
x



exp()(
s
xg
,)



t
s
d
x
x






)).)()((exp()(





t
s
s
dxgxfxg



(18)

More over, the time derivative of the process noise coefficient )(
t
xg of equation (15) can be
written as
ttt
xxgxg

)()(



),)()()((
tttt
xgxfxg





after some calculations, the integral counterpart of the above equation can be stated as


)).))(
)(
)()(
((exp()()(





dxg
xg
xfxg
xgxg
t
s
ts





(19)
Stochastic Control12
Equation (18) in combination with equation (19) leads to


).)
)(
)(
)()((exp()(






t
s
t
s
t
d
xg
xf
xgxfxg
x






(20)
Furthermore, the Taylor series expansion of the functional derivative
s
t
x


in powers of
)( ts  can be stated as

).)(())()((
2
tsOts
x

s
ts
xx
s
t
t
t
s
t










(21)

From equation (20), we have


),(
t
t
t
xg
x




(22)

).
)(
)(
)()()((
t
t
ttt
s
t
xg
xf
xgxfxgts
x
s








(23)

After retaining the first two terms of the right-hand side of equation (21) and equations (22)-

(23) in combination with equation (21) lead to

)()( stxg
x
t
s
t



)
)(
)(
)()()((
t
t
ttt
xg
xf
xgxfxg




)(1)(( stxg
t



)

)(
)()()()(
(
t
tttt
xg
xfxgxgxf



),
thus

).())
)(
)(
)(()(1)(()( xp
xg
xf
xgstxg
x
xx
s
t
t






(24)

The autocorrelation
)( stR 

of the OU process satisfying the stochastic differential
equation
t
cor
t
cor
t
dB
D
dtd




21


 becomes

.)(
cor
st
cor
e
D

stR





 (25)
Equations (24)-(25) in conjunction with equation (16) give

).))(1(()( p
g
f
gg
x
g
x
Dfp
x
xp
cor















The Kolmogorov-Fokker-Planck equation and the Kolmogorov backward equation are
exploited to analyse the Itô stochastic differential equation by deriving the evolution of the
conditional moment. The evolutions of conditional mean and variance are the special cases
of the conditional moment evolution. The conditional mean and variance evolutions are
infinite dimensional as well as involve higher-order moments. For these reasons,
approximate mean and variance evolutions are derived and examined involving numerical
experiments. Alternatively, the Carleman linearization to the exact stochastic differential
equation resulting the bilinear stochastic differential equation has found applications in
developing the approximate estimation procedure. The Carleman linearization transforms a
finite dimensional non-linear system into a system of infinite dimensional linear systems
(Kowalski and Steeb 1991).
The exact solution of the Fokker-Planck equation is possible for the simpler form of the
stochastic differential equation, e.g.


.
tt
adBdx

(26)

The Fokker-Planck equation for equation (26) becomes

.
),,(
2

1
),,(
2
0
2
2
0
00
x
txtxp
a
t
txtxp
tt







Consider the process ),0(
2
taN and its probability density

ta
x
t
e
ta

txtxp
2
2
0
2
2
0
2
1
),,(





satisfies equation (26). However, the closed-form solution to the Fokker-Planck equation for
the non-linear stochastic differential equation is not possible, the approximate solution to
the Fokker-Planck equation is derived. The Galerkin approximation to the Fokker-Planck
equation received some attention in literature. The Galerkin approximation can be applied
to the Kushner equation as well. More generally, the usefulness of the Galerkin
approximation to the partial differential equation and the stochastic differential equation for
the approximate solution can be explored. The theory of the Galerkin approximation is
grounded on the orthogonal projection lemma. For a greater detail, an authoritative book,
computational Galerkin methods, authored by C A J Fletcher can be consulted (Fletcher
1984).
The Fokker-Planck equation 13
Equation (18) in combination with equation (19) leads to


).)

)(
)(
)()((exp()(





t
s
t
s
t
d
xg
xf
xgxfxg
x






(20)
Furthermore, the Taylor series expansion of the functional derivative
s
t
x



in powers of
)( ts  can be stated as

).)(())()((
2
tsOts
x
s
ts
xx
s
t
t
t
s
t










(21)

From equation (20), we have



),(
t
t
t
xg
x



(22)

).
)(
)(
)()()((
t
t
ttt
s
t
xg
xf
xgxfxgts
x
s









(23)

After retaining the first two terms of the right-hand side of equation (21) and equations (22)-
(23) in combination with equation (21) lead to

)()( stxg
x
t
s
t



)
)(
)(
)()()((
t
t
ttt
xg
xf
xgxfxg





)(1)(( stxg
t



)
)(
)()()()(
(
t
tttt
xg
xfxgxgxf



),
thus

).())
)(
)(
)(()(1)(()( xp
xg
xf
xgstxg
x
xx

s
t
t





(24)

The autocorrelation
)( stR


of the OU process satisfying the stochastic differential
equation
t
cor
t
cor
t
dB
D
dtd




21



 becomes

.)(
cor
st
cor
e
D
stR





 (25)
Equations (24)-(25) in conjunction with equation (16) give

).))(1(()( p
g
f
gg
x
g
x
Dfp
x
xp
cor















The Kolmogorov-Fokker-Planck equation and the Kolmogorov backward equation are
exploited to analyse the Itô stochastic differential equation by deriving the evolution of the
conditional moment. The evolutions of conditional mean and variance are the special cases
of the conditional moment evolution. The conditional mean and variance evolutions are
infinite dimensional as well as involve higher-order moments. For these reasons,
approximate mean and variance evolutions are derived and examined involving numerical
experiments. Alternatively, the Carleman linearization to the exact stochastic differential
equation resulting the bilinear stochastic differential equation has found applications in
developing the approximate estimation procedure. The Carleman linearization transforms a
finite dimensional non-linear system into a system of infinite dimensional linear systems
(Kowalski and Steeb 1991).
The exact solution of the Fokker-Planck equation is possible for the simpler form of the
stochastic differential equation, e.g.


.
tt

adBdx  (26)

The Fokker-Planck equation for equation (26) becomes

.
),,(
2
1
),,(
2
0
2
2
0
00
x
txtxp
a
t
txtxp
tt







Consider the process ),0(
2

taN and its probability density

ta
x
t
e
ta
txtxp
2
2
0
2
2
0
2
1
),,(





satisfies equation (26). However, the closed-form solution to the Fokker-Planck equation for
the non-linear stochastic differential equation is not possible, the approximate solution to
the Fokker-Planck equation is derived. The Galerkin approximation to the Fokker-Planck
equation received some attention in literature. The Galerkin approximation can be applied
to the Kushner equation as well. More generally, the usefulness of the Galerkin
approximation to the partial differential equation and the stochastic differential equation for
the approximate solution can be explored. The theory of the Galerkin approximation is
grounded on the orthogonal projection lemma. For a greater detail, an authoritative book,

computational Galerkin methods, authored by C A J Fletcher can be consulted (Fletcher
1984).
Stochastic Control14
3. A stochastic Duffing-van der Pol system
The second-order fluctuation equation describes a dynamical system in noisy environment.
The second-order fluctuation equation can be regarded as

).,,,(
tttt
BxxtFx




The phase space formulation allows transforming a single equation of order
n into a
system of
n first-order differential equations. Choose
1
xx
t

21
xx 

,
),,,,(
212 t
BxxtFx





by considering a special case of the above system of equations, we have

,
21
dtxdx 
,),,(),,(
2122122 t
dBxxtgdtxxtfdx 

in matrix-vector format


tt
dBxxtGdtxxtfd ),,(),,(
2121


, (27)

where

 


,,,
2121
TT

t
xx



,,),(
22
T
t
fxtf 



.,0),(
22
T
t
gtG 



The stochastic Duffing-van der Pol system can be formulated in the form of equation (27)
(Sharma 2008), where


,
)
),(
2
12

3
121
2










xbxaxxx
x
tf
t


,
0
),(
1










n
B
t
x
tG


(28)

and
),( tG
t

is the process noise coefficient matrix. The Fokker-Planck equation can be
stated as (Sage and Melsa1971, p.100)




t
ttp
t
),,(
0
0






i
i
ti
ttptf


),,(),(
0
0

,
),,(),()(
2
1
,
0,
2
0




ji
ji
t
j
i
T

ttpt


GG
(29)

where
.), ,,(
21
T
n
xxx

Equation (29) in combination with equation (28) leads to the
Fokker-Planck equation for the stochastic system of this chapter, i.e.

2
2
2
1
1
2
x
p
x
x
p
x
x
p

x
t
p












pbx
x
p
axp
2
1
2
3
1







.)12)(2(
2
22
1
2
2
2
12
pxnn
x
p
xbx
n
B








Alternatively, the stochastic differential system can be analysed qualitatively involving the
Itô differential rule, see equation (13) of the chapter. The energy function for the stochastic
system of this chapter is

.
422
1
),(

4
1
2
1
2
221
x
a
x
xxxE 

(30)


From equations (13), (28), and (30), we obtain
)()((),(
2
12
3
12122
3
1121
xbxaxxxxxaxxxxdE 

dtx
n
B
)
2
1

2
1
2



.
1 t
n
B
dwx



After a simple calculation, we have the following SDE:
.)
2
1
2
1
(
1
22
1
22
1
2
2
2
2 tut

n
Bu
n
B
dvdwxdtxxbxxdE



The qualitative analysis of the stochastic problem of this chapter using the multi-
dimensional Itô differential rule illustrates the contribution of diffusion parameters to the
stochastic evolution of the energy function. The energy evolution equation suggests the
system will exhibit either increasing oscillations or decreasing depending on the choice of
the parameters
,, b

and the diffusion parameters .,
uB


The numerical experiment
also confirms the qualitative analysis of this chapter, see figures (1)-(2). This chapter
discusses a Fokker-Planck analysis of the SDvdP system, recently analysed and published
by one of the Authors (Sharma 2008).
Making use of the Fokker-Planck equation, Kolmogorov backward equation, the evolutions
of condition mean and variances (Jazwinski 1970, p. 363) can be stated as

The Fokker-Planck equation 15
3. A stochastic Duffing-van der Pol system
The second-order fluctuation equation describes a dynamical system in noisy environment.
The second-order fluctuation equation can be regarded as


).,,,(
tttt
BxxtFx




The phase space formulation allows transforming a single equation of order
n into a
system of
n first-order differential equations. Choose
1
xx
t


21
xx


,
),,,,(
212 t
BxxtFx




by considering a special case of the above system of equations, we have


,
21
dtxdx


,),,(),,(
2122122 t
dBxxtgdtxxtfdx




in matrix-vector format


tt
dBxxtGdtxxtfd ),,(),,(
2121



, (27)

where

 


,,,

2121
TT
t
xx



,,),(
22
T
t
fxtf 



.,0),(
22
T
t
gtG 



The stochastic Duffing-van der Pol system can be formulated in the form of equation (27)
(Sharma 2008), where


,
)
),(

2
12
3
121
2










xbxaxxx
x
tf
t


,
0
),(
1










n
B
t
x
tG


(28)

and
),( tG
t

is the process noise coefficient matrix. The Fokker-Planck equation can be
stated as (Sage and Melsa1971, p.100)




t
ttp
t
),,(
0
0






i
i
ti
ttptf


),,(),(
0
0

,
),,(),()(
2
1
,
0,
2
0




ji
ji
t
j

i
T
ttpt


GG
(29)

where
.), ,,(
21
T
n
xxx

Equation (29) in combination with equation (28) leads to the
Fokker-Planck equation for the stochastic system of this chapter, i.e.

2
2
2
1
1
2
x
p
x
x
p
x

x
p
x
t
p












pbx
x
p
axp
2
1
2
3
1







.)12)(2(
2
22
1
2
2
2
12
pxnn
x
p
xbx
n
B








Alternatively, the stochastic differential system can be analysed qualitatively involving the
Itô differential rule, see equation (13) of the chapter. The energy function for the stochastic
system of this chapter is

.
422

1
),(
4
1
2
1
2
221
x
a
x
xxxE 

(30)


From equations (13), (28), and (30), we obtain
)()((),(
2
12
3
12122
3
1121
xbxaxxxxxaxxxxdE 

dtx
n
B
)

2
1
2
1
2



.
1 t
n
B
dwx



After a simple calculation, we have the following SDE:
.)
2
1
2
1
(
1
22
1
22
1
2
2

2
2 tut
n
Bu
n
B
dvdwxdtxxbxxdE



The qualitative analysis of the stochastic problem of this chapter using the multi-
dimensional Itô differential rule illustrates the contribution of diffusion parameters to the
stochastic evolution of the energy function. The energy evolution equation suggests the
system will exhibit either increasing oscillations or decreasing depending on the choice of
the parameters
,, b

and the diffusion parameters .,
uB


The numerical experiment
also confirms the qualitative analysis of this chapter, see figures (1)-(2). This chapter
discusses a Fokker-Planck analysis of the SDvdP system, recently analysed and published
by one of the Authors (Sharma 2008).
Making use of the Fokker-Planck equation, Kolmogorov backward equation, the evolutions
of condition mean and variances (Jazwinski 1970, p. 363) can be stated as