BO GIAO DUC VA DAO TAO
TR£fC‘ING DAY HOC KINH TG TP. HO CHI MINH

CflUYEN N6ANfl : DIEU KfllEN fl§C KINH TE
MA S0 :

5.02.20

LUANANTIENSIKINMTE

NGU#IHU0NGD

MNOAHOC:

-GS.TSTR %NGV

TP. HO CHI MINH - 2001

KHANG


LII CAM
DOAN

Tfii xin cam doan day la cfing trinh nghien cilu cua rieng toi. C:ie so lieu,
ket qua néu trong luan :in la trung thitc.

Nh’itng ket luan cua lua( n :in chita tu’’ng ditclc cfi ng bet trong bat ky cong
trinh nao kh:ic.

Ngitdi cam dean

NGUYEN VAN SI


UjCLTjC
MU DAU

i

CHCI€ING 1

4

XICH MARKOV VA CAC TRANG THAI

4

1.1. Tinh markov va xich Markov

4

1.1.1. C:ic dJnh nghia cci ban

4

1.1.2. Ma tr(an xac suat chuyén

5

1.1.3. Phan phoi hftu han chiéu.


8

1.1.4. Phan phoi ban dau

8

1.2. Phan ld’p c:ic trang th:ii

9

1.2.1 Cac trang thiii lien thfi ng va s phan lfi’p.
1.2.1.1 Cac trang thiii lien thong.
1.2.1.2. Chu ky cua trang th:ii
1.2.2. Trang th:ii hoi quy va trang th:ii khong hoi quy

9
9
10
10

1.2.2.1. C:i c dinh nghia.

i0

1.2.2.2. Tieu chuan hoi quy va khong hoi quy

13

1.3. Xii c suat gidi han


13

1.3.1. Dinh ly Ergodic

13

1.3.2 Thcli gian la) p trung binh

16

1.4 Xich Markov http thu

16

1.5.
h Markov vfii them gian lien tuc

Xic
20

1.5.1.
h nghia va cac khéi niem cd ban

Din
20

1.5.2.
i trlnh sinh va huy


Qu:
22

1.5.3.
‘i han cua xac suat

Gid
23

1.5.4.
:i trinh Poisson va qua trinh dem

Qu
25


1.5.5. Qu:i trinh Poisson co phan loai

29

1.5.6. Qu:i trinh doi mcii

29

1.5.6.1. DJnh nghia

29

1.5.6.2. Phfin phoi va trung binh cua N (t)


30

CHLI€ING 2

32

CAC MO HINH XICH MARK0V

32

2.1. Mo hinh kiém ke

32

2.2. Mo hinh phuc vu d:inn dong.

35

2.3. Xich Markov chay lien tiép

2.3.1. Dang ma tran xii c suat chuyen.
2.3.2. Ciic vi du
2.×.. Mo hinh pha ri hoach thJ phan

38
2. 5 Mo hinh trang th:ii hap thy

40

2.6 . II’ng dung cua qu:i trinh Poisson


41

2.7. Mo hinh tang trifcl ng tuyén tinh vfii s nhap cif .

42

2.8. I.I’ng dung cua qu:i trinh dot mdi

44

CLUNG 3

49

NG DUNG CUA XICH MARK0V

49

3.1. I.I’ng dung mo hinh phan hoach thJ phan cho 2 h5ng hang khong ci
Viet Nam
3.1.1 him ma tran xac suat chuyén va kiem tra tinh Markov

49
50

3.1.1.1 Thuat giai tim ma tran x:I c suat chuyén

50


3.1.1.2. Kiém tra tinh Markov

52

3.1.2. Litu do kiém tra tinh Markov va tim ma tr(an chuyen trong bai

toan phan hoach thJ phan cho hai h5ng hang khfing

54

3.1.3. Chitctng trlnh phan mém kiem tra tinh Markov va tim ma tran

chuyen trong bai tod n phan hoach th[ phan cua hat hang hang khong

55


3.1.4. Phfin hoach thi| phan cho hat h5ng hang khong vfii so lieu th¿c te 62
3.1.4.1. Kiem tra tinh Markov va ti’m ma tran x:ie suat chuyen

62

3.1.4.2. Phan hoach thi phan

63

3.2. Phan tich s bien dong cua gié vang tar TP Ho Chi Minh

65


3.2. 1.him ma tran xiic su$t chuyén va kiém tra tinh Markov

67

3.2. 1.1 Phifdng phép tim mak(an xiic su5t chuyén

67

3.2. 1.2. Kiém tra tinh Markov .

69

3.2.2 Litu do kiem tra tinh Markov va la( p ma tra( n chuyén de dii doiin
st
tang giam cua gi:i vang

72

3.2.3 Chifdng trinh ph:i n mém kiem tra tinh Markov va lap ma
tr3an chuyén de dy doan st tang giam cua gia vang
3.2.4. Ket n,ua vfii so lieu th c te

73
92

3.2.4.1. Ma tran x:ic suat chuyen va tinh Markov

82

3.2.4.2 Ap dung mo hinh xich Markov http thu


84

3.2.5. Xac suat gifii han

86

3.2.6. Kiem tra trang th:ii hoi qui

88

3.2.6.1. Thua(t giai

88

3.2.6.2. Litu do kiem tra trang thai hoi qui

90

3.2.6.3. Chu’cJng trinh ph$n mém de kiem tra trang thiii hoi qui

90

KET LUAN

95

1/ Nh’ifng két qua da hoan thanh

95


2/ Nhu”ng ddng gop mcii cila luan :in

96

3/ Nhifng van dé co the :ip dung tiép.

97

DANH MQC TAI L@U THAM KSO

99


1

Trong viec nghien ci?u nham nang cao hi(eu qua cua mo( t he› thong kinh
te viec t’if trang th:ii hien tai dp bao trang th:ii tiidng lai cua he thong la dieu

het si?c cap thiet. Bdi lC, nhcl do c:ic nha kinh te, c:ie nha doanh nghiep hoach
d)nh c:ie chien lu’dc kinh doanh, lpa chon quyet dJnh toi itu trong cii c rang
bupc, dira c:ic bien ph:i p t:ie dpng de loi kéo kh:ich hang .
Khi tien hanh c:ic nghien clit nay, chiing ta luon dung do vfii c:ic nhan‘ to
ngau nhien, chiing t:ic dpng mpi ncli, mpi fire. Nhat la trong giai doan kinh te thi
tru’fing hien nay, c:i c yéu to ngau nhién t:ie dpng cang du’ dpi cang da dang hctn
bao gif het.
Vi vay cii c mo hinh xac suat trci thanh c:ic cong cu quan trpng khong the
thieu trong cac nghien cit’u nay.
Mo hinh Xich Markov ditclc nha bac hpc Nga la A.A Markov (1856-1922)
difa ra tiI dau the ky 20 da ditclc nhieu nha toé n hpc phat trien them va sit dung

vao trong nhieu linh vie nghien ctiti khoa hpc : sinh hpc, vat ly hpc, x5 hpi hpc,

van hpc, kinh té hpc .
Trong nh’itng nam gan day, fi c:ic nu’fic kinh té phiit trien. Xich Markov
diidc sit dung nhieu trong kinh tfi : van de phan hoach th) phan, van de phuc vu
d:inn dong, van de dp trC hang hfi a, van de doi mdi cfc thiet bJ .
D niffic ta viec nghien ciiu quit trinh Markov ciing da ducic quan tain tit
lau : c:ie gi:to sit Nguyen B:ie Van, Nguyen Duy Tien, Da) ng Hiing Thang (
Dai hpc Tong hdp Ha Npi ), Nguyen Ho Quynh ( Dai hpc Birch khoa Ha Npi
), Nguyen Van Thu ( Vien To:in hpc Ha Noi ) . . v. .v. . d5 co nhieu cong trlnh
c6 gia tri. nghien ci’tu ve qu:i trinh Markov. Tuy nhien nhiing cong trinh dfi la
CaC


2

cong trinh thu:in tii y ly thuyét va mtii nhpn la ciic dJnh ly gifii han. Trong cong
trinh nay chiing tfii mong muon st dung cac két qua cua ly thuyet Xich Markov
vfii so lieu thpc té de tra left nhtfng cau hoi ct the trong kinh doanh. Hai ncli
chiing toi lifa chpn :ip dung la : hang khfi ng dan dung va kinh doanh vang ba c da

quy, nhting linh vie ma chiing tot xem la khé mui nhpn trong nén kinh th quoc
gia hien nay. Cu the la chil ng tfii giai quyfit 2 bai toan.
- Phan hoach thJ phan giffa hat hang hang khong tuyén bay TP Ho Chi
Minh — Ha Npi
- DQ doé n tang, giam gi:i vang tar TP Ho Chi Minh.
Theo nhan xét ciia Phfi Giiio sit Nguyen Bac Van — mpt trong c:ie chuyen
gia hang dau ve ly thuyét xii c suat u n.itic ta tih. viec cli9r. iTlG I.'•.nf. rich
Marr:ov de giai quyet ve van de dat ra d day la toi tin “ khong mpt phu’cIng
phiip nao kh:1c co the dat tfii difclc “.

Tat nhien qua hai van de giai quyét trong luan :in chung toi con muon xay
d ng mpt quy trlnh th c hien nhftng bai to:in tilfing th thupc 2 mo hinh nay vfii
phan mem vi tinh co the xem la cha) t ché.
Npi dung luan :in cua chiing toi "in 33 chilctng
- ChiI‹Jng 1 trinh bay nhftng yéu linh can ban can cho viec hiéu nhting mo hlnh
chiing toi trinh bay ci chu’fing 2. Cl chiming nay chilng tfii chi chting minh nhitng
kit qua quan trpng . Chi?ng minh nhu’’ng két qua kh:ic t dpc gia co the tim trong
tai li u [7] cua Gi:t o sit Nguyen Duy Tien — mpt tai lieu trinh bay kha day du
nhftng ket qua cap nh(at nhat ve qu:i trlnh Markov.

- Chifctng 2 : Trinh bay ciic mo hinh ring dung Xich Markov vao kinh té. Trong
chiicing nay, ngoai 2 mo hinh chung toi da sit dung cu the o chu’‹Jng 3, chiing tot
con trinh bay mpt “ mo hinh khac ma do khufin kho luan :in va do chita du so


lieu, nen chiing toi chifa :ip dung nhifng chiing toi nhan thay co the ring dung tot
trong kinh te niffic ta.
- Chitcing 3 : Trinh bay 2 :ip dung th c té d5 tién hanh co két qua.
Ciing theo lfii nhan xét cua Phfi Gi:to sit Nguyen B:ic Van thi “ day la

luan :in cap tien st dau tien d Viet Nam ve ung dung mo hinh Markov vao thiic

té “.


4

XICH MARKOV VA CAC TR@NG Tl½é‹l
1.1. Tirih markov va xiclx Markov
1.1.1. C:ie di.nh nghia cd ban

Gia sit chiing ta can nghien ct st tién trien theo thcii gian cua mpt he kinh
te, vat ly hoac sinh thii i nao dfi .
Ky hieu X(t) la vJ tri cua he fi thcli diem t. Tap help c:ie vi trf co the co cua
he ditpc gpi la khong gian trang théi.
Gia st tru’fic thcli diem s he n trang th:ii nao dfi , con tar thcii diem s he ct
bãang thai i. Tì cnn uié t tai thoi oiem I (t > S‘) trong titclng lai he cI trang thai j

vfii x:ie suat la bao nhiéu. Neu xii c suat nay chi phu thupc vao s, t, i, j thi dieu
nay co nghia la st tien trien cua h e( trong titling lai chi phu thu pc vao hien tai
va
d pc lap vfii qua khi?. Tinh chat nay gpi la tinh Markov.

co tinh chat nay gpi

He( la quit trinh Markov.
Chang han, neu gpi X(t) la dan so ci tai thcli diem t (trong titling lai) thi
co the xem X(t) chi phu thuo( c vao dan so hien tar va doe lap vfii qu:i khi’t .
Nfii chung c:i c he khfing co tri nhfi hoac sit’c y lh nhting he co tinh Markov.

Ta ky hieu E la tap gom cac giii tri. cua X(t) va E gpi la khong gian trang
th:ii cua X(t).
Nfi u X(t) co tinh Markov va E d:inh so du’cic thi X(t) dupc gpi la xich
Markov.

Tru’cing hcip, néu t = 0, 1, 2, ...thi ta co khai niem xich Markov vfii th’oi
gian
rcii rac. Con neu t c [0,s) thl ta c6 kh:ii niem xich Markov vfii thdi gian lien tuc.
Tinh Markov dope dinh nghia theo ngon ngft to:in hpc nhif sau



Dinh nghia

Ta not rang X(t) co tinh Markov néu

jcE.
Ta xem

ID hi(en tai vfi

, la tifclng lai, ( , t i... ., t,.,) la qua khu. Do do

biéu thiic tren chinh la tinh Markov cua X(t).
Da)t p (s, i, t, j) = P{X(I) = j / X(s) = i} , (s < t)
Do la x:ic suat co dieu kien de he tar thdi diem s d trang thai i, din thcli
diem t he chuyen sang trang théi j. Vi the ta gpi p(s, i, t, j) la xii c suat chuyen
cua he. Néu x:ie suat chuyen chi phu thuoc vao (t-s) to’c la
p(s, i, t, j) = p (s + h, i, I + h, j) thi ta nfii he la thuan nhat theo thfii gian.
Trong !i:ar. tire :i«y, I'm khong nfii gi them, chung ta chi xét xich Markov
thuan nhat.
1.1.2. Ma tran x:ic su3t chuj en
Gia sit (X,; n = 0, 1, 2,...) la xich Markov rcli rac va thuan nhat. Khi dfi co
m9t khong gian x:ic suat (Cl, A, P), X, : Cl --› E la cii c bien ngau nhien nh a(n
gia tri trong tap dem difpc E, E la khong gian trang th:ii, cac phfin tit cua no
du’cIc ky hieu la i, j, k,..
Tinh Markov va tinh thuan nhat cua (X,) co nghia la


6

P(ABC)

P(AC / B) = P(B)

P(BC)P(A / BC)
P(B)

_ P(B).P(C /B).P(A /B)
P(B)

= P(C/B).P(A/B)

ti?c la qua khil vii tifdng lai d9c la)p vfii nhau khi ta cho triffic hie›n tai.

Tif cong thit’c x:i c suat day du suy ra ma tran IP= (p;f ) co tinh chat

X:i c suét chuyén sau n bu’fic dircic dJnh nghia nhit sau

ij"' = ìããã= s / xã = i} =

(x• = J / x = i}

Day la xac suat de he› tar thfii diem xuat ph:it d trang th:ii i, sau n bride
chuyén sang trang thai j.
Rfi ring ta cb [9jj"' '

ij. Ta qui ifdc

p,

(0)


1 néu i = j


0 néu i ×j
Tif cfing thif’c x:i c suat day du va ttt tinh Markov ta co


Vcli mpi n = 0, 1, 2, ...
(1.1)
(1.2)
PhifDng trinh (1. 1) la phu’ctng trinh ngifclc; (1.2) la phifdng trinh thuan.
Mpt $ach tong quat hdn vfii mpi

n, in = 0, 1, 2, ... ta co phitdng trinh

Chii'ng minh (1.1)
He xuat ph:it tit trang th:ii i, sau n + 1 biific chuyen sang trang thai j la
két qua cua viec he xuat phii t tif trang th:ii i, sau m9t bitfic chuyen sang trang
thai k nao dfi, the roi h(e xuat phat tit trang thiii k, sau n bitfic tiép theo
chuyen sang trang th:ii j.

Tif cong thilc xac suat day du va tinh Markov ta co
p,""' = P{ Yen = j / X = i}
X n+i = ; / X = i, X; = k}. P{ X = k / X = i}
(n)

dieu nay d5 chu'ng minh cong thtic (1.1).
Cac cong thiic (1.2) va (1.3) difdc chting minh tifcing .
Ciic phiinng trinh (1.1), (1.2) va (1.3) co dang ma tran nhif
sau


TO db ta suy ra

JP(‘) = JP‘.


8

1.1.3. Phfi n phoi hCu han chiéu.
Phan phoi hifu han chieu cua qua trinh Markov difcic tinh theo cong thilc

Bieu thiic d ve tr:ii du’cic gpi la phan phoi h’ifu han chieu.
That vay

Do dinh nghia cua qu:i trinh Markov.

Dieu nay da chi?ng minh (1.4)

1.1.4. Phan phoi ban dau
Dinh nghia Phan phoi ban dau cua he tai thfii diem n difcic cho bfii cong
thiic sau

j"' 'P{X, = j}; n = 0, 1, 2, ...; j c E
(1.5)
(0)
a phan phoi ban dau cua he.
Da) t 11( = (pj(‘), j c E) va gpi

Ta qui itdc II(‘ =


c E)


9

Nhit vay, mo hinh cua mo( t xich Markov rfii rac va thuan nhat la bp ba
(X„ II, IP ) trong dfi

(X,)

la day c:ic biéu ngau nhien rcii rac

11

la phan phoi ban d$u

IP

la ma tran x:ic suat chuyén

1.2 Phfi n ldp c:ic trang thiii
1.2.1 Cii c trang th:ii lién thong va s phfin ldp.
1.2.1.1 C:ic trang th:ii lién thfi ng.
DJnh nghia
Ta nfii rang trang th:ii
hO Pij

(n)

j


dat dirpc tit trang thai

i néu ton tai n ± 0 sao

— 1 néu i = j va p, (0) — 0 néu i × j). Trong
ij'

> o eta quy iiClC

tru’dng help nhu’ the ta ky hieu i --+ j.
Hai trang th:ii i va j diiclc gpi la lien thong vdi nhau néu i ---› j va j --+ i.
Trong tru’ctng hdp dfi ta ky hieu la i +-› j.
De dang thay rang ‹-+ co c:i c tinh chat phan xa, doi xilng, bat cau. Tire la.
I} i +-› i (vi j

j''

'1)j

2) i +-› j thi j ‹-+ i ,

3 i ‹-+ j va j +-› k thi i +-› k.
Nhif v(ay ‹-+ la quan he tifdng difclng. Do dd, theo quan he nay, toan bp

khong gian trang thai difcic chia thanh c:ic ldp rem nhau, hai trang thai bat ky


thuoc ciing mpt léip tién thong vcii nhau, con hai trang thai bat ky thupc hai lélp
khac nhau khong the tién thong vdi nhau.

DJnh nghia
Xich Markov dif‹1c gpi la toi gian néu hai trang thai bat ky ci’ia no lien thong vcii
nhau. Nhif vay xich toi gian khi va chi khi E gom diing mot l‹ip; xich khong toi gién
co it nhat hai l‹ip khac rong, rdi nhau E = E; w Eg w ...

Trong nhieu triffing help co the xem moi E (k = 1, 2,..) la khong gian trang
th:ii cua xich Markov toi gian. ii the E„ E ,... dildc gpi la 1clp toi gian cua xich.
Nhir va( y viec xét xich Markov co the gut ve viec xét c:i c xich Markov tfii gian.

1.2.1.2. Chu ky cua trang thiii

D]nh nghia
Chu ky d(i) cua trang thiii i la itfic chung lfin nhat cua tat ca cac so nguyen
n

1 thoa m5n /°, ) > 0. Neu Pi'" ) - 0 dfii vfii tat ca n ? 1 thi da) t d(i) = 0.

D]nh ly

Néu i ‹-› j thi d(i) = d(j).

Dinh nghia Ta riot rang trang théi i khong co chu ky néu

1.2.2. Trang th:ii h‹ii quy va trang th:ii khong hfii quy
1.2.2.1. C:ie di.nh nghia.
Gia sit (X,) la xich Markov. Xét trang th:1i co dJnh i c E.

Ta da) t

‘) = P{X, = j, X; ×j, ..., X, .i ìJ ữ = i} , j e E.


d(i) = 1.


11

Chilng minh

co (rcli nhau) Ak. trong dfi Ak la bien co he xuat ph:it tit’ i lan dau tien rat vao j
tai thfii diem k roi lai xuat ph:it tit j sau dfi r‹Ji vao j tai thcli diem n — k. Ta

n
k —-\

nao dfi.

DJnh nghia
i du’dc gpi la trang th:ii hoi guy néu fit -- 1
i du’pc gpi la trang th:ii khfing hoi quy néu fit < 1.
Theo dinh nghia nay i la hoi quy neu va chi néu he xuat phd t tit i, vdi xac
suét 1 he lai tr0 ve i tai thdi diem h’ifu han nao dfi. Trong khi d6 i la khong hoi
quy co nghia la bien co he› xufit ph:It tif i, trd lai i ‘it nhat mpt lan co x:ie suat

bang :ti <

DO tinh Markov ta suy ra bien co he xuat phat ttr

i, trci lai i it



12

nhat 2 lan co xii c suat bang /'ii ' .. bien co he xuat ph:it tit i, trcl lai i it nhat k

Gia sir M la bien ngau nhien dem so I:in he trd lai i.
Ta thay rang, néu i khfing hoi quy thi
k
P{M ? k / X = i} = (f

(k = 1,2,...) ,

IE(M / X = i) =
tu”c la M co phan phcii hinh hpc
Dac biet, P{M = ‹×› / X = i} = 0. Vay, he xuat phiit tu’ i he trcl ve i vo so

lan vfii xac suat 0.
Trong tritfing hip i la hot quy thi
P{M ? k / X = i} = lim

Do do, ta co the tinh gi:i trJ trung binh cua no
n)
, dfi la thcii gian trung binh he trcl lai i.
/’il
I
n--0

Dinh nghia
Gia sit i la trang thiii hoi quy. Ta nfii
i la trang thai hot guy difcing. néu p; <
‹x›. i la trang th:ii khfing néu


p; =

‹×›.
Theo dJnh nghia tren i la trang thiii diifing co nghia la thfii gian chfi dcii de
h(e xuat ph:it tit i trci ve i la hflu han trong khi do thcii gian nay bang ‹x› neu i
la trang thiii khfing.


1.2.2.2. Tiéu chuan hfii quy va khfing hfii quy

DJnh ly
(i) Trang thai i la hoi guy khi va chi khi

hoac tilting ditclng, trang th:ii i la khfi ng hoi guy néu va chi néu

(1.6)
(ii) Néu i --› j hot quy thi j --› i va j cling hoi quy.
(iii) Neu i ‹-› j va j hoi quJ' thi fit —— 1.

1.3.X:ic suat gidi han
1.3.1. DJnh ly Ergodic
Ta xét nhffng dieu ki(n de xich Markov co tinh chat Ergodic theo nghia ton tar

c:ic gifii han
x = lim p,j"' khong php thuoc vao i

Y nghia cua dieu nay ID trong tit‹Jng lai xa xoi, xd c suat de he n trang th:ii
j khfing phu thuo( c vao hien tar.
DJnh ly Gia sit JP -( ij) la ma tran x:ie suat chuyén cua xich Markov (X,)


cd khong gian trang thiii hftu han E = {1, 2,..., N}.


14

(i) Néu IP chinh quy theo nghia sau ton tai no sao cho

thi ‘n tai ciic
so

×i ×p sao
cho

(1.8)

(1.9)

(ii) Ngitcic lgi, néu ton tai c:ie so ×i.-... eN ihoa m5n c:i c dieu kien
(1.8)
va (1.9) thi sé ton tar no thoa m5n (1.7).
(iii) C:ie so n„..-. nN la nghiem cua he phu’cIng trlnh
, '

iPkj

• jcE

(1. 10)


k

ii vay ta chi cfin chif’ng to
M$"'’ IIIj"'---9• 0 khi n --› ‹x› ; b'j = 1, 2, ..., N


Gia sit s — minp!‘0) > 0. Khi do
1‘ “

(n)
k

Tit do ta suy ra:

dieu d‹Jn giam, co day con hpi tu tfii 0,

(ii} Hien nhien tit (1.8) va (1.9) suy ra (1.7) vi so trang th:ii la hiiu han.
(iii) (1.10 lh he qua tr¿c tiép cua (1 .9 . Tha)t va) y, vi so trpng th:1i la
h’iiu han, nen


16

= lim

lim

1.3.2 Thdi gian lap trung binh
Ta xét quit trlnh xich Markov la Ergodic


DJnh nghia.
Thcii gian trung blnh trot qua giiia nhflng lan quit trinh cl trang th:ii j difclc gpi
la thcli gian lap trung binh cua trang thai j.

Dinh ly .
Thfii gian la( p trung binh cua trang thiii j bang

1

j = 1,..., N )

(1. 11)

1.4 Xich Markov hap thq
Gia siI (x,) la Xich Markov vfii khfing gian trang thai E = {0, 1,..., N
J Gia sit 0, 1, ..., r — 1 la cac trang thai truyen ting theo nghia

Pit

1, vfii r

Q
O

R
I

D day O la ma tra) n cap (N — r + 1) x r ma tat ca céc phan tit deu bang
0 I la ma tran dcln vi cap (N - r + 1) x (N — r + 1)
Q l1 ma tran cap r x r ma qij — Pij di 0 ± i, j fi r — 1


Tinh trpc tiép ta cd


17

0

I

Bang phu’cIng ph:ip gut nap, ta ditclc

O
Gpi

I

' la so lfin trung binh he cI trang th:1i j tinh den thfii diem n va

xuat ph:i t t’it trang th:ii i.

WO

He thiic nay diing cho mpi i, jcE. Nhitng no co y nghia nhat khi i,j la

ciic trang th:ii truyen it‘ng.
Liic dd, tit (1.12) ta co

p$ = q}


vfii 0 I i, j fi r -1


18

1 néu i = j
0 nfi u i a j

(1.14)
r —l

A=0

(ii—I)

(1.
15)

Chuyen (1.14) qua gicii han ta co (n --›‹×›)
W = I + QW

lim

(1.16)

zi)

(1. 17)
va ttr (1. 16), ta suy ra W — QW = (I — Q) W = I
nen

la

W = ( I — Q) ' la ma tran dao cua ma tra( n (I — Q). Ta gpi la ma tr(an W

ma tran cfi ban i?ng vdi ma tran Q. Thanh phan thii (i, j ) la phan tit W,j cua ma

1.18)
DaG ii 'E T
tii’ kang th:ii i.

= i] thcii gian trung binh cho den ltic hap thu khi bat dau


19

y =0n=0

n=0y =0

r —1

r -l

n=0

7’—1

(1.19)

¿=0


1=0

Bay gif ta xét din x:ic suat chain. Ta biét rang c:i c trang thiii r, r +1, ...,N

(1.21 )

—— P X-— k

vdi i = 0, 1, ..., r — 1 va k = r, r
+1,..., N trong dfi T la thcii diem chain.

T = min {n é 0:

r fi x, fi N}
= k I *0 '
vfii i = 0, 1, ..., r — 1 va k = r, r +1,..., N

(1.22)
Cho n ---• ‹x› trong (1.21) ta c6 xac suat
chain.