HOANG VAN NAM

MINISTRY OF EDUCATION AND TRAINING
HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY
---------------------------------------

Hoang Van Nam

COMPUTER SCIENCE

DIFFICULT SITUATIONS RECOGNITION SYSTEM FOR
VISUALLY-IMPAIRED AID USING A MOBILE KINECT

MASTER THESIS OF SCIENCE
COMPUTER SCIENCE

2014B
Ha Noi – 2016

123doc
Mang
Ln
thay vì
h■■ng
l■im■i
s■
cam
tr■
h■u
m■t
k■t

nghi■m
t■im■t
là
s■
cáwebsite
nhân
mang
kho
m■ith■
kinh
m■
l■i
d■n
vi■n
nh■ng
cho
doanh
■■u
kh■ng
ng■■i
quy■n
chia
t■ th■c
dùng,
l■
s■l■i
v■i
và
hi■n
t■t

cơng
h■n
mua
ngh■a
nh■t
2.000.000
ngh■
báncho
tài
v■
hi■n
ng■■i
li■u
c■a
tài
th■
hàng
mình
li■u
dùng.
hi■n
■■u
■
thìt■t
Khi
■■i,
s■p
Vi■t
c■
khách

b■n
t■i,
l■nh
Nam.
ngh■a
online
hàng
v■c:
Táctr■
khơng
v■
tài
phong
thành
chính
c■a
khác
chun
c■a
thành
tíngì
d■ng,
hàng
so
nghi■p,
viên
v■i
tri■u
cơng
c■a

b■n
hồn
nhà
ngh■
123doc
g■c.
bán
h■o,
thơng
B■n
và
hàng
■■
n■p
có
tin,
l■i
cao
th■
ti■n
ngo■i
chuy■n
tính
phóng
vào
ng■,...Khách
trách
tài
giao
to,kho■n

nhi■m
thu
sang
nh■
c■a
■■i
■■n
hàng
tùy123doc,
v■i
v■
ý.
cót■ng
qu■n
th■b■n
d■
ng■■i
lýChào
dàng
s■ dùng.
■■■c
m■ng
tra c■u
M■c
h■■ng
b■n
tàitiêu
li■u
■■n
nh■ng

hàng
m■t
v■i■■u
quy■n
cách
123doc.
c■a
chính
l■i123doc.net
sau
xác,n■p
nhanh
ti■n
tr■
chóng.
trên
thành
website
th■ vi■n tài li■u online l■n nh■t Vi■t Nam, cung c■p nh■ng tài li■u ■■c không th■ tìm th■y trên th■ tr■■ng ngo■i tr■ 123doc.net.
Nhi■u event thú v■, event ki■m ti■n thi■t th■c. 123doc luôn luôn t■o c■ h■i gia t■ng thu nh■p online cho t■t c■ các thành viên c■a website.

Mangh■n
Ln
Th■a
Xu■t
Sau
Nhi■u
123doc
Link
khi

h■■ng
phát
thu■n
l■i
event
cam
s■
nh■n
xác
m■t
tr■
t■
h■u
k■t
s■
thú
nghi■m
t■i
th■c
ýxác
n■m
t■■ng
m■t
d■ng
v■,
là
s■
nh■n
s■
website

ra
mang
event
kho
m■i
■■■c
■■i,
1.
t■o
tLink
t■
th■
m■
l■i
c■ng
ki■m
■■ng
d■n
123doc
CH■P
g■i
vi■n
xác
nh■ng
cho
■■u
■■ng
ti■n
v■
th■c

h■
kh■ng
ng■■i
NH■N
■ã
■■a
quy■n
th■ng
thi■t
chia
t■ng
s■
ki■m
dùng,
l■
ch■
CÁC
s■
■■■c
th■c.
s■
l■i
b■■c
v■i
ti■n
và
email
chuy■n
■I■U
t■t

cơng
h■n
mua
123doc
g■i
online
kh■ng
nh■t
b■n
2.000.000
v■
ngh■
bán
KHO■N
sang
b■ng
ln
cho
■■a
■ã
tài
■■nh
hi■n
■■ng
ng■■i
li■u
ph■n
ln
ch■
tài

TH■A
tài
v■
th■
li■u
hàng
t■o
email
li■u
thơng
ky,
dùng.
tríhi■n
THU■N
hi■u
c■
c■a
b■n
■■u
■b■n
tin
t■t
h■i
Khi
■■i,
qu■
mình
vui
Vi■t
xác

c■
■ã
khách
gia
lịng
b■n
nh■t,
minh
trong
l■nh
■■ng
Nam.
t■ng
Chào
■■ng
online
hàng
uy
tài
v■c:
l■nh
thu
Tác
m■ng
ky,
tín
kho■n
tr■
nh■p
nh■p

khơng
b■n
tài
phong
v■c
cao
thành
b■n
chính
vui
email
nh■t.
tài
email
online
oLink
khác
chun
■■n
li■u
lịng
thành
tínb■n
Mong
c■a
xác
gì
cho
d■ng,
và

■■ng
v■i
so
nghi■p,
viên
th■c
kinh
■ã
mình
t■t
123doc.
123doc.net!
v■i
mu■n
cơng
■■ng
nh■p
c■a
c■
doanh
s■
b■n
vàcác
hồn
mang
■■■c
ngh■
123doc
click
email

ký
g■c.
online.
thành
v■i
h■o,
Chúng
vào
l■i
thơng
B■n
g■i
c■a
và
123doc.netLink
CH■P
cho
viên
linkí
Tính
■■
v■
n■p
có
mình
tơi
tin,
c■ng
c■a
cao

■■a
th■
■■n
cung
NH■N
ti■n
ngo■i
và
tính
mình
website.
phóng
■■ng
ch■
th■i
click
vào
c■p
CÁC
ng■,...Khách
trách
xác
trong
email
tài
■i■m
D■ch
vào
xã
to,kho■n

■I■U
th■c
nhi■m
h■i
thu
linkơng
l■nh
b■n
tháng
V■
nh■
m■t
s■
KHO■N
c■a
■ã
v■c
(nh■
■■i
hàng
■■■c
tin
tùy
ngu■n
5/2014;
■■ng
123doc,
tài
v■i
xác

■■■c
ý.
có
li■u
TH■A
g■i
t■ng
minh
th■
tài
ky,
123doc
và
v■
mơ
ngun
b■n
b■n
d■
ng■■i
THU■N
tài
kinh
■■a
t■
dàng
kho■n
s■
vui
v■■t

d■■i
doanh
tri
dùng.
■■■c
ch■
lịng
tra
th■c
m■c
email
■ây)
email
c■u
■■ng
Chào
online.
M■c
h■■ng
q
100.000
cho
tài
b■n
b■n
m■ng
tiêu
báu,
nh■p
li■u

Tính
b■n,
■ã
nh■ng
■ã
hàng
phong
m■t
l■■t
■■n
email
■■ng
b■n
tùy
■■ng
■■u
quy■n
cách
truy
thu■c
■■n
th■i
phú,
c■a
ký
ky,
c■a
c■p
chính
v■i

■i■m
v■i
■a
mình
l■i
b■n
vào
123doc.net
m■i
123doc.netLink
d■ng,
123doc.net!
sau
xác,
các
vui
tháng
vàngày,
n■p
click
lịng
“■i■u
nhanh
giàu
5/2014;
ti■n
s■
vào
■■ng
tr■

giá
Kho■n
Chúng
chóng.
h■u
trên
linkc■a
thành
tr■
xác
123doc
nh■p
2.000.000
website
■■ng
th■c
Th■a
tơi
th■
website.
cung
email
v■■t
s■
vi■n
th■i
Thu■n
■■■c
c■p
c■a

thành
mong
m■c
tài D■ch
v■
li■u
mình
g■i
viên
100.000
mu■n
S■
online
v■
và
V■
■■ng
D■ng
click
■■a
t■o
(nh■
l■■t
l■n
ký,
D■ch
■i■u
vào
ch■
nh■t

■■■c
truy
l■t
link
email
ki■n
V■”
vào
c■p
Vi■t
123doc
mơtop
sau
cho
b■n
m■i
Nam,
t■200
■ây
d■■i
cho
ngày,
■ã
cung
các
các
(sau
■■ng
g■i
■ây)

s■
website
c■p
users
■ây
h■u
ky,
cho
nh■ng
■■■c
có
b■n
2.000.000
b■n,
ph■
thêm
vui
tài
bi■n
tùy
g■i
lịng
thu
li■u
thu■c
t■t
thành
nh■t
nh■p.
■■c

■■ng
T■i
vào
t■i
viên
khơng
t■ng
Chính
nh■p
Vi■t
các
■■ng
th■i
“■i■u
th■
Nam,
vì
email
v■y
■i■m,
ký,
tìm
t■
Kho■n
c■a
l■t
123doc.net
th■y
l■chúng
vào

mình
tìm
trên
Th■a
top
ki■m
và
tơi
th■
200
ra
click
Thu■n
cóthu■c
■■i
tr■■ng
các
th■
vào
nh■m
website
c■p
v■
top
link
ngo■i
S■
3nh■t
■áp
123doc

Google.
D■ng
ph■
tr■
■KTTSDDV
■ng
123doc.net.
bi■n
■ã
D■ch
Nh■n
nhu
g■i
nh■t
c■u
V■”
■■■c
theo
t■i
chia
sau
Vi■t
quy■t
danh
■ây
s■ Nam,
tài
(sau
hi■u
...li■u

t■
■ây
do
ch■t
l■c■ng
■■■c
tìm
l■■ng
ki■m
■■ng
g■i
và
thu■c
t■t
bình
ki■m
T■i
ch■n
top
ti■n
t■ng
3 Google.
là
online.
th■i
website
■i■m,
Nh■n
ki■m
chúng

■■■c
ti■ntơi
online
danh
có th■
hi■u
hi■u
c■p
do
qu■
nh■t
c■ng
và ■KTTSDDV
uy
■■ng
tín nh■t.
bình ch■n
theo quy■t
là website
... ki■m ti■n online hi■u qu■ và uy tín nh■t.

Lnh■n
123doc
Sau
Th■a
Xu■t
khi
h■■ng
phát
thu■n

cam
nh■n
m■t
t■k■t
s■
t■i
ýxác
n■m
t■■ng
d■ng
là
s■
nh■n
website
ra
mang
■■i,
1.
t■o
t■l■i
c■ng
■■ng
d■n
123doc
CH■P
nh■ng
■■u
■■ng
h■
NH■N

■ã
quy■n
th■ng
chia
t■ng
ki■m
CÁC
s■s■
l■i
b■■c
ti■n
vàchuy■n
■I■U
t■t
mua
online
kh■ng
nh■t
bán
KHO■N
sang
b■ng
cho
■■nh
thay
ng■■i
ph■n
tài
TH■A
vìv■

li■u
m■i
thơng
dùng.
tríTHU■N
hi■u
m■t
c■atin
Khi
qu■
mình
cá
xác
khách
nhân
nh■t,
minh
trong
Chào
kinh
hàng
uy
tài
l■nh
m■ng
doanh
tín
kho■n
tr■
v■c

cao
thành
b■n
t■
email
nh■t.
tàith■c
■■n
li■u
thành
b■n
Mong
hi■n
và
v■i
viên
kinh
■ã
123doc.
123doc.net!
mu■n
ngh■a
■■ng
c■a
doanh
mang
123doc
v■
kýonline.
c■a

v■i
Chúng
l■ivà
123doc.netLink
mình
cho
Tính
n■p
tơi
c■ng
thì
■■n
cung
ti■n
s■p
■■ng
th■i
vào
c■p
t■i,
xác
tài
■i■m
D■ch
xã
ngh■a
kho■n
th■c
h■itháng
V■

m■t
s■
v■
c■a
(nh■
■■■c
c■a
ngu■n
5/2014;
123doc,
■■■c
c■a
g■i
tài
123doc
hàng
v■
mơ
ngun
b■n■■a
t■
tri■u
s■
v■■t
d■■i
tri
■■■c
ch■
nhà
th■c

m■c
■ây)
email
bán
h■■ng
q
100.000
cho
hàng
b■n
báu,
b■n,
nh■ng
l■i
■ã
phong
l■■t
chuy■n
tùy
■■ng
quy■n
truy
thu■c
phú,
ky,
c■p
giao
■a
l■i
b■n

vào
m■i
sang
d■ng,
sau
các
vuingày,
n■p
■■n
lịng
“■i■u
giàu
ti■n
s■
■■ng
v■
giá
Kho■n
h■u
qu■n
trên
tr■
nh■p
2.000.000
website
■■ng
Th■a
lý hồn
email
th■i

Thu■n
h■o,
c■a
thành
mong
v■
■■
mình
viên
mu■n
S■
cao
và
■■ng
D■ng
tính
click
t■otrách
ký,
D■ch
■i■u
vàol■t
link
nhi■m
ki■n
V■”
vào
123doc
top
sau

cho
■■i
200
■ây
cho
v■i
■ãcác
các
(sau
g■i
t■ng
website
users
■ây
ng■■i
■■■c
cóph■
dùng.
thêm
bi■n
g■i
thu
M■c
t■t
nh■t
nh■p.
T■i
tiêu
t■i
t■ng

hàng
Chính
Vi■tth■i
■■u
Nam,
vì v■y
■i■m,
c■a
t■123doc.net
l■
123doc.net
chúng
tìm ki■m
tơiracó
tr■
thu■c
■■i
th■
thành
nh■m
c■p
topth■
3nh■t
■áp
Google.
vi■n
■KTTSDDV
■ng
tàiNh■n
nhu

li■uc■u
online
■■■c
theo
chia
l■n
quy■t
danh
s■nh■t
tài
hi■u
...li■u
Vi■t
do
ch■t
Nam,
c■ng
l■■ng
cung
■■ng
và
c■p
bình
ki■m
nh■ng
ch■n
ti■ntài
là
online.
website

li■u ■■cki■m
khơng
ti■n
th■
online
tìm th■y
hi■utrên
qu■th■
và tr■■ng
uy tín nh■t.
ngo■i tr■ 123doc.net.
Ln
Th■a
Xu■t
Sau
Nhi■u
123doc
Mang
thayh■n
khi
vì
h■■ng
phát
thu■n
l■i
event
m■i
cam
s■
nh■n

m■t
tr■
t■
h■u
m■t
k■t
s■
thú
nghi■m
t■i
ýxác
n■m
t■■ng
m■t
d■ng
v■,
là
s■
cá
nh■n
website
ra
nhân
mang
event
kho
m■i
■■i,
1.
t■o

t■
th■
kinh
m■
l■i
c■ng
ki■m
■■ng
d■n
123doc
CH■P
vi■n
nh■ng
cho
doanh
■■u
■■ng
ti■n
h■
kh■ng
ng■■i
NH■N
■ã
quy■n
th■ng
thi■t
chia
t■t■ng
ki■m
th■c

dùng,
l■
CÁC
s■
th■c.
s■
l■i
b■■c
v■i
ti■n
và
hi■n
chuy■n
■I■U
t■t
cơng
h■n
mua
123doc
online
kh■ng
ngh■a
nh■t
2.000.000
ngh■
bán
KHO■N
sang
b■ng
ln

cho
tài
■■nh
v■
hi■n
ng■■i
li■u
ph■n
ln
c■a
tài
TH■A
tài
v■
th■
li■u
hàng
t■o
mình
li■u
thơng
dùng.
tríhi■n
THU■N
hi■u
c■
c■a
■■u
■
thìtin

t■t
h■i
Khi
■■i,
qu■
s■p
mình
Vi■t
xác
c■
khách
gia
b■n
t■i,
nh■t,
minh
trong
l■nh
Nam.
t■ng
Chào
ngh■a
online
hàng
uy
tài
v■c:
l■nh
thu
Tác

m■ng
tín
kho■n
tr■
nh■p
khơng
v■
tài
phong
v■c
cao
thành
b■n
chính
c■a
email
nh■t.
tài
online
khác
chun
■■n
c■a
li■u
thành
tínb■n
Mong
gì
cho
d■ng,

và
hàng
v■i
so
nghi■p,
viên
kinh
■ã
t■t
123doc.
123doc.net!
v■i
mu■n
tri■u
cơng
■■ng
c■a
c■
doanh
b■n
các
hồn
nhà
mang
ngh■
123doc
ký
g■c.
online.
thành

bán
v■i
h■o,
Chúng
l■i
thơng
B■n
và
hàng
123doc.netLink
cho
viên
Tính
■■
n■p
có
tơi
tin,
c■ng
l■i
c■a
cao
th■
■■n
cung
ti■n
ngo■i
chuy■n
tính
website.

phóng
■■ng
th■i
vào
c■p
ng■,...Khách
trách
xác
tài
■i■m
D■ch
giao
xã
to,kho■n
th■c
nhi■m
h■i
thu
sang
tháng
V■
nh■
m■t
s■
c■a
(nh■
■■i
■■n
hàng
■■■c

tùy
ngu■n
5/2014;
123doc,
v■i
v■
■■■c
ý.
cóg■i
t■ng
qu■n
th■
tài
123doc
v■
mơ
ngun
b■n
d■
ng■■i
lý,
■■a
t■
dàng
s■
cơng
v■■t
d■■i
tri
dùng.

■■■c
ch■
tra
th■c
ngh■
m■c
■ây)
email
c■u
M■c
h■■ng
q
hi■n
100.000
cho
tài
b■n
tiêu
báu,
li■u
b■n,
th■
nh■ng
■ã
hàng
phong
m■t
l■■t
hi■n
tùy

■■ng
■■u
quy■n
cách
truy
thu■c
■■i,
phú,
ky,
c■a
c■p
chính
■a
b■n
l■i
b■n
vào
123doc.net
m■i
d■ng,
sau
online
xác,
các
vuingày,
n■p
lịng
“■i■u
nhanh
giàu

khơng
ti■n
s■
■■ng
tr■
giá
Kho■n
chóng.
h■u
trên
khác
thành
tr■
nh■p
2.000.000
website
■■ng
Th■a
gìth■
so
email
vi■n
th■i
v■i
Thu■n
c■a
thành
b■n
mong
tài v■

li■u
mình
g■c.
viên
mu■n
S■
online
và
B■n
■■ng
D■ng
click
t■o
l■n
cóký,
D■ch
■i■u
vào
th■
nh■t
l■t
link
phóng
ki■n
V■”
vào
Vi■t
123doc
top
sau

cho
to,
Nam,
200
thu
■ây
cho
■ã
cung
nh■
các
các
(sau
g■iwebsite
tùy
c■p
users
■ây
ý.nh■ng
■■■c
cóph■
thêm
tài
bi■n
g■i
thu
li■u
t■t
nh■t
nh■p.

■■c
T■it■i
khơng
t■ng
Chính
Vi■tth■i
th■
Nam,
vì v■y
■i■m,
tìm
t■123doc.net
th■y
l■chúng
tìm
trên
ki■m
tơi
th■
racóthu■c
■■i
tr■■ng
th■nh■m
c■p
top
ngo■i
3nh■t
■áp
Google.
tr■

■KTTSDDV
■ng
123doc.net.
Nh■n
nhu c■u
■■■c
theo
chiaquy■t
danh
s■ tài
hi■u
...li■udo
ch■t
c■ng
l■■ng
■■ng
vàbình
ki■mch■n
ti■n là
online.
website ki■m ti■n online hi■u qu■ và uy tín nh■t.
Chia
m■t
u■t
Nhi■u
Mang
Ln
123doc
Th■a
Xu■t

Sau
tri■n
phát
khi
h■n
member
s■
h■■ng
phát
khai
thu■n
l■i
event
s■
cam
nh■n
câu
t■
m■t
tr■
t■
event
h■u
ýk■t
s■
chuy■n
thú
nghi■m
t■i
ýkhơng

t■■ng
xác
n■m
t■■ng
m■t
d■ng
v■,
là
khuy■n
s■
nh■n
website
ra
mang
m■y
event
t■o
kho
thành
m■i
■■i,
1.
t■o
t■
mãi
c■ng
th■
n■i
m■
l■i

c■ng
ki■m
■■ng
d■n
cơng
123doc
CH■P
th■
vi■n
b■t
nh■ng
cho
■■ng
■■u
■■ng
ti■n
trên
n■p
h■
c■a
kh■ng
ng■■i
NH■N
■ã
quy■n
th■ng
123doc
thi■t
chia
ki■m

v■i
c■ng
t■ng
ki■m
dùng,
l■
CÁC
s■
nh■ng
th■c.
ti■n
s■
l■i
b■■c
■■ng
v■i
ti■n
-và
ki■m
chuy■n
■I■U
t■t
cơng
online
h■n
mua
123doc
online
■u
kh■ng

123doc
nh■t
5■ãi
2.000.000
ngh■
bán
KHO■N
tri■u
b■ng
sang
b■ng
ln
cho
c■c
tài
■■nh
■ã
hi■n
ch■
tài
ng■■i
li■u
ph■n
ln
k■
tài
TH■A
xu■t
li■u
tài

v■
v■i
th■
li■u
h■p
hàng
t■o
li■u
thơng
s■c
dùng.
trí
hi■u
7hi■n
THU■N
hi■u
d■n.
tài
c■
c■a
■■u
■■■ng
li■u!
tin
qu■
t■t
h■i
Khi
■■i,
qu■

mình
■■ng
Vi■t
xác
c■
khách
gia
nh■t,
Nghe
trong
b■n
nh■t,
minh
trong
l■nh
Nam.
t■ng
Chào
b■online
có
uy
hàng
danh
l■
uy
tài
v■c:
l■nh
thu
Tác

v■
tín
m■ng
nhé,
tín
kho■n
tr■
sách
cao
nh■p
khó
khơng
tài
phong
v■c
cao
tr■■c
thành
b■n
chính
nh■t.
tin
Top
email
nh■t.
tài
online
khác
nh■ng
chun

■■n
li■u
tiên
thành
danh
tín
Mong
b■n
Mong
gì
cho
d■ng,
và
hãy
v■i
■ây
so
thu
nghi■p,
viên
kinh
■ã
mu■n
t■t
123doc.
123doc.net!
cùng
v■i
mu■n
cao

là
cơng
■■ng
c■a
c■
doanh
b■n
con
nh■t
mang
tìm
các
hồn
mang
ngh■
123doc
s■
ký
g■c.
hi■u
online.
thành
tháng
v■i
l■i
hồn
h■o,
Chúng
l■i
thơng

B■n
thơng
cho
và
123doc.netLink
cho
viên
t■o
tồn
Tính
■■
n■p
có
c■ng
tơi
tin,
c■ng
tin
c■
c■a
cao
th■
chính
■■n
cung
ti■n
ngo■i
v■
h■i
■■ng

tính
website.
phóng
■■ng
Khách
th■i
vào
c■p
xác
gia
ng■,...Khách
trách
xác
xã
tài
t■ng
■i■m
mà
D■ch
xã
to,
hàng
h■i
kho■n
th■c
nhi■m
h■i
BQT
thu
thu

m■t
tháng
V■
có
nh■
m■t
s■
nh■p
123doc
c■a
th■
(nh■
■■i
hàng
ngu■n
■■■c
tùy
ngu■n
5/2014;
123doc,
d■
v■i
online
■■■c
ý.
có
■ã
dàng
tài
g■i

t■ng
th■
tài
thu
123doc
ngun
cho
v■
mơ
ngun
b■n
tra
d■
ng■■i
th■p
t■t
■■a
t■
c■u
dàng
s■
v■■t
tri
d■■i
c■
■■■c
tri
dùng.
■■■c
ch■

tài
th■c
các
tra
th■c
m■c
li■u
■ây)
email
c■u
sau
thành
q
M■c
h■■ng
q
m■t
100.000
cho
■■t
tài
báu,
b■n
tiêu
báu,
viên
li■u
cách
b■n,
t■ng

nh■ng
phong
■ã
hàng
phong
c■a
m■t
l■■t
chính
tùy
■■ng
k■t
■■u
website.
phú,
quy■n
cách
truy
thu■c
phú,
doanh
xác,
ky,
c■a
c■p
■a
chính
■a
nhanh
l■i

b■n
vào
d■ng,
thu
123doc.net
m■i
d■ng,
sau
xác,
các
vui
tháng
chóng.
ngày,
n■p
giàu
lịng
“■i■u
nhanh
giàu
11
ti■n
giá
s■
■■ng
tr■
giá
uy
Kho■n
chóng.

h■u
tr■
trên
tín
thành
tr■
nh■p
■■ng
cao
2.000.000
website
■■ng
Th■a
th■
nh■t.
email
th■i
vi■n
th■i
Thu■n
Mong
mong
c■a
thành
mong
tài v■
li■u
mình
mu■n
mu■n

viên
mu■n
S■
online
và
■■ng
D■ng
mang
t■o
click
t■o
l■n
■i■u
ký,
D■ch
■i■u
vào
l■i
nh■t
l■t
cho
link
ki■n
ki■n
V■”
vào
Vi■t
c■ng
123doc
cho

top
sau
cho
Nam,
■■ng
cho
200
■ây
cho
■ã
cung
các
các
các
(sau
g■i
xãusers
website
h■i
c■p
users
■ây
m■t
nh■ng
có
■■■c
cóph■
thêm
ngu■n
thêm

tài
bi■n
g■i
thu
thu
li■u
tài
t■t
nh■p.
nh■t
nh■p.
ngun
■■c
T■it■i
Chính
khơng
t■ng
Chính
Vi■t
tri th■c
th■i
vìth■
Nam,
vìv■y
v■y
q
■i■m,
tìm
123doc.net
t■123doc.net

báu,
th■y
l■chúng
tìm
phong
trên
ki■m
tơi
ra
th■
ra
phú,
có
■■i
thu■c
■■i
tr■■ng
th■
■Sau
nh■m
nh■m
c■p
top
ngo■i
h■n
■áp
3nh■t
■áp
Google.
m■t

■ng
tr■
■KTTSDDV
■ng
123doc.net.
n■m
nhu
Nh■n
nhuc■u
rac■u
■■i,
■■■c
chia
theo
chia
123doc
s■
quy■t
danh
s■tàitài
hi■u
li■u
■ã
...li■u
t■ng
ch■t
do
ch■t
c■ng
b■■c

l■■ng
l■■ng
■■ng
kh■ng
vàvàki■m
bình
ki■m
■■nh
ch■n
ti■n
ti■n
v■
online.
là
online.
tríwebsite
c■a mình
ki■m
trong
ti■nl■nh
online
v■c
hi■u
tài li■u
qu■và
vàkinh
uy tín
doanh
nh■t.online


Nhi■u
Mang
Ln
123doc
Th■a
Xu■t
Sau
khi
h■n
h■■ng
phát
thu■n
l■i
event
s■
cam
nh■n
m■t
tr■
t■
h■u
k■t
s■
thú
nghi■m
t■i
ýxác
n■m
t■■ng
m■t

d■ng
v■,
là
s■
nh■n
website
ra
mang
event
kho
m■i
■■i,
1.
t■o
t■
th■
m■
l■i
c■ng
ki■m
■■ng
d■n
123doc
CH■P
vi■n
nh■ng
cho
■■u
■■ng
ti■n

h■
kh■ng
ng■■i
NH■N
■ã
quy■n
th■ng
thi■t
chia
t■ng
ki■m
dùng,
l■
CÁC
s■
th■c.
s■
l■i
b■■c
v■i
ti■n
vàchuy■n
■I■U
t■t
cơng
h■n
mua
123doc
online
kh■ng

nh■t
2.000.000
ngh■
bán
KHO■N
sang
b■ng
ln
cho
tài
■■nh
hi■n
ng■■i
li■u
ph■n
ln
tài
TH■A
tài
v■
th■
li■u
hàng
t■o
li■u
thơng
dùng.
tríhi■n
THU■N
hi■u

c■
c■a
■■u
■ tin
t■t
h■i
Khi
■■i,
qu■
mình
Vi■t
xác
c■
khách
gia
b■n
nh■t,
minh
trong
l■nh
Nam.
t■ng
Chào
online
hàng
uy
tài
v■c:
l■nh
thu

Tác
m■ng
tín
kho■n
tr■
nh■p
khơng
tài
phong
v■c
cao
thành
b■n
chính
email
nh■t.
tài
online
khác
chun
■■n
li■u
thành
tínb■n
Mong
gì
cho
d■ng,
và
v■i

so
nghi■p,
viên
kinh
■ã
t■t
123doc.
123doc.net!
v■i
mu■n
cơng
■■ng
c■a
c■
doanh
b■n
các
hồn
mang
ngh■
123doc
ký
g■c.
online.
thành
v■i
h■o,
Chúng
l■i
thơng

B■n
và
123doc.netLink
cho
viên
Tính
■■
n■p
có
tơi
tin,
c■ng
c■a
cao
th■
■■n
cung
ti■n
ngo■i
tính
website.
phóng
■■ng
th■i
c■p
thay
ng■,...Khách
trách
xác
■i■m

D■ch
xã
to,
vì th■c
nhi■m
m■i
h■i
thutháng
V■
nh■
m■t
s■(nh■
■■i
hàng
■■■c
tùy
ngu■n
5/2014;
cáv■i
nhân
■■■c
ý.
cóg■i
t■ng
th■
tài
123doc
kinh
v■
mơ

ngun
d■
ng■■i
doanh
■■a
t■
dàng
v■■t
d■■i
tri
dùng.
ch■
t■
tra
th■c
m■c
■ây)
th■c
email
c■u
M■c
q
100.000
cho
tài
hi■n
b■n
tiêu
báu,
li■u

b■n,
ngh■a
■ã
hàng
phong
m■t
l■■t
tùy
■■ng
■■u
cách
truy
v■
thu■c
phú,
ky,
c■a
c■a
c■p
chính
■a
b■n
vào
mình
123doc.net
m■i
d■ng,
xác,
các
vuingày,

thì
lịng
“■i■u
nhanh
giàu
s■p
s■
■■ng
tr■
giá
t■i,
Kho■n
chóng.
h■u
thành
tr■
ngh■a
nh■p
2.000.000
■■ng
Th■a
th■
email
v■vi■n
th■i
Thu■n
c■a
c■a
thành
mong

tài
c■a
v■
li■u
mình
viên
hàng
mu■n
S■
online
và
■■ng
D■ng
tri■u
click
t■o
l■n
ký,
D■ch
■i■u
vào
nhà
nh■t
l■t
link
bán
ki■n
V■”
vào
Vi■t

123doc
hàng
top
sau
cho
Nam,
200
l■i
■ây
cho
■ã
chuy■n
cung
các
các
(sau
g■iwebsite
c■p
users
■ây
giao
nh■ng
■■■c
cósang
ph■
thêm
tài
bi■n
g■i
■■n

thu
li■u
t■t
nh■t
v■
nh■p.
■■c
T■i
qu■n
t■i
khơng
t■ng
Chính
Vi■t
lý th■i
quy■n
th■
Nam,
vì v■y
■i■m,
tìm
l■i
t■123doc.net
th■y
l■
sau
chúng
tìm
trên
n■p

ki■m
tơi
th■
ti■n
racóthu■c
■■i
tr■■ng
trên
th■nh■m
c■p
website
top
ngo■i
3nh■t
■áp
Google.
tr■
■KTTSDDV
■ng
123doc.net.
Nh■n
nhu c■u
■■■c
theo
chiaquy■t
danh
s■ tài
hi■u
...li■udo
ch■t

c■ng
l■■ng
■■ng
vàbình
ki■mch■n
ti■n là
online.
website ki■m ti■n online hi■u qu■ và uy tín nh■t.

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


MINISTRY OF EDUCATION AND TRAINING
HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

--------------------------------------Hoang Van Nam

DIFFICULT SITUATIONS RECOGNITION SYSTEM FOR
VISUALLY-IMPAIRED AID USING A MOBILE KINECT

Department :

COMPUTER SCIENCE

MASTER THESIS OF SCIENCE
…......................................

SUPERVISOR :
1. Dr. Le Thi Lan

Ha Noi – 2016

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


CỘNG HÒA XÃ HỘI CHỦ NGHĨA VIỆT NAM
Độc lập – Tự do – Hạnh phúc

BẢN XÁC NHẬN CHỈNH SỬA LUẬN VĂN THẠC SĨ
Họ và tên tác giả luận văn : …………………………………........……………..
Đề tài luận văn: ………………………………………….....……………...............….
Chuyên ngành:……………………………...…………………........................…..........
Mã số SV:………………………………….. …………………....................................…...
Tác giả, Người hướng dẫn khoa học và Hội đồng chấm luận văn xác
nhận tác giả đã sửa chữa, bổ sung luận văn theo biên bản họp Hội đồng
ngày….........................………… với các nội dung sau:
……………………………………………………………………………………………………..…………
…………………………………………………………………………………………..…………………………
…………………………………………………………………………..…………………………………………
…………………………………………………………..…………………………………………………………
…………………………………………..…………………………………………………………………………
…………………………..……………………………………………………………………………………..

Ngày
Giáo viên hướng dẫn

tháng

năm


Tác giả luận văn

CHỦ TỊCH HỘI ĐỒNG

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Declaration of Authorship
I, Hoang Van Nam, declare that this thesis titled, ’Difficult situations recognition for
visual-impaired aid using mobile Kinect’ and the work presented in it are my own. I
confirm that:




This work was done wholly or mainly while in candidature for a research degree
at this University.



Where any part of this thesis has previously been submitted for a degree or any
other qualification at this University or any other institution, this has been clearly
stated.



Where I have consulted the published work of others, this is always clearly attributed.



Where I have quoted from the work of others, the source is always given. With
the exception of such quotations, this thesis is entirely my own work.



I have acknowledged all main sources of help.



Where the thesis is based on work done by myself jointly with others, I have made
clear exactly what was done by others and what I have contributed myself.

Signed:


Date:

i

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

Abstract
International Research Institute MICA
Computer Vision Department
Master of Science
Difficult situations recognition for visual-impaired aid using mobile Kinect

by Hoang Van Nam

By 2014, according to figures from some organization, here are more than one million
people in the Vietnam living with sight loss, about 1.3% of Vietnamese people. Although
the big impact to the daily living, especially with the ability to move, read, communicate
with another, only a small percentage of blind or visually impaired people live with
assistive device or animal such as a dog guide. Motivated by the significant changes in
technology have take place in the last decade, especially in the introduction of varies
types of sensors as well as the development in the field of computer vision, I present
in this thesis a difficult situations recognition system for visually impaired aid using a
mobile Kinect. This system is based on data captured from Kinect and using computer
vision technique to detect obstacle. At the current prototype, I only focused on detecting
obstacle in the indoor environment like public building and two types of obstacle will
be exploited: general obstacle in the moving way and staircases-which causes a big
dangerous to the visually impaired people. The 3D imaging techniques were used to
detect the general obstacle including: plane segmentation, 3D point clustering and the
mixed strategy between depth and color image is used to detect the staircase based on
detecting the stair edges and its structure. The system is very reliable with the detection
rate is about 82.9% and the time to process each frame is 493 ms.

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Acknowledgements
I am so honor to be here the second time, in one of the finest university in Vietnam to
write those grateful words to people who have been supporting, guiding me from the
very first moment when I was a university student until now, when I am writing my
master thesis.
I am grateful to my supervisor, Dr. Le Thi Lan, whose expertise, understanding, generous guidance and support made it possible for me to work on a topic that was of great
interest to me. It was a pleasure to work with her.
Special thanks to Dr. Tran Thi Thanh Hai, Dr. Vu Hai and Dr. Nguyen Thi Thuy
(VNUA) and all of the members in the Computer Vision Department, MICA Institute
for their sharp comments, guidance for my works which helps me a lot in how to study
and to do research in right way and also the valuable advices and encouragements that
they gave to me during my thesis.
I would like to express my gratitude to Prof. Veelaert Peter, Dr. Luong Quang Hiep
and Mr. Michiel Vlaminck at Ghent University, Belgium for their supporting. It’s been
a great honor to cooperate and work with them.
Finally, I would especially like to thank my family and friends for their continues love,
support they have given me through my life, helps me pass through all the frustrating,
struggling, confusing. Thanks for everything that helped me get to this day.

Hanoi, 19/02/2016
Hoang Van Nam


iii

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Contents
Declaration of Authorship

i

Abstract

ii


Acknowledgements

iii

Contents

iv

List of Figures

vi

List of Tables

ix

Abbreviations

x

1 Introduction
1.1 Motivation . . . . . . . . . . . . . . . . . . .
1.2 Definition . . . . . . . . . . . . . . . . . . . .
1.2.1 Assistive systems for visually impaired
1.2.2 Difficult situations . . . . . . . . . . .
1.2.3 Mobile Kinect . . . . . . . . . . . . . .
1.2.4 Environment Context . . . . . . . . .
1.3 Difficult Situations Recognition System . . .
1.4 Thesis Contributions . . . . . . . . . . . . . .


. . . .
. . . .
people
. . . .
. . . .
. . . .
. . . .
. . . .

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.

.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.

.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.


.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

1
1

2
2
3
5
11
12
13

2 Related Works
14
2.1 Assistive systems for visually impaired people . . . . . . . . . . . . . . . . 14
2.2 RGB-D based assistive systems for visually impaired people . . . . . . . . 18
2.3 Stair Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
3 Obstacle Detection
3.1 Overview . . . . . . . . . . . . .
3.2 Data Acquisition . . . . . . . . .
3.3 Point Cloud Registration . . . .
3.4 Plane Segmentation . . . . . . .
3.5 Ground & Wall Plane Detection

.
.
.
.
.

.
.
.
.

.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.

.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.

.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.

.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.
.

.
.
.
.

.

.
.
.
.
.

25
25
26
27
30
32

iv

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Contents
3.6
3.7

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.

.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.


.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.

.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.

.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.

32
34
34
35
45
46
48

4 Experiments
4.1 Dataset . . . . . . . . . . . . . . . . . .

4.2 Difficult situation recognition evaluation
4.2.1 Obstacle detection evaluation . .
4.2.2 Stair detection evaluation . . . .

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.

.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.

.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.
.
.
.

.

.
.
.

.
.
.
.

49
49
51
51
53

3.8

Obstacle Detection . . . . . . . . . .
Stair Detection . . . . . . . . . . . .
3.7.1 Stair definition . . . . . . . .
3.7.2 Color-based stair detection .
3.7.3 Depth-based stair detection .
3.7.4 Result fusion . . . . . . . . .
Obstacle information representation

v
.
.
.
.

.
.
.

5 Conclusions and Future Works
58
5.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
5.2 Future Works . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
6 Publications

60

Bibliography

61

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


List of Figures
1.1
1.2
1.3
1.4
1.5

1.6
1.7
1.8
1.9
1.10
1.11
1.12
1.13
1.14

1.15
1.16
2.1
2.2
2.3
2.4
2.5
2.6


A Comprehensive Assistive Technology (CAT) Model provided by [12] .
A model for activities attribute and mobility provided by [12] . . . . . .
Distribution of frequencies of head-level accidents for blind people [18] .
Distribution of frequencies of tripping resulting a fall [18] . . . . . . . .
A typical example of depth image (A) raw depth image, (B) depth image
is visualized by jet color map and the colorbar shows the real distance
with each color value, (C) Reconstructed 3D scene . . . . . . . . . . . .
A stereo images that taken from OpenCV library and the calculated depth
image (A) left image, (B) right image, (C) depth image (disparity map)
Some existed stereo camera. From left to right: Kodak stereo camera,
View-Master Personal stereo camera, ZED, Duo 3D Sensor . . . . . . .
Time of flight systems from [3] . . . . . . . . . . . . . . . . . . . . . . .
Some ToF cameras. From left to right: DepthSense, Fotonic, Microsoft
Kinect v2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Structured light cameras. From left to right: PrimeSense, Microsoft
Kinect v1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Structured light systems from [3] . . . . . . . . . . . . . . . . . . . . . .
Figure from [16], (A) raw IR image with pattern, (B) depth image . . .
Figure from [16] (A) Errors for structured light cameras, (B) Quantization
errors in different distances of a door: 1m, 3m, 5m . . . . . . . . . . . .
Prototype of system using mobile Kinect, (A) Kinect with battery and
belt, (B) Backpack with laptop (C)Mobile Kinect is mounted on human
body . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Two different environments that I tested with (A) Our office build (B)
Nguyen Dinh Chieu secondary school . . . . . . . . . . . . . . . . . . . .
Prototype of our obstacle detection and warning system . . . . . . . . .
Robot-Assisted Navigation from [17] (A) RFID tag, (B) Robot (C) Navigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
NXT Robot System from [6] (A) The system’s Block Diagram, (B) NXT
Robot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Mobile robot from [22] [21] . . . . . . . . . . . . . . . . . . . . . . . . .
BrainPort vision substitution device [32] . . . . . . . . . . . . . . . . . .
Obstacle detection process from [30] . . . . . . . . . . . . . . . . . . . .
Stair detection from [26] (A) Input image (B)(C)Frequency as a output
of Gabor filter (D)Stair detection result . . . . . . . . . . . . . . . . . .

.
.
.
.

3
4
4
5

.

6

.

7

.
.

7
8


.

8

.
.
.

8
9
9

. 10

. 11
. 12
. 13

. 15
.
.
.
.

16
16
18
20

. 21


vi

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


List of Figures
A near-approach for stair detection in [13] (A) Input image with detected
stair region, (B) Texture energy, (C)Input image with detected lines are
stair candidates, (D)Optical flow maps in this image, there is a significant
changing in the line in the edge of stair . . . . . . . . . . . . . . . . . .
2.8 Example of segmentation and classification in [24] . . . . . . . . . . . .
2.9 Stair modeling(left) and features in each plane [24] . . . . . . . . . . . .
2.10 Stair detection algorithm proposed in [29] (A) Detected line in the edge

image (using color infomation) (B) Depth profiles in each line (red line:
pedestrian crosswalk, blue: down stair, green: upstair) . . . . . . . . . .

vii

2.7

3.1
3.2
3.3
3.4
3.5
3.6

3.7
3.8
3.9
3.10
3.11
3.12
3.13

3.14
3.15
3.16
3.17
3.18
3.19
3.20
3.21

3.22
3.23
3.24
3.25
3.26
3.27

Obstacle Detection Flowchart . . . . . . . . . . . . . . . . . . . . . . .
Kinect mounted on body . . . . . . . . . . . . . . . . . . . . . . . . . . .
Coordinate Transformation Process . . . . . . . . . . . . . . . . . . . . .
Kinect Coordinate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Point Cloud rotation using normal vector of ground plane (while arrow):
left: before rotating, right: after rotating . . . . . . . . . . . . . . . . . .
Normal vector estimation algorithms [15] (a) Normal vector of the center
point can be calculated by a cross product of two vectors of four neighbor
points (red), (b) Normal vector estimation in a scene . . . . . . . . . .
Plane segmentation result using algorithm proposed in [15]. Each plane
is represented by a distinctive color. . . . . . . . . . . . . . . . . . . . .
Detected Ground and Walls plane (ground: blue, wall: red) . . . . . . .
Human Segmentation Data by Microsoft Kinect SDK (a) Color Image,
(b) Human Mask . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Detected Obstacles (a) Color Image, (b) Detected Obstacles . . . . . .
Model of stair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Coordinate transformation models from [7] . . . . . . . . . . . . . . . .
Projective chirping: a) A real world object that generate a projection
with ”chirping” - ”periodicity-in-perspective” b) Center raster of image
c) Best fit projective chirp . . . . . . . . . . . . . . . . . . . . . . . . . .
A pin-hole camera model with stair . . . . . . . . . . . . . . . . . . . . .
A vertical Gabor filter kernel. . . . . . . . . . . . . . . . . . . . . . . . .
Gabor filter applied on a color image (a) Original (b) Filtered Image. .

Thresholding the grayscale image (a) Original (b) Thresholded Image.
Example of thinning image using morphological. . . . . . . . . . . . . .
Thresholding the grayscale image (a) Original (b) Thresholded Image.
Six points vote for a line will make an intersection in Hough space, this
intersection has higher intensity than neighbor pixels. . . . . . . . . . .
Hough space (a) Line in the original space (b) Three curves vote for this
line in Hough space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Hough space on stair image (a) Original image (b) Hough space. . . . .
Chirp pattern detection (a) Hough space (b) Original image with detected
chirp pattern. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Point cloud of stair (a) Original color image (b)Point cloud data created
from color and depth image. . . . . . . . . . . . . . . . . . . . . . . . .
Detected steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Detected planes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Detected stair on point cloud. . . . . . . . . . . . . . . . . . . . . . . . .

. 22
. 23
. 23

. 24
.
.
.
.

26
27
28
29


. 30

. 31
. 31
. 33
.
.
.
.

34
34
35
36

.
.
.
.
.
.
.

38
38
39
40
40
41

42

. 42
. 43
. 43
. 44
.
.
.
.

45
46
47
47

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


List of Figures

viii

3.28 Obstacle position quantization for sending warning message to visually
impaired people. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
4.1
4.2
4.3

4.4

4.5
4.6

Depth image encoding (A) Original, (B) Visualized Image (C) Encoded
Image. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Detection time of each step in our proposed method. . . . . . . . . . . . .
Example stair image to evaluation (A)Positive sample from MICA dataset
(B) Negative sample from MICA dataset (C) Positive sample from MONASH
dataset (D) Negative sample from MONASH dataset. . . . . . . . . . . .
Detected stair in Tian’s based method (A-F) and detected stair in my
proposal method (G-I) (A) Color image (B) Depth image (C) Edges (D)
Line segments (E) Detected concurrent lines (F) Depth values on detected
lines (G) Detected stair with blue lines are false stair edge and green

lines are stair edge (H) Edges Image, (I) Detected peaks in Hough map
corresponding to lines in Figure G. . . . . . . . . . . . . . . . . . . . . . .
Miss detection in Tian’s based method because of missed depth on stair(AF) and detected stair in my proposed method (G-I). . . . . . . . . . . . .
Miss detection in Tian’s based method because of missed depth on stair(AF) and detected stair in my proposed method (G-I). . . . . . . . . . . . .

50
52

54

55
56
57

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep


List of Tables
2.1

Comparison between assistive robot and wearable device . . . . . . . . . . 14

4.1
4.2
4.3
4.4
4.5
4.6

Database specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . .
Pixel level evaluation result (TP,FP,FN: million pixels). . . . . . . . . .
Object level evaluation result (TP,FP,FN: objects). . . . . . . . . . . . .
Stair dataset for evaluation . . . . . . . . . . . . . . . . . . . . . . . . .
Stair detection result of the proposed method on different datasets . . .
Comparison of the proposed method and the method of Tian et al. [29]
on MICA dataset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.
.
.
.
.

50

52
52
53
53

. 55

ix

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Abbreviations
PCL


Point Cloud Library

CAT

Comprehensive Assistive Technology

TDU

Tongue Display Unit

IR

Infrared

OpenCV

Open Computer Vision

RGB

Red Green Blue

RGB-D

Red Green Blue and Depth

ToF

Time of Flight


x

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Chapter 1

Introduction
1.1

Motivation

According to the official statistic of National Eye Hospital in 2002, Vietnam has about

900.000 blind people, including about 400.000 who are totally blind. By 2014, according
to figures from some organizations, the number of blind people in Vietnam is about 1,2
to 1,4 million people, it’s still a large number in comparison with other countries. In
the worldwide, the visually impaired population is estimated to number in excess of 285
million by the investigation of World Health Organization (August 2014)1 . About 90% of
them live in developing countries with low-income settings. Visually impaired has made
a big impact in their daily living. Especially they can not read the document, the ability
to move and to communicate with other people is compromised because the information
is received primarily through vision. All of the above things have led blindness become
the public heath problem in all over the world.
Nowadays, with the significant developing in the technology, lots of assistive devices
has been released in order to help visually impaired people in daily life. But although
many researchers and companies are concerned with making better and cheap device
to improve the comfort of visually impaired people, the research in this field still remains many unsolved issues and in general, those devices still cannot replace traditional
methods such as the white cane or guided dog.
Take the motivation on the significant changes in technology have take place in the last
decade, especially in the introduction of varies types of sensors as well as the development
in the field of computer vision, my thesis aims to build a prototype of system to help
visually impaired people avoid the obstacle in the environment using Kinect sensor.
1

/>
1

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Chapter 1. Introduction

2

With the Kinect, the benefit is that we can make a reliable system by using depth and
color information to detect the obstacle with an affordable price. In my thesis, due to
the lack of time, I only focus on indoor environment, more specifically in the public
building such as apartment or office in order to detect some general objects encountered
on the moving way and stair which may cause danger to the visually impaired people.
My thesis is organized as follows:
First, I shall give some definitions in the context of my work and the contributions in
this thesis.
In chapter 2, I shall review briefly some other works that related to my system such
as existing assistive devices, obstacle detection algorithms/systems, its advantages and
disadvantages.
In chapter 3, a framework for obstacle detection will be developed and I shall present
the details of each module and also the entire system, analyzing and assessing them.

In the next chapter, I shall give some experiments results of my system, including how
to prepare the dataset, how to make an evaluation and the final results.
In the final chapter, I end this work by giving some conclusions and future works to
make the system more complete and effective.

1.2

Definition

1.2.1

Assistive systems for visually impaired people

According to [12], assistive systems for visually impaired people can be understood as
an equipment, devices or systems which can be used to overcome the gap between a
disabled person wants to do and what the social allows them to do. In short, such
kind of system must be able to help the visually impaired people to do the things that
normal people can do. And this system can be model by the Comprehensive Assistive
Technology (CAT) Model as shown in Fig 1.1. The top level of this model consist of
four components that can be used to define all assistive technology systems:
• Context (in which the assistive technology will be used).
• Person (what kind of user can use this system)
• Activities (what activities that assistive system can help the visually impaired
people, can be seen more clearly in Fig 1.2)

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Chapter 1. Introduction

3

• Assistive Technology (technology will be used to make a system)
Most of the existing systems are aimed at solving one specific aspect of each branch in
the model: work on bounded defined context, with some certain types of users to help
them in specific activities in daily life. In the framework of my master thesis, to simplify
the system, I just focused on some certain aspects of this model and I will explain in
detail in the next sections. In short, I applied my system with the local settings of
context, in a small public building such as office, department and the users are the
visually impaired students at the Nguyen Dinh Chieu Secondary school to help them
avoid obstacles in a moving way.
Local settings
Culture &
Social context


National context
Context

Characteristics
Comprehensive
Assistive
Technology
model

Person
Activities
Assistive Technology

Attitudes
Social aspects
Activity specification
Design issues
System technology issues
End user issues

Figure 1.1: A Comprehensive Assistive Technology (CAT) Model provided by [12]

1.2.2

Difficult situations

Fig 1.2 shows detail information of activities branch in the model CAT (see Fig 1.1).
As shown in the figure, there are a lot of services that can be used in assistive systems
for visually impaired people such as mobility, daily living, cognitive activities, education

and employment, recreational activities, communication and access to information. But
most of exist works focus on the mobility component in the activities model because of
its important role for visually impaired people daily life.

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Chapter 1. Introduction

4

Reaching and lifting

Sitting and standing

Short distance locomotion
inside & outside

Mobility

Long & medium distance
locomotion
Movement on ramps,
slopes, stairs & hills

Our Focus

Daily living
Obstacle avoidance

Activities
Cognitive activities

Navigation and orientation

Education and Employment

Sitting and standing

Recreational activities

Communication & Access to information

Access to environment


Figure 1.2: A model for activities attribute and mobility provided by [12]

According to the survey of R.Manduchi [18] in 2011 with 300 respondents who are
legally blind or blind, there were half of the respondents said that they had an headlevel accident at least once in a week and about 30% respondents fell down at least once
a month (see Fig 1.3 and Fig 1.4). Therefore, helping visually impaired people in the
moving process is always an interested topic for researchers, social organizations and
companies. In fact, many products have been released, and also have some particular
success like the system proposed in [11], [10], [1] and [4].

Figure 1.3: Distribution of frequencies of head-level accidents for blind people [18]

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep



Chapter 1. Introduction

5

Figure 1.4: Distribution of frequencies of tripping resulting a fall [18]

In the context of my thesis, I aim to develop a system which can detect the obstacles
in visually impaired people’s moving way which are the main cause of the accidents
mentioned above. The scenario in this project is that visually impaired people want to
move on the hallway inside a public building, so they need to avoid obstacles including
moving or static objects and to go up/down the stair. Obstacle in my case can be defined
as objects laying on the ground or in front of the visually impaired people that he/she
can be harmed while moving if encountered these objects. Although obstacle’s class is
very important with the visually impaired people to distinguish which is more dangerous
and which is not but in my work, I just try to detect obstacle in the scene without saying
its name (make a classification). And within the framework of this thesis, I also focus on
detection another special object that often appears in the building and is very dangerous
for the visually impaired people, that is the stair. Moreover, the proposed system will
only give a warning to the blind people using Tongue Display Unit (TDU) which has
been already developed by Thanh-Huong Nguyen in 2013 [23]. In brief, my proposed
system aims to solve two aspects of mobility component of the activities model (see
Fig. 1.2): obstacle avoidance and movement on ramps, slopes, stair & hill and with
the second aspect, the current system just stop at the level of given warning distance of
stairs to the visually impaired people in order to assist them in going up/down stairs.

1.2.3

Mobile Kinect

1. Introduction To assist visually impaired persons in those difficult situations, in

my thesis, I proposed using a Kinect sensor to capture the information of the environment in order to detect obstacles if they appear. There are a lot of advantages
when using Kinect in this system since it is a popular RGB-D camera with cheap
price. But firstly, I will give some brief information about the depth camera where
Kinect is the typical example.

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Chapter 1. Introduction

6

Depth camera is actually a sensor which has the capacity to provide depth information (depth image or depth map). A depth map is an image that contains
information relating to the distance of the surface of scene objects from a viewpoint, for example in the Fig. 1.5. An intensity value of each pixel in a depth

map represents a distance from a point in the object to the camera. Therefore,
3D information of the scene can be reconstructed by using depth image (as shown
in Fig. 1.5-C). The benefit of the depth image that is not affected by lighting
conditions.

(a)

(b)

(c)

Figure 1.5: A typical example of depth image (A) raw depth image, (B) depth image
is visualized by jet color map and the colorbar shows the real distance with each color
value, (C) Reconstructed 3D scene

In recent years, with the development of technology, especially in the field of
sensor fabrication industry, there are a lot of cameras have been placed on the
market which is capable of capturing the depth information. Those devices can be
separated into several groups by used technology such as stereo camera: ZED, for
example, Time-of-Flight (ToF) like ZCam, the structured light camera like Kinect,
long range 3D camera. Each device has it own advantages, disadvantages and only
suitable for a particular use case.
2. Stereo Camera
Stereo camera is the kind of camera was used in the robotics since its early days.
Take the ideas of human binocular vision, it contains two or more cameras with
precisely known relative offsets. Depth information can be calculated by matching
similar point in the overlapped region between images. Hence, 3D distance to
matching points can be determined using triangulation like illustrated in Fig 1.6.
However, the camera is used in this case is still the color camera. As a result, it is
still affected by the changing of lighting conditions. On the other hand, the depth

image is calculated by matching algorithms, so it works very poorly when the
scene is texture-less, for example image of wall, building,. There are many stereo
cameras that are available on the market due to the ease of making such as Kodak

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Chapter 1. Introduction

7

stereo camera, View-Master Personal stereo camera, ZED2 , Duo 3D Sensor3 , as
illustrated in Fig 1.7.


(a)

(b)

(c)

Figure 1.6: A stereo images that taken from OpenCV library and the calculated
depth image (A) left image, (B) right image, (C) depth image (disparity map)

Figure 1.7: Some existed stereo camera. From left to right: Kodak stereo camera,
View-Master Personal stereo camera, ZED, Duo 3D Sensor

3. Time of Flight (ToF) camera
Time of Flight (ToF) cameras use the same principle as laser radar, which is
instead of transmitting a single beam, short pulses of Infrared(IR) light is sent.
The camera will get the return time from pixels across its field of view. And the
distance was measured by comparing the phase of the modulated return pulses
with those emitted by the laser (Fig 1.8). But ToF cameras is also suffered from
similar limitations as a ime of flight sensors, including ambiguity of measurements,
multiple reflection, sensitivity to material reflectance, background lighting and do
not operate well outdoors in strong sunlight. Some of the popular ToF cameras
can be listed such as: DepthSense4 , Fotonic5 , Microsoft Kinect v2 (see Fig 1.9)
4. Structured light camera
Structured light camera is another approach to measure depth information by
using “structured light”, which is a pattern of light such as array of lines. The
scene will be viewed at an angle like illustrating in the Fig 1.11. If the pattern is
projected onto a flat wall, the camera will see straight lines but if the scene is very
complex then it will see a more complex profile. By analyzing this profiles across
2


/>
4
/>5
/>3

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Chapter 1. Introduction

8

Figure 1.8: Time of flight systems from [3]


Figure 1.9: Some ToF cameras. From left to right: DepthSense, Fotonic, Microsoft
Kinect v2

the field to map, depth information can be calculated. With traditional method,
structured lights is grids or arrays of lines but it’s affected by a noise. So that, in
some newer devices such as PrimeSense or Microsoft Kinect v1 (see Fig 1.10), a
codes will be added in to a light to make the camera is almost zero repetition across
the scene. The Kinect v1 uses a randomly distributed speckled pattern and each
speckle looks differently flat at different distance, due to a special lens as can be
seen in the Fig 1.12. But this kind of depth sensor also have some limitations such
as the errors grow with the square of the distance to objects, strong quantization
effects (see Fig) and some limitations that similar with ToF system like sensitivity
to material reflectance, do not operate well in strong sunlight.

Figure 1.10: Structured light cameras. From left to right: PrimeSense, Microsoft
Kinect v1

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Chapter 1. Introduction

9

Figure 1.11: Structured light systems from [3]

(a)

(b)

Figure 1.12: Figure from [16], (A) raw IR image with pattern, (B) depth image

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Chapter 1. Introduction

10

(a)

(b)

Figure 1.13: Figure from [16] (A) Errors for structured light cameras, (B) Quantization errors in different distances of a door: 1m, 3m, 5m

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Chapter 1. Introduction

11

5. Mobile Kinect
In my thesis, I used Microsoft Kinect v1 as a capture device of the system due to
its usability and availability. To make Kinect become more flexibility, I have added
some components and making it become “Mobile Kinect”, which is Kinect with a
external battery so it can be moved to anywhere without worrying about electrical
sources (holes, cables) and it is easy to replace this external battery. To attach on
the human body, Kinect has been mounted into a single belt, so that the Kinect
can be fix mounted easily on the body. An other important part of mobile Kinect
is a laptop which plays the role of main processor. It contains a data acquisition
and obstacle detection modules. The reason of choosing laptop because Kinect is
the commercial device and it have been developed for the video game purposes,
so Kinect cannot operate without a PC. And because of the restrictions in Kinect
data cable length, a computer must be placed nearby the Kinect (the whole system
can be seen in Fig 1.14).

(a)

(b)

(c)


Figure 1.14: Prototype of system using mobile Kinect, (A) Kinect with battery and
belt, (B) Backpack with laptop (C)Mobile Kinect is mounted on human body

Officially, Kinect runs with 12V sources provided by an adapter comes with it by
default. In the experiment, it can operate when the voltage drops down to 8.5V
with the running current is about 0.3-0.4A. So I designed battery with 8-packs
AAA batteries of which can provides 8x1.5A=12A. And the time to drop down
from 12V-8.5V is about 1.5-2 hours in our experiment, which mean the mobile
Kinect can runs 1.5-2 hours with the battery.

1.2.4

Environment Context

The environment for development and testing system is public building as mentioned
before. More specifically, I just focus on a specific use case, that is walking along the
corridors of building. There are two major type of corridor in our context, that is
two sides wall and half opening with one side is the glass windows or opened and the
other side is the solid wall. But in our experiment, I aim to develop a system with the

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep


Chapter 1. Introduction

12

half opening corridor because it’s very popular type in public building such as school,
office and some apartment. In the context of my thesis, I tested the system with two
different environments. One is our office building at B1-Hanoi University of Science and
Technology, the others is Nguyen Dinh Chieu secondary school for blind pupils (see Fig
1.15). Because depth data is strongly affected by sun light, so the requirement with
environment is that it must not be lighted too strong (in the shady day or in the two
sides wall building, where the sun light cannot reach to).
The use-case was used is that the user (visually impaired people) want to move to another
place in the building and to do that, he/she must go along the corridor, walk up/down
the stair then move to destination point. Ignoring the problem with path finding, in
my work, I only aim to obstacle avoidance. Obstacles in both cases are the objects that
blocked the moving way such as: distinguisher, trash bin, column, wall and human in
front of the user (can be seen in Fig 1.15).

(a)


(b)

Figure 1.15: Two different environments that I tested with (A) Our office build (B)
Nguyen Dinh Chieu secondary school

1.3

Difficult Situations Recognition System

In conclusion, my entire system to recognize the difficult situations can be demonstrated
as in Fig 1.16. In this case, the prototype system will be fix mounted on the visually
impaired people. To interact with the user, a tactile visual substitution module from
[23] has been used to give the warning about obstacle in front of he/she. The mobile
Kinect will be mounted on the human hip to capture depth and color information. Those
information will be processed by a recognition module in the laptop behind user. After
obstacle has been detected, laptop will send a correspondence command to tactile visual
substitution module in order to give warning message. The message representation has
been integrated into this module and presented in [23]. So my main work is how to send
a correct command to the feedback module.

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz

luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep do an to nghiep docx 123docz
luan van hay luan van tot nghiep