Geometrical-Based Lip-Reading using Template Probabilistic Multi-Dimension Dynamic Time Warping

Geometrical-Based Lip-Reading using Template Probabilistic Multi-Dimension Dynamic Time Warping

By identifying lip movements and characterizing their associations with speech sounds, the performance of speech recognition systems can be improved, particularly when operating in noisy environments. In this paper, we present a geometrical-based automatic lip reading system that extracts the lip re...

Full description

Bibliographic Details
Main Authors: M. Z., Ibrahim, Mulvaney, D. J.
Format: Article
Language:English
Published: Elsevier 2015
Subjects:
TK Electrical engineering. Electronics Nuclear engineering
Online Access:http://umpir.ump.edu.my/id/eprint/12795/
http://umpir.ump.edu.my/id/eprint/12795/
http://umpir.ump.edu.my/id/eprint/12795/
http://umpir.ump.edu.my/id/eprint/12795/1/Geometrical-Based%20Lip-Reading%20Using%20Template%20Probabilistic%20Multi-Dimension%20Dynamic%20Time%20Warping.pdf
id ump-12795
recordtype eprints
spelling ump-127952016-06-21T02:45:09Z http://umpir.ump.edu.my/id/eprint/12795/ Geometrical-Based Lip-Reading using Template Probabilistic Multi-Dimension Dynamic Time Warping M. Z., Ibrahim Mulvaney, D. J. TK Electrical engineering. Electronics Nuclear engineering By identifying lip movements and characterizing their associations with speech sounds, the performance of speech recognition systems can be improved, particularly when operating in noisy environments. In this paper, we present a geometrical-based automatic lip reading system that extracts the lip region from images using conventional techniques, but the contour itself is extracted using a novel application of a combination of border following and convex hull approaches. Classification is carried out using an enhanced dynamic time warping technique that has the ability to operate in multiple dimensions and a template probability technique that is able to compensate for differences in the way words are uttered in the training set. The performance of the new system has been assessed in recognition of the English digits 0 to 9 as available in the {CUAVE} database. The experimental results obtained from the new approach compared favorably with those of existing lip reading approaches, achieving a word recognition accuracy of up to 71 with the visual information being obtained from estimates of lip height, width and their ratio. Elsevier 2015-07 Article PeerReviewed application/pdf en http://umpir.ump.edu.my/id/eprint/12795/1/Geometrical-Based%20Lip-Reading%20Using%20Template%20Probabilistic%20Multi-Dimension%20Dynamic%20Time%20Warping.pdf M. Z., Ibrahim and Mulvaney, D. J. (2015) Geometrical-Based Lip-Reading using Template Probabilistic Multi-Dimension Dynamic Time Warping. Journal of Visual Communication and Image Representation , 30. 219 - 233. ISSN 1047-3203 http://dx.doi.org/10.1016/j.jvcir.2015.04.013 DOI: 10.1016/j.jvcir.2015.04.013
repository_type Digital Repository
institution_category Local University
institution Universiti Malaysia Pahang
building UMP Institutional Repository
collection Online Access
language English
topic TK Electrical engineering. Electronics Nuclear engineering
spellingShingle TK Electrical engineering. Electronics Nuclear engineering
M. Z., Ibrahim
Mulvaney, D. J.
Geometrical-Based Lip-Reading using Template Probabilistic Multi-Dimension Dynamic Time Warping
description By identifying lip movements and characterizing their associations with speech sounds, the performance of speech recognition systems can be improved, particularly when operating in noisy environments. In this paper, we present a geometrical-based automatic lip reading system that extracts the lip region from images using conventional techniques, but the contour itself is extracted using a novel application of a combination of border following and convex hull approaches. Classification is carried out using an enhanced dynamic time warping technique that has the ability to operate in multiple dimensions and a template probability technique that is able to compensate for differences in the way words are uttered in the training set. The performance of the new system has been assessed in recognition of the English digits 0 to 9 as available in the {CUAVE} database. The experimental results obtained from the new approach compared favorably with those of existing lip reading approaches, achieving a word recognition accuracy of up to 71 with the visual information being obtained from estimates of lip height, width and their ratio.
format Article
author M. Z., Ibrahim
Mulvaney, D. J.
author_facet M. Z., Ibrahim
Mulvaney, D. J.
author_sort M. Z., Ibrahim
title Geometrical-Based Lip-Reading using Template Probabilistic Multi-Dimension Dynamic Time Warping
title_short Geometrical-Based Lip-Reading using Template Probabilistic Multi-Dimension Dynamic Time Warping
title_full Geometrical-Based Lip-Reading using Template Probabilistic Multi-Dimension Dynamic Time Warping
title_fullStr Geometrical-Based Lip-Reading using Template Probabilistic Multi-Dimension Dynamic Time Warping
title_full_unstemmed Geometrical-Based Lip-Reading using Template Probabilistic Multi-Dimension Dynamic Time Warping
title_sort geometrical-based lip-reading using template probabilistic multi-dimension dynamic time warping
publisher Elsevier
publishDate 2015
url http://umpir.ump.edu.my/id/eprint/12795/
http://umpir.ump.edu.my/id/eprint/12795/
http://umpir.ump.edu.my/id/eprint/12795/
http://umpir.ump.edu.my/id/eprint/12795/1/Geometrical-Based%20Lip-Reading%20Using%20Template%20Probabilistic%20Multi-Dimension%20Dynamic%20Time%20Warping.pdf
first_indexed 2023-09-18T22:14:43Z
last_indexed 2023-09-18T22:14:43Z
_version_ 1777415260170354688

Similar Items