A lip geometry approach for feature-fusion based audio-visual speech recognition

M. Z., Ibrahim and Mulvaney, D. J. (2014) A lip geometry approach for feature-fusion based audio-visual speech recognition. In: 6th International Symposium on Communications, Control and Signal Processing, ISCCSP 2014 , 21 - 23 May 2014 , Athens, Greece. pp. 644-647. (6877957). ISBN 9781479928903

Preview	Pdf A lip geometry approach for feature-fusion based audio.pdf Download (541kB) | Preview
	Pdf A lip geometry approach for feature-fusion based audio_FULL.pdf Restricted to Repository staff only Download (254kB) | Request a copy

Abstract

This paper describes a feature-fusion audio-visual speech recognition (AVSR) system that extracts lip geometry from the mouth region using a combination of skin color filter, border following and convex hull, and classification using a Hidden Markov Model. By defining a small number of highly descriptive geometrical features relevant to the recognition task, the approach avoids the poor scalability (termed the `curse of dimensionality') that is often associated with featurefusion AVSR methods. The paper describes comparisons of the new approach with conventional appearance-based methods, namely the discrete cosine transform and the principal component analysis techniques, when operating under simulated ambient noise conditions that affect the spoken phrases. The experimental results demonstrate that, in the presence of audio noise, the geometrical method significantly improves speech recognition accuracy compared with appearance-based approaches, despite the new method requiring significantly fewer features.

Item Type:	Conference or Workshop Item (Lecture)
Additional Information:	Indexed by Scopus
Uncontrolled Keywords:	Lip geometry; Feature fusion; Audio-visual speech recognition; OpenCV
Subjects:	T Technology > TK Electrical engineering. Electronics Nuclear engineering
Faculty/Division:	Faculty of Electrical & Electronic Engineering Institute of Postgraduate Studies
Depositing User:	Mrs Norsaini Abdul Samat
Date Deposited:	28 Dec 2022 04:07
Last Modified:	28 Dec 2022 04:07
URI:	http://umpir.ump.edu.my/id/eprint/29900
Download Statistic:	View Download Statistics

Actions (login required)

View Item