Learning to use an artificial visual cue in speech identification

Stephens, Joseph D. W.; Holt, Lori L.

doi:doi:10.1121/1.3479537

Volume/Page
Keyword
DOI
Citation
Advanced

Volume: Page/Article:

Search Content Type

article doi:

Citation:

You are not logged in You are logged out of this journal. Log In

Journal of the Acoustical Society of America / Volume 128 / Issue 4 / SPEECH PERCEPTION [71]

Previous Article | Next Article

Learning to use an artificial visual cue in speech identification ^a

^a Portions of this work were presented at the 46th meeting of the Psychonomic Society and the 151st meeting of the Acoustical Society of America.

J. Acoust. Soc. Am. Volume 128, Issue 4, pp. 2138-2149 (2010); (12 pages)

Joseph D. W. Stephens and Lori L. Holt

Department of Psychology and Center for the Neural Basis of Cognition, Carnegie Mellon University, Pittsburgh, Pennsylvania 15213

Alerts
- Alert Me When Cited
- Alert Me When Corrected
Tools
Share
- Email Abstract
- Connotea
- CiteULike
- del.icio.us
- BibSonomy
- Tweet this Article
- Add to Facebook

Abstract

References (32)

Citing Articles (1)

Article Objects (8)

Related Content

Full Text: Read Online (HTML) | Download PDF | Buy PDF (US$30) | View Cart

Visual information from a speaker’s face profoundly influences auditory perception of speech. However, relatively little is known about the extent to which visual influences may depend on experience, and extent to which new sources of visual speech information can be incorporated in speech perception. In the current study, participants were trained on completely novel visual cues for phonetic categories. Participants learned to accurately identify phonetic categories based on novel visual cues. These newly-learned visual cues influenced identification responses to auditory speech stimuli, but not to the same extent as visual cues from a speaker’s face. The novel methods and results of the current study raise theoretical questions about the nature of information integration in speech perception, and open up possibilities for further research on learning in multimodal perception, which may have applications in improving speech comprehension among the hearing-impaired.

ACKNOWLEDGMENTS

This research was supported by NIH award 1 F31 DC007284-01 to JDWS, by National Science Foundation award BCS-0345773 to LLH, by a grant from the Bank of Sweden Tercentenary Foundation to LLH, and by the Center for the Neural Basis of Cognition. The authors thank Christi Gomez for help with data collection and manuscript preparation.

Article Outline

INTRODUCTION
METHOD
1. General method
2. Participants
3. Stimuli
  Example audiovisual stimuli may be viewed at the following URL: http://www.psy.cmu.edu/~lholt/php/gallery_audiovisual.php.
  1. Distributions
  2. Auditory stimuli
    1. Auditory stimulus creation
    2. Auditory stimulus selection
  3. Artificial visual stimuli
  4. Natural visual stimuli
4. Procedure
  1. General procedure
  2. Audiovisual training tasks
    1. “Explicit” training task
    2. “Incidental learning” task
  3. Visual identification (pre-test and post-test)
  4. Factorial audiovisual identification
  5. Audiovisual mismatch identification in noise
RESULTS
1. Visual identification
2. Factorial audiovisual identification
  1. Visual influence
  2. Model comparisons
    1. FLMP
    2. SCM
    3. Model fits
3. Audiovisual mismatch identification in noise
  1. Intelligibility
  2. McGurk effect
DISCUSSION
1. Summary of current findings
2. Theoretical implications
3. Implications for speech comprehension in adverse conditions
4. Conclusion

RELATED DATABASES

To view database links for this article, you need to log in.

KEYWORDS and PACS

Keywords

hearing, speech intelligibility, speech recognition, visual perception

PACS

43.71.Rt

Sensory mechanisms in speech perception
43.71.An

Models and theories of speech perception
43.71.Es

Vowel and consonant perception; perception of words, sentences, and fluent speech
43.71.Ft

Development of speech perception

ARTICLE DATA

History

Received 05 Jan 2010
Accepted 22 Jul 2010
Revised 20 May 2010

Digital Object Identifier

http://dx.doi.org/10.1121/1.3479537

PUBLICATION DATA

ISSN

0001-4966 (print)

Publisher

$abstract.society.value is a Member of CrossRef

Acoustical Society of America

For access to fully linked references, you need to log in.

For access to citing articles, you need to log in.

Figures (6) Tables (2)

Access to article objects (figures, tables, multimedia) requires a subscription; log in to view available files.
(Access to supplementary files, where available, is free for this journal.)

Learning to use an artificial visual cue in speech identification ^a