• Volume/Page
  • Keyword
  • DOI
  • Citation
  • Advanced
   
 
 
 

You are not logged in You are logged out of this journal. Log In

Journal of the Acoustical Society of America / Volume 131 / Issue 2 / SPEECH PERCEPTION [71]

Faciliation of Mandarin tone perception by visual speech in clear and degraded audio: Implications for cochlear implants a

a Portions of this work were presented in “Multimodal perception of Mandarin tone for Cochlear Implant Users,” Interspeech, Brisbane, September 2008.

J. Acoust. Soc. Am. Volume 131, Issue 2, pp. 1480-1489 (2012); (10 pages)

Damien Smith and Denis Burnham

MARCS Auditory Laboratories, University of Western Sydney, NSW, 1797, Australia

Full Text: Read Online (HTML) | Download PDF | Buy PDF (US$30) | View Cart
Cochlear implant (CI) users in tone language environments report great difficulty in perceiving lexical tone. This study investigated the augmentation of simulated cochlear implant audio by visual (facial) speech information for tone. Native speakers of Mandarin and Australian English were asked to discriminate between minimal pairs of Mandarin tones in five conditions: Auditory-Only, Auditory-Visual, CI-simulated Auditory-Only, CI-simulated Auditory-Visual, and Visual-Only (silent video). Discrimination in CI-simulated audio conditions was poor compared with normal audio, and varied according to tone pair, with tone pairs with strong non-F0 cues discriminated the most easily. The availability of visual speech information also improved discrimination in the CI-simulated audio conditions, particularly on tone pairs with strong durational cues. In the silent Visual-Only condition, both Mandarin and Australian English speakers discriminated tones above chance levels. Interestingly, tone-naïve listeners outperformed native listeners in the Visual-Only condition, suggesting firstly that visual speech information for tone is available, and may in fact be under-used by normal-hearing tone language perceivers, and secondly that the perception of such information may be language-general, rather than the product of language-specific learning. This may find application in the development of methods to improve tone perception in CI users in tone language environments.

© 2012 Acoustical Society of America

ACKNOWLEDGMENTS

The authors thank Dr. Brett Swanson and Professor Jim Patrick, Cochlear Ltd., Lane Cove, Sydney, Australia, for their assistance; Amanda Reid for assistance with manuscript editing and preparation, Dr. Nan Xu for assistance with the corpus; Professor Marcus Taft for testing space at the University of New South Wales; and all participants for their cooperation. The research was jointly supported by Cochlear Ltd. and an Australian Research Council Linkage grant (LP0562532) to the second author.

Article Outline

  1. INTRODUCTION
  2. METHOD
    1. Participants
    2. Corpus
    3. Recording and editing
    4. Acoustic analyses
    5. Cochlear implant simulation
    6. Procedure
    7. Design and counterbalancing
  3. RESULTS
    1. Auditory and auditory-visual conditions
      1. Normal audio comparisons
      2. Simulated cochlear implant audio comparisons
    2. Visual-only condition
    3. Acoustic cues and discrimination performance: Regression analysis
      1. AO, AV, and VO Conditions
      2. CI audio conditions
  4. DISCUSSION

RELATED DATABASES

To view database links for this article, you need to log in.

KEYWORDS, PACS, and IPC

Keywords

cochlear implants, hearing, speech intelligibility

PACS

  • 43.71.Es

    Vowel and consonant perception; perception of words, sentences, and fluent speech

  • 43.71.Rt

    Sensory mechanisms in speech perception

  • 43.66.Ts

    Auditory prostheses, hearing aids

  • 43.71.Hw

    Cross-language perception of speech

International Patent Classification (IPC)

  • A61F2/18

    Internal ear or nose parts, e.g. ear-drums

ARTICLE DATA

History
Received 23 May 2011
Accepted 05 Dec 2011
Revised 28 Nov 2011
Digital Object Identifier
http://dx.doi.org/10.1121/1.3672703

PUBLICATION DATA

ISSN

0001-4966 (print)  

Publisher

$abstract.society.value is a Member of CrossRef Acoustical Society of America

For access to fully linked references, you need to log in.

Figures (4) Tables (3)

Access to article objects (figures, tables, multimedia) requires a subscription; log in to view available files.
(Access to supplementary files, where available, is free for this journal.)

Access to article objects (figures, tables, multimedia) requires a subscription; log in to view available files.
(Access to supplementary files, where available, is free for this journal.)



Close

Your Recent History Recent History Help

Recently Viewed

  1. Faciliation of Mandarin tone perception by visual speech in clear and degraded audio: Implications for cochlear implants

Home Publications Meetings Contact ASA Site Map Back to Top

Copyright © Acoustical Society of America

close