SuperLectures.com

SYNTHESIZING VISUAL SPEECH TRAJECTORY WITH MINIMUM GENERATION ERROR

Speech Synthesis

Full Paper at IEEE Xplore

Presented by: Lijuan Wang, Author(s): Lijuan Wang, Microsoft Research Asia, China; Yi-Jian Wu, Microsoft Corporation, China; Xiaodan Zhuang, Beckman Institute / University of Illinois at Urbana-Champaign, China; Frank K. Soong, Microsoft Research Asia, China

In this paper, we propose a minimum generation error (MGE) training method to refine the audio-visual HMM to improve visual speech trajectory synthesis. Compared with the traditional maximum likelihood (ML) estimation, the proposed MGE training explicitly optimizes the quality of generated visual speech trajectory, where the audio-visual HMM modeling is jointly refined by using a heuristic method to find the optimal state alignment and a probabilistic descent algorithm to optimize the model parameters under the MGE criterion. In objective evaluation, compared with the ML-based method, the proposed MGE-based method achieves consistent improvement in the mean square error reduction, correlation increase, and recovery of global variance. It also improves the naturalness and audio-visual consistency perceptually in the subjective test.


  Speech Transcript

|

  Slides

Enlarge the slide | Show all slides in a pop-up window

0:00:16

  1. slide

0:00:45

  2. slide

0:02:32

  3. slide

0:06:00

  4. slide

0:08:16

  5. slide

0:09:26

  6. slide

0:10:58

  7. slide

0:14:24

  8. slide

0:14:51

  9. slide

0:15:09

 10. slide

0:16:14

 11. slide

0:17:40

 12. slide

  Comments

Please sign in to post your comment!

  Lecture Information

Recorded: 2011-05-26 15:25 - 15:45, Panorama
Added: 15. 6. 2011 19:16
Number of views: 28
Video resolution: 1024x576 px, 512x288 px
Video length: 0:22:45
Audio track: MP3 [7.71 MB], 0:22:45