Click here to Skip to main content
Click here to Skip to main content
Go to top

Converting Text-To-Speech and and using mouth motion animation

, 27 Nov 2001
Rate this:
Please Sign up or sign in to vote.
This program shows how to convert text to speech and use mouth motion

Introduction

Parts of this code are based on James Matthews' original article Visemes: Representing Mouth Positions.

Figure 1: GUI of application program

This project shows I can convert from Text into Speech that followed by mouth motion. Before you build this project, I suggest you to read my article , "Simple Program for Text to Speech Using SAPI". These steps for building project:

  1. New Application

  2. Setting SAPI on project

  3. Building GUI of project

  4. Coding

  5. Testing

1. New Application

Create your application with MFC AppWizard (exe) and type SpeakMouth as project name. Click OK button. In Step 1 of MFC AppWizard, choice Dialog based and click Finish button.

Figure 2 : New application.

Figure 3: Application based on Dialog. Click Finish button

2. Setting SAPI on Project

Setting SAPI on this project like the article , "Simple Program for Text to Speech Using SAPI".

3. Building GUI of project

Figure 4: GUI Model

From figure 4, you can build and design like hat. This ID and properties of GUI component:

No Component Name ID Add variable using ClassWizard
1 Picture IDC_MOUTH_IMG CStatic :: m_cMouth
2 Group Box IDC_STATIC -
3 Edit Box IDC_TEXT CString :: m_sText
4 Button IDC_SPEAK_BTN -

Model GUI of this project shows on figure 4. After you build GUI, then you must import files bitmap consist of graphics of mouth (figure 5) in Resource View of project. Choice files bitmap in folder res (figure 6). I have put files bitmap of mouth shape in folder res in this project, and you can find these files bitmap in your Speech SDK too. These files bitmap you must import:

  • mic eyes closed.bmp
  • mic.bmp
  • mic_eyes_narrow.bmp
  • mic_mouth_2.bmp
  • mic_mouth_3.bmp
  • mic_mouth_4.bmp
  • mic_mouth_5.bmp
  • mic_mouth_6.bmp
  • mic_mouth_7.bmp
  • mic_mouth_8.bmp
  • mic_mouth_9.bmp
  • mic_mouth_10.bmp
  • mic_mouth_11.bmp
  • mic_mouth_12.bmp
  • mic_mouth_13.bmp

Figure 5: Import file bitmap into project

Figure 6: Choice file Batmap

After you put all files bitmap into Resource View of project, you also must set properties of these files bitmap ie. assign ID of files bitmap. You can do it with right clicking of your mouse into each file bitmap (figure 7) and you'll see window like figure 8. Here's ID of files bitmap:

File Bitmap ID
  mic eyes closed.bmp  IDB_MICEYESCLO
  mic.bmp  IDB_MICFULL
  mic_eyes_narrow.bmp  IDB_MICEYESNAR
  mic_mouth_2.bmp  IDB_MICMOUTH2
  mic_mouth_3.bmp  IDB_MICMOUTH3
  mic_mouth_4.bmp  IDB_MICMOUTH4
  mic_mouth_5.bmp  IDB_MICMOUTH5
  mic_mouth_6.bmp  IDB_MICMOUTH6
  mic_mouth_7.bmp  IDB_MICMOUTH7
  mic_mouth_8.bmp  IDB_MICMOUTH8
  mic_mouth_9.bmp  IDB_MICMOUTH9
  mic_mouth_10.bmp  IDB_MICMOUTH10
  mic_mouth_11.bmp  IDB_MICMOUTH11
  mic_mouth_12.bmp  IDB_MICMOUTH12
  mic_mouth_13.bmp  IDB_MICMOUTH13

Figure 7: Setting properties of file bitmap

Figure 8: Properties of file bitmap

4. Coding

There are three classes in this project (figure 8) ie:

  • CAboutDlg
  • CSpeakMouthApp
  • CSpeakMouthDlg

Figure 9: Classes of project

Call ClassWizard  to add variable into class CSpeakMouthDlg, type m_sError as name of variable and its type is CString, protected.

Below steps of coding processes:

  1. In the file StdAfx.h, put these code lines for accessing SAPI COM
  2.  #include <atlbase.h>
     extern CComModule _Module;
     #include <atlcom.h>
    
  3. On the class CSpeakMouthDlg, add the code lines at the top of the file SpeakMouthDlg.h
  4. #include "sapi.h"
    #include <sphelper.h>
    
    // CONTANTS OF MOUTH
    #define CHARACTER_WIDTH     128
    #define CHARACTER_HEIGHT    128
    #define WEYESNAR            14 // eye positions
    #define WEYESCLO            15
    
        Beside that, add code lines on the top of class CDialog (file SpeakMouthDlg.cpp):
     #define WM_RECOEVENT	WM_USER+101	
     /////////////////////////////////////////////////////////
     // Mouth Mapping Array (from Microsoft's TTSApp Example)
    
     const int g_iMapVisemeToImage[22] = {
         0,  // SP_VISEME_0 = 0,   // Silence
        11, // SP_VISEME_1,        // AE, AX, AH
        11, // SP_VISEME_2,        // AA
        11, // SP_VISEME_3,        // AO
        10, // SP_VISEME_4,        // EY, EH, UH
        11, // SP_VISEME_5,        // ER
        9,  // SP_VISEME_6,        // y, IY, IH, IX
        2,  // SP_VISEME_7,        // w, UW
        13, // SP_VISEME_8,        // OW
        9,  // SP_VISEME_9,        // AW
        12, // SP_VISEME_10,       // OY
        11, // SP_VISEME_11,       // AY
        9,  // SP_VISEME_12,       // h
        3,  // SP_VISEME_13,       // r
        6,  // SP_VISEME_14,       // l
        7,  // SP_VISEME_15,       // s, z
        8,  // SP_VISEME_16,       // SH, CH, JH, ZH
        5,  // SP_VISEME_17,       // TH, DH
        4,  // SP_VISEME_18,       // f, v
        7,  // SP_VISEME_19,       // d, t, n
        9,  // SP_VISEME_20,       // k, g, NG
        1	// SP_VISEME_21,      // p, b, m
     };
    
     ////////////////////////////////////////////////////////////
    
  5. On the class CSpeakMouthDlg, add a member method (you just click on right mouse, and choice Add Member Method). Method:  InitMouthImageList()::BOOL, InitializationSAPI()::BOOL and DestroySAPI()::BOOL , all in protected mode. Beside that, Add variables, here's the code lines: 
  6.    CString m_sError;
       BOOL InitMouthImageList();
       BOOL DestroySAPI();
       BOOL InitializationSAPI();
    	
       ///////////////////////////////////////////////////////
       // Speech API Variables
    	
       CComPtr<ISpVoice> IpVoice;
       CImageList m_cMouthList;
       int	m_iMouthBmp;
       CRect m_cMouthRect;
       ///////////////////////////////////////////////////////
    
  7. Now, you can implement your method that you have created
  8.  void CSpeakMouthDlg::OnDestroy() 
    {
    
        DestroySAPI();
        CDialog::OnDestroy();
    
    }
    
    BOOL CSpeakMouthDlg::InitializationSAPI()
    {
        if (FAILED(CoInitialize(NULL))) {
            m_sError=_T("Error intialization COM");
            return FALSE;
        }
    
        HRESULT hRes;   
        hRes = IpVoice.CoCreateInstance(CLSID_SpVoice);
    
        if (FAILED(hRes)) {
            m_sError=_T("Error creating voice");
            return FALSE;
        }
    
        hRes = IpVoice->SetInterest(SPFEI(SPEI_VISEME), 
            SPFEI(SPEI_VISEME));    
        if (FAILED(hRes)) {
            m_sError=_T("Error creating interest...seriously");
            return FALSE;
        }
    
        hRes = IpVoice->SetNotifyWindowMessage(m_hWnd, 
            WM_RECOEVENT, 0, 0);
        if (FAILED(hRes)) {
            m_sError=_T("Error setting notification window");
            return FALSE;
        }
    
        return TRUE;
    }
    
    BOOL CSpeakMouthDlg::DestroySAPI()
    {
        if (IpVoice) {
            IpVoice.Release();
        }
        return TRUE;
    }
    
    BOOL CSpeakMouthDlg::InitMouthImageList()
    {
        m_cMouth.GetClientRect(&m_cMouthRect);
        m_cMouth.ClientToScreen(&m_cMouthRect);
        ScreenToClient(&m_cMouthRect);
    
        CBitmap bmp;
        m_cMouthList.Create(CHARACTER_WIDTH, 
            CHARACTER_HEIGHT, ILC_COLOR32 | ILC_MASK, 1, 0);
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICFULL));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH2));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH3));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH4));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH5));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH6));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH7));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH8));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH9));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH10));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH11));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH12));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICMOUTH13));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICEYESNAR));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        bmp.LoadBitmap(MAKEINTRESOURCE(IDB_MICEYESCLO));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        bmp.Detach();
    
        m_cMouthList.SetOverlayImage(1, 1);
        m_cMouthList.SetOverlayImage(2, 2);
        m_cMouthList.SetOverlayImage(3, 3);
        m_cMouthList.SetOverlayImage(4, 4);
        m_cMouthList.SetOverlayImage(5, 5);
        m_cMouthList.SetOverlayImage(6, 6);
        m_cMouthList.SetOverlayImage(7, 7);
        m_cMouthList.SetOverlayImage(8, 8);
        m_cMouthList.SetOverlayImage(9, 9);
        m_cMouthList.SetOverlayImage(10, 10);
        m_cMouthList.SetOverlayImage(11, 11);
        m_cMouthList.SetOverlayImage(12, 12);
        m_cMouthList.SetOverlayImage(13, 13);
        m_cMouthList.SetOverlayImage(14, WEYESNAR);
        m_cMouthList.SetOverlayImage(15, WEYESCLO);
    
        return TRUE;
    }
    
  9. In the class CSpeakMouthDlg, the method OnInitDialog() must have the following code for SAPI initialization
  10. // Set the icon for this dialog.  
    // The framework does this automatically
    //  when the application's main window 
    // is not a dialog
    SetIcon(m_hIcon, TRUE); // Set big icon
    SetIcon((HICON)(LoadImage(AfxGetResourceHandle(), 
            MAKEINTRESOURCE(IDR_MAINFRAME), 
            IMAGE_ICON, 16, 16, 0)), FALSE);
    
    m_sError=_T("");
    if (!InitializationSAPI()) {
        AfxMessageBox(m_sError);        
        DestroySAPI();
    }
    
    InitMouthImageList();
    
    return TRUE;  
    
    
  11. In the construction of class CSpeakMouthDlg::CSpeakMouthDlg(CWnd* pParent=NULL)
  12.  m_iMouthBmp = 0;
    
  13. Now, you must create the method to handle the event of converting speech into mouth motion. You can send the message handler ie: WM_RECOEVENT that you have defined as WM_USER + 101. In file SpeakMouthDlg.h, you type:
  14. // Generated message map functions
    //{{AFX_MSG(CSpeakMouthDlg)
    virtual BOOL OnInitDialog();
    afx_msg void OnSysCommand(UINT nID, LPARAM lParam);
    afx_msg void OnPaint();
    afx_msg HCURSOR OnQueryDragIcon();
    afx_msg void OnDestroy();
    afx_msg void OnSpeak();
    //}}AFX_MSG
    // add message handler
    afx_msg LRESULT OnMouthEvent(WPARAM, LPARAM); 
    DECLARE_MESSAGE_MAP()
    and in the file SpeakMouthDlg.cpp, add these code lines
    BEGIN_MESSAGE_MAP(CSpeakMouthDlg, CDialog)
    //{{AFX_MSG_MAP(CSpeakMouthDlg)
    ON_WM_SYSCOMMAND()
    ON_WM_PAINT()
    ON_WM_QUERYDRAGICON()
    ON_WM_DESTROY()
    ON_BN_CLICKED(IDC_SPEAK_BTN, OnSpeak)
    //}}AFX_MSG_MAP
    // add message handler
    ON_MESSAGE(WM_RECOEVENT, OnMouthEvent)  
    END_MESSAGE_MAP()    
    

    This is the implementation of the method OnMouthEvent() :

    LRESULT CSpeakMouthDlg::OnMouthEvent(WPARAM wParam, 
                                         LPARAM lParam) 
    {
        CSpEvent event;  
    
        while (event.GetFrom(IpVoice) == S_OK) {
            switch (event.eEventId) {
                case SPEI_VISEME:
                    m_iMouthBmp = 
                        g_iMapVisemeToImage[event.Viseme()];
                    InvalidateRect(m_cMouthRect, false);
                    break;
            }
        }
    
        return 0;
    }
    
  15. Now, you add a method to handle the button click (IDC_SPEAK_BTN) event. Type OnSpeak as the name of the method, and add these code lines as it's implementation:
  16.  void CSpeakMouthDlg::OnSpeak() 
     {
        UpdateData();
    	
        USES_CONVERSION;
        IpVoice->Speak(m_sText.AllocSysString(), SPF_ASYNC, 
            NULL);		
     }
    
  17. So, you can debug and run your program

5. Testing

To test your application program, you can run it and type word a into the Edit Box. After that, click the Speak button.

Reference

Speech SDK 5.1 from Microsoft Inc.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

Share

About the Author

Agus Kurniawan
Founder PE College
Indonesia Indonesia
He gradueted from Sepuluh Nopember Institute of Technology (ITS) in Department of Electrical Engineering, Indonesia. His programming interest is VC++, C#, VB, VB.NET, .NET, VBScript, Delphi, C++ Builder, Assembly,and ASP/ASP.NET. He's founder for PE College(www.pecollege.net), free video tutorial about programming, infrastructure, and computer science. He's currently based in Depok, Indonesia. His blog is http://geeks.netindonesia.net/blogs/agus and http://blog.aguskurniawan.net

Comments and Discussions

 
QuestionHow to add additional character to the source code given above with the click of checkbox? PinmemberMember 1072421513-May-14 15:56 
QuestionMouth Animation doesnt work PinmemberMember 1072421511-May-14 3:06 
AnswerAnimation Mouth PinmemberRupayani25-Apr-11 3:20 
Can you tell me!
How to create it in vb.net!Please help!
Can you send to luhrupa@gmail.com
QuestionHow to set own voice PinmemberVC_RYK4-Feb-10 20:15 
QuestionStop Speaking Code example PinmemberAhmed Faraz23-May-07 9:47 
AnswerRe: Stop Speaking Code example Pinmembercristitomi23-Sep-07 19:21 
Questionhow to set each syllable to each wav file in vb.net Pinmembershariati8-Aug-06 20:38 
Generaldirectx and tts Pinmemberwtatters5-Apr-05 11:49 
GeneralPlease Help! Pinmembermehta_janani3-Feb-05 3:51 
QuestionHow to realize Japanese Text-to-Speech? Pinmembernjdibfm20-Jan-05 18:08 
AnswerRe: How to realize Japanese Text-to-Speech? PinsussChandru713-Mar-05 2:04 
GeneralAbout recognition Pinmembersinmorn24-May-04 14:42 
GeneralPhoneme extraction Pinmembersridahran8-Mar-04 22:20 
QuestionHow to retrieve from database PinmemberVilma S30-Jan-04 3:00 
GeneralGreat, but... PinsussJessicaLampe26-Jan-04 3:31 
GeneralRe: Great, but... Pinmemberrhdunn15-Jan-05 23:56 
Generaldownload bmps seperate PinsussAnonymous25-Jan-04 9:15 
Questionhow to get rid from speech sdk? PinmemberFARTal_error31-Jul-03 14:20 
QuestionHow to execute Visual C++? PinmemberiProgram (WangYue)23-Apr-03 1:14 
Generalsapi.h Pinmembermaitre cesar22-Apr-03 1:27 
GeneralRe: sapi.h PinmemberRavi Bhavnani23-Apr-03 1:42 
QuestionHow interesting it is! PinmemberRedwings20-Dec-02 11:11 
Generalsapi.h PinsussAnonymous15-Aug-02 0:35 
GeneralRe: sapi.h PinmemberBrakanjan22-Aug-02 9:24 
GeneralRe: sapi.h Pinmemberrhdunn16-Jan-05 3:53 
QuestionHow to get sound? PinmemberAnonymous7-Jun-02 23:11 
AnswerRe: How to get sound? Pinmembereponk25-Feb-03 17:05 
Generalmouth motion animation PinmemberCodeProjectSQ29-Nov-01 16:49 
GeneralRe: mouth motion animation PinmemberJoeSox1-Sep-03 7:48 
QuestionIdentical code ? PinmemberKonstantin Boukreev29-Nov-01 14:18 
AnswerRe: Identical code ? PinmemberAnony mouse3-Dec-01 15:26 
GeneralRe: Identical code ? PinmemberKonstantin Boukreev3-Dec-01 23:37 
GeneralRe: Identical code ? PinmemberCatalin5-Mar-02 1:37 
GeneralRe: Identical code ? PinmemberVladimir L.19-Apr-02 7:19 
GeneralRe: Identical code ? PinmemberVladimir L.21-Apr-02 2:06 
AnswerRe: Identical code ? Pinmemberzizz_jr24-Dec-02 23:08 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web04 | 2.8.140916.1 | Last Updated 28 Nov 2001
Article Copyright 2001 by Agus Kurniawan
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid