Click here to Skip to main content
13,352,408 members (60,107 online)
Click here to Skip to main content
Add your own
alternative version


71 bookmarked
Posted 27 Nov 2001

Converting Text-To-Speech and and using mouth motion animation

, 27 Nov 2001
Rate this:
Please Sign up or sign in to vote.
This program shows how to convert text to speech and use mouth motion


Parts of this code are based on James Matthews' original article Visemes: Representing Mouth Positions.

Figure 1: GUI of application program

This project shows I can convert from Text into Speech that followed by mouth motion. Before you build this project, I suggest you to read my article , "Simple Program for Text to Speech Using SAPI". These steps for building project:

  1. New Application

  2. Setting SAPI on project

  3. Building GUI of project

  4. Coding

  5. Testing

1. New Application

Create your application with MFC AppWizard (exe) and type SpeakMouth as project name. Click OK button. In Step 1 of MFC AppWizard, choice Dialog based and click Finish button.

Figure 2 : New application.

Figure 3: Application based on Dialog. Click Finish button

2. Setting SAPI on Project

Setting SAPI on this project like the article , "Simple Program for Text to Speech Using SAPI".

3. Building GUI of project

Figure 4: GUI Model

From figure 4, you can build and design like hat. This ID and properties of GUI component:

No Component Name ID Add variable using ClassWizard
1 Picture IDC_MOUTH_IMG CStatic :: m_cMouth
2 Group Box IDC_STATIC -
3 Edit Box IDC_TEXT CString :: m_sText
4 Button IDC_SPEAK_BTN -

Model GUI of this project shows on figure 4. After you build GUI, then you must import files bitmap consist of graphics of mouth (figure 5) in Resource View of project. Choice files bitmap in folder res (figure 6). I have put files bitmap of mouth shape in folder res in this project, and you can find these files bitmap in your Speech SDK too. These files bitmap you must import:

  • mic eyes closed.bmp
  • mic.bmp
  • mic_eyes_narrow.bmp
  • mic_mouth_2.bmp
  • mic_mouth_3.bmp
  • mic_mouth_4.bmp
  • mic_mouth_5.bmp
  • mic_mouth_6.bmp
  • mic_mouth_7.bmp
  • mic_mouth_8.bmp
  • mic_mouth_9.bmp
  • mic_mouth_10.bmp
  • mic_mouth_11.bmp
  • mic_mouth_12.bmp
  • mic_mouth_13.bmp

Figure 5: Import file bitmap into project

Figure 6: Choice file Batmap

After you put all files bitmap into Resource View of project, you also must set properties of these files bitmap ie. assign ID of files bitmap. You can do it with right clicking of your mouse into each file bitmap (figure 7) and you'll see window like figure 8. Here's ID of files bitmap:

File Bitmap ID
  mic eyes closed.bmp  IDB_MICEYESCLO
  mic.bmp  IDB_MICFULL
  mic_eyes_narrow.bmp  IDB_MICEYESNAR
  mic_mouth_2.bmp  IDB_MICMOUTH2
  mic_mouth_3.bmp  IDB_MICMOUTH3
  mic_mouth_4.bmp  IDB_MICMOUTH4
  mic_mouth_5.bmp  IDB_MICMOUTH5
  mic_mouth_6.bmp  IDB_MICMOUTH6
  mic_mouth_7.bmp  IDB_MICMOUTH7
  mic_mouth_8.bmp  IDB_MICMOUTH8
  mic_mouth_9.bmp  IDB_MICMOUTH9
  mic_mouth_10.bmp  IDB_MICMOUTH10
  mic_mouth_11.bmp  IDB_MICMOUTH11
  mic_mouth_12.bmp  IDB_MICMOUTH12
  mic_mouth_13.bmp  IDB_MICMOUTH13

Figure 7: Setting properties of file bitmap

Figure 8: Properties of file bitmap

4. Coding

There are three classes in this project (figure 8) ie:

  • CAboutDlg
  • CSpeakMouthApp
  • CSpeakMouthDlg

Figure 9: Classes of project

Call ClassWizard  to add variable into class CSpeakMouthDlg, type m_sError as name of variable and its type is CString, protected.

Below steps of coding processes:

  1. In the file StdAfx.h, put these code lines for accessing SAPI COM
  2. #include <atlbase.h>
    extern CComModule _Module;
    #include <atlcom.h>
  3. On the class CSpeakMouthDlg, add the code lines at the top of the file SpeakMouthDlg.h
  4. #include "sapi.h"
    #include <sphelper.h>
    #define CHARACTER_WIDTH     128
    #define CHARACTER_HEIGHT    128
    #define WEYESNAR            14 // eye positions
    #define WEYESCLO            15
        Beside that, add code lines on the top of class CDialog (file SpeakMouthDlg.cpp):
     #define WM_RECOEVENT	WM_USER+101	
     // Mouth Mapping Array (from Microsoft's TTSApp Example)
     const int g_iMapVisemeToImage[22] = {
         0,  // SP_VISEME_0 = 0,   // Silence
        11, // SP_VISEME_1,        // AE, AX, AH
        11, // SP_VISEME_2,        // AA
        11, // SP_VISEME_3,        // AO
        10, // SP_VISEME_4,        // EY, EH, UH
        11, // SP_VISEME_5,        // ER
        9,  // SP_VISEME_6,        // y, IY, IH, IX
        2,  // SP_VISEME_7,        // w, UW
        13, // SP_VISEME_8,        // OW
        9,  // SP_VISEME_9,        // AW
        12, // SP_VISEME_10,       // OY
        11, // SP_VISEME_11,       // AY
        9,  // SP_VISEME_12,       // h
        3,  // SP_VISEME_13,       // r
        6,  // SP_VISEME_14,       // l
        7,  // SP_VISEME_15,       // s, z
        8,  // SP_VISEME_16,       // SH, CH, JH, ZH
        5,  // SP_VISEME_17,       // TH, DH
        4,  // SP_VISEME_18,       // f, v
        7,  // SP_VISEME_19,       // d, t, n
        9,  // SP_VISEME_20,       // k, g, NG
        1	// SP_VISEME_21,      // p, b, m
  5. On the class CSpeakMouthDlg, add a member method (you just click on right mouse, and choice Add Member Method). Method:  InitMouthImageList()::BOOL, InitializationSAPI()::BOOL and DestroySAPI()::BOOL , all in protected mode. Beside that, Add variables, here's the code lines: 
  6. CString m_sError;
    BOOL InitMouthImageList();
    BOOL DestroySAPI();
    BOOL InitializationSAPI();
    // Speech API Variables
    CComPtr<ISpVoice> IpVoice;
    CImageList m_cMouthList;
    int  m_iMouthBmp;
    CRect m_cMouthRect;
  7. Now, you can implement your method that you have created
  8.  void CSpeakMouthDlg::OnDestroy() 
    BOOL CSpeakMouthDlg::InitializationSAPI()
        if (FAILED(CoInitialize(NULL))) {
            m_sError=_T("Error intialization COM");
            return FALSE;
        HRESULT hRes;   
        hRes = IpVoice.CoCreateInstance(CLSID_SpVoice);
        if (FAILED(hRes)) {
            m_sError=_T("Error creating voice");
            return FALSE;
        hRes = IpVoice->SetInterest(SPFEI(SPEI_VISEME), 
        if (FAILED(hRes)) {
            m_sError=_T("Error creating interest...seriously");
            return FALSE;
        hRes = IpVoice->SetNotifyWindowMessage(m_hWnd, 
            WM_RECOEVENT, 0, 0);
        if (FAILED(hRes)) {
            m_sError=_T("Error setting notification window");
            return FALSE;
        return TRUE;
    BOOL CSpeakMouthDlg::DestroySAPI()
        if (IpVoice) {
        return TRUE;
    BOOL CSpeakMouthDlg::InitMouthImageList()
        CBitmap bmp;
            CHARACTER_HEIGHT, ILC_COLOR32 | ILC_MASK, 1, 0);
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.Add(&bmp, RGB(255,0,255));
        m_cMouthList.SetOverlayImage(1, 1);
        m_cMouthList.SetOverlayImage(2, 2);
        m_cMouthList.SetOverlayImage(3, 3);
        m_cMouthList.SetOverlayImage(4, 4);
        m_cMouthList.SetOverlayImage(5, 5);
        m_cMouthList.SetOverlayImage(6, 6);
        m_cMouthList.SetOverlayImage(7, 7);
        m_cMouthList.SetOverlayImage(8, 8);
        m_cMouthList.SetOverlayImage(9, 9);
        m_cMouthList.SetOverlayImage(10, 10);
        m_cMouthList.SetOverlayImage(11, 11);
        m_cMouthList.SetOverlayImage(12, 12);
        m_cMouthList.SetOverlayImage(13, 13);
        m_cMouthList.SetOverlayImage(14, WEYESNAR);
        m_cMouthList.SetOverlayImage(15, WEYESCLO);
        return TRUE;
  9. In the class CSpeakMouthDlg, the method OnInitDialog() must have the following code for SAPI initialization
  10. // Set the icon for this dialog.  
    // The framework does this automatically
    //  when the application's main window 
    // is not a dialog
    SetIcon(m_hIcon, TRUE); // Set big icon
            IMAGE_ICON, 16, 16, 0)), FALSE);
    if (!InitializationSAPI()) {
    return TRUE;  
  11. In the construction of class CSpeakMouthDlg::CSpeakMouthDlg(CWnd* pParent=NULL)
  12.  m_iMouthBmp = 0;
  13. Now, you must create the method to handle the event of converting speech into mouth motion. You can send the message handler ie: WM_RECOEVENT that you have defined as WM_USER + 101. In file SpeakMouthDlg.h, you type:
  14. // Generated message map functions
    virtual BOOL OnInitDialog();
    afx_msg void OnSysCommand(UINT nID, LPARAM lParam);
    afx_msg void OnPaint();
    afx_msg HCURSOR OnQueryDragIcon();
    afx_msg void OnDestroy();
    afx_msg void OnSpeak();
    // add message handler
    afx_msg LRESULT OnMouthEvent(WPARAM, LPARAM); 
    and in the file SpeakMouthDlg.cpp, add these code lines
    BEGIN_MESSAGE_MAP(CSpeakMouthDlg, CDialog)
    // add message handler

    This is the implementation of the method OnMouthEvent() :

    LRESULT CSpeakMouthDlg::OnMouthEvent(WPARAM wParam, 
                                         LPARAM lParam) 
        CSpEvent event;  
        while (event.GetFrom(IpVoice) == S_OK) {
            switch (event.eEventId) {
                case SPEI_VISEME:
                    m_iMouthBmp = 
                    InvalidateRect(m_cMouthRect, false);
        return 0;
  15. Now, you add a method to handle the button click (IDC_SPEAK_BTN) event. Type
    as the name of the method, and add these code lines as it's implementation:
  16.  void CSpeakMouthDlg::OnSpeak() 
        IpVoice->Speak(m_sText.AllocSysString(), SPF_ASYNC, 
  17. So, you can debug and run your program

5. Testing

To test your application program, you can run it and type word a into the Edit Box. After that, click the Speak button.


Speech SDK 5.1 from Microsoft Inc.


This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


About the Author

Agus Kurniawan
Founder PE College
Indonesia Indonesia
He gradueted from Sepuluh Nopember Institute of Technology (ITS) in Department of Electrical Engineering, Indonesia. His programming interest is VC++, C#, VB, VB.NET, .NET, VBScript, Delphi, C++ Builder, Assembly,and ASP/ASP.NET. He's founder for PE College(, free video tutorial about programming, infrastructure, and computer science. He's currently based in Depok, Indonesia. His blog is and

You may also be interested in...

Comments and Discussions

QuestionHow to add additional character to the source code given above with the click of checkbox? Pin
Member 1072421513-May-14 16:56
memberMember 1072421513-May-14 16:56 
QuestionMouth Animation doesnt work Pin
Member 1072421511-May-14 4:06
memberMember 1072421511-May-14 4:06 
AnswerAnimation Mouth Pin
Rupayani25-Apr-11 4:20
memberRupayani25-Apr-11 4:20 
Can you tell me!
How to create it in!Please help!
Can you send to
QuestionHow to set own voice Pin
VC_RYK4-Feb-10 21:15
memberVC_RYK4-Feb-10 21:15 
QuestionStop Speaking Code example Pin
Ahmed Faraz23-May-07 10:47
memberAhmed Faraz23-May-07 10:47 
AnswerRe: Stop Speaking Code example Pin
cristitomi23-Sep-07 20:21
membercristitomi23-Sep-07 20:21 
Questionhow to set each syllable to each wav file in Pin
shariati8-Aug-06 21:38
membershariati8-Aug-06 21:38 
Generaldirectx and tts Pin
wtatters5-Apr-05 12:49
memberwtatters5-Apr-05 12:49 
GeneralPlease Help! Pin
mehta_janani3-Feb-05 4:51
membermehta_janani3-Feb-05 4:51 
QuestionHow to realize Japanese Text-to-Speech? Pin
njdibfm20-Jan-05 19:08
membernjdibfm20-Jan-05 19:08 
AnswerRe: How to realize Japanese Text-to-Speech? Pin
Chandru713-Mar-05 3:04
sussChandru713-Mar-05 3:04 
GeneralAbout recognition Pin
sinmorn24-May-04 15:42
membersinmorn24-May-04 15:42 
GeneralPhoneme extraction Pin
sridahran8-Mar-04 23:20
membersridahran8-Mar-04 23:20 
QuestionHow to retrieve from database Pin
Vilma S30-Jan-04 4:00
memberVilma S30-Jan-04 4:00 
GeneralGreat, but... Pin
JessicaLampe26-Jan-04 4:31
sussJessicaLampe26-Jan-04 4:31 
GeneralRe: Great, but... Pin
rhdunn16-Jan-05 0:56
memberrhdunn16-Jan-05 0:56 
Generaldownload bmps seperate Pin
Anonymous25-Jan-04 10:15
sussAnonymous25-Jan-04 10:15 
Questionhow to get rid from speech sdk? Pin
FARTal_error31-Jul-03 15:20
memberFARTal_error31-Jul-03 15:20 
QuestionHow to execute Visual C++? Pin
iProgram (WangYue)23-Apr-03 2:14
memberiProgram (WangYue)23-Apr-03 2:14 
Generalsapi.h Pin
maitre cesar22-Apr-03 2:27
membermaitre cesar22-Apr-03 2:27 
GeneralRe: sapi.h Pin
Ravi Bhavnani23-Apr-03 2:42
memberRavi Bhavnani23-Apr-03 2:42 
QuestionHow interesting it is! Pin
Redwings20-Dec-02 12:11
memberRedwings20-Dec-02 12:11 
Generalsapi.h Pin
Anonymous15-Aug-02 1:35
sussAnonymous15-Aug-02 1:35 
GeneralRe: sapi.h Pin
Brakanjan22-Aug-02 10:24
memberBrakanjan22-Aug-02 10:24 
GeneralRe: sapi.h Pin
rhdunn16-Jan-05 4:53
memberrhdunn16-Jan-05 4:53 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.180111.1 | Last Updated 28 Nov 2001
Article Copyright 2001 by Agus Kurniawan
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid