Click here to Skip to main content
11,479,170 members (60,855 online)
Click here to Skip to main content

The singular non-modality of MFC modal dialogs

, 6 Apr 2003 Ms-PL 204.2K 1.6K 51
Rate this:
Please Sign up or sign in to vote.
Explains the pseudo modality of CDialog based modal dialogs and a problem with the CDialog::EndDialog implementation

Introduction

For the non-programmer, modal dialog boxes are those that refuse to go away till you dismiss them, which is usually done by clicking on the OK or Cancel buttons. For the programmer (the non-VB ones anyway), modal dialog boxes are those that disable their immediate parent window before they are created, and enable their parent window when they are closed. Thus the essence of a modal dialog is that you cannot do anything to the parent window till you close the modal dialog. Modeless dialogs are comparatively more polite and less fussy in that they do not insist on being dismissed to allow you to access their parent windows. I trust that by now, any infinitesimal doubts any of you might have had regarding the modality of a dialog has been absolutely eradicated.

MFC has a class called CDialog which is a CWnd derived class that is specifically used for the creation and display of dialog boxes on screen - both modal and modeless dialogs are supported. Modeless dialogs are created using Create() and you have to destroy them on your own using DestroyWindow() and they behave just like any normal modeless window. Modal dialogs are created and shown using the formidable DoModal() method of the CDialog class. And you close a modal dialog by calling EndDialog() or alternatively by calling OnOK() or OnCancel() both of which internally call EndDialog(). The CDialog::EndDialog method will call the EndDialog Win32 API function which is defined in user32.dll. EndDialog (the API function) will not immediately close the dialog, rather it will set a flag which will instruct the message queue to exit the loop, destroy the dialog window and enable the parent window. So far so good; everything seems so peaceful and serene, and the little birds in the tree across the yard are tweeting in a beautiful voice.

The non-modality factor

The Win32 API supports various dialog box related functions which include functions for creating both modeless and modal dialogs. There are the CreateDialogXXX set of functions like CreateDialog, CreateDialogIndirect etc. which are used to create modeless dialog boxes and the DialogBoxXXX set of functions like DialogBox, DialogBoxIndirect etc. which create modal dialogs. And as mentioned in the previous section there is also the EndDialog function which is used to end modal dialogs and only modal dialogs. Modeless dialogs must be terminated by calling DestroyWindow directly. The basic reason why you should not attempt to use EndDialog on a modeless dialog is that modeless dialogs do not have their own modal message loop nor do they disable their parent window, which basically nullifies the utility value of using EndDialog on them.

Alright, now this is what will give you a solid kick - MFC CDialog based modal dialogs are not modal dialogs. Just in case you did not read that right the first time, here it goes again - MFC CDialog based modal dialogs are not modal dialogs. I would have said it a third time but the fear of being termed a parrot thwarts me from doing so. Open dlgcore.cpp and examine the source code and you'll see that this astonishing statement is true. The MFC CDialog class uses the CreateDialogIndirect API function to create a pseudo-modal dialog and if you look up CreateDialogIndirect on MSDN you'll see that it is used to create a modeless dialog. The MFC command routing mechanism uses a combination of message maps and virtual functions to achieve what it does and a true modal dialog will totally wreck this mechanism because then the modal message loop is controlled outside the scope of the MFC command routing machinery. Thus the developers had no choice but to creat a pseudo-modal modeless dialog and then document it very intensely as a modal dialog.

Basically these are the summarized steps that are done when a pseudo-modal dialog is created via CDialog::DoModal -

  • A boolean flag bEnableParent is set to FALSE
  • If the parent window is enabled, then it is disabled and bEnableParent is set to TRUE
  • The dialog box is created using CreateDialogIndirect
  • A message pump is maintained using CWnd::RunModalLoop
  • Once the modal loop exits (when CDialog::EndDialog is called) the parent window is enabled if bEnableParent is TRUE
  • The dialog window is destroyed by calling DestroyWindow
  • And DoModal returns the argument passed to EndDialog

As you can see the developers have taken great pains to ensure that the pseudo-modal dialogs behave exactly as modal dialogs are supposed to. They have even tried to accommodate for unusual scenarios where there might be two modal dialogs both having the same parent - this is where the bEnableParent comes into play. Thus when the second modal dialog comes up, it will not set bEnableParent to TRUE because the parent window is already disabled. Thus when it is dismissed it will not enable the parent window which is exactly what is needed because another dialog modal to the same parent window is still active on screen.

EndDialog - a tiny flaw

As mentioned a couple of times already in this chapter, modal dialogs are dismissed using EndDialog. Let's see what EndDialog looks like (for those interested, it is defined in dlgcore.cpp)

void CDialog::EndDialog(int nResult)
{
    ASSERT(::IsWindow(m_hWnd));

    if (m_nFlags & (WF_MODALLOOP|WF_CONTINUEMODAL))
        EndModalLoop(nResult);

    ::EndDialog(m_hWnd, nResult);
}

CWnd::EndModalLoop is called to exit the message loop maintained by CWnd::RunModalLoop which is quite fine and then the EndDialog API function is called to terminate the modal dialog. This is also fine and Hey, hey, hey, just wait a cotton picking minute there!!!! The EndDialog API function is to be used only to close modal dialogs, but here we are attempting to use it to close a pseudo-modal dialog which is actually nothing but a modeless dialog disguised to reflect a modality it does not truly possess. [stunned silence...] Well, after that initial shock, let's all relax a bit. After all it's not the end of the world and EndDialog is not really a very harmful function call; it will only attempt to end a non-existent modal message loop and will enable the parent window of the dialog window. The former attempt will fail obviously and the latter attempt will just do something which we would have done anyway on our own, because the moment the pseudo-modal pump exits, DoModal will re-enable the parent window. So, is everything well and good, are the birds chirping peacefully once again?

The bug

Remember all that bEnableParent stuff we discussed a couple of paragraphs earlier, and how I mentioned how this flag is used to make sure that when multiple modal dialogs exist together that have the same parent window, this flag prevents the premature enabling of the parent window when one of the modal dialog siblings are dismissed? Well guess what? The careless programmer at Microsoft who coded this particular function had just rendered all those precautionary measures totally null and void. Because now any MFC modal dialog that is closed using EndDialog (which also means OnOK and OnCancel because those functions will internally call EndDialog) will enable it's parent window irrespective of what value bEnableParent holds. This basically means if you have two or more MFC modal dialogs which have the same parent window, then the moment you dismiss any one of those modal dialogs, all the other modal dialogs will lose their modality because now the parent window has been re-enabled.

Steps to reproduce this bug

  • Create an MFC dialog based application
  • Add a new dialog to it and associate a class with it called CChildDialog
  • Set two timers in the OnInitDialog method of the main dialog :-
  • SetTimer(1000,1000,NULL);	
    SetTimer(2000,2000,NULL);
  • Bring up a pseudo-modal dialog each, in the timer handler :-
  • void CModalDemoDlg::OnTimer(UINT nIDEvent)
    {
        KillTimer(nIDEvent);
        CChildDialog dlg(this);
        dlg.DoModal();
    
        CDialog::OnTimer(nIDEvent);
    }
  • Run the program and wait 2 seconds by which time you will have the main dialog and two modal child dialogs
  • Try to access the main dialog and you will be unable to do so because of the presence of the two modal dialogs
  • Now dismiss one of the child dialogs using the OK or the Cancel button
  • Now try to access the main dialog and you'll see to your astonishment that you will be able to do so, despite the presence of a modal dialog on screen

The project I have attached was created using VC++.NET 2003 Final Beta and I apologize to all those of you who do not have that version. But following the above mentioned steps shouldn't take you more than a few minutes at most. What is so amazing is that this bug has gone unnoticed through several versions of MFC. I checked as far back as VC++ 6 and that has the exact same problem too. As far as I see it, all someone's got to do is to comment out or delete the call to the EndDialog API call. I am guessing that what happened was this - the MFC developers were so much used to calling the native API equivalents from their wrapper functions (for example they would call the MessageBox API from inside CWnd::Messagebox) that someone must have automatically typed in that line without thinking and the QA guys must also have overlooked this error. And this problem remained mostly unknown because it was a very rare situation for a program to have a multi-modal window architecture where several modal dialog windows have the same parent window.

Work-around

Okay, so until Microsoft corrects this bug, what we could do (other than having to correct the MFC code and recompile MFC) is to override the OK and Cancel button handlers and to use this code instead of the default :-

{
    if (m_nFlags & (WF_MODALLOOP|WF_CONTINUEMODAL))
        EndModalLoop(IDOK); // or IDCANCEL
}

Notice how we are not calling the base class (if we do then ::EndDialog(...) gets called and all our efforts are wasted). If you want to exit the modal dialog in a place outside the OK/Cancel button handlers you should use the same code except that you might want to return a different value (like perhaps IDYES for example).

Conclusion

For all I know I might be the biggest jack-ass in town and there might be a perfectly legitimate reason for this matter, but something tells me that, that is a very remote contingency. By the way I'd like to thank Shog9 for sending me the VC++ 6 version of dlgcore.cpp at a rather late hour of the night (which to him is the equivalent of noon for most of us). I'd also like to explicitly mention here that this article is by no means an attempt to mock the amazing set of Microsoft programmers who developed the MFC library.

Version History

  • April 5 2003 - Article first published
  • April 7 2003 - Article updated with Work-around to the problem

License

This article, along with any associated source code and files, is licensed under The Microsoft Public License (Ms-PL)

Share

About the Author

Nish Nishant

United States United States
Nish Nishant is a Software Architect/Consultant based out of Columbus, Ohio. He has over 15 years of software industry experience in various roles including Lead Software Architect, Principal Software Engineer, and Product Manager. Nish is a recipient of the annual Microsoft Visual C++ MVP Award since 2002 (13 consecutive awards as of 2014).

Nish is an industry acknowledged expert in the Microsoft technology stack. He authored
C++/CLI in Action for Manning Publications in 2005, and had previously co-authored
Extending MFC Applications with the .NET Framework for Addison Wesley in 2003. In addition, he has over 140 published technology articles on CodeProject.com and another 250+ blog articles on his
WordPress blog. Nish is vastly experienced in team management, mentoring teams, and directing all stages of software development.

Contact Nish : You can reach Nish on his google email id voidnish.

Website and Blog

Comments and Discussions

 
AnswerSlight Correction for Modeless CDialog's [modified] Pin
Member 445110530-Nov-09 12:32
memberMember 445110530-Nov-09 12:32 
GeneralAccess violation if parent window is closed before MFC CDialog-based modal dialog is closed Pin
doroboy20-Mar-08 23:54
memberdoroboy20-Mar-08 23:54 
GeneralRe: Access violation if parent window is closed before MFC CDialog-based modal dialog is closed Pin
Member 445110530-Nov-09 12:34
memberMember 445110530-Nov-09 12:34 
GeneralWindow or Dialog with Desktop as parent Pin
Moti@IT22-Jun-07 4:00
memberMoti@IT22-Jun-07 4:00 
Question"Enter" and "ESC" Keys could dismiss a modal dialog, anyway to prevent this? Pin
Sstar7-Aug-06 19:23
memberSstar7-Aug-06 19:23 
AnswerRe: "Enter" and "ESC" Keys could dismiss a modal dialog, anyway to prevent this? Pin
lzinggl16-Oct-06 1:55
memberlzinggl16-Oct-06 1:55 
GeneralThank you !! Pin
Fastfootskater12-Aug-07 21:52
memberFastfootskater12-Aug-07 21:52 
GeneralThanks Nish Pin
Paul Hooper12-Jul-06 2:52
memberPaul Hooper12-Jul-06 2:52 
GeneralC++ version suffers similar issue Pin
ThrashMaster13-Apr-06 11:56
memberThrashMaster13-Apr-06 11:56 
GeneralProblem with modal dialog stack Pin
hickory3-Feb-06 5:59
memberhickory3-Feb-06 5:59 
GeneralRe: Problem with modal dialog stack Pin
hickory16-Feb-06 2:12
memberhickory16-Feb-06 2:12 
GeneralRe: Problem with modal dialog stack Pin
hickory16-Feb-06 2:18
memberhickory16-Feb-06 2:18 
GeneralRe: Problem with modal dialog stack Pin
hickory16-Feb-06 9:59
memberhickory16-Feb-06 9:59 
AnswerRe: Problem with modal dialog stack Pin
abc13a18-Sep-07 22:27
memberabc13a18-Sep-07 22:27 
GeneralMessage loops Pin
Stephen Hewitt11-Jan-06 13:36
memberStephen Hewitt11-Jan-06 13:36 
QuestionHow to enable parent AND one child Pin
Sven Appenrodt24-May-05 0:34
memberSven Appenrodt24-May-05 0:34 
GeneralVC6 also needs UpdateData() in OnOK() Pin
bruce2g1-May-05 22:26
memberbruce2g1-May-05 22:26 
GeneralMultiple Modal Dialogs Pin
Dinplor18-Apr-05 9:12
memberDinplor18-Apr-05 9:12 
GeneralNon-modality of modal dialogs Pin
Yadnesh24-Nov-03 3:04
memberYadnesh24-Nov-03 3:04 
QuestionQuestion "DoModal" ? Pin
Petrisor20-Nov-03 6:54
memberPetrisor20-Nov-03 6:54 
AnswerRe: Question "DoModal" ? Pin
DavidCrow25-Feb-05 7:15
memberDavidCrow25-Feb-05 7:15 
QuestionWhat about scenario with one modal dialog and two frame windows? Pin
domKing25-Oct-03 5:50
memberdomKing25-Oct-03 5:50 
GeneralThe correct way is Pin
JimmyO17-Apr-03 4:47
memberJimmyO17-Apr-03 4:47 
void CChildDialog::EndDialog(int nResult)
{
ASSERT(::IsWindow(m_hWnd));

if (m_nFlags & (WF_MODALLOOP|WF_CONTINUEMODAL))
EndModalLoop(nResult);
DestroyWindow();
}
Big Grin | :-D
GeneralRe: The correct way is [Actually no!] Pin
Nishant S17-Apr-03 6:47
editorNishant S17-Apr-03 6:47 
Questiony cant we access some of the parent's member variables? Pin
Miguel Lopes14-Apr-03 3:44
memberMiguel Lopes14-Apr-03 3:44 
GeneralI also think it's a bug Pin
Ernest Laurentin7-Apr-03 6:58
memberErnest Laurentin7-Apr-03 6:58 
GeneralRe: I also think it's a bug Pin
Rob Manderson7-Apr-03 8:41
editorRob Manderson7-Apr-03 8:41 
GeneralWhy didnt you Pin
Andreas Saurwein7-Apr-03 5:43
memberAndreas Saurwein7-Apr-03 5:43 
GeneralRe: Why didnt you Pin
Anonymous24-Jun-03 7:27
sussAnonymous24-Jun-03 7:27 
Generalasynchronous calls and single thread Pin
laoran21-Dec-03 23:52
memberlaoran21-Dec-03 23:52 
GeneralModal, ApplicationModal, SystemModal Pin
Patje5-Apr-03 10:57
memberPatje5-Apr-03 10:57 
GeneralRe: Modal, ApplicationModal, SystemModal Pin
Nishant S5-Apr-03 12:26
editorNishant S5-Apr-03 12:26 
GeneralFor people who do not have Everett... Pin
Nishant S5-Apr-03 10:41
editorNishant S5-Apr-03 10:41 
GeneralIf only the source was better commented... Pin
Shog95-Apr-03 6:27
memberShog95-Apr-03 6:27 
GeneralRe: If only the source was better commented... Pin
Nishant S5-Apr-03 10:35
editorNishant S5-Apr-03 10:35 
QuestionWhat about Win32 API? Pin
Paolo Messina5-Apr-03 5:48
memberPaolo Messina5-Apr-03 5:48 
AnswerRe: What about Win32 API? Pin
Nishant S5-Apr-03 5:59
editorNishant S5-Apr-03 5:59 
QuestionWhy din't you ? Pin
Smitha Vijayan5-Apr-03 4:33
memberSmitha Vijayan5-Apr-03 4:33 
AnswerRe: Why din't you ? Pin
Nishant S5-Apr-03 5:10
editorNishant S5-Apr-03 5:10 
GeneralRe: Why din't you ? Pin
Smitha Vijayan5-Apr-03 6:40
memberSmitha Vijayan5-Apr-03 6:40 
GeneralRe: Why din't you ? Pin
DavidCrow15-May-03 10:06
memberDavidCrow15-May-03 10:06 
QuestionHow did you find this? Pin
Gary R. Wheeler5-Apr-03 2:33
memberGary R. Wheeler5-Apr-03 2:33 
AnswerRe: How did you find this? Pin
Nishant S5-Apr-03 4:24
editorNishant S5-Apr-03 4:24 
GeneralMaybe it's a bug Pin
Rob Manderson4-Apr-03 22:34
editorRob Manderson4-Apr-03 22:34 
GeneralRe: Maybe it's a bug Pin
Nishant S5-Apr-03 4:27
editorNishant S5-Apr-03 4:27 
GeneralRe: Maybe it's a bug Pin
Marc Clifton5-Apr-03 4:46
memberMarc Clifton5-Apr-03 4:46 
GeneralRe: Maybe it's a bug Pin
Greg_G24-Jun-03 7:43
memberGreg_G24-Jun-03 7:43 
GeneralRe: Maybe it's a bug Pin
aihtdikh30-Sep-04 21:14
memberaihtdikh30-Sep-04 21:14 
Generalstacked boxes Pin
.S.Rod.4-Apr-03 21:29
member.S.Rod.4-Apr-03 21:29 
GeneralRe: stacked boxes Pin
Nishant S5-Apr-03 4:34
editorNishant S5-Apr-03 4:34 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.150520.1 | Last Updated 7 Apr 2003
Article Copyright 2003 by Nish Nishant
Everything else Copyright © CodeProject, 1999-2015
Layout: fixed | fluid