how to know when end playing wav file using mmsysem library in C++

Depends on the callback mechanism you have adopted in waveOutOpen() API.

1) If you choose to have a window or thread receive callback information, the following messages are sent to the window procedure function to indicate the progress of waveform-audio output: MM_WOM_OPEN, MM_WOM_CLOSE, and MM_WOM_DONE.
MM_WOM_DONE indicates the end of playback.

2) If you choose to have a function receive callback information, the following messages are sent to the function to indicate the progress of waveform-audio output: WOM_OPEN, WOM_CLOSE, and WOM_DONE.
WOM_DONE also indicates the end of playback.

References:
- WOM_DONE[^]
- MM_WOM_OPEN[^]
- MM_WOM_CLOSE[^]