Your best bet is to add another bool: ignoreClick.
Set it to true in the MouseUp handler if it was doing a drag and check it in the Click handler.
Reset it in two places: at the end of the Click handler, and in the MouseDown event just to be on the safe side.
I got the order of execution wrong: when you release the mouse you get events in this order:
Click
MouseClick
Up
So it's even easier: all you have to do is check if you are dragging and ignore the event if you are - the Up event handler will clear the drag flag so subsequent ones will work.