How to handle overlapping mouse input in GUI?

Sorry for the late answer, but I hope I can still help. I had my own little GUI and 3D engine project going, but said goodbye to Mickeysoft when they abandoned XNA and wanted to have everything the Windows 8 way.

There is no general answer to your question. It depends very much on how your UI works and what kind of structures you have implemented to store the current state of the UI. If the elements of your UI can overlap, then you will have to establish some way to determine the Z-Order of the elements and move them to the back or to the front when needed.

The elements of my UI were controls. A control always had a parent and could itself be parent to any number of child controls. The children were clipped to the space of their parents. Any child (or part of it) outside the frame of the parent would not be rendered or considered to receive any kind of input.

The root object of the UI was an invisible control that was always present and occupied the entire screen space. It was the only control without a parent. Any forms or layouts would have to be children of this root control. The controls themselves did not contain any Z-order information.

When processing mouse input, I simply walked the tree and tested the mouse position against the boundaries of the controls. If a control was 'hit', then I followed down into the list of its children to see if there was an even better fit. Up to here the Z-order was simply defined by the structure of my control tree. The work was done if I got no more than a single result.

To deal with overlapping controls, I needed only one more convention: The order of the children in the list (which was kept in the parent) determined their Z-order. The first ones would be in the back, the ones at the end would be in the front. Changing the Z-position of a control became simply a matter of repositioning it at the beginning or end of the list.

This way I could easily draw them by going through the list from the front to the back. The hit test for mouse input could not stop anymore at the first result. It had to cover the entire object tree. If there was more than one result, the later one would be rendered in front of the first one, so the last result would always be what I was looking for.

I hope this helped you a little. Choosing a good data structure and using its properties can be an alternative to explicit implementations.