A popular phrase among developers is to "Fail fast and fail often." I'm not saying this is wrong, but I'm not saying this is right either. The idea to fail fast is to get immediate feedback that something bad happened. Letting an unhandled exception close the application isn't always acceptable, either. I don't think it's acceptable at all, even for work tools to have an unhandled exception, but that means you've came up with a way to handle every single exception possible. That is very unlikely. Software development is a craft in this regard because you have to find a balance.
My rule of thumb is if I can recover from an exception, then I need to ensure that my application can recover. It's not exactly rocket science. I believe the key is to ensure that you never throw a generic exception. Each exception that you throw needs to be specific to the issue or problem at hand. That way you catch that specific exception and not a similar exception from lower level classes. You also never want to handle a generic exception, either. Sometimes you won't be able to recover from a specific exception and so it's perfectly fine to let it get deferred to a higher level class to handle it correctly.
The balance comes when you're at the highest level and you're getting exceptions that were thrown levels down. You may not be able to catch all of them, because you may not know what exceptions were not handled in the levels below. In .NET it's really easy to send your department information when an unhandled exception occurs. It's not the best way and it may not always be the right way, but sometimes it's the only way to properly find out what type of exceptions are being thrown. I'd rather have the application crash then to have my end user using an application where the state is unknown. This can lead to more problems and issues.
Software development is something you craft and with time you'll be able to learn and generate better exception handlers. Don't throw generic exceptions and don't handle generic exceptions, because if it's generic then you have no idea what really happened. Leaving an end user with the ability to do more damage is never the way to go.