Transitioning from ASP.NET, javascipt to ASP.NET MVC 4/6 and using ajax, jquery

The tool you use depends largely on your needs. I can tell you that right now is a very exciting time to be delving into some of the architecture/framework options out there; there's a lot of power and options available out there.

First of all, depending on your server capabilities/expected load, you can opt to go for a SPA (Single-Page Application) focused framework, like AngularJS, to handle the heavy lifting of your application on the client and use an MVC/WebAPI back-end ostensibly as a data-access layer. You can use the vanilla MS MVC implementation (which is built around jQuery) and almost instantly build full-stack (but less snazzy) applications. You can even go through and re-engineer your ASP.NET using .NET 4.5 to get better performance and features.

Every option has a learning curve, but I think you'll find that vanilla MVC is the easiest to settle into. Razor views are simple to work with, intuitive, and have all the power of an ASP page with almost none of the clutter. jQuery is basically a JavaScript wrapper that has a ton of pre-built functionality, such as simplified DOM traversal and selection, AJAX wrapper classes, JSON parsing, and best of all, a plug-in friendly framework.

I'll stop sounding like a pitch-man now. Your concerns in order:
1. Yes, AJAX does not require post-back and can update from server to client immediately. It is most likely the right tool for the job, but I would use JSON rather than XML.

2. The jQuery extension jQueryUI has options for popovers, modals, etc. If you want to be more forward-leaning, use the Bootstrap/jQuery combo that comes built-into MVC5. KendoUI has some good pay-to-play options, but there are many others out there.

3. Learning curve for Bootstrap/jQueryUI is low and quick. There's a lot of information out there about both.

4. jQuery is JavaScript in easy mode. If you want more power have a look at some of the other frameworks out there. I'm working angular at the moment, but there are several options that can be complementary. Difficulty varies (largely with capability).

5. MVC works more like an OOP approach to web applications, so if you can think in OOP/N-Tier design it's a snap. If you can't...well...I don't think it's a very complex implementation. As long as you can understand how Data, Business, and UI are connected you should be just fine.

There's a robust tutorial for vanilla MVC here:
http://www.codeproject.com/Articles/207797/Learn-MVC-Model-View-Controller-step-by-step-in

Best of luck!