In my opinion Microsoft is the best company around when it comes to supporting developers. Information on the general Windows Mobile APIs are freely available to everyone; no registration or fees required. The source code is not open, but a majority of the APIs are publically documented. You can find these APIs reference on the Windows Mobile Development Center in MSDN. For Windows Mobile I could divide the APIs into one of 2 categories; native and managed.
If you are just starting with Windows Mobile development you will want to use the Managed APIs. All of these APIs are implemented through .Net. When using these APIs a lot of low level tasks such as some aspects of memory management are taken care of for you. The Native APIs give one direct access to more of the device at the cost of less protection from one's mistakes and more effort being demanded to perform certain tasks. Since versions of the .Net framework are supported on both the desktop and on windows mobile devices you may want to start with some simple desktop application development with .Net before developing on a Windows Mobile device. After getting a foundation in the .Net framework pick up the book "Microsoft Mobile Development Handbook" by Andy Wigley, Daniel Moth, and Peter Foor.
The bear minimum toolset that one would need to develop for Windows Mobile would be the .Net framework and one of the Windows Mobile SDKs. These are free downloads. However, I would strongly encourage you to invest in Visual Studio 2008 Professional. It will provide a much more complete development environment then using the command line tools of the SDK.
You usually have access to the file system of a Windows Mobile device (though some businesses or phone carriers may restrict certain areas of the device for their own security or policy reasons). giving you the ability to copy programs to the device as desired.