To answer your question I will first quote from a book
Silverlight Recipes, A Problem Solution Approach that "Silverlight supports the Microsoft ASF and MP4 File Formats (also known as MPEG-4 File Format version 2)"
WMV and WMA files follows the ASF file container structures. MP4 and MOV can also be decoded by Silverlight as they are common variants of the MP4 container structure.
So, based on that understanding I can say Silverlight can take WMV, WMA, MP4 and MOV.
But there is almost always a "BUT", that even though Silverlight can consume the above container formatsor structures, it can only decode if those file formats are encoded using Windows Media VC-1/H.264 (or equiv. in MPEG-4 part 10 and the MPEG-4 Advanced Video Codec) standards.
Now, that is according to the Author, out-of-the-box support in Silverlight.
And yes, Silverlight does not support out-of-the-box AVI file structures. For these types you will have to write your own code which you can then hook up with Silverlight to render frames from an avi video as in the code project article link below:
Play AVI files in Silverlight 4 using MediaElement and MediaStreamSource[
^]