Correct, you have to go with silverlight only. Prism is one of the popular MVVM frame work. there no mandatory that you need to use all of these things, You can create your app with out them as well. But Building WPF/Silverlight is advisable.
Strictly speaking, PRISM is not an MVVM framework. Yes, it has some MVVM features, but it is better regarded as an application composition framework. If the OP didn't build his original app with PRISM then he's got a whole host of work in trying to convert it into PRISM.
*pre-emptive celebratory nipple tassle jiggle* - Sean Ewington