I am not sure if i understood your question completely.
When you publish your service, it will output dlls of your class library project in a publish folder. Then you would need to run 'heat' command that will output a wix file referencing the files/dlls of publish folder to be included in the installer. After this step, you would need to build your wix project to get the installer. For more info on heat command refer to following link
Harvest Tool (Heat)