It's difficult to trace or help with the above information. You need to see some logs if any and related data. You also should have shared the error you get once the website is down.
There might be some sort of memory leak.
Have a read on this informative Microsoft MSDN article on IIS 6.0
] to configure and learn all about it.
Also, have a look at this similar discussion: ALL sites go down - IIS restart doesn't work - Connection Reset error in browser