I developed a web scraper (using C#) that should be able to make thousands of requests each time.
The problem is that the website's server will block my IP after a number of requests.
1- How to prevent being blocked?
2- How to know when will the website's server will block my IP? I mean how to know my limit whether being certain amount of traffic or certain number of requests.