Click here to Skip to main content
15,798,200 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:

Can somebody please help me out in doing Web Data Scraping.
Problem is that like I want to scrape a web site (ex:- with some information like
item Description , manufacturer ,Supplier Par no, price, etc of the some of the products in their site.

Are there any external tools to do this or Can we write a scraping program..

Can somebody please help me out in this feature.


1 solution

I don't know of a tool to do exactly this, but there are similar tools and you could build one yourself.

I once did this to download all links from a webpage. You might want to look at one of the web crawlers on Code Project and borrow some of their ideas.

Basically, you use a WebRequest to get the HTML for a page. Then, you can use various means to extract the data you want. I chose to use regular expressions, because they were quick and dirty and I was never going to use the application again. You could also parse the HTML.

You'd have to look for patterns, like any TD element that contains the string "Item Description", then look at the next TD defined after that (where your target data will be held). You could then apply that algorithm to every page that uses that pattern. You could also have the web crawler search every page on for you, that way you can have it search the pages and the contents of the pages automatically.
Share this answer

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900