Usually website has an RSS file. Then we can parse it to have the latest news, however, there are some that didn't make this RSS file so we should parse directly HTML of this website.
You can download this sample here.
Using the Code
First of all, we should add to the reference the
<a href="http://htmlagilitypack.codeplex.com/">Htmlagilitypack</a>. You can download it from nuget on your Visual Studio.
P.S.: If you are working on Windows Phone, it will have some problems with that DLL, so you must add these two DLL files,
System.Xml.Xpath. You can also find it on nuget.
We create a new function that takes as parameter the website that you want to parse:
Then, we send a request to the website to get all HTML pages:
HttpClient http = new HttpClient();
var response = await http.GetByteArrayAsync(website);
String source = Encoding.GetEncoding("utf-8").GetString(response, 0, response.Length - 1);
source = WebUtility.HtmlDecode(source);
HtmlDocument resultat = new HtmlDocument();
P.S.: You should pay attention to the Encoding, each website has an Encoding. In this example, it uses
utf-8, you can find it on the attribute
charset on the website HTML.
After that, we inspect the element that we want to parse and get its id or class, then we can retrieve it easily.
As you can see in the picture, we want to parse information of these devices that are all wrapped in
ul, but before that we must find the ancestor
div that has an id or a class, in this example the
div has a class named
So now, we will filter the HTML with only the content of this
div, then we get all tag of
li that contains information that we want to get.
List<HtmlNode> toftitle = resultat.DocumentNode.Descendants().Where
(x => (x.Name == "div" && x.Attributes["class"] != null &&
After each filter you do, it is preferred to breakpoint the project to verify our work.
As a result, we get 11
divs that have class named
block_content, so you should verify which item contains information that we want to get. In this example, it's item N°6.
var li = toftitle.Descendants("li").ToList();
foreach (var item in li)
var link = item.Descendants("a").ToList().GetAttributeValue("href", null);
var img = item.Descendants("img").ToList().GetAttributeValue("src", null);
var title = item.Descendants("h5").ToList().InnerText;
Inside each item of
li, we will get the link, image and Title.
Descendants allow you to get all tag with specified name inside the item.
GetAttributeValue allows you to get the attribute of the tag.
InnerText allows you to get Text between tags.
InnerHtml allows you to get HTML.
Difficulty of parsing HTML depends on the structure of the website.