Click here to Skip to main content
15,608,053 members
Please Sign up or sign in to vote.
1.00/5 (1 vote)
See more:
I want to scrape some data from a website. the website requires to login first. I can't do it in code with the help of htmlagilitypack. I also tried to do this with webbrowsercontrol, but it doesn't work properly. Please, help me.

What I have tried:

I tried to login with htmlagility pack but couldn't do it. also I have search google extensively, couldn't get specific answer.

p.s. If any website that doesn't require login then htmlagilitypack works fine.
Updated 20-Apr-22 12:45pm
Richard MacCutchan 20-Apr-22 7:51am    
If the website requires a login then that is to protect it from webscrapers.
Graeme_Grant 20-Apr-22 18:48pm    
I believed the same until a couple of days ago when I read this: Web scraping is legal, US appeals court reaffirms – TechCrunch[^]
Dave Kreskowiak 20-Apr-22 21:26pm    
It may be legal but that in no way means some company will allow you to do it. You still have to face the terms of service from the company, which will prevent you from doing it.
Graeme_Grant 20-Apr-22 21:28pm    
I'm not saying that I agree with it but this sets a precedence and overrides any terms of service. Companies should be concerned.
Dave Kreskowiak 20-Apr-22 21:30pm    
No, it does not override Terms of Service. Think about it, if the company has proprietary data that only they generate and they keep it behind a subscription, does the law give you the right to walk into the company and demand the data for free? NO IT DOESN'T!

1 solution

I did a quick Google Search using: htmlagility pack credentials[^]

Found several solutions, including this one: [SOLVED] => HtmlAgilityPack and Authentication[^]

If this is not suitable, then there are several other solutions using the search above.
Share this answer

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900