Click here to Skip to main content
15,791,675 members
Please Sign up or sign in to vote.
1.00/5 (1 vote)
See more:
Hello there,

I'm new to data science, and I've been attempting to scrape data from a website using Python and BeautifulSoup, as described in this article, but I keep encountering an issue that I can't seem to fix. I'm using the following code:
import requests
from bs4 import BeautifulSoup

url = ''
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

data = soup.find('div', class_='info').text

However, when I execute the code, I see the following error message:
AttributeError: 'NoneType' object has no attribute 'text'

What I have tried:

I double-checked the HTML of the website, and the element with class "info" is clearly present. I have no idea what is producing this problem or how to repair it. Any assistance would be much appreciated!
Updated 25-Jul-23 3:11am
Graeme_Grant 25-Jul-23 6:31am    
Page scrapping is not allowed on most websites. Do they have an API?
Member 15627495 25-Jul-23 6:56am    
data var is an array ( you can have 0, 1, lot of values return). you need a loop and indexes to retrieve value by ".text"

1 solution

data = soup.find('div', class_='info').text

The message is telling you that soup.find did not find a div with that class type, so the find result is None. So when you try to access the text attribute it raises the AttributeError exception. And if you go to Example Domain[^] and inspect the page source you can see that there is no such div.
Share this answer

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900