How to get a notification using web scrapping - Python

Question

1.00/5 (1 vote)

See more:

Hi,

I want to use python web scrapping using selenium and want to get a notification when the date is modified of the content.

URL : Reporting framework 2.9 - European Banking Authority[^]

in this website on the weekly and monthly basis the content gets updated and whatever content gets updated a date is mentioned against them and i want to get a notification in my email that xxxx.....EBA2.9 (example) is updated. Can anyone help me out.

What I have tried:

so far i have tried :

from selenium import webdriver
from selenium.webdriver.common.keys import Keys #to search the particular key

DR = webdriver.Chrome()  #created instance "DR" of the browser
DR.get("https://eba.europa.eu/risk-analysis-and-data/reporting-frameworks/reporting-framework-2.9")

Details = DR.find_elements_by_css_selector('div>p>strong')
print(Details)

for dtl in Details:
    DetailText = dtl.find_element_by_css_selector('div.a')
    print(DetailText.text)

but didn't get the result.

Posted 22-Sep-19 0:28am

Sunil KK

Updated 22-Sep-19 0:29am

v2

Add a Solution

Comments

[no name] 22-Sep-19 12:05pm

"If you would like to be updated on the EBA's work, subscribe to the EBA email alerts."

Sunil KK 23-Sep-19 0:38am

Thanks Schmitz for the response, but i want to do this part using web scrapping.

ZurdoDev 23-Sep-19 10:23am

Then you will have to parse the html and find exactly what you are looking for.

Sunil KK 24-Sep-19 2:58am

Hi Dev, I actually get an notification whenever any latest content gets uploaded by EBA. Eg. if you visit the link (i mentioned above) you will find date against of content like (Updated 19 August 2019) [ZIP, 2994KB]. If this date gets changed i need a notification

ZurdoDev 24-Sep-19 9:34am

I don't understand what you want us to do. You said you do "get an notification" and then you said "i need a notification"

Sunil KK 25-Sep-19 0:38am

Hi Dev, Sorry for the confusion. I actually wants to get an notification if the dates gets updated. Let me try to give you an example :

3. DPM v2.9

a. DPM database (Updated 19 August 2019) [ZIP, 81129KB]

b. DPM dictionary (Updated 19 August 2019) [XLS, 940KB]

c. DPM table layout and data point categorisation (Updated 19 August 2019) [ZIP, 7214KB]

d. Changes compared to previous version phase 1 (Updated 28 May 2019) [ZIP, 6263KB]

e. Changes compared to previous version phase 2 (Updated 19 August 2019) [ZIP, 2994KB]

now suppose, EBA updated some content for "e" in that date will be updated from "Updated 19 August 2019" to "Updated 25 September 2019"

Now, i want to get an notification in the below format :

3. DPM v2.9

e. Changes compared to previous version phase 2 (Updated 25 September 2019) [ZIP, 2994KB]

I hope, i am able to elaborate what exactly i want using web scrapping.

ZurdoDev 25-Sep-19 8:31am

If you want to get a notification when their site changes you have to ask them. I don't understand what you want us to do.

If they won't notify you then you need to write code that parses their html everyday looking for the date and then do whatever you want.

Sunil KK 27-Sep-19 7:00am

Hi Dev,

exactly, i want this "If they won't notify you then you need to write code that parses their html everyday looking for the date and then do whatever you want." could you help me to build that code which parse the HTML ?

ZurdoDev 27-Sep-19 7:43am

No. This is exactly what my first comment was 3 days ago. Where are you stuck? You have some code above. What's wrong?

Sunil KK 27-Sep-19 12:01pm

In my Code, i am getting the title, but when i am trying to print those titles it is giving me an error and i don't know how to use those title to get an notification.

ZurdoDev 27-Sep-19 12:06pm

Step 1 is to figure out how to tell when something is new. Is there a spot on the page that has an updated date, for example?

Sunil KK 27-Sep-19 12:10pm

Yes, this is the link just for your reference

https://eba.europa.eu/risk-analysis-and-data/reporting-frameworks/reporting-framework-2.9

you can see "2. Validation rules (Updated on 10 September 2019) [XLSX, 4.2MB]" this kind of statement where we can see the date.

ZurdoDev 27-Sep-19 12:14pm

If you look at the source they do not have that information inside an element with a name or id so there is no way to get it directly. Like I said the first time, you'll have to figure out a way to parse the data and find what you want. You might be able to look for "(Updated on") for example, but they could change it and it won't work anymore.

Sunil KK 28-Sep-19 7:08am

I agree with your point there is no name or id give inside an element. But i have to do this, do you have any idea ? any suggestion>

ZurdoDev 28-Sep-19 9:05am

Look for "updated on(" and then try to parse it. Or, contact the site. They may have an RSS feed or some other way already.

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)