Click here to Skip to main content
15,867,568 members
Home / Discussions / JavaScript
   

JavaScript

 
AnswerRe: need help Pin
Nathan Minier1-Oct-15 6:39
professionalNathan Minier1-Oct-15 6:39 
GeneralRe: need help Pin
Member 119720431-Oct-15 8:10
Member 119720431-Oct-15 8:10 
GeneralRe: need help Pin
Member 1197204310-Oct-15 11:15
Member 1197204310-Oct-15 11:15 
GeneralRe: need help Pin
Nathan Minier13-Oct-15 1:23
professionalNathan Minier13-Oct-15 1:23 
QuestionSet up a counter to count the number of times an alert is triggered in a website. Pin
Rajesh_198025-Sep-15 9:18
Rajesh_198025-Sep-15 9:18 
AnswerRe: Set up a counter to count the number of times an alert is triggered in a website. Pin
Wombaticus8-Oct-15 1:25
Wombaticus8-Oct-15 1:25 
Questionhow to perform color effect on image in imagemagick? Pin
Rajiv Rahi24-Sep-15 2:24
professionalRajiv Rahi24-Sep-15 2:24 
QuestionCannot get dynamic javascript content from web page Pin
elelont223-Sep-15 3:08
elelont223-Sep-15 3:08 
Hi,
i am trying to use pyQT and python to get the dynamic content from a web page. The problem is that i still only get the static content. What could be wrong with the code below?

Code is based on this link: https://impythonist.wordpress.com/2015/01/06/ultimate-guide-for-scraping-javascript-rendered-web-pages/[^]
Python
import sys
import time
from PyQt4.QtGui import *
from PyQt4.QtCore import *
from PyQt4.QtWebKit import *
from lxml.html import fromstring, tostring, iterlinks

class Render(QWebPage):
  def __init__(self, url):
    self.app = QApplication(sys.argv)
    QWebPage.__init__(self)
    self.loadFinished.connect(self._loadFinished)
    self.mainFrame().load(QUrl(url))
    self.app.exec_()
    print("inside 1")

  def _loadFinished(self, result):
    self.frame = self.mainFrame()
    self.app.quit()
    print("inside 2")

  #def userAgentForUrl(self, url):
  #  return 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36 OPR/32.0.1948.25'

url = 'http://www.somepage.com'

r = Render(url)
print("inside 3")

print("Sleeping..")
time.sleep(5)
print("Sleeping done")

result = r.mainFrame().toHtml()
print(result.encode('utf-8'))

I added the sleep(5) to ensure that the dynamic content has time to load but is does not help.

Why doesn't the r.mainFrame() contain the valid dynamically created page contents? Is it not updated after the pageloaded event?

Regards
AnswerRe: Cannot get dynamic javascript content from web page Pin
Richard MacCutchan23-Sep-15 3:51
mveRichard MacCutchan23-Sep-15 3:51 
QuestionCallback is not a function error Pin
MadDashCoder14-Sep-15 21:36
MadDashCoder14-Sep-15 21:36 
AnswerRe: Callback is not a function error Pin
Richard Deeming15-Sep-15 2:06
mveRichard Deeming15-Sep-15 2:06 
GeneralRe: Callback is not a function error Pin
MadDashCoder15-Sep-15 6:39
MadDashCoder15-Sep-15 6:39 
GeneralRe: Callback is not a function error Pin
Richard MacCutchan15-Sep-15 9:09
mveRichard MacCutchan15-Sep-15 9:09 
GeneralRe: Callback is not a function error Pin
MadDashCoder15-Sep-15 10:40
MadDashCoder15-Sep-15 10:40 
GeneralRe: Callback is not a function error Pin
Richard MacCutchan15-Sep-15 10:45
mveRichard MacCutchan15-Sep-15 10:45 
GeneralRe: Callback is not a function error Pin
MadDashCoder15-Sep-15 14:21
MadDashCoder15-Sep-15 14:21 
GeneralRe: Callback is not a function error Pin
Richard MacCutchan15-Sep-15 21:20
mveRichard MacCutchan15-Sep-15 21:20 
GeneralRe: Callback is not a function error Pin
MadDashCoder15-Sep-15 23:29
MadDashCoder15-Sep-15 23:29 
GeneralRe: Callback is not a function error Pin
Richard MacCutchan16-Sep-15 0:01
mveRichard MacCutchan16-Sep-15 0:01 
GeneralRe: Callback is not a function error Pin
MadDashCoder16-Sep-15 9:33
MadDashCoder16-Sep-15 9:33 
GeneralRe: Callback is not a function error Pin
Richard MacCutchan16-Sep-15 21:26
mveRichard MacCutchan16-Sep-15 21:26 
QuestionJavascript and JQuery? Pin
Member 119720439-Sep-15 8:35
Member 119720439-Sep-15 8:35 
AnswerRe: Javascript and JQuery? Pin
Blikkies10-Sep-15 18:47
professionalBlikkies10-Sep-15 18:47 
AnswerRe: Javascript and JQuery? Pin
Member 1201497027-Sep-15 6:56
Member 1201497027-Sep-15 6:56 
AnswerRe: Javascript and JQuery? Pin
aarif moh shaikh1-Oct-15 0:32
professionalaarif moh shaikh1-Oct-15 0:32 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.