Click here to Skip to main content
Click here to Skip to main content
Go to top

Retrieve any Web Content via AJAX, from any URL (domain), in Any Browser, without XSS Error: Part 2

, 5 Jan 2014
Rate this:
Please Sign up or sign in to vote.
Drop this folder, start the Mongoose Web Server and boom, you can retrieve any web content via AJAX.

Note On Updates

For readers who've read the previous version of this article, you can skip to the Current Updates section where I discuss a cheap alternative solution to this problem which unfortunately doesn't work.

Introduction and Background

I did not know there was going to be a part 2 to this article, but I've been working on solving a specific problem and I made a breakthrough which led to success. I have been trying to:

Create the ability to retrieve any data, from any domain via AJAX without encountering an XSS (Cross-Site Scripting) error.

Problem Solved

I solved this problem in Part 1 of this (now) series: http://www.codeproject.com/Articles/641756/GrabberProxy-Retrieve-any-web-content-via-AJAX-fro

Additional Challenge

I also wanted to create a solution that could be dropped into a directory and run without the confusing installation / configuration of a web server like IIS (Microsoft Internet Information Server) and the additional overhead of running ASP.NET.

I wanted it to __just work__. Don't we all?

Technologies Used to Solve the Problem

I think this article and solution are pretty cool because I had to use a few technologies working together to solve it. If you really take a close look, you'll get a nice introduction to the following tech:

  1. JavaScript
  2. AJAX
  3. jQuery
  4. PHP - I haven't used PHP in about 10 years, but I figured out the code I needed in about 20 minutes.
  5. Mongoose Web Server
  6. Twitter BootstrapJS (for UI styling)
Mongoose Web Server: Totally Cool

I have used the Mongoose Web Server in the past so after finishing Part 1 of this series, I decided to take a look at what features it might offer to help me solve my additional challenge.

Note: You can learn much more about this great project here.

Complete Configured and Ready To Go

I was thoroughly amazed to find that I could create my entire re-usable project which allows another user to grab my project folder, drop it on their drive and run the server immediately.

Everything is configured for you and ready to go. I hope you are as amazed at how well this works as I am. Now, I'll show you how it works. It's all very simple really, but getting to the solution took some thought.

Get The Code Going

First, I'll tell you how to get the thing going and then I'll explain the code that makes it work.

To run the project, follow these simple steps:

  1. Download the zip file and drop the contents onto your drive.
  2. After you do that, you'll see a directory structure which looks like what you see in Figure 1 below.
  3. Double-click the Mongoose.exe: that will start the web server up on port 53327 -- Note: You may see a warning from Windows since the web server opens the port (it will look similar to Figure 2 below).
  4. After Mongoose starts, it will place an icon in the task bar tray.
  5. Right-click that tray icon and a menu will appear (See Figure 3) .
  6. Click the Start browser menu item and the index.php page (from the weboort directory) will load in your default browser.
  7. That right there should be enough to amaze you. A drop in web server running PHP. Very cool.
Figure 1

Figure 2

Figure 3

Running GetDef.htm: Lookup Definition of Any English Word

Now that you've started your browser, we want to try the first script out which will allow you to look up any English word in the dictionary. To do that, load the GetDef.htm page from the \webroot directory by typing the following URL in your browser (or clicking the following link):

http://localhost:53327/GetDef.htm (It makes it a lot easier to help you follow the steps since I know you have that web server running on that port.)

After clicking the URL, you will see a page which looks like the following:

By the way, it is BootstrapJS which makes the button look nice and the text box to highlight as you move in and out of it.

Type a English term in the edit box and press the [Get Definition] button.

I used the term "rear view mirror"

Here's how it works. Inside the GetDef.htm, you'll find a JavaScript method named load_content(). That method contains the following code:

$.get("http://localhost:53327/LocalLookup.php?defterm=" + lookupTerm, 
    function (data){     
      // call jQuery's parseJSON method to parse the returned definition from its JSON source
      var definition = $.parseJSON(data);
      // select the jsonLoader div and put the definition in it.
            $("#jsonLoader").text(definition.term);
            }
      );

$.get : jQuery AJAX

$.get is the jQuery method for making an AJAX call.

That method takes a URL target and a function which will run upon success.

In our case, our URL target is the LocalLookup.php -- our proxy to our remote URL. We pass a QueryString with a variable named defterm and set it equal to the value the user entered for the English term.

LocalLookup.php: Our Proxy to the Outside World

Since AJAX calls can only call local resources, the LocalLookup.php is the proxy which will get the remote URL for us. Let's take a look at that PHP code. It's very simple and self-documenting.

<?php
 // 1. get the passed in term (off the querystring) that we want to retrieve the definition for
 // 2. call urlencode (php method) in case odd chars are in the querystring variable.
 $qstringTerm =  urlencode($_REQUEST['defterm']);
 //echo $qstring; // this line for testing purposes
 // 1. retrieve the data from the remote server
 // 2. store it in a local php variable.
 definition = file_get_contents('http://newtonsaber.com/temp/deflookup.aspx?term=' . $qstringTerm);
 // 1. write out the returned data to our local web request.
 echo $definition;
?>

Explanation of PHP Script

You see, we are actually passing our lookup term along to our remote resource, waiting for the response and then echoing the response back to our local request. That is what makes the AJAX work on a remote target.

Now, let's try one that allows you to retrieve any URL. Load the following page from the /webroot/ directory by typing the URL or clicking it:

You will see a page that is very similar to the GetDef.htm except, now you can type any URL and it will be loaded into the DIV box. Here's an example of the Wall Street Journal page (WSJ.com) being retrieved and loaded:

Note: As I mentioned in Part 1 of this article, some pages do leak out of the DIV due to styles they have which affect the entire page.

Here's the slightly altered PHP code:

<?php
 // 1. get the passed in term (off the querystring) that we want to retrieve the definition for
 $qstringURL =  $_REQUEST['URL'];
 //echo $qstring; // for testing purposes
 // 1. retrieve the data from the remote server
 // 2. store it in a local php variable.
 $definition = file_get_contents($qstringURL);
 // 1. write out the returned data to our local web request.
 echo $definition;
?>

WARNING: CAVEAT

There is very little error checking in this, so make sure you keep your URL in the correct format (http://<domain>) or you will get odd errors. I leave the error checking to you.

Hopefully you see, with these two examples how cool all of this is.

Current Updates

After I created the first entry in this series, I brainstormed for any other (hopefully easier) ways of retrieving the remote content as if it were a local resource.

At one point I thought, "Hey, why can't I just retrieve a local page which redirects (via JavaScript location.href) to the remote resource?"

I implemented the code (using jQuery) in the following way:

// attempt to load the local.htm into the "#json2" <DIV>   
$("#json2").load("http://localhost:53327/local.htm");  

The requested page (local.htm) resides on your local server and is a basic HTML skeleton which contains a script at the top of the page which does the following:

 <head> 
     <script> 
     // this does not work in the AJAX call, because it is as if two calls are made.
      location.href="http://newtonsaber.com/temp/deflookup.aspx?term=forward";
   </script>  

I was hoping the jQuery load method would wait on the entire return before continuing, but it does not.

What Actually Happens?

Actually, you get two requests. The first one is the jQuery.load request, but then it gets canceled and the location.href takes over and writes a new document into your web browser.

That is why this shortcut method does not work.

Benefits of GrabberProxy

I also wanted to touch upon the benefits of the GrabberProxy by making you think about how it supports a stronger pattern for developing web or cloud-based software.

Everything Is a Remote Factory

You see, with the GrabberProxy, everything becomes a remote factory which you can point at to get the data you want. This allows us to implement a layered solution which means you can easily create (multiple) views (on the client) side. Think about that for a while and I believe you'll see more benefit from this idea. It's some things to think about.

History

  • Part 2: August 27, 2013
  • Part 2: Article update August 28, 2013
    • Ideas on other possible solutions to the idea of retrieve remote content via AJAX
    • Also detailed reason why someone might want to use the GrabberProxy.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

newton.saber
Architect
United States United States
My newest book is Learn Python, Think Python (amazon link opens in new window/tab)
 
My previous book is Object-Oriented JavaScript (See it at Amazon.com)
 
My book, Learn JavaScript - amazon.com link is available at Amazon.
 
My upcoming book, Learn AngularJS - Think AngularJS, will be releasing later in 2014.
 
You can learn more about me and my other books, at, NewtonSaber.com
Follow on   Twitter

Comments and Discussions

 
QuestionGoogle Bookmarks access Pinmembermarc borgers6-Jan-14 9:16 
QuestionSimple - Just Add headers to response PinmemberNick Kowalewicz6-Jan-14 5:27 
AnswerRe: Simple - Just Add headers to response Pinmembernewton.saber8-Jan-14 9:43 
QuestionWild goose chase PinmemberMember 90401373-Sep-13 5:31 
AnswerRe: Wild goose chase Pinmembernewton.saber3-Sep-13 7:17 
QuestionBy the way PinmemberPeter Hawke27-Aug-13 13:20 
AnswerRe: By the way Pinmembernewton.saber8-Jan-14 7:47 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web04 | 2.8.140926.1 | Last Updated 5 Jan 2014
Article Copyright 2014 by newton.saber
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid