Note On Updates
For readers who've read the previous version of this article, you can skip to the Current Updates section where I discuss a cheap alternative solution to this problem which unfortunately doesn't work.
Introduction and Background
I did not know there was going to be a part 2 to this article, but I've been working on solving a specific problem and I made a breakthrough which led to success. I have been trying to:
Create the ability to retrieve any data, from any domain via AJAX without encountering an XSS (Cross-Site Scripting) error.
I solved this problem in Part 1 of this (now) series: http://www.codeproject.com/Articles/641756/GrabberProxy-Retrieve-any-web-content-via-AJAX-fro
I also wanted to create a solution that could be dropped into a directory and run without the confusing installation / configuration of a web server like IIS (Microsoft Internet Information Server) and the additional overhead of running ASP.Net.
I wanted it to __just work__. Don't we all?
Technologies Used To Solve The Problem
I think this article and solution are pretty cool because I had to use a few technologies working together to solve it. If you really take a close look you'll get a nice intro to the following tech:
- PHP - I haven't used PHP in about 10 years, but I figured out the code I needed in about 20 minutes.
- Mongoose Web Server
- Twitter BootstrapJS (for UI styling)
Mongoose Web Server : Totally Cool
I have used the Mongoose Web Server in the past so after finishing Part 1 of this series, I decided to take a look at what features it might offer to help me solve my additional challenge.
Note: You can learn much more about this great project at https://code.google.com/p/mongoose/
Complete Configured and Ready To Go
I was thoroughly amazed to find that I could create my entire re-usable project which allows another user to grab my project folder drop it on their drive and run the server immediately.
Everything is configured for you and ready to go. I hope you are as amazed at how well this works as I am. Now, I'll show you how it works. It's all very simple really, but getting to the solution took some thought.
Using the code
Get The Code Going
First I'll tell you how to get the thing going and then I'll explain the code that makes it work.
To run the project follow these simple steps:
- download the zip file and drop the contents onto your drive.
- After you do that you'll see a directory structure which looks like what you see in Figure 1 below.
- double-click the Mongoose.exe : that will start the web server up on port 53327 -- Note: you may see a warning from Windows since the web server is open the port (it will look similar to Figure 2 below).
- after Mongoose starts it will place an icon in the task bar tray
- Right-click that tray icon and a menu will appear (See Figure 3) .
- Click the Start browser menu item and the index.php page (from the weboort directory) will load in your default browser.
- That right there should be enough to amaze you. A drop in web server running PHP. Very cool.
Running GetDef.htm : Lookup Definition of Any English Word
Now that you've started your browser, we want to try the first script out which will allow you to look up any English word in the dictionary. To do that load the GetDef.htm page from the \webroot directory by typing the following URL in your browser (or clicking the following link):
http://localhost:53327/GetDef.htm (It makes it a lot easier to help you follow the steps since I know you have that web server running on that port.)
After clicking the URL you will see a page which looks like the following:
By the way, It is BootstrapJS whichmakes the button look nice and the text box to highlight as you move in and out of it.
Type a English term in the edit box and press the [Get Definition] button.
I used the term "rear view mirror"
$.get("http://localhost:53327/LocalLookup.php?defterm=" + lookupTerm,
// call jQuery's parseJSON method to parse the returned definition from its JSON source
var definition = $.parseJSON(data);
// select the jsonLoader div and put the definition in it.
$.get : jQuery AJAX
$.get is the jQuery method for making an AJAX call.
That method takes a URL target and a function which will run upon success.
In our case our URL target is the
LocalLookup.php -- our proxy to our remote URL. We pass a QueryString with a variable named
defterm and set it equal to the value the user entered for the English term.
LocalLookup.php: Our Proxy To the Outside World
Since AJAX calls can only call local resources, the LocalLookup.php is the proxy which will get the remote URL for us. Let's take a look at that PHP code. It's very simple and self-documenting.
$qstringTerm = urlencode($_REQUEST['defterm']);
definition = file_get_contents('http://newtonsaber.com/temp/deflookup.aspx?term=' . $qstringTerm);
Explanation of PHP Script
You see we are actually passing our lookup term along to our remote resource, waiting for the response and then echoing the response back to our local request. That is what makes the AJAX work on a remote target.
Now let's try one that allows you to retrieve any URL. Load the following page from the /webroot/ directory by typing the URL or clicking it:
You will see a page that is very similar to the GetDef.htm except, now you can type any URL and it will be loaded into the DIV box. Here's an example of the Wall Street Journal page (WSJ.com) being retrieved and loaded:
Note; As I mentioned in Part 1 of this article some pages do leak out of the DIV due to styles they have which affect the entire page.
Here's the slightly altered PHP code:
$qstringURL = $_REQUEST['URL'];
$definition = file_get_contents($qstringURL);
WARNING : CAVEAT
There is very little error checking in this so make sure you keep your URL in the correct format (http://<domain>) or you will get odd errors. I leave the error checking to you.
Hopefully you see, with these two examples how cool all of this is.
After I created the first entry in this series I brainstormed for any other (hopefully easier) ways of retrieving the remote content as if it were a local resource.
I implemented the code (using jQuery) in the following way.
The requested page (local.htm ) resides on your local server and is a basic HTML skeleton which contains a script at the top of the page which does the following:
// this does not work in the AJAX call, because it is as if two calls are made.
I was hoping the jQuery load method would wait on the entire return before continuing, but it does not.
What Actually Happens?
Actually, you get two requests. The first one is the jQuery.load request, but then it gets canceled and the location.href takes over and writes a new document into your web browser.
That is why this shortcut method does not work.
Benefits of GrabberProxy
I also wanted to touch upon the benefits of the GrabberProxy by making you think about how it supports a stronger pattern for developing web or cloud-based software.
Everything Is a Remote Factory
You see, with the GrabberProxy, everything becomes a remote factory which you can point at to get the data you want. This allows us to implement a layered solution which means you can easily create (multiple) views (on the client) side. Think about that for a while and I believe you'll see more benefit from this idea. It's some things to think about.
Part 2: August 27, 2013.
- Part 2: Article update August 28, 2013 - ideas on other possible solutions to the idea of retrieve remote content via AJAX. Also detailed reason why someone might want to use the GrabberProxy.