Challenge-Me: Javascript Including Engine - Part I

Foreword

First of all, with this article I initiate the Challenge-Me series of articles with the intent of covering problems that are hard or near impossible to solve.

And second, I’ve split the original large article into 3 smaller articles for 2 reasons: better readability and because every solution to this problem can make a stand alone article.

Part I

Introduction

JavaScript is a fantastic language. Its flexibility is unimaginable. But it has its drawbacks derived from its initial scope: to bring life to the static html pages.
One of these drawbacks is the missing functionality of including in js files other js files.
While a few years ago this was not such a big issue, nowadays it has become a big issue due to increase volume and complexity of JavaScript applications.

The Challenge

Primary Challenge:
To create an engine that will implement the JavaScript’s missing functionality of including in js files other js files.

Secondary Challenges:
- To work for online and offline applications
- To show if there are circular dependencies, and which files are involved
- Do not use eval

What would be the benefits? Easy development of large js applications that can be structured into related files, creating a logical dependency tree.

Secondary challenges are for establishing high quality standards (If something worth doing, then it is worth doing right).

 - Why not only online (since about 90% of the js code is used online)? - Because of the speed and simplicity of testing large amount of js code without passing through an http server.

 - What about circular dependencies? It isn’t pleasant at all to see that your application never finishes loading or you run out of memory due to circular dependencies, so determining that there is a circular dependency and stopping the loading process is a must. Also, pointing out which files form it would be even better by saving a lot of the developer’s time spent for looking for the needle in the haystack if the application has a complex tree of files and the circular dependency involves more than 10 of them.

 - Regarding the eval() function: it is very handy for executing js code (and this could be very useful for what we want to do), but there is a catch: it handles resources badly, and this makes it totally inappropriate for executing large portion of code as is in our case.

Solution 1

The Logic

Since the actual solution comes as a continuous improvement of previous solutions I’ll present the entire way to my latest solution.

My first solution about 4 years ago was to put in every file of the growing js application a comment ”//Requires files:” and when I want to use a file I look what files it requires, put them on a list, then for every file on the list I do the same thing until I reach independent files, creating in the process the order in that files need to be placed in the document’s head.

This system can work for 10 to 20 files maximum, after that it’s very hard to use it, and even before I started doing that, I knew that this is a temporary solution until I will have time to find a more appropriate solution.

I remembered this rudimentary solution because it actually describes the algorithm that is needed, and what remains to be done is to automate the process (and it’s not as easy as it seems)

So lets see how the implementation can be done: every file will have an include section formed by an $include function that will dynamically add to the HTML document’s head the script objects that will load the files specified by the function’s parameters.

And yet another problem: when the execution of a script starts, it goes all the way (it isn’t stopped because a new script object appeared before the current one, and does not load and execute that script first as we would have wanted)

This means that if you have in the FileA.js the statement var A = 5;
And in FileB.js $include(“FileA.js”); alert(A);

The result will be undefined and not 5 as expected!

So we can deduce that this solution works fine only if all files contain only function declarations (always) and object declarations if they're not related to objects from other files.

If you organize your code like C# code with no declarations outside classes then there is no problem.

Another aspect that must be taken under consideration is that the src attribute of the script tag contains the path to the script relative to the html document in which it is loaded; since the pages are logically placed into different folders, to include the same file we’ll have different src attributes.

But js files from a web project are usually all placed in one folder (possible with subfolders), so it would be better to create our including engine to include files relative to this folder rather than to the html document.


The Implementation

var IncludingEngine = {};
IncludingEngine.FindPath = function()
{
    var scripts = document.getElementsByTagName("script");
    var foundNo = 0;
    
    for (var i=0; i<scripts.length; i++)
    {
        if (scripts[i].src.match(this.FileName))
        { 
            this.Path = scripts[i].src.replace(this.FileName, "");
            foundNo ++;
        }
    }
    
    if (foundNo == 0)
        throw new Error("The name of this file isn't " + 
                    this.FileName + "!\r\nPlease change it back!");
    if (foundNo > 1)
        throw new Error("There are " + foundNo + " files with the name " +
                    this.FileName + "!\r\nThere can be only one!");
}



IncludingEngine.Init = function ()
{
    this.FileName = "IncludingEngine.js";
    this.Path = "";
    //the root path of the engine; all files included by this engine will be relative to this path
    this.FilesLoaded = new Array(); 
    //will store the files already loaded in order not to load (execute) a file many times
    
    this.FindPath();
}



function $include()
{
    for (var i = 0; i < arguments.length; i ++)
    {
        //check if the file was already loaded (this way we avoid circular dependencies also)
        if (!IncludingEngine.FilesLoaded[arguments[i]])
        {
            var script;
            script = document.createElement("script");
            script.src = IncludingEngine.Path + arguments[i];
            script.type = "text/javascript";
            document.getElementsByTagName("head")[0].appendChild(script);
            IncludingEngine.FilesLoaded[arguments[i]] = 1;
        }
    }
    
}

First we find the path where the IncludingEngine.js is, because next files will be specified relative to this path.

We’ll also have a hashtable that will keep track of the included files, so that they will not be executed twice or more.

The $include function (I used $ because it makes it look special) will add to the document’s head the script objects that will load the files specified by the arguments.

Notes:

 - You can also use document.write to add scripts to the document’s head (actually that was used in my original solution).
 - You have probably already seen this kind of approach because it’s easy to deduce, but it is far from meeting the challenge.
 - In part II I will present a very, very different approach.
 - the latest version of this implementation can be found at IncludingEngine.jsFramework.com

Here are all the parts of "Javascript Including Engine" article: Part I, Part II, Part III.
Also you may want to visit the IncludingEngine website for more info.