TOOL

• Download source - 2.50 MB
• Download demo - 2.23 MB
• Download TOOLForge - 1.38 MB

Big News in This Release

TOOLForge, the IDE for TOOL, has been enhanced to allow visual creation of XML Forms. This enhancement will save anyone interested in using XMLForms with TOOL the pain of creating form definition files by hand. There is more work to be done here with the more advanced XMLForm control types, but I'm wagering the existing feature set will address most user's needs. However, this program can only be released in non-source form. There is just too much commercial library code in this project to have it any other way. See below for more information on how to work with TOOLForge to debug TOOL scripts.

Introduction

TOOL (Tiny Object Oriented Language) is an easily-embedded, object-oriented, C++-like-language interpreter. The language, and indeed a significant part of the core of the TOOL engine, is based on the BOB project, a work that was originally developed by David Betz covered in previously published issues of Dr. Dobb's Journal.

The original language interpreter was implemented entirely in K and R style C. The TOOL interpreter uses an object-oriented design of the original BOB interpreter, and also includes countless dozens of other enhancements to the original project. The object oriented redesign was accomplished by packaging the functionality of the original project into a small set of inter-operable C++ classes. These classes were then aggregated into a single container class that acts as a facade over the entire system.

One of the primary purposes for the redesign of the original interpreter was to bring the code up to “more modern standards”. However, the key benefit of packaging the interpreter into a single container class is that any application can easily create an "interpreter context" simply by invoking a single class. Applications can also run multiple "stand-alone" interpreter sessions simultaneously simply by having multiple instances of the wrapper-class in scope at once.

The purpose of this article is to introduce the TOOL interpreter and language from the perspective of a person who has a desire to include a scripting solution as part of his project. To this end, the following topics will be discussed:

• The design of the C++ classes that make up the TOOL core.
• How to integrate TOOL into your application.
• How to extend TOOL for your custom requirements.
• How to program in TOOL.
• An examination of the TOOL classes framework, which is a set of classes written in the TOOL language and which provides a set of tested and ready-to-use TOOL classes and that also illustrates the entire set of TOOL intrinsic functions (the TOOL API).

These days, interest in interpreters seems to be at an all time high, and there are many interpreters and technologies to choose from. This being the case, why should anyone consider the TOOL engine as part of a project’s solution? A short list of considerations follows below, which have driven me to include TOOL as part of my own products.

• Lack of Dependencies. When considering the inclusion of a scripting system that is not an integral component of an Operating System installation, in order for your product to run, it is dependant on the correct installation and configuration of a third party product. This kind of dependency is a potential “break point” and therefore is one that I prefer to avoid if at all possible.
• Easily Embedded. Truly embedding a third-party scripting-engine into your solution is not always an easy prospect; especially if part of your goal is to exchange data between your application context and the scripting context (in effect incorporating the interpreter to the degree that it is an integral part of the project, and not just “stuck on the side of it”). Often times, data must be “marshaled between” the two contexts using APIs or syntax which is not always that easily understood or straightforward. So, to address this, one of my goals with TOOL was to make the exchange of data between the two contexts as simple as possible. To this end, I have included an application context object into the design of TOOL. This application context allows for a simple, loosely coupled, yet effective, means of exchanging data between the application environment and the script context.
• Easily Extended. There are certain occasions when part of the reason for embedding a scripting system into a project is to expose specific portions of an application’s functions into the scripted context (in effect making your programs programmable). With most interpreters available today, the prospect of extending them can be a daunting one. In examining the source for some of the more common interpreters available at this time, I found dozens and dozens (in some cases hundreds) of files whose organization is not readily apparent to the casual observer. This being the case, it can be extremely difficult as an “outsider” to understand where and what to edit to extend the functionality of these engines. And if your project is “typical”, it most likely has tight deadlines that will not afford you the chance to take the time required to understand the internal organization of these projects to the point of being able to modify their run-times. And, even if you succeed in that task, now you are potentially the proud owner of a customized, “one-off” version of that distribution that brings its own set of maintenance challenges and may require constant validation of your extension with all future versions of the selected engine. In direct contrast, TOOL remains an easily pliable engine and is very easily extended. The techniques to extend TOOL will be thoroughly discussed in this article.
• No Licensing Hassles. Even so called “open source” interpreter engines can be encumbered with additional license fees and other costs if they are included as part of a commercial product. For “low volume” products, these kinds of costs can be prohibitive. And between you, me, and the water cooler it seems somehow wrong that anyone feels they have the right to charge monies for something that is “supposed to be free” so I tend to shy away from these situations on principle.

The design of the C++ classes that make up the TOOL

As was mentioned, TOOL is comprised of a small number of core classes. The bulk of the files included in this project are extensions to that basic engine. Since there is not enough space to cover all these classes in great depth, I’ll have to leave the extended analysis to the reader, and therefore my goal will be to attempt to impart an understanding of the core classes within the TOOL engine. I hope to give you enough information to allow you to get your bearings in the project; but down in the guts understanding is best gained by wrestling the project to the mat.

Listing 1. Simple example of how to drive the TOOL engine. What we are shown here is a data exchange variable called “oArgs” that will be used to share data between the application and the scripting engine. “oArgs” is then passed into an instance of a VMFacade object which is the container for the entire TOOL scripting engine. All that is needed to run a script then is issue a call to compile and to execute on the facade object. The script can return variables back to the application through the oArgs container.

int main(int argc, char* argv[])
{
  SCRIPT_ARGUMENTS oArgs;
 
  oArgs.insert( SCRIPT_ARGUMENTS::value_type( std::string( "TestVector"    ) , 
                                              new CAppContextVariant( "On" ) ) );
  oArgs.insert( SCRIPT_ARGUMENTS::value_type( std::string( "TestStack"     ) , 
                                              new CAppContextVariant( "On" ) ) );
  oArgs.insert( SCRIPT_ARGUMENTS::value_type( std::string( "TestQueue"     ) , 
                                              new CAppContextVariant( "On" ) ) );
  oArgs.insert( SCRIPT_ARGUMENTS::value_type( std::string( "TestMap"       ) , 
                                              new CAppContextVariant( "On" ) ) );
  oArgs.insert( SCRIPT_ARGUMENTS::value_type( std::string( "TestByteArray" ) , 
                                              new CAppContextVariant( "On" ) ) );
  oArgs.insert( SCRIPT_ARGUMENTS::value_type( std::string( "TestString"    ) , 
                                              new CAppContextVariant( "On" ) ) );
  
  VMFacade* poScript = new VMFacade();
  poScript->SetVar( &oArgs );
 
  poScript->CompileFile( "C:\\Projects\\ScriptEngineTester\\CoreClasses.tool" );
  poScript->Execute( "TestAll" );
 
  return( 0 );
}

The first class that the users of TOOL are going to encounter is the VMFacade class (declared in VMFacade.h and implemented in VMFacade.cpp). This class is the single wrapper class that aggregates all components of the entire TOOL system; and it will be the one that you’ll want to instantiate in your projects to get instances of the scripting engine. You can see from the short example in Listing #1 just how simple this class is to use. After you create an instance of this class, you simply need to compile a file (or a text block if you’d prefer) with a call to CompileFile() or CompileText().

If you need to share variables from your program with the script engine, or if you need to retrieve results from the execution of a script, you can easily accomplish this by placing parameters/variables into an instance of a SCRIPT_ARGUMENTS class. This STL derived map is set up to store all the variable types that can be exchanged between the application and the scripting environment. All variables in the application context are stored under a string key, and the value type is a variant type of object that can be one of the following types: String, Long, Boolean, Double, DWORD, DateTime, ByteArrray, and a NULL object type. The CappContextVariant class is declared in the file: VMCoreVirtualMachine.h.

The script can pull values out of the container by name while it is running using a call to the TOOL API function: GetAppEnvField(). The script can also use the context to return variables created in the script’s execution back to the hosting application using the TOOL API function: SetAppEnvField().

In order to facilitate communication between the hosting process and the scripting environment then: simply declare an instance of a SCRIPT_ARGUMENTS class, place variables into it, and then share that context with the VMFacade class through a call to the SetVar() on the VMFacade object. After the script has completed, the hosting application can examine this map for any results sent back to the application from the scripting engine.

With all this setup being accomplished, the next thing to do is to invoke the scripting engine with a call to the Execute() method on the facade. One nice thing about the TOOL engine is that you can specify the entry point for the script execution simply by declaring the entry point in the call to Execute(). This capability can be handy in the case where you may have multiple script programs included in a single script file and you want to “select which one to run” based on logic in the main application. Another way this feature can be used, is to define the “standard interface” for all your scripts and then simply define all scripts so that they have the same entry point(s).

I’ve used both approaches in my projects. The first approach is handy if you would like to have multiple entry points (say for example, a kind of a macro language system available to your project) in a single script file. The second approach is handy if you want to describe a set of standalone script files that all will be invoked by the hosting program in the same fashion as “scripted objects”. In my case, I’ve employed an Init/Run semantic quite successfully wherein all scripts must have Init() and Run() functions in them that do the work specific to that script.

Peeking behind the VMFacade class, the next major class encountered is the VMCore class (declared in VMCore.h and implemented in VMCore.cpp). This class is also a facade class over the rest of the system, but does more of the detailed operations required to stitch the TOOL runtime altogether. As such, this class aggregates all the other major components of the TOOL engine. The VMCore class drives the actual compilation and execution operations of the TOOL system. It also initiates all the other major components in TOOL. As part of constructing the TOOL engine, this class provides all the major components with pointer references to the other major components. Finally, this class also catches all exceptions thrown from the TOOL engine. Exceptions can be thrown during compilation if there are syntax errors detected in the script. Exceptions can also be thrown during script execution if one of the integrity checks performed by the TOOL engine fails. Finally, exceptions can be initiated by the script itself through a call to the TOOL API function: Throw().

Part of the reason for the deep level of “cross connection” between the major TOOL components is historical, in that the original design and implementation of TOOL was based on the BOB engine. Even though the major objects were designed based on broad functional areas, there were some areas where the separation could not be performed completely (just the same as in one cannot completely separate memory from the CPU in a “real” computer system). Even so, I’m hopeful that you will find my repackaging of the BOB project in to this particular set of classes is sensible and cohesive.

Looking more closely at what is contained in the VMCore, you can see that “at the bottom” of the TOOL system is a VMCoreVirtualChip (declared in VMCoreVirtualChip.h). This class is the “CPU” for the TOOL system. In this class, you will find such things as a vector for the interpreted code, the program counter, the stack and frame pointers for the machine, as well as the main heap storage areas.

At the next level out, there is the VMCoreVirtualMachine class (declared in VMCoreVirtualMachine.h) which fills the role of managing the TOOL heap and acts as the main memory manager class for the TOOL system. That being the case, this class will also work with the VMCoreVirtualChip to define and manage a simple dictionary of all variables and literals contained within a script program. Derived from the VMCoreVirtualMachine class is the VMCoreVirtualOpSys class. This class is where all the intrinsic functions (essentially the entire API to the TOOL engine) are defined and implemented. The VMCoreVirtualOpSys is just that, it is the “operating system” that is offered by the TOOL engine and is also the class where extensions to the engine are located. When you would like to add a new intrinsic function to your version of TOOL, you will likely implement it in the VMCoreVirtualOpSys class.

Still continuing on our journey out from the TOOL core, you’ll see that there is a VMCoreCompiler class which is inherited from the VMCoreParserScanner class. The parser-scanner object is the one that will scan and tokenize the script text, while the compiler will drive the scanner as well as operate against the scanner-parser output to assemble the array of op-codes that will be executed when the script is run.

At the highest level of TOOL, is the VMCoreInterpreter class that is responsible for executing the vector of compiled code that was produced by the VMCoreCompiler. The interpreter acts as a large switching-scanner. It scans the op-codes produced by the compiler, and then based on the op-code detected, will perform a specific set of actions.

How does the TOOL Engine work?

TOOL is implemented as a hybrid of a compiler and an interpreter (an inter-piler?). When a function is defined, it is compiled into instructions for a stack-oriented, byte-code machine. When the function is invoked, those byte-code instructions are interpreted. The advantage of this approach over a straight interpreter is that syntax analysis is done only once, at compile time. This speeds up function execution considerably and opens up the possibility of building a run-time-only system that doesn't include the compiler at all. In fact, in some of my scripts (which are not overly complex, but are not all that simple either), I’ve gotten over 150 complete executions a second through the system. Your mileage may vary of course, but TOOL, while nowhere near as fast as compiled code is no slouch either.

The virtual machine that executes the byte-codes generated by the TOOL compiler has a set of registers, a stack, and a heap. These constructs are contained in the VMVirtualChip class. All instructions get their arguments from and return their results to the stack. Literals are stored in the code object itself and are referred to by offset. Branch instructions test the value on the top of the stack (without popping the stack) and branch accordingly. Function arguments are passed on the stack, and function values are returned on the top of the stack. We’ll examine this concept again in more depth when the techniques for extending the TOOL engine are discussed.

In TOOL script classes, all member functions are virtual. This means that when a member function is invoked, the interpreter must determine which implementation of the member function to invoke. This is done by the SEND op-code, which uses a selector from the stack (actually, just a string containing the name of the member function) with the method dictionary associated with the object's class to determine which member function to use. If the lookup fails, the dictionary from the base class is examined. This continues, following the base-class chain until either a member function is found or there is no base class. If a member function is found to correspond to the selector, it replaces the selector on the stack and control is transferred to the member function, just as it would have been for a regular function. If no member function is found, an error is reported and the interpreter aborts. One extension I’ve added to this mechanism is a “cast call” with a syntax of “*->”. What this call does is to allow the script-writer to “cast” the class to a specific one of its base classes (named on the left hand side of the operator) and then to invoke the selector named on the right hand side of the operator. Several examples of this operator in action can be found in the script file “coreclasses.tool”. This operator serves two primary purposes. One is that it makes the call to a base class explicit, which is especially useful if multiple parent classes share the same function name, and also serves to comment to the script code as to what the author’s specific intentions are with regards to the execution of that class method.

TOOL variable types have been extended to support the following basic data types: Longs, Doubles, Bytes, Strings, Tokenizers, ByteArrays, Vectors, Classes, DateTimes, Queues, Maps, Stacks, WaitObjects, NT Kernel-handles, DWORDs, NT-File handles, File-Find handles, zib-files, SQLite databases, ODBC connections, and nil. Internally, the interpreter uses four more types: classes, compiled bytecode functions, built-in function headers, and variables. Wherever a value can be stored, a tag indicates the type of value presently stored there. This is the “v_type” field on the VMVariant class, declared in “VMCoreGlobal.h”.

Objects, vectors, and bytecode objects are all represented by an array of value structures. In the case of bytecode objects, the first element in the vector is a pointer to the string of bytecodes for the function, and the rest are the literals referred to by the bytecode instructions. Class objects are vectors, where the first element is a pointer to the class object and the remaining elements are the values of the nonstatic member variables for the object. Built-in functions are just pointers to the C functions that implement the built-in function. Variables are pointers to dictionary entries for the variable. There is a dictionary for global symbols and one for classes. Each class also has a dictionary for data members and member functions.

In addition to the stack, TOOL uses a heap to store objects, vectors, and strings. The current implementation of TOOL uses the C heap and the C functions malloc and free to manage heap space and uses a compacting memory manager. However, I must admit that the heap management in this version of TOOL is not completely garbage collected. This is due to the evolving nature of the engine and the fact that some of the new data types are actually C++ classes managed by the VMVariant object. This is why the TOOL API call “FreeObject” has crept into the TOOL API. This function is included so that the heap allocated C++ class can be cleaned up when the script is done using it. I recognize fully that this is a weakness in the current implementation, and one that I do plan to address in a newer version of the engine. But, for now, I’ve gotten used to the “evil of it” and have not therefore made a priority out of addressing the shortcoming of the current implementation.

Operator Overloading

In the TOOL interpreter, an operator can have several different meanings based on the LHS/RHS values for the operator. Consider the case of the array index operator “[]”. In this case, the operator can have several different interpretations. It should perform one action if the LHS is a Map object, and a complete different operation if the LHS is a Vector object. A similar approach is needed for mathematical operations, such as “+” or “-” where if the operation is performed on variables of different types, then one of them must be “promoted” to the other variable type. The interested reader is encouraged to study the VMInterpreter’s handling of the OP_ADD and OP_VREF op-codes for more details on how to extend operator overloading. This is an important topic to have a grasp of if you wish to add more variable types to the TOOL engine and wish to have specific behaviors for different “standard operators”.

Function Overloading

In the TOOL API, intrinsic functions can be “overloaded” in that they can be coded to accept different parameter counts and types. One example of this (and there are many others) is the HandlerNewDateTime() method in the VMCoreVirtualOpSys class. This handler is coded to accept either 1 or 6 arguments. If one argument is passed, then its type is examined to verify that it is a DT_DATETIME type. If six arguments are passed, then each of these arguments are examined to verify that they are DT_INTEGER types.

The most extreme implementation of this feature is found in the function HandlerStringFormat() which is an implementation of an elliptical function; meaning that it can accept any number and type of arguments, which is what you’d expect from a sprintf type of function.

This flexibility in defining functions is a natural outcome of the way the interpreter pushes function arguments on to the stack. The interpreter has no information regarding the number and type of arguments needed by any intrinsic function. It will simply push all arguments found for the call on to the stack. This feature enables any intrinsic function to be coded to accept all types of arguments from the interpreter.

There is a “flipside” to this flexibility however, and that is that all intrinsic functions should perform sanity checks on all arguments and argument counts to verify that what it is about to operate on is what is expected in the function.

All Ashore, That’s Going Ashore

Well that’s about a whirlwind of a tour that I can provide without getting completely bogged down in all the details that bring TOOL together. Hopefully, I have provided enough of an introduction to the lay of the land in the TOOL engine so that you can have some solid ground to stand on while you analyze the inner workings of TOOL. However, take comfort in knowing that you can easily use and extend TOOL without knowing how all the inside pieces and parts work. I just thought I’d show your around the project a bit, so that when you go to kick the tires on this beasty yourself, you’ll know where to find the fenders.

Programming in TOOL

The examples below present a simple example program; a function for computing factorials written using the TOOL language. Note: Readers are also encouraged to review the sample scripts included with this distribution.

// factorial program

//

Factorial( iValue )
{
  return( ( iValue == 1 ) ? 1 : ( iValue * Factorial( iValue - 1 ) ) ); 
}

Run( ;iLoop )
{
  iLoop = Long( 0 );
  for ( iLoop = 1; iLoop < 10 iLoop++ )
  {
    Echo( StringFormat( "Factorial for % is %", iLoop, Factorial( iLoop ) ) );
  }
}

You can see that this program looks a lot like its C counterpart. The only noticeable difference is the lack of a declaration for the type of the parameter iValue in the arguments passed to the Factorial function and for the return type of this same function. This is due to the fact that variable types do not need to be declared in TOOL; although it is still considered "good style" to declare and type all variables for clarity and maintenance purposes. A side effect of this is that any variable can take on a value of any type.

Other points worth noticing in this first sample are the variables declared in the Run function's formal parameter list. Here, you can see that the variable iLoop follows a semi-colon. This syntax is used to define the iLoop variable as being in "local scope" to the Run function. By default all variables in TOOL scripts adopt global scope unless they are declared using this specialized local scope syntax.

Important note: Even though TOOL is a weakly typed language from the script writer's perspective, the TOOL interpreter will type-check the type of a variable passed into its routines. If the variable is not of the right type, the interpreter will halt the execution of the script.

Again, what you should notice primarily is how much the example program above looks a lot like a similar program written in C. Also, notice that the program uses the StringFormat function to create a formatted string, and the Echo function to display the results. These functions will be explained in greater detail later in this document. The StringFormat function in TOOL prints each of its arguments in succession into an output string. It is capable of printing arguments of any type and automatically formats them appropriately.

In addition to supporting C-like expressions and control constructs, TOOL also supports C++-like classes. Again, since TOOL is a typeless language, the syntax for class definitions is somewhat different from C++, but it is similar enough that it should be easy to move from one to the other.

The next example shows a simple class definition that defines a class called Foo with members m_A and m_B and a static member called m_Last as well as a static member function GetLast. Unlike C++, it is not necessary to declare all member functions within the class definition; only the static member functions need be declared. It is necessary, however, to declare all data members in the class definition.

// class declaration

//

class Foo
{
  m_A;
  m_B;
 
  static m_Last;
  static GetLast();
}
 
Foo::Foo( AValue, BValue )
{
  m_A    = AValue;
  m_B    = BValue;
  m_Last = this;
  return( this );
}
 
Foo::GetLast()
{
  return( m_Last );
}

As in C++, new objects of a class are initialized using a constructor function, which has the same name as the class itself. In this example, the constructor takes two arguments, which are the initial values for the member variables m_A and m_B. It also remembers the last object instance of the type Foo created in the static member variable m_Last. Lastly, the constructor returns the new object. For those of you not familiar with C++, the variable this refers to the object for which the member function is being called. It is an implicit parameter passed to every non-static member function. In this case, it is the new object just created.

In TOOL, all class data members are implicitly protected. The only way to access or modify the value of a member variable is through a member function. If you need to access a member variable outside a member function, you must provide access to member functions to do this.

We'll continue to refine the Foo class below to show how to set the value of a member variable. Finally, we'll show a member function that displays the numbers between m_A and m_B for any object of the Foo class type, and a Run function that creates some objects and manipulates them. The new operator creates a new object of the class whose name follows it. The expressions in parentheses after the class name are the arguments to be passed to the constructor function.

// continuing to define the Foo class

//

Foo::GetAValue()
{
  return( m_A );
}
 
Foo::GetBValue()
{
  return( m_B );
}
 
Foo::SetAvalue( NewAValue )
{
  m_A = NewAValue;
}
 
Foo::SetBValue( NewBValue )
{
  m_B = NewBValue;
}
 
Foo::GetSpan()
{
  return( m_B - m_A );
}
 
Run( ;poFoo1, poFoo2 )
{
  poFoo1 = new Foo(  1 , 2 );
  poFoo2 = new Foo( 11, 22 );
 
  Echo( "Foo1 Span Is: " + poFoo1->GetSpan() );
  Echo( "Foo2 Span Is: " + poFoo2->GetSpan() );  
}

TOOL also supports an inheritance model similar to the Java language, in that it allows one class to be derived from another. The derived class will inherit the behavior of the base class and possibly add some behavior of its own. TOOL only supports single inheritance; therefore, each class can have at most one base class. The next code example defines a class Bar derived from the base class Foo defined earlier.

The class Bar will have member variables m_A and m_B inherited from its parent Foo as well as the additional member variable m_C. The constructor for Bar needs to initialize this new member variable and do the initialization normally done for objects of class Foo. The example below illustrates how this is done:

// class derivation in TOOL

//

class Bar : Foo
{
  m_C;
}
 
Bar::Bar( AValue, BValue, CValue )
{
  Foo*->Foo( AValue, BValue );
 
  m_C = CValue;
  return( this );
}

This example illustrates another difference between TOOL and C++ objects. In C++, constructor functions cannot be called to initialize already existing objects. This is allowed in TOOL, however, so the Foo constructor can be used to do the common initialization of the Foo and Bar classes. Careful readers may notice the *-> operation used in the Bar constructor. In TOOL, this is called the "cast-call operator". What this operator does is to cast "this" to the base-class-type named on the left-hand-side of the operator, and then invoke the function on the right-hand-side of the operator in the named base class. This syntax is especially useful for function overloading in long class hierarchies because this can be explicitly cast to the proper parent class type before the invocation of the function call.

A Quick Mention of Programming Style

Since TOOL is a weakly typed language, meaning that the type of variables is not required to be specified, I found it useful to use some means of classifying the data-type of a script variable (for the purpose of making the code more understandable to others). For this reason, a simplified form of Hungarian Notation is used in all script samples included with this article. Following are samples of the notation used:

Data Type      Hungarian "Marker"         Example
String         s                          sMyString
Number         i                          iValue
Object         o                          oValue

I don’t mean to promote Hungarian Notation in any specific way, as I know that just as many folks don’t like it as I do, I simply find it to be a useful when striving to understand large projects.

More on a Suggested Programming Style

TOOL is a weakly typed language meaning that the type of variables does not have to be specified. Therefore, some means of classifying the data stored in a variable (for the purpose of making the code more understandable to others) would be of great benefit. Therefore, a simplified form of Hungarian Notation is encouraged in all TOOL program scripts. Following are samples of the notation used:

Data Type	Suggested Marker	Example
String	s	sText
Long	l or i	lNumber
Double	dbl	dblValue
Byte	b	bFlag
Stack	ostk	ostkOfItems
Vector	ovec	ovecOfItems
Map	omap	omapOfItems
DateTime	dt	dtNow
Queue	oqueue	oqueueOfItems
Handle	h	hFile
Color	clr	clrBlack
DWORD	dw	dwValue
Database Connection	oDB	oDBSource
Calculator	ocalc	ocalcFormula
Class Instance	po	poFoo

Creating and Allocating TOOL Variables

There are several types of variables in TOOL. They are all listed below along with the TOOL call for creating the variable. Readers familiar with C++/C#/Java can think of these API calls as calls to a constructor for a variable.

Vector();           // accepts no arguments


String( 255 );      // reserve 255 characters for length

String( "Hello" );  // create and assign value to string


Queue();            // accepts no arguments

 
Stack();            // accepts no arguments

 
Map();              // accepts no arguments

 
Tokenizer( sToTokenize, sDelimiter ); 
 
ByteArray();              // create byte array with default parameters

 
ByteArray( Long( 25 ) );  // create byte array with 25 elements

 
ByteArray( Long( 25 ), Long( 5 ) ); // create byte array with 25

                                    // elements and grow factor of 5

                                    // elements

 
Long();                   // create a long with value zero

Long( 50 );               // assign value to a long variable

 
Handle( 0 );              // create a handle type MUST have init value

 
DateTime();               // create a datetime value equal to 'now'

DateTime( dtOther );      // copy construct a date time variable

DateTime( 2004, 01, 01, 12, 15, 0 );  // create date time with init

                                      // value of 01/01/2004 12:15:00

 
Color();                 // create color with RGB 255,255,255

Color( 0, 0, 0, );       // create color with RGB 0,0,0

Color( clrOther );       // copy construct a color variable

 
DWORD();                 // create a DWORD with value of zero

DWORD( dwOther );        // copy construct a DWORD variable

DWORD( Long( 5 ) );      // construct a DWORD from a number value

 
Database();              // create an ODBC database variable

 
Calculator();            // create an calculator / equation handler

 
Double();                // create a double with a value of 0.00

Double( Long( 6 ) );     // create a double from a long value

Double( dblOther );      // copy construct a double variable

Double( "123.45" );      // create a double from a string value

 
Byte();                  // create a byte initialized to zero/false

Byte( bOther );          // copy construct a byte type variable

Byte( Long( 10 ) );      // set byte variable from number 

                         // NOTE: limit of 0 - 255 for the number

 
Page();                  // report page object

Table();                 // table object in a report page

Color();                 // color object in a report page

 
Database();              // odbc database connection object

 
Calculator();            // function evaluation object

 
NullValue();             // TOOL NULL Value

 
MiniDatabase();          // SQL LITE Wrapper Object

 
UnZipper();              // zlib Wrapper Object for unzip operations

 
ZipMaker();              // zkib Wrapper Object for zip operations

 
FileBuilder();           // string collection object for creating file output

 
DelimitedFile();         // wrapper object for delimited file I/O

 
ExcelExporter();         // Wrapper for an excel workbook export object

TOOL Run Time Type Checking

Since TOOL is a relatively weakly-typed language, as was explained previously in another topic; while at the same time the TOOL interpreter validates that all arguments passed to the TOOL API functions are of the proper type, TOOL offers an entire suite of RTTI (Run Time Type Identification) functions for script writers in order to assist them in writing solid and reliable script programs.

Use of these functions allows the creation of more robust scripts due to the argument validations that can be performed on all variables. An additional use of these functions may be to write scripts that will conditionally branch based on the type of variable being tested. There will be examples of both types of usage later in this document. The complete list of RTTI functions is listed below:

IsNull( oVarToTest );          // is the variable NULL?

IsClass( oVarToTest );         // is the variable a TOOL script class?

IsVector( oVarToTest );        // is the variable a vector type?

IsNumber( oVarToTest );        // is the variable a long number?

IsString( oVarToTest );        // is the variable a string?

IsFile( oVarToTest );          // is the variable a file?

IsKernelObject( oVarToTest );  // is the variable an NT kernel object?

IsHandle( oVarToTest );        // is the variable an NT style handle?

IsDateTime( oVarToTest );      // is the variable a TOOL date time type?

IsDWord( oVarToTest );         // is the variable a DWORD?

IsNTHandle( oVarToTest );      // is the variable an NT handle?

IsFindHandle( oVarToTest );    // is the variable a "file find handle"?

IsQueue( oVarToTest );         // is the variable a TOOL queue type?

IsStack( oVarToTest );         // is the variable a TOOL stack type?

IsHashTable( oVarToTest );     // is the variable a TOOL map type?

IsConstant( oVarToTest );      // is the variable declared as const?

IsConstCast( oVarToTest );     // is the variable cast as const?

IsDouble( oVarToTest );        // is the variable a double?

IsByte( oVarToTest );          // is the variable a byte?

IsByteArray( oVarToTest );     // is the variable a byte array type?

IsDatabase( oVarToTest );      // is the variable a database connection?

IsReportPage( oVarToTest );    // is the variable a report page?

IsPageRegion( oVarToTest );    // is the variable a report page region?

IsPageTable( oVarToTest );     // is the variable a report page table?

IsColor( oVarToTest );         // is the variable a report page color?

IsMiniDatabase( oVarToTest );  // is the variable a sqlite database wrapper?

IsUnZipper( oVarToTest );      // is the variable a zlib unzipper wrapper?

IsZipMaker( oVarToTest );      // is the variable a zlib zipper wrapper?

IsMiniRowSet( oVarToTest );    // is the variable a SQLite rowset wrapper?

IsFileBuilder( oVarToTest );   // is the variable a file builder object?

IsDelimitedFile( oVarToTest ); // is the variable a delimited file I/O wrapper?

IsExcelExporter( oVarToTest ); // is the variable an Excel workbook 

                               // creator wrapper?

Note: See the class CRunTimeTypeInfo in the file ToolBox.Tool for a class that pulls all the RTTI APIs into a single TOOL class. The class sample shown is the base class for all TOOL framework classes, since all TOOL classes use variable type checks to insure the proper type of variables are passed to TOOL API functions from the TOOL framework.

TOOL Date Time Operations

Data processing and reporting often needs to deal with dates or date ranges. In fact, date time processing is likely to be "second most common" for data processing/reporting data types, with only string data being more frequently used. Because DateTimes are such a common requirement, TOOL offers a built in data type for that type of value; and also offers a set of API functions that offer the most common types of operations used for DateTime based values. These functions are listed below and a class wrapper for this family of functions can be found in the class called CDateTime which can be found in the ToolBox.Tool file.

dtNow   = GetDate();
dtLater = DateAdd( sModPart, dtBase, iValue );
dtEarly = DateSub( sModPart, dtBase, iValue );
iPart   = DatePart( sDatePart );
dtNew   = DateSetHourPart( dtToChange, iHour );
dtNew   = DateSetMinsPart( dtToChange, iMins );
dtNew   = DateSetSecsPart( dtToChange, iSecs );
dtNew   = DateSetHMS( dtToChange, iHour, iMins, iSecs );
sText   = DateToString( dtDate );
dtNew   = StringToDate( sText );
bLess   = DateCompare( dtBase, dtOther );
iSpan   = DateDiff( dtBase, dtOther );

TOOL File and Directory Operations

Directory Related Functions

Directory operations are very common for certain classes of data processing applications. To enable these types of jobs, TOOL includes several functions specific to those related tasks. The following classes illustrate the use of these types of functions.

The first TOOL class to study for this section of the TOOL API is found in the ToolBox.Tool file called CCurrentDirectory which implements a class wrapper over the TOOL functions related to manipulation of the current process directory. The current process directory is typically set by those scripts that are going to operate primarily from a single directory and use that directory as the "default path root" for all file-related operations.

The next directory related TOOL class wraps all those functions that actually manipulate the directory structure of a disk. There are TOOL API functions for moving, copying, comparing, renaming, and deleting entire directory trees on hard disks. See the CDirectory class in the ToolBox.Tool file for more information.

Directory and File Traversal Related Functions

Certain data transport tasks need to periodically scan file and directory structures as part of their processing. To support these types of jobs, TOOL offers a set of functions for traversal of directories and files. Class wrappers for these types of functions can be found in the ToolBox.Tool file in the CFind class.

For a practical example that uses these classes, refer to the simple network file backup program found in the TOOL Network Drive Operations topic. That example will demonstrate how TOOL can be put to use for purposes related to system administration in addition to tasks dedicated to data I/O, transformation and reporting.

The fact that TOOL can be used for various tasks outside of strictly data transport and reporting, adds additional value to the TOOL language in that the TOOL system has enough facilities included that allow it to be used for all types of operations.

File Related Functions

The TOOL API also offers file related functions for working with system files. TOOL provides a set of functions for inspecting file attributes, another set of functions for performing both text-based file I/O and for serialization of TOOL variables, ini-file I/O and file parsing. Classes that illustrate each of these portions of the API can be found in the ToolBox.Tool file.

See the CFileInfo in the ToolBox.Tool file for a class that implements a class wrapper for the file attributes and file management APIs built into the TOOL API. See the CTextFile class in the ToolBox.Tool file for a TOOL class that encapsulates all the TOOL API functions for text-file I/O.

Finally, see the CToolDataFile class in the ToolBox.Tool file which provides a wrapper over the TOOL API for the serialization of TOOL variable values. The class operates with a vector interface for reading and writing TOOL variables. The way TOOL serialization currently works, is that the script developer will "load a vector" with all the variables they want to save into a TOOL data file. The vector elements will be written in order to the data file. For restoring the values, the script simply reads back the vector data from the file.

Note, that at this time, collections of data (stacks, queues, maps, etc.) can not be serialized automatically. However, if demand warrants this type of functionality, it can easily be added to the TOOL run-time engine.

The final specialized area of TOOL file I/O processing is with regards to INI file processing. Since INI files are a convenient and easy to use method of storing simple hierarchies of configuration data, they are still a prevalent method of storing application parameters. So, as a convenience, TOOL also offers a set of functions for INI files. A class that provides a wrapper over these API functions is the CIniFile class found in the ToolBox.Tool file.

TOOL Network Drive Operations

TOOL offers a pair of functions that are related to connecting to network drives. There is one function to connect to a networked shared directory and another to disconnect from the same. The sample class named CNetworkDrive in the ToolBox.Tool file shows a class wrapper for these TOOL API functions.

In the CNetworkDrive class, you can see the straight-forward manner in which TOOL scripts can connect to network drives. There is another interesting item in this sample class however, and it is the Connect method; specifically, the test for:

sDriveLetter = "ERROR";

This shows another overloaded operator for string operations where TOOL string type variables can be directly tested for equality in addition to the StringCompare function. As an example of how to use the network drive class (and other classes) included in the ToolBox.Tool file, the following program shows how a TOOL program could be written to perform a network data file backup operation:

Run()
{
  CNetworkDrive poNetOut = new CNetworkDrive();
 
  // since the '\' character is an escape character within strings

  // i.e., \t for tab

  //       \r carriage-return 

  //       \n for line-feed

  //       \" for embedded-quote

  //

  // we need to use \\ for each \ character that should be made 

  // part of a string

  //

  // so in the string below: "\\\\MyServer\\BackupShare"

  //        will resolve to: "\\MyServer\BackupShare"

  //

  if ( poNetOut->Connect( "\\\\MyServer\\BackupShare", 
                          "Admin", 
                          "Password" )
  {
    sNetDrive = poNetOut->GetDriveLetter();
 
    // this example will back up three other computers to 

    // the single backup server

    //

    for ( int iLoop = 0; iLoop < 2; iLoop++ )
    {
      // based on the machine being backed up, set up for

      // that copy operation

      //

      if ( iLoop == 0 )
      {
        BackUpSingleServer( sNetDrive, 
                            "Workstatation1",
                            "ImportantWork" );  
      }
      else
      if ( iLoop == 1 )
      {
        BackUpSingleServer( sNetDrive, 
                            "Workstatation2",
                            "FinanceData" ); 
      }
      else
      if ( iLoop == 2 )
      {
        BackUpSingleServer( sNetDrive, 
                            "Workstatation3",
                            "CustomerData" );  
 
      }
    }
  }
}
 
// notice the parameter list in the function immediately below.

// the first three parameters are inputs to this function and

// as such are provided by the calling function.

//

// the next two parameters, which follow a ; character, are 

// locally scoped variables

//

BackUpSingleServer( sNetDrive, 
                    sServerName,
                    sServerShare,  
                    ; sTargetCopyPath,
                      sLastBackUpPath )
{
  CNetworkDrive poNetIn  = new CNetworkDrive();
  sTargetCopyPath  = StringCopy( sNetDrive );
  sLastBackUpPath  = StringCopy( sNetDrive );
 
  // set up the target paths for the backup file

  // storage

  //

  sTargetCopyPath += StringFormat( "\\%.Backup", 
                                   sServerName );
 
  sLastBackUpPath += StringFormat( "\\%.Prev.Backup", 
                                   sServerName );
 
  if ( poNetIn->Connect( StringFormat( "\\\\%\\%",
                                       sServerName,
                                       sServerShare ),
                                       "Admin",
                                       "Password" ) )
  {
    sInDrive = poNetIn->GetDriveLetter();
 
    // create a directory object to manage the data

    // copy

    //

    CDirectory poDir = new CDirectory( sInDrive );
 
    poDir->CopyDirectory( sTargetCopyPath, 
                          sLastBackUpPath );
 
   if ( poDir->CompareToOther( sTargetCopyPath ) )
   {
      // file backup success

    }
    else
    {
      Echo( FormatString( "Failure during backup of \\\\%\\%.\r\n",
                          sServerName, sServerShare ) );
    }
    poNetIn->Disconnect();
  }
  else
  {
    Echo( FormatString( "Failed to backup: \\\\%\\%.\r\n",
                        sServerName, sServerShare ) );
  }
}

The TOOL program above illustrates how to instantiate and make use of TOOL language class objects in a program. This demonstration program also shows how TOOL can be put to multiple uses in an organization in addition to its use as a data-management and reporting system by showing how a simple network backup program can be written in TOOL.

TOOL Registry Operations

Since TOOL developers may need to have their scripts interact with the system's registry, TOOL offers APIs for interacting with this aspect of the host system as well. The support offered by TOOL is enough to use the Registry for I/O of strings and numbers. Since nearly all TOOL data types can be converted to these two variable types, the TOOL API is so constrained for simplicity and ease of use (a recurring design goal for TOOL).

If additional functionality is required, then it will not be a difficult task to extend TOOL to support any new requirements with regards to operating with the system registry.

As is typical for this document, a class wrapper for this section of the TOOL API is found in the ToolBox.Tool file and in the TOOL class called CRegistry.

TOOL System Environment Operations

Periodically, in certain types of system instillations, the operating system's environment variables are used for certain purposes. For operations in these types of installations, TOOL provides functions that allow TOOL scripts to read/write to the system environment variables. The TOOL class called CEnvironment in the ToolBox.Tool file illustrates this portion of the TOOL API.

TOOL Process Control Functions

TOOL offers functions for starting, checking-on, and monitoring other processes. This allows TOOL to inter-operate with already existing programs, and/or to coordinate the activities of the system via the control over processes which are external to the TOOL run-time engine. TOOL can simply spawn a secondary process with the following call:

ProcStartNoWait( sProcessCommandLine );

When this API is invoked, the TOOL run-time will forward the process-command-line to the operating system for execution and then will return immediately, so as to not wait for the newly-started process. A related function is to query the operating system to determine if a particular process is running in the operating system where the TOOL runtime is hosted. This call is shown below:

IsProcessRunning( sProcessName );

These two functions can allow TOOL to be utilized as a simple-to-use watchdog program (a program that keeps another process running). The following example illustrates this concept:

WatchDog( sProcessName, sProcessCmdLine )
{
  while ( 1 )
  {
    if ( !IsProcessRunning( sProcessName ) )
    {
      // if process is not running, then start it

      //

      ProcStartNoWait( sProcessCmdLine );
    }
    // 

    // since this is an infinite loop, yield control on 

    // each pass through the loop

    //

    Sleep( 5000 );
  }
}

Another interesting TOOL API for starting external processes is:

StartLoggedProcess( sProcssesCmdLine, sFullPathToLogFile );

This API will start up a process and then will capture all data from that process' "stdout" to a log file passed to this function. This means of starting processes is very useful for running simple utility programs from the TOOL system, and then capturing the output from those utilities to a file for future trouble-shooting or analysis.

The next TOOL API for starting processes is one that allows a TOOL script to get a process handle from the operating system for the process just started. This handle then can be used as a "wait handle" so that TOOL can wait for the process to complete prior to continuing.

hHandle = StartProcessGetHandle( sProcessCmdLine );
....
ReleaseHandle( hHandle );

The ReleaseHandle call should be called from the script when interest in the process handle has ceased. The final process control function offered by TOOL is one that allows TOOL scripts to monitor and analyze the output of a slave process. When this API is invoked, TOOL will start and "attach to the output from" a process. What this allows is for all output produced by the slave process' "std output" and/or "std error" streams to be read and analyzed by the TOOL script. The API function that does this follows:

ProcStartErrorCheck( sRunLine, oVecErrorList );

The following function illustrates how to use this TOOL API function:

StartProgramAndCheckResults( sCommandLine; oVecErrorList, bSuccess )
{
  oVecErrorList = Vector();
  bSuccess      = Byte( 0 );
 
  // add a series of "error strings" to a list of phrases

  // the output from the slave process will be examined against the

  // error list. If the slave process produces any of these phrases

  // the results of the process execution will be reported as an

  // error

  //

  VectorAddItem( oVecErrorList , "ERROR"                       );
  VectorAddItem( oVecErrorList , "UNAVAILABLE"                 );
  VectorAddItem( oVecErrorList , "DOES NOT EXIST"              );
  VectorAddItem( oVecErrorList , "INCORRECT SYNTAX"            );
  VectorAddItem( oVecErrorList , "NO SUCH FILE OR DIRECTORY"   );
  VectorAddItem( oVecErrorList , "COLUMN DOES NOT ALLOW NULLS" );
  VectorAddItem( oVecErrorList , "INVALID OBJECT NAME"         );
  VectorAddItem( oVecErrorList , "INVALID COLUMN NAME"         );
  VectorAddItem( oVecErrorList , "CHOOSE ANOTHER"              );
  VectorAddItem( oVecErrorList , " LEVEL"                      );
  VectorAddItem( oVecErrorList , "FAILED"                      );
  VectorAddItem( oVecErrorList , "FILE NOT FOUND"              );
 
  // run a slave process and verify its output.

  // NOTE: The TOOL script will pause at this point until the

  //       slave process ends. Utility type programs that exit

  //       after completion of a task(s) are the most appropriate

  //       type of process to start with this TOOL API function.

  //

  bSuccess = ProcStartErrorCheck( sCommandLine, oVecErrorList );
 
  if ( !bSuccess )
  {
    // since Throw will exit this, make sure to free all

    // memory used here

    //

    VectorClear( oVecErrorList );
    FreeObject( oVecErrorList );
 
    Throw( StringFormat( "The program % returned an error.\r\n", 
                         sCommandLine ) );
  }
  VectorClear( oVecErrorList );
  FreeObject( oVecErrorList );
}

TOOL Inter-Process Synchronization Primitives

TOOL offers a complete set of process-synchronization variable-types. These variable types are important in those installations where TOOL script programs will coordinate actions with other external programs/processes through the signaling of these cross-process variables. These variables will also be used internally in TOOL multi-threaded-scripts.

The explanation of the appropriate use of these variable-types is beyond the scope of this document. For that type of information, the reader is referred to any reference that deals with multi-threaded programming techniques.

A class wrapper for each of the TOOL synchronization variable types is found in the ToolBox.Tool file. The first variable type you may want to examine is a class wrapper for a semaphore, which as you may recall is an inter-process variable that has a "count" associated with it. The variable can be "acquired" simultaneously by as many processes/threads as the "count" supports. See the CSemaphore class in ToolBox.Tool for that class.

The next type of inter-process variable supported in TOOL is the Mutex. This type of variable can only be "acquired" by a single process and/or thread at any time. This type of variable is a "mutual exclusion" gate and serves as a global "single threaded control" across the entire operating system. The TOOL class wrapper for that variable type is called CMutex and can be found in the ToolBox.Tool file.

The final type of inter-process variable supported in TOOL is an Event. An event can be used to "send signals" between processes and/or threads for purposes of coordinating activities between them. Properly understood and used, Events can lead to powerful interactions between completely separate processes. The CEvent class in the ToolBox.Tool file implements an event class.

Now that we've covered the creation and simple management of the inter-process variables supported in TOOL, the next topic (naturally) is how to use them in "wait sets" the use of which allows TOOL to coordinate its actions with other external processes. The CWaitSet class in ToolBox.Tool shows one possible way to use the TOOL wait-related.

Working with TOOL Collection Objects

TOOL offers several data types for managing collections of data. Included in the TOOL installation are sample programs that illustrate how to work with each collection type. It is hoped that the reader will start to become more familiar with class declarations in the TOOL language at the same time that they are being introduced to the collections API functions themselves.

Determining Element Counts of any Collection

TOOL offers a SizeOf function for determining the number of elements in any collection type variable. The following types of variables can be passed to this function:

• String
• Vector
• Stack
• Queue
• Map
• ByteArray

as follows:

lElements = SizeOf( oCollectionTypeVar );

TOOL String Operations

Strings are probably the most common data types used in nearly all data processing tasks. Due to that situation, string manipulation is probably the most common types of operations performed in data processing.

Because of this, the TOOL API for Strings is the "largest single section" dedicated to a single data type. The complete list of String-related API functions is listed below. Since the String-related API is as rich as it is, the sample class is likely to be the largest sample included in the samples provided with this distribution.

iResult = StringCompare( sOne, sTwo ); // returns -1 if sOne < sTwo

                                       //          0 if sOne = sTwo

                                       //          1 if sOne > sTwo

sResult = FillString( sBase, iStartAt, iSpanFor, sFillWith );
iIndex  = IndexOf( sBase, sToFind );
sResult = TrimString( sToTrim, sType ); // sType = "l" trim left

                                        //       = "r" trim right

                                        //       = "b" trim both

iLength = StringLength( sText );
sResult = ToLower( sInput );
sResult = ToUpper( sInput );
sResult = SubString( sInput, iStartAt, iLength );
sResult = StringGetAt( sInput, iIndex );
sResult = StringConcat( sInput, sToAppend );
sResult = StringReplace( sInput, sToReplace, sReplaceWith );
sResult = StringFormat( sFormatString, ... );
sResult = StringBoundedBy( sInput, sLowerToken, sUpperToken );
sResult = BoundedStringReplace( sIn, sLowToken, sUpToken, sNewText );
sResult = TrimQuotes( sInput );
sOutput = StringCopy( sInput );
sResult = PutQuote( sInput );
sResult = PadLeft( sInput, iPadSize );
sResult = PadRight( sInput, iPadSize );
bNumber = IsStringNumeric( sInput );

A rather large TOOL class that implements a class wrapper over the TOOL String-based API functions is the CString class found in the ToolBox.Tool file. That class contains a large amount of functionality (since there are so many types of operations typically performed on the string type data), so it is the largest one presented so far in this document.

The class also illustrates a few of the string-to-other-data-type conversion functions offered by TOOL. Even so, it is a straight-forward class and as such should offer no difficulties to the reader.

Another very important string function offered by TOOL is the "StringFormat" function. This elliptic function is very useful in building a single string from any number of data values appended into a format string. See the examples below:

sResult = StringFormat( "Today's data is: %", GetDate() );
sResult = StringFormat( "The value mapped in the under the key % is %",
                        iKey, 
                        MapFindByKey( oMap, iKey ) );

With the StringFormat function, the format string contains a '%' character at each position where the 'string value' to the next format argument is to be inserted in the output string.

To 'escape' the '%' character and output an actual percent sign, use '%%' which will output a single percent character into the resultant string at that location.

While there is no practical limit to the number of arguments that can be passed to the StringFormat function, the resultant string is limited to 4-KBytes in length. Another very important String operator is the '+' sign which can also be used for string concatenation, as follows:

sString  = "The quick brown fox";
sString += "jumped over the lazy dog";
sString += "and the cow \"jumped\" over the moon";

In the example above, you can also see that the '\' character can be used to "escape" a double-quote that should be embedded in the resultant string.

Tokenizing Strings in TOOL

To perform simple tokenization on string type variables, TOOL offers a built in tokenizer. A class wrapper for TOOL's String tokenizer functions is found in the CTokenizer class in the ToolBox.Tool file.

TOOL Vector Operations

TOOL offers a set of functions specific to operating on Vector type variables. For all these functions, the first argument to each of these functions is the Vector object upon which the function should operate against. Each of these operations will be introduced below.

TOOL Vector objects can contain a "mixed collection" of objects, meaning that each of the elements stored in the collection can be of different types.

VectorSetAtIndex( oVector, iIndex, vValue );
VectorGetAtIndex( oVector, iIndex );
VectorAddItem( oVector, vValue );
VectorDelAtIndex( oVector, iIndex );
VectorClear( oVector );

See the CVector class in the file ToolBox.Tool for a TOOL class that provides a class wrapper for all the Vector related functions offered by TOOL. That example illustrates once again how to declare a TOOL class as well as offering a rather complete implementation of a Vector object wrapper for TOOL vector-related functions.

In examining the CVector class example, one can see that the Vector class not only provides a complete class wrapper to all TOOL vector related functions; but that it extends the CRunTimeTypeInfo class introduced in the discussion of RTTI related functions; and that calls to the base class are used to validate arguments passed prior to making calls into the TOOL engine. For each of these validation calls, you can see the use of the cast-call-operator.

Note: It is considered good form to validate all arguments prior to invoking the API functions if there is any doubt as to the type of variable being passed to the TOOL runtime engine. Adopting this philosophy will help to make your TOOL scripts more robust and reliable and will prevent abrupt halts of your scripts by the run-time should you pass incorrect variable types into the TOOL API functions.

Through class design techniques, it is possible to implement TOOL collections that are "type checked" as is shown in the extension to the CVector class called CNumberSet in the ToolBox.Tool file. This class shows an example of a type constrained collection that will only store a collection of numbers.

In the CNumberSet example class, one can see that the SetAtIndex and AppendItem class functions have been overridden so that RTTI checks can be performed on the arguments passed to the class functions. You can also see the cast-call-operator usage again in several of these function calls including the class constructor.

In addition, the CNumberSet class also illustrates the use of three more specialized Vector operations; specifically the Max, Min, and Avg TOOL API functions which operate against all the numbers contained in a Vector variable and return the appropriate value from the operation.

TOOL Byte Array Operations

One of the TOOL data collection variable types is the Byte Array variable type. This type is similar to the Vector type explained in another section of this document, with one very important difference: the Byte Array only stores arrays of bytes. This class is useful as a "buffer class" for forming particular arrangements of byte values.

In looking over the list of functions shown below, you can see some of the Byte Array related functions offered by the TOOL API. You can see that the ByteArrayAppend and ByteArrayInsertAtIndex functions have several different overloads and can work with several types of formal arguments. This type of function overloading is possible due to the stack-oriented design of the TOOL engine, and the technique is used when it makes sense to offer an overloaded implementation.

ByteArrayGetAtIndex( oByteArray, iIndex );
ByteArraySetAtIndex( oByteArray, iIndex, bValue );
ByteArrayAppend( oByteArray, bAByte   );
ByteArrayAppend( oByteArray, sAString );
ByteArrayAppend( oByteArray, oAnotherByteArray );
ByteArrayInsertAtIndex( oByteArray, iIndex, bAByte   );
ByteArrayInsertAtIndex( oByteArray, iIndex, sAString );
ByteArrayInsertAtIndex( oByteArray, iIndex, oAnotherByteArray );
ByteArrayDelAtIndex( oByteArray, iStartDeletAtIndex, iCountToRemove );
ByteArrayClear( oByteArray );

The class CByteArray in ToolBox.Tool file is a sample class wrapper for the Byte Array variable type API functions offered by TOOL. In the class example, you will again see the use of the cast-call-operators and RTTI functions being used. After a review of the Byte Array class wrapper, and through studying the other classes in the ToolBox.Tool file, you should start to get a pretty good idea as to how to construct a class in the TOOL language.

TOOL Stack Operations

TOOL offers a Stack object for those scripts that can benefit from such a LIFO collection type. Like the Vector type, the Stack can store various variable types in a single collection. A Stack is rather a simple collection type, so the number of API functions dedicated to Stack is more limited than for other TOOL collection types. See the list below:

StackPush( oStack, oToPush );
StackPop( oStack );
StackPeek( oStack );
StackClear( oStack );

Refer again to the ToolBox.Tool file for the CStack class, which is a simple TOOL Stack class wrapper as shown to illustrate the use of all the stack related functions in TOOL. As you can see, this Stack class is relatively simple due to the fact that the Stack related API functions are limited by the straight-forward nature of the Stack variable type.

TOOL Map Operations

Currently, TOOL offers only one type of associative container; the Map which allows a simple storage container for keyed-value pairs. The TOOL Map data type constricts the types of variables that can be used as keys into the collection; but like all other TOOL containers, the values stored in the container can be any TOOL value type, and multiple variable types can be stored in a single map. If an illegal key type is attempted to be used with the TOOL Map, the TOOL Run-Time-Engine will halt execution of the TOOL script.

Like other TOOL data collections, the Map container is designed to be simple and straight-forward, so the number of Map related functions is limited to those functions required to work with the data type. The list of Map related API calls is listed below:

MapRemoveKey( oMap, aKey );
MapFindByKey( oMap, aKey );
MapHasKey( oMap, aKey );
MapClear( oMap );

For an example implementation of a TOOL Map class wrapper, see the CMap class in the ToolBox.Tool file. In order to implement a CMap class that is constrained to a single key or value type, one would need to revise the IsLegalKeyType and/or the IsLegalValueType class methods to provide a more restrictive set of tests on the keys and values allowed to be stored in the underlying Map variable.

TOOL Queue Operations

Another classic data collection offered by TOOL is the Queue data type. The TOOL Queue is an implementation of the FIFO data collection commonly used in many programming projects. The entire Queue related API is shown below:

EnQueue( oQueue, oToPush );
DeQueue( oQueue );
QueueClear( oQueue );

The TOOL class implementation of a Queue class is the CQueue class in the Toolbox.Tool file. Like the Stack data type, the TOOL Queue implementation is straightforward and does not require a large amount of functions to implement. From the samples provided, you can also see how easy it is to work with a TOOL queue.

TOOL Application Runtime Environment

Parameters can be passed into the TOOL runtime context for use as "environment variables" for the duration of that TOOL program. TOOL scripts can gain access to these variables with the calls explained below.

The RunTool.exe program can be invoked with command-line-arguments that will be forwarded to a TOOL script's runtime context. See the following example for how this could be done:

For other TOOL programs that are running in a TOOL run-time environment, there are two additional API calls that can be used to get and set variables within the program context. For fetching a variable from the application server environment, use the API "GetAppEnvField". An important distinction between "GetAppEnvField" and "GetProperty" is that "GetProperty" will always return a string type variable, whereas "GetAppEnvField" can return variables of the following types:

• String
• Long
• Byte (boolean)
• Double
• DWORD
• DateTime
• ByteArray

depending on the type of the data that was stored in the application context with the call to the API "SetAppEnvField". (Note that the above list of variable types are the only ones that will be allowed to be stored in the application context through the "SetAppEnvField" call. If any other type of variable is passed to the "SetAppEnvField" function, a run-time error will result and execution of the TOOL script will be terminated.) The sample below shows how these API functions may be used.

// store the variables in the application context

//

SetAppEnvField( "THE_STRING", "Hello"    );
SetAppEnvField( "THE_NUMBER", 1234567    );
SetAppEnvField( "FALSE_BOOL", Byte( 0 )  );
SetAppEnvField( "TRUE_BOOL",  Byte( 1 )  );
SetAppEnvField( "DOUBLE_VAL", 1234.5678  );
SetAppEnvField( "NEW_YEARS",  DateTime() );
 
// retrieve varialbes from the application context

//

xFetched = GetAppEnvField( "THE_STRING" );
xFetched = GetAppEnvField( "DOUBLE_VAL" );

See the TOOL class "CAppEnvironment" in the file ToolBox.Tool for a class that implements the application environment interface.

TOOL Database Operations

Since one of the primary purposes envisioned for TOOL is use in tasks related to data gathering, manipulation and storage of database data, TOOL must offer a database I/O system. The goal of this section of the TOOL API was to provide enough functionality to "get the job done" while attempting to avoid a lot of unnecessary complexity.

The interface method used by TOOL to connect to databases is the ODBC drivers offered by all major DBMS vendors. The purpose for this is that this level of interface is the lowest-common-denominator for all database connections (in fact even more 'advanced interfaces' are typically built as additional code layers over ODBC). In addition, it is a ubiquitous technology at this point and therefore will allow TOOL to "connect to virtually anything" on an equal basis.

Internally, TOOL has a sophisticated ODBC wrapper technology that allows access to database results to be processed without the TOOL programmer having to deal with the 'grungy details' of dynamically binding to the results of "any conceivable query" that are a part of TOOL jobs.

The database functions offered by TOOL are designed for a simple and straight-forward application of database I/O. The "command interface" between TOOL scripts and the connected databases is through SQL-strings passed to the DBMS via the TOOL-managed database connections. Again, the primary design consideration is for ease of use and portability between all database types/vendors rather than to build TOOL too tightly to