Extending a C++ Application with Lua 5.2

• Download source (no EXE) - 91.6 KB
• Download source - 362.9 KB

Introduction

Many applications would be more adaptable if end users were able to modify the application behavior though some kind of a scripting language. Some commercial applications provide such a facility. Examples include Visual Basic scripting in Microsoft Office or the use of Lua in video games such as World of Warcraft. The scripting language uses the application as a platform which provides a set of services the end user accesses and manipulates with the scripting language.

There are quite a few alternatives for an embedded language with a variety of open source and closed source engines available as well as the choice of creating one from scratch. Some of the most visible are JavaScript, Lua, and Python though there are many others. The general approach for embedding a script engine with Microsoft Windows is to use a Dynamic Link Library (DLL) containing the script engine and accessing the services in the engine with a set of function calls.

In this article, we will look at using the Lua 5.2 scripting engine embedded into a test harness written in C++ and developed with Visual Studio 2005. The necessary components for Lua are available through the lua.org web site. For our purposes, we downloaded Lua 5.2 as a compiled Windows DLL which provided the include files, the library file to link against, and the actual DLL which must be included with the application executable. While this example uses Visual Studio 2005, it should be compatible with later versions of Visual Studio.

This article provides an overview of a particular implementation of using the Lua 5.2 scripting engine. There is much more information provided in the documentation on lua.org as well as other web sites. However the test harness provided here is a starting place for your own experiments.

Background

Developing an application as a domain specific platform rather than a monolithic application is a different philosophy for product development. Such a domain specific platform is a kind of toolkit offering a variety of software services for end users to implement their own version of a domain specific application. The idea of a plug-in or add-in has been used in a variety of software including Microsoft Office and Microsoft Visual Studio as well as other Integrated Development environments such as Eclipse and image manipulation applications such as Adobe Photoshop. The World of Warcraft MMORPG has a large number of add-ons as do several other similar games. Some games offer tools that allow people to add additional content creating a modding community.

A commercial point of sale application, GenPOS, has several features which lend to a tool kit philosophy.

• The Layout Manager utility to design screen layouts for positioning windows and buttons and decorations
• The control string functionality to automate some workflows
• A large variety of parameters which modify software behavior
• A database of mnemonics allowing for the display of messages in a variety of languages
• An interface for dynamically changing parameters and mnemonics through a remote access interface
• An interface for pulling the financial and operational data from the terminal

However, there are some areas of change which are handled by the actual source code and which require the development agency to make source code changes in order to modify the behavior. For instance, one such area is printed receipt changes (some alternates are provided through parameter and mnemonic provisioning). A short list of other limitations would include:

• Existing control string functionality lacks state testing and jumps
• Unable to dynamically modify screen behavior and displayed information based on current state

Rather than attempting to enhance and improve the current and very simple control string functionality in order to provide additional capabilities for scripting the point of sale, we have decided to look into other possible enhancements that would provide much more product changeability without a great deal of development effort. Just as we did with the introduction of the Layout Manager utility to allow customers to design their own screen layouts and workflows, we want a fairly flexible mechanism that will allow Resellers of the point of sale to provide value added services to their customers through scripting. We also intend to provide sufficient access to application services that Resellers and End Customers will be able to modify application behavior on their own. Finally, we wanted to use a programming language that would have some degree of user community with resources for people wanting to use it.

We have done a few simple experiments of using the Lua 5.2 scripting engine within the point of sale source code to get some idea of the difficulty of inserting the functionality into the application. It appears from the experiments that much of what we will want to expose as application services are already available as various functions and can be readily made available to the Lua scripting engine with some adapters.

Using the Lua 5.2 Scripting Engine

The Lua scripting engine is written in the C programming language and using it in a C or C++ application is quite straightforward. There are a number of resources available on the web that provide bits and pieces of using the Lua 5.2 scripting engine in an application. The function interface for the 5.2 version of Lua does have a few changes from the previous versions primarily in the initialization and startup interface.

The basic initialization is as follows:

1. Create a new Lua state using the lua_newstate() function
2. Load up the standard Lua library if it is wanted luaL_openlibs()

Once the initialization is done, you can then run a Lua script by doing the following:

1. Load a Lua chunk or Lua script into the engine with one of the load functions such as luaL_loadfile()
2. Execute the Lua chunk using one of the call functions such as lua_pcall()

If you are wanting to load a Lua script file and then execute functions within the Lua script from an application, you must execute the Lua chunk that has been loaded. When a Lua chunk is loaded, it is compiled and loaded into the Lua scripting engine however it has not been executed. The Lua chunk will create any globals or functions when it is run however until the Lua chunk is executed, any globals or functions defined in the Lua script will not be available to the application. The same also applies to any globals that the application is providing to the Lua scripting engine as part of the environment being provided to the Lua engine. The application must first create the variables and functions and use the lua_setglobal() function to make them available. In the file UtilityFunctions.cpp where the method int LuaSimpleWrapper::TriggerGlobalCall() is defined is an example of dynamically building a Lua function call on the Lua virtual stack and then using the lua_pcall() function in the Lua scripting engine to execute the function with the provided arguments.

All of the various Lua scripting engine functions or services use a handle or pointer to a data structure containing the state information of the Lua scripting engine. This handle is specified as lua_State * or a pointer to a lua_State variable. Each time that the function lua_newstate() is called, the return value will be a pointer to a lua_State data structure or a NULL pointer if the call fails for some reason. This data structure is a kind of unique session variable allowing multiple Lua scripting engine sessions simultaneously. When the various Lua scripting engine function calls are used in the application or when a service provided by the application to the Lua scripting engine is invoked, the lua_State data structure provides the session environment so that the Lua session states are unique. This in turn implies that any functions provided by the application to the Lua scripting engine should be fully reentrant or there should be some kind of monitor or semaphore or critical region method used to enforce access to non-shareable services to provide thread safe access.

Using the Code

The source code in this Visual Studio 2005 project is composed of three C++ source files along with a sample Lua source to test out some of the basic functionality. The main is in Parser01.cpp which loads the specified Lua source file and then invokes several different functions. The file UtilityFunctions.cpp contains the source code for the LuaSimpleWrapper methods. The file InitEnviron.cpp contains the additional functions that we want our application to provide to the Lua environment.

The approach we are considering using is that a file containing a Lua script is specified at the time the point of sale application starts up. As part of starting up, the point of sale will start a thread in which the Lua scripting engine will be initialized and the specified Lua source file loaded and executed. The Lua chunk will remain in memory and as events happen within the point of sale application some of those events are transferred to the Lua script for processing. The test harness in this example program is an exploration of the approach we are considering.

In the function in the Lua source code provided below, we use several of the functions that are provided in the file InitEnviron.cpp in order to manipulate wide character strings which are not standard Lua strings. Standard Lua text strings are char strings (C style single byte character strings). These additional functions provide a way for a Lua script to perform string actions such as concatenation or comparison on wide character strings.

The following shows a small Lua function that takes three parameters and performs a series of actions. The function name xxfunc is a global name that is available to the application by using the lua_getglobal() function to retrieve it from the Lua global dictionary and put the handle to the function onto the Lua virtual stack. The parameters to the function are then pushed onto the Lua virtual stack using one of the push functions such as lua_pushstring() and then the function is executed by using the lua_pcall() Lua scripting engine function.

-- a sample Lua global function that can be invoked from the application or from Lua
function xxfunc (myMessage, wide1, wide2)
    trace("## xxfunc() called.")
    trace("  "..myMessage)
    trace ("  myFrame index "..myFrame.FrameIndex)
    trace ("  myFrame2 index "..myFrame2.FrameIndex)
    trace ("  myFrame3 index "..myFrame3.FrameIndex)
    
    -- compare two wide char strings that were passed in as arguments
    trace ("  compare wide "..wcscmp(wide1, wide2))
    
    -- generate a wide char string from a Lua string
    local widestring = wcscre("WIDE1")
    trace ("  compare with generated "..wcscmp (wide1, widestring))
    
    -- try out the wcscat and the wcscre functions to generate a string
    traceW (wcscat (wcscre("  concat two "), wcscat(wide1, wide2)))
    
    traceW (wcscat (wcscre("  concat multi "), wide1, wcscre(" "), wide2))
    
    -- tryout wcscat with a non-string argument which should be skipped.
    traceW (wcscat (wcscre("  concat multi "), 2, wcscre(" "), wide2))
   
    local myMemEntry = GetMnemonic (15)
    if (myMessage) then
        if (myMessage.Type == "FRAMEWORK") then
            local myNem = GetMnemonic (20)
        end
    end
end

The above Lua function in the Lua script that has been loaded can be invoked from the application. Using a helper function from the LuaSimpleWrapper class, TriggerGlobalCall(), we can invoke the Lua function with the following lines of C++ source. The TriggerGlobalCall() method takes a descriptive string which indicates the global Lua function (function or function assigned to a variable or a table entry) to invoke along with a description of the argument types. The arguments follow the descriptive string. This type of variable function call can be a source of run time errors since there are several separate pieces of source which must agree: (1) the descriptive string in the TriggerGlobalCall() call, (2) the actual arguments provided in the TriggerGlobalCall() call, and (3) the Lua function that is being invoked.

if (myLua.TriggerGlobalCall ("xxfunc:s,w,w", 
          "TriggerGlobalCall", L"WIDE1", L"WIDE2") < 0) {
    cout << "%% " << myLua.GetLastErrorString() << endl ;
}

The descriptive string uses a comma separated list of single letter argument types in which the letter s indicates a char string, the letter w indicates a wide character string, the letter d indicates a double, the letter i indicates an integer, and the letter f indicates a function address. In the code for the TriggerGlobalCall() method, the commas between the letters are ignored and they are actually just a way to make the descriptive string easier to recognize.

The above TriggerGlobalCall() method invoking the Lua function xxfunc() will result in the following output. This output is generated by the Lua function using the parameters specified in the TriggerGlobalCall() argument list.

## xxfunc() called.
TriggerGlobalCall
myFrame index 0
myFrame2 index 1
myFrame3 index 2
compare wide -1
compare with generated 0
concat two WIDE1WIDE2
concat multi WIDE1 WIDE2
concat multi  WIDE2
getTransactionMnemonic() 15

The TriggerGlobalCall() method we provide in the LuaSimpleWrapper class allows for a table value to be specified allowing accessing a function that has been assigned to a key in a Lua table. The format for the descriptive string is "tablename.key" in which "tablename" is the global name of a Lua table and "key" is the table key used to access the function.

// specify a function to be invoked by the OnEvent() handler
// specify a different event type which is not in the Lua script
if (myLua.TriggerGlobalCall ("myFrame2.OnEvent:s,f", "EVENT_TYPE_J2", SimpleFunc) < 0) {
    cout << "%% " << myLua.GetLastErrorString() << endl;
}

Exporting a C/C++ Function to the Lua Engine

To create a helper C or C++ function to be used in a Lua script, the C/C++ application must provide the function body and then use the appropriate Lua engine functions to make the new function available to the Lua engine. The function calls used in the application to provide the new helper function to the Lua engine push values onto the Lua virtual stack and then call the lua_setglobal() function to make the application function available to the Lua scripting engine as a global function.

lua_pushcclosure (lua, concatMultiWideStrings, 0);
lua_setglobal (lua, "wcscat");

Lua supports the idea of a closure for functions. A closure allows an application to specify one or more values that will be available to the application function when the Lua scripting engine invokes it. These values can be updated by the application function, a feature used in this example to provide a unique value through the incrementing of a counter associated with the application function. An example of the C++ source for this can be found in the int LuaSimpleWrapper::InitLuaEnvironment() method which provides the CreateFrame() function to the Lua scripting engine.

// CreateFrame() function that will create a frame with an index
// This function uses two variables which are used to store the
// frame data allowing it to be used by the application in order to
// send events to a specific frame object in the Lua code.
// we access the array of the list of objects, m_ListOfObjects[], with objectindex
// and we access the specific frame for the object with frameindex.
lua_pushnumber(m_luaState, 0);                // frameindex, count of frames for this Lua state object, init to zero
lua_pushnumber(m_luaState, m_MyObjectCount);  // objectindex, which Lua state object am I?

// create the C closure with the above two arguments, 
lua_pushcclosure (m_luaState, ParserLuaCreateGlobalFrame, 2);
lua_setglobal (m_luaState, "CreateFrame");

A C++ function being exported has a source code body like the following. This function, concatMultiWideStrings () uses a series of Lua engine functions to process what is on the Lua virtual stack and to then return the result to the Lua engine. The following function shows the use of the lua_State * parameter providing the session environment for the function. This particular function allows for multiple strings to be concatenated together. The Lua scripting engine provides a count for the number of arguments available on the Lua virtual stack. We can also use the provided function lua_type() to determine the type of the argument allowing us to skip those arguments that are not of the correct type

Actually the Lua scripting engine will perform variable type transformations however in our case we want to use only type LUA_TSTRING because any transformation that Lua will do from some other type to the string type will result in a C style single byte character string rather than a double byte wide character string that we are expecting.

// concatenate multiple wide strings
//  const wchar_t *wcscat(wchar_t *wcharSt1, const wchar_t *wcharSt2, const wchar_t *wcharSt3, ...)
static int concatMultiWideStrings (lua_State *lua)
{
    int  nPushCount = 0;
    int  nArgIndex = 1;
    int argc = lua_gettop(lua);
    wchar_t  tempBuffer[2048];

    if (argc > 0) {
        wchar_t    *pWideString = &tempBuffer[0];
        size_t     iLen = 1;

        while (nArgIndex <= argc) {
            if (lua_type(lua, nArgIndex) == LUA_TSTRING) {
                const wchar_t *msgX = (wchar_t *) lua_tostring (lua, nArgIndex);
                while (*msgX) {*pWideString++ = *msgX++; iLen++; }
            }
            nArgIndex++;
        }
        *pWideString = 0;  // final zero terminator
        lua_pushlstring (lua, (char *)(&tempBuffer), iLen * sizeof(wchar_t));
        nPushCount++;
    }

    return nPushCount;
}

Points of Interest

The first version of this test harness was very simple beginning with just a simple Lua script that would load and run writing a "Hello World" to the console using the Lua output functions. As the script and the test harness became more complex while investigating the potential of embedding Lua, it quickly became evident that some way of dumping the Lua virtual stack was a necessity in order to understand how the communication between the Lua engine and the application was working. Several times sudden breaks in behavior required using the stack dump function while stepping through the C++ source with the debugger in order to understand what was happening and to determine a fix for the behavior.

The dynamic nature of Lua, much like loose type languages such as JavaScript, can encourage the adventurous programmer to write some really interesting source code however it can also result in source code that is difficult to debug and difficult to test. The difficulty is compounded by using two different languages and the lack of debugging facilities in Visual Studio for Lua.

When implementing the method int LuaSimpleWrapper::TriggerGlobalCall () we ran into test cases which resulted in the Lua scripting engine performing an application close or exit resulting in a Windows error dialog when running in the Visual Studio debugger. We decided that the problem was that some values were being pushed onto the Lua virtual stack and, due to the error condition, were not being properly processed. In order to address the problem, when an error was detected, we cleared the Lua virtual stack using the following C++ source code sequence. In the test harness we have two different tests, one test that the specified global name exists and the second test is that if a key value is specified using the "global.key" syntax, then the global must be a table otherwise it is an error.

lua_getglobal (m_luaState, globalName);
if (lua_type(m_luaState, lua_gettop(m_luaState)) == LUA_TNIL) {
    // if the global variable does not exist then we will bail out with an error.
    strcpy_s (m_lastLuaError, sizeof(m_lastLuaError), "Global variable not found: ");
    strcat_s (m_lastLuaError, sizeof(m_lastLuaError), globalName);
    m_lastState = LuaDescripParse;
    // error so we will just clear the Lua virtual stack and then return
    // if we do not clear the Lua stack, we leave garbage that will cause
    // problems with later function calls from the application.
    // we do this rather than use lua_error() because this function is called
    // from the application and not through Lua.
    lua_settop (m_luaState, 0);
    return -1;
}