Building a simple C++ script compiler from Scintilla and CINT.

• Download demo - 694 Kb
• Download source - 3,340 Kb

Introduction

I ran across two Open Source projects recently. CINT, a C/C++ script engine, and Scintilla, a multi-language editor with syntax highlighting among other features.

So the possibilities here are obvious. A simple application that would allow the rapid entry, execution and debugging of C/C++ scripts. What a handy tool for a developer such as myself. I would imagine this would also be of interest to anyone learning the C/C++ language.

Application Goals

There is tremendous opportunity here for features. But I decided to keep the feature set as small as possible so as to focus on the integration of the CINT engine and the Scintilla editor.

The features provided will enable you to enter your C/C++ code into the editor, execute it, then the program will rub your nose in all the numerous syntax errors you have inflicted on the digital world.

This application also statically links both projects and I will cover the details needed to make this happen. Often with small applications like this, you know, without full blown installers, it's nicer to static link to avoid frustrating users with missing DLL's. My personal preference is to static link if possible unless I have an installer that is capable of verifying the integrity of the installation. Disk space is cheaper than time these days.

Our project

The project is a MFC dialog application created with VC6. I avoided using MFC specific classes for integrating CINT or Scintilla so you should have no problem porting this code to a non-MFC project.

Linking to Scintilla

From the Scintilla Website - Scintilla is an editor control providing support for syntax styling, error indicators, code completion and call tips. The selection margin can contain markers like those used in debuggers to indicate breakpoints and the current line. Styling choices are more open than with many editors, allowing the use of proportional fonts, bold and italics, multiple foreground and background colors and multiple fonts.

I downloaded and extracted Scintilla into the sub folder scintilla in our project folder. We find the VC project files in the sub folder ./scintilla/vcbuild. After adding the SciLexer.dsp project to our workspace, we find it builds without error. Great!

By default Scintilla compiles to a DLL. We would like to static link, so we will add a linker response file to create a static library. I created two files, one for the release version, and another for the debug version.

Linker Response File (rsp_scintilla.txt) - Release Version

/nologo /subsystem:windows ./Release/*.obj /out:../bin/s_scintilla.lib

Linker Response File (rsp_scintillad.txt) - Debug Version

/nologo /subsystem:windows ./Debug/*.obj /out:../bin/sd_scintilla.lib

Now we add a Post-Build Step to each the Release and Debug versions calling the appropriate response file.

Post-Build Step - Release Version

link -lib @rsp_scintilla.txt

Post-Build Step - Debug Version

link -lib @rsp_scintillad.txt

Build Scintilla and you should find that the files sd_scintilla.lib and s_scintilla.lib have been created in the scintilla/bin folders. These are the libs we will link to.

We need to add Scintilla headers to our project, so I prefer to do this in the Stdafx.h file, since the Scintilla library files will probably not change much. So, in Stdafx.h, let's add the following includes...

// Include Scintilla parser

#include "scintilla/include/SciLexer.h"

#include "scintilla/include/Scintilla.h"

Last step here, we need to link to the Scintilla lib files. Open the Stdafx.cpp file and add the following....

#ifdef _DEBUG
#   pragma comment( lib, "scintilla/bin/sd_scintilla.lib" )
#else
#   pragma comment( lib, "scintilla/bin/s_scintilla.lib" )
#endif

This will link us to the debug or release version as needed. You could also add the lib files to the project settings, I prefer this method.

Since the Scintilla project was a DLL, and we're now linking statically, we need to reproduce the startup and shutdown procedures it would have done in the DLL. You can usually find this by searching the project for the function DllMain(). Sure enough, we find DllMain() in Scintilla. It registers the editor window class among other things. We will add this code to our startup. In InitInstance() add...

// Initialize the Scintilla

if ( !Scintilla_RegisterClasses( AfxGetApp()->m_hInstance ) )
{   AfxMessageBox( "Scintilla failed to initiailze" );
    return FALSE;
} // end if

Add the Scintilla shutdown code to ExitInstance()...

    // Release Scintilla

    Scintilla_ReleaseResources();

Great! We are now linked to the Scintilla library. Let's move on to CINT.

Linking to CINT

From the CINT Website - CINT is a C/C++ interpreter aimed at processing C/C++ scripts.

CINT is a part of ROOT. ROOT has a fascinating feature set and it would be interesting to integrate as well. I would have done so except that ROOT is covered by the LGPL, and being a commercial developer myself, this license would never allow me to use the work in an actual project. Scintilla and CINT are covered by more commercial friendly licenses. I would deffinately like to update this project in the future to integrate ROOT as well.

I downloaded and extracted CINT into the sub folder cint in our project folder. Unfortunately the VC project files for CINT are actually wrappers around a MAK file. Because of this I chose to just create another project and add the files that I needed. I created the project files in the sub folder ./cint/libcint. This project natively creates static libraries, so there is nothing else to do here but compile.

Upon compiling, I ran into a small snag. Two functions, G__SetGlobalcomp() and G__ForceBytecodecompilation() are defined in both Method.cxx and v6_dmystrm.c and both files are needed to compile CINT. The v6_dmystrm.c versions are the ones we want, so, I surrounded the two functions in Method.cxx with #if 0. I also had to do the same in Apiifold.cxx. I'm sure this is probably an oversight since the project is primarily for UNIX. Hopefully this will be resolved in a later version.

Despite this minor hiccup, things are now compiling nicely. Let's add it to our project by added the following includes to the the Stdafx.h file...

// CINT

#include "cint/G__ci.h"

#include "cint/src/Global.h"

We need to link as well, so we add the following lines to the Stdafx.cpp file...

#ifdef _DEBUG
#   pragma comment( lib, "cint/libcint/Debug/libcint.lib" )
#else
#   pragma comment( lib, "cint/libcint/Release/libcint.lib" )
#endif

Creating the Scintilla Editor and Output windows

Now that both libraries are linked in, we can start the fun part of actually using them. To start let's create the Scintilla editor, by calling the function InitialiseEditor() from OnInitDialog(). We require a list of C/C++ keywords and a color scheme to complete the initialization. I used a color scheme I like. The original Twilight scheme from the old Borland compiler for DOS from which I learned C++ many moons ago. It should be straight forward to modify the colors to your own taste.

// C++ keywords

static const char g_cppKeyWords[] = 

    // Standard

    "asm auto bool break case catch char class const "
    "const_cast continue default delete do double "
    "dynamic_cast else enum explicit extern false finally "
    "float for friend goto if inline int long mutable "
    "namespace new operator private protected public "
    "register reinterpret_cast register return short signed "
    "sizeof static static_cast struct switch template "
    "this throw true try typedef typeid typename "
    "union unsigned using virtual void volatile "
    "wchar_t while "

    // Extended

    "__asm __asume __based __box __cdecl __declspec "
    "__delegate delegate depreciated dllexport dllimport "
    "event __event __except __fastcall __finally __forceinline "
    "__int8 __int16 __int32 __int64 __int128 __interface "
    "interface __leave naked noinline __noop noreturn "
    "nothrow novtable nullptr safecast __stdcall "
    "__try __except __finally __unaligned uuid __uuidof "
    "__virtual_inheritance";


/// Scintilla Colors structure

struct SScintillaColors
{   int         iItem;
    COLORREF    rgb;
};

// A few basic colors

const COLORREF black = RGB( 0, 0, 0 );
const COLORREF white = RGB( 255, 255, 255 );
const COLORREF green = RGB( 0, 255, 0 );
const COLORREF red = RGB( 255, 0, 0 );
const COLORREF blue = RGB( 0, 0, 255 );
const COLORREF yellow = RGB( 255, 255, 0 );
const COLORREF magenta = RGB( 255, 0, 255 );
const COLORREF cyan = RGB( 0, 255, 255 );

/// Default color scheme

static SScintillaColors g_rgbSyntaxCpp[] = 
{
    {   SCE_C_COMMENT,          green },
    {   SCE_C_COMMENTLINE,      green },
    {   SCE_C_COMMENTDOC,       green },
    {   SCE_C_NUMBER,           magenta },
    {   SCE_C_STRING,           yellow },
    {   SCE_C_CHARACTER,        yellow },
    {   SCE_C_UUID,             cyan },
    {   SCE_C_OPERATOR,         red },
    {   SCE_C_PREPROCESSOR,     cyan },
    {   SCE_C_WORD,             cyan },
    {   -1,                     0 }
};

void CCintDlg::InitialiseEditor() 
{
    // Punt if we already have a window

    if ( ::IsWindow( m_hwndEditor ) ) return;

    // Create editor window

    m_hwndEditor = CreateWindowEx(  0, "Scintilla", "", 
                                        WS_CHILD | WS_VISIBLE | WS_TABSTOP | 
                                        WS_CLIPCHILDREN,
                                        10, 10, 500, 400,
                                        GetSafeHwnd(), NULL /*(HMENU)GuiID*/, 
                                        AfxGetApp()->m_hInstance, NULL );   

    // Did we get the editor window?

    if ( !::IsWindow( m_hwndEditor ) )
    {   TRACE( "Unable to create editor window\n" );
        return;
    } // end if


    // CPP lexer

    SendEditor( SCI_SETLEXER, SCLEX_CPP );

    // Set number of style bits to use

    SendEditor( SCI_SETSTYLEBITS, 5 );
    
    // Set tab width

    SendEditor( SCI_SETTABWIDTH, 4 );

    // Use CPP keywords

    SendEditor( SCI_SETKEYWORDS, 0, (LPARAM)g_cppKeyWords );

    // Set up the global default style. These attributes are used wherever no explicit choices are made.

    SetAStyle( STYLE_DEFAULT, white, black, 10, "Courier New" );

    // Set caret foreground color

    SendEditor( SCI_SETCARETFORE, RGB( 255, 255, 255 ) );

    // Set all styles

    SendEditor( SCI_STYLECLEARALL );

    // Set selection color

    SendEditor( SCI_SETSELBACK, TRUE, RGB( 0, 0, 255 ) );
    
    // Set syntax colors

    for ( long i = 0; g_rgbSyntaxCpp[ i ].iItem != -1; i++ ) 
        SetAStyle( g_rgbSyntaxCpp[ i ].iItem, g_rgbSyntaxCpp[ i ].rgb );
}

I also used Scintilla for the output window. Just to show a different method. I accessed the output window more directly using ::SendMessage().

void CCintDlg::InitialiseOutput() 
{
    // Punt if we already have a window

    if ( ::IsWindow( m_hwndOutput ) ) return;

    // Create editor window

    m_hwndOutput = CreateWindowEx(  0, "Scintilla", "", 
                                        WS_CHILD | WS_VISIBLE | WS_TABSTOP | 
                                        WS_CLIPCHILDREN,
                                        10, 10, 500, 400,
                                        GetSafeHwnd(), NULL /*(HMENU)GuiID*/, 
                                        AfxGetApp()->m_hInstance, NULL );   

    // Did we get the editor window?

    if ( !::IsWindow( m_hwndEditor ) )
    {   TRACE( "Unable to create editor window\n" );
        return;
    } // end if


    // Set number of style bits to use

    ::SendMessage( m_hwndOutput, SCI_SETSTYLEBITS, 5, 0L );

    // Set tab width

    ::SendMessage( m_hwndOutput, SCI_SETTABWIDTH, 4, 0L );

    // Set foreground color

    ::SendMessage( m_hwndOutput, SCI_STYLESETFORE, STYLE_DEFAULT, (LPARAM)RGB( 255, 255, 255 ) );

    // Set background color

    ::SendMessage( m_hwndOutput, SCI_STYLESETBACK, STYLE_DEFAULT, (LPARAM)RGB( 0, 0, 0 ) );

    // Set font

    ::SendMessage( m_hwndOutput, SCI_STYLESETFONT, STYLE_DEFAULT, (LPARAM)"Courier New" );

    // Set selection color

    ::SendMessage( m_hwndOutput, SCI_SETSELBACK, (WPARAM)TRUE, (LPARAM)RGB( 0, 0, 255 ) );
    
    // Set all styles

    ::SendMessage( m_hwndOutput, SCI_STYLECLEARALL, 0, 0L );

}

To position the windows, I used the following code. It's straight forward as we are just dealing with plain old Window handles...

BOOL CCintDlg::Size()
{
    // Ensure valid window

    if ( !::IsWindow( GetSafeHwnd() ) )
        return FALSE;

    // Get window size

    RECT rect, ctrl;
    GetClientRect( &rect );
    CopyRect( &ctrl, &rect );

    // Position the editor window

    ctrl.bottom -= ( 6 * 24 );
    CWnd *pWnd = CWnd::FromHandle( m_hwndEditor );
    if ( pWnd ) pWnd->MoveWindow( &ctrl );

    // Position the output window

    ctrl.top = ctrl.bottom;
    ctrl.bottom = rect.bottom;
    pWnd = CWnd::FromHandle( m_hwndOutput );
    if ( pWnd ) pWnd->MoveWindow( &ctrl );

    return TRUE;
}

Executing the code with CINT

Of course, we want to be able to execute the script, and have any output generated accessible in a cut-and-pasteable window. CINT sends the output from the script to the standard output stream STDOUT. We need to intercept this data. So to make this simple, I appropriated the class CHookStdio from the Open Source project rulib (don't worry, no license violation here). CHookStdio allows us to easily hook STDOUT and access the data. Note that we have to pass a parameter to CHookStdio indicating how much buffer space we need. Be aware of this size when writing your scripts. I set this to 64K.

So the steps to execute our script are now...

Get the text from Scintilla

Hook STDOUT

Send to CINT for processing

Check for CINT errors

Write the hooked STDOUT data to the output window

And here are the details.

void CCintDlg::OnExecute() 
{
    // Reset CINT

    G__scratch_all();
    g_sCintLastError = "";

    // Reset Scintilla

    SendEditor( SCI_MARKERDELETEALL, 0 );

    // Clear output window

    ::SendMessage( m_hwndOutput, SCI_SETTEXT, 0, (WPARAM)"" );

    // Get the editor window handle

    CWnd *pWnd = CWnd::FromHandle( m_hwndEditor );
    if ( !pWnd ) return;

    // Get the script text

    CString strScript;
    pWnd->GetWindowText( strScript );
    if ( strScript.IsEmpty() ) return;

    // Must add to get proper line number

    strScript = "#line 0\r\n" + strScript;

    // Hook stdio output

    CHookStdio hs( STD_OUTPUT_HANDLE );

    // Set error callback function

    G__set_errmsgcallback( &CCintDlg::CintError );

    // Execute the program

    if ( !G__int( G__exec_text( (LPCTSTR)strScript ) ) )
    {
        // Initilaize error markers

        SendEditor( SCI_MARKERDEFINE, 0, SC_MARK_SHORTARROW );
        SendEditor( SCI_MARKERSETFORE, 0, RGB( 80, 0, 0 ) );
        SendEditor( SCI_MARKERSETBACK, 0, RGB( 255, 0, 0 ) );

        // Set error marker to proper line

        int nErrLine = G__lasterror_linenum();
        SendEditor( SCI_MARKERADD, nErrLine - 1, 0 );

        // Show the error string

        ShowError( g_sCintLastError.c_str() );

        return;

    } // end if                


    // Set foreground color

    ::SendMessage( m_hwndOutput, SCI_STYLESETFORE, STYLE_DEFAULT, (LPARAM)RGB( 255, 255, 255 ) );
    ::SendMessage( m_hwndOutput, SCI_STYLECLEARALL, 0, 0L );

    // Get output

    char buf[ 64 * 1024 ] = "";
    buf[ hs.Read( buf, sizeof( buf ) - 1 ) ] = 0;

    // Show script output

    if ( *buf ) ::SendMessage( m_hwndOutput, SCI_SETTEXT, 0, (WPARAM)buf );
}

This function also includes code for highlighting errors in the script. Here is a screenshot of what an error looks like.

Conclusion

That sums it up. So have fun.

One of the many practical uses I can think of for this project is generating tedious lookup tables which often come up when optimizing code. Now I can create a simple script and run it when I needed, instead of wasting time with an entire project just to generate a simple table. (Although I have to admit to using PHP for this lately).

Here is a script to generate a square root lookup table...

// Start the table

printf( "const double g_rdLookup_Sqrt[] = \r\n{\r\n\t" );

// Generate items

for ( int i = 0; i < 100; i++ )
{
    // Add value separator

    if ( i ) printf( ", " );

    // Break into reasonable pieces

    if ( i && !( i % 8 ) ) printf( "\r\n\t" );

    // Calculate value

    printf( "%g", (double)sqrt( (double)i ) );
}

// Complete the table

printf( "\r\n};" );

And here's the output...

const double g_rdLookup_Sqrt[] = 
{
    0, 1, 1.41421, 1.73205, 2, 2.23607, 2.44949, 2.64575, 
    2.82843, 3, 3.16228, 3.31662, 3.4641, 3.60555, 3.74166, 3.87298, 
    4, 4.12311, 4.24264, 4.3589, 4.47214, 4.58258, 4.69042, 4.79583, 
    4.89898, 5, 5.09902, 5.19615, 5.2915, 5.38516, 5.47723, 5.56776, 
    5.65685, 5.74456, 5.83095, 5.91608, 6, 6.08276, 6.16441, 6.245, 
    6.32456, 6.40312, 6.48074, 6.55744, 6.63325, 6.7082, 6.78233, 6.85565, 
    6.9282, 7, 7.07107, 7.14143, 7.2111, 7.28011, 7.34847, 7.4162, 
    7.48331, 7.54983, 7.61577, 7.68115, 7.74597, 7.81025, 7.87401, 7.93725, 
    8, 8.06226, 8.12404, 8.18535, 8.24621, 8.30662, 8.3666, 8.42615, 
    8.48528, 8.544, 8.60233, 8.66025, 8.7178, 8.77496, 8.83176, 8.88819, 
    8.94427, 9, 9.05539, 9.11043, 9.16515, 9.21954, 9.27362, 9.32738, 
    9.38083, 9.43398, 9.48683, 9.53939, 9.59166, 9.64365, 9.69536, 9.74679, 
    9.79796, 9.84886, 9.89949, 9.94987
};

History

void CCintDlg::OnExecute() 
{
    USES_CONVERSION;

	// Reset CINT
	G__scratch_all();
	g_sCintLastError = "";

	// Reset Scintilla
	SendEditor( SCI_MARKERDELETEALL, 0 );

	// Clear output window
	::SendMessage( m_hwndOutput, SCI_SETTEXT, 0, (WPARAM)"" );

	// Get the editor window handle
	CWnd *pWnd = CWnd::FromHandle( m_hwndEditor );
	if ( !pWnd ) return;

	// Get the script text
	CString strScript;
	pWnd->GetWindowText( strScript );
	if ( strScript.IsEmpty() ) return;

	// Must add to get proper line number
	strScript = "#line 0\r\n" + strScript;

	// Hook stdio output
	CHookStdio hs( STD_OUTPUT_HANDLE, 64 * 1024 );

	// Set error callback function
	G__set_errmsgcallback( &CCintDlg::CintError );

	// Execute the program
	if ( !G__int( G__exec_text( T2A(strScript) ) ) )
	{
		// Initilaize error markers
		SendEditor( SCI_MARKERDEFINE, 0, SC_MARK_SHORTARROW );
		SendEditor( SCI_MARKERSETFORE, 0, RGB( 80,  0,  0) );
		SendEditor( SCI_MARKERSETBACK, 0, RGB(255,  0,  0) );

		// Set error marker to proper line
		int nErrLine = G__lasterror_linenum();
		SendEditor( SCI_MARKERADD, nErrLine - 1, 0 );

		// Show the error string
		ShowError( A2T(g_sCintLastError.c_str()) );

		return;

	} // end if				   

	// Set foreground color
	::SendMessage( m_hwndOutput, SCI_STYLESETFORE, STYLE_DEFAULT, (LPARAM)RGB( 255, 255, 255 ) );
	::SendMessage( m_hwndOutput, SCI_STYLECLEARALL, 0, 0L );

	// Get output
	char buf[ 64 * 1024 ] = "";
	buf[ hs.Read( buf, sizeof( buf ) - 1 ) ] = 0;

	// Show script output
	if ( *buf ) ::SendMessage( m_hwndOutput, SCI_SETTEXT, 0, (WPARAM)buf );
}