Click here to Skip to main content
Click here to Skip to main content
Add your own
alternative version

a Tiny Parser Generator v1.2

, 21 Sep 2010
@TinyPG is a utility that makes it easier to write and try out your own parser/compiler.
TinyPG_v1.0_binaries.zip
Release
Examples
BNFGrammar v1.0.tpg
simple expression1.tpg
simple expression2.tpg
Templates
TinyPG.exe
TinyPG_v1.0_sources.zip
TinyPG v1.0
bin
Debug
Release
TinyPG.vshost.exe
TinyPG.vshost.exe.manifest
Compiler
backup
vssver2.scc
vssver2.scc
Controls
DockExtender
vssver2.scc
vssver2.scc
Debug
vssver2.scc
Examples
BNFGrammar v0.2.tpg
BNFGrammar v1.0.tpg
simple expression1.tpg
simple expression2.tpg
test.tpg
vssver2.scc
mssccprj.scc
obj
Debug
TempPE
Release
TempPE
Properties.Resources.Designer.cs.dll
Properties
vssver2.scc
Templates
vssver2.scc
TinyPG.csproj.user
TinyPG.suo
vssver2.scc
TinyPG_v1.1_sources.zip
TinyPG v1.1
bin
Release
Examples
BNFGrammar v1.0.tpg
BNFGrammar v1.1.tpg
simple expression1.tpg
simple expression2.tpg
Templates
TinyPG.exe
Compiler
vssver2.scc
Controls
DockExtender
vssver2.scc
vssver2.scc
Debug
vssver2.scc
Examples
BNFGrammar v0.2.tpg
BNFGrammar v1.0.tpg
BNFGrammar v1.1.tpg
simple expression1.tpg
simple expression2.tpg
vssver2.scc
Highlighter
vssver2.scc
mssccprj.scc
Properties
vssver2.scc
Templates
vssver2.scc
TinyPG.csproj.user
TinyPG.suo
vssver2.scc
TinyPG_v1.2_binaries.zip
BNFGrammar v1.0.tpg
BNFGrammar v1.1.tpg
GrammarHighlighter.tpg
simple expression1.tpg
simple expression2.tpg
C#
VB
TinyPG.exe
TinyPG_v1.2_sources.zip
TinyPG v1.2
bin
Release
Examples
BNFGrammar v1.0.tpg
BNFGrammar v1.1.tpg
GrammarHighlighter.tpg
simple expression1.tpg
simple expression2.tpg
Templates
C#
VB
TinyPG.exe
CodeGenerators
CSharp
vssver2.scc
VBNet
vssver2.scc
vssver2.scc
Compiler
BNFGrammar.tpg
vssver2.scc
Controls
DockExtender
vssver2.scc
vssver2.scc
Debug
vssver2.scc
Examples
BNFGrammar v0.2.tpg
BNFGrammar v1.0.tpg
BNFGrammar v1.1.tpg
BNFGrammar_vb v1.1.tpg
GrammarHighlighter.tpg
GrammarHighlighter_vb.tpg
simple expression1.tpg
simple expression1_vb.tpg
simple expression2.tpg
simple expression2_vb.tpg
vssver2.scc
Highlighter
GrammarHighlighter v1.2.tpg
vssver2.scc
mssccprj.scc
Properties
vssver2.scc
Templates
C#
vssver2.scc
VB
vssver2.scc
vssver2.scc
TinyPG.csproj.user
TinyPG.suo
vssver2.scc
TinyPG_v1_3_binaries.zip
BNFGrammar 1.3.tpg
BNFGrammar_vb 1.3.tpg
GrammarHighlighter v1.3.tpg
GrammarHighlighter_vb.tpg
simple expression1.tpg
simple expression2.tpg
TinyPG.exe
TinyPG.vshost.exe
TinyPG.vshost.exe.manifest
TinyPG_v1_3_sources.zip
TinyPG v1.3
TinyPG
bin
Debug
Examples
BNFGrammar 1.3.tpg
BNFGrammar v1.0.tpg
BNFGrammar v1.1.tpg
BNFGrammar_vb 1.3.tpg
GrammarHighlighter v1.3.tpg
GrammarHighlighter.tpg
GrammarHighlighter_vb.tpg
simple expression1.tpg
simple expression2.tpg
Templates
C#
VB
TinyPG.exe
TinyPG.pdb
TinyPG.vshost.exe
Release
Examples
BNFGrammar 1.3.tpg
BNFGrammar_vb 1.3.tpg
GrammarHighlighter v1.3.tpg
GrammarHighlighter_vb.tpg
simple expression1.tpg
simple expression2.tpg
Templates
C#
VB
TinyPG.exe
TinyPG.pdb
TinyPG.vshost.exe
TinyPG.vshost.exe.manifest
CodeGenerators
CSharp
vssver2.scc
VBNet
vssver2.scc
vssver2.scc
Compiler
BNFGrammar 1.3.tpg
vssver2.scc
Controls
DockExtender
vssver2.scc
vssver2.scc
Debug
vssver2.scc
Examples
BNFGrammar 1.3.tpg
BNFGrammar_vb 1.3.tpg
csharpstruct.tpg
GrammarHighlighter v1.3.tpg
GrammarHighlighter_vb.tpg
simple expression1.tpg
simple expression1_vb.tpg
simple expression2.tpg
simple expression2_vb.tpg
vssver2.scc
Highlighter
GrammarHighlighter v1.3.tpg
vssver2.scc
mssccprj.scc
obj
Debug
TempPE
Properties.Resources.Designer.cs.dll
TinyPG.Controls.AutoComplete.resources
TinyPG.Controls.RegExControl.resources
TinyPG.csproj.GenerateResource.Cache
TinyPG.exe
TinyPG.MainForm.resources
TinyPG.pdb
TinyPG.Properties.Resources.resources
Release
TempPE
Properties.Resources.Designer.cs.dll
TinyPG.Controls.AutoComplete.resources
TinyPG.Controls.RegExControl.resources
TinyPG.csproj.GenerateResource.Cache
TinyPG.exe
TinyPG.MainForm.resources
TinyPG.pdb
TinyPG.Properties.Resources.resources
Properties
vssver2.scc
Templates
C#
vssver2.scc
VB
vssver2.scc
vssver2.scc
TinyPG.csproj.user
TinyPG.suo
vssver2.scc
TinyPG.UnitTests
bin
Release
Examples
BNFGrammar 1.3.tpg
BNFGrammar v1.0.tpg
BNFGrammar v1.1.tpg
BNFGrammar_vb 1.3.tpg
GrammarHighlighter v1.3.tpg
GrammarHighlighter_vb.tpg
simple expression1.tpg
simple expression2.tpg
Templates
C#
VB
TinyPG.exe
TinyPG.UnitTests.dll
localtestrun.testrunconfig
mssccprj.scc
Properties
Testfiles
BNFGrammar v1.1.tpg
BNFGrammar_vb v1.1.tpg
GrammarHighlighter.tpg
GrammarHighlighter_vb.tpg
simple expression1.tpg
simple expression1_vb.tpg
simple expression2.tpg
simple expression2_vb.tpg
vssver2.scc
Testfiles
BNFGrammar v1.1.tpg
BNFGrammar_vb v1.1.tpg
GrammarHighlighter.tpg
GrammarHighlighter_vb.tpg
simple expression1.tpg
simple expression1_vb.tpg
simple expression2.tpg
simple expression2_vb.tpg
vssver2.scc
TestResults
TinyPG.testrunconfig
TinyPG.UnitTests.csproj.user
vssver2.scc
// Generated by TinyPG v1.2 available at www.codeproject.com

using System;
using System.Collections.Generic;
using System.Text;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using System.Drawing;
using System.Threading;

namespace <%Namespace%>
{
    // Summary:
    //     System.EventArgs is the base class for classes containing event data.
    [Serializable]
    [ComVisible(true)]
    public class ContextSwitchEventArgs : EventArgs
    {
        public readonly ParseNode PreviousContext;
        public readonly ParseNode NewContext;

        // Summary:
        //     Initializes a new instance of the System.EventArgs class.
        public ContextSwitchEventArgs(ParseNode prevContext, ParseNode nextContext)
        {
            PreviousContext = prevContext;
            NewContext = nextContext;
        }
    }

    // delegate for firing context switch events
    public delegate void ContextSwitchEventHandler(object sender, ContextSwitchEventArgs e);

    /// <summary>
    /// Takes control over the RichTextBox and will color the text accoording to the rules of the parser and the scanner
    /// this control extender will also support Undo/Redo functionality.
    /// </summary>
    public class TextHighlighter : IDisposable
    {
        private class UndoItem
        {
            /// <summary>
            /// contains the information for an undo/redo action
            /// </summary>
            /// <param name="text">the full text to be undone/redone</param>
            /// <param name="position">position of the caret after the un/redo action</param>
            /// <param name="scroll">position of the scrollbars after un/redo action</param>
            public UndoItem(string text, int position, Point scroll)
            {
                Text = text;
                Position = position;
                ScrollPosition = scroll;
            }

            public string Text;
            public int Position;
            public Point ScrollPosition;
        }


        // some winapís required
        [DllImport("user32", CharSet = CharSet.Auto)]
        private extern static IntPtr SendMessage(IntPtr hWnd, int msg, int wParam, IntPtr lParam);

        [DllImport("user32.dll")]
        private static extern bool PostMessageA(IntPtr hWnd, int nBar, int wParam, int lParam);

        [DllImport("user32.dll", CharSet = CharSet.Auto)]
        private static extern int GetScrollPos(int hWnd, int nBar);

        [DllImport("user32.dll")]
        private static extern int SetScrollPos(IntPtr hWnd, int nBar, int nPos, bool bRedraw);

        private const int WM_SETREDRAW = 0x000B;
        private const int WM_USER = 0x400;
        private const int EM_GETEVENTMASK = (WM_USER + 59);
        private const int EM_SETEVENTMASK = (WM_USER + 69);
        private const int SB_HORZ = 0x0;
        private const int SB_VERT = 0x1;
        private const int WM_HSCROLL = 0x114;
        private const int WM_VSCROLL = 0x115;
        private const int SB_THUMBPOSITION = 4;
        private const int UNDO_BUFFER = 100;

        private int HScrollPos
        {
            get { return GetScrollPos((int)Textbox.Handle, SB_HORZ); }
            set
            {
                SetScrollPos((IntPtr)Textbox.Handle, SB_HORZ, value, true);
                PostMessageA((IntPtr)Textbox.Handle, WM_HSCROLL, SB_THUMBPOSITION + 0x10000 * value, 0);
            }
        }

        private int VScrollPos
        {
            get { return GetScrollPos((int)Textbox.Handle, SB_VERT); }
            set
            {
                SetScrollPos((IntPtr)Textbox.Handle, SB_VERT, value, true);
                PostMessageA((IntPtr)Textbox.Handle, WM_VSCROLL, SB_THUMBPOSITION + 0x10000 * value, 0);
            }
        }

        // public shared members
        public ParseTree Tree;
        public readonly RichTextBox Textbox;

        // private members
        private Parser Parser;
        private Scanner Scanner;
        private IntPtr stateLocked = IntPtr.Zero;

        private int UndoIndex = -1;
        private List<UndoItem> UndoList;

        private ParseNode currentContext;
        public event ContextSwitchEventHandler SwitchContext;

        private Thread threadAutoHighlight;


        private void Do(string text, int position)
        {
            
            if (stateLocked != IntPtr.Zero) return;

            UndoItem ua = new UndoItem(text, position, new Point(HScrollPos, VScrollPos));
            UndoList.RemoveRange(UndoIndex, UndoList.Count - UndoIndex);
            UndoList.Add(ua);
            if (UndoList.Count > UNDO_BUFFER)
                UndoList.RemoveAt(0);

            // make undo/redo a little smarter, remove single strokes
            // reducing nr of undo states
            if (UndoList.Count > 7)
            {
                bool canRemove = true;
                UndoItem nextItem = ua;
                for (int i = 0; i < 6; i++)
                {
                    UndoItem prevItem = UndoList[UndoList.Count - 2 - i];
                    canRemove &= (Math.Abs(prevItem.Text.Length - nextItem.Text.Length) <= 1 && Math.Abs(prevItem.Position - nextItem.Position) <= 1);
                    nextItem = prevItem;
                }
                if (canRemove)
                {
                    UndoList.RemoveRange(UndoList.Count - 6, 5);
                }
            }
            UndoIndex = UndoList.Count;
        }

        public void ClearUndo()
        {
            UndoList = new List<UndoItem>();
            UndoIndex = 0;
        }

        public void Undo()
        {
            if (!CanUndo) return;

            UndoIndex--;
            if (UndoIndex < 1)
                UndoIndex = 1;

            // implement undo action here
            UndoItem ua = UndoList[UndoIndex-1];
            RestoreState(ua);
        }

        public void Redo()
        {
            if (!CanRedo) return;

            UndoIndex++;
            if (UndoIndex > UndoList.Count)
                UndoIndex = UndoList.Count;

            UndoItem ua = UndoList[UndoIndex-1];
            RestoreState(ua);

        }

        private void RestoreState(UndoItem item)
        {
            Lock();
            // restore state
            Textbox.Rtf = item.Text;
            Textbox.Select(item.Position, 0);
            HScrollPos = item.ScrollPosition.X;
            VScrollPos = item.ScrollPosition.Y;
            
            Unlock();
        }

        public bool CanUndo
        {
            get { return UndoIndex > 0; }
        }

        public bool CanRedo
        {
            get { return UndoIndex < UndoList.Count; }
        }

        public TextHighlighter(RichTextBox textbox, Scanner scanner, Parser parser)
        {
            Textbox = textbox;
            Scanner = scanner;
            Parser = parser;

            ClearUndo();

            //Tree = Parser.Parse(Textbox.Text);
            Textbox.TextChanged += new EventHandler(Textbox_TextChanged);
            textbox.KeyDown += new KeyEventHandler(textbox_KeyDown);
            Textbox.SelectionChanged += new EventHandler(Textbox_SelectionChanged);
            Textbox.Disposed += new EventHandler(Textbox_Disposed);

            SwitchContext = null;
            currentContext = Tree;

            threadAutoHighlight = new Thread(AutoHighlightStart);
            threadAutoHighlight.Start();
        }


        public void Lock()
        {
            // Stop redrawing:  
            SendMessage(Textbox.Handle, WM_SETREDRAW, 0, IntPtr.Zero);
            // Stop sending of events:  
            stateLocked = SendMessage(Textbox.Handle, EM_GETEVENTMASK, 0, IntPtr.Zero);
            // change colors and stuff in the RichTextBox  
        }

        public void Unlock()
        {
            // turn on events  
            SendMessage(Textbox.Handle, EM_SETEVENTMASK, 0, stateLocked);
            // turn on redrawing  
            SendMessage(Textbox.Handle, WM_SETREDRAW, 1, IntPtr.Zero);

            stateLocked = IntPtr.Zero;
            Textbox.Invalidate();
        }
  
        void textbox_KeyDown(object sender, KeyEventArgs e)
        {
            // undo/redo
            if (e.KeyValue == 89 && e.Control) // CTRL-Y
                Redo();
            if (e.KeyValue == 90 && e.Control) // CTRL-Z
                Undo();
        }

        void Textbox_TextChanged(object sender, EventArgs e)
        {
            if (stateLocked != IntPtr.Zero) return;

            Do(Textbox.Rtf, Textbox.SelectionStart);
            
            HighlightText();
        }

        void Textbox_SelectionChanged(object sender, EventArgs e)
        {
            if (stateLocked != IntPtr.Zero) return;

            if (SwitchContext == null) return;
            ParseNode newContext = GetCurrentContext();
            
            if (currentContext == null) 
                currentContext = newContext;
            if (newContext == null) return;

            if (newContext.Token.Type != currentContext.Token.Type)
            {
                SwitchContext.Invoke(this, new ContextSwitchEventArgs(currentContext, newContext));
                currentContext = newContext;
            }
            
        }

        /// <summary>
        /// this handy function returns the section in which the user is editing currently
        /// </summary>
        /// <returns></returns>
        public ParseNode GetCurrentContext()
        {
            ParseNode node = FindNode(Tree, Textbox.SelectionStart);
            return node;
        }

        private ParseNode FindNode(ParseNode node, int posstart)
        {
            if (node == null) return null;

            if (node.Token.StartPos <= posstart && (node.Token.StartPos + node.Token.Length) >= posstart)
            {
                foreach (ParseNode n in node.Nodes)
                {
                    if (n.Token.StartPos <= posstart && (n.Token.StartPos + n.Token.Length) >= posstart)
                        return FindNode(n, posstart);
                }
                return node;
            }
            else
                return null;
        }

        /// <summary>
        /// use HighlighText to start the text highlight process from the caller's thread.
        /// this method is not used internally. 
        /// </summary>
        public void HighlightText()
        {
            lock (treelock)
            {
                textChanged = true;
                currentText = Textbox.Text;
            }
        }

        // highlight the text (used internally only)
        private void HighlightTextInternal()
        {
            Lock();

            int hscroll = HScrollPos;
            int vscroll = VScrollPos;

            int selstart = Textbox.SelectionStart;

            HighlighTextCore();

            Textbox.Select(selstart,0);

            HScrollPos = hscroll;
            VScrollPos = vscroll;

            Unlock();
        }

        /// <summary>
        /// this method should be used only by HighlightText or RestoreState methods
        /// </summary>
        private void HighlighTextCore()
        {
            //Tree = Parser.Parse(Textbox.Text);
            StringBuilder sb = new StringBuilder();
            if (Tree == null) return;

            ParseNode start = Tree.Nodes[0];
            HightlightNode(start, sb);

            AddRtfHeader(sb);
            AddRtfEnd(sb);

            Textbox.Rtf = sb.ToString();

        }

        // thread start for the automatic highlighting
        private static object treelock = new object();
        private bool isDisposing;
        private bool textChanged;
        private string currentText;

        private void AutoHighlightStart()
        {
            ParseTree _tree;
            string _currenttext = "";
            while (!isDisposing)
            {
                bool _textchanged;
                lock (treelock)
                {
                    _textchanged = textChanged;
                    if (textChanged)
                    {
                        textChanged = false;
                        _currenttext = currentText;
                    }
                }
                if (!_textchanged)
                {
                    Thread.Sleep(200);
                    continue;
                }

                _tree = (ParseTree)Parser.Parse(_currenttext);
                lock (treelock)
                {
                    if (textChanged)
                        continue;
                    else
                        Tree = _tree; // assign new tree
                }
                
                Textbox.Invoke(new MethodInvoker(HighlightTextInternal));

            }
        }

        /// <summary>
        /// inserts the RTF codes to highlight text blocks
        /// </summary>
        /// <param name="node">the node to highlight, will be appended to sb</param>
        /// <param name="sb">the final output string</param>
        private void HightlightNode(ParseNode node, StringBuilder sb)
        {
            if (node.Nodes.Count == 0)
            {
                switch (node.Token.Type)
                {
<%HightlightTokens%>
                    default:
                        sb.Append(@"{{\cf0 ");
                        break;
                }
                sb.Append(node.Token.Text.Replace(@"\", @"\\").Replace("{", @"\{").Replace("}", @"\}").Replace("\n", "\\par\n"));
                sb.Append(@"}");
            }

            foreach (ParseNode n in node.Nodes)
            {
                HightlightNode(n, sb);
            }
        }

        // define the color palette to be used here
        private void AddRtfHeader(StringBuilder sb)
        {
            sb.Insert(0, @"{\rtf1\ansi\deff0{\fonttbl{\f0\fnil\fcharset0 Consolas;}}{\colortbl;<%RtfColorPalette%>}\viewkind4\uc1\pard\lang1033\f0\fs20");
        }

        private void AddRtfEnd(StringBuilder sb)
        {
            sb.Append("}");
        }

        void Textbox_Disposed(object sender, EventArgs e)
        {
            Dispose();
        }

        #region IDisposable Members

        public void Dispose()
        {
            isDisposing = true;
            threadAutoHighlight.Join(1000);
            if (threadAutoHighlight.IsAlive)
                threadAutoHighlight.Abort();
        }

        #endregion
    }
}

By viewing downloads associated with this article you agree to the Terms of Service and the article's licence.

If a file you wish to view isn't highlighted, and is a text file (not binary), please let us know and we'll add colourisation support for it.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Herre Kuijpers
Architect Capgemini
Netherlands Netherlands
Currently Herre Kuijpers is employed at Capgemini Netherlands for over 10 years, where he developed skills with all kinds of technologies, methodologies and programming languages such as c#, ASP.Net, Silverlight, VC++, Javascript, SQL, UML, RUP, WCF. Currently he fulfills the role of software architect in various projects.
 
Herre Kuijpers is a very experienced software architect with deep knowledge of software design and development on the Microsoft .Net platform. He has a broad knowledge of Microsoft products and knows how these, in combination with custom software, can be optimally implemented in the often complex environment of the customer.

| Advertise | Privacy | Mobile
Web01 | 2.8.140826.1 | Last Updated 21 Sep 2010
Article Copyright 2008 by Herre Kuijpers
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid