Syntax colorizing with Speller engine using TOM and CSAPI

• Download demo project source – 48.8 Kb
• Download demo project bin – 21.2 Kb

Introduction

The demo enhances a CRichEditCtrl that provides basic syntax highlighting for C/C++ or any other languages. The CRichEditCtrl also provides an interface that checks spelling using the CSAPI and gives a screen feedback by a coloring underline. This demo uses intensively the TOM interface. The check of spelling occurs only in the comments part of a source code or in strings. The control provides highlighting keywords not only in the source code but also for Doxygen in the comments.

Implementation

For a better implementation, the project has been organized into three classes. CSyntaxcolor manages comments, strings and keyword detection. CSpellingChecker manages the spelling of keywords and manages dictionaries. CSyntaxColorSpellChecker inherits from CSyntaxcolor and overloads the highlighting to provide a coloring underline feedback for the CRichEditCtrl.

Strategy

In order to avoid problems with the redraw of the control, I used Text Object Model (TOM) interface. This interface allows to manipulate runs of characters without sad effects on the screen, and gives more functions than the Win32 API. You can, for example, disable the undo during the modifications of colors. For the spelling, I used CSAPI. There is only some documentation about this API, but it eases considerably the implementation of a spell engine. It allows you to use many standardized dictionaries in your application. I am not sure if it is an official API from Microsoft.

Points of Interest

The main problem to handle syntax highlighting correctly is to manage the multi-line comments without re-computing each time colors of the complete file. The change of only one character could affect the entire file. Re-computing the complete file takes time and slows down the process of typing. To detect the modification of the multi-line comment structure, there is no clean solution. My solution has been to overload the notification EN_MSGFILTER of the CRichEditCtrl. When the user types a character able to change the state of the multi-line comments, I set a flag. And I do a fast analysis of the current line in order to know if the comment is opened or closed.

void CDemoEditorView::OnMsgfilterEdit(NMHDR* pNMHDR, LRESULT* pResult) 
{
  MSGFILTER *pMsgFilter = reinterpret_cast<MSGFILTER *>(pNMHDR);
  if(pMsgFilter->msg == WM_CHAR)
  {
    switch(pMsgFilter->wParam)
    {
      case '*':
      case '/':
        {
          m_bChangeAll = true;
          break;// '*' comment could affect all the file
        }
    }
    CHARRANGE cr;
    GetRichEditCtrl().GetSel(cr);
    if(cr.cpMax -cr.cpMin)// when we remove, it could remove '*/' or '/*'
    {
      m_bChangeAll = true;
    }
  }
  if(pMsgFilter->msg == WM_KEYDOWN)
  {
    switch(pMsgFilter->wParam)
    {
      case VK_BACK:
      case VK_DELETE:
      case VK_RETURN:
        m_bChangeAll = true;
        break;
    }
  }
  if(m_bChangeAll)
  {
    CHARRANGE cr;
    GetRichEditCtrl().GetSel(cr);
    m_nextedComments  = 
     m_colorizer.AnalyseMultiCommentLine(GetRichEditCtrl().LineFromChar(cr.cpMin));
  }
  *pResult = 0;
}

Using Colorizer

I consider the colorization of a text as a set of styles in a run of characters. A style includes all properties for a string (color, font, size, …). The ITextRange interface handles the concept of style through the functions GetSyle and SetStyle. Most of the time, this property is not used in a default implementation, but this LONG is stored in the run. I have stored in this value, the index of the highlight class. It is now very easy to quickly continue a multi line comment style when I evaluate only a part of the file. The style is stored directly in the run of characters. The main job of coloring is done in the function ColorizeRange(…). You can customize the keywords with the function AddKeyList(…). It will be easy to customize this class for HTML, XML, or other languages. You can customize the comments and string detection by changing CreateCommentsAndStringsTags(…). Also, you can change and add more highlight styles by changing CreateDefaultStyle().

void CSyntaxColor::CreateDefaultStyle()
{
    m_tStyle.SetSize(styleMax);
    // black for normal
    m_tStyle[normal]    = CKeyStyle(RGB(0,0,0));
    // blue for syntax
    m_tStyle[syntax]    = CKeyStyle(RGB(0,0,255));
    // Gray for directives
    m_tStyle[directive] = CKeyStyle(RGB(192,192,192));
    // blue for pragma
    m_tStyle[pragma]    = CKeyStyle(RGB(0,0,255));
    // green  multi comment green
    m_tStyle[mcomments] = CKeyStyle(RGB(0,200,0));
    // green  single comment green
    m_tStyle[scomments] = CKeyStyle(RGB(0,200,0));
    // blue string
    m_tStyle[sstring]   = CKeyStyle(RGB(0,0,255));
    // blue string
    m_tStyle[dstring]   = CKeyStyle(RGB(0,0,255));
    // red keyword comments and bold
    m_tStyle[highlightComments]= CKeyStyle(RGB(255,0,0),bold);
}

For each word in the NormalSlot or the CommentSlot, ColorizeRange calls OnColorKeyWord(int istart ,int iend ,LPCSTR pword, HighLightStyle style) to colorize a keyword.

Using the Speller

To implement the spelling checker in syntax colorizing, I have created a new class derived from CSyntaxcolor. The class includes an object CSpellingChecker and overload OnColorKeyWord(…). The new class continues the job of OnColorKeyWord done in CSyntaxcolor, and checks where the colorization occurs, checks the spelling, and sets the underline to signal a misspelled word.

bool CSyntaxColorSpellChecker::OnColorKeyWord(int istart, int iend, 
                          LPCSTR pword, HightLightStyle style)
{
    if(CSyntaxColor::OnColorKeyWord(istart,iend,pword,style)) return true;
    long newunderline = 0;
    if(style == scomments || style == mcomments || 
                             style == sstring || style == dstring )
    {
        newunderline = m_cSpeller.CheckWord(pword) ? 0: SPELL_UNDERLINE;
    }
    m_pRange->SetRange(istart,iend);
    ITextFont *pFont;
    m_pRange->GetFont(&pFont);
    pFont->SetUnderline(newunderline);
    pFont->Release();
    return false;
}

You can change the language of the speller by passing another LID to the function Initialize(lidFrench, lidDutch,…). Just make sure before that the dictionary and the spell engine is rightly installed in the registry. CSpellingChecker also provides some tools to manage dictionaries and words suggestion through a set of functions. Suggestion word comes not only from the main dictionary (.LEX) but also from your user dictionary.

bool CheckWord(LPCSTR pWord);
bool SuggestWords(LPCSTR pWord,int max, CStringArray & tList );
bool IgnoreAlways(LPCSTR pWord);
bool AddToUserDic(LPCSTR pWord);

It is possible to add more user dictionaries (read only) in the function InitEngine, if you modify the SIB structure for each call of SuggestWords and CheckWord. These extra dictionaries can be constructed automatically from your source code, in order to check the spelling of function names, variables, etc…

The Demo

The demo just implements the three classes in a text editor. I have added syntax highlighting and spell checking in comments to a basic application created by Visual C++. The editor is configured for C++ keywords highlight and Doxygen keywords in the comments.

History

• 11/11/2004 - first version.