This is part 1 in a multi-article series on building your own IDE. The series will show how to extend the Visual Studio Isolated Shell to create an IDE. Each article in the series will build on the previous article to add new features to the environment. This article will create the My C language service in C# using the Irony Compiler Construction Kit. The language was initially created in the ManagedMyC example that is included with the Visual Studio SDK. The SDK example uses the Managed Babel System.
The first download supplied with this article contains the completed My C language service sample discussed in this article. It requires Visual Studio 2008 Standard edition or higher, and the Microsoft Visual Studio 2008 SDK.
In order to run the solution:
A language service in Visual Studio provides language-specific support (syntax coloring and highlighting, statement completion, brace matching, parameter information tooltips, etc.). A language service is implemented by creating a Visual Studio Integration Package. A project template is provided when the Visual Studio SDK is installed.
Managed language services are created using the Managed Package Framework. In addition, the Managed Babel System uses MPF, and provides a scanner and parser, MPLex, and MPPG. MPLex and MPPG are a variation of the Garden Point Scanner Generator (GPLex) and the Garden Point Parser Generator (GPPG) created by the Queensland University of Technology. These tools take a LEX-like and YACC-like specification and generate a scanner and parser in C#.
The Irony Compiler Construction Kit combines the lexical and parser specification in one C# file. Also, since the file is already in C#, no code generation takes place. This creates a solution that is easier to debug and maintain.
The second download is a compressed file containing a template to generate a Visual Studio Integration Package for an Irony Language Service. In order to use the template, download and unzip the contents to a temporary location. The two DLL files, Wizard.dll and IronyLanguageServiceWizard.dll, should be placed in the Global Assembly Cache (C:\Windows\assembly). The third file, IronyLanguageServicePackage.zip, should be placed in the Visual Studio user templates directory, typically "My Documents\Visual Studio 2008\Templates\ProjectTemplates". This will install a Visual Studio template that can be accessed in the New Project dialog. It will appear in the My Templates section when C# is selected.
When creating a new Irony Language Service Package, a wizard is run to gather information specific to the language that is being created.
The first page requests basic information about the language. The company name will be used as the namespace for all the classes. Also, if your language supports multiple file extensions, separate each extension with a semicolon.
Next, the wizard requests the features your language will support in Visual Studio. Leaving the options checked does not create any additional work, because the necessary integration is already included in the package.
Package Load Keys (PLK) are required for VSPackages to load successfully. The Visual Studio SDK provides a Developer Load Key (DLK) that makes it possible for VSPackages to load without a PLK during development. If the language being developed will be used on machines without the Visual Studio SDK, a PLK is a must. A PLK can be generated here.
After running the Irony Language Service Wizard, there should be a Grammar.cs file open in the editor. This is where the Irony Grammar will be defined. The LEX and YACC files included with the ManagedMyC sample are being used for the language specification.
First, declare the terminals. Terminals include keywords, operators, comments, variables, etc.
CommentTerminal blockComment = new CommentTerminal("block-comment", "/*", "*/");
CommentTerminal lineComment = new CommentTerminal("line-comment", "//",
"\r", "\n", "\u2085", "\u2028", "\u2029");
NonGrammarTerminals.Add(blockComment);
NonGrammarTerminals.Add(lineComment);
NumberLiteral number = new NumberLiteral("number");
IdentifierTerminal identifier = new IdentifierTerminal("identifier");
Next, declare the non-terminals:
NonTerminal program = new NonTerminal("program");
NonTerminal declarations = new NonTerminal("declaration");
NonTerminal declaration = new NonTerminal("declaration");
NonTerminal simpleDeclarations = new NonTerminal("simple-declarations");
NonTerminal simpleDeclaration = new NonTerminal("simple-declaration");
NonTerminal semiDeclaration = new NonTerminal("semi-declaration");
NonTerminal parenParameters = new NonTerminal("paren-parameters");
NonTerminal parameters = new NonTerminal("parameters");
NonTerminal classOption = new NonTerminal("class-option");
NonTerminal variableType = new NonTerminal("variable-type");
NonTerminal block = new NonTerminal("block");
NonTerminal blockContent = new NonTerminal("block-content");
NonTerminal statements = new NonTerminal("statements");
NonTerminal statement = new NonTerminal("statement");
NonTerminal parenExpressionAlways = new NonTerminal("paren-expression-always");
NonTerminal parenExpression = new NonTerminal("paren-expression");
NonTerminal forHeader = new NonTerminal("for-header");
NonTerminal forBlock = new NonTerminal("for-block");
NonTerminal semiStatement = new NonTerminal("semi-statement");
NonTerminal arguments = new NonTerminal("arguments");
NonTerminal parenArguments = new NonTerminal("paren-arguments");
NonTerminal assignExpression = new NonTerminal("assign-expression");
NonTerminal expression = new NonTerminal("expression");
NonTerminal booleanOperator = new NonTerminal("boolean-operator");
NonTerminal relationalExpression = new NonTerminal("relational-expression");
NonTerminal relationalOperator = new NonTerminal("relational-operator");
NonTerminal bitExpression = new NonTerminal("bit-expression");
NonTerminal bitOperator = new NonTerminal("bit-operator");
NonTerminal addExpression = new NonTerminal("add-expression");
NonTerminal addOperator = new NonTerminal("add-operator");
NonTerminal multiplyExpression = new NonTerminal("multiply");
NonTerminal multiplyOperator = new NonTerminal("multiply-operator");
NonTerminal prefixExpression = new NonTerminal("prefix-expression");
NonTerminal prefixOperator = new NonTerminal("prefix-operator");
NonTerminal factor = new NonTerminal("factor");
NonTerminal identifierExpression = new NonTerminal("identifier-expression");
Then, define the rules for the non-terminals. Also, in Irony, the root non-terminal must be specified.
this.Root = program;
program.Rule = declarations;
declarations.Rule = MakeStarRule(declarations, declaration);
declaration.Rule
= classOption + variableType + identifier + parameters + block
| classOption + identifier + parenParameters + block
| variableType + identifier + parenParameters + block
| identifier + parenParameters + block
| simpleDeclaration;
simpleDeclarations.Rule = MakePlusRule(simpleDeclarations, simpleDeclaration);
simpleDeclaration.Rule = semiDeclaration + ";";
semiDeclaration.Rule
= semiDeclaration + "," + identifier
| classOption + variableType + identifier
| variableType + identifier;
parameters.Rule
= parameters + "," + variableType + identifier
| variableType + identifier;
parenParameters.Rule
= ToTerm("(") + ")"
| "(" + parameters + ")";
classOption.Rule
= ToTerm("static")
| "auto"
| "extern";
variableType.Rule
= ToTerm("int")
| "void";
block.Rule
= ToTerm("{") + "}"
| "{" + blockContent + "}";
blockContent.Rule
= simpleDeclarations + statements
| simpleDeclarations
| statements;
statements.Rule = MakePlusRule(statements, statement);
statement.Rule
= semiStatement + ";"
| "while" + parenExpressionAlways + statement
| "for" + forHeader + statement
| "if" + parenExpressionAlways + statement
| "if" + parenExpressionAlways + statement + "else" + statement;
parenExpressionAlways.Rule = parenExpression;
parenExpression.Rule = ToTerm("(") + expression + ")";
forHeader.Rule = "(" + forBlock + ")";
forBlock.Rule = assignExpression + ";" + expression + ";" + assignExpression;
semiStatement.Rule
= assignExpression
| "return" + expression
| "break"
| "continue";
arguments.Rule
= expression + "," + arguments
| expression;
parenArguments.Rule
= ToTerm("(") + ")"
| "(" + arguments + ")";
assignExpression.Rule
= identifier + "=" + expression
| expression;
expression.Rule
= relationalExpression + booleanOperator + expression
| relationalExpression;
booleanOperator.Rule
= ToTerm("&&")
| "||";
relationalExpression.Rule
= bitExpression + relationalOperator + bitExpression
| bitExpression;
relationalOperator.Rule
= ToTerm(">")
| ">="
| "<"
| "<="
| "=="
| "!=";
bitExpression.Rule
= addExpression + bitOperator + bitExpression
| addExpression;
bitOperator.Rule
= ToTerm("|")
| "&"
| "^";
addExpression.Rule
= multiplyExpression + addOperator + addExpression
| prefixExpression;
addOperator.Rule
= ToTerm("+") | "-";
multiplyExpression.Rule
= prefixExpression + multiplyOperator + multiplyExpression
| prefixExpression;
multiplyOperator.Rule
= ToTerm("*")
| "/";
prefixExpression.Rule
= prefixOperator + factor
| factor;
prefixOperator.Rule = ToTerm("!");
factor.Rule
= identifierExpression + parenArguments
| identifierExpression
| number
| parenExpression;
identifierExpression.Rule
= identifier
| identifierExpression + "." + identifier;
Last, the keywords are defined for the My C language. This is necessary for the syntax coloring. All terminals in the keyword list will be displayed as keywords in the code editor for Visual Studio.
this.MarkReservedWords("break", "continue", "else", "extern", "for",
"if", "int", "return", "static", "void", "while");
There are two options for testing the grammar: Grammar Explorer and Visual Studio Experimental Hive. The next two sections are a walkthrough in setting up the environment for either option.
One of the many nice things about Irony is that it includes a tool for testing Irony grammars, called Grammar Explorer.
To setup the environment, right-click the language service project and select Properties. Select the Debug tab and choose Start External Program, and point it to the Irony.GrammarExplorer executable in the Resources directory.
The first time Grammar Explorer is run, the grammar will need to be added. To add a grammar into Grammar Explorer, click the button next to the drop down and click Add Grammar from the menu.
Find the language service DLL and click Open.
The grammar should appear in the list and be selected, so the only thing to do here is click OK.
Once the grammar is added, it can be selected from the drop down. Now, it will display the grammar errors.
The shift-reduce conflict on the else is in the original YACC grammar, and is caught by MPPG when the ManagedMyC example is compiled. This is a common problem with C grammars. Here is an example demonstrating the conflict:
if (x>y) if (x<z) foo(); else bar();
The compiler faces the problem of deciding whether to reduce "if (x<z) foo();" to an if_statement and then shift the "else", or if the "else" should be shifted first: both would be equally valid under the grammar as stated, but reduction would result in this:
if (x>y)
{
if (x<z)
{
foo();
}
}
else
{
bar();
}
... associating the "else" with the first "if", whereas shifting the "else" produces...
if (x>y)
{
if (x<z)
{
foo();
}
else
{
bar();
}
}
... with the "else" associated with the second "if".
Irony provides a way to resolve this issue, which is presented below. However, if left alone, the grammar will still perform correctly. This is because the default behaviour of Irony is to shift, which produces the correct result in C and similar languages.
To setup the environment, right-click the language service project and select Properties. Select the Debug tab and choose Start External Program, and point it to the Visual Studio 2008 executable (default: C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE\devenv.exe). Also, add /ranu /rootsuffix Exp to the command line arguments.
Build and run the project in Visual Studio (F5). This will launch the experimental hive. To test the language service, open a file with the .myc extension, or create a text file and save it with the .myc extension. Two files were included in the demo project: short.myc and thing.myc.
This section covers additions or changes made to the My C sample grammar to demonstrate additional features of Irony.
This warning is displayed anytime a non-terminal's rule contains a single non-terminal. There are several ways to resolve this issue. First, consider whether the non-terminal is really necessary. If the non-terminal should be there, the issue can be resolved by merging the two non-terminals.
The My C grammar has a program non-terminal that is causing this warning. This non-terminal was included from the original YACC file, and was probably placed there for expansion. To resolve the warning, the program and the declarations non-terminals are merged. Note that the program non-terminal must follow declarations so that the assignment is made after its rule is initialized.
this.Root = program;
declarations.Rule = MakeStarRule(declarations, declaration);
//Must follow declarations so that assignment
//is made after rule is initialized.
program.Rule = declarations.Rule;
Irony includes a method for resolving shift-reduce conflicts, PreferShiftHere(). It allows you to tell the compiler where to shift. So, the error from earlier can be resolved by the following:
statement.Rule
= semiStatement + ";"
| "while" + parenExpressionAlways + statement
| "for" + forHeader + statement
| "if" + parenExpressionAlways + statement
| "if" + parenExpressionAlways + statement +
PreferShiftHere() + "else" + statement;
Irony also supports number prefixes and suffixes. For example, the C language uses the prefix 0x to specify hexadecimal numbers. It also uses the f suffix to designate a value as a float. This can be defined in Irony with the following:
NumberLiteral number = new NumberLiteral("number");
number.AddPrefix("0x", NumberFlags.Hex);
number.AddSuffixCodes("f", TypeCode.Single);
In YACC, brace pairs are matched for each rule.
Block
: OpenBlock CloseBlock {Match(@1, @2); }
| OpenBlock BlockContent CloseBlock {Match(@1, @3); }
;
In Irony, it is a little different. Any terminal that should be matched in Visual Studio is registered in the grammar with the function RegisterBracePair. The My C grammar registers parenthesis and curly braces.
this.RegisterBracePair("{", "}");
this.RegisterBracePair("(", ")");
Another strength of Irony is the ability to define operator precedence. The My C language gives a higher precedence to the multiply and divide operators.
this.RegisterOperators(1, "+", "-");
this.RegisterOperators(2, "*", "/");
When I first discovered Visual Studio language services, I used the Managed Babel System. After working with a YACC grammar, I quickly found that it was difficult to debug and would be hard to maintain in the future. Irony seems to solve both of these issues, and performs just as well. Irony also uses BNF expressions, so it was a very easy transition from YACC.
| You must Sign In to use this message board. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
