Lexical Analyzer In Java

See more:

Java

write the lexical analysis phase

See more: Java C#3.5
To keep it simple we will start with only:
• one variable type ﴾"int"﴿
• basic math (+, -, *, /)
• Print command to output results
(Basically it will be little more than a simple calculator).

must load in the source file ﴾as a command‐line argument﴿ and process it line-by-line, removing whitespace and comments and categorizing each word or symbol as a token. A token represents an atomic unit to be parsed, and is typically realized as one or a short series of characters in a source file such as "int", "+", or "42". You will then output ﴾to standard out﴿ a tokenized version of the file, as described in detail below.
If we get to develop our compiler then will maybe using the tools Flex and Bison. This project will only require the use of Flex, which will handle lexical analysis for us, taking as input a set of regular expressions associated with each token type.

Here is the contents of an example input file (Create syntax for selected language for source file; only for 4 lines of code below):

example1.java: (4 lines of code below)
Hide Copy Code

val test_num = 3 * (7.2 + 12.1); // This is my comment.
// The next line intentionally left blank!;

System.out.println(test_num);

Example2.java: (5 lines of code below, not scanning for bool should handle unknown token)
Hide Copy Code

val test_num = 3 * (7.2 + 12.1); // This is my comment.
// The next line intentionally left blank!;

System.out.println(test_num);
bool not_done = 0;

executable should produce:
Hide Copy Code

TYPE: val
ID: test_num
ASCII_CHAR: =
VAL_LITERAL: 3
ASCII_CHAR: *
ASCII_CHAR: (
VAL_LITERAL: 7.2
ASCII_CHAR: +
VAL_LITERAL: 12.1
ASCII_CHAR: )
ASCII_CHAR: ;

I know that I have to build arrays for the ASCII characters as well as the Literals...etc. My question is how do I get it to read from a text file, line by line?