How to Design and Build the "Simplest Possible" Programming Language?

Quote:

I have always wished for my computer to be as easy to use as my telephone; my wish has come true because I can no longer figure out how to use my telephone.

Quote:

What is the simplest thing that could possibly work?

The article Fundamental Pragmatics for Successful Programmers states a key principle for designing software applications:

Before writing code, design your data carefully!

We should apply the same rule for language design. Let’s first think about data, before thinking about code.

Many programming languages provide the following data types:

• primitive data types: boolean, number (integer and float), character, string

• collections: list, set, map

• structures (records), such as a person structure composed of fields identifier, name, city

• null: the absence of a value

Some languages also provide enums, tuples, binary data, and maybe some more.

What should we do if we want to keep it as simple as possible?

It turns out that we just need one data type. Yes, one single data type: a list whose elements can be strings or lists. Put more simply: a tree of strings.

Here are some simple examples:

It is easy to show that all above mentioned data types can be represented by just using a tree of strings data type.

Let’s look at each case:

• Primitive data types can be represented by a list containing one string representing the value:

  // boolean
  [ "yes" ] [ "no" ] // we could also use "true" and "false", but let's stick with "yes" and "no"
  
  // numbers
  [ "123" ] [ "-123.45" ] 
  // character
  [ "a" ] 
  // string (or enum)
  [ "big" ] 

• Collections:

  • a list or a tuple is a tree with only leaf nodes:

    [ "a" "b" "a" ]

  • a set is a list with unique elements:

    [ "a" "b" "c" ]

  • a map can be represented by a list whose elements are lists with two elements providing a key/value pair:

    [ [ "1" "one" ] [ "2" "two" ] ]

• A structure can be represented with a map:

  [
     [ "name"       "Albert" ]
     [ "birthday"   "1879-03-14" ]
     [ "birthplace" "Ulm" ]
  ]

• null is represented by an empty list:

  []

• Binary data can be coded as strings containing hexadecimal characters

  [ "F0A1" "BC78" ]

To simplify coding let’s provide the following practical and intuitive syntax sugar:

• The surrounding square brackets can be omitted for lists containing one element. Instead of …​

  [ "Hello" ]

  ... we can simply write:

  "Hello"

• The surrounding quotes can be omitted for numbers. Instead of …​

  [ "1" "2" "3" ]

  ... we can simply write:

  [ 1 2 3 ]

• Commas can optionally be inserted between list elements. Instead of …​

  [ 1 12.34 5 ]

  ... we can write:

  [ 1, 12.34, 5 ]

• Elements can appear on separate lines and white space is ignored. Instead of …​

  [ 7456.2345 76.23 83676.1223 ]

  ... we can write:

  [
     7456.2345
       76.23
    83676.1223
  ]

Remember:

Simplix has one data type: a tree of strings.

Example: [ "a" "b" [ 1 2 ] "c" ]

Basic Syntax Rules

// a single line comment
const pi = 3.1415926 // TODO: check if precision ok

///
    multi
    line
    comment
.///

///
    comment
    ///
        nested
        comment
    .///
    blah blah blah
.///

const max_index = 100
var sum = 0

repeat for each long_variable_name \
    in long_expression_denoting_a_list

const cities = [ "London"
    "New York"
    "Paris" ]

const foo = bar (
    "long expression 1"
    "long expression 2" )

if condition then
    instruction
    instruction
.

color
make_delicious_coffee
XML_file_20

happy
Happy
HAPPY

Functions

function add ( op1 op2 ) -> sum

Services

Instructions

function foo ( input_1 input_2 ) -> result
    instruction
    instruction
    instruction
.

The usefulness of any programming language largely depends on the set of readily available libraries and frameworks. If we are lucky then we can quickly create impressive applications by just writing some glue-code to make existing software components work together seamlessly. Good libraries leverage our power to write quality software in a minimum of time.

Most programming languages are delivered with a standard library that provides very basic functionality needed for common tasks. The same must be done in Simplix. The standard library should provide useful functions for:

• working with strings and lists

• working with directories and files

• date/time functions

• input/output operations, such as reading from and writing to the OS’s standard input, output, and error devices, files, URLs, etc.

• interacting with the OS (running OS commands/scripts and external applications, getting command line arguments, getting and setting environment variables, exiting the OS process with success or with an error code)

Having a good standard library is important, but not enough. Depending on the application we want to write, we need much more: working with HTML, JSON, and XML documents, using network communications, creating and changing media files (images, sound, video), etc. etc. We do not want to re-invent the wheel and lose time writing this code ourselves. We want to use existing libraries.

To achieve this, we just need to apply an important rule mentioned in Fundamental Pragmatics for Successful Programmers / Part 2:

Use existing software whenever possible!

In the case of my proof-of-concept implementation of Simplix (more on this later) I decided to use Java as 'existing software'.

To make Simplix and Java work together, we just have to add two simple features:

• Allow Java source code to be embedded in Simplix code. Here is a simple example:

  function hello
  
      out.write_line ( "Hello from Simplix." )
  
      java
          // This is Java code!
          System.out.println ( "Hello from Java." );
      end java
  .

  Output:

  Hello from Simplix.
  Hello from Java.

• Allow any .jar file (Java package) to be used by a Simplix application, by just copying it into a lib directory.

This increases the usefulness of Simplix tremendously. We can now:

• use any existing Java software in a Simplix application. The Java software can be written in any Java Virtual Machine (JVM) language, such as Java, Scala, Clojure, PPL, etc.

• call Java code from Simplix code, call Simplix code from Java code, and exchange in-memory data between Simplix and Java

• write parts of a Simplix application in another JVM language, or even in C, via Java’s native interface

• use a Simplix application or library in another JVM application

An additional advantage is that writing Simplix’s standard library now becomes very easy. In fact, the standard library is just a wrapper around existing Java code. Here is an example of a Simplix matches_regex function that takes a string and a regex as input, and tells us if the string matches the regex:

Note the Conversions class that acts as a Simplix/Java bridge. It contains methods to convert Simplix data to Java data and vice-versa.

A nice side effect of using Java as target environment is that Java’s slogan "write once, run anywhere" now becomes true for Simplix too: a Simplix application can run on any system that supports Java: Mac, Unix/Linux, Windows and embedded devices.

Remember:

Simplix is delivered with a standard library.

Java libraries (.jar files) can be used in a Simplix application.

Let us now see how to create and run applications.

Note: If you want to try out the steps below, you must first install Simplix.

Run

sim run

Hello from test_app.

C:\tests\simplix>sim create_app test_app
C:\tests\simplix>cd test_app
C:\tests\simplix\test_app>sim run
Hello from test_app.

Develop

The result of compiling a Simplix application is a standard Java .jar file in the application’s run directory (e.g. run/my_app.jar).

You can use any existing Java deployment technique to deploy and distribute your Simplix application as a one-click installation procedure (not discussed here).

Note that you don’t need to install Simplix on the user’s machine and you don’t need to distribute the Simplix source code. Only the standard Java Runtime Environment (JRE) needs to be installed (if not done already), and the .jar files produced by the Simplix compiler must be deployed.

At the time of writing, the total size for a small application like those we will see later is about 50 kB (including Simplix’s standard library, but excluding the Java runtime).

To write a simple 'Hello world' application, open a terminal in the directory of your choice, and type:

This command creates file hello_app/src/start.sim. Open this file with your preferred editor and modify it so that it looks like this:

Explanations: out.write_line denotes function write_line that is defined in service out. Service out is part of Simplix’s standard library and contains functions to work with the OS’s standard output device. "Hello world" is the string that is passed as input to function write_line.

To run the application, go to directory hello_app (the application’s root directory) and type:

Output in your terminal:

Here is a print-screen of the terminal session on Windows:

If you prefer a GUI version of the program, you can change the code to:

Running this code displays:

After writing the Simplix compiler (more on that later), I wanted to know how many lines of code were needed. So I wrote a tool to do that - in Simplix.

I started with the following function that counts the lines of code in a single file:

1. The function takes a file path as input and returns the number of non empty lines in the file.

2. Variable line_count is initialized to zero.

3. file_reader.read_lines (i.e. function read_lines in service file_reader, part of Simplix’s standard library) reads the lines in file file. We use a loop to iterate over them. Each line will be contained in variable line.

4. We use function string.matches_regex (part of Simplix’s standard library) to check if the line is not empty.

5. In case of a non-empty line we increment line_count. Note that we cannot write line_count = line_count + 1, because Simplix doesn’t support operators (at least not in it’s current version).

6. At the end of the loop, line_count contains the number of non-empty lines, which is returned as the function’s result.

public static long countNonEmptyLinesInFile ( File file ) throws IOException {

    return Files.lines ( file.toPath() )
        .filter ( line -> line.matches ( ".*[^ \\t].*" ) )
        .count();
}

The above function can be called like this:

Note: You can use Windows or Unix path separators (\ or /) in directory paths. They are implicitly converted to the correct character, depending on the OS you are using. If you prefer to use the Windows backslash, then you must escape it (as in C, C#, Java, etc.): "C:\\tests\\simplix\\name.txt".

The next step is to write function count_non_empty_lines_in_files_of_directory, which takes a directory path as input, and returns the total number of lines found in all files contained in the directory tree.

We can then call this function from the start function. The whole code looks like this:

In the above example we use the application to count the lines of code in itself (C:/tests/simplix/count_lines_of_code/src). The result looks like this:

An obvious improvement of the tool would be to allow the directory (or a list of directories) to be specified as a command line argument. To make the tool even more versatile, the regex could also be provided as an optional command line argument. The default value would be the one shown in the code above. We’ll see later how to work with command line arguments.

It must be easy to interact with the operating system (OS). We need the following essential operations:

• read from the standard input device and write to the standard output and error devices.

• read command line arguments provided when the Simplix application is started.

• set the exit code when the Simplix application exits (0 = success, other integer values represent an error code).

• call OS commands and scripts, and execute other applications installed on the system.

• get and set environment variables.

Let’s look at some examples.

Standard input/output/error

function start

    const n1 = in.read_line()                 (1)
    const n2 = in.read_line()
    out.write_line ( number.add ( n1 n2 ) )   (2)
.

function start

    const n1 = read_number()
    const n2 = read_number()
    const result = number.add ( n1 n2 )
    out.write_line ( [ n1 " + " n2 " = " result ] )
.

function read_number -> number

    out.write_line ( "Please enter a number and type <Enter>:" )
    const result = in.read_line()
    if not check.is_number ( result ) then                           (1)
        err.write_line ( [ "ERROR: " result " is not a number!" ] )  (2)
        process.exit_with_error ( 1 )                                (3)
    .
    return result
.

Please enter a number and type <Enter>:
12
Please enter a number and type <Enter>:
5
12 + 5 = 17

Please enter a number and type <Enter>:
123.45
Please enter a number and type <Enter>:
-0.45
123.45 + -0.45 = 123.00

Please enter a number and type <Enter>:
34598349857923847592345928345872398745
Please enter a number and type <Enter>:
9837459834959234957923748.983274598345987
34598349857923847592345928345872398745 + 9837459834959234957923748.983274598345987 = 34598349857933685052180887580830322493.983274598345987

Please enter a number and type <Enter>:
foo.12
ERROR: foo.12 is not a number!

cd run

echo 10> input.txt
echo 20>> input.txt
java -jar OS_in_out_err.jar < input.txt > output.txt 2> errors.txt

Please enter a number and type <Enter>:
Please enter a number and type <Enter>:
10 + 20 = 30

Command Line Arguments

function start
    display_arguments()                                                      (1)
.

function display_arguments

    const arguments = process.arguments()                                    (2)
    out.write_line ( [ "There are " list.size(arguments) " arguments." ] )

    out.write_lines ( arguments )                                            (3)

    // alternative using a loop
    repeat for each argument in arguments                                    (4)
        out.write_line ( argument )
    .
.

There are 0 arguments.

@echo off
cd run
java -jar OS_CL_arguments.jar arg1 "C:\Program Files\Test" "" 123

There are 4 arguments.
arg1
C:\Program Files\Test

123
arg1
C:\Program Files\Test

123

Process Exit Code

function start

    const ok = GUI_dialogs.ask_yes_no ( "Is everything ok?" )

    if ok then
        process.exit_with_success()
    else
        process.exit_with_error ( 2 )
    .
.

@echo off

cd run

java -jar OS_exit_code.jar
set EXIT_CODE=%ERRORLEVEL%
if %EXIT_CODE% EQU 0 goto success
if %EXIT_CODE% NEQ 0 goto failure

:success
echo Success
goto end

:failure
echo Failure code %EXIT_CODE%
goto end

:end

Success

Failure code 2

Calling External Programs

function start

    OS_command.start_and_continue ( [
        "notepad.exe"
        "C:/tests/Simplix/config.txt" ] )
.

OS_command.run_and_wait ( [ "cmd" "/c" "dir" ] )

OS_command.start_and_contine ( [ "cmd" "/c" "dir" ">" "dir.txt" ] )

function start

    const result = OS_command.run_and_wait ( "C:/tests/Simplix/my_backup.bat" )
    if number.equals ( result 0 ) then
        out.write_line ( "Backup ok." )
    else
        const message = [ "ERROR: Backup could not be done (error " result ")." ]
        err.write_line ( message )
        GUI_dialogs.error ( message )
    .
.

@echo off
echo Doing backup
exit 3

OS_command.run_and_wait ( [ "file.bat", "p1", "p 2" "" 123 ] )

Besides interacting with the OS, being able to use and interact with Java software is one of the most useful and powerful features of Simplix, because now a huge set of battle-proven software is ready to be used in any Simplix application.

Suppose we need a web server. We can use Tomcat, one of the best web servers available. Let’s quickly see how to do this, without digging into details.

After create a new application, create directory lib in your application’s root directory. This directory can contain any number of .jar files to be used in the Simplix application.

The .jar files for Tomcat are available here. Download the 'embedded' version, and extract all files into directory lib (or a sub-directory of lib). Here is the list of files:

Now modify file src/start.sim so that it looks like this:

Directory C:\tests\Simplix\web_app contains the web application files. Create at least one index.html file with the following content:

Run the Simplix application, open a browser, and go to http://localhost:8080/:

You can now use the full power of Tomcat (Java Server Pages, servlets, etc.) to create whatever you want.

Let’s close this chapter with some code snippets showing other useful features.

Text File Operations

function demo

    const file = "test.txt"
    const data = [ "line_1", "line_2" ]
    file_writer.write_lines ( file, data )
    file_writer.append_line ( file, "line_3" )

    const lines = file_reader.read_lines ( file )
    repeat for each line in lines
        out.write_line ( line )
    .
.

line_1
line_2
line_3

Data Persistence

function storage_test

    const config = [
        [ "address"      "127.0.0.1" ]
        [ "port"         "8080" ]
        [ "default_page" "index.html" ]
    ]

    // store config to file config.txt
    const file = "config.txt"
    storage.store_to_file ( config, file )

    // load config from config.txt
    const restored_config = storage.restore_from_file ( file )

    assert.is_equal ( config, restored_config )
.

Structures

// create a new book
function create ( ISBN, title, author ) -> book
    return [ ISBN, title, author ]
.

// get a book's ISBN
function ISBN ( book ) -> ISBN
    return list.get ( book, 1 ) // get the first element stored in book
.

// get a book's title
function title ( book ) -> title
    return list.get ( book, 2 )
.

// get a book's author
function author ( book ) -> author
    return list.get ( book, 3 )
.

function to_string ( book ) -> string
    return [ "Book "
             title ( book )
             " by "
             author ( book ) ]
.

function test
    const Effective_Java = book.create ( "978-0134685991", "Effective Java", "Joshua Bloch" )
    const ISBN = book.ISBN ( Effective_Java )
    assert.is_equal ( ISBN, "978-0134685991" )
    out.write_line ( book.to_string ( Effective_Java ) )
.

Book Effective Java by Joshua Bloch

Data inspection

function demo
    const data = [ "a" "b" [ "b1" "b2" [ "b21" ] ] "c" ]
    out.write ( tree.to_string_tree ( data ) )
.

a
b
    b1
    b2
        b21
c

function demo
    const data = [ "a" "b" [ "b1" "b2" [ "b21" ] ] "c" ]
    tree_GUI.show_as_string_tree ( data )
.

function accept_constant_instruction -> SPL_constant_instruction_AST or null

    if not accept_keyword ( SPL_keywords.const ) then
        return null
    .

    const tokens = require_current_instruction_tokens
    const id = require_id on_null: return null
    require_symbol ( SPL_symbols.equal_sign )
    const expression = require_expression on_null: return null

    return SPL_constant_instruction_AST.create ( tokens, id, expression )
.

While Simplix’s type system is incredibly simple and very easy to learn and understand, it also has a number of serious drawbacks:

• It lacks expressiveness

  Every object in Simplix is potentially a tree of strings. Because there is just one type, there is no specific type information associated with an object reference (input/output argument, constant, variable). Hence, the kind of data stored in an object reference is only expressed by its name, not by its type.

  For example, look at the following function definition in Simplix:

  function make_coffee ( size, milk, sugar ) -> cup_of_coffee

  Now let’s look at a similar method definition in a C#/Java-like language:

  public CupOfCoffee makeCoffee ( CupOfCoffeeSize size, boolean milk, boolean sugar )

  There are a number of questions that arise only in the Simplix version. For example:

  • What are valid values for input argument size? Is it the amount of coffee in centi-liters? In the C#/Java version, we can lookup type CupOfCoffeeSize and see (for example) that it is an enumerated type with values small, medium, and large.

  • What are valid values for input arguments milk and sugar? Is it an amount or a boolean?

  • What is the data structure of the result returned? In Simplix, cup_of_coffee is just a tree of strings. The best we can do is to provide a service for structure cup_of_coffee, as demonstrated in a previous example. In the C#/Java version, the data structure can be looked up in the API of CupOfCoffee.

• It is error-prone

  As there is just one data type, the compiler cannot detect type incompatibility errors. For example, calling the above function make_coffee with the title of a movie assigned to input argument milk doesn’t produce a compiler error. The best we can do is to check the input argument in the function’s body (e.g. assert.is_yes_no ( milk )), so that at least a runtime error will be generated as soon as possible.

• In some cases, Simplix’s data structures are space- and time-inefficient

  Here are two striking examples:

  • Space-inefficiency:

    Storing a boolean value as a list that contains the string "yes" or "no" is a shameful waste of memory.

  • Time-inefficiency:

    Adding two numbers (e.g. number.add ( 2 3 )) involves three cumbersome steps:

    • The two numbers, stored as strings, must be parsed and converted into internal number representations

    • The numbers are added

    • The result is converted into a string that is inserted into a new list.

• There is no specific support for the 'absence of a value'

  The 'absence of a value' can be represented with an empty list in Simplix. But there is no distinction between nullable and non-nullable types. There is no language support to ensure that the 'absence of a value' is handled explicitly. Other languages provide compile-time null-safety, or the Maybe/Optional pattern to achieve this. In Simplix it is the programmer’s responsibility to handle the 'absence of a value' correctly.

If we write small applications not involving big data then the above limitations are often irrelevant. However, their relevance increases with the size of code, data, and the number of programmers working on the code base. Simplistic type systems are a bad fit for developing big applications. A well designed, powerful type system helps a lot and can dramatically improve productivity, maintainability, and reliability in a software development project.

To keep it simple, Simplix just provides the basic building blocks to write code: functions and instructions.

Hence, there are a number of features that experienced programmers will miss. Here are some examples:

• There are no operators (and hence no operator precedence rules). Instead of writing a = b + c, we have to write a = number.add ( b c )

• There are no higher-order functions. Functions in Simplix cannot receive functions as input values or return a function as result. Hence, there are no anonymous functions (lambdas) and no closures. This limits the ability to write generic, reusable code.

• There are no object oriented concepts like interfaces, classes, type inheritance and polymorphism.

• There are no advanced functional programming concepts supported, such as pattern matching, currying, etc.

• There is no built-in support for concurrent/parallel processing. These features require the use of Java libraries (standard or 3th party) or OS processes.

It goes without saying that the current proof-of-concept version of Simplix lacks a lot of features found in professional development environments: IDE support, unit testing, build tools, etc. etc.

Quote:

Everything should be as simple as possible, but not simpler!

What is the simplest thing that could possibly work?