Click here to Skip to main content
Click here to Skip to main content

Building an AI Chatbot using a Regular Expression Engine

, 21 Jun 2007
Rate this:
Please Sign up or sign in to vote.
This article describes how to build an AI Chatterbot using a popular, Regular Expression-based open source Chatterbot engine: Verbots

Screenshot - VerbotSDK.gif

Introduction

This article describes how to build an Artificial Intelligence (AI) Chatterbot using an open source, Regular Expression-based natural language processing engine called Verbot. Chatterbot technology is not new, but has been growing in popularity and usage as people experiment with new ways of interacting with computers. Chatterbots, Animated Characters, Speech Recognition and Voice Synthesis are being used to facilitate new human–computer interaction (HCI) channels.

Background

The original chatterbot, ELIZA was developed by Joseph Weizenbaum in 1966. ELIZA was a virtual psychotherapist that would attempt to engage the user by staying on topic and sounding concerned with his/her problems. In 1994, Michael Mauldin -- the creator of the first Verbot, Sylvie -- coined the term "ChatterBot." Since then, Verbots have evolved and matured into the current Verbot 4 version created by Conversive. In 2006, Conversive released the Verbot SDK as open source to allow people to easily create their own 'bots and embed them in their own applications. This article describes some of the basics behind the Verbot Engine SDK and how the SDK can be integrated into another application.

How the Verbot works

Before delving into code, I thought it would be useful to explain at a high level how the Verbot SDK works. All chatterbots have an idea of a "KnowledgeBase" or script, which tells the 'bot what keywords or phrases to recognize. The corresponding output is then sent to the user when a match is found. Verbot KnowledgeBases are made up of a collection of rules that are the fundamental building blocks of Verbot conversations. Rules must have one or more Inputs, which recognize what the user says. They must also have one or more Outputs, which are what the 'bot responds when an input in the rule is matched. When the Verbot Engine loads a KnowledgeBase file, it turns the simple text inputs found in the script into regular expressions that can then be used as the basis for matching.

For example, let's say we have an input which is:

What is your name?

The resulting regular expression generated by the engine looks like this:

^(|.*?\b|.*?\s)What\b.+?\bis\b.+?\byour\b.+?\bname(|\b.*?|\s.*?)$

This may be a bit daunting if you are unfamiliar with Regular Expression syntax. Even if you are familiar, it may still seem a bit cryptic. Basically, the engine has put wildcards between words, as well as at the start and end of the phrase. This regular expression will match to, "So, what the heck is your name?" but not to, "What's your name?" To match to this, we could use the Synonym feature in Verbots, which you can learn more about at their website along with other, more advanced scripting topics. Also, there is a good Regular Expression syntax reference over at dotnetcoders if you'd like to learn more about regex.

Verbot KnowledgeBases (.vkb extension) are XML files that define the rules, inputs and outputs of the KnowledgeBase. They may also reference other Verbot 4 file types such as Synonyms, Replacement Profiles and Code Modules. Verbots allows you to "compile" your VKB files into a more compact binary "Compiled KnowledgeBase" (.ckb extension) file format.

Testing the application

Execute the VerbotWindowsApplicationSample.exe file found within the VerbotWindowsApplicationSample\bin\Release folder. Once the application is running, choose the File -> Load... menu item and browse to the VerbotWindowsApplicationSample\Resources\sample.ckb file to open the sample KnowledgeBase. Once the file is loaded into the engine, you may interact with the 'bot. Try typing "notepad" to test out one of the rules included in the sample KnowledgeBase.

Notice how the program has started Windows Notepad. This is behavior coded into the Sample application to parse commands in the script. I'll talk more about this later.

Using the code

To use the Verbot Engine SDK inside your own application, simply include the Verbot4Library reference in your project, create an instance of the Verbot4Engine and call either AddCompiledKnowledgeBase or AddKnowledgeBase to load your 'bot's knowledge. To interact with the 'bot when user input comes in, create a State object to uniquely identify your user. Add the paths to the KnowledgeBases you want your user to interact with. Then call the GetReply method on the Verbot4Engine object to find a match in the loaded script(s).

Let's look at the getReply() function in our VerbotWinApp.cs file:

private void getReply()
{
    string stInput = this.inputTextBox.Text.Trim();
    this.inputTextBox.Text = "";
    Reply reply = this.verbot.GetReply(stInput, this.state);
    if(reply != null)
    {
        this.outputTextBox.Text = reply.Text;
        this.parseEmbeddedOutputCommands(reply.AgentText);
        this.runProgram(reply.Cmd);
    }
    else
        this.outputTextBox.Text = "No reply found.";
}

This method takes the text in the input text box and sends it to the engine's GetReply method. It gets a Reply object back, which we can then process. The Reply object has a member "Text" that we will just put in the output box. It also has an "AgentText" member, which is a version of the Text meant to be sent to a Text-To-Speech (TTS) engine. It has a "Cmd" (Command) member, too, that we can use for special processing as necessary.

Finally, take a look at the parseEmbeddedOutputCommands and runEmbeddedOutputCommand methods. These methods do custom processing based on our requirements. This is where the "run" command opens Notepad, as I mentioned above.

Depending on how your own application works, you can make customizations to how the Reply object is handled. For example, in an online 'bot engine you may want to process URLs by opening a popup window, etc.

Creating knowledge

There are some freely available tools to help you create a Verbot KnowledgeBase. Verbot GPL Editor and VCompiler are a couple of options to get you started. You can also use the Verbot Editor, which costs $10 to unlock with a registration code. If you'd like to create your own KnowledgeBase creation tool, the code is pretty straightforward. You may also want to look at the source code for the Verbot GPL Editor I mentioned above as a good reference. Here is a summary of the steps required to create a KnowledgeBase in code:

  1. First, create an instance of the KnowledgeBase class:
    KnowledgeBase kb = new KnowledgeBase();
  2. Then we'll want to create a rule to add to our KB:
    Rule r = kb.AddRule();
  3. Finally, for each rule you create you'll want to have at least one input and one output. We could manually create and add inputs and outputs or we could simply call the AddInput and AddOutput methods in the rule class:
    r.AddInput("Input Text", "");
    r.AddOutput("Output Text", "", "");

Now we've got a KnowledgeBase object with one rule that has one input and one output. Of course, in a real application we wouldn't want to hard-code the "Input Text" and "Output Text" fields and would probably want to allow the user to add these themselves, but you get the idea. Next, we probably want to save our KB to a file. To do this, we need an instance of the XMLToolbox class. Then we simply call the SaveXML method to save the file:

XMLToolbox xmlToolbox = new XMLToolbox(typeof(KnowledgeBase));
xmlToolbox.SaveXML(kb, @"C:\mykbpath.vkb");

Then we want to be able to load our KnowledgeBase into the Verbot Engine and be able to interact with it. Here is the code to do that:

Verbot4Engine engine = new Verbot4Engine();
KnowledgeBaseItem kbi = new KnowledgeBaseItem();
kbi.Fullpath = @"C:\";
kbi.Filename = "mykbpath.vkb";
engine.AddKnowledgeBase(kb, kbi);
State state = new State();
state.CurrentKBs.Add(@"C:\mykbpath.vkb");

Finally, to interact with the loaded knowledge bases just call the GetReply method on the Engine, as discussed above.

Further reading

If you are interested in learning more about Verbots, visit the Verbot community forums or browse Verbots wiki to learn more about scripting your own Verbot.

History

  • 22 March, 2007 -- First draft of article.
  • 28 March, 2007 -- Updated article for new version of SDK 4.1.3.2, which includes a simple console application as a sample and a new KnowledgeBase method, AddRule, which returns the new rule. Also, small changes in article instructions were made.
  • 21 June, 2007 -- Article edited and moved to the main CodeProject.com article base.

License

This article, along with any associated source code and files, is licensed under The GNU General Public License (GPLv3)

Share

About the Author

MattsterP
Web Developer
United States United States
Matt Palmerlee is a Software Engineer that has been working in the Microsoft.NET environment developing C# WebServices, Windows Applications, Web Applications, and Windows Services since 2003.

Comments and Discussions

 
Generalnice job Pinmembergreenknt16-Apr-09 4:44 
QuestionSynonyms - SDK (C#) PinmemberPhil Bartie18-Dec-08 11:18 
GeneralNeed help to load KB PinmemberDinesh Vitharana24-May-08 10:21 
AnswerRe: Need help to load KB PinmemberLars Brandt15-Aug-08 11:07 
GeneralLoad knowladge base Pinmemberahmedmakki14-May-08 4:19 
General[Message Deleted] PinmemberDanny Rodriguez27-Jan-08 9:12 
QuestionWannt integration with AIM, MSN, YAhoo??? Pinmembervijaymodi_8121-Jun-07 18:51 
QuestionDo they have rules here about people coming on and schleping their products? Pinmembersherifffruitfly21-Jun-07 15:47 
AnswerRe: Do they have rules here about people coming on and schleping their products? PinmemberMattsterP22-Jun-07 6:34 
GeneralRe: Do they have rules here about people coming on and schleping their products? Pinmembersherifffruitfly22-Jun-07 6:39 
GeneralRe: Do they have rules here about people coming on and schleping their products? PinmemberMattsterP25-Jun-07 10:14 
QuestionHow to create a rule/knowledgebase Pinmembermerlin98128-Mar-07 10:47 
AnswerRe: How to create a rule/knowledgebase [modified] PinmemberMattsterP28-Mar-07 12:59 
GeneralRe: How to create a rule/knowledgebase Pinmembermerlin98129-Mar-07 0:12 
Thank you for the fast and detailed response. One question: what are the conditions when adding a new rule? I see we can pass an empty string, but what else can be pass?
GeneralRe: How to create a rule/knowledgebase PinmemberMattsterP29-Mar-07 6:28 
GeneralRe: How to create a rule/knowledgebase Pinmembermerlin98129-Mar-07 9:01 
GeneralRe: How to create a rule/knowledgebase PinmemberMattsterP29-Mar-07 10:32 
GeneralRe: How to create a rule/knowledgebase PinmemberMattsterP29-Mar-07 15:01 
GeneralRe: How to create a rule/knowledgebase Pinmembermerlin98130-Mar-07 4:32 
AnswerRe: How to create a rule/knowledgebase Pinmemberchineng8731-Mar-10 18:51 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140827.1 | Last Updated 21 Jun 2007
Article Copyright 2007 by MattsterP
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid