Click here to Skip to main content
Licence 
First Posted 21 Oct 2003
Views 139,107
Bookmarked 51 times

ID3 Decision Tree Algorithm in C#

By | 21 Oct 2003 | Article
Sample of ID3 Decision Tree Algorithm in C#

Introduction

The algorithm ID3 (Quinlan) uses the method top-down induction of decision trees. Given a set of classified examples a decision tree is induced, biased by the information gain measure, which heuristically leads to small trees. The examples are given in attribute-value representation. The set of possible classes is finite. Only tests, that split the set of instances of the underlying example languages depending on the value of a single attribute are supported.

Details

Depending on whether the attributes are nominal or numerical, the tests either

  • have a successor for each possible attribute value, or
  • split according to a comparison of an attribute value to a constant, or depending on if an attribute value belongs to a certain interval or not.

The algorithm starts with the complete set of examples, a set of possible tests and the root node as the actual node. As long as the examples propagated to a node do not all belong to the same class and there are tests left,

  • a test with highest information gain is chosen,
  • the corresponding set of successors is created for the actual node,
  • each example is propagated to the successor given by the chosen test,
  • ID3 is called recursively for all successors.

How it works

The core of sample is builded with 3 classes (Attribute, TreeNode and DecisionTreeID3).

  • TreeNode - are the nodes of the decision tree;
  • Attribute - is the class with have a name e any possible values;
  • DecisionTreeID3 - is the class what get a data samples and generate a decision tree.

License

This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here

About the Author

Roosevelt

Web Developer

Brazil Brazil

Member



Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board. (secure sign-in)
 
Search this forum  
 FAQ
    Noise  Layout  Per page   
  Refresh
Questionneed help in ID3 Decision Tree Algorithm in C# Pinmemberramireddyrasagna17:29 6 Feb '12  
Questionhelp in id3 decision tree algorithm in c# Pinmemberramireddyrasagna11:26 5 Feb '12  
GeneralNice code....but this code just for two class decision [modified] Pinmemberh4ckm03d2:24 27 Jul '11  
Generalhelp,id3 decision tree Pinmemberdhanyamc21:06 7 Jun '11  
Generalhelp id3 decision tree Pinmemberdhanyamc8:22 7 Jun '11  
GeneralRebuilt based on this code PinmemberTim Claason7:28 1 Feb '11  
Generalspliting criterion Pinmemberjamilehy12:04 2 Oct '10  
QuestionHow you defined new attributes in main? Pinmemberjamilehy17:28 30 Sep '10  
AnswerRe: How you defined new attributes in main? Pinmemberpol_chu9:31 2 Oct '10  
GeneralRe: How you defined new attributes in main? Pinmemberjamilehy11:40 2 Oct '10  
GeneralRe: How you defined new attributes in main? Pinmemberpol_chu19:16 2 Oct '10  
GeneralAn error report [modified] Pinmemberpol_chu22:44 9 Sep '10  
Generalerror Pinmemberlqht8:05 25 Apr '10  
GeneralError Pinmemberstrepto16:21 25 Apr '10  
Generalhi Pinmemberrodalyn18:15 15 Mar '10  
Generalhelp:id3 decision tree Pinmemberstd_11:34 12 Mar '10  
GeneralRe: help:id3 decision tree PinmemberSamBarakaba22:36 15 Oct '10  
GeneralRe: help:id3 decision tree Pinmemberleavetrace23:08 24 Oct '10  
GeneralRe: help:id3 decision tree Pinmemberleavetrace23:13 24 Oct '10  
GeneralRe: help:id3 decision tree Pinmemberleavetrace23:16 24 Oct '10  
GeneralRe: help:id3 decision tree Pinmemberleavetrace23:18 24 Oct '10  
GeneralMy vote of 1 PinmemberCode Deamon4:27 20 Apr '09  
Questioncan not work with more than two class Pinmemberleavetrace18:50 1 Jan '09  
GeneralI think there's a mistake, correct me if i'm wrong Pinmemberrock_me6:56 1 Jul '08  
I have changed your code into VB language, cause my task is in VB. I think there's no big difference. This is my training data.

        '===================================================================================
        'Exam Result Sample
        '===================================================================================
        col = result.Columns.Add("NFails")
        col.DataType = GetType(String)
 
        col = result.Columns.Add("NMarg")
        col.DataType = GetType(String)
 
        col = result.Columns.Add("Att")
        col.DataType = GetType(String)
 
        col = result.Columns.Add("Ext")
        col.DataType = GetType(String)
 
        col = result.Columns.Add("Ant")
        col.DataType = GetType(String)
 
        col = result.Columns.Add("Result")
        col.DataType = GetType(String)
 
        result.Rows.Add(New Object() {"0", "0", "good", "no", "P", "P"})
        result.Rows.Add(New Object() {"0", "0", "poor", "yes", "F", "P"})
        result.Rows.Add(New Object() {"0", "0", "good", "yes", "F", "P"})
        result.Rows.Add(New Object() {"3", "0", "good", "no", "F", "F"})
        result.Rows.Add(New Object() {"3", "1", "poor", "no", "F", "F"})
 
        result.Rows.Add(New Object() {"3", "0", "good", "no", "P", "F"})
        result.Rows.Add(New Object() {"3", "2", "good", "yes", "P", "R"})
        result.Rows.Add(New Object() {"2", "1", "poor", "no", "F", "F"})
        result.Rows.Add(New Object() {"2", "2", "good", "yes", "P", "R"})
        result.Rows.Add(New Object() {"1", "0", "poor", "yes", "P", "R"})
 
        result.Rows.Add(New Object() {"1", "1", "good", "yes", "F", "R"})
        result.Rows.Add(New Object() {"1", "1", "good", "no", "F", "R"})
        result.Rows.Add(New Object() {"1", "0", "poor", "no", "F", "F"})
        result.Rows.Add(New Object() {"3", "2", "good", "no", "P", "F"})
        result.Rows.Add(New Object() {"2", "2", "good", "no", "F", "R"})
 
        result.Rows.Add(New Object() {"2", "1", "good", "yes", "P", "R"})
        result.Rows.Add(New Object() {"2", "0", "poor", "no", "F", "F"})
       '===================================================================================
 
in function InternalMountTree, there are some lines :
 
            If aSample.Rows.Count = 0 Then
                Return New TreeNode(New Attribute(getMostCommonValue(samples, targetAttr)))
            Else
                Dim dc3 As DecisionTreeID3 = New DecisionTreeID3()
                Dim ChildNode As TreeNode = dc3.mountTree(aSample, targetAttr, _
                DirectCast(aAttributes.ToArray(GetType(Attribute)), Attribute()))
                root.addTreeNode(ChildNode, value)
            End If
 
there's no need to return the treenode, just add a new treenode so it doesn't break the recursive way. The codes become like this :
 
            If aSample.Rows.Count = 0 Then
                Dim ChildNode As New TreeNode(New Attribute(getMostCommonValue(samples, targetAttr)))
                root.addTreeNode(ChildNode, value)
                'Return New TreeNode(New Attribute(getMostCommonValue(samples, targetAttr)))
            Else
                Dim dc3 As DecisionTreeID3 = New DecisionTreeID3()
                Dim ChildNode As TreeNode = dc3.mountTree(aSample, targetAttr, _
                DirectCast(aAttributes.ToArray(GetType(Attribute)), Attribute()))
                root.addTreeNode(ChildNode, value)
            End If
 
Please correct me if i'm wrong because i've also changed some procedures and functions to match with my task. And you were so great about making this algorithm become so simple... Thank you so much! Smile | :)
Best Regards,
dee
QuestionDo u interested in association rule? Pinmemberalice_nhan4:39 27 Nov '07  

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Mobile
Web04 | 2.5.120529.1 | Last Updated 22 Oct 2003
Article Copyright 2003 by Roosevelt
Everything else Copyright © CodeProject, 1999-2012
Terms of Use
Layout: fixed | fluid