Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
Wikipedia does has the definition of ID3 algorithm but it is very complex. Is there any link that explains the concept in a easy to follow manner.
Wikipedia has lots of complex theories and formulas. I did understand what is entropy. But I did not understand what is Gain. Your basic explanation or your pointers to some good links will definitely help me.
I think the Wiki page basically nails the notion, though a few examples would be helpful. You could check the U Florida page and the ICL pages on this topic. And when it comes to such topics (Machine Learning and Data Mining), you should always consider reading materials by Andrew Moore @ CMU which I find to be very helpful.
But also, here’s my two cents:
Entropy denotes the expect value of information (uncertainty) contained in a message. Since we are considering decision trees here, Gain is the difference in Entropy when we split on an attribute (i.e., the difference between the original Entropy and the weight average of the Entropy of the different branches). Since this is the reduced uncertainty, this is just the information we gained through the splitting on that attribute.
Also, we used Data Mining: A Tutorial-based Approach as text when we did data mining course, and that a a good and easy book.