A Data Science Central Community
As a part of a series of posts discussing how a machine learning classifier works, I ran decision tree to classify a XY-plane, trained with XOR patterns or linearly separable patterns.
1. Simple (non-overlapped) XOR pattern
It worked well. Its decision boundary was drawn almost perfectly parallel to the assumed true boundary, i.e. XY axes.
2. Complex (overlapped) XOR pattern without pruning
Awful result, it appears to never follow the true boundary.
3. Complex XOR pattern with pruning
Just a little improved, but it still appears to be overfitted.
4. Two-classes linearly separable pattern
Never parallel to the true boundary.
5. Three-classes linearly separable pattern
Even worse... it appears to get more overfitted than the case of 2-classes.
Throughout these experiments, I found decision tree alone is easy to get overfitted; it obviously requires any further additional methods to get generalized, e.g. ensemble learning.
Join Data Science Central to comment on this post.