Subscribe to DSC Newsletter

Decision tree vs. linearly separable or non-separable pattern

As a part of a series of posts discussing how a machine learning classifier works, I ran decision tree to classify a XY-plane, trained with XOR patterns or linearly separable patterns.

1. Simple (non-overlapped) XOR pattern

It worked well. Its decision boundary was drawn almost perfectly parallel to the assumed true boundary, i.e. XY axes.

2. Complex (overlapped) XOR pattern without pruning

Awful result, it appears to never follow the true boundary.

3. Complex XOR pattern with pruning

Just a little improved, but it still appears to be overfitted.

4. Two-classes linearly separable pattern

Never parallel to the true boundary.

5. Three-classes linearly separable pattern

Even worse... it appears to get more overfitted than the case of 2-classes.

Throughout these experiments, I found decision tree alone is easy to get overfitted; it obviously requires any further additional methods to get generalized, e.g. ensemble learning.

click here to read full article.

Join Data Science Central to comment on this post.

Views: 4385


You need to be a member of AnalyticBridge to add comments!

Join AnalyticBridge

On Data Science Central

© 2021   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service