Does Bagging Help to Prevent Overfitting in Decision Trees?

Introduction

Decision trees are a class of machine learning algorithms well known for their ability to solve both classification and regression problems, and not to forget the ease of interpretation they offer. However, they suffer from overfitting and can fail to generalize well if not controlled properly.

In this article, we will discuss what is overfitting, to what extent a decision tree overfits the training data, why it is an issue, and how it can be addressed.

Then, we will get ourselves acquainted with one of the ensemble techniques i.e., bagging, and see if it can be used to make decision trees more robust.

We will cover the following:

• Create our regression dataset using NumPy.
• Train a decision tree model using scikit-learn.
• Understand what overfitting means by looking at the performance of the same model on the training set and test set.
• Discuss why overfitting is more common in non-parametric models such as decision trees (and of course learn what is meant by the term non-parametric) and how it can be prevented using regularization.
• Understand what bootstrap aggregation (bagging in short) is and how it can potentially help with overfitting.
• Finally, we will implement the bagging version of the decision tree and see if it helps or not

  Tags: Data Science Decision Tree Getting Started Machine Learning Overfitting

  Add Fav

Comment