Low Bias high variance
Question
You came to know that your model is suffering from low bias and high variance. Which algorithm should you use to tackle it? Why?
in progress
0
Interview Question
4 years
3 Answers
1466 views
Contributor 0
Answers ( 3 )
we should use regularization since these methods use shrinking coefficients techniques to increase bias thereby decreasing variance. we can also perform feature selection
1) If we are dealing with Linear Regression, then we can try Ridge and Lasso regression as they
make the target variable less sensitive to the changes in the independent variables.
2) If we are using decision trees, they are associated with high variance by the nature itself by
which they make predictions, they we should try ensemble methods like Random Forest as
they build many trees and average out the results which help in reducing the variance.
3) If you are using Support Vector Machines, you can make the margin classifier wide enough
by tuning the cost parameter.
So, at the end of the day it is not about using a specific algorithm when the model is suffering from
high variance, it is about fine tuning the model using same algorithm to reduce the variance.
Every model you apply has hyper-parameters which can be tuned so as to control the learning process.
This can help you in reducing variance.
If our model is suffering from low bias and high variance then our model is suffering from overfitting. To prevent overfitting, we can use regularization L1 or L2. If we are using a neural network we can introduce dropout. We can also use early stopping to prevent overfitting.