What are the metrics to measure the performance of your Linear Regression model? Most asked question
Question
Almost all the interviewer asks this question
solved
0
Statistics
55 years
12 Answers
4210 views
Grand Master 0
Answers ( 12 )
There are many metrics to measure the performance of Linear Regression. Mostly used in Econometrics and Academics is R-squared and Restricted R-squared. However, there are many others used in Data Sciences, like Root Mean Squared Error. You can refer to the following link for many others in the metrics module of the Sklearn package. https://scikit-learn.org/stable/modules/model_evaluation.html#regression-metrics
Instead of Restricted R-squared, I meant is Adjusted R-squared.
The evaluation metrics varies based on the problem statement and requirements. For example extracting total amount from invoice using OCR, we can consider the R2 metrics because we are more concerned about the information explained by the model.
Apart from R2 and R2 adjusted (use to check significance of added variable, it considers degree of freedom), Root Mean Squared Error and Root Mean log squar d error.
One of the most common metric is R^2.
R^2 gives the measure of the amount of variance explained by the predictor variables which
is present in the target variable.
The other metrics include Mean Squared Error (MSE), Mean Absolute Error(MAE)
Mean Absolute Percentage Error(MAPE), Root Mean Squared Error (RMSE)
There are many metrics in the Data science community to measure the performance of a Machine learning Problem:
Confusion Matrix
F1 Score
Gain and Lift Charts
Kolmogorov Smirnov Chart
AUC – ROC
Log Loss
Gini Coefficient
Concordant – Discordant Ratio
Root Mean Squared Error
Cross Validation (Not a metric though!
R Squared/Adjusted R squared
Mostly we use the R Squared/ Adjusted R squared but that also depends upon the problem statement and requirements
-> Let us first understand what is R-squared:
R-squared or R2 explains the degree to which your input variables explain the variation of your output / predicted variable. So, if R-square is 0.8, it means 80% of the variation in the output variable is explained by the input variables. So, in simple terms, higher the R squared, the more variation is explained by your input variables and hence better is your model.
However, the problem with R-squared is that it will either stay the same or increase with addition of more variables, even if they do not have any relationship with the output variables. This is where “Adjusted R square” comes to help. Adjusted R-square penalizes you for adding variables which do not improve your existing model.
Hence, if you are building Linear regression on multiple variable, it is always suggested that you use Adjusted R-squared to judge goodness of model. In case you only have one input variable, R-square and Adjusted R squared would be exactly same.
Typically, the more non-significant variables you add into the model, the gap in R-squared and Adjusted R-squared increases.
Most commonly known evaluation metrics for linear regression model
1. R-squared (R2), which is the proportion of variation in the outcome that is explained by the predictor variables.
2.Root Mean Squared Error (RMSE), which measures the average error performed by the model in predicting the outcome for an observation.
3.Residual Standard Error (RSE), also known as the model sigma, is a variant of the RMSE adjusted for the number of predictors in the model. The lower the RSE, the better the model.
4.Mean Absolute Error (MAE), like the RMSE, the MAE measures the prediction error. Mathematically, it is the average absolute difference between observed and predicted outcomes, MAE = mean(abs(observeds – predicteds)).
There are many metrics to measure the performance of Linear Regression Model:
Root Mean Square Error
Mean Squared Error
Accuracy Score
ROC-AUC Score
R-square
Adjusted R square
Mean Absolute Percentage Error
Mean Absolute Error
Linear regression is a linear model, e.g. a model that assumes a linear relationship between the input variables (x) and the single output variable (y). We call the difference between the actual value and the model’s estimate a RESIDUAL.
1. MAE : Mean Absolute Error calculate the residual for every data point, taking only the absolute value of each so that negative and positive residuals do not cancel out. We then take the average of all these residuals
2. RMSE: Root mean squared error is the square root of the MSE. Because the MSE is squared, its units do not match that of the original output. RMSE converts the error metric back into similar units, making interpretation easier. Since the MSE and RMSE both square the residual, they are similarly affected by outliers
3. MAPE: The mean absolute percentage error is the percentage equivalent of MAE.
4. R-squared (R2): The proportion of variation in the outcome that is explained by the predictor variables. So, if R-square is 0.7, it means 70% of the variation in the output variable is explained by the input variables.
5. Adjusted R2: One of the pitfalls of the R-squared is that it will always improve as we increase the number of variables. The Adjusted R-Squared fixes this problem. It adds a penalty to the model. Notice that there are two additional variables in the model.
For the R-Squared and the Adjusted R-Squared, the closer the value to 1, the better is the performance of our model.
Generally, the most commonly used matrices, for measuring regression quality and comparing models are-
Adjusted R2, AIC, BIC, and Cp.
In the regression model, the most commonly known evaluation metrics include:
1) R-squared (R2), which is the proportion of variation in the outcome that is explained by the predictor variables. In multiple regression models, R2 corresponds to the squared correlation between the observed outcome values and the predicted values by the model. The Higher the R-squared, the better the model.
2) Root Mean Squared Error (RMSE), which measures the average error performed by the model in predicting the outcome for an observation. Mathematically, the RMSE is the square root of the mean squared error (MSE), which is the average squared difference between the observed actual outcome values and the values predicted by the model. So, MSE = mean((observeds – predicteds)^2) and RMSE = sqrt(MSE). The lower the RMSE, the better the model.
3) Residual Standard Error (RSE), also known as the model sigma, is a variant of the RMSE adjusted for the number of predictors in the model. The lower the RSE, the better the model. In practice, the difference between RMSE and RSE is very small, particularly for large multivariate data.
4) Mean Absolute Error (MAE), like the RMSE, the MAE measures the prediction error. Mathematically, it is the average absolute difference between observed and predicted outcomes, MAE = mean(abs(observeds – predicteds)). MAE is less sensitive to outliers compared to RMSE.
The problem with the above metrics is that they are sensitive to the inclusion of additional variables in the model, even if those variables don’t have significant contribution in explaining the outcome. Put in other words, including additional variables in the model will always increase the R2 and reduce the RMSE. So, we need a more robust metric to guide the model choice.
performance metrics include sensitivity(recall), specificity, precision, f1-score, AUC, mean-squared-error, mean-absolute error, R², Adjusted R² etc and are used according to the task and data used for the task. In this article, we would discuss metrics
There are 3 main metrics for this:
1. R Square/Adjusted R Square
2. Mean Square Error(MSE)/Root Mean Square Error(RMSE)
3. Mean Absolute Error(MAE)
R Square/Adjusted R Square:
R Square measures how much of variability in dependent variable can be explained by the model. It is square of Correlation Coefficient(R) and that is why it is called R Square.
R Square is calculated by the sum of squared of prediction error divided by the total sum of square which replace the calculated prediction with mean. R Square value is between 0 to 1 and bigger value indicates a better fit between prediction and actual value.
R Square is a good measure to determine how well the model fits the dependent variables. However, it does not take into consideration of overfitting problem. If your regression model has many independent variables, because the model is too complicated, it may fit very well to the training data but performs badly for testing data. That is why Adjusted R Square is introduced because it will penalise additional independent variables added to the model and adjust the metric to prevent overfitting issue.
Mean Square Error(MSE)/Root Mean Square Error(RMSE):
While R Square is a relative measure of how well the model fits dependent variables, Mean Square Error is an absolute measure of the goodness for the fit.
MSE is calculated by the sum of square of prediction error which is real output minus predicted output and then divide by the number of data points. It gives you an absolute number on how much your predicted results deviate from the actual number.
Root Mean Square Error(RMSE) is the square root of MSE. It is used more commonly than MSE because firstly sometimes MSE value can be too big to compare easily. Secondly, MSE is calculated by the square of error, and thus square root brings it back to the same level of prediction error and make it easier for interpretation.
Mean Absolute Error(MAE):
Mean Absolute Error(MAE) is similar to Mean Square Error(MSE). However, instead of the sum of square of error in MSE, MAE is taking the sum of absolute value of error.
MSE gives larger penalisation to big prediction error by square it while MAE treats all errors the same.
The various metrics used to evaluate the performance of a Linear Regression model are:
1. Mean Squared Error(MSE)
2. Root-Mean-Squared-Error(RMSE).
3. Mean-Absolute-Error(MAE).
4. R² or Coefficient of Determination.
5. Adjusted R²