Describe the mathematical mechanism of how regularization works for linear models.

Question

Anonymous · Accepted Answer

So Lasso and Ridge regression are used to prevent overfitting for the general
linear regression.in [http://regression.in] both these methods , we add an extra
penalty term to penalise the overfit linear regression equation and hence helps
in getting a more generalized answer. Ridge regression equation is given as cost
function + λ(slope)^2.Let's say we have a perfectly fit line with error = 0 and
slope = 2 and λ = 1.so [http://1.so] , the error = 4.so [http://4.so] , to
minimize the error , we reduce the slope to let's say slope = 1.5.Now
[http://1.5.Now] that we have reduced the slope to 1.5 , we get error as well ,
which is very small numerically. So [http://numerically.so] , the new error is
very small + 1(1.5)^2 which is close 2.5-3.So [http://2.5-3.So] ,we have reduced
the error.Lasso regression also works in the same way , but it additionally
helps in feature selection also. [http://also.in] Lasso regression equation is
given by cos function + λ*|slope|.Since we take the modulus of slope here , the
features having very small slopes will become 0 and irrelevant features will be
removed.

Describe the mathematical mechanism of how regularization works for linear models.

Practice this question with AI

Go Premium

Community Answers

Unlock Full Access

Practice More Questions