Regularization is the one of the central ideas on the Machine Learning. Without it we often can get overfitting. Consider, please, using of Least Squares method to perform regression for 10 points. Using polynom of degree 10 we can support error = 0, but for new point error will be large. Regularization avoids this situation.