What is regularization?

Regularization is a technique to reduce overfitting by adding a penalty term to the loss function that discourages overly complex models. L1 regularization (Lasso) adds the sum of absolute weights — it can drive weights to exactly zero, performing automatic feature selection. L2 regularization (Ridge) adds the sum of squared weights — it shrinks weights toward zero but rarely makes them exactly zero. Elastic Net combines L1 and L2. The regularization strength λ controls the trade-off between fitting the data and model simplicity.