What is a loss function?

A loss function (also called a cost function or objective function) measures how far the model's predictions are from the true labels. It is the signal that gradient descent minimizes. Common loss functions: Mean Squared Error (MSE) for regression, Cross-Entropy Loss for classification, Mean Absolute Error (MAE) for regression with outliers. Choosing the right loss function is important — MSE penalizes large errors more than MAE because it squares the difference.