What is the bias-variance tradeoff?

The bias-variance tradeoff is a fundamental ML concept describing the two sources of prediction error. Bias is error from overly simplistic assumptions — a high-bias model underfits. Variance is error from sensitivity to small fluctuations in training data — a high-variance model overfits. Total expected error = Bias² + Variance + Irreducible Noise. Increasing model complexity reduces bias but increases variance, and vice versa. The goal is to find the sweet spot with low bias and low variance, achieved through regularization, cross-validation, and ensembling.