What is the learning rate?

The learning rate (η) controls how large a step gradient descent takes when updating weights. A learning rate too large causes the loss to diverge or oscillate. A learning rate too small leads to very slow convergence and can get stuck in local minima. Finding the right learning rate is crucial. Techniques include learning rate schedules (step decay, cosine annealing), learning rate warmup, and adaptive optimizers like Adam and RMSProp that automatically adjust the effective learning rate per parameter.