What is an activation function?

An activation function introduces non-linearity into a neural network, allowing it to learn complex patterns. Without activation functions, a neural network with multiple layers would collapse into a single linear transformation. Common activation functions: ReLU (max(0, x)) — most widely used in hidden layers; Sigmoid (outputs 0–1) — used in binary classification output; Softmax — used in multi-class output layers (outputs probability distribution); Tanh — outputs -1 to 1, used in some RNN architectures.