What is k-Nearest Neighbors (KNN)?

K-Nearest Neighbors (KNN) is a non-parametric, lazy learning algorithm that classifies a new point by finding the k training points closest to it (using Euclidean or other distance metrics) and taking a majority vote. For regression, it averages the neighbors' values. "Lazy" means it stores all training data and does no training — computation happens at prediction time, making inference slow for large datasets. Choosing the right k is important: too small causes overfitting, too large causes underfitting. Feature scaling is essential for KNN.