What is the curse of dimensionality?

The curse of dimensionality refers to the phenomena that arise when analyzing data in high-dimensional spaces that do not occur in low-dimensional settings. As the number of features grows, the volume of the space increases exponentially, causing data to become increasingly sparse. This means distance metrics become less meaningful (all points appear equidistant), models need exponentially more data to generalize, and computation becomes expensive. Solutions include dimensionality reduction (PCA, t-SNE, UMAP), feature selection, and regularization.