What is contrastive learning and what is SimCLR?

Contrastive learning is a self-supervised learning paradigm that trains representations by comparing similar and dissimilar pairs. The objective: push representations of similar samples (positive pairs) close together and push dissimilar samples (negative pairs) apart in embedding space. SimCLR creates positive pairs by applying two random augmentations to the same image and treats all other images in the batch as negatives. It uses a projection head (MLP) on top of the encoder during training (discarded at fine-tuning time). SimCLR achieves strong performance on downstream tasks without any labels. Related: MoCo, BYOL (no negative pairs), SimSiam.