What is Self-Supervised Learning? How does it differ from Supervised Learning?
Self-Supervised Learning (SSL) is a paradigm where the model generates its own supervisory signal from the raw data — no human labels needed.
How it differs from Supervised — supervised needs human-labeled examples; SSL invents pretext tasks from the data itself (e.g., predict the next word, fill in the masked word, rotate an image and predict the angle).
Examples — BERT uses Masked Language Modeling; GPT uses Next-Token Prediction; SimCLR uses contrastive learning on images.
Why it matters — unlocks training on internet-scale data without labeling, powering today's foundation models.
Big picture: SSL is technically a form of unsupervised learning, but its pretext tasks make it look supervised. It's the backbone of modern LLMs.
Self-supervised learning is the engine behind modern LLMs and foundation models. Mention masked language modeling (BERT) and next-token prediction (GPT) as canonical examples — interviewers love when you connect concepts to known systems.