What is overfitting? How can you detect and prevent it?

Question

Accepted Answer

Overfitting happens when a model learns the noise and quirks of the training data instead of the true underlying pattern. It performs great on training data but poorly on unseen data. How to detect — training accuracy stays high, validation accuracy plateaus or drops; large gap between training and validation loss. Causes — too complex a model, too few training examples, too many epochs, leakage, lack of regularization. How to prevent: More data or data augmentation . Regularization (L1, L2, Dropout, Batch Normalization). Early stopping based on validation loss. Cross-validation to spot overfitting early. Simpler model or pruning (e.g., shallower trees). Ensembling (Bagging, Random Forest). Don't just define overfitting — explain why it happens (model has too much capacity for the data) and how to detect it (validation error rising while training error keeps falling). Mention at least three concrete prevention techniques.