What is the Bias-Variance Tradeoff? How do you diagnose which one your model suffers from?
The Bias-Variance Tradeoff describes the tension between two sources of error in a model.
Bias — error from oversimplifying the problem. Model can't capture the underlying pattern. Symptom: high training error AND high validation error (underfitting).
Variance — error from being too sensitive to the training data. Model captures noise. Symptom: low training error BUT high validation error (overfitting).
Total error = Bias² + Variance + Irreducible Noise. Reducing one often increases the other.
How to diagnose:
• High training error → high bias (underfitting). Fix: bigger model, more features, less regularization.
• Big gap between training and validation → high variance (overfitting). Fix: more data, regularization, simpler model, dropout.
This is the #1 most-asked ML interview question. Don't just define bias and variance — also explain how to diagnose: high training error = bias; low training but high validation error = variance.
Naming the symptom and the fix is what separates a good answer.