What is the difference between parametric and non-parametric models? Give examples of each.

Question

Accepted Answer

The distinction is about the number of parameters a model has — and whether that number depends on the dataset size. Parametric models — have a fixed number of parameters , regardless of data size. Make strong assumptions about data distribution. Faster, but limited expressiveness. Examples: Linear Regression, Logistic Regression, Naive Bayes, Neural Networks (with fixed architecture). Non-parametric models — number of parameters grows with the data . Make few assumptions about distribution; more flexible but slower and more data-hungry. Examples: K-Nearest Neighbors (KNN), Decision Trees, Kernel SVM, Gaussian Processes. Trade-off: parametric = simpler, faster, less flexible; non-parametric = flexible, more accurate on complex data, but expensive at inference. The key insight is whether the model's complexity grows with the data. Parametric = fixed capacity; Non-parametric = grows with data. KNN is the textbook non-parametric example — make sure to mention it.