Hiprup

What is the difference between a model parameter and a hyperparameter?

Both control how a model behaves, but they're set at different times.

  • Model Parameters — values learned by the model from data during training. The model updates them via gradient descent. Examples: weights and biases of a neural network, coefficients of linear regression.

  • Hyperparameters — values set by the engineer before training. They control the learning process itself. Examples: learning rate, number of epochs, number of trees in Random Forest, K in KNN.

How they're chosen: parameters are learned; hyperparameters are tuned (Grid Search, Random Search, Bayesian Optimization).

A simple but easily confused distinction. Use weights vs learning rate as your canonical pair: weights are learned during training (parameters); learning rate is set before training (hyperparameter).

What is the difference between a model parameter and a hyperparameter? | Hiprup