Statistics for AIML - Regression Metrics - Degree Of freedom Tutorial

The number of independent pieces of information used to calculate the statistic is called the degrees of freedom. The degrees of freedom of a statistic depend on the sample size:

Degree Of freedom = n (sample size) - 1

• When the sample size is small, there are only a few independent pieces of information, and therefore only a few degrees of freedom.
• When the sample size is large, there are many independent pieces of information, and therefore many degrees of freedom.

As DF increases the t-distribution reaches closer to the normal distribution. At low DF, we have fat tails. If DF > 30, then t-distribution is as good as normal distribution.

In predictive modeling, the degrees of freedom often refers to the number of parameters in the model that are estimated from data.

This linear regression model has two degrees of freedom because there are two parameters in the model that must be estimated from a training dataset. Adding one more variable to the data would add one more degree of freedom for the model.