Overfitting

STAT 20: Introduction to Probability and Statistics

Concept Questions

Overfitting or not?

01:00

Which scenarios (open pollev.com) are examples of overfitting (either in real life or in statistics)?

Sometimes, it’s helpful to draw out data frames for each scenario. What are the response variable, important predictors, and erroneous predictors which might lead to overfitting? The options which involve overfitting are the “high degree polynomial”, sneaker and study method scenarios.

How are \(RSS\) and \(R^2\) related?

01:00

Choose the correct option from the items at pollev.com.

As we increase RSS, \(R^2\) should decrease. As we decrease RSS, \(R^2\) should increase.

Where is overfitting worse?

01:00

Suppose I overfit my model to the training data. In which scenario (for which training data) would I expect the test set performance to be significantly worse? Assume that the testing sets A and B look like their corresponding training sets.

If you overfit the data to training set A, you will likely have a complex curve (maybe a polynomial). This curve will fit poorly a testing set that looks similar to A. If you overfit to B, since the data has low variance you will have a positively sloping line. The testing set performance will be very similar to the training set performance (it will not be markedly worse).

Answer: Training set A

Break

05:00

Problem Set: Overfitting

40:00