# Textbook error 101 -- A low p-value for the full model does not mean that the model is a good predictor of the response

On page 606, of Lock et al “Statistics: Unlocking the Power of Data”, the authors state in item D “The p-value from the ANOVA table is 0.000 so the model as a whole is effective at predicting grade point average.” Ah no.

```
library(data.table)
library(mvtnorm)
rho <- 0.5
n <- 10^5
Sigma <- diag(2)
Sigma[1,2] <- Sigma[2,1] <- rho
X <- rmvnorm(n, mean=c(0,0), sigma=Sigma)
colnames(X) <- c("X1", "X2")
beta <- c(0.01, -0.02)
y <- X%*%beta + rnorm(n)
fit <- lm(y ~ X)
summary(fit)
```

```
##
## Call:
## lm(formula = y ~ X)
##
## Residuals:
## Min 1Q Median 3Q Max
## -5.0500 -0.6750 0.0003 0.6730 4.8276
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.002480 0.003158 0.785 0.43232
## XX1 0.011213 0.003651 3.071 0.00213 **
## XX2 -0.019416 0.003651 -5.319 1.05e-07 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.9987 on 99997 degrees of freedom
## Multiple R-squared: 0.0002849, Adjusted R-squared: 0.0002649
## F-statistic: 14.25 on 2 and 99997 DF, p-value: 6.489e-07
```

A p-value is not a measure of the predictive capacity of a model because the p-value is a function of 1) signal, 2) noise (unmodeled error), and 3) sample size while predictive capacity is a function of the signal:noise ratio. If the signal:noise ratio is tiny, the predictive capacity is small but the p-value can be tiny if the sample size is large.