about

1. concepts in statistical modeling

2. probability distributions

3. the linear model

best references

best references

model & data

• data are not sacrossanct
• search for a minimal and suitable model

the data

• continuous or dicrete variable?
• how many replicates?
• what are the predictor variables?
• what is the pattern?

concepts

• maximum likelihood
• principle of parsimony
  • Ocram's razor

maximum likelihood

given the data and the model:

what are the parameter values that make the data more plausible?

principle of parsimony

principle of parsimony

• models with fewer possible parameters
• linear models preferable to non-linear
• less assumptions
• minimally adequate models
• simpler explanations

best model is just a model

• all models are wrong
• some models are better than others
• we are never sure of the correct model
• the simpler the model, the better -- but not simplistic

statistical distributions

continuous distributions

df_n <- data.frame(val = rnorm(1000, mean = 0, sd = 1))
df_ln <- data.frame(val = exp(rnorm(1000)))Copy code 

continuous distributions

statistical model

relation between variables: prediction

relation between variables: extrapolation

the linear model

is the response variable normal?

is the response variable normal?

what is relationship between the predictor and the response variable?

assumptions

• relationship between x and y is linear

• normality of residuals

• homoscedasticity -- homogeneity of residuals variance

• independence of residuals error terms

parameter estimation

least squares method

least squares method

least squares method

least squares method

linear model in R

mod <- lm(y1 ~ x1)
summary(mod)Copy code 

## 
## Call:
## lm(formula = y1 ~ x1)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -9.1424 -4.0088  0.9982  2.8714  6.1706 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)  
## (Intercept)    1.632      4.520   0.361   0.7287  
## x1             4.076      1.384   2.945   0.0216 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 5.361 on 7 degrees of freedom
## Multiple R-squared:  0.5534,    Adjusted R-squared:  0.4895 
## F-statistic: 8.672 on 1 and 7 DF,  p-value: 0.02156Copy code 

uncertainty in the estimate

estimation of coefficients

coef(mod)Copy code 

## (Intercept)          x1 
##    1.632390    4.075949Copy code 

confidence interval

confint(mod)Copy code 

##                  2.5 %    97.5 %
## (Intercept) -9.0566136 12.321394
## x1           0.8031237  7.348775Copy code 

linear model residue

linear model variance partitioning

sum of squares from the linear model

$$S{S}_{total}=S{S}_{between}+S{S}_{error}$$

total sum of squares

total sum of squares

$$S{S}_{total}=\sum _{i=1}^n(y_i-\overline{y}{)}^2$$

$$S{S}_{total}=450.35$$

residual sum of squares

residual sum of squares

$$S{S}_{error}=\sum _{i=1}^n(y_i-{\hat{y}}_i{)}^2$$

$$S{S}_{error}=201.15$$

model sum of squares

$$S{S}_{total}=S{S}_{between}+S{S}_{error}$$

$$S{S}_{between}=S{S}_{total}-S{S}_{error}$$

$$S{S}_{between}=450.35-201.15$$

$$S{S}_{between}=249.2$$

variance partitioning

$$S{S}_{total}=450.35$$

$$S{S}_{between}=249.2$$

$$S{S}_{error}=201.15$$

variance partitioning

anova table

anova(mod)Copy code 

## Analysis of Variance Table
## 
## Response: y1
##           Df Sum Sq Mean Sq F value  Pr(>F)  
## x1         1 249.20 249.200  8.6723 0.02156 *
## Residuals  7 201.15  28.735                  
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Copy code 

coefficient of determination $R^2$

$$R^2=\frac{S{S}_{between}}{S{S}_{t}otal}$$

$$R^2=\frac{249.2}{450.35}$$

$$R^2=0.5533$$

coefficient of determination $R^2$

summary(mod)Copy code 

## 
## Call:
## lm(formula = y1 ~ x1)
## 
## Residuals:
##     Min      1Q  Median      3Q     Max 
## -9.1424 -4.0088  0.9982  2.8714  6.1706 
## 
## Coefficients:
##             Estimate Std. Error t value Pr(>|t|)  
## (Intercept)    1.632      4.520   0.361   0.7287  
## x1             4.076      1.384   2.945   0.0216 *
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## Residual standard error: 5.361 on 7 degrees of freedom
## Multiple R-squared:  0.5534,    Adjusted R-squared:  0.4895 
## F-statistic: 8.672 on 1 and 7 DF,  p-value: 0.02156Copy code 

todo

• lm tutorial

• git add, commit, and push of the day